US20190243886A1 - Methods and systems for improving machine learning performance - Google Patents
Methods and systems for improving machine learning performance Download PDFInfo
- Publication number
- US20190243886A1 US20190243886A1 US16/125,343 US201816125343A US2019243886A1 US 20190243886 A1 US20190243886 A1 US 20190243886A1 US 201816125343 A US201816125343 A US 201816125343A US 2019243886 A1 US2019243886 A1 US 2019243886A1
- Authority
- US
- United States
- Prior art keywords
- documents
- natural language
- document
- language model
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 100
- 238000010801 machine learning Methods 0.000 title description 18
- 230000007786 learning performance Effects 0.000 title description 2
- 238000003058 natural language processing Methods 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims description 40
- 230000015654 memory Effects 0.000 claims description 36
- 238000012545 processing Methods 0.000 abstract description 21
- 230000006870 function Effects 0.000 abstract description 13
- 230000001976 improved effect Effects 0.000 abstract description 7
- 238000004891 communication Methods 0.000 description 38
- 238000010586 diagram Methods 0.000 description 17
- 238000000605 extraction Methods 0.000 description 11
- 238000012549 training Methods 0.000 description 9
- 239000000047 product Substances 0.000 description 6
- 238000012804 iterative process Methods 0.000 description 4
- 241000282412 Homo Species 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 238000007796 conventional method Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 239000004984 smart glass Substances 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000001010 compromised effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000008093 supporting effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/01—Social networking
-
- G06F17/241—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/243—Natural language query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24532—Query optimisation of parallel queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/288—Entity relationship models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/93—Document management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G06F17/2241—
-
- G06F17/272—
-
- G06F17/2785—
-
- G06F17/28—
-
- G06F17/2809—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/137—Hierarchical processing, e.g. outlines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/169—Annotation, e.g. comment data or footnotes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/221—Parsing markup language streams
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/42—Data-driven translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
Description
- This application is a continuation of U.S. patent application Ser. No. 14/964,510, filed Dec. 9, 2015, and titled “METHODS AND SYSTEMS FOR IMPROVING MACHINE LEARNING PERFORMANCE,” which claims the benefits of U.S. Provisional Application 62/089,736, filed Dec. 9, 2014, and titled, “METHODS AND SYSTEMS FOR ANNOTATING NATURAL LANGUAGE PROCESSING,” U.S. Provisional Application 62/089,742, filed Dec. 9, 2014, and titled, “METHODS AND SYSTEMS FOR IMPROVING MACHINE PERFORMANCE IN NATURAL LANGUAGE PROCESSING,” U.S. Provisional Application 62/089,745, filed Dec. 9, 2014, and titled, “METHODS AND SYSTEMS FOR IMPROVING FUNCTIONALITY IN NATURAL LANGUAGE PROCESSING,” and U.S. Provisional Application 62/089,747, filed Dec. 9, 2014, and titled, “METHODS AND SYSTEMS FOR SUPPORTING NATURAL LANGUAGE PROCESSING,” the disclosures of which are incorporated herein by reference in their entireties and for all purposes.
- This application is also related to US non provisional applications (Attorney Docket No. 1402805.00006_IDB006), titled “METHODS FOR GENERATING NATURAL LANGUAGE PROCESSING SYSTEMS,” (Attorney Docket No. 1402805.00007_IDB007), titled “ARCHITECTURES FOR NATURAL LANGUAGE PROCESSING,” (Attorney Docket No. 1402805.00012_IDB012), titled “OPTIMIZATION TECHNIQUES FOR ARTIFICIAL INTELLIGENCE,” (Attorney Docket No. 1402805.00013_IDB013), titled “GRAPHICAL SYSTEMS AND METHODS FOR HUMAN-IN-THE-LOOP MACHINE INTELLIGENCE,” (Attorney Docket No. 1402805.000015_IDB015), titled “METHODS AND SYSTEMS FOR MODELING COMPLEX TAXONOMIES WITH NATURAL LANGUAGE UNDERSTANDING,” (Attorney Docket No. 1402805.00016_IDB016), titled “AN INTELLIGENT SYSTEM THAT DYNAMICALLY IMPROVES ITS KNOWLEDGE AND CODE-BASE FOR NATURAL LANGUAGE UNDERSTANDING,” (Attorney Docket No. 1402805.00017_IDB017), titled “METHODS AND SYSTEMS FOR LANGUAGE-AGNOSTIC MACHINE LEARNING IN NATURAL LANGUAGE PROCESSING USING FEATURE EXTRACTION,” (Attorney Docket No. 1402805.00018_IDB018), titled “METHODS AND SYSTEMS FOR PROVIDING UNIVERSAL PORTABILITY IN MACHINE LEARNING,” and (Attorney Docket No. 1402805.00019_IDB019), titled “TECHNIQUES FOR COMBINING HUMAN AND MACHINE LEARNING IN NATURAL LANGUAGE PROCESSING,” each of which are filed concurrently herewith, and the entire contents and substance of all of which are hereby incorporated in total by reference in their entireties and for all purposes.
- The subject matter disclosed herein generally relates to processing data. In some example embodiments, the present disclosures relate to systems and methods for improving machine performance in natural language processing.
- In some embodiments, methods and systems for improving machine performance in natural language processing are presented. In some embodiments, a method may include: generating a natural language model by a natural language platform; storing the natural language model in a first stateless format; accessing a plurality of documents to be classified by the natural language model; storing the plurality of documents in a second stateless format; and classifying, by the natural language platform, at least one document among the plurality of documents while the at least one document is stored in the second stateless format using the natural language model while stored in the first stateless format.
- In some embodiments of the method, storing the natural language model in the first stateless format comprises storing the natural language model in a language agnostic format.
- In some embodiments of the method, storing the plurality of documents in a second stateless format comprises storing all configuration and auxiliary data used to process each document among the plurality of documents with a combination of said document and the natural language model.
- In some embodiments, the method further comprises performing an intelligent queuing operation on a subset of the documents within the plurality of documents while classifying the at least one document, wherein the subset of documents is distinct from the at least one document.
- In some embodiments, the method further comprises performing a discover topics operation to discover documents that are classified into a specified label while classifying the at least one document.
- In some embodiments of the method, accessing the plurality of documents to be classified by the natural language model comprises retrieving a subset of the plurality of documents from a database; and classifying the at least one document occurs while retrieving the subset of the plurality of documents, wherein the at least one document is distinct from the subset of the plurality of documents.
- In some embodiments of the method, storing the natural language model in a stateless format comprises storing replicas of the natural language model each into a server among a plurality of parallelized servers.
- In some embodiments, a natural language processing system is presented and comprises: a plurality of server machines communicatively coupled in parallel, each of the plurality of servers comprising a memory and at least one processor, each of the plurality of servers configured to: store, in said memory of said server, a replica of a natural language model in a first stateless format; access a plurality of documents to be classified by said replica of the natural language model; store the plurality of documents in a second stateless format; and classify at least one document among the plurality of documents while the at least one document is stored in the second stateless format using said replica of the natural language model while stored in the first stateless format.
- In some embodiments, a non-transitory computer readable medium is presented comprising instructions that, when executed by a process, cause the processor to perform operations comprising: generating a natural language model; storing the natural language model in a first stateless format; accessing a plurality of documents to be classified by the natural language model; storing the plurality of documents in a second stateless format; and classifying at least one document among the plurality of documents while the at least one document is stored in the second stateless format using the natural language model while stored in the first stateless format.
- Some embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings.
-
FIG. 1 is a network diagram illustrating an example network environment suitable for aspects of the present disclosure, according to some example embodiments. -
FIG. 2 is a diagram showing an example system architecture for performing aspects of the present disclosure, according to some example embodiments. -
FIG. 3 is a high level diagram showing various examples of types of human communications and what the objectives may be for a natural language model to accomplish, according to some embodiments. -
FIG. 4 is a diagram showing an example flowchart for how different data structures within the system architecture may be related to one another, according to some example embodiments. -
FIG. 5 is a diagram describing further details of an example implementation of a stateless storage of a natural language model, according to some embodiments. -
FIG. 6 is a diagram describing further details of a feature selection module that may be used to improve model training performance of the natural language platform, according to some embodiments. -
FIG. 7 is a block diagram illustrating components of a machine, according to some example embodiments, able to read instructions from a machine-readable medium and perform any one or more of the methodologies discussed herein. - Example methods, apparatuses, and systems (e.g., machines) are presented for improving performance when performing natural language processing techniques using human annotations applied to machine learning techniques of natural language.
- Aspects of the present disclosure are presented for assisting customers or users to accurately and expediently process human communications brought upon by the capabilities of the digital age. The modes of human communications brought upon by digital technologies have created a deluge of information that can be difficult for human readers to handle alone. Companies and research groups may want to determine trends in the human communications to determine what people generally care about for any particular topic, whether it be what car features are being most expressed on Twitter®, what political topics are being most expressed on Facebook®, what people are saying about the customer's latest product in their customer feedback page, and so forth. It may be desirable for companies to aggregate and then synthesize the thousands or even millions of human communications from the many different modes available in the digital age (e.g., Twitter®, blogs, email, etc.). Processing all this information by humans alone can be overwhelming and cost-inefficient. Methods today may therefore rely on computers to apply natural language processing in order to interpret the many human communications available in order to analyze, group, and ultimately categorize the many human communications into digestible patterns of communication.
- Aspects of the present disclosure include novel methods for combining natural language machine learning processing of the millions of individual human communications with human annotations of the machine results to best refine how the machines process all the data. The human annotations help the machine learning techniques resolve inevitable ambiguities in the human communications, as well as provide intelligence or meaning to communications the machine does not accurately comprehend. The human annotations can then enable computers to provide better natural language results of the human communications, which can then in turn be better refined by more human annotations as necessary. This cyclical or iterative process can converge to provide companies or users of the present disclosures with accurate summaries and analysis of the thousands or millions of human communications in the user's subject matter area.
- In addition, aspects of the present disclosure may construct machine learning models based on this iterative process that can be specifically tailored to a user's unique needs or subject matter area. For example, the words important to categorizing communications in biotechnology may be different than the words important to categorizing communications in the automobile industry. The biotechnology user may desire to tailor the machine learning model to better understand articles related to biotechnology, while the automobile industry user may desire to tailor the machine learning model to better understand customer feedback emails. As another example, the language, grammar, and idioms used in social media may vary drastically from communications in professional writings, e.g., legal or medical journals. A user focusing on Twitter® communications may desire to tailor the machine learning model to better determine when tweets of adolescent teens convey positive sentiment or negative sentiment, while a user focusing on legal documents may desire to tailor the machine learning model to better understand whether a legal decision is favorable or unfavorable. As another example, answers to poll questions or customer surveys can be determined without polling or conducting any survey, based on analyzing public communications, e.g., tweets, Disqus™ comments and so forth. The machine learning model can be trained through the iterative process utilizing human annotations described herein to more easily determine what actual public sentiment may that might otherwise be determined through polling or surveys.
- Once tuned to a user's specific needs through the iterative process described, aspects of the present disclosure allow for these tailored machine learning models to be applied to any number of present and future human communications. In some cases, the machine learning models can act as a filter of sorts, to discern and parse out what communications are relevant to the user before humans or even other machine language techniques process and analyze the data further.
- In some example embodiments, a comprehensive system for producing these catered natural language models is presented. The comprehensive system may include an application program interface (API) to perform much of the functionality described herein. The API may be configured to improve performance in generating the natural language models by specially integrating specific functions and modules designed to reduce memory usage and reduce processing time while still providing accurate results. The comprehensive system may also include a series of background modules configured to provide certain functionality for the API to help achieve these performance benchmarks. The comprehensive system may also include a user interface to allow users to supplement the machine learning techniques with human annotations, as each user may have different focuses for processing data, each with specific vocabulary and language nuances more catered to the user's purposes.
- In addition, the present disclosures describe how performance when utilizing the natural language models is improved through a novel design of storing the natural language model and the documents to be classified by the natural language model in a stateless format. This allows for a number of performance improvements, such as performing classification predictions while still ingesting the documents, and querying the current results for particular information while the model continues to process new documents.
- Examples merely demonstrate possible variations. Unless explicitly stated otherwise, components and functions are optional and may be combined or subdivided, and operations may vary in sequence or be combined or subdivided. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of example embodiments. It will be evident to one skilled in the art, however, that the present subject matter may be practiced without these specific details.
- Referring to
FIG. 1 , a network diagram illustrating anexample network environment 100 suitable for performing aspects of the present disclosure is shown, according to some example embodiments. Theexample network environment 100 includes aserver machine 110, adatabase 115, afirst device 120 for afirst user 122, and asecond device 130 for asecond user 132, all communicatively coupled to each other via anetwork 190. Theserver machine 110 may form all or part of a network-based system 105 (e.g., a cloud-based server system configured to provide one or more services to the first andsecond devices 120 and 130). Theserver machine 110, thefirst device 120, and thesecond device 130 may each be implemented in a computer system, in whole or in part, as described below with respect toFIG. 7 . The network-basedsystem 105 may be an example of a natural language platform configured to generate natural language models as described herein. Theserver machine 110 and thedatabase 115 may be components of the natural language platform configured to perform these functions. While theserver machine 110 is represented as just a single machine and thedatabase 115 where is represented as just a single database, in some embodiments, multiple server machines and multiple databases communicatively coupled in parallel or in serial may be utilized, and embodiments are not so limited. - Also shown in
FIG. 1 are afirst user 122 and asecond user 132. One or both of the first andsecond users first user 122 may be associated with thefirst device 120 and may be a user of thefirst device 120. For example, thefirst device 120 may be a desktop computer, a vehicle computer, a tablet computer, a navigational device, a portable media device, a smartphone, or a wearable device (e.g., a smart watch or smart glasses) belonging to thefirst user 122. Likewise, thesecond user 132 may be associated with thesecond device 130. As an example, thesecond device 130 may be a desktop computer, a vehicle computer, a tablet computer, a navigational device, a portable media device, a, smartphone, or a wearable device (e.g., a smart watch or smart glasses) belonging to thesecond user 132. Thefirst user 122 and asecond user 132 may be examples of users or customers interfacing with the network-basedsystem 105 to utilize a natural language model according to their specific needs. In other cases, theusers users users system 105 through thedevices - Any of the machines,
databases 115, or first orsecond devices FIG. 1 may be implemented in a general-purpose computer modified (e.g., configured or programmed) by software (e.g., one or more software modules) to be a special-purpose computer to perform one or more of the functions described herein for that machine,database 115, or first orsecond device FIG. 7 . As used herein, a “database” may refer to a data storage resource and may store data structured as a text file, a table, a spreadsheet, a relational database (e.g., an object-relational database), a triple store, a hierarchical data store, any other suitable means for organizing and storing data or any suitable combination thereof. Moreover, any two or more of the machines, databases, or devices illustrated inFIG. 1 may be combined into a single machine, and the functions described herein for any single machine, database, or device may be subdivided among multiple machines, databases, or devices. - The
network 190 may be any network that enables communication between or among machines,databases 115, and devices (e.g., theserver machine 110 and the first device 120). Accordingly, thenetwork 190 may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof. Thenetwork 190 may include one or more portions that constitute a private network, a public network (e.g., the Internet), or any suitable combination thereof. Accordingly, thenetwork 190 may include, for example, one or more portions that incorporate a local area network (LAN), a wide area network (WAN), the Internet, a mobile telephone network (e.g., a cellular network), a wired telephone network (e.g., a plain old telephone system (POTS) network), a wireless data network (e.g., WiFi network or WiMax network), or any suitable combination thereof. Any one or more portions of thenetwork 190 may communicate information via a transmission medium. As used herein, “transmission medium” may refer to any intangible (e.g., transitory) medium that is capable of communicating (e.g., transmitting) instructions for execution by a machine (e.g., by one or more processors of such a machine), and can include digital or analog communication signals or other intangible media to facilitate communication of such software. - Referring to
FIG. 2 , a diagram 200 is presented showing an example system architecture for performing aspects of the present disclosure, according to some example embodiments. The example system architecture according to diagram 200 represents various data structures and their interrelationships that may comprise a natural language platform, such as the natural language platform 170, or the network-basedsystem 105. These various data structures may be implemented through a combination of hardware and software, the details of which may be apparent to those with skill in the art based on the descriptions of the various data structures described herein. For example, anAPI module 205 includes one or more API processors, where multiple API processors may be connected in parallel. In some example embodiments, the repeating boxes in the diagram 200 represent identical servers or machines, to signify that the system architecture in diagram 200 may be scalable to an arbitrary degree. TheAPI module 205 may represent a point of contact for multiple other modules, includes adatabase module 210, acache module 215,background processes module 220,applications module 225, and even an interface forusers 235 in some example embodiments. TheAPI module 205 may be configured to receive or access data fromdatabase module 210. The data may include digital forms of thousands or millions of human communications. Thecache module 215 may store in more accessible memory various information from thedatabase module 210 or fromusers 235 or other subscribers. Because thedatabase module 210 andcache module 215 show accessibility throughAPI module 205, theAPI module 205 can also support authentication and authorization of the data in these modules. Thebackground module 220 may be configured to perform a number of background processes for aiding natural language processing functionality. Various examples of the background processes include a model training module, a cross validation module, an intelligent queuing module, a model prediction module, a topic modeling module, an annotation aggregation module, an annotation validation module, and a feature extraction module. These various modules are described in more detail below as well as in non-provisional applications (Attorney Docket Nos. 1402805.00006_IDB006, 1402805.00007_IDB007, 1402805.00012_IDB012, 1402805.00013_IDB013, 1402805.00016_IDB016, 1402805.00017_IDB017, and 1402805.00019_IDB019), each of which again are incorporated by reference in their entireties. TheAPI module 205 may also be configured to support display and functionality of one or more applications inapplications module 225. - In some example embodiments, the
users 235 may access theAPI module 205, in some cases enabling theusers 235 to create their own applications using the system architecture of diagram 200. Theusers 235 may be other examples of theusers annotators 230 may have access to applications already created inapplications module 225. - In some embodiments, the system architecture according to diagram 200 may be scalable and reproducible at various client sites. Thus, the
database modules 210 and thecache module 215 may be implemented specifically for each client, such that each client does not share memory capacity with another client to ensure better privacy. - In some embodiments, the
API module 205 may be implemented in a plurality of servers communicatively coupled in parallel, for example in a cloud environment. Load balancing may be performed across the plurality of servers to automatically distribute processing and memory use within theAPI module 205. In addition, each of the plurality of servers may be configured to store an identical copy of a natural language model in a corresponding memory that provides quick access, such as in RAM. This form of storing the natural language models may be referred to as a “blob.” In some example embodiments, this is handled with agent-based operationalized training that are deployed on AWS instances. In some example embodiments, the horizontal scalability supported by some example embodiments of theAPI module 205 assist in this process. In this way, each of the servers may quickly access their respective natural language models stored in such memory. In addition, the natural language models may be stored in memory as a stateless data structure that is language agnostic. This allows theAPI module 205 overall to utilize the natural language model in real-time and generally dramatically improves the performance of making predictions with the natural language model by dramatically reducing latency. In addition, this configuration allows for an arbitrary degree of scalability, further augmenting the versatility of this design architecture. Additional details about the universal portability of the natural language models are described in application (Attorney Docket No. 1402805.00018_IDB018), which again is incorporated herein by reference. - In some embodiments,
API module 205 is configured to produce a document in a stateless form using a natural language model according to the following example transformation process: - 1. The document text is partitioned into a sequence of tokens and plurality of associated tags, each token representing a character sequence, morpheme, or word from the original text. More details about the tokenization process are described in application (Attorney Docket No. 1402805.00016_IDB016), which is incorporated herein by reference.
- 2. The document text, tokens, tags, document metadata and auxiliary data are used by a feature extracting algorithm as described in application (Attorney Docket No. 1402805.00017_IDB017), which is incorporated herein by reference. The feature extracting algorithm is configured to use the set of feature types and associated configuration parameters stored with the natural language model. Similarly, any auxiliary data needed by a feature type (for example, to designate documents longer or shorter than a median document length) are stored within the natural language model.
- 3. A machine learning prediction is generated by combining the stored probabilities for each feature extracted in step 2.
- 4. A rule-based prediction is generated by applying rules and associated weights stored in the natural language model to the document text, if any.
- The predictions generated in step 2 and step 3 are combined according to the ratio of the number of extracted features to the number of matching rules.
- In some embodiments, all configuration and auxiliary data used in steps 1-4 is either contained in the natural language model “blob” or provided with each document as needed to process it (for example, the document text), thereby providing stateless document processing.
- These steps are illustrated in the
flowchart 500 ofFIG. 5 .Blocks - In some embodiments, the feature types supported by the feature extracting algorithm applied in step 2 are configured to perform the same transformations regardless of the language or languages used to write the document. In these embodiments, feature extraction is performed only with respect to the tokens and each token's associated tags, thereby allowing the same training and prediction process to be performed equally effectively across any languages supported by the tokenizer. Such embodiments are considered language-agnostic since the same feature extracting, model training, and document processing algorithms may be used to create natural language models which understand an arbitrary number of written languages, without requiring special programming.
- The process described above provides an example for configuring each document intended for classification to be stored in a stateless format. This process, combined with processes for generating and storing the natural language model into a stateless “blob,” allows for the natural language platform to not need to be configured into a particular state before beginning classification of the documents using the natural language model. Further examples processes for storing a natural language model in a stateless “blob” format are described in application (Attorney Docket No. 1402805.00018_IDB018), again incorporated herein by reference.
- In contrast, conventional methods do not generate natural language models stored in a stateless format. Conventional methods therefore tend to require that documents be ingested and pre-processed before being allowed to perform any classification or even declare the action of classifying the documents, because the natural language platform needs to reach a certain state in order to utilize the natural language model. As a result, large wait times must occur where a user or client cannot even begin to utilize the model until all of the pre-selected number of documents is processed. No documents can then be processed in a near real-time fashion due to the model and the documents requiring a particular state before processing, unlike the methods described herein due to the stateless nature of the natural language model and the documents to be processed.
- In some embodiments, a user or client of the natural language model may be apportioned a dedicated environment in memory for utilizing their particular natural language model. In essence, memory of each server in the cloud environment may be partitioned for each client, allowing use of their respective natural language models while still achieving fast performance such as real-time capabilities.
- In some embodiments, the feature extraction module may also be stored in memory of each of the parallelized servers of the
API module 205. The feature extraction module may also be stored as a stateless and language agnostic data structure within each memory. This configuration may allow for a high degree of flexibility and versatility when extracting features from text. Additional details about the feature extraction aspects are discussed in application (Attorney Docket No. 1402805.00017_IDB017), which again is incorporated herein by reference. - In some embodiments, the stateless, language-agnostic models generated by the model training process such as in
background module 220 are configured to limit the number of features stored in the model, for example, to at most 100,000 features per label. In these embodiments, the model training process includes a feature selection algorithm that is configured to efficiently select which features in documents extracted by the feature extraction module should be used by the natural language model when making predictions. For example, the feature selection algorithm may order features according to the amount of information entropy each feature provides, by counting the number of times each feature occurs in documents annotated for each label relative to other documents where the feature appears. If fewer features can be selected while still achieving comparable predictive performance, then natural language models may be stored more efficiently, allowing for more efficient use of memory. Limiting the number of features stored in a model reduces the size and memory requirement in order to use the model, thereby enabling use of the models in more resource-constrained environments, or allowing a larger number of models to remain resident in memory of each of the parallelized servers for theAPI module 205. -
Example flowchart 600 ofFIG. 6 exemplifies this process for performing feature selection throughsteps Steps step 610 and consistent with the description above. Once it is determined that the predictive performance is sufficiently compromised due to the exclusion of the next feature, then the remaining list of features may be stored as the total list of features associated with said label. Other examples for performing feature selection are described in application (Attorney Docket No. 1402805.0006_IDB006), again incorporated herein by reference. - Referring to
FIG. 3 , a high level diagram 300 is presented showing various examples of types of human communications and what the objectives may be for a natural language model to accomplish. Here, various sources of data, sometimes referred to as a collection ofdocuments 305, may be obtained and stored in, forexample database 115, client data store 155, ordatabase modules 210, and may represent different types of human communications, all capable of being analyzed by a natural language model. Examples of the types ofdocuments 305 include, but are not limited to, posts in social media, emails or other writings for customer feedback, pieces of or whole journalistic articles, commands spoken or written to electronic devices, transcribed call center recordings; electronic (instant) messages; corporate communications (e.g., SEC 10-k, 10-q); confidential documents and communications stored on internal collaboration systems (e.g., SharePoint, Notes), and pieces of or whole scholarly texts. - In some embodiments, at
block 310, it may be desired to classify any of thedocuments 305 into a number of enumerated categories or topics, consistent with some of the descriptions mentioned above. This may be referred to as performing a document-scope task. For example, auser 130 in telecommunications may supply thousands of customer service emails related to services provided by a telecommunications company. Theuser 130 may desire to have a natural language model generated that classifies the emails into predetermined categories, such as negative sentiment about their Internet service, positive sentiment about their Internet service, negative sentiment about their cable service, and positive sentiment about their cable service. As previously mentioned, these various categories for which a natural language model may classify the emails into, e.g. “negative” sentiment about “Internet service,” “positive” sentiment about “Internet service,” “negative” sentiment about “cable service,” etc., may be referred to as “labels.” Based on these objectives, atblock 315, a natural language model may be generated that is tailored to automatically classify these types of emails into these types of labels. - As another example, in some embodiments, at
block 320, it may be desired to extract specific subsets of text from documents, consistent with some of the descriptions mentioned above. This may be another example of performing a span-scope task, in reference to the fact that this function focuses on a subset within each document (as previously mentioned, referred to herein as a “span”). For example, auser 130 may desire to identify all instances of a keyword, key phrase, or general subject matter within a novel. As another example, a company may want to extract phrases that correspond to products or product features (e.g., “iPhone 5” or “battery life”). Certainly, this span scope task may be applied to multiple novels or other documents. Here too, based on this objective, atblock 315, a natural language model may be generated that is tailored to perform this function for a specified number of documents. - As another example, in some embodiments, at
block 325, it may be desired to discover what categories the documents may be thematically or topically organized into in the first place, consistent with descriptions above about topic modeling. In some cases, theuser 130 may utilize the natural language platform only to perform topic modeling and to discover what topics are most discussed in a specified collection ofdocuments 305. To this end, the natural language platform may be configured to conduct topic modeling analysis atblock 330. Topic modeling is discussed in more detail below, as well as in applications (Attorney Docket Nos. 1402805.00012_IDB012, 1402805.00013_IDB013, 1402805.00016_IDB016, 1402805.00017_IDB017, and 1402805.00019_IDB019), each of which again are incorporated herein by reference in their entireties. In some cases, it may be desired to then generate a natural language model that categorizes thedocuments 305 into these newfound topics. Thus, after performing thetopic modeling analysis 230, in some embodiments, the natural language model may also be generated atblock 315. - Referring to
FIG. 4 , a diagram 400 is presented showing an example flowchart for how different data structures within the system architecture may be related to one another, according to some example embodiments. Here, thecollections data structure 410 represents a set ofdocuments 435 that in some cases may generally be homogenous. Adocument 435 represents a human communication expressed in a single discrete package, such as a single tweet, a webpage, a chapter of a book, a command to a device, or a journal article, or any part thereof. Eachcollection 410 may have one ormore tasks 430 associated with it. Atask 430 may be thought of as a classification scheme. For example, acollection 410 of tweets may be classified by its sentiment, e.g. a positive sentiment or a negative sentiment, where each classification constitutes atask 430 about acollection 410. Alabel 445 refers to a specific prediction about a specific classification. For example, alabel 445 may be the “positive sentiment” of a human communication, or the “negative sentiment” of a human communication. In some cases,labels 445 can be applied to merely portions ofdocuments 435, such as paragraphs in an article or particular names or places mentioned in adocument 435. For example, alabel 445 may be a “positive opinion” expressed about a product mentioned in a human communication, or a “negative opinion” expressed about a product mentioned in a human communication. In some example embodiments, a task may be a sub-task of another task, allowing for a hierarchy or complex network of tasks. For example, if a task has a label of “positive opinion,” there might be sub-tasks for types of “positives opinions,” like “intention to purchase the product,” “positive review,” “recommendation to friend,” and so on, and there may be subtasks that capture other relevant information, such as “positive features.” -
Annotations 440 refer to classifications imputed onto acollection 410 or adocument 435, often times by human input but may also be added by programmatic means, such as interpolating from available metadata (e.g., customer value, geographic location, etc.), generated by a pre-existing natural language model, or generated by a topic modeling process. As an example, anannotation 440 applies alabel 445 manually to adocument 435. In other cases,annotations 440 are provided byusers 235 from pre-existing data. In other cases,annotations 440 may be derived from human critiques of one ormore documents 435, where the computer determines whatannotation 440 should be placed on a document 435 (or collection 410) based on the human critique. In other cases, with enough data in a language model,annotations 440 of acollection 410 can be derived from one or more patterns of pre-existing annotations found in thecollection 410 or asimilar collection 410. - In some example embodiments, features 450 refer to a library or collection of certain key words or groups of words that may be used to determine whether a
task 430 should be associated with acollection 410 ordocument 435. Thus, eachtask 430 has associated with it one ormore features 450 that help define thetask 430. In some example embodiments, features 450 can also include a length of words or other linguistic descriptions about the language structure of adocument 435, in order to define thetask 430. For example, classifying adocument 435 as being a legal document may be based on determining if thedocument 435 contains a threshold number of words with particularly long lengths, words belonging to a pre-defined dictionary of legal-terms, or words that are related through syntactic structures and semantic relationships. In some example embodiments, features 450 are defined by code, while in other cases features 450 are discovered by statistical methods. In some example embodiments, features 450 are treated independently, while in other cases features 450 are networked combinations of simpler features that are used in combination utilizing techniques like “deep-learning.” In some example embodiments, combinations of the methods described herein may be used to define thefeatures 450, and embodiments are not so limited. One or more processors may be used to identify in adocument 435 the words found infeatures data structure 450 to determine what task should be associated with thedocument 435. - In some example embodiments, a work unit's
data structure 455 specifies when humans should be tasked to further examine adocument 425. Thus, human annotations may be applied to adocument 435 after one ormore work units 455 is applied to thedocument 435. Thework units 455 may specify how many human annotators should examine thedocument 435 and in what order of documents should document 435 be examined. In some example embodiments,work units 455 may also determine what annotations should be reviewed in aparticular document 435 and what the optimal user interface should be for review. - In some example embodiments, the
data structures subscribers 405 may have associatedAPI keys 415, which may represent one or more authentication data structures used to authenticate subscribers and provide access to thecollections 410.Groups 420 may represent a grouping of subscribers based on one or more common traits, such assubscribers 405 belonging to the same company.Individual users 425 capable of accessing thecollections 410 may also result from one ormore groups 420. In addition, in some cases, eachgroup 420,user 425, orsubscriber 405 may have associated with it a more personalized or customized set ofcollections 510,documents 435,annotations 440, tasks, 430, features 450, and labels 445, based on the specific needs of the customer. - In some example embodiments, an API module is presented that is configured to drive processing of the system architecture described in
FIGS. 1 and 2 . An example of the API module isAPI module 205. In addition, theAPI module 205 may also enableusers 235 to access many of the functionality provided by the system architecture, as well as support data storage for any and all human communications to be analyzed by theusers 235 via, e.g.,database module 210 andcache module 215. In some example embodiments, theAPI module 205 is also configured to handle an arbitrary amount of customers orusers 235 and data at any given time, as well as satisfactorily perform the functions theusers 235 want. TheAPI module 205 may also support the display and functionality of any applications inapplication module 225, and may connect to any and all backgroundsupport systems module 220. In some example embodiments, theAPI module 205 also provides authentication services to verify and authenticateusers 235. - Aspects of the present disclosure allow for the
API module 205 to process tasks from an arbitrary number of users simultaneously. In addition, the arbitrary number of users may also access the natural language process techniques in an arbitrary number of languages, provided the system architecture of the present disclosures have been implemented to support the desired languages. In some cases, the arbitrary number of users may also access an arbitrary number of queries and human communications per use. The techniques described herein can therefore refer to techniques for improving the scalability of natural language processing. The following are a number of improvements toward these ends, according to some example embodiments. - In some example embodiments, the speed of retrieval of documents from the
API module 205 across a network connection, e.g., the Internet, is improved. For example, for a single request of documents by auser 235, multiple documents can be retrieved. For example, two different network protocol standards are combined to retrieve multiple documents using a streaming fetch mechanism, based on a single request. - In some embodiments, a natural language model may be generated for each language as specified by the user or client. Each model tailored to a particular language may be trained using annotations compiled in the particular language. Each model tailored to a specific language may be stored on different servers. This is facilitated by the fact that the models are natively stored as stateless data structures that are language agnostic, and because the feature extraction module is also language agnostic. For example, the features of documents extracted by the feature extraction module may be extracted one time and may be available for use in all languages of the natural language model. Only some features may be used for a given language, while other features may be used for another given language. For example, the feature extraction module may identify 100 features of a collection of documents, and a Spanish implemented natural language model may utilize just 10 of the features while an English implemented natural language model may utilize 50 of the features. It is possible that some of the features used in the Spanish implemented model may also be used in the English implemented model, and the remaining features not used in either model may be utilized in other language-specific models.
- In some embodiments, an intelligent queuing process such as included in background
support system module 220 may be used to create language-specific models from a document set containing documents written in a plurality of languages. For example, the intelligent queuing module may recognize that the features extracted from a first subset of documents written in a first language never co-occur with features extracted from a second subset of documents written in a second language. In some embodiments, the intelligent queuing process may select one or more documents for annotation from each of the subset of documents, thereby creating a natural language model for each language represented amongst the annotated documents. Additional details about the intelligent queuing process are described in more details in application (Attorney Docket No. 1402805.00012), again incorporated herein by reference. - In some example embodiments, a streaming fetch of a corpus, e.g., a collection of words from a collection of documents, via JSON combined with a multipart/mixed HTTP protocol can allow for improved document retrieval. In some example embodiments, this method can be much faster for traffic across the network, and hence faster for
users 235, than a highly parallelized approach, since costs for overhead from a single request is processed only once per batch. During this document retrieval process, particular transforms using the tokenizer and feature extraction modules may be performed on the documents by theAPI module 205 that reduces some processes during the model training phase. - In some embodiments, the
API module 205 may process the document according to one or more existing natural language models during the document retrieval, thereby eliminating the overhead of processing each document individually in the topic modeling and intelligent queuing processes. Processing the document while other documents are being retrieved may be made possible because of the stateless nature of the natural language model and the documents. Because all of the inputs needed to classify documents are stored in a stateless format, the natural language platform may be configured to simultaneously retrieve documents while processing other documents. Furthermore, due to processing documents during document retrieval, additional functions that the client may opt for may be made possible. For example, this allows for the intelligent queuing process and the discover topics functionality to occur while processing the documents. Further detailed descriptions of intelligent queuing are described in application (Attorney Docket No. 1402805.00012_IDB012), and further descriptions of the discover topics functionality are discussed in application (Attorney Docket No. 1402805.00015_IDB015), both of which again are incorporated herein by reference. In contrast, conventionally, an API module may simply retrieve documents from thedatabase module 210 without being able to perform any additional processing at the same time. - In some example embodiments, another issue to be resolved or improved includes efficiently loading and retrieving request information. This request information can include information about the credentials of the
user 235, authentication information, and various metadata about the collection of documents theuser 235 intends to retrieve. TheAPI module 205 retrieves this request information from memory, for example adatabase module 210. However, if the request information is repetitive across multiple requests from auser 235, the operations for loading and retrieving said request information can be cumbersome and may slow down process time. - In some example embodiments, this request information may be synchronized across multiple servers, e.g., via a
cache module 215, referred to as cache synchronization. In some embodiments, cache synchronization includes methods for notifying the multiple servers about any changes in the request information. In addition, each individual server may be configured to independently determine whether said server has the latest request information, and if not, obtain an update of the latest request information. - In some example embodiments, another issue to be resolved or improved includes efficiently keeping track of a search cursor when a
user 235 makes requests to retrieve a specified number of documents. Auser 235 may ask for the first 1000 documents in a collection of documents, for example. Theuser 235 may then ask for the next 1000 documents, i.e., documents #1001-2000. The search cursor helps keep track of what indexed document theuser 235 has left off at. For higher indexed documents, some methods determine where the search cursor should be by counting from the beginning of the index for each request. This can be more inefficient the more documents there are that need to be searched. The time taken to perform this operation by some methods scales linearly (order N) with the number of documents in the database or system. - In some embodiments, performance may be improved for performing topic modeling through providing a random or pseudorandom tag for each document in the
database module 210, according to some embodiments. In some cases, a client or user may opt to perform a truncated topic modeling session by limiting the amount of time anAPI module 205 may take to conduct topic modeling using the topic modeling module. For example, the user may opt to learn what topics may be generated or discovered in a collection of documents after only 10 minutes of processing. To do this, a limited number of documents are retrieved, sufficient to be processed and grouped into topics within only 10 minutes. If there are many more documents available than may be processed, then to obtain a closely representative set of documents of the entire collection when performing the truncated topic modeling, a random subset of documents should be retrieved. Conventional retrievals, such as retrieving documents consecutively starting from a particular index, are not likely to achieve this random sampling of subject matter. Rather, according to some embodiments, each document may be applied a random or pseudorandom tag or index. The retrieval of the documents may then be based on an ordering of the documents by this random or pseudorandom tag. In some embodiments, retrieval may start at a random or pseudorandom value as well. In this way, the documents may be retrieved in a random order to achieve a more representative sampling of the entire set of documents. - Referring to
FIG. 7 , the block diagram illustrates components of amachine 700, according to some example embodiments, able to readinstructions 724 from a machine-readable medium 722 (e.g., a non-transitory machine-readable medium, a machine-readable storage medium, a computer-readable storage medium, or any suitable combination thereof) and perform any one or more of the methodologies discussed herein, in whole or in part. Specifically,FIG. 7 shows themachine 700 in the example form of a computer system (e.g., a computer) within which the instructions 724 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing themachine 700 to perform any one or more of the methodologies discussed herein may be executed, in whole or in part. - In alternative embodiments, the
machine 700 operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, themachine 700 may operate in the capacity of aserver machine 110 or a client machine in a server-client network environment, or as a peer machine in a distributed (e.g., peer-to-peer) network environment. Themachine 700 may include hardware, software, or combinations thereof, and may, as example, be a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a cellular telephone, a smartphone, a set-top box (STB), a personal digital assistant (PDA), a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing theinstructions 724, sequentially or otherwise, that specify actions to be taken by that machine. Further, while only asingle machine 700 is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute theinstructions 724 to perform all or part of any one or more of the methodologies discussed herein. - The
machine 700 includes a processor 702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), or any suitable combination thereof), amain memory 704, and astatic memory 706, which are configured to communicate with each other via abus 708. Theprocessor 702 may contain microcircuits that are configurable, temporarily or permanently, by some or all of theinstructions 724 such that theprocessor 702 is configurable to perform any one or more of the methodologies described herein, in whole or in part. For example, a set of one or more microcircuits of theprocessor 702 may be configurable to execute one or more modules (e.g., software modules) described herein. - The
machine 700 may further include a video display 710 (e.g., a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, a cathode ray tube (CRT), or any other display capable of displaying graphics or video). Themachine 700 may also include an alphanumeric input device 712 (e.g., a keyboard or keypad), a cursor control device 714 (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, an eye tracking device, or other pointing instrument), astorage unit 716, a signal generation device 718 (e.g., a sound card, an amplifier, a speaker, a headphone jack, or any suitable combination thereof), and anetwork interface device 720. - The
storage unit 716 includes the machine-readable medium 722 (e.g., a tangible and non-transitory machine-readable storage medium) on which are stored theinstructions 724 embodying any one or more of the methodologies or functions described herein, including, for example, any of the descriptions ofFIGS. 1-6 . Theinstructions 724 may also reside, completely or at least partially, within themain memory 704, within the processor 702 (e.g., within the processor's cache memory), or both, before or during execution thereof by themachine 700. Theinstructions 724 may also reside in thestatic memory 706. - Accordingly, the
main memory 704 and theprocessor 702 may be considered machine-readable media 722 (e.g., tangible and non-transitory machine-readable media). Theinstructions 724 may be transmitted or received over anetwork 726 via thenetwork interface device 720. For example, thenetwork interface device 720 may communicate theinstructions 724 using any one or more transfer protocols (e.g., HTTP). Themachine 700 may also represent example means for performing any of the functions described herein, including the processes described inFIGS. 1-6 . - In some example embodiments, the
machine 700 may be a portable computing device, such as a smart phone or tablet computer, and have one or more additional input components (e.g., sensors or gauges) (not shown). Examples of such input components include an image input component (e.g., one or more cameras), an audio input component (e.g., a microphone), a direction input component (e.g., a compass), a location input component (e.g., a GPS receiver), an orientation component (e.g., a gyroscope), a motion detection component (e.g., one or more accelerometers), an altitude detection component (e.g., an altimeter), and a gas detection component (e.g., a gas sensor). Inputs harvested by any one or more of these input components may be accessible and available for use by any of the modules described herein. - As used herein, the term “memory” refers to a machine-readable medium 722 able to store data temporarily or permanently and may be taken to include, but not be limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. While the machine-readable medium 722 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed
database 115, or associated caches and servers) able to storeinstructions 724. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing theinstructions 724 for execution by themachine 700, such that theinstructions 724, when executed by one or more processors of the machine 700 (e.g., processor 702), cause themachine 700 to perform any one or more of the methodologies described herein, in whole or in part. Accordingly, a “machine-readable medium” refers to a single storage apparatus ordevice devices - Furthermore, the machine-readable medium 722 is non-transitory in that it does not embody a propagating signal. However, labeling the tangible machine-readable medium 722 as “non-transitory” should not be construed to mean that the medium is incapable of movement; the medium should be considered as being transportable from one physical location to another. Additionally, since the machine-readable medium 722 is tangible, the medium may be considered to be a machine-readable device.
- Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.
- Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute software modules (e.g., code stored or otherwise embodied on a machine-readable medium 722 or in a transmission medium), hardware modules, or any suitable combination thereof. A “hardware module” is a tangible (e.g., non-transitory) unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a
processor 702 or a group of processors 702) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein. - In some embodiments, a hardware module may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations. For example, a hardware module may be a special-purpose processor, such as a field programmable gate array (FPGA) or an ASIC. A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware module may include software encompassed within a general-
purpose processor 702 or otherprogrammable processor 702. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations. - Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses 708) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
- The various operations of example methods described herein may be performed, at least partially, by one or
more processors 702 that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured,such processors 702 may constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented module” refers to a hardware module implemented using one ormore processors 702. - Similarly, the methods described herein may be at least partially processor-implemented, a
processor 702 being an example of hardware. For example, at least some of the operations of a method may be performed by one ormore processors 702 or processor-implemented modules. As used herein, “processor-implemented module” refers to a hardware module in which the hardware includes one ormore processors 702. Moreover, the one ormore processors 702 may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples ofmachines 700 including processors 702), with these operations being accessible via a network 726 (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API). - The performance of certain operations may be distributed among the one or
more processors 702, not only residing within asingle machine 700, but deployed across a number ofmachines 700. In some example embodiments, the one ormore processors 702 or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the one ormore processors 702 or processor-implemented modules may be distributed across a number of geographic locations. - Unless specifically stated otherwise, discussions herein using words such as “processing,” “computing,” “calculating,” “determining,” “presenting,” “displaying,” or the like may refer to actions or processes of a machine 700 (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or any suitable combination thereof), registers, or other machine components that receive, store, transmit, or display information. Furthermore, unless specifically stated otherwise, the terms “a” or “an” are herein used, as is common in patent documents, to include one or more than one instance. Finally, as used herein, the conjunction “or” refers to a non-exclusive “or,” unless specifically stated otherwise.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/125,343 US20190243886A1 (en) | 2014-12-09 | 2018-09-07 | Methods and systems for improving machine learning performance |
US17/131,486 US20210232761A1 (en) | 2014-12-09 | 2020-12-22 | Methods and systems for improving machine learning performance |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201462089745P | 2014-12-09 | 2014-12-09 | |
US201462089742P | 2014-12-09 | 2014-12-09 | |
US201462089747P | 2014-12-09 | 2014-12-09 | |
US201462089736P | 2014-12-09 | 2014-12-09 | |
US14/964,510 US20160162569A1 (en) | 2014-12-09 | 2015-12-09 | Methods and systems for improving machine learning performance |
US16/125,343 US20190243886A1 (en) | 2014-12-09 | 2018-09-07 | Methods and systems for improving machine learning performance |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/964,510 Continuation US20160162569A1 (en) | 2014-12-09 | 2015-12-09 | Methods and systems for improving machine learning performance |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/131,486 Continuation US20210232761A1 (en) | 2014-12-09 | 2020-12-22 | Methods and systems for improving machine learning performance |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190243886A1 true US20190243886A1 (en) | 2019-08-08 |
Family
ID=56094482
Family Applications (20)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/964,510 Abandoned US20160162569A1 (en) | 2014-12-09 | 2015-12-09 | Methods and systems for improving machine learning performance |
US14/964,517 Active US10127214B2 (en) | 2014-12-09 | 2015-12-09 | Methods for generating natural language processing systems |
US14/964,522 Abandoned US20160162458A1 (en) | 2014-12-09 | 2015-12-09 | Graphical systems and methods for human-in-the-loop machine intelligence |
US14/964,520 Abandoned US20160162457A1 (en) | 2014-12-09 | 2015-12-09 | Optimization techniques for artificial intelligence |
US14/964,528 Abandoned US20160162464A1 (en) | 2014-12-09 | 2015-12-09 | Techniques for combining human and machine learning in natural language processing |
US14/964,511 Active - Reinstated US9495345B2 (en) | 2014-12-09 | 2015-12-09 | Methods and systems for modeling complex taxonomies with natural language understanding |
US15/294,156 Abandoned US20170235813A1 (en) | 2014-12-09 | 2016-10-14 | Methods and systems for modeling complex taxonomies with natural language understanding |
US16/125,343 Abandoned US20190243886A1 (en) | 2014-12-09 | 2018-09-07 | Methods and systems for improving machine learning performance |
US16/181,102 Abandoned US20190303428A1 (en) | 2014-12-09 | 2018-11-05 | Methods for generating natural language processing systems |
US16/185,843 Abandoned US20190311024A1 (en) | 2014-12-09 | 2018-11-09 | Techniques for combining human and machine learning in natural language processing |
US16/197,190 Abandoned US20190311025A1 (en) | 2014-12-09 | 2018-11-20 | Methods and systems for modeling complex taxonomies with natural language understanding |
US16/198,453 Abandoned US20200234002A1 (en) | 2014-12-09 | 2018-11-21 | Optimization techniques for artificial intelligence |
US16/289,481 Abandoned US20200034737A1 (en) | 2014-12-09 | 2019-02-28 | Architectures for natural language processing |
US16/749,689 Abandoned US20200184146A1 (en) | 2014-12-09 | 2020-01-22 | Techniques for combining human and machine learning in natural language processing |
US16/796,812 Abandoned US20210150130A1 (en) | 2014-12-09 | 2020-02-20 | Methods for generating natural language processing systems |
US16/811,737 Active 2036-04-16 US11599714B2 (en) | 2014-12-09 | 2020-03-06 | Methods and systems for modeling complex taxonomies with natural language understanding |
US17/119,902 Active US11288444B2 (en) | 2014-12-09 | 2020-12-11 | Optimization techniques for artificial intelligence |
US17/131,486 Abandoned US20210232761A1 (en) | 2014-12-09 | 2020-12-22 | Methods and systems for improving machine learning performance |
US17/166,493 Abandoned US20210232762A1 (en) | 2014-12-09 | 2021-02-03 | Architectures for natural language processing |
US17/182,178 Abandoned US20210232763A1 (en) | 2014-12-09 | 2021-02-22 | Graphical systems and methods for human-in-the-loop machine intelligence |
Family Applications Before (7)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/964,510 Abandoned US20160162569A1 (en) | 2014-12-09 | 2015-12-09 | Methods and systems for improving machine learning performance |
US14/964,517 Active US10127214B2 (en) | 2014-12-09 | 2015-12-09 | Methods for generating natural language processing systems |
US14/964,522 Abandoned US20160162458A1 (en) | 2014-12-09 | 2015-12-09 | Graphical systems and methods for human-in-the-loop machine intelligence |
US14/964,520 Abandoned US20160162457A1 (en) | 2014-12-09 | 2015-12-09 | Optimization techniques for artificial intelligence |
US14/964,528 Abandoned US20160162464A1 (en) | 2014-12-09 | 2015-12-09 | Techniques for combining human and machine learning in natural language processing |
US14/964,511 Active - Reinstated US9495345B2 (en) | 2014-12-09 | 2015-12-09 | Methods and systems for modeling complex taxonomies with natural language understanding |
US15/294,156 Abandoned US20170235813A1 (en) | 2014-12-09 | 2016-10-14 | Methods and systems for modeling complex taxonomies with natural language understanding |
Family Applications After (12)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/181,102 Abandoned US20190303428A1 (en) | 2014-12-09 | 2018-11-05 | Methods for generating natural language processing systems |
US16/185,843 Abandoned US20190311024A1 (en) | 2014-12-09 | 2018-11-09 | Techniques for combining human and machine learning in natural language processing |
US16/197,190 Abandoned US20190311025A1 (en) | 2014-12-09 | 2018-11-20 | Methods and systems for modeling complex taxonomies with natural language understanding |
US16/198,453 Abandoned US20200234002A1 (en) | 2014-12-09 | 2018-11-21 | Optimization techniques for artificial intelligence |
US16/289,481 Abandoned US20200034737A1 (en) | 2014-12-09 | 2019-02-28 | Architectures for natural language processing |
US16/749,689 Abandoned US20200184146A1 (en) | 2014-12-09 | 2020-01-22 | Techniques for combining human and machine learning in natural language processing |
US16/796,812 Abandoned US20210150130A1 (en) | 2014-12-09 | 2020-02-20 | Methods for generating natural language processing systems |
US16/811,737 Active 2036-04-16 US11599714B2 (en) | 2014-12-09 | 2020-03-06 | Methods and systems for modeling complex taxonomies with natural language understanding |
US17/119,902 Active US11288444B2 (en) | 2014-12-09 | 2020-12-11 | Optimization techniques for artificial intelligence |
US17/131,486 Abandoned US20210232761A1 (en) | 2014-12-09 | 2020-12-22 | Methods and systems for improving machine learning performance |
US17/166,493 Abandoned US20210232762A1 (en) | 2014-12-09 | 2021-02-03 | Architectures for natural language processing |
US17/182,178 Abandoned US20210232763A1 (en) | 2014-12-09 | 2021-02-22 | Graphical systems and methods for human-in-the-loop machine intelligence |
Country Status (1)
Country | Link |
---|---|
US (20) | US20160162569A1 (en) |
Families Citing this family (306)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US10002189B2 (en) | 2007-12-20 | 2018-06-19 | Apple Inc. | Method and apparatus for searching using an active ontology |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
US8676904B2 (en) | 2008-10-02 | 2014-03-18 | Apple Inc. | Electronic devices with voice command and contextual data processing capabilities |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US20120311585A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Organizing task items that represent tasks to perform |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US10417037B2 (en) | 2012-05-15 | 2019-09-17 | Apple Inc. | Systems and methods for integrating third party services with a digital assistant |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US10430506B2 (en) | 2012-12-10 | 2019-10-01 | International Business Machines Corporation | Utilizing classification and text analytics for annotating documents to allow quick scanning |
US10199051B2 (en) | 2013-02-07 | 2019-02-05 | Apple Inc. | Voice trigger for a digital assistant |
US10652394B2 (en) | 2013-03-14 | 2020-05-12 | Apple Inc. | System and method for processing voicemail |
US10748529B1 (en) | 2013-03-15 | 2020-08-18 | Apple Inc. | Voice activated device for use with a voice-based digital assistant |
WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
CN110442699A (en) | 2013-06-09 | 2019-11-12 | 苹果公司 | Operate method, computer-readable medium, electronic equipment and the system of digital assistants |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
US10296160B2 (en) | 2013-12-06 | 2019-05-21 | Apple Inc. | Method for extracting salient dialog usage from live data |
EP3480811A1 (en) | 2014-05-30 | 2019-05-08 | Apple Inc. | Multi-command single utterance input method |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US10169826B1 (en) | 2014-10-31 | 2019-01-01 | Intuit Inc. | System and method for generating explanations for tax calculations |
AU2015349722C1 (en) | 2014-11-20 | 2021-06-10 | Arizona Board Of Regents On Behalf Of Arizona State University | Systems and methods for generating liquid water from air |
US10387970B1 (en) | 2014-11-25 | 2019-08-20 | Intuit Inc. | Systems and methods for analyzing and generating explanations for changes in tax return results |
US20160162569A1 (en) * | 2014-12-09 | 2016-06-09 | Idibon, Inc. | Methods and systems for improving machine learning performance |
US10152299B2 (en) | 2015-03-06 | 2018-12-11 | Apple Inc. | Reducing response latency of intelligent automated assistants |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US10872384B1 (en) | 2015-03-30 | 2020-12-22 | Intuit Inc. | System and method for generating explanations for year-over-year tax changes |
US10460227B2 (en) | 2015-05-15 | 2019-10-29 | Apple Inc. | Virtual assistant in a communication session |
US10200824B2 (en) | 2015-05-27 | 2019-02-05 | Apple Inc. | Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10318591B2 (en) * | 2015-06-02 | 2019-06-11 | International Business Machines Corporation | Ingesting documents using multiple ingestion pipelines |
US9578173B2 (en) | 2015-06-05 | 2017-02-21 | Apple Inc. | Virtual assistant aided communication with 3rd party service in a communication session |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
WO2016205286A1 (en) * | 2015-06-18 | 2016-12-22 | Aware, Inc. | Automatic entity resolution with rules detection and generation system |
US20160378747A1 (en) | 2015-06-29 | 2016-12-29 | Apple Inc. | Virtual assistant for media playback |
US10607298B1 (en) | 2015-07-30 | 2020-03-31 | Intuit Inc. | System and method for indicating sections of electronic tax forms for which narrative explanations can be presented |
US10140983B2 (en) * | 2015-08-28 | 2018-11-27 | International Business Machines Corporation | Building of n-gram language model for automatic speech recognition (ASR) |
US10740384B2 (en) | 2015-09-08 | 2020-08-11 | Apple Inc. | Intelligent automated assistant for media search and playback |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10331312B2 (en) | 2015-09-08 | 2019-06-25 | Apple Inc. | Intelligent automated assistant in a media environment |
WO2017044134A1 (en) * | 2015-09-11 | 2017-03-16 | Hewlett Packard Enterprise Development Lp | Human-readable cloud structures |
US20170098161A1 (en) | 2015-10-06 | 2017-04-06 | Evolv Technologies, Inc. | Augmented Machine Decision Making |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10956666B2 (en) | 2015-11-09 | 2021-03-23 | Apple Inc. | Unconventional virtual assistant interactions |
US10102201B2 (en) * | 2015-11-30 | 2018-10-16 | Soundhound, Inc. | Natural language module store |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US9760556B1 (en) | 2015-12-11 | 2017-09-12 | Palantir Technologies Inc. | Systems and methods for annotating and linking electronic documents |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10346546B2 (en) * | 2015-12-23 | 2019-07-09 | Oath Inc. | Method and system for automatic formality transformation |
US10740573B2 (en) | 2015-12-23 | 2020-08-11 | Oath Inc. | Method and system for automatic formality classification |
US10572524B2 (en) * | 2016-02-29 | 2020-02-25 | Microsoft Technology Licensing, Llc | Content categorization |
US9747143B1 (en) * | 2016-03-30 | 2017-08-29 | International Business Machines Corporation | Multi platform based event processing |
TWI718284B (en) | 2016-04-07 | 2021-02-11 | 美商零質量純水股份有限公司 | Solar thermal unit |
RU2632143C1 (en) * | 2016-04-11 | 2017-10-02 | Общество С Ограниченной Ответственностью "Яндекс" | Training method of rating module using the training selection with the interference labels |
RU2628431C1 (en) * | 2016-04-12 | 2017-08-16 | Общество с ограниченной ответственностью "Аби Продакшн" | Selection of text classifier parameter based on semantic characteristics |
GB201608576D0 (en) * | 2016-05-16 | 2016-06-29 | Pro Intex It Ltd | Functional behaviour test system and method |
CN115228249A (en) | 2016-05-20 | 2022-10-25 | 环球源公司 | System and method for water extraction control |
US11227589B2 (en) | 2016-06-06 | 2022-01-18 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179049B1 (en) * | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
RU2637883C1 (en) * | 2016-06-20 | 2017-12-07 | Общество С Ограниченной Ответственностью "Яндекс" | Method of establishing training object for training machine training algorithm |
US10635704B2 (en) * | 2016-07-15 | 2020-04-28 | Cisco Technology, Inc. | Automatic ontology generation for internet of things applications |
US10769592B1 (en) | 2016-07-27 | 2020-09-08 | Intuit Inc. | Methods, systems and computer program products for generating explanations for a benefit qualification change |
US10762472B1 (en) | 2016-07-27 | 2020-09-01 | Intuit Inc. | Methods, systems and computer program products for generating notifications of benefit qualification change |
US10872315B1 (en) | 2016-07-27 | 2020-12-22 | Intuit Inc. | Methods, systems and computer program products for prioritization of benefit qualification questions |
US11055794B1 (en) | 2016-07-27 | 2021-07-06 | Intuit Inc. | Methods, systems and computer program products for estimating likelihood of qualifying for benefit |
US10579941B2 (en) * | 2016-09-01 | 2020-03-03 | Facebook, Inc. | Systems and methods for recommending pages |
US10474753B2 (en) | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
US10043516B2 (en) | 2016-09-23 | 2018-08-07 | Apple Inc. | Intelligent automated assistant |
US10650086B1 (en) * | 2016-09-27 | 2020-05-12 | Palantir Technologies Inc. | Systems, methods, and framework for associating supporting data in word processing |
US20180114274A1 (en) * | 2016-10-26 | 2018-04-26 | Intuit Inc. | Methods, systems and computer program products for generating and presenting explanations for tax questions |
WO2018081628A1 (en) | 2016-10-28 | 2018-05-03 | Roam Analytics, Inc. | Dataset networking and database modeling |
WO2018081633A1 (en) | 2016-10-28 | 2018-05-03 | Roam Analytics, Inc. | Semantic parsing engine |
US10963789B2 (en) | 2016-11-28 | 2021-03-30 | Conduent Business Services, Llc | Long-term memory networks for knowledge extraction from text and publications |
US10878488B2 (en) * | 2016-11-29 | 2020-12-29 | Samsung Electronics Co., Ltd. | Electronic apparatus and method for summarizing content thereof |
US11281993B2 (en) | 2016-12-05 | 2022-03-22 | Apple Inc. | Model and ensemble compression for metric learning |
US10679008B2 (en) * | 2016-12-16 | 2020-06-09 | Microsoft Technology Licensing, Llc | Knowledge base for analysis of text |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US11373752B2 (en) * | 2016-12-22 | 2022-06-28 | Palantir Technologies Inc. | Detection of misuse of a benefit system |
US11204787B2 (en) | 2017-01-09 | 2021-12-21 | Apple Inc. | Application integration with a digital assistant |
US11954098B1 (en) * | 2017-02-03 | 2024-04-09 | Thomson Reuters Enterprise Centre Gmbh | Natural language processing system and method for documents |
US10298757B2 (en) * | 2017-02-23 | 2019-05-21 | Accenture Global Solutions Limited | Integrated service centre support |
US20180268004A1 (en) * | 2017-03-17 | 2018-09-20 | Microsoft Technology Licensing, Llc | Rule hierarchies for graph adaptation |
US11514058B2 (en) | 2017-03-17 | 2022-11-29 | Microsoft Technology Licensing, Llc | Context rules for a graph |
US11270686B2 (en) | 2017-03-28 | 2022-03-08 | International Business Machines Corporation | Deep language and acoustic modeling convergence and cross training |
US10679009B2 (en) * | 2017-04-21 | 2020-06-09 | Tata Consultancy Services Limited | System and method for belief based human-bot conversation |
US11880746B1 (en) * | 2017-04-26 | 2024-01-23 | Hrb Innovations, Inc. | Interface for artificial intelligence training |
US10963501B1 (en) * | 2017-04-29 | 2021-03-30 | Veritas Technologies Llc | Systems and methods for generating a topic tree for digital information |
DK201770383A1 (en) | 2017-05-09 | 2018-12-14 | Apple Inc. | User interface for correcting recognition errors |
US10417266B2 (en) | 2017-05-09 | 2019-09-17 | Apple Inc. | Context-aware ranking of intelligent response suggestions |
DK180048B1 (en) | 2017-05-11 | 2020-02-04 | Apple Inc. | MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION |
US10395654B2 (en) | 2017-05-11 | 2019-08-27 | Apple Inc. | Text normalization based on a data-driven learning network |
US10726832B2 (en) | 2017-05-11 | 2020-07-28 | Apple Inc. | Maintaining privacy of personal information |
DK201770439A1 (en) | 2017-05-11 | 2018-12-13 | Apple Inc. | Offline personal assistant |
US11301477B2 (en) | 2017-05-12 | 2022-04-12 | Apple Inc. | Feedback analysis of a digital assistant |
DK179496B1 (en) | 2017-05-12 | 2019-01-15 | Apple Inc. | USER-SPECIFIC Acoustic Models |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK201770428A1 (en) | 2017-05-12 | 2019-02-18 | Apple Inc. | Low-latency intelligent automated assistant |
DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
DK201770432A1 (en) | 2017-05-15 | 2018-12-21 | Apple Inc. | Hierarchical belief states for digital assistants |
US20180336892A1 (en) | 2017-05-16 | 2018-11-22 | Apple Inc. | Detecting a trigger of a digital assistant |
US10403278B2 (en) | 2017-05-16 | 2019-09-03 | Apple Inc. | Methods and systems for phonetic matching in digital assistant services |
US10311144B2 (en) | 2017-05-16 | 2019-06-04 | Apple Inc. | Emoji word sense disambiguation |
DK179549B1 (en) | 2017-05-16 | 2019-02-12 | Apple Inc. | Far-field extension for digital assistant services |
US20180336275A1 (en) | 2017-05-16 | 2018-11-22 | Apple Inc. | Intelligent automated assistant for media exploration |
US11340925B2 (en) | 2017-05-18 | 2022-05-24 | Peloton Interactive Inc. | Action recipes for a crowdsourced digital assistant system |
US11056105B2 (en) | 2017-05-18 | 2021-07-06 | Aiqudo, Inc | Talk back from actions in applications |
US11043206B2 (en) | 2017-05-18 | 2021-06-22 | Aiqudo, Inc. | Systems and methods for crowdsourced actions and commands |
US10466963B2 (en) | 2017-05-18 | 2019-11-05 | Aiqudo, Inc. | Connecting multiple mobile devices to a smart home assistant account |
WO2018213788A1 (en) | 2017-05-18 | 2018-11-22 | Aiqudo, Inc. | Systems and methods for crowdsourced actions and commands |
US10657328B2 (en) | 2017-06-02 | 2020-05-19 | Apple Inc. | Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling |
EP3631644B8 (en) * | 2017-06-02 | 2022-07-20 | Thinkspan, LLC | Universal data scaffold based data management platform |
US11447407B2 (en) | 2017-07-14 | 2022-09-20 | Source Global, PBC | Systems for controlled treatment of water with ozone and related methods therefor |
CN109272003A (en) * | 2017-07-17 | 2019-01-25 | 华东师范大学 | A kind of method and apparatus for eliminating unknown error in deep learning model |
US11062792B2 (en) | 2017-07-18 | 2021-07-13 | Analytics For Life Inc. | Discovering genomes to use in machine learning techniques |
US11139048B2 (en) | 2017-07-18 | 2021-10-05 | Analytics For Life Inc. | Discovering novel features to use in machine learning techniques, such as machine learning techniques for diagnosing medical conditions |
CN110019770A (en) * | 2017-07-24 | 2019-07-16 | 华为技术有限公司 | The method and apparatus of train classification models |
US11531998B2 (en) * | 2017-08-30 | 2022-12-20 | Qualtrics, Llc | Providing a conversational digital survey by generating digital survey questions based on digital survey responses |
AU2018329660B2 (en) | 2017-09-05 | 2023-11-09 | Source Global, PBC | Systems and methods to produce liquid water extracted from air |
AU2018329665B2 (en) | 2017-09-05 | 2023-11-16 | Source Global, PBC | Systems and methods for managing production and distribution of liquid water extracted from air |
US10565444B2 (en) * | 2017-09-07 | 2020-02-18 | International Business Machines Corporation | Using visual features to identify document sections |
US10445429B2 (en) | 2017-09-21 | 2019-10-15 | Apple Inc. | Natural language understanding using vocabularies with compressed serialized tries |
US10755051B2 (en) | 2017-09-29 | 2020-08-25 | Apple Inc. | Rule-based natural language processing |
US20190102697A1 (en) * | 2017-10-02 | 2019-04-04 | International Business Machines Corporation | Creating machine learning models from structured intelligence databases |
WO2019071202A1 (en) | 2017-10-06 | 2019-04-11 | Zero Mass Water, Inc. | Systems for generating water with waste heat and related methods therefor |
KR102365621B1 (en) * | 2017-10-20 | 2022-02-21 | 구글 엘엘씨 | Capturing detailed structures in patient-physician conversations for use in clinical documentation |
CN107832305A (en) * | 2017-11-28 | 2018-03-23 | 百度在线网络技术(北京)有限公司 | Method and apparatus for generating information |
US10636424B2 (en) | 2017-11-30 | 2020-04-28 | Apple Inc. | Multi-turn canned dialog |
MX2020005896A (en) * | 2017-12-06 | 2021-01-08 | Zero Mass Water Inc | Systems for constructing hierarchical training data sets for use with machine-learning and related methods therefor. |
US20190179883A1 (en) * | 2017-12-08 | 2019-06-13 | International Business Machines Corporation | Evaluating textual annotation model performance |
US10979457B2 (en) | 2017-12-20 | 2021-04-13 | Check Point Public Cloud Security Ltd | Cloud security assessment system using near-natural language compliance rules |
US10803108B2 (en) * | 2017-12-20 | 2020-10-13 | International Business Machines Corporation | Facilitation of domain and client-specific application program interface recommendations |
US10942955B2 (en) * | 2017-12-21 | 2021-03-09 | Shanghai Xiaoi Robot Technology Co., Ltd. | Questioning and answering method, method for generating questioning and answering system, and method for modifying questioning and answering system |
US20190197433A1 (en) * | 2017-12-22 | 2019-06-27 | Wipro Limited | Methods for adaptive information extraction through adaptive learning of human annotators and devices thereof |
US10942954B2 (en) * | 2017-12-22 | 2021-03-09 | International Business Machines Corporation | Dataset adaptation for high-performance in specific natural language processing tasks |
US10599783B2 (en) * | 2017-12-26 | 2020-03-24 | International Business Machines Corporation | Automatically suggesting a temporal opportunity for and assisting a writer in writing one or more sequel articles via artificial intelligence |
US11768852B2 (en) * | 2017-12-27 | 2023-09-26 | Marlabs Incorporated | System and method for data analysis and presentation of data |
US10963495B2 (en) | 2017-12-29 | 2021-03-30 | Aiqudo, Inc. | Automated discourse phrase discovery for generating an improved language model of a digital assistant |
US10963499B2 (en) * | 2017-12-29 | 2021-03-30 | Aiqudo, Inc. | Generating command-specific language model discourses for digital assistant interpretation |
US10929613B2 (en) | 2017-12-29 | 2021-02-23 | Aiqudo, Inc. | Automated document cluster merging for topic-based digital assistant interpretation |
US10733982B2 (en) | 2018-01-08 | 2020-08-04 | Apple Inc. | Multi-directional dialog |
US10733375B2 (en) | 2018-01-31 | 2020-08-04 | Apple Inc. | Knowledge-based framework for improving natural language understanding |
WO2019161339A1 (en) | 2018-02-18 | 2019-08-22 | Zero Mass Water, Inc. | Systems for generating water for a container farm and related methods therefor |
US11449762B2 (en) | 2018-02-20 | 2022-09-20 | Pearson Education, Inc. | Real time development of auto scoring essay models for custom created prompts |
US11741849B2 (en) | 2018-02-20 | 2023-08-29 | Pearson Education, Inc. | Systems and methods for interface-based machine learning model output customization |
US10789959B2 (en) | 2018-03-02 | 2020-09-29 | Apple Inc. | Training speaker recognition models for digital assistants |
US10579737B2 (en) * | 2018-03-06 | 2020-03-03 | Adobe Inc. | Natural language image editing annotation framework |
US10963972B1 (en) * | 2018-03-06 | 2021-03-30 | Wells Fargo Bank, N.A. | Adaptive life advisor system |
US10685655B2 (en) | 2018-03-07 | 2020-06-16 | International Business Machines Corporation | Leveraging natural language processing |
US10592604B2 (en) | 2018-03-12 | 2020-03-17 | Apple Inc. | Inverse text normalization for automatic speech recognition |
US11474836B2 (en) | 2018-03-13 | 2022-10-18 | Microsoft Technology Licensing, Llc | Natural language to API conversion |
US10747560B2 (en) * | 2018-03-20 | 2020-08-18 | Microsoft Technology Licensing, Llc | Computerized task guidance across devices and applications |
RU2691855C1 (en) * | 2018-03-23 | 2019-06-18 | Общество с ограниченной ответственностью "Аби Продакшн" | Training classifiers used to extract information from natural language texts |
US10818288B2 (en) | 2018-03-26 | 2020-10-27 | Apple Inc. | Natural assistant interaction |
US10909331B2 (en) | 2018-03-30 | 2021-02-02 | Apple Inc. | Implicit identification of translation payload with neural machine translation |
US20190332789A1 (en) * | 2018-04-27 | 2019-10-31 | Microsoft Technology Licensing, Llc | Hierarchical access rights and role based access |
US10928918B2 (en) | 2018-05-07 | 2021-02-23 | Apple Inc. | Raise to speak |
US11145294B2 (en) | 2018-05-07 | 2021-10-12 | Apple Inc. | Intelligent automated assistant for delivering content from user experiences |
US20210074271A1 (en) * | 2018-05-08 | 2021-03-11 | 3M Innovative Properties Company | Hybrid batch and live natural language processing |
AU2019265024A1 (en) | 2018-05-11 | 2020-12-03 | Source Global, PBC | Systems for generating water using exogenously generated heat, exogenously generated electricity, and exhaust process fluids and related methods therefor |
US10951482B2 (en) | 2018-05-16 | 2021-03-16 | Microsoft Technology Licensing, Llc | Device identification on a building automation control network |
US11456915B2 (en) | 2018-05-21 | 2022-09-27 | Microsoft Technology Licensing, Llc | Device model templates |
US10984780B2 (en) | 2018-05-21 | 2021-04-20 | Apple Inc. | Global semantic word embeddings using bi-directional recurrent neural networks |
US11281995B2 (en) * | 2018-05-21 | 2022-03-22 | International Business Machines Corporation | Finding optimal surface for hierarchical classification task on an ontology |
US11386266B2 (en) | 2018-06-01 | 2022-07-12 | Apple Inc. | Text correction |
DK201870355A1 (en) | 2018-06-01 | 2019-12-16 | Apple Inc. | Virtual assistant operation in multi-device environments |
US10892996B2 (en) | 2018-06-01 | 2021-01-12 | Apple Inc. | Variable latency device coordination |
DK179822B1 (en) | 2018-06-01 | 2019-07-12 | Apple Inc. | Voice interaction at a primary device to access call functionality of a companion device |
DK180639B1 (en) | 2018-06-01 | 2021-11-04 | Apple Inc | DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT |
US11076039B2 (en) | 2018-06-03 | 2021-07-27 | Apple Inc. | Accelerated task performance |
US10783328B2 (en) * | 2018-06-04 | 2020-09-22 | International Business Machines Corporation | Semi-automatic process for creating a natural language processing resource |
US10629205B2 (en) * | 2018-06-12 | 2020-04-21 | International Business Machines Corporation | Identifying an accurate transcription from probabilistic inputs |
US11232255B2 (en) * | 2018-06-13 | 2022-01-25 | Adobe Inc. | Generating digital annotations for evaluating and training automatic electronic document annotation models |
CN108733359B (en) * | 2018-06-14 | 2020-12-25 | 北京航空航天大学 | Automatic generation method of software program |
US20190385711A1 (en) * | 2018-06-19 | 2019-12-19 | Ellipsis Health, Inc. | Systems and methods for mental health assessment |
US10664472B2 (en) * | 2018-06-27 | 2020-05-26 | Bitdefender IPR Management Ltd. | Systems and methods for translating natural language sentences into database queries |
US11456082B2 (en) * | 2018-07-03 | 2022-09-27 | International Business Machines Corporation | Patient engagement communicative strategy recommendation |
US10839618B2 (en) | 2018-07-12 | 2020-11-17 | Honda Motor Co., Ltd. | Applied artificial intelligence for natural language processing automotive reporting system |
US11399030B2 (en) * | 2018-07-23 | 2022-07-26 | Kyndryl, Inc. | Ontology based control of access to resources in a computing system |
US11334375B2 (en) * | 2018-07-23 | 2022-05-17 | Google Llc | Intelligent home screen of cloud-based content management platform |
US11636333B2 (en) * | 2018-07-26 | 2023-04-25 | Tesla, Inc. | Optimizing neural network structures for embedded systems |
US11144581B2 (en) * | 2018-07-26 | 2021-10-12 | International Business Machines Corporation | Verifying and correcting training data for text classification |
US11392794B2 (en) * | 2018-09-10 | 2022-07-19 | Ca, Inc. | Amplification of initial training data |
US11183195B2 (en) * | 2018-09-27 | 2021-11-23 | Snackable Inc. | Audio content processing systems and methods |
US11010561B2 (en) | 2018-09-27 | 2021-05-18 | Apple Inc. | Sentiment prediction from textual data |
US10839159B2 (en) | 2018-09-28 | 2020-11-17 | Apple Inc. | Named entity normalization in a spoken dialog system |
US11462215B2 (en) | 2018-09-28 | 2022-10-04 | Apple Inc. | Multi-modal inputs for voice commands |
US11170166B2 (en) | 2018-09-28 | 2021-11-09 | Apple Inc. | Neural typographical error modeling via generative adversarial networks |
CN110969330A (en) * | 2018-09-30 | 2020-04-07 | 曹西军 | Enterprise competitive barrier assessment method and system |
WO2020082038A1 (en) | 2018-10-19 | 2020-04-23 | Zero Mass Water, Inc. | Systems and methods for generating liquid water using highly efficient techniques that optimize production |
US20200124566A1 (en) | 2018-10-22 | 2020-04-23 | Zero Mass Water, Inc. | Systems and methods for detecting and measuring oxidizing compounds in test fluids |
US10943185B1 (en) | 2018-10-23 | 2021-03-09 | Bank Of America Corporation | Supervised machine-learning training platform with isolated update testing and bi-directional update propogation |
US11475898B2 (en) | 2018-10-26 | 2022-10-18 | Apple Inc. | Low-latency multi-speaker speech recognition |
US10979448B2 (en) * | 2018-11-02 | 2021-04-13 | KnowBe4, Inc. | Systems and methods of cybersecurity attack simulation for incident response training and awareness |
US10341491B1 (en) * | 2018-11-26 | 2019-07-02 | Capital One Services, Llc | Identifying unreported issues through customer service interactions and website analytics |
US10956671B2 (en) | 2018-11-30 | 2021-03-23 | International Business Machines Corporation | Supervised machine learning models of documents |
US20200175416A1 (en) * | 2018-11-30 | 2020-06-04 | Jpmorgan Chase Bank, N.A. | Methods for sharing machine learning based web service models |
US11216892B1 (en) * | 2018-12-06 | 2022-01-04 | Meta Platforms, Inc. | Classifying and upgrading a content item to a life event item |
US11093708B2 (en) * | 2018-12-13 | 2021-08-17 | Software Ag | Adaptive human to machine interaction using machine learning |
EP3906508B1 (en) * | 2018-12-31 | 2024-03-13 | Intel Corporation | Securing systems employing artificial intelligence |
US11638059B2 (en) | 2019-01-04 | 2023-04-25 | Apple Inc. | Content playback on multiple devices |
US11328007B2 (en) * | 2019-02-04 | 2022-05-10 | International Business Machines Corporation | Generating a domain-specific phrasal dictionary |
US11676043B2 (en) | 2019-03-04 | 2023-06-13 | International Business Machines Corporation | Optimizing hierarchical classification with adaptive node collapses |
US11157702B2 (en) * | 2019-03-06 | 2021-10-26 | International Business Machines Corporation | Utilizing varying coordinates related to a target event to provide contextual outputs |
EP3938931A4 (en) * | 2019-03-11 | 2022-12-07 | Parexel International, LLC | Methods, apparatus and systems for annotation of text documents |
US20200294083A1 (en) * | 2019-03-12 | 2020-09-17 | Jacob M. Del Hagen | Interface method and system for enabling an advertisement sponsor to input data concerning leads generated in response to advertisements |
US11645110B2 (en) * | 2019-03-13 | 2023-05-09 | International Business Machines Corporation | Intelligent generation and organization of user manuals |
US11348573B2 (en) | 2019-03-18 | 2022-05-31 | Apple Inc. | Multimodality in digital assistant systems |
US11715030B2 (en) | 2019-03-29 | 2023-08-01 | Red Hat, Inc. | Automatic object optimization to accelerate machine learning training |
US11334818B2 (en) | 2019-04-03 | 2022-05-17 | Singularity System Inc. | System and method for real-time training of machine learning model using small training data set |
US20220180066A1 (en) * | 2019-04-04 | 2022-06-09 | Singularity Systems Inc. | Machine learning processing pipeline optimization |
US10614345B1 (en) | 2019-04-12 | 2020-04-07 | Ernst & Young U.S. Llp | Machine learning based extraction of partition objects from electronic documents |
CN113747962A (en) | 2019-04-22 | 2021-12-03 | 环球源公司 | Water vapor adsorption air drying system and method for producing liquid water from air |
US11392757B2 (en) * | 2019-04-26 | 2022-07-19 | Figure Eight Technologies, Inc. | Management of annotation jobs |
US11423908B2 (en) | 2019-05-06 | 2022-08-23 | Apple Inc. | Interpreting spoken requests |
DK201970509A1 (en) | 2019-05-06 | 2021-01-15 | Apple Inc | Spoken notifications |
US11475884B2 (en) | 2019-05-06 | 2022-10-18 | Apple Inc. | Reducing digital assistant latency when a language is incorrectly determined |
US11307752B2 (en) | 2019-05-06 | 2022-04-19 | Apple Inc. | User configurable task triggers |
US11748613B2 (en) * | 2019-05-10 | 2023-09-05 | Baidu Usa Llc | Systems and methods for large scale semantic indexing with deep level-wise extreme multi-label learning |
US11140099B2 (en) | 2019-05-21 | 2021-10-05 | Apple Inc. | Providing message response suggestions |
US11277453B2 (en) * | 2019-05-24 | 2022-03-15 | International Business Machines Corporation | Media communication management |
US11289073B2 (en) | 2019-05-31 | 2022-03-29 | Apple Inc. | Device text to speech |
DK201970510A1 (en) | 2019-05-31 | 2021-02-11 | Apple Inc | Voice identification in digital assistant systems |
US11496600B2 (en) | 2019-05-31 | 2022-11-08 | Apple Inc. | Remote execution of machine-learned models |
DK180129B1 (en) | 2019-05-31 | 2020-06-02 | Apple Inc. | User activity shortcut suggestions |
US11468890B2 (en) | 2019-06-01 | 2022-10-11 | Apple Inc. | Methods and user interfaces for voice-based control of electronic devices |
US11360641B2 (en) | 2019-06-01 | 2022-06-14 | Apple Inc. | Increasing the relevance of new available information |
US11531703B2 (en) | 2019-06-28 | 2022-12-20 | Capital One Services, Llc | Determining data categorizations based on an ontology and a machine-learning model |
US10489454B1 (en) | 2019-06-28 | 2019-11-26 | Capital One Services, Llc | Indexing a dataset based on dataset tags and an ontology |
US11113518B2 (en) | 2019-06-28 | 2021-09-07 | Eygs Llp | Apparatus and methods for extracting data from lineless tables using Delaunay triangulation and excess edge removal |
US11354504B2 (en) * | 2019-07-10 | 2022-06-07 | International Business Machines Corporation | Multi-lingual action identification |
US20210026897A1 (en) * | 2019-07-23 | 2021-01-28 | Microsoft Technology Licensing, Llc | Topical clustering and notifications for driving resource collaboration |
US11521019B2 (en) | 2019-08-06 | 2022-12-06 | Bank Of America Corporation | Systems and methods for incremental learning and autonomous model reconfiguration in regulated AI systems |
US11294884B2 (en) | 2019-08-09 | 2022-04-05 | International Business Machines Corporation | Annotation assessment and adjudication |
US11188517B2 (en) | 2019-08-09 | 2021-11-30 | International Business Machines Corporation | Annotation assessment and ground truth construction |
US11915465B2 (en) | 2019-08-21 | 2024-02-27 | Eygs Llp | Apparatus and methods for converting lineless tables into lined tables using generative adversarial networks |
WO2021056255A1 (en) | 2019-09-25 | 2021-04-01 | Apple Inc. | Text detection using global geometry estimators |
US11442944B2 (en) | 2019-10-18 | 2022-09-13 | Thinkspan, LLC | Algorithmic suggestions based on a universal data scaffold |
WO2021077038A1 (en) | 2019-10-18 | 2021-04-22 | Taylor Brian Samuel | Scalable scaffolding and bundled data |
US10810709B1 (en) | 2019-11-21 | 2020-10-20 | Eygs Llp | Systems and methods for improving the quality of text documents using artificial intelligence |
US10922476B1 (en) * | 2019-12-13 | 2021-02-16 | Microsoft Technology Licensing, Llc | Resource-efficient generation of visual layout information associated with network-accessible documents |
TWI798513B (en) * | 2019-12-20 | 2023-04-11 | 國立清華大學 | Training method of natural language corpus for the decision making model of machine learning |
CN111104514B (en) * | 2019-12-23 | 2023-04-25 | 北京百度网讯科技有限公司 | Training method and device for document tag model |
US11775862B2 (en) * | 2020-01-14 | 2023-10-03 | Microsoft Technology Licensing, Llc | Tracking provenance in data science scripts |
US20210233008A1 (en) * | 2020-01-28 | 2021-07-29 | Schlumberger Technology Corporation | Oilfield data file classification and information processing systems |
US11625934B2 (en) | 2020-02-04 | 2023-04-11 | Eygs Llp | Machine learning based end-to-end extraction of tables from electronic documents |
US11354513B2 (en) | 2020-02-06 | 2022-06-07 | Adobe Inc. | Automated identification of concept labels for a text fragment |
US11416684B2 (en) * | 2020-02-06 | 2022-08-16 | Adobe Inc. | Automated identification of concept labels for a set of documents |
US11176329B2 (en) | 2020-02-18 | 2021-11-16 | Bank Of America Corporation | Source code compiler using natural language input |
US11250128B2 (en) | 2020-02-18 | 2022-02-15 | Bank Of America Corporation | System and method for detecting source code anomalies |
US11568153B2 (en) | 2020-03-05 | 2023-01-31 | Bank Of America Corporation | Narrative evaluator |
US20210326718A1 (en) * | 2020-04-16 | 2021-10-21 | Microsoft Technology Licensing, Llc | Machine learning techniques to shape downstream content traffic through hashtag suggestion during content creation |
US11934441B2 (en) | 2020-04-29 | 2024-03-19 | International Business Machines Corporation | Generative ontology learning and natural language processing with predictive language models |
US11061543B1 (en) | 2020-05-11 | 2021-07-13 | Apple Inc. | Providing relevant data items based on context |
US11043220B1 (en) | 2020-05-11 | 2021-06-22 | Apple Inc. | Digital assistant hardware abstraction |
US11341339B1 (en) * | 2020-05-14 | 2022-05-24 | Amazon Technologies, Inc. | Confidence calibration for natural-language understanding models that provides optimal interpretability |
US11443082B2 (en) | 2020-05-27 | 2022-09-13 | Accenture Global Solutions Limited | Utilizing deep learning and natural language processing to convert a technical architecture diagram into an interactive technical architecture diagram |
US11704580B2 (en) | 2020-05-31 | 2023-07-18 | International Business Machines Corporation | Automated combination of predictions made by different prediction systems |
US11556788B2 (en) * | 2020-06-15 | 2023-01-17 | International Business Machines Corporation | Text-based response environment action selection |
US20210390250A1 (en) * | 2020-06-15 | 2021-12-16 | Canon Kabushiki Kaisha | Information processing apparatus |
US11393456B1 (en) * | 2020-06-26 | 2022-07-19 | Amazon Technologies, Inc. | Spoken language understanding system |
CN111522891A (en) * | 2020-07-03 | 2020-08-11 | 支付宝(杭州)信息技术有限公司 | Method and device for determining relationship closeness between two entities and electronic equipment |
US11490204B2 (en) | 2020-07-20 | 2022-11-01 | Apple Inc. | Multi-device audio adjustment coordination |
US11438683B2 (en) | 2020-07-21 | 2022-09-06 | Apple Inc. | User identification using headphones |
US11829720B2 (en) | 2020-09-01 | 2023-11-28 | Apple Inc. | Analysis and validation of language models |
US20220199078A1 (en) * | 2020-12-22 | 2022-06-23 | Samsung Electronics Co., Ltd. | Electronic apparatus, system comprising electronic apparatus and server and controlling method thereof |
WO2022155539A1 (en) * | 2021-01-14 | 2022-07-21 | Virtuosource Llc | A rules-based decision support system for assessment of digital content involving natural language |
WO2022159443A1 (en) | 2021-01-19 | 2022-07-28 | Source Global, PBC | Systems and methods for generating water from air |
US11741313B2 (en) * | 2021-02-08 | 2023-08-29 | International Business Machines Corporation | Question based chatbot generator from web API specifications |
US11645464B2 (en) | 2021-03-18 | 2023-05-09 | International Business Machines Corporation | Transforming a lexicon that describes an information asset |
CN115237856A (en) * | 2021-04-23 | 2022-10-25 | 伊姆西Ip控股有限责任公司 | Method, device and computer program product for marking files |
US20230008868A1 (en) * | 2021-07-08 | 2023-01-12 | Nippon Telegraph And Telephone Corporation | User authentication device, user authentication method, and user authentication computer program |
US11443102B1 (en) * | 2021-08-13 | 2022-09-13 | Pricewaterhousecoopers Llp | Methods and systems for artificial intelligence-assisted document annotation |
US11645462B2 (en) | 2021-08-13 | 2023-05-09 | Pricewaterhousecoopers Llp | Continuous machine learning method and system for information extraction |
US20230091581A1 (en) * | 2021-09-21 | 2023-03-23 | Bank Of America Corporation | Personal Data Discovery |
US20230088315A1 (en) * | 2021-09-22 | 2023-03-23 | Motorola Solutions, Inc. | System and method to support human-machine interactions for public safety annotations |
US11516157B1 (en) * | 2021-09-30 | 2022-11-29 | Dell Products, L.P. | Machine learning facilitated drafting of response to communication |
US20230134796A1 (en) * | 2021-10-29 | 2023-05-04 | Glipped, Inc. | Named entity recognition system for sentiment labeling |
US11546323B1 (en) | 2022-08-17 | 2023-01-03 | strongDM, Inc. | Credential management for distributed services |
US11736531B1 (en) | 2022-08-31 | 2023-08-22 | strongDM, Inc. | Managing and monitoring endpoint activity in secured networks |
CN115294773B (en) * | 2022-09-29 | 2023-02-14 | 深圳市城市交通规划设计研究中心股份有限公司 | Bus lane optimal configuration method, electronic device and storage medium |
CN115904482B (en) * | 2022-11-30 | 2023-09-26 | 杭州巨灵兽智能科技有限公司 | Interface document generation method, device, equipment and storage medium |
US11916885B1 (en) | 2023-01-09 | 2024-02-27 | strongDM, Inc. | Tunnelling with support for dynamic naming resolution |
US11765207B1 (en) * | 2023-03-17 | 2023-09-19 | strongDM, Inc. | Declaring network policies using natural language |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080059435A1 (en) * | 2006-09-01 | 2008-03-06 | Thomson Global Resources | Systems, methods, software, and interfaces for formatting legal citations |
US7376635B1 (en) * | 2000-07-21 | 2008-05-20 | Ford Global Technologies, Llc | Theme-based system and method for classifying documents |
US20140304264A1 (en) * | 2013-04-05 | 2014-10-09 | Hewlett-Packard Development Company, L.P. | Mobile web-based platform for providing a contextual alignment view of a corpus of documents |
Family Cites Families (64)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5970490A (en) * | 1996-11-05 | 1999-10-19 | Xerox Corporation | Integration platform for heterogeneous databases |
NZ520663A (en) * | 2000-02-10 | 2004-05-28 | Involve Technology Inc | System for creating and maintaining a database of information utilizing user defined keyword relevance ratings |
US7730113B1 (en) * | 2000-03-07 | 2010-06-01 | Applied Discovery, Inc. | Network-based system and method for accessing and processing emails and other electronic legal documents that may include duplicate information |
AU2001271891A1 (en) * | 2000-07-07 | 2002-01-21 | Criticalpoint Software Corporation | Methods and system for generating and searching ontology databases |
US6513059B1 (en) * | 2000-08-24 | 2003-01-28 | Cambira Corporation | Adaptive collaborative intelligent network system |
US20020152202A1 (en) * | 2000-08-30 | 2002-10-17 | Perro David J. | Method and system for retrieving information using natural language queries |
US7146399B2 (en) * | 2001-05-25 | 2006-12-05 | 2006 Trident Company | Run-time architecture for enterprise integration with transformation generation |
EP1473639A1 (en) * | 2002-02-04 | 2004-11-03 | Celestar Lexico-Sciences, Inc. | Document knowledge management apparatus and method |
US7548847B2 (en) * | 2002-05-10 | 2009-06-16 | Microsoft Corporation | System for automatically annotating training data for a natural language understanding system |
US8543564B2 (en) * | 2002-12-23 | 2013-09-24 | West Publishing Company | Information retrieval systems with database-selection aids |
US20040254950A1 (en) * | 2003-06-13 | 2004-12-16 | Musgrove Timothy A. | Catalog taxonomy for storing product information and system and method using same |
BRPI0412778A (en) * | 2003-07-22 | 2006-09-26 | Kinor Technologies Inc | access to information using ontology |
US20050278362A1 (en) * | 2003-08-12 | 2005-12-15 | Maren Alianna J | Knowledge discovery system |
GB0320205D0 (en) * | 2003-08-28 | 2003-10-01 | British Telecomm | Method and apparatus for storing and retrieving data |
US20050060140A1 (en) * | 2003-09-15 | 2005-03-17 | Maddox Paul Christopher | Using semantic feature structures for document comparisons |
US7533090B2 (en) * | 2004-03-30 | 2009-05-12 | Google Inc. | System and method for rating electronic documents |
US7809548B2 (en) * | 2004-06-14 | 2010-10-05 | University Of North Texas | Graph-based ranking algorithms for text processing |
US20070106499A1 (en) * | 2005-08-09 | 2007-05-10 | Kathleen Dahlgren | Natural language search system |
US7664746B2 (en) * | 2005-11-15 | 2010-02-16 | Microsoft Corporation | Personalized search and headlines |
US20070150802A1 (en) * | 2005-12-12 | 2007-06-28 | Canon Information Systems Research Australia Pty. Ltd. | Document annotation and interface |
US7835911B2 (en) * | 2005-12-30 | 2010-11-16 | Nuance Communications, Inc. | Method and system for automatically building natural language understanding models |
EP1840764A1 (en) * | 2006-03-30 | 2007-10-03 | Sony France S.A. | Hybrid audio-visual categorization system and method |
US7664644B1 (en) * | 2006-06-09 | 2010-02-16 | At&T Intellectual Property Ii, L.P. | Multitask learning for spoken language understanding |
US8131756B2 (en) | 2006-06-21 | 2012-03-06 | Carus Alwin B | Apparatus, system and method for developing tools to process natural language text |
JP4311432B2 (en) * | 2006-09-29 | 2009-08-12 | ブラザー工業株式会社 | Information processing apparatus and program |
US7774198B2 (en) * | 2006-10-06 | 2010-08-10 | Xerox Corporation | Navigation system for text |
US7953736B2 (en) * | 2007-01-04 | 2011-05-31 | Intersect Ptp, Inc. | Relevancy rating of tags |
US8738606B2 (en) * | 2007-03-30 | 2014-05-27 | Microsoft Corporation | Query generation using environment configuration |
US8214743B2 (en) * | 2007-08-07 | 2012-07-03 | International Business Machines Corporation | Data management techniques |
WO2009050521A2 (en) * | 2007-10-17 | 2009-04-23 | Iti Scotland Limited | Computer-implemented methods displaying, in a first part, a document and in a second part, a selected index of entities identified in the document |
JP4518165B2 (en) * | 2008-03-11 | 2010-08-04 | 富士ゼロックス株式会社 | Related document presentation system and program |
US20100153318A1 (en) * | 2008-11-19 | 2010-06-17 | Massachusetts Institute Of Technology | Methods and systems for automatically summarizing semantic properties from documents with freeform textual annotations |
US8165974B2 (en) * | 2009-06-08 | 2012-04-24 | Xerox Corporation | System and method for assisted document review |
US8713018B2 (en) * | 2009-07-28 | 2014-04-29 | Fti Consulting, Inc. | System and method for displaying relationships between electronically stored information to provide classification suggestions via inclusion |
US20110029517A1 (en) * | 2009-07-31 | 2011-02-03 | Shihao Ji | Global and topical ranking of search results using user clicks |
US8682649B2 (en) * | 2009-11-12 | 2014-03-25 | Apple Inc. | Sentiment prediction from textual data |
US20120064501A1 (en) * | 2010-04-08 | 2012-03-15 | Sukkarieh Jana Z | Systems and Methods for Evaluation of Automatic Content Scoring Technologies |
US8498950B2 (en) * | 2010-10-15 | 2013-07-30 | Yahoo! Inc. | System for training classifiers in multiple categories through active learning |
US8725739B2 (en) * | 2010-11-01 | 2014-05-13 | Evri, Inc. | Category-based content recommendation |
US20120271774A1 (en) * | 2011-04-21 | 2012-10-25 | Hirevue, Inc. | Interview frameworks |
WO2012166581A2 (en) * | 2011-05-27 | 2012-12-06 | Ctc Tech Corp. | Creation, use and training of computer-based discovery avatars |
JP5866728B2 (en) * | 2011-10-14 | 2016-02-17 | サイバーアイ・エンタテインメント株式会社 | Knowledge information processing server system with image recognition system |
US11410072B2 (en) * | 2011-10-21 | 2022-08-09 | Educational Testing Service | Computer-implemented systems and methods for detection of sentiment in writing |
WO2013088287A1 (en) * | 2011-12-12 | 2013-06-20 | International Business Machines Corporation | Generation of natural language processing model for information domain |
US9652452B2 (en) * | 2012-01-06 | 2017-05-16 | Yactraq Online Inc. | Method and system for constructing a language model |
US9170715B1 (en) * | 2012-02-03 | 2015-10-27 | CounterPointe Digital Technologies LLC | System and method for mass visualization of real estate properties |
US8832162B2 (en) * | 2012-03-25 | 2014-09-09 | Think Computer Corporation | Method and system for storing, categorizing and distributing information concerning relationships between data |
US8943135B2 (en) * | 2012-07-24 | 2015-01-27 | Fard Johnmar | System and method for measuring the positive or negative impact of digital and social media content on intent and behavior |
US20140188665A1 (en) | 2013-01-02 | 2014-07-03 | CrowdChunk LLC | CrowdChunk System, Method, and Computer Program Product for Searching Summaries of Online Reviews of Products |
GB2509539A (en) * | 2013-01-08 | 2014-07-09 | Ibm | Production rule engine |
US20140257990A1 (en) * | 2013-03-06 | 2014-09-11 | TipTap, Inc. | Method and system for determining correlations between personality traits of a group of consumers and a brand/product |
US9122681B2 (en) * | 2013-03-15 | 2015-09-01 | Gordon Villy Cormack | Systems and methods for classifying electronic information using advanced active learning techniques |
WO2014183089A1 (en) | 2013-05-09 | 2014-11-13 | Metavana, Inc. | Hybrid human machine learning system and method |
US9348815B1 (en) * | 2013-06-28 | 2016-05-24 | Digital Reasoning Systems, Inc. | Systems and methods for construction, maintenance, and improvement of knowledge representations |
US9772994B2 (en) | 2013-07-25 | 2017-09-26 | Intel Corporation | Self-learning statistical natural language processing for automatic production of virtual personal assistants |
US9990422B2 (en) * | 2013-10-15 | 2018-06-05 | Adobe Systems Incorporated | Contextual analysis engine |
US9547640B2 (en) * | 2013-10-16 | 2017-01-17 | International Business Machines Corporation | Ontology-driven annotation confidence levels for natural language processing |
US10754925B2 (en) * | 2014-06-04 | 2020-08-25 | Nuance Communications, Inc. | NLU training with user corrections to engine annotations |
US20160019282A1 (en) * | 2014-07-16 | 2016-01-21 | Axiom Global Inc. | Discovery management method and system |
US9886247B2 (en) * | 2014-10-30 | 2018-02-06 | International Business Machines Corporation | Using an application programming interface (API) data structure in recommending an API composite |
US9860308B2 (en) * | 2014-11-25 | 2018-01-02 | International Business Machines Corporation | Collaborative creation of annotation training data |
US9852132B2 (en) * | 2014-11-25 | 2017-12-26 | Chegg, Inc. | Building a topical learning model in a content management system |
US11295071B2 (en) * | 2014-12-09 | 2022-04-05 | 100.Co, Llc | Graphical systems and methods for human-in-the-loop machine intelligence |
US20160162569A1 (en) * | 2014-12-09 | 2016-06-09 | Idibon, Inc. | Methods and systems for improving machine learning performance |
-
2015
- 2015-12-09 US US14/964,510 patent/US20160162569A1/en not_active Abandoned
- 2015-12-09 US US14/964,517 patent/US10127214B2/en active Active
- 2015-12-09 US US14/964,522 patent/US20160162458A1/en not_active Abandoned
- 2015-12-09 US US14/964,520 patent/US20160162457A1/en not_active Abandoned
- 2015-12-09 US US14/964,528 patent/US20160162464A1/en not_active Abandoned
- 2015-12-09 US US14/964,511 patent/US9495345B2/en active Active - Reinstated
-
2016
- 2016-10-14 US US15/294,156 patent/US20170235813A1/en not_active Abandoned
-
2018
- 2018-09-07 US US16/125,343 patent/US20190243886A1/en not_active Abandoned
- 2018-11-05 US US16/181,102 patent/US20190303428A1/en not_active Abandoned
- 2018-11-09 US US16/185,843 patent/US20190311024A1/en not_active Abandoned
- 2018-11-20 US US16/197,190 patent/US20190311025A1/en not_active Abandoned
- 2018-11-21 US US16/198,453 patent/US20200234002A1/en not_active Abandoned
-
2019
- 2019-02-28 US US16/289,481 patent/US20200034737A1/en not_active Abandoned
-
2020
- 2020-01-22 US US16/749,689 patent/US20200184146A1/en not_active Abandoned
- 2020-02-20 US US16/796,812 patent/US20210150130A1/en not_active Abandoned
- 2020-03-06 US US16/811,737 patent/US11599714B2/en active Active
- 2020-12-11 US US17/119,902 patent/US11288444B2/en active Active
- 2020-12-22 US US17/131,486 patent/US20210232761A1/en not_active Abandoned
-
2021
- 2021-02-03 US US17/166,493 patent/US20210232762A1/en not_active Abandoned
- 2021-02-22 US US17/182,178 patent/US20210232763A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7376635B1 (en) * | 2000-07-21 | 2008-05-20 | Ford Global Technologies, Llc | Theme-based system and method for classifying documents |
US20080059435A1 (en) * | 2006-09-01 | 2008-03-06 | Thomson Global Resources | Systems, methods, software, and interfaces for formatting legal citations |
US20140304264A1 (en) * | 2013-04-05 | 2014-10-09 | Hewlett-Packard Development Company, L.P. | Mobile web-based platform for providing a contextual alignment view of a corpus of documents |
Also Published As
Publication number | Publication date |
---|---|
US20160162456A1 (en) | 2016-06-09 |
US20160162464A1 (en) | 2016-06-09 |
US20210150130A1 (en) | 2021-05-20 |
US20160162457A1 (en) | 2016-06-09 |
US20210232760A1 (en) | 2021-07-29 |
US20190311024A1 (en) | 2019-10-10 |
US20200034737A1 (en) | 2020-01-30 |
US20160162476A1 (en) | 2016-06-09 |
US10127214B2 (en) | 2018-11-13 |
US20190311025A1 (en) | 2019-10-10 |
US11288444B2 (en) | 2022-03-29 |
US20160162569A1 (en) | 2016-06-09 |
US20210232763A1 (en) | 2021-07-29 |
US11599714B2 (en) | 2023-03-07 |
US20210232762A1 (en) | 2021-07-29 |
US20210232761A1 (en) | 2021-07-29 |
US20210165955A1 (en) | 2021-06-03 |
US20160162458A1 (en) | 2016-06-09 |
US20190303428A1 (en) | 2019-10-03 |
US20200184146A1 (en) | 2020-06-11 |
US9495345B2 (en) | 2016-11-15 |
US20200234002A1 (en) | 2020-07-23 |
US20170235813A1 (en) | 2017-08-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20210232761A1 (en) | Methods and systems for improving machine learning performance | |
US20240078386A1 (en) | Methods and systems for language-agnostic machine learning in natural language processing using feature extraction | |
CN107368515B (en) | Application page recommendation method and system | |
US9147154B2 (en) | Classifying resources using a deep network | |
US20210110111A1 (en) | Methods and systems for providing universal portability in machine learning | |
US11651015B2 (en) | Method and apparatus for presenting information | |
CN107798622B (en) | Method and device for identifying user intention | |
US20220121668A1 (en) | Method for recommending document, electronic device and storage medium | |
CN112101042A (en) | Text emotion recognition method and device, terminal device and storage medium | |
CN110990527A (en) | Automatic question answering method and device, storage medium and electronic equipment | |
CN110738056B (en) | Method and device for generating information | |
CN106383865B (en) | Artificial intelligence based recommended data acquisition method and device | |
CN114691850A (en) | Method for generating question-answer pairs, training method and device of neural network model | |
CN114610867A (en) | Label description information generation method and device, electronic equipment and storage medium | |
CN116701604A (en) | Question and answer corpus construction method and device, question and answer method, equipment and medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: IDIBON, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ERLE, SCHUYLER D.;MUNRO, ROBERT J.;CALLAHAN, BRENDAN D.;AND OTHERS;SIGNING DATES FROM 20160119 TO 20160226;REEL/FRAME:055945/0346 |
|
AS | Assignment |
Owner name: IDIBON (ASSIGNMENT FOR THE BENEFIT OF CREDITORS), LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IDIBON, INC.;REEL/FRAME:055978/0362 Effective date: 20160519 |
|
AS | Assignment |
Owner name: HEALY, TREVOR, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IDIBON (ASSIGNMENT FOR THE BENEFIT OF CREDITORS), LLC;REEL/FRAME:056057/0325 Effective date: 20161010 |
|
AS | Assignment |
Owner name: AIPARC HOLDINGS PTE. LTD., SINGAPORE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEALY, TREVOR;REEL/FRAME:056083/0123 Effective date: 20181006 |
|
AS | Assignment |
Owner name: AI IP INVESTMENTS LTD, VIRGIN ISLANDS, BRITISH Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AIPARC HOLDINGS PTE. LTD.;REEL/FRAME:056096/0278 Effective date: 20210114 |
|
AS | Assignment |
Owner name: 100.CO, LLC, FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AI IP INVESTMENTS LTD.;REEL/FRAME:056145/0509 Effective date: 20210414 |
|
AS | Assignment |
Owner name: 100.CO TECHNOLOGIES, INC., FLORIDA Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNOR:100.CO, LLC;REEL/FRAME:062131/0714 Effective date: 20221214 |
|
AS | Assignment |
Owner name: DAASH INTELLIGENCE, INC., FLORIDA Free format text: CHANGE OF NAME;ASSIGNOR:100.CO TECHNOLOGIES, INC.;REEL/FRAME:064347/0117 Effective date: 20230118 |
|
AS | Assignment |
Owner name: 100.CO GLOBAL HOLDINGS, LLC, FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DAASH INTELLIGENCE, INC.;REEL/FRAME:064420/0108 Effective date: 20230713 |
|
AS | Assignment |
Owner name: AI IP INVESTMENTS LTD., UNITED KINGDOM Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:100.CO GLOBAL HOLDINGS, LLC;REEL/FRAME:066636/0583 Effective date: 20231220 |