WO2020131004A1

WO2020131004A1 - Domain-independent automated processing of free-form text

Info

Publication number: WO2020131004A1
Application number: PCT/US2017/068911
Authority: WO
Inventors: Ahmet AKYAMAC; Rajarshi BHOWMIK
Original assignee: Nokia Technologies Oy; Nokia USA Inc
Current assignee: Nokia Technologies Oy; Nokia USA Inc
Priority date: 2017-12-29
Filing date: 2017-12-29
Publication date: 2020-06-25
Anticipated expiration: 2020-06-29

Abstract

A method and apparatus for improved free-form text processing that is domain agnostic and automatically identifies key topics for textual data which does not require prior training or supervision, pre-labelling or annotation of the data, or domain expertise. A multi-stage process (i.e., a text processing pipeline) is utilized that is domain-independent, automated and unsupervised. Given a set of free-form text documents, the text processing pipeline leverages both intradocument and corpus-wide (i.e., inter-document) word occurrence statistics for high-quality key phrase extraction. Automatic feature representation is performed using the extracted key phrases and dimensionality reduction is applied that minimizes the distance between near-similar documents in the vector space. For each text document, the text processing pipeline assigns either a hierarchical category or a topic identified by topical key words.

Description

Domain-Independent Automated Processing of Free-Form Text

TECHNICAL FIELD

[0001] The present invention relates generally to data analytics, and, more particularly, to a text processing pipeline that is domain-independent, automated, and unsupervised.

BACKGROUND OF THE EMBODIMENTS

[0002] The use and application of unstructured and textual data has become quite prevalent, with some reports indicating that such data can comprise up to 80% of all data stored by service providers. Some typical examples include textual free-form, unstructured and semi-structured data from customer care tickets, surveys, social media, machine logs, alarm and alerting systems, and diagnostics. There is an important business need to rapidly and automatically understand the key topics and topic categories represented in collections of text. This information can be used in lieu of the complete text corpus, and is often a key enabler of data analytics activities such as automatic machine learning and personalized data exploration. This combination of automation and personalization of the machine learning tasks enables data analytics techniques to be rapidly and widely deployed for use in diverse settings, and a wide variety of applications and business organizations.

[0003] Currently, methods that operate on textual data are typically viewed on a scale having two extremes. On one end of the scale, domain experts manually and extensively analyze textual data to generate a variety of particular insights; for example, to uncover systemic issues in network deployment, or to track hidden patterns in diagnostics information to debug problems, etc. As will be appreciated, this is a very time-consuming process, and often results in a very small set of addressed problems given the resource requirements.

[0004] On the other end of the scale, natural language processing techniques such as semantic analysis are used to automatically summarize textual data. These techniques often require an extensive training set of previously labelled or annotated textual data to develop models, and are often domain-specific. For example, in the telecommunications and networking-based textual data domains, special challenges are presented in applying semantic analysis, as these domains often contain extraneous, non-standard abbreviations, or synonym ized text.

[0005] Prior text processing and categorization techniques can be broadly classified into two categories: rule-based approaches and semantic/linguistic property based approaches. Rule-based approaches require significant domain knowledge to extract key information from text. For example, a regular expression based pattern extraction technique for log analysis was proposed in“Semi-automatic Discovery of Extraction Patterns for Log Analysis”, (white paper by Carasso, David. 2007, available from Splunk, Inc. at www.splunk.com) where the natural language processing (NLP) employs a natural language toolkit (NLTK) supporting part of speech (POS) pattern matching using regular expressions. Further, in the insurance business domain, there exists an intelligent text analyzer (see, Glasgow, Barry, Alan Mandell, Dan Binney, Lila Ghemri, and David Fisher. 1998.“MITA: An Information-Extraction Approach to the Analysis of Free-Form Text in Life Insurance Applications”, Al Magazine 19:59, doi:10.1609/aimag.v19i1.1354) that leverages standard natural language processing tools along with knowledge engineering and machine learning for information extraction from free-form text. However, the extraction module uses if-then rules using pattern matching mechanism. Thus, these rule- based techniques are often domain specific and do not generalize well for other types of free-form text data.

[0006] With respect to semantic/linguistic property based approaches these leverage the semantic relations among words and phrases to extract key information. One such approach proposes using a knowledge graph-based framework for processing structured log files using semantic relationships and ontology (see, Nimbalkar, Piyush, Varish Mulwad, Nikhil Puranik, Anupam Joshi, and Tim Finin. 2016. “Semantic Interpretation of Structured Log Files", 2016 IEEE 17th International Conference on Information Reuse and Integration (IRI), IEEE 549-555). However, this method is not suitable for unstructured text data of variable lengths and formats.

[0007] Categorization of unstructured text data depends heavily on the extraction of keywords or phrases that entail identifying the key concept of the text. In this categorization field, automatic key-phrase extraction is an established line of approaches which can be broadly categorized into two classes: supervised and unsupervised. In several known supervised approaches (see: Riloff, Ellen, and Wendy Lehnert. 1994, Information extraction as a basis for high-precision text classification" , ACM Transactions on Information Systems (TOIS) 296-333; Turney, Peter D. 2000, “Learning algorithms for keyphrase extraction." Information retrieval (Springer) 303-336; and Hulth, Anette 2003, “Improved Automatic Keyword Extraction Given More Linguistic Knowledge ", Proceedings of the 2003 conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 216-223) key phrase annotated datasets are leveraged to train the models that learn to determine whether a candidate phrase is a key phrase or not.

[0008] Alternatively, unsupervised approaches for key phrase extraction can be categorized into several groups (see, Hasan, Kazi Saidul, and Vincent Ng. 2014.“Automatic Keyphrase Extraction: A Survey of the State of the Art’, Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland: Association for Computational Linguistics. 1262-1273). In one approach a ranking of candidate phrases is obtained based on intra-document word co-occurrence statistics (see, Mihalcea, Rada, and Paul Tarau, 2004,“TextRank: Bringing Order into Text’, EMNLP, 404- 411 ) which uses a graph-based ranking algorithm to assign ranking scores to candidate key phrases. Another approach is proposed which uses certain heuristics to generate candidate phrases and assigns ranking scores using degree and frequency of words in the word co-occurrence matrix (see, Rose, Stuart, Dave Engel, Nick Cramer, and Wendy Cowley, 2010,“Automatic Keyword Extraction from Individual Documents" , Text Mining: Applications and Theory, 1 - 20). In another approach corpus-wide word co-occurrence statistics are leveraged to identify relevant topics for the documents in the corpus and performs topical key phrase extraction (see, Liu, Zhiyuan, Wenyi Huang, Yabin Zheng, and Maosong Sun, 2010, “Automatic Key Phrase Extraction via Topic Decomposition" , Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 366- 376). However, these approaches all have their own drawbacks. TextRank uses a fixed size window to generate candidate key phrases, imposing a limit to the length of the key phrases. Also, words with stronger connectivity (i.e. higher degree) are preferred as key words, which may not always be relevant to the main topic of the document. RAKE, on the other hand, tends to assign higher scores to longer candidate key phrases than the shorter ones, even in cases where the shorter ones may be more relevant. Topical Key Phrase Ranking is more suitable for longer text documents where each document is typically a mixture of multiple topics. For short length text document that usually covers a single topic (e.g. a customer care ticket talks about one particular problem, one line of a machine log tells about one particular event), the quality of the extracted key phrases is very poor. Another drawback of Topical Key Phrase ranking is, lesser represented topics in the corpus are difficult to discover.

[0009] Therefore, a need exists for an improved free-form text processing technique that is domain agnostic and automatically identifies key topics for textual data, and does not require prior training or supervision, pre labelling or annotation of the data, or domain expertise.

BRIEF SUMMARY OF THE EMBODIMENTS

[0010] In accordance with various embodiments, a method and apparatus are provided for improved free-form text processing that is domain agnostic and automatically identifies key topics for textual data which does not require prior training or supervision, pre-labelling or annotation of the data, or domain expertise. [0011] In accordance with an embodiment, a multi-stage process (i.e. , a text processing pipeline) is utilized that is domain-independent, automated and unsupervised. Illustratively, given a set of free-form text documents, the text processing pipeline leverages both intra-document and corpus-wide (i.e., inter document) word occurrence statistics for high-quality key phrase extraction. Automatic feature representation is performed using the extracted key phrases and dimensionality reduction is applied that minimizes the distance between near-similar documents in the vector space. Further, for each text document, the text processing pipeline assigns either a hierarchical category or a topic identified by topical key words.

[0012] In this way, the free-form text processing achieved by the embodiments are (i) domain-independent: the technique is applicable to a diverse set of free-form text data; (ii) automated: automatically extracted key phrases and their corresponding ranking scores are used as features for each document; and there is no requirement of any human intervention (for data cleaning, for example) and/or any domain expertise (for feature engineering, for example); (iii) unsupervised: all stages in the text processing pipeline are unsupervised and do not require any annotated or labeled training data (thereby eliminating the need to obtain such training data which in many cases is difficult to reliably obtain); (iv) leveraging both intra-document and corpus-wide occurrence statistics; and (v) improving decision making: the text processing pipeline is able to decide whether a text document is categorical or not based on the density of the clusters generated by a hierarchical clustering of the text documents. In other words, the text processing pipeline assigns a categorical label to a text document if it is a part of a very dense cluster, otherwise, if the cluster density is below a threshold value, the text processing pipeline assigns a topic to the text document which is identified by a set of topical key words.

[0013] In accordance with the embodiments, as noted above, a combination of intra-document and corpus-wide co-occurrence statistics are leveraged to identify key phrases of variable length. In particular, a single document key phrase extraction technique is applied to identify possible key phrases from each text document solely based on the intra-document word co occurrence statistics. Illustratively, a list of common stop words, lexical markers and punctuations is provided as phrase delimiters. Based on such phrase delimiters, a set of candidate phrases is generated. The candidate phrases are then split, illustratively, into words for constructing the word co-occurrence matrix. The candidate key phrases are then scored and ranked using this single document word co-occurrence matrix. But as mentioned earlier, such extracted key phrases are biased towards longer key phrases compared to the shorter ones. Therefore, there is a need to fine tune these extracted key phrases. To do so, corpus-wide inter-document word co-occurrence statistics are used. This approach helps to assign a relevance score to each of the previously extracted key phrases. This relevance score is used to rank and prune the set of candidate key phrases to obtain the most relevant key phrases for each document. Next, each text document is represented as a sparse, high-dimensional vector using the extracted relevant key phrases and their respective ranking scores as features, and a dimensionality reduction is performed using a locality-sensitive hashing technique. Further, an agglomerative hierarchical clustering technique is applied to cluster similar text documents to facilitate the assignment of both topics and sub-topics to enhance the clustering results, and a measure of the density of the clusters generated is made. If the cluster density is higher than a pre-defined threshold value, the text documents are annotated to belong to that cluster, and the cluster is considered to be categorical. Otherwise, the cluster is not considered categorical. Instead, the top-K most frequent phrases from all the documents within the cluster are selected as representatives of the topic for the cluster, and the text documents are annotated to be part of this topic.

[0014] These and other advantages of the embodiments will be apparent to those of ordinary skill in the art by reference to the following detailed description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG. 1 shows a flowchart of illustrative operations for automated domain-independent free-form text processing in accordance with an

embodiment;

[0016] FIG. 2 shows plate notation model for a Latent Dirichlet

Allocation (LDA) variant in accordance with an embodiment;

[0017] FIG. 3 shows a flowchart of illustrative operations for word generation for text documents in accordance with an embodiment;

[0018] FIGs. 4A - 4I shows an illustrative scenario and exemplary results in applying automated domain-independent free-form text processing in a technical service ticket/customer care domain in accordance with an embodiment; and

[0019] FIG. 5 is a high-level block diagram of an exemplary computer in accordance with an embodiment.

DETAILED DESCRIPTION

[0020] In accordance with various embodiments, a method and apparatus are provided for improved free-form text processing that is domain agnostic and automatically identifies key topics for textual data which does not require prior training or supervision, pre-labelling or annotation of the data, or domain expertise. In accordance with an embodiment, a text processing pipeline is utilized that is domain-independent, automated, and unsupervised. Illustratively, given a set of free-form text documents, the text processing pipeline leverages both intra-document and corpus-wide word occurrence statistics for high-quality key phrase extraction. Automatic feature representation is performed using the extracted key phrases and dimensionality reduction is provided that minimizes the distance between near-similar documents in the vector space. Further, for each text document, the text processing pipeline assigns either a hierarchical category or a topic identified by topical key words. [0021] FIG. 1 shows a flowchart of illustrative operations 100 for automated domain-independent free-form text processing in accordance with an embodiment. The processing of the free-form text occurs, illustratively, in a multi-stage text processing pipeline which will now be discussed in more detail. At step 105, the textual data input that is to be processed is received, presented, or otherwise provided (e.g., textual free-form, unstructured and semi-structured data from customer care tickets, surveys, social media, machine logs, alarm and alerting systems, and diagnostics, to name just a few input types). Illustratively, the textual data input may be in the form of one or more so-called text documents (e.g., a technical/customer care service ticket). At step 110, key-phrase extraction is applied to each text document as each document is essentially a collection of words. In accordance with an embodiment, RAKE is applied to identify the extracted key-phrases from each text document, at step 115. Steps 110 and 115 illustratively comprise a single document key-phrase extraction state (i.e. , intra-document processing; Stage 1 ). At step 110, a list of common stop words, lexical markers and punctuations is provided as phrase delimiters. Based on such phrase delimiters, a set of candidate key phrases are generated using any number of well-known automated keyword extraction techniques such as RAKE. For example, in a customer care application these key phrases might be“service outage”, “technical dispatched” or“instruction manual not adhered to”, to name just a few.

[0022] The candidate key phrases are then split, illustratively, into words for constructing the so-called word co-occurrence matrix. Illustratively, RAKE uses two measures - the frequency and degree of a word to compute the score of a candidate key phrase. The degree of a word is measured as the sum of its frequency in the document and the number of its co-occurrence with other words across all candidate key phrases of the text document. If a candidate phrase has multiple words, the scores of all the words are summed up. After computing the scores, a descending order ranking is obtained for the candidate phrases and a selection is made of some percentage (illustratively, the top seventy percent (70%)) thereof are selected as the extracted key-phrases. [0023] As such, the results of Stage 1 (i.e. , the intra-document processing) and Stage 2 (i.e., the inter-document processing) result in the automated generation of key words and key phrases that may be useful in ultimately examining and categorizing the input documents. Essentially, in the context of the embodiments herein, these key words and key phrase are akin to a generated“dictionary” for further use in the automated processing of the free- text input documents. Next, at step 120, a word distribution of topics and topic distribution of the documents is obtained. Illustratively these are obtained using a variant of Latent Dirichlet Allocation (LDA). Steps 125-130 represent a topical key word ranking stage (i.e., Stage 2). As will be appreciated, LDA is a well- known generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Referring briefly to FIG. 2, a variant of LDA is shown (as specially configured for the embodiments herein) in plate notation model 200. With plate notation, the dependencies among the many variables can be captured concisely. The boxes are so-called “plates” representing replicates. The outer plate represents documents, while the inner plate represents the repeated choice of topics and words (e.g., word W_m>n 255) within a document. M 235 denotes the number of documents, N 240 the number of words in a document in accordance with the embodiment shown.

[0024] As shown in plate notation model 200, the domain specific jargons are assumed to have a Dirichlet prior cp_D 205 parameterized by b 210, the topic distribution r 215 is assumed to have a Dirichlet prior parameterized by a 220 and p 250 is the Dirichlet prior with parameter g 220 for choosing between topic words and domain specific jargons. For each topic k, a word distribution is assumed with a Dirichlet prior cp_k parameterized by b. z_dm 225 denotes the topic of the m-th text document in the corpus and y_m,n 230 is a binary variable indicating whether the n-th word of the m-th document is a topic word or a domain specific jargon. [0025] As will be appreciated, traditional LDA assumes each document to be a mixture of a few topics. However, this assumption does not necessarily hold true in accordance with the embodiment herein as, for example, each ticket(s) or log(s) or diagnostic(s) can essentially be directed to one topic. As noted above, plate notation model 200 is a specially configured LDA variant for use in the embodiments herein. Additionally, there are certain domain specific jargons which are not associated to a specific topic. However, such words cannot be removed from the text as they bear significant semantic meaning. In the LDA variant embodiment shown in FIG. 2, an assumption is made that each word in each text document is either a domain specific jargon or a topic word.

[0026] FIG. 3 is flowchart of operations 300 detailing the word generation process for each text document. In this process, as detailed above, the domain specific jargons are assumed to have a Dirichlet prior cp_D 245 parameterized by b 210, the topic distribution t is assumed to have a Dirichlet prior parameterized by a and p is the Dirichlet prior with parameter g for choosing between topic words and domain specific jargons. At step 310, the Dirichlet prior cp_D 245 is chosen. At steps 320 and 330, for each topic k, we assume a word distribution with a Dirichlet prior cp_k parameterized by b. z,j denotes the topic of the m-th text document in the corpus and y_{m n} 230 is a binary variable indicating whether the n-th word of the m-th document. In steps 310- 330, Dir(), Categorical() and Bernoulli() represent probability distribution functions such as Dirichlet, Categorical and Bernoulli distribution functions, respectively. Step 330 provides details of the generation process of each document d_m. First, one of the K (K is the number of possible topics) topics is sampled as the representative topic of d_m from the categorical distribution of topics with parameter T, illustratively, Categorical(T). For the n-th word of document d_m illustratively, w_m,n, a binary random variable y_m,n is sampled from the Bernoulli distribution parameterized by TT. If the value of the random variable y_m,n is 0, then the word w_{m n} is identified to be a domain specific jargon and hence, is sampled from the Categorical distribution of domain specific jargons with the Dirichlet prior (P_{D .} On the other hand, if the value of the random variable y_m,n is 1 , then the word w_m,n is identified to be a topical word for the previously chosen topic of the document and is sampled from the Categorical distribution of the chosen topic with the Dirichlet prior cp_{k .}

[0027] Referring back to FIG. 1 , having completed the single document key-phrase and topical word ranking stages (i.e. , Stage 1 and Stage 2), as detailed above, topical key phrase ranking is performed at step 135 (i.e., Stage 3). In accordance with the embodiment, at step 135, the topical key word ranking obtained in the topical word ranking stage is used to obtain a ranking of the candidate key phrases generated in Stage 1. Any automatic key phrase extraction technique may be utilized at step 135, illustratively, the technique described herein above in Liu, Zhiyuan, Wenyi Huang, Yabin Zheng, and Maosong Sun. 2010. “Automatic Key Phrase Extraction via Topic Decomposition" , Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 366- 376, may be used. After ranking the candidate phrases in descending order, a selection is made of a certain percentage (e.g., the top thirty percent (30%)) as the key phrases of the text document.

[0028] At step 140 (i.e., Stage 4), each text document is represented as a sparse, high-dimensional vector (i.e., a collection of features) using the extracted key phrases and their respective ranking scores as features, and step 145 (i.e., Stage 5) dimensionality reduction is performed using a locality-sensitive hashing technique (for example, the so-called“SimHash” technique described in Charikar, Moses S. 2002, “Similarity Estimation Techniques from Rounding Algorithms.” Proceedings of the 34th Annual ACM Symposium on Theory of Computing, ACM, 380-388). This reduces the dimensionality and obtains a dense representation for each text document, and also ensures that near-similar text documents are close to one another in the vector space of reduced dimensionality. Of course, if processing speed and/or available memory are not issues, step 145 may be optional given the resulting compression obtained thereof improves overall speed performance. [0029] At step 150 (i.e. , Stage 6), an agglomerative hierarchical clustering technique such as scipy.clustering. agglomerative is applied to cluster similar text documents. In this way, hierarchical clustering enables the automated assignment of both topics and sub-topics to enhance the clustering results such that the documents are clustered with each cluster being assigned specific key words and/or key phrases. This allows facilities the examination and manipulation of the clusters as a function of the automatically assigned key words and/or key phrases given the assigned key words and/or key phrases are a subset of the results from Stage 3, as detailed above. At step 155 (i.e., Stage 7), there is a generation of categories (and/or groups) of the text documents by topic that is facilitated by measuring the density of the clusters generated in Stage 6. If the cluster density is higher than a pre-defined threshold value, the text documents are annotated to belong to that cluster, and the cluster is considered to be categorical. Otherwise, the cluster is not considered categorical. Instead, the top-K most frequent phrases from all the documents within the cluster are selected as representatives of the topic for the cluster, and the text documents are annotated to be part of this topic.

[0030] Advantageously, the embodiments facilitate the free-form text processing that is automated, domain-independent, topic-independent, and document size-independent having a high quality level. Further, the embodiments herein may be utilized for a wide variety of use cases. Some examples include, but are not limited to, (i) categorization of unstructured machine log messages: (ii) understanding groups and main topics of problems faced by customers as described in free form text in customer care tickets; (iii) revealing groups of topics with positive, neutral and negative sentiments from free-form customer survey verbatims; and (iv) preparing unstructured textual information by either categorizing it, or extracting topical key phrases to be used in lieu of the full text as part of the pre-processing steps of automated machine learning.

[0031] FIGs. 4A - 4I shows an illustrative scenario and exemplary results in applying automated domain-independent free-form text processing in a technical service ticket/customer care domain in accordance with an embodiment. In particular, FIG. 4A shows service ticket 400 which is textual data input (see, step 105 of FIG. 1 ) in accordance with the embodiment. Service ticket 400 has original text 405 which is representative of a textual excerpt from an illustrative network operations trouble ticket (i.e. , ticket / in a set of n tickets, 1 ... n). For example, as shown in FIG. 4B, service ticket 400 is shown as part of the set of 235 ( n = 235) service tickets that also includes service ticket 410, 415, and 420, respectively.

[0032] In accordance with embodiment, the completion of Stage 1 processing (as described herein above; see FIG. 1 ) results in the extraction of a plurality of candidate key phrases for each document. For example, FIG. 4C shows such results for service ticket 400 where plurality of candidate key phrases 425 have been extracted which include the following exemplary key phrases:“procedure integrating emc vnx storage array” 425-1 ,“boot screen” 425- 2, and“local team” 425-3, to highlight just a few.

[0033] FIG. 4D shows results 430 from the completion of Stage 2 processing (as described herein above; see FIG. 1 ) for topical key word 430-1 and normalized ranking 430-2 having a plurality of key words 430-3 by normalized rank. Fig 4D shows the top 25 topical key words in descending order of normalized rank, out of a total of 5126 key words extracted from n = 235 service tickets in this illustrative example. In this way, the top ranked key phrases may be identified in Stage 3 (as described herein above; see FIG. 1 ) with FIG. 4E showing results 435 for the top twelve (12) ranked key phrases (i.e., ranked key phrase 435-1 through 435-11 in descending order) out of a total of 79 key phrases for ticket 400 (i.e., ticket /).

[0034] In the illustrative example under discussion, Stage 4 (as described herein above; see FIG. 1 ) processing results in a total of 10,852 key phrases identified from n = 235 service tickets. As shown in FIG. 4F, feature vector 440 for ticket 400 (i.e., ticket /) has 10,852 dimensions (i.e., dimension 440-1 through 440-DM) with each dimension representing a respective one of the 10,852 key phrases. Flere, the feature vector for ticket 400 (i.e., ticket /) is non- zero (the shaded regions shown in FIG. 4F) for the key phrases found in that ticket. Thus, the shaded boxes in FIG. 4F indicate key phrases that are found in ticket 400. It is to be understood that FIG. 4F is a partial representation of the feature vector for ticket 400 illustrating 5 shaded boxes, which in the full representation has a total of 79 shaded boxes corresponding to the 79 key phrases in ticket 400 out of the total 10,852 key phrases identified from n = 235 service tickets. FIG. 4F shows the high dimensional representation of ticket 400. This feature vector is sparse, wastes memory, and increases computational requirements. As such, in accordance with the embodiments herein, dimensionality reduction is applied in Stage 5 (as previously described herein above; see FIG. 1 ).

[0035] More particularly, results 445 shown in FIG. 4G is a compact feature representation for ticket 400 after dimensionality reduction is applied in accordance with the embodiment. Advantageously, the previous 10,852 dimensions (i.e. , dimension 440-1 through 440-DM) have been significantly reduced to a set of thirty-two (32) dimensions (i.e., dimension 445-1 through 445- DM) each dimension represented by bit 445-1 and value 445-3.

[0036] FIG. 4H shows results 450 which are the clusters (i.e., cluster 450-1 through 450-12 resulting from using hierarchical agglomerative clustering in Stage 6 (as described herein above; see FIG. 1 ). Note, in this example, ticket 400 is in cluster 450-10. From here, Stage 7 (as described herein above; see FIG. 1 ) processing results in a category or topic assignment. In this illustrative example, the density of cluster 450-10 was found to be lower than the pre defined threshold value. Thus, as shown in FIG. 4I, results 460 show no category (i.e., category 460-1 ) being assigned and topic assignment 465 having the assignment of topics 465-1 , 465-2, 465-3, 465-4, 465-5, 465-6, 465-7, 465-8, 465-9, and 465-10 for the cluster in which ticket 400 resides (i.e., cluster 450-10). In this way, in accordance with embodiment, for each text document, the text processing pipeline automatically assigns either a hierarchical category or a topic identified by topical key words (for example, “disk” or“system” as shown in results 460). These automatically-assigned categories and/or topical key words can then be utilized to more efficiently process, categorize, operate on, and/or ultimately resolve the free-form text (i.e. , the original input) in a rapid way without the need for extensive prior training or supervision, pre-labelling or annotation of the data, or domain expertise.

[0037] As detailed above, the various embodiments herein can be embodied in the form of methods and apparatuses for practicing those methods. The disclosed methods may be performed by a combination of hardware, software, firmware, middleware, and computer-readable medium (collectively “computer”) installed in and/or communicatively connected to a user device. FIG. 5 is a high-level block diagram of an exemplary computer 500 that may be used for implementing a method for domain-independent automating processing of free-form text in accordance with the various embodiments herein. Computer 500 comprises a processor 510 operatively coupled to a data storage device 520 and a memory 530. Processor 510 controls the overall operation of computer 500 by executing computer program instructions that define such operations. Communications bus 560 facilitates the coupling and communication between the various components of computer 500. Of course, computer 500 may be any type of computing device such a computer, tablet, server, mobile device, smart phone, to name just a few. The computer program instructions may be stored in data storage device 520, or a non-transitory computer readable medium, and loaded into memory 530 when execution of the computer program instructions is desired.

[0038] Thus, the steps of the disclosed method (see, e.g., FIGs. 1 and 3) and the associated discussion herein above) can be defined by the computer program instructions stored in memory 530 and/or data storage device 520 and controlled by processor 510 executing the computer program instructions. For example, the computer program instructions can be implemented as computer executable code programmed by one skilled in the art to perform the illustrative operations defined by the disclosed method. Accordingly, by executing the computer program instructions, processor 510 executes an algorithm defined by the disclosed method. Computer 500 also includes one or more communication interfaces 550 for communicating with other devices via a network (e.g., a wireless communications network) or communications protocol (e.g., Bluetooth®). For example, such communication interfaces may be a receiver, transceiver or modem for exchanging wired or wireless communications in any number of well-known fashions. Computer 500 also includes one or more input/output devices 540 that enable user interaction with the user device (e.g., camera, display, keyboard, mouse, speakers, microphone, buttons, etc.).

[0039] Processor 510 may include both general and special purpose microprocessors, and may be the sole processor or one of multiple processors of computer 500. Processor 510 may comprise one or more central processing units (CPUs), for example. Processor 510, data storage device 520, and/or memory 530 may include, be supplemented by, or incorporated in, one or more application-specific integrated circuits (ASICs) and/or one or more field programmable gate arrays (FPGAs).

[0040] Data storage device 520 and memory 530 each comprise a tangible non-transitory computer readable storage medium. Data storage device 520, and memory 530, may each include high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR RAM), or other random access solid state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices such as internal hard disks and removable disks, magneto-optical disk storage devices, optical disk storage devices, flash memory devices, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), digital versatile disc read-only memory (DVD-ROM) disks, or other non-volatile solid state storage devices.

[0041] Input/output devices 540 may include peripherals, such as a camera, printer, scanner, display screen, etc. For example, input/output devices 540 may include a display device such as a cathode ray tube (CRT), plasma or liquid crystal display (LCD) monitor for displaying information to the user, a keyboard, and a pointing device such as a mouse or a trackball by which the user can provide input to computer 500.

[0042] It should be noted that for clarity of explanation, the illustrative embodiments described herein may be presented as comprising individual functional blocks or combinations of functional blocks. The functions these blocks represent may be provided through the use of either dedicated or shared hardware, including, but not limited to, hardware capable of executing software. Illustrative embodiments may comprise digital signal processor (“DSP”) hardware and/or software performing the operation described herein. Thus, for example, it will be appreciated by those skilled in the art that the block diagrams herein represent conceptual views of illustrative functions, operations and/or circuitry of the principles described in the various embodiments herein. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo code, program code and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer, machine or processor, whether or not such computer, machine or processor is explicitly shown. One skilled in the art will recognize that an implementation of an actual computer or computer system may have other structures and may contain other components as well, and that a high level representation of some of the components of such a computer is for illustrative purposes.

[0043] The foregoing Detailed Description is to be understood as being in every respect illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is not to be determined from the Detailed Description, but rather from the claims as interpreted according to the full breadth permitted by the patent laws. It is to be understood that the embodiments shown and described herein are only illustrative of the principles of the present invention and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the invention. Those skilled in the art could implement various other feature combinations without departing from the scope and spirit of the invention.

Claims

What is claimed is:

1. A method for analyzing textual input, the method comprising:

receiving the textual input, the textual input comprising a plurality of documents with each document of the plurality of documents comprising a collection of words;

for each document in the plurality of documents:

extracting a respective plurality of candidate key words therefrom; and

extracting, using the respective plurality of candidate key words from each document of the plurality of documents, a respective plurality of candidate key phrases therefrom;

generating a plurality of topical key words from the plurality of documents;

individually scoring and ranking each topical key word in the plurality of topical key words;

for each candidate key phrase in the respective plurality of candidate key phrases of each document:

identifying particular ones of the topical key words in the candidate key phrase; and

individually scoring the candidate key phrase using the individual scores of the particular ones of the topical key words identified in the candidate key phrase;

individually ranking each candidate key phrase in the respective pluralities of candidate key phrases using the individual scoring of the candidate key phrase;

generating, using the individual ranking of each candidate key phrase, a respective feature vector for each document in the plurality of documents;

clustering the respective feature vectors into a plurality of clustered feature vectors; and

generating, from the plurality of clustered feature vectors, one or more outputs that are representative of the textual input.

2. The method for analyzing textual input of claim 1 , wherein the extracting the respective plurality of candidate key phrases further comprises:

defining a plurality of phrase delimiters using a plurality of common stop words, a plurality of lexical markers, and a plurality of punctuations; and

using the plurality of phrase delimiters for the extracting of the respective plurality of candidate key phrases.

3. The method for analyzing textual input of claim 1 , wherein the textual input is one of free-form text, unstructured text, and semi-structured text.

4. The method for analyzing textual input of claim 2, the method further comprising:

receiving the plurality of common stop words, the plurality of lexical markers, and the plurality of punctuations.

5. The method for analyzing textual input of claim 3 wherein the textual input is related to one of a technical service ticket, a social media stream, a machine log, a diagnostic tool, and an alarm system.

6. The method for analyzing textual input of claim 2 wherein at least one candidate key phrase in the respective plurality of candidate key phrases is extracted using RAKE automated keyword extraction.

7. The method for analyzing textual input of claim 1 further comprising:

generating a co-occurrence matrix from the respective pluralities of candidate key phrases, and

using the co-occurrence matrix in the scoring of each candidate key phrase, wherein the individual scoring of the candidate key phrase is established in part by a summation of the individual scores of the particular ones of the topical key words identified in the candidate key phrase.

8. The method for analyzing textual input of claim 1 further comprising:

performing dimensionality reduction on each respective feature vector for each document in the plurality of documents to obtain a compact feature representation for each document.

9. The method of 8 wherein the plurality of candidate key phrases and their respective individual ranking are features used to obtain the compact feature representation.

10. The method for analyzing textual input of claim 1 , wherein the clustering the respective feature vectors into the plurality of clustered feature vectors uses agglomerative hierarchical clustering for clustering particular ones of the documents of the plurality of documents and associating the particular ones of the documents with a respective cluster of the plurality of clustered feature vectors.

11. The method for analyzing textual input of claim 10, wherein the clustering of the particular ones of the documents of the plurality of documents automatically assigns one or more of the candidate key phrases to the particular ones of the documents clustered.

12. The method for analyzing textual input of claim 10, wherein the clustering of the particular ones of the documents of the plurality of documents automatically assigns one or more of the topical key words to the particular ones of the documents clustered.

13. The method for analyzing textual input of claim 12, the method further comprising:

measuring a density of the plurality of clustered feature vectors; and if the density measured is above a pre-defined threshold value then assigning a categorical reference for the particular ones of the documents associated with the respective cluster; and

if the density measured is equal to or below the pre-defined threshold value then assigning one or topics for the particular ones of the documents associated with the respective cluster by selecting a subset of the one or more of the topical key words assigned to the particular ones of the documents associated with the respective cluster.

14. The method of claim 13, the method further comprising:

annotating the particular ones of the documents associated with the respective cluster with the categorical reference assigned.

15. An apparatus for analyzing textual input, comprising:

a processor; and

a memory storing instructions that when executed cause the processor to perform operations comprising:

receiving the textual input, the textual input comprising a plurality of documents with each document of the plurality of documents comprising a collection of words; for each document in the plurality of documents:

extracting a respective plurality of candidate key words therefrom; and extracting, using the respective plurality of candidate key words from each document of the plurality of documents, a respective plurality of candidate key phrases therefrom;

generating a plurality of topical key words from the plurality of documents;

individually ranking each topical key word in the plurality of topical key words; for each candidate key phrase in the respective plurality of candidate key phrases of each document:

identifying particular ones of the topical key words in the candidate key phrase; and individually scoring the candidate key phrase using the individual scores of the particular ones of the topical key words identified in the candidate key phrase;

16. The apparatus of claim 15 wherein the extracting the respective plurality of candidate key phrases operation further comprises:

17. The apparatus of claim 15 wherein the operations further comprise:

generating a co-occurrence matrix from the respective pluralities of candidate key phrases; and

18. The apparatus of claim 15, wherein the operations further comprise:

19. The apparatus of claim 18 wherein the operations further comprise:

20. The apparatus of claim 15 wherein the textual input is one of free-form text, unstructured text, and semi-structured text relating to one of a technical service ticket, a social media stream, a machine log, a diagnostic tool, and an alarm system.