US20060184526A1 - Customer feedback acquisition and processing system - Google Patents

Customer feedback acquisition and processing system Download PDF

Info

Publication number
US20060184526A1
US20060184526A1 US11/403,977 US40397706A US2006184526A1 US 20060184526 A1 US20060184526 A1 US 20060184526A1 US 40397706 A US40397706 A US 40397706A US 2006184526 A1 US2006184526 A1 US 2006184526A1
Authority
US
United States
Prior art keywords
database
customer feedback
records
block
concept
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/403,977
Inventor
Robert Bossemeyer
Dennis Connolly
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AT&T Intellectual Property I LP
Original Assignee
AT&T Intellectual Property I LP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US09/356,668 priority Critical patent/US6510427B1/en
Priority to US10/242,118 priority patent/US7058625B2/en
Application filed by AT&T Intellectual Property I LP filed Critical AT&T Intellectual Property I LP
Priority to US11/403,977 priority patent/US20060184526A1/en
Publication of US20060184526A1 publication Critical patent/US20060184526A1/en
Assigned to AT&T INTELLECTUAL PROPERTY I, L.P. reassignment AT&T INTELLECTUAL PROPERTY I, L.P. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: SBC PROPERTIES, L.P.
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06QDATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce, e.g. shopping or e-commerce
    • G06Q30/02Marketing, e.g. market research and analysis, surveying, promotions, advertising, buyer profiling, customer management or rewards; Price estimation or determination
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/912Applications of a database
    • Y10S707/917Text
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99931Database or file accessing
    • Y10S707/99933Query processing, i.e. searching
    • Y10S707/99936Pattern matching access

Abstract

A customer feedback acquisition and processing system is disclosed. Customer feedback, which may optionally include voice signals, is captured and stored in a database. The database can be searched to develop a subset of records pertaining to an area of interest. A data mining tool can then be used on the subset to identify trend(s) in the customer feedback records. The database tool assigns relevance scores to each word (“concept”) in one or more fields of the records in the subset. It then combines the concepts and develops new relevance scores for the combined concepts to identify trend(s) in the customer feedback records.

Description

  • The present patent document is a continuation of U.S. application Ser. No. 10/242,118, filed Sep. 12, 2002, pending, which is a divisional of U.S. application Ser. No. 09/356,668, filed Jul. 19, 1999, now U.S. Pat. No. 6,510,427, and the entire disclosure of each of these applications is incorporated herein by reference.
  • FIELD OF THE INVENTION
  • The present invention relates generally to information acquisition and processing systems and, more particularly, to a customer feedback acquisition and processing system for use in obtaining, organizing and analyzing customer feedback related to products and services.
  • BACKGROUND OF THE INVENTION
  • For years, businesses that sell products or provide services have provided their customers or clients with avenues to register comments, complaints or suggestions relevant to the products or services provided by the business. These avenues have included customer feedback postcards and toll free numbers that consumers may call to speak with a service representative.
  • Traditionally, received feedback postcards have been filed in an order relevant to the products or services to which they pertain. Any analysis of the data provided by the postcards required a data analyst to individually process handwritten, and sometimes unreadable, postcards to determine trends in the customer comments on the postcards.
  • Service representatives answering toll free lines have traditionally completed paper-based customer comment forms as they speak to the customer. These handwritten comment forms were processed in a manner similar to the postcards. Specifically, they were filed and hand-analyzed by a data analyst at a later time.
  • In recent years, the advent of the computer has modified how customer feedback is acquired, retained and processed. Handwritten data from feedback postcards may now be keyed or scanned into, and stored by, a computer in an electronic format. Similarly, computer use has simplified the acquisition of information that is provided by customers during calls. Specifically, service representatives may now use a computer terminal with an interface that allows the input of various pieces of information including, for example, an identification of the product or service about which the call was made, the time and date of the call and the comments made by the caller. Additionally, the popularity of network communications over the Internet now allows businesses to receive customer comments via electronic mail (email) and web page feedback techniques.
  • Although the use of computers has simplified the acquisition of customer feedback from telephone calls, the value of the acquired data is dependent on the level of detail the receiving service representatives enter into their terminal user interfaces. Ideally, the service representatives would enter all of the callers' comments into the terminal. However, while some service representatives may enter lengthy customer comments, others may enter very brief descriptions. These brief descriptions may or may not be succinct and descriptive sentences that are meaningful. Accordingly, the quality of the information acquired depends solely on the quality of the service representative's characterization of the telephone call with the customer.
  • Once information pertaining to customer feedback has been acquired electronically, it is useful to analyze the collected data to determine, for example, where improvements in products or services should be made. As noted, the information acquired for each product or service may be analyzed by an analyst who looks for trends in the feedback. Hand analysis of the data is a time consuming and arduous task. If a business offers many products or services, many person-hours must be spent analyzing the customer feedback to determine a trend in the data.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a schematic illustration of a customer feedback acquisition and processing system constructed in accordance with the teachings of the invention.
  • FIG. 2 is a more detailed schematic illustration of the data acquisition processor shown in FIG. 1.
  • FIG. 3 is an exemplary data structure that may be created by the data formatter of FIG. 2 and stored in the database of FIG. 1.
  • FIG. 4 is a more detailed schematic illustration of the database processor of FIG. 1.
  • FIG. 5 is an exemplary search result that may be obtained by execution of the search engine and queue generator of FIG. 4 in response to certain exemplary search criteria.
  • FIG. 6 is an exemplary search queue that may be obtained by execution of the search engine and queue generator, the relevance computer and the relevance sorter of FIG. 4.
  • FIG. 7 is an exemplary search queue that may be obtained by performing a merge cycle on the search queue of FIG. 6.
  • FIG. 8 is an exemplary search queue that may be obtained by performing a merge cycle on the search queue of FIG. 7.
  • FIG. 9 is an exemplary search queue that may be obtained by performing a merge cycle on the search queue of FIG. 8.
  • FIG. 10 is a more detailed schematic illustration of the relevance computer of FIG. 4.
  • FIG. 11 is a more detailed schematic illustration of the merger of FIG. 4.
  • FIGS. 12A-12D are a flowchart illustrating exemplary programmed steps performed by the data acquisition processor of FIG. 1.
  • FIGS. 13A-13B are a flowchart illustrating an exemplary create database index routine.
  • FIGS. 14A-14B are a flow chart illustrating an exemplary main routine implemented by the database processor of FIG. 1.
  • FIGS. 15A-15B are a flow chart illustrating an exemplary initialize search queue routine called by the main routine of FIGS. 14A-14B.
  • FIG. 16 is a flow chart illustrating an exemplary computer relevance scores routine called by the main routine of FIGS. 14A-14B.
  • FIGS. 17A-17C is a flow chart illustrating an exemplary merger routine called by the main routine of FIGS. 14A-14B.
  • FIG. 18 is a flow chart illustrating the calculate global frequency routine.
  • DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • A customer feedback acquisition and processing system (CFAPS) constructed in accordance with the teachings of the invention is schematically illustrated in FIG. 1. Generally, the CFAPS 10 includes a plurality of service representative terminals 12, a data acquisition processor 14, a database 16, a database processor 18 and one or more data analyst terminals 20. A service representative working at one of the service representative terminals 12 may receive customer feedback from either a telephone call via a PSTN connection or from an electronic message (e.g., an email message or a web page message) via a network. The receipt of one telephone call, one email message or one web page message is referred to herein as one customer feedback instance.
  • When a customer feedback instance occurs, a customer feedback message is created and forwarded to the data acquisition processor 14 for storage in the database 16 as a record. For example, if the customer feedback instance is initiated by a telephone call, the service representative will converse with the customer. The service representative may summarize the pertinent information from the conversation and enter the summarized information (e.g., profile information) into the service representation terminal 12 via a graphical user interface. The data entered by the service representative and the audio from the call is forwarded to the data acquisition processor 14 in a customer feedback message. The data acquisition processor 14 parses the message into segments and stores the segments in certain predefined fields of a record in the database 16. Preferably, the voice data from the telephone call is recorded in its entirety and converted from audio information to text information by the data acquisition processor 14 using a large vocabulary speech recognition technology. The text information from the audio data is then stored in association with the summary information provided by the service representative. By way of examples, not limitations, the profile information provided by the service representative may optionally include an identification of the product or service to which the customer feedback instance pertains, the date and/or time the customer feedback instance was received, the reason for the customer feedback, and/or comments regarding the customer feedback. As will be appreciated by persons of ordinary skill in the art, various standardized codes may be defined to represent the product or service about which the feedback was received (e.g., KEP70=a two line telephone, TDG1A=a big button telephone, etc.). Additionally, a standardized code may be used to represent various reasons for the customer feedback (e.g., R02=product return, R03=product complaint, etc.). Therefore, a customer feedback instance that is received because of a product return of a big button telephone may be represented by codes such as TDG1A and R02.
  • The data acquisition processor 14 composites the profile information from the service representative and the text information from the customer (if any) into a single record which is written to the database 16. One record is created for each customer feedback instance. Accordingly, the database 16 includes as many records as there are customer feedback instances. The data acquisition processor 14 is described in further detail below.
  • Although persons of ordinary skill in the art will readily appreciate that the service representation terminal(s) 12 can be implemented in many ways without departing from the scope or spirit of the invention, in the preferred embodiment, the service representation terminals 12 are implemented by networked personal computers.
  • For the purpose of analyzing data in the database 16, the CFAPS 10 is provided with the database processor 18. As shown in FIG. 1, the database processor 18 communicates information to and receives information from both the database 16 and the data analyst terminal 20. The database processor 18 is responsive to commands received from the data analyst terminal 20 to access the database 16 and, among other things, to copy, change and/or manipulate the records stored therein. The data analyst terminal 20 preferably includes a graphical user interface (GUI) that allows a data analyst using the data analyst terminal 20 to control the database processor 18. Using the GUI of the data analyst terminal 20, the data analyst may command the database processor 18 to perform various functions on the records stored in the database 16. These functions may include, but are not limited to, copying records, sorting records, searching for particular records, determining the relevance of a word or group of words (referred to hereinafter as concepts) appearing in the records and combining concepts and determining the relevance of the combined concepts. For example, a data analyst may wish to search for all data relevant to product returns for a specific product (e.g., the big button telephone). To carry out such a search, the analyst may enter search criteria specifying the product (e.g., TDG1A for the big button telephone) and feedback rationale (e.g., R02 for product returns) into product and reason fields in the GUI of the data analyst terminal 20. The database processor 18 is described in further detail below with reference to FIGS. 4 and 10-11.
  • Although for descriptive purposes the data acquisition processor 14 is shown as a stand-alone entity, persons of ordinary skill in the art will readily appreciate that the data acquisition processor 14 may be implemented in either the service representative terminals 12 or the database 16 without departing from the scope or spirit of the invention. Likewise, persons of ordinary skill in the art will recognize that, although for ease of illustration the database processor 18 is shown as a stand-alone entity in FIG. 1, the database processor 18 may be implemented in either the data analyst terminal 20 or the database 16 without departing from the scope or the spirit of the invention. Preferably, however, a data acquisition processor 14 is resident in each of the service representative terminals 12 and a database processor 18 is resident in each of the data analyst terminals 20. Additionally, although a plurality of service representative terminals 12 and a single data analyst terminal are shown in FIG. 1, different numbers of service representative terminal(s) 12 (including one) and/or data analyst terminal(s) 20 may be employed without departing from the scope or spirit of the invention. Moreover, it will be readily understood by persons of ordinary skill in the art that the service representative terminal 12 and the data analyst terminal 20 may optionally be combined into a single terminal that performs both the data acquisition and the data processing functions described herein.
  • As shown in FIG. 2, the data acquisition processor 14 preferably includes a signal identifier 25, a speech recorder 26, a speech recognizer 28, and a data formatter 32. As its name suggests, the signal identifier 25 examines the input signals received from the service representative terminal(s) 12 and determines whether the received signal is an audio signal (e.g., from a conversation on a telephone). If the examined input signal is an audio signal, it is routed to the speech recorder 26. Otherwise, the input signal is sent directly to the data formatter 32.
  • Assuming a customer feedback instance is associated with a telephone call, the audio from the call is preferably communicated to the speech recorder 26. The speech recorder 26 stores the audio from the call in either analog or digital format until the speech recognizer 28 is activated. When activated, the speech recognizer 28 converts the stored audio from the speech recorder 26 into text information that can be more easily stored and searched in the database 16. Persons of ordinary skill in the art will appreciate that, if the speech recognizer 28 is adapted to process audio data in real time, the speech recorder 26 may optionally be eliminated from the data acquisition processor 14. The speech recognizer 28 may be implemented by any of the well known speech recognition software packages which are commercially available from manufactures such as Dragon Systems, Philips, Lucent and Nuance.
  • The profile information entered by the service representative, any text data received from the network (e.g., when the subject customer feedback instance is associated with an email message or with data received via a web page), and/or any text data developed by the speech recognizer 28 from voice or other audio data (e.g., when the subject customer feedback instance is associated with a telephone call), is communicated to the data formatter 32. The data formatter 32 writes the received data into appropriate field(s) of a record and the record is stored in the database 16. The record may be operated on at a later time by the database processor 18 as explained below.
  • FIG. 3 is an exemplary representation of a data structure 40 that may reside in the database 16 and may be filled with records 42 completed by the data formatter 32 of the data acquisition processor 14. The illustrated data structure 40 has a plurality of rows and columns. Each row is associated with a single record 42. Each column is associated with a predefined field. More specifically, the data structure 40 contains a product code field 44 for receiving a code associated with a product discussed in a customer feedback instance, date and time fields 46, 48 for storing data indicating when the customer feedback instance was recorded in the database 16, a reason code field 50 for receiving a code associated with the reason the customer initiated the instance, a comment field 52 for storing any comments that the service representative chooses to enter, and a textual information or customer comment field 54 for storing the text data generated by the speech recognizer 28 or received from the network (e.g., in an email message or web page form), if any.
  • As shown in FIG. 4, the database processor 18 preferably includes a search engine and queue generator 60, a relevance finder 62, a relevance sorter 64, and a merger 66. Whenever a user initiates a search of the database 16, search criteria are communicated from the GUI of the data analyst terminal 20 to the search engine and queue generator 60. The search engine 60 responds by searching the database 16 for all records meeting the search criteria. For example, search criteria pertaining to product returns for the big button telephone may correspond to R02 and TDG1A for reason code and product code, respectively. If the search engine and queue generator 60 receives such search criteria, it responds by employing conventional word search techniques to search the product code field 44 and the reason code field 50 of every record 42 in the database 16. The records 42 meeting the subject search criteria are compiled and displayed in a search result 70 such as that shown in FIG. 5. The search result 70 is stored in a memory such as the memory associated with the data analyst terminal 20 or the storage device containing the database 16.
  • Preferably, the search result 70 is composed of a number of rows and columns. Each row represents a record 42 corresponding to a customer feedback instance meeting the search criteria. The columns comprise certain 10 fields from the identified records. Preferably, the displayed fields include at least the fields identified in the search criteria and the comment field 52. In the illustrated example shown in FIG. 5, the product code (USOC) field 44, the reason code field 50 and the comment field 52 of each record 42 meeting the search criteria are displayed. Preferably, the search result 70 also displays the number 74 of records identified in the search (e.g., 919 records). Of course, persons of ordinary skill in the art will appreciate that other fields such as the textual information or customer comment field 54 can be displayed in the search result 70 without departing from the scope or the spirit of the invention.
  • Returning to FIG. 4, the search result 70 as well as the contents of the entire database 16 are available to the relevance finder 62. As will be discussed in detail with reference to FIG. 10, the relevance finder 62 processes each word in one or more predetermined fields (e.g., the comment field 52) of each record 42 in the search result 70 to generate a search queue 76 (see FIG. 6) that includes a concept list (e.g., a list of every word appearing in the predetermined field(s) of the search result 70). The relevance finder 62 compares each concept 77 in the concept list of the search queue 76 to each word in the predetermined Field(s) of every record 42 stored in the entire database 16 to develop a set count for each concept 77 (i.e., the total number of records 42 in the database including a given concept 77 is the set count for that concept). It also compares each concept 77 in the search queue 76 to each word in the predetermined field(s) of the records 42 in the search result 70 to develop a subset count for each concept 77 (i.e., the total number of records 42 in the search result 70 including a given concept 77 is the subset count for that concept). For each concept 77, the relevance finder 62 then employs the set count to calculate a global frequency and the subset count to calculate a local frequency. The global and local frequencies of each concept are then used by the relevance finder 62 to determine a relevance metric (or relevance score) 78 for each concept 77. The relevance finder 62 stores the relevance scores 78 in association with their corresponding concepts 77 in the search queue 76. After relevance scores 78 have been calculated for every concept in the search queue 76, the relevance sorter 64 (FIG. 4) sorts all of the concepts 77 in the search queue 76 according to the magnitudes of their relevance metrics 78. A representative sorted search queue 76 output by the relevance sorter 64 is shown in FIG. 6.
  • As shown in FIG. 6, each row in the search queue 76 represents a particular concept 77. For instance, in the illustrated example, the first three concepts 77 in the queue 76 are BIG, ENOUGH, and LOUD, all of which are shown in the concept column 98. The search queue 76 also includes 5 relevance 100 and count 102 columns. The relevance column 100 contains the relevance score 78 of each concept 77 listed in the concept column 98. The count column 102 contains a value indicative of the number of records in the search result 70 that include the corresponding concept 77 (i.e., the local record count). For example, the fast mw in the search queue 76 indicates that the concept BIG appears in forty-one of the records 42 in the search result 70 and that the relevance score 78 of BIG is 0.0832. Because the relevance score 78 is based on a comparison of the frequency with which a particular concept appears in the general population of the database 16 with the frequency with which that concept appears in a subset of that population (i.e., the search result 70), a concept 77 can have a large value in the count column 102 and a low value in the relevance score column 100. For example, although in the example shown in FIG. 6 the concept ENOUGH has a larger count than the concept BIG, it has a lower relevance score 78 than BIG because, while ENOUGH is found in more records 42 in the search result 70 than BIG, the concept ENOUGH also appears in more records in the database 16 than does the concept BIG. The more frequently a concept 77 appears in the records 42 in the search result 70 and the less frequently that same concept 77 appears in the general population of the database 16, the more relevant that concept 77 is to the specified search criteria. In other words, the relevance score 78 of a concept 77 is directly proportional to the number of records 42 in the search result 70 containing that concept 77, and inversely proportional to the number of records 42 in the general population of the database 16 containing that same concept 77. The sorted search queue 76 (FIG. 6) developed by the relevance sorter 64 is communicated to the data analyst terminal 20 for display to the data analyst.
  • While the sorted search queue 76 shown in FIG. 6 includes only concepts 77 containing a single word, combinations of concepts 77 (which may also be called compound concepts or generically “concepts”) may in fact yield relevance metrics 78 that are higher than the relevance metrics 78 of the singular concepts. Accordingly, for the purpose of creating compound concepts (i.e., a string of two or more concepts 77), the database processor 18 is further provided with the merger 66 (see FIG. 4). The merger 66 15 preferably combines the concepts 77 in the search queue 76 in pairs in every possible, non-redundant way (but without combining more than two concepts (compound or singular) from the existing queue in a single new concept) to create compound concepts. The merger 66 passes the compound concepts to the relevance finder 62, which determines the relevance scores 78 of the compound concepts and communicates the calculated relevance metrics 78 back to the merger 66. If, after the relevance scores 78 of each pair of concepts 77 in the search queue 76 are computed, none of the compound concepts created by the merger 66 in the current merger cycle have a higher relevance metric 78 than the concepts from which they are created (e.g., the relevance score of compound concept BIG, TELEPHONE is not greater than the relevance score of the concept BIG and also greater than the relevance score of the concept TELEPHONE), the merger 66 ceases its operation without changing the search queue 76. If, however, at least one newly created compound concept has a higher relevance score 78 than the concepts 77 from which it was created, the merger 66 cooperates with the relevance sorter 64 to update the search queue 76 to include the newly-created compound concept with the highest relevance score 78 and which meets the above-noted criterion (e.g., has a higher relevance score than any of the singular concepts from which it was created) (see FIG. 7). As used herein, a merger cycle is defined as a cycle wherein a relevance score is computed for every possible combination of two concepts in a search queue as of a fixed time.
  • The merger 66 continues to merge concepts together until the relevance score 78 of every possible pair of two existing concepts in the search queue 76 (thereby excluding new concepts created in the current merger cycle) is determined. In the example of FIG. 7, the merger 66 has completed one merger cycle on the search queue of FIG. 6 (i.e., every concept in the queue 76 has been paired with every other concept in the queue 76 one time, a relevance score 78 has been computed for each pair, and the new concept with the highest relevance score which is also above the highest relevance score 78 previously contained in the queue 76 has been added to the queue), and the merger 66 has found that the new concept with the highest relevance score, namely, “LOUD, ENOUGH”, has a relevance score 78 which is higher than any of the previously calculated relevance scores 78 shown in FIG. 6. Accordingly, the concept “LOUD, ENOUGH” has been added to the search queue 76.
  • The results of the first merger cycle (FIG. 7) are displayed at the data analyst terminal 20. If the data analyst wishes to initiate another merger cycle, he/she indicates so by interacting with the GUI. The merger 66 responds by pairing each concept 77 in the new search queue 76 with every other concept 77 in that search queue 76, by cooperating with the relevance finder 64 to compute a relevance score 78 for each pair, and by cooperating with the relevance sorter 64 to add the new concept with the highest relevance score 78 to the queue if that score 78 is greater than the highest relevance score 78 in the queue 76 at the end of the last merger cycle. In the example shown in FIGS. 6-8, the performance of a second merger cycle resulted in the addition of the new concept “RING, LOUD, ENOUGH” to the search queue 76 (see FIG. 8). That new concept has a higher relevance score (i.e., 5664.7368) than the concepts from which it was created (i.e., “RING”, “LOUD” and “ENOUGH”) (FIG. 7).
  • The results of the second merger cycle (FIG. 8) are displayed at the data analyst terminal 80. If the data analyst requests performance of another merger cycle, the merger 66, the relevance finder 62 and the relevance sorter 64 will function precisely as they did in connection with the second merger cycle described above. The results of a subsequent merger cycle in the illustrated example are shown in FIG. 9. As depicted in that figure, the third merger cycle resulted in the addition of the new concept “DOES, NOT, RING, LOUD, ENOUGH”. (Four intervening merger cycles had added four other concepts to the queue 76 shown in FIG. 8, namely, “DOESNT, RING, LOUD, ENOUGH”, “STATES, RINGER, LOUD, ENOUGH”, “RINGER, NOT, LOUD, ENOUGH”, and “LOUD, OF, NEARING, ENOUGH”.) The new search queue 76 (FIG. 9) is displayed at the data analyst terminal 20. The data analyst can initiate as many cycles as he/she would like. However, eventually performing additional merger cycles will fail to identify any concept with a relevance score which is higher than the relevance scores of the concepts from which it was created, and additional merger cycles will, therefore, not change the search queue 76.
  • Persons of ordinary skill in the art will appreciate that, although the above described database processor 18 stops between merger cycles to provide the data analyst with enhanced control over the data analysis process, the merger 66 can be adapted to automatically continue to conduct merger cycles until an interrupt is received from the data analyst terminal or until the search queue 76 is not changed by a merger cycle (whichever occurs first), without departing from the scope or spirit of the invention.
  • Turning to a more detailed discussion of the relevance finder 62, for the purpose of determining the number of records in which a concept 77 appears, the relevance finder 62 is provided with a counter 110 (see FIG. 10). When the search engine and queue generator 60 complete the search result 70, it initiates preparation of the search queue 76 by creating a list of every word/concept found in a predetermined field(s) of every record 42 in the search queue 76. During this process, the counter 110 counts the number of records 42 within which each such word/concept appears. The predetermined field used to create the search queue 76 is preferably the comments field 52, but other fields such as the textual information (customer comments) field 54 could be used in this role either in place of, or in addition to, the comments field 52 without departing from the scope or spirit of the invention. The counter 100 also polls the database 16 to determine how many records 42 within the entire database 16 contain the subject concept/word in the predetermined field(s). Preferably, in addition to the count number, the counter 110 creates a subset list for the search queue identifying by record number (or some other unique identifier) the records 42 containing the subject concept in the subset. The subset list for each concept is stored in association with the corresponding concept. The subset count and set count (i.e., the number of records in the subset (search result 70) and the number of records in the database 16 as a whole containing the concept) are then passed to a frequency generator 112.
  • The frequency generator 112 calculates a global frequency for each concept 77 by dividing the set count (which is representative of the number of records 42 in the database 16 in which the concept 77 appears) by the total number of records 42 in the database 16. The frequency generator 112 also calculates a local frequency for each concept 77 by dividing the subset count (which is representative of the number of records 42 in the search result 70 in which the concept appears) by the total number of records 42 in the search result 70. Both the global frequency and the local frequency are provided to the relevance calculator 114, which calculates the relevance score 78 of each concept 77 by dividing the local frequency value of the corresponding concept 77 by the global frequency value for that same concept 77. The relevance scores 78 of the concepts 77 are communicated to the relevance sorter 64 shown in FIG. 4 which uses the scores 78 to sort the concepts 77 within the search queue 76.
  • A more detailed illustration of the merger 66 is shown in FIG. 11. For the purpose of selecting pairs of concepts 77 in the search queue 76 to create new concepts, the merger 66 is provided with a selector 120. The selector 120 sequentially pairs every concept 77 in the search queue 76 as it existed prior to the current merger cycle with every other concept 77 in that search queue 76 to create new concepts. Each new concept is passed to an intersection generator 122 which determines the number (count) of records 42 within the subset (i.e., the search result 70) that contain the new concept within the predetermined field(s), and the number (count) of records 42 in the database 16 that contain the new concept within the predetermined field(s).
  • To shorten the processing time required to develop these counts, rather than re-examining all of the records 42 in the search result 70, the intersection generator 122 compares the subset lists created by the counter 110 for the concepts 77 being combined to identify the records 42 containing both concepts. In other words, if, for example, the concepts LOUD and ENOUGH are being combined by the selector 120, the intersection generator 122 retrieves the subset list identifying records within the search result 70 containing the concept LOUD, and the subset list identifying records within the search result 70 containing the concept ENOUGH. It then identifies which (if any) records appears in both subset lists. The number of records 42 appearing in both subset lists define the local record count of the number 10 of records in the search result 70 containing the concept LOUD, ENOUGH. The intersection generator 122 stores the identified count in association with the proposed new concept (LOUD, ENOUGH). It also creates a new subset list identifying (e.g., by record number) the records 42 containing the new concept and stores that new subset list in association with the new concept (LOUD, ENOUGH).
  • Because the number of records in the database 16 is generally very large, it is impractical to maintain a “set list” (i.e., a list analogous to the subset list but identifying the records in the database 16 as a whole containing a given concept) for each concept. Specifically, both memory and processing time limitations generally preclude the use of such set lists to determine a global record count (i.e., the number of records in the entire database 16 including a given concept) for a concept. The memory limitations are implicated by the length such set lists would often attain. The processing time limitations are implicated by the amount of time it would take to compare such extensively long lists.
  • To address this problem, and to avoid repeatedly reading every record 42 in the database 16 to develop global record counts, the CFAPS 10 periodically performs an off-line examination of the database 16. Such an examination develops a global index of every single word (i.e., every one word concept 77) appearing in the database 16 and a global record count (i.e., a number indicating the number of records 16 in the database 16 as a whole containing a given concept) for each concept 77. This index is stored in a non-volatile storage medium such as a hard drive where it can be accessed as the need arises.
  • The global index is utilized to develop approximations of the global frequency (i.e., the global record count divided by the total number of records 42 in the database 16) of each compound concept. In particular, if it is desired to develop a global frequency for the compound concept AB, where concept A and concept B are one word concepts, the relevance finder 62 retrieves the global record counts for concept A and concept B from the global index, and divides those global record counts by the number of records in the database 16 to determine the global frequency for each of concept A and concept B. The frequency generator 112 of the relevance finder 62 then multiplies the global frequencies of concept A and concept B to create an estimated global frequency for the compound concept AB. Were the concepts A and B statistically independent, the estimation would be highly accurate. Since, however, in compound concepts of high relevancy, the underlying concepts will not be statistically independent, multiplying their frequencies will result in a relatively poor estimation of the global frequency of the compound concept. Specifically, in such circumstances, the estimated value will always be lower than the true global frequency. This means that the relevance score of a compound concept including statistically dependent concepts will be higher than the score 78 it would have if the true global frequency of the compound concept was calculated. The enhancement in the relevance score 78 introduced by the noted approximation is reflected in the large relevance scores of the compound concepts appearing in FIGS. 7-9.
  • It will be appreciated that it will eventually become necessary to compute a relevance score 78 for a new compound concept including one or more compound concepts. In such circumstances, the global index will not have a count for the compound concepts used to build the new current compound concept. Therefore, to calculate the relevance score of the new compound concept, the relevance finder 62 employs the estimated value(s) of the compound concept(s) used to build the new compound concept. For example, if new compound concept ABCD is made from compound concept AB and compound concept CD, the estimated global frequency of the compound concept AB is multiplied with the estimated global frequency of the compound concept CD to determine an estimated global frequency for the compound concept ABCD.
  • The subset count (local record count) determined by the intersection generator 122 is communicated to the frequency generator 112 which then calculates a local frequency as explained above (see FIG. 10). (The local frequency equals the subset count divided by the total number of records 42 in the search result 70.) The relevance calculator 114 then develops a relevance score 78 for the new concept from the global frequency and the local frequency as described above. The relevance score 78 is returned to the improvement detector 124 of the merger 66 (see FIG. 11).
  • The improvement detector 124 compares the relevance score 78 received from the relevance calculator 114 for the new concept being tested to the highest relevance score 78 obtained by a new concept in the current merger cycle thus far. If it does not have the highest relevance score 78 of any concept created in the current merger cycle, it is discarded and the next new concept 77 is tested. This process continues until the improvement detector 124 identifies the concept with the highest relevance score 78 created during the current merger cycle. The improvement detector 124 then compares the relevance score 78 of that compound concept (i.e., the new concept developed in the current merger cycle) to the relevance score 78 of each concept forming the new compound concept. If the relevance score 78 of the new concept developed in the current merger cycle is lower than any one of the relevance scores of the concepts forming the new compound concept, no additional concept is added to the queue 76. Otherwise, the new concept is added to the search queue 76 in the appropriate position indicated by its relevance score 78 (i.e., at the highest position).
  • Persons of ordinary skill in the art will appreciate that, while the intersection generator 122 is preferably included to reduce processing time, if desired the intersection generator 122 can be eliminated and the output of the selector 120 delivered to the counter 110 of the frequency finder 62 (instead of the frequency generator 112) to initiate a full examination of all of the records in the search result 70 without departing from the scope or spirit of the invention. Similarly, persons of ordinary skill in the art will appreciate that the approximation technique for calculating the global frequency of compound concepts described above can be replaced with a brute force counting technique or with a “set list” technique analogous to the subset list technique described above without departing from the scope or spirit of the invention.
  • Persons of ordinary skill in the art will also appreciate that, although in the preferred embodiment, the search queue 76 is developed from the comment fields 52 of the records in the search result 70, the search queue 76 could alternatively (or additionally) be developed from the textual information fields 54 of the records without departing from the scope or spirit of the invention. Use of the comment fields 52 for queue generation is preferred, however, because they are believed to typically contain much of the same information as the textual information fields 54, but in condensed fashion. As a result, using the comment fields 52 should reduce processing time.
  • A more detailed explanation of the structure and operation of the software implementing the preferred embodiment of the CFAPS 10 will now be provided in connection with the flow charts appearing in FIGS. 12-17. Persons or ordinary skill in the art will appreciate that, although for ease of discussion, the structure and operation of the software will be described in the context of a series of steps occurring in a particular order, the steps or variations thereof can be performed in other temporal sequences without departing from the scope or spirit of the invention.
  • Turning first to the steps executed by the data acquisition processor 14 (FIG. 12A), the data acquisition processor 14 initially awaits receipt of a customer feedback input message from a service representative terminal 12 (block 200). Upon receipt of an input message, the signal identifier 25 (FIG. 2) examines the input message to determine if it includes an audio signal (block 202). If so, the audio signal is muted to the speech recorder 26. The speech recognizes 28 then translates the audio signal to text and delivers the text to the data formatter 32 (block 204). Since an input message with an audio component (e.g., a recording of a telephone call) will always be associated with a new customer feedback instance, the data formatter 32 creates a new record 42 in the database 16, (block 206), fills in the date and time fields 46, 48 of the new record 42 with the current date and time (block 208) and then writes the text received from the speech recognizer 28 in the textual information field 54 of the new record (block 210). If the input message including the audio signal also included non-audio components (block 212), the non-audio portion(s) are stamped with an identifier referencing the newly created record 42 (block 214), and then control proceeds to block 230 (FIG. 12C). Otherwise, control returns to block 200 until another input signal is received.
  • If the input message did not include an audio signal (block 202), control proceeds to block 216 (FIG. 12B). If the data formatter 32 identifies the received input message as an electronic submission such as a web page form or an email message (block 216), it attempts to identify any predefined codes within the message that would divide the message into segments (e.g., a product code segment, a reason code segment, a service representative comment segment, and/or a customer feedback text segment (block 218). Since an electronic submission is always associated with a new customer feedback instance, the data formatter 32 creates a new record 42 in the database (block 220). It then records the current date and time in the date and time fields 46, 48 of the new record 42 (block 222). The data formatter 32 then writes any identified segment(s) of the input message to the corresponding field(s) in the new record (block 224). Any unidentifiable 20 text in the input message (e.g., text without a code such as might be present in an email message) is written to the textual information field 54 of the new record (block 224). Control then returns to block 200 (FIG. 12A).
  • To handle customer feedback instances that are not accompanied by an audio recording and are not originated by electronic submission, the service representatives are provided with the ability to request creation of a new record. When such a request is received (block 225), the data formatter 32 creates a new record 42 in the database 16, records the current date and time in the date and time fields 46, 48 of the new record 42 (block 227), and stamps the input message with an identifier associating the message with the newly crated record 42 (block 228). Control then proceeds to block 230 (FIG. 12C).
  • If the received input message does not include an audio component (block 202), does not request creation of a new record 42 (block 225), is not an electronic submission (block 216), or does include an audio component in addition to other components (block 212), control proceeds to block 230. If the input message includes a code identifying the message (or a portion 15 thereof) as a service representative comment, control proceeds to block 232. If the input message does not identify the comment as being associated with an existing record 42, an error message is sent to the service representative terminal 12 originating the input message (block 234). Control then returns to block 200 for processing of the next received message.
  • If, on the other hand, the input message associated with the comment does identify an existing record 42 (block 232), the data formatter 32 writes the comment to the comment field 52 of the identified record 42 (block 236). If the input message includes additional components (block 238), control proceeds to block 244. Otherwise control returns to block 200 for processing the next input message (which, of course, may optionally be waiting in a message queue).
  • If, at block 230, the data formatter 32 determines that the input message does not include a comment portion, or if, after processing an input message including a comment portion, additional components of the input message remain for processing (block 238), control proceeds to block 244. At block 244, the data formatter 32 determines whether the input message includes a code identifying the message (or a portion thereof) as a reason code. If so, control proceeds to block 246. If the input message does not identify an existing record 42, the data formatter 32 returns an error message to the service representative terminal 12 originating the message (block 248). Control then returns to block 200.
  • If, on the other hand, the input message does identify an existing 15 record (block 246), the data formatter 32 writes the reason code associated with the input message into the reason code field 50 of the identified record 42 (block 250). If the input message includes additional components (block 252), control proceeds to block 260 (FIG. 12D). Otherwise, control returns to block 200.
  • If, at block 244, the date formatter 32 determines that the input message does not include a reason code, or if after processing an input message including a reason code, additional components of the input message remain for processing (block 252), control proceeds to block 260. If, at block 260 (FIG. 12D), the data formatter 32 determines that the input message includes a code identifying the message (or a portion thereof) as a product code, control proceeds to block 262. If the input message does not identify an existing record 42 (block 262), the data formatter 32 returns an error message to the service representative terminal 12 originating the message (block 264). Control then returns to block 200.
  • If, on the other hand, the input message does identify an existing record (block 262), the data formatter 32 writes the product code associated with the input message into the product code field 44 of the identified record 42 (block 266). If the input message includes remaining, unprocessed components, those components are unidentifiable and the data formatter 32 sends an error message to the originating service representative terminal 12 (block 270). Otherwise, control returns to block 200.
  • Similarly, if at block 260, the data formatter 32 determines that the input message does not include a product code, the input message is unidentifiable and the data formatter 32 returns an error message to the service representative terminal 12 originating the message (block 270). Control then returns to block 200.
  • The operation of the database processor 18 will now be explained in connection with FIGS. 13-18.
  • The create database index routine is shown in FIGS. 13A-13B. Although the index is not shown in the figures, the create database index routine is executed to create an index of every one word concept in the predefined field(s) of the records 42 in the entire database 16. This index identifies each such one word concept and its corresponding global record count. As explained above, the information contained in the index is used in estimating the global frequency of compound concepts. The presence of the index (which is typically stored on a hard drive) expedites the operation of the rest of the program by eliminating the need to repeatedly poll the database 16 to determine global record counts. If the database 16 is updated from time to time, it is also necessary to periodically update the index by, for example, re-executing the create database index routine to create a new index.
  • When the create database index routine is initiated, the search engine and queue generator 60 creates a new index in a non-volatile memory such as a hard drive associated with the data analyst terminal 20 or the storage device containing the database 16 (block 273). The search engine and queue generator 60 then determines which field(s) of the records 42 in the database 16 are to be used to create the index (block 274). As mentioned above, the field(s) employed in this role is typically the comment field 52 alone. However, other fields including, by way of example, not limitation, the textual information field 54, could be used in place of, or in addition to the comment field 52 without departing from the scope or spirit of the invention.
  • As shown in FIG. 13A, after the field to be used in generating the index is identified (block 274), the search engine and queue generator 60 sets a record counter X to zero and a word counter Y to one (block 275). The record counter X is then incremented by one (block 276) and the field identified at block 274 is retrieved from the fast record in the database 16 (block 278). A first word in the retrieved field is then selected (block 279) and examined to determine if the selected word already appears in the index (block 280). If the selected word is not in the index (block 280), the search engine and queue generator 60 writes that word to the index (block 281) and sets a word count variable associated with the newly added word (concept) to one (block 282). Although, due to the memory and processing constraints mentioned above it is not done in the currently preferred embodiment, a set list could optionally be created (block 283). Control then proceeds to block 284. If the word selected at block 279 is already in the index (block 280), control passes directly to block 284 from block 280.
  • At block 284, the search engine and queue generator 60 determines whether the current word has already been seen in the current record 42. If this is the fast instance of the word (concept) 77 appearing in the current record, then the global record count for the current word is incremented by one (block 285) (FIG. 13B). If a set list is being created, an identifier identifying the current record is written to the set list (block 286). If this is not the first time the word 77 has been seen in the current record, control proceeds to block 287. Block 284 is provided to ensure that the global record count for the concepts 77 written to the index reflect the number of records 42 in which that concept 77 appears, not the total number of times that the concept 77 appears in the database 16 (e.g., if the word “BIG” appears twice in one record, the global record count for the word “BIG” will only be incremented by one).
  • At block 287 (FIG. 13B), the search engine and queue generator 60 determines whether the current word is the last word in the field of the current record. If not, the word counter Y is incremented by one (block 288) and control returns to block 279 (FIG. 13A). Control will continue to loop between blocks 279-288 until each word in the field of the current record is examined. After each word in the current record has been examined (block 287), the search engine and queue generator 60 determines whether there are more records 42 in the database 16 to examine (block 289) (FIG. 13B). If so, control returns to block 276 (FIG. 13A). Control will continue to loop through blocks 276-289 until every word and every record 42 in the index has been examined. When that examination is complete (block 289), the index will be complete. Every single word concept in the database can be looked up in the completed index to identify a global record count for the looked-up word.
  • The main routine executed by the database processor 18 is shown in FIGS. 14A-14B. As shown in FIG. 14A, at startup, the database processor 18 performs various conventional housekeeping tasks such as initializing variables (block 298). After the housekeeping tasks are completed, control proceeds to blocks 302.
  • At block 302, the database processor 18 enters a loop wherein it awaits entry of search criteria from a data analyst terminal 20. When search criteria are received, the search engine and queue generator 60 (FIG. 4) creates a data structure in an associated memory (such as the memory of the data analyst terminal 20 or the memory device containing the database 16) to store a new search result 70 (block 304). It then clears a record counter R (block 306), and then enters a loop defined by blocks 310-318 to populate the newly defined search result 70.
  • In particular, at block 310, the search engine and queue generator 60 increments the record count by one. It then retrieves the record (or a portion thereof) associated with the record count from the database 16 and reads a 10 predefined field from that record 42 (block 312). The predefined field is identified in the search criteria entered by the user, and may comprise the product code field 44, the date field 46, the time field 48, the reason code field 50, the comments code field 52, and/or the textual information field 54. In the example shown in FIG. 5, the predetermined fields are the product code field 44 and the reason code field 50.
  • After the predetermined field is read (block 312), the search engine and queue generator 60 determines whether the search criteria specified by the data analyst are met in that field (block 314). This determination is preferably performed by comparing a term entered by the data analyst 20 against the term(s) contained in the predetermined field(s). If a match is found, the criteria are met and control proceeds to block 316. Otherwise, the search criteria are not met and control proceeds to block 318. If the search criteria are met (block 314), the search engine and queue generator 60 adds the record 42 being examined to the newly defined search result 70 (block 316). The search engine and queue generator 60 then determines whether the record 42 currently being examined is the last record 42 in the database 16 (block 318). If not, control returns to block 310 where the record count is incremented and the next record 42 is retrieved for comparison against the search criteria. Control will continue to loop through blocks 310-318 until every record 42 in the database 16 is examined for compliance with the search criteria. After every record 42 is so examined, the search result 70 (see FIG. 5) is displayed at the data analyst terminal 20 (block 324). The search engine and queue generator 60 then calls the initialize search queue routine shown in FIGS. 15A-15B (block 326).
  • When the initialize search queue routine is initiated, the search engine and queue generator 60 creates a new search queue 76 in a memory such as the memory associated with the data analyst terming 20 or the storage device containing the database 16 (block 330). The search engine and queue generator 60 then determines which field(s) of the records 42 in the search result 70 are to be used to create the search queue 76 (block 332). As mentioned above, the field employed in this role is typically the comment field 52 alone. However, any other field(s) including, by way of example, not limitation, the textual information field 54, could be used in place of, or in addition to, the comment field 52 without departing from the scope or spirit of the invention.
  • After the field to be used in generating the search queue 76 is identified (block 332), the search engine and queue generator 60 sets a record counter X to zero and a word counter Y to one (block 334). The record counter X is then incremented by one (block 336) and the field identified at block 332 is retrieved from the first record in the search result (block 338). A first word in the retrieved field is then selected (block 340) and examined to determine if the selected word already appears in the search queue 76 (block 342). If the selected word is not in the search queue 76 (block 342), the search engine and queue generator 60 writes that word to the search queue 76 and sets a word count variable (i.e., the local record count) associated with the newly added word (concept) to zero (block 346). It then creates a subset list for the new concept (block 347). Control then proceeds to block 348. If the word selected at block 340 is already in the search queue 76 (block 342), control passes directly to block 348.
  • At block 348, the search engine and queue generator 60 determines whether the current word has already been seen in the current record 42. If this is the first instance of the word (concept) 77 appearing in the current record, then the local record count for the current word is incremented by one (block 350) and the identifier of the record being examined is added to the subset list (block 351). If this is not the first time the word has appeared in the current record 42, control proceeds to block 352 (FIG. 15B). Block 348 is provided to ensure that the local record count for the concepts 77 written to the search queue 76 reflect the number of records in which that concept 77 appears, not the total number of times that concept 77 appears in the search result 70 (e.g., if the word “BIG” appears twice in one record, the local record count for the word “BIG” will only be incremented by one).
  • At block 352 (FIG. 15B), the search engine and queue generator 60 determines whether the current word is the last word in the field of the current record. If not, the word counter Y is incremented by one (block 354) and control returns to block 340 (FIG. 15A). Control will continue to loop between blocks 340-354 until each word in the field of the current record is examined. After each word has been examined (block 352), the search engine and queue generator 60 determines whether there are more records 42 in the search result 70 to examine (block 356) (FIG. 15B). If so, control returns to block 336 (FIG. 15A). Control will continue to loop through blocks 336-356 until every word in every record 42 in the search result 70 has been examined. When that examination is complete (block 356), the concept list 98 and local record count column 102 of the search queue 76 will be complete and control will return to block 370 of the main routine (FIG. 14B).
  • Once the search queue 76 has been populated with the concept list 98 and the local record counts list 102, the search engine and queue generator 60 calls the compute relevance scores routine (block 370). As shown in FIG. 16, the counter 110 of the relevance finder 62 initiates the compute relevance scores routine by setting loop counter C to zero (block 372). The loop counter C is then incremented by one (block 374). The first concept in the search queue 76 is then retrieved (block 376). The counter 110 then retrieves the global record count for the first concept in the search queue 76 from the index (block 378). The frequency generator 112 then calculates and stores the global frequency for the current concept by dividing the global record count (i.e. the number of records in the database 16 that include the current concept) by the total number of records 42 in the database 16 (block 380).
  • The counter 110 of the relevance finder 62 then retrieves the local record count for the first concept in the search queue 76 (block 394). The frequency generator 112 then calculates the local frequency for the current concept by dividing the local record count by the number of records in the search result (block 396) (FIG. 16). The relevance calculator 114 then retrieves the global frequency and the local frequency for the concept identified at block 444 (blocks 446-448). The relevance calculator 114 then calculates a relevance score 78 for the current concept by dividing its global frequency by its local frequency (block 450). The relevance calculator 114 then stores the calculated relevance score 78 in the search queue 76 in association with the current concept (block 452). The relevance calculator 114 next determines whether the relevance score 78 for the last concept in the search queue 76 has been written to the search queue 76 (block 454). If not, control returns to block 374. Control will continue to loop through blocks 374-454 until the relevance score 78 of every concept 77 in the search queue 76 has been calculated and written to the search queue 76. After the relevance score 78 of the last concept 77 in this search queue 76 has been calculated (block 454), control returns to block 470 of the main routine shown in FIG. 14B.
  • At block 470, the relevance sorter 64 sorts the concepts 77 in the search queue 76 based on their relevance scores 78 using a conventional sorting algorithm such as a bubble sort routine. The database processor 18 then displays the sorted search queue 76 (see FIG. 6) at the data analyst terminal 20 (block 472).
  • The database processor 18 then enters a loop where it awaits a request to perform a merger cycle (block 474) or a request to initiate a new search (block 476) from the data analyst. If the data analyst requests that a new search be performed (block 476), control returns to block 302 of FIG. 14A where the data analyst is given the opportunity to enter new search criteria. If, on the other hand, the data analyst requests the initiation of a merger cycle (block 474), control proceeds to block 478 (FIG. 14B) where the database processor 18 calls the merger routine.
  • As shown in FIG. 17A, the selector 120 of the merger 66 initiates the merger routine by setting a loop counter C1 to zero (block 500), by setting a high relevance score variable to zero (block 501), by incrementing the loop counter C1 by one (block 502), by setting a sub-loop counter C2 to the loop counter value C1 plus one (block 504), and by retrieving the first concept from the search queue 76 (block 506).
  • At block 512, the selector 120 retrieves the concept specified by the sub-loop counter C2 from the search queue 76. For example, if this is the first time through the search queue 76 shown in FIG. 6, the loop counter C1 equals one, the sub-loop counter C2 equals two, and therefore, concept C1 equals “BIG” and concept C2 equals “ENOUGH”. The selector 120 then creates a new concept by combining the concept specified by the loop counter C1 with the concept specified by the sub-loop counter C2 (block 514). The frequency generator 112 of the relevance finder 62 then calls the calculate global frequency routine (FIG. 18) to develop a new global frequency for the new compound concept (block 520).
  • As shown in FIG. 18, the frequency generator 112 initiates the calculate global frequency routine by setting a compound frequency variable and a loop counter Y to one (blocks 600 and 602). It then counts the number of single words contained in the new compound concept (block 604). A loop limit variable Z is then set to the number of words counted in the compound concept (block 610).
  • At block 612, the frequency generator 112 retrieves the global record count for the fast word in the new compound concept from the index. It then calculates the global frequency for the first word in the new compound concept (block 614) and multiplies the calculated global frequency with the value in the compound frequency variable to develop a new value of the compound frequency variable (block 616).
  • The frequency generator 112 next determines whether the value in the loop counter Y equals the value in the loop limit variable Z (block 618). If so, the global frequency for the new compound variable has been computed and control returns to block 524 of FIG. 17B. Otherwise, the loop counter Y is incremented by one (block 620) and control returns to block 512 (FIG. 18). Control will continue looping through blocks 612-6 until the individual global frequencies of the single words in the new compound concept have been multiplied together (e.g., if three single words A, B & C make up the new compound concept then the loop stops executing when the global frequency of word A has been multiplied with the global frequency of word B and the product of the global frequencies of words A and B has been multiplied with the global frequency of word Q. Once this has been achieved, the global frequency of the new compound concept has been estimated, and control returns to block 524 of FIG. 17B.
  • At block 524 (FIG. 17B), the selector 120 retrieves the subset lists for the concepts specified by the loop counter C1 and the sub-loop counter C2. The intersection generator 122 then creates a new subset list for the new concept by identifying the intersection of the subset list for the concepts specified by the loop and sub-loop counters C1, C2 (block 530). The frequency generator 112 then counts the number of records 42 identified in the new subset list (block 532) and calculates the local frequency for the new concept (block 534); The relevance calculator 114 then calculates a relevance score 78 for the new concept from the global and local frequencies as explained above (block 536).
  • At block 540, the improvement detector 124 of the merger 66 compares the relevance score 78 of the new concept to the value in the highest relevance score variable. If the relevance score 78 of the new concept is greater than the value in the highest relevance score variable, the improvement detector 124 stores the new concept, its global frequency and its subset list in memory (block 542). The improvement detector 124 also replaces the value in the highest relevance score variable with the relevance score 78 of the new concept and stores the relevance scores of each of the concepts (singular or compound) comprising the new compound concept in memory (block 544).
  • If the relevance score 78 of the new concept does not exceed the value in the highest relevance score variable (block 540), the new concept together with its global frequency, its relevance and its subset list is discarded. At block 546, the selector 120 of the merger 66 determines whether the concept specified by the sub-loop counter C2 is the last concept in the search queue 76. If not, the selector 120 increments the sub-loop counter C2 by one (block 548) and control returns to block 512 (FIG. 17A). Control will continue looping through blocks 512-548 until every concept in the search queue 76 has been paired with the concept specified by the loop counter C1 (block 546). When this occurs, control proceeds from block 546 to block 550.
  • At block 550 (FIG. 17B), the selector 120 of the merger 66 determines whether the loop counter C1 is equal to the sub-loop counter C2 minus one. If so, every concept that was in the search queue 76 prior to the initiation of the current merger cycle has been paired with every other concept that was in the search queue 76 prior to the current merger cycle, and control proceeds to block 551 (FIG. 17C). At block 551, the improvement detector 124 determines whether the value in the highest relevance score variable exceeds the individual relevance scores of each of the concepts (singular or compound) comprising the new compound concept. For example, if the compound concept identified as having the highest relevance score in the current merger cycle (i.e., identified via blocks 500-550) is BIG, BUTTON, TELEPHONE, the improvement detector compares: (a) the relevance score of the concept BIG, BUTTON, TELEPHONE to the relevance score of the concept BIG, (b) compares the relevance score of BIG, BUTTON, TELEPHONE to the relevance score of the concept BUTTON, and (c) compares the relevance score of BIG, BUTTON, TELEPHONE to the relevance score of the concept TELEPHONE. If these comparisons indicate that the relevance score of BIG, BUTTON, TELEPHONE is greater than the relevance score of the concept BIG, is greater than the relevance score of the concept BUTTON, and is greater than the relevance score of the concept TELEPHONE, control proceeds to block 552 where the new concept (e.g., BIG, BUTTON, TELEPHONE) is added to the search queue 76. Otherwise, no new concept is added to the search queue 76 during the current routine. Regardless of whether a concept is added to the search queue 76, control returns to block 511 of the main routine (FIG. 14B) and the search queue 76 is displayed at the analyst terminal. The database processor 18 then enters the loop represented by blocks 474 and 476 of FIG. 14B until a request to perform another merger cycle (block 474) or to initiate a new search (block 476) is received as explained above.
  • Returning to block 550 of FIG. 17B, if the loop counter C1 is not equal to the sub-loop counter C2 minus one, control returns to block 502 (FIG. 17A) where the loop counter C1 is incremented. The sub-loop counter C2 is then reset (block 504). The database processor 18 continues to execute the loop specified by blocks 502-550 until a relevance score 78 has been calculated for every new concept that can be created by combining any two of the concepts present in the search queue 76 as it existed at the initiation of the current merger cycle (block 550). Once this task is completed (block 550), one new concept is added to the search queue 76 (if appropriate) (block 552) and control returns to (block 511) of FIG. 14B.
  • Those skilled in the art will appreciate that, although the teachings of the invention have been described in connection with certain examples, there is no intention to limit the invention thereto. On the contrary, the intention of this patent is to cover all methods and apparatus fairly falling within the scope of the appended claims either literally or under the doctrine of equivalents.

Claims (21)

1. A customer feedback acquisition and processing system comprising:
a service representative terminal for receiving customer feedback messages;
a data acquisition processor in communication with the service representative terminal for developing electronic records including text representative of the customer feedback messages;
a database in communication with the data acquisition processor for storing the records developed by the data acquisition processor;
a data analyst terminal for receiving query inputs; and
a database processor in communication with the database and the data analyst terminal, the database processor being responsive to a query input received from the data analyst terminal to analyze the text in the records stored in the database to identify a trend in the customer feedback messages, the trend associated with a plurality of different customers for a product or service.
2. A customer feedback acquisition and processing system as defined in claim 1 wherein the customer feedback messages comprise at least one of a voice signal received over a telephone, an audio signal, an email message, and data entered in a form from a web page.
3. A customer feedback acquisition and processing system as defined in claim 1 wherein the data acquisition processor comprises a speech recognizer for translating audio signals to text.
4. A customer feedback acquisition and processing system as defined in claim 1 wherein the data acquisition processor comprises a data formatter for determining whether a received customer feedback message includes at least one of a service representative comment, a customer comment, a product code, a reason code, a date and a time.
5. A customer feedback acquisition and processing system as defined in claim 4 wherein the data acquisition processor stores at least one segment of the received customer feedback message in a corresponding one of a service representative field, a customer comment field, a product code field, a reason code field, a date field, and a time field in a record in the database.
6. A customer feedback acquisition and processing system as defined in claim 5 wherein each of the records in the database corresponds to a customer feedback instance.
7. A customer feedback acquisition and processing system as defined in claim 1 wherein the database processor further comprises a search engine and queue generator which is responsive to the query input to produce a search result identifying a subset of the records in the database.
8. A customer feedback acquisition and processing system as defined in claim 7 wherein the database processor further comprises a relevance finder that computes a relevance score for each of a plurality of concepts in a search queue.
9. A customer feedback acquisition and processing system as defined in claim 8 wherein the relevance finder comprises a counter that develops a set count at least approximating a number of records in the database and that develops a subset count.
10. A customer feedback acquisition and processing system as defined in claim 9 wherein the relevance finder further comprises a frequency generator that generates a global frequency and that generates a local frequency.
11. A customer feedback acquisition and processing system as defined in claim 10 wherein the relevance finder further comprises a relevance calculator that calculates the relevance score for the given concept by dividing the local frequency by the global frequency.
12. A method of acquiring and analyzing customer feedback comprising the acts of:
receiving customer feedback messages;
developing electronic records including text representative of the customer feedback messages;
storing the developed records in a database;
receiving a query input; and
responding to the query input by analyzing the text in the records to identify a trend in the customer feedback messages associated with a plurality of different customers for a product or service.
13. A method as defined in claim 12 wherein the act of developing electronic records further comprises determining whether a received customer feedback message includes at least one of a service representative comment, a customer comment, a product code, a reason code, a date and a time.
14. A method as defined in claim 13 wherein the act of storing the records further comprises storing at least one segment of the received customer feedback message in a corresponding one of a service representative field, a customer comment field, a product code field, a reason code field, a date field, and a time field in a record in the database.
15. A method as defined in claim 12 wherein the act of responding to the query input by analyzing the text in the records further comprises the acts of:
producing a search result identifying a subset of the records in the database; and
producing a first search queue from the search result.
16. A method as defined in claim 15 wherein the act of responding to the query input by analyzing the text in the records further comprises the act of computing a relevance score.
17. A method as defined in claim 16 wherein the act of computing a relevance score further comprises the acts of:
developing a set count; and
developing a subset count.
18. A method as defined in claim 17 wherein the act of computing a relevance score further comprises the acts of:
generating a global frequency; and
generating a local frequency.
19. A method as defined in claim 18 wherein the act of computing a relevance score further comprises the acts of calculating the relevance score by dividing the local frequency by the global frequency.
20. A method as defined in claim 16 further comprising the act of sorting based on the relevance scores.
21. In a computer readable storage medium having stored therein data representing instructions executable by a programmed processor for use with a database comprising a plurality of records containing text data, the storage medium comprising instructions for the database mining tool comprising:
receiving customer feedback messages;
developing electronic records including text representative of the customer feedback messages; storing the developed records in a database;
receiving a query input; and
responding to the query input by analyzing the text in the records to identify a trend in the customer feedback messages associated with a plurality of different customers.
US11/403,977 1999-07-19 2006-04-12 Customer feedback acquisition and processing system Abandoned US20060184526A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US09/356,668 US6510427B1 (en) 1999-07-19 1999-07-19 Customer feedback acquisition and processing system
US10/242,118 US7058625B2 (en) 1999-07-19 2002-09-12 Customer feedback acquisition and processing system
US11/403,977 US20060184526A1 (en) 1999-07-19 2006-04-12 Customer feedback acquisition and processing system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US11/403,977 US20060184526A1 (en) 1999-07-19 2006-04-12 Customer feedback acquisition and processing system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/242,118 Continuation US7058625B2 (en) 1999-07-19 2002-09-12 Customer feedback acquisition and processing system

Publications (1)

Publication Number Publication Date
US20060184526A1 true US20060184526A1 (en) 2006-08-17

Family

ID=23402417

Family Applications (3)

Application Number Title Priority Date Filing Date
US09/356,668 Expired - Lifetime US6510427B1 (en) 1999-07-19 1999-07-19 Customer feedback acquisition and processing system
US10/242,118 Active 2020-08-21 US7058625B2 (en) 1999-07-19 2002-09-12 Customer feedback acquisition and processing system
US11/403,977 Abandoned US20060184526A1 (en) 1999-07-19 2006-04-12 Customer feedback acquisition and processing system

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US09/356,668 Expired - Lifetime US6510427B1 (en) 1999-07-19 1999-07-19 Customer feedback acquisition and processing system
US10/242,118 Active 2020-08-21 US7058625B2 (en) 1999-07-19 2002-09-12 Customer feedback acquisition and processing system

Country Status (1)

Country Link
US (3) US6510427B1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040264670A1 (en) * 2003-06-24 2004-12-30 International Business Machines Corporation Method for managing resources in a multi-channeled customer service environment
US20080288348A1 (en) * 2007-05-15 2008-11-20 Microsoft Corporation Ranking online advertisements using retailer and product reputations
US20090276417A1 (en) * 2008-05-02 2009-11-05 Ido Shapira Method and system for database query term suggestion
US20110264478A1 (en) * 2010-04-21 2011-10-27 Sanyo Electric Co., Ltd. Attachment device, information collection device, and method for obtaining information about reasons for return
US8108359B1 (en) * 2007-12-14 2012-01-31 Symantec Corporation Methods and systems for tag-based object management
US20140052842A1 (en) * 2012-08-17 2014-02-20 International Business Machines Corporation Measuring problems from social media discussions
US9535902B1 (en) 2013-06-28 2017-01-03 Digital Reasoning Systems, Inc. Systems and methods for entity resolution using attributes from structured and unstructured data
US9633002B1 (en) 2014-06-27 2017-04-25 Digital Reasoning Systems, Inc. Systems and methods for coreference resolution using selective feature activation
US9697192B1 (en) 2013-06-28 2017-07-04 Digital Reasoning Systems, Inc. Systems and methods for construction, maintenance, and improvement of knowledge representations

Families Citing this family (89)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6938029B1 (en) * 1999-03-31 2005-08-30 Allan Y. Tien System and method for indexing recordings of observed and assessed phenomena using pre-defined measurement items
US20020007285A1 (en) * 1999-06-18 2002-01-17 Rappaport Alain T. Method, apparatus and system for providing targeted information in relation to laboratory and other medical services
US8041805B2 (en) * 1999-08-30 2011-10-18 Opinionlab, Inc. System and method for reporting to a website owner user reactions to particular web pages of a website
US7085820B1 (en) 1999-08-30 2006-08-01 Opinionlab, Inc. System and method for reporting to a website owner user reactions to particular web pages of a website
US6421724B1 (en) 1999-08-30 2002-07-16 Opinionlab, Inc. Web site response measurement tool
US6677963B1 (en) * 1999-11-16 2004-01-13 Verizon Laboratories Inc. Computer-executable method for improving understanding of business data by interactive rule manipulation
US7962359B2 (en) * 2000-04-06 2011-06-14 Autopoll, Inc. Method and system for collecting and disseminating survey data over the internet
US6606581B1 (en) * 2000-06-14 2003-08-12 Opinionlab, Inc. System and method for measuring and reporting user reactions to particular web pages of a website
JP2002007560A (en) * 2000-06-19 2002-01-11 Az-Navi:Kk Recollection database system and medium with recorded recollection support program
EP1195675A1 (en) * 2000-07-14 2002-04-10 Sun Microsystems, Inc. Comment driven processing
US6631184B1 (en) * 2000-07-24 2003-10-07 Comverse Ltd. System for community generated feedback and/or rating
US20020044687A1 (en) * 2000-10-16 2002-04-18 Emailcard Feedback Systems, Inc. Customer feedback system
US8868448B2 (en) 2000-10-26 2014-10-21 Liveperson, Inc. Systems and methods to facilitate selling of products and services
US7526439B2 (en) 2001-08-06 2009-04-28 Proficient Systems, Incorporated Systems and methods to facilitate selling of products and services
US9819561B2 (en) 2000-10-26 2017-11-14 Liveperson, Inc. System and methods for facilitating object assignments
US20060015390A1 (en) * 2000-10-26 2006-01-19 Vikas Rijsinghani System and method for identifying and approaching browsers most likely to transact business based upon real-time data mining
US6765593B2 (en) * 2001-01-24 2004-07-20 International Business Machines Corporation Method and system for evaluating serviceability of a computer system
USRE46973E1 (en) 2001-05-07 2018-07-31 Ureveal, Inc. Method, system, and computer program product for concept-based multi-dimensional analysis of unstructured information
US8589413B1 (en) * 2002-03-01 2013-11-19 Ixreveal, Inc. Concept-based method and system for dynamically analyzing results from search engines
US20020174170A1 (en) * 2001-05-21 2002-11-21 Serge Ioffe Method and system for distributing messages to agents
US20030005087A1 (en) * 2001-05-24 2003-01-02 Microsoft Corporation Server side sampling of databases
KR20020090705A (en) * 2001-05-29 2002-12-05 현대자동차주식회사 Method of developing vehicles by information providers at an on-line
US6711569B1 (en) * 2001-07-24 2004-03-23 Bright Planet Corporation Method for automatic selection of databases for searching
US7146409B1 (en) * 2001-07-24 2006-12-05 Brightplanet Corporation System and method for efficient control and capture of dynamic database content
US7613717B1 (en) * 2001-10-26 2009-11-03 Teradata Us, Inc. Automated system for rating customer feedback
US7493559B1 (en) * 2002-01-09 2009-02-17 Ricoh Co., Ltd. System and method for direct multi-modal annotation of objects
US7010496B2 (en) * 2002-02-06 2006-03-07 Accenture Global Services Gmbh Supplier performance reporting
WO2004012126A2 (en) * 2002-07-29 2004-02-05 Opinionlab, Inc. System and method for providing substantially real-time access to collected information concerning user interaction with a web page of a website
US7370285B1 (en) 2002-07-31 2008-05-06 Opinionlab, Inc. Receiving and reporting page-specific user feedback concerning one or more particular web pages of a website
US7478121B1 (en) * 2002-07-31 2009-01-13 Opinionlab, Inc. Receiving and reporting page-specific user feedback concerning one or more particular web pages of a website
US6985907B2 (en) * 2002-08-14 2006-01-10 Ford Motor Company Computer-implemented method and system for attributing applicable condition codes to field claims
US20040049417A1 (en) * 2002-09-09 2004-03-11 Opinionlab, Inc. Receiving and reporting page-specific user feedback concerning one or more particular web pages of a website
US20040139113A1 (en) * 2002-10-29 2004-07-15 Listou Robert Eugene Computer program for rapidly editing and visually distinguishing elements of presentation of parametric text data objects and associated files
US20040172323A1 (en) * 2003-02-28 2004-09-02 Bellsouth Intellectual Property Corporation Customer feedback method and system
US7877265B2 (en) * 2003-05-13 2011-01-25 At&T Intellectual Property I, L.P. System and method for automated customer feedback
US7827487B1 (en) 2003-06-16 2010-11-02 Opinionlab, Inc. Soliciting user feedback regarding one or more web pages of a website without obscuring visual content
US7339347B2 (en) * 2003-08-11 2008-03-04 Reserve Power Cell, Llc Apparatus and method for reliably supplying electrical energy to an electrical system
US20050076089A1 (en) * 2003-09-18 2005-04-07 Fonseca Arnoldo Medeiros Method and system for communication from anonymous sender(s) to known recipient(s) for feedback applications
US7698345B2 (en) 2003-10-21 2010-04-13 The Nielsen Company (Us), Llc Methods and apparatus for fusing databases
US20050204378A1 (en) * 2004-03-10 2005-09-15 Shay Gabay System and method for video content analysis-based detection, surveillance and alarm management
US7321903B2 (en) 2004-05-06 2008-01-22 International Business Machines Corporation Method for unified collection of content analytic data
US20060265368A1 (en) * 2005-05-23 2006-11-23 Opinionlab, Inc. Measuring subjective user reaction concerning a particular document
US8521763B1 (en) 2005-09-09 2013-08-27 Minnesota Public Radio Computer-based system and method for processing data for a journalism organization
US8738732B2 (en) * 2005-09-14 2014-05-27 Liveperson, Inc. System and method for performing follow up based on user interactions
US9432468B2 (en) 2005-09-14 2016-08-30 Liveperson, Inc. System and method for design and dynamic generation of a web page
US8396468B1 (en) * 2005-09-15 2013-03-12 At&T Mobility Ii Llc Assessing performance and quality of a mobile communication service
US8509761B2 (en) * 2005-09-15 2013-08-13 At&T Mobility Ii Llc Location based services quality assessment
US8587630B1 (en) 2005-09-15 2013-11-19 At&T Mobility Ii Llc Assessing performance and quality of a mobile communication service
US8620293B2 (en) * 2005-09-15 2013-12-31 At&T Mobility Ii Llc Location-based transmitter selection and handoff
US20070067405A1 (en) * 2005-09-20 2007-03-22 Eliovson Joshua M Moderated anonymous forum
US8571320B2 (en) 2005-12-12 2013-10-29 Core Wireless Licensing S.A.R.L. Method and apparatus for pictorial identification of a communication event
US20070218834A1 (en) * 2006-02-23 2007-09-20 Ransys Ltd. Method and apparatus for continuous sampling of respondents
US8775237B2 (en) 2006-08-02 2014-07-08 Opinionlab, Inc. System and method for measuring and reporting user reactions to advertisements on a web page
US7809602B2 (en) * 2006-08-31 2010-10-05 Opinionlab, Inc. Computer-implemented system and method for measuring and reporting business intelligence based on comments collected from web page users using software associated with accessed web pages
JP4771915B2 (en) * 2006-11-15 2011-09-14 京セラミタ株式会社 Device that converts Html text, METHOD, AND PROGRAM
US20080163225A1 (en) * 2006-12-29 2008-07-03 Fragala Thomas F Method and/or system for task scheduling
US7908260B1 (en) 2006-12-29 2011-03-15 BrightPlanet Corporation II, Inc. Source editing, internationalization, advanced configuration wizard, and summary page selection for information automation systems
US7958106B2 (en) * 2007-03-14 2011-06-07 Tacoda Llc System and method for determining client metadata using a dynamic rules engine
US7814084B2 (en) * 2007-03-21 2010-10-12 Schmap Inc. Contact information capture and link redirection
US8977631B2 (en) * 2007-04-16 2015-03-10 Ebay Inc. Visualization of reputation ratings
US8260687B2 (en) * 2007-04-16 2012-09-04 Ebay Inc. Distributed commerce application-widget
JP5033724B2 (en) * 2007-07-12 2012-09-26 株式会社沖データ Document retrieval apparatus and an image forming apparatus, a document retrieval system
US20090138797A1 (en) * 2007-11-28 2009-05-28 Robert Eugene Listou Automated sequential imaging of selected computer produced pages of data associated with designated text data objects listed on automated permutations of data tables
US20090234665A1 (en) * 2008-03-11 2009-09-17 Electronic Data Systems Corporation System and method for customer feedback
US7865455B2 (en) * 2008-03-13 2011-01-04 Opinionlab, Inc. System and method for providing intelligent support
US20090248647A1 (en) * 2008-03-25 2009-10-01 Omer Ziv System and method for the quality assessment of queries
US20110238430A1 (en) * 2008-04-23 2011-09-29 ProvidedPath Software, inc. Organization Optimization System and Method of Use Thereof
US8359621B2 (en) * 2008-05-05 2013-01-22 At&T Intellectual Property I, Lp System and method for monitoring delivery of media content by a media communication system
US8799200B2 (en) 2008-07-25 2014-08-05 Liveperson, Inc. Method and system for creating a predictive model for targeting webpage to a surfer
US8762313B2 (en) 2008-07-25 2014-06-24 Liveperson, Inc. Method and system for creating a predictive model for targeting web-page to a surfer
US8805844B2 (en) 2008-08-04 2014-08-12 Liveperson, Inc. Expert search
US9892417B2 (en) 2008-10-29 2018-02-13 Liveperson, Inc. System and method for applying tracing tools for network locations
US20100205024A1 (en) * 2008-10-29 2010-08-12 Haggai Shachar System and method for applying in-depth data mining tools for participating websites
WO2010120679A2 (en) * 2009-04-12 2010-10-21 The Brookeside Group, Inc. Emotivity and vocality measurement
US9245243B2 (en) * 2009-04-14 2016-01-26 Ureveal, Inc. Concept-based analysis of structured and unstructured data using concept inheritance
US20110004474A1 (en) * 2009-07-02 2011-01-06 International Business Machines Corporation Audience Measurement System Utilizing Voice Recognition Technology
US8635237B2 (en) * 2009-07-02 2014-01-21 Nuance Communications, Inc. Customer feedback measurement in public places utilizing speech recognition technology
US8332232B2 (en) * 2009-11-05 2012-12-11 Opinionlab, Inc. System and method for mobile interaction
US9767212B2 (en) 2010-04-07 2017-09-19 Liveperson, Inc. System and method for dynamically enabling customized web content and applications
US9350598B2 (en) 2010-12-14 2016-05-24 Liveperson, Inc. Authentication of service requests using a communications initiation feature
US8918465B2 (en) 2010-12-14 2014-12-23 Liveperson, Inc. Authentication of service requests initiated from a social networking site
CN102769832B (en) * 2011-05-06 2016-09-28 中兴通讯股份有限公司 A method and system for multimedia creation
US8943002B2 (en) 2012-02-10 2015-01-27 Liveperson, Inc. Analytics driven engagement
US8805941B2 (en) 2012-03-06 2014-08-12 Liveperson, Inc. Occasionally-connected computing interface
US9563336B2 (en) 2012-04-26 2017-02-07 Liveperson, Inc. Dynamic user interface customization
US9672196B2 (en) 2012-05-15 2017-06-06 Liveperson, Inc. Methods and systems for presenting specialized content using campaign metrics
US20140236649A1 (en) * 2013-02-21 2014-08-21 Atlassian Pty Ltd Method and Apparatus for Performing Remote Operations in an Issue Tracking Environment
US9118763B1 (en) 2014-12-09 2015-08-25 Five9, Inc. Real time feedback proxy
CN109844717A (en) 2016-08-14 2019-06-04 利维帕尔森有限公司 The system and method that real time remote for mobile applications controls

Citations (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4430726A (en) * 1981-06-18 1984-02-07 Bell Telephone Laboratories, Incorporated Dictation/transcription method and arrangement
US4841387A (en) * 1987-12-15 1989-06-20 Rindfuss Diane J Arrangement for recording and indexing information
US4908866A (en) * 1985-02-04 1990-03-13 Eric Goldwasser Speech transcribing system
US5136655A (en) * 1990-03-26 1992-08-04 Hewlett-Pacard Company Method and apparatus for indexing and retrieving audio-video data
US5303299A (en) * 1990-05-15 1994-04-12 Vcs Industries, Inc. Method for continuous recognition of alphanumeric strings spoken over a telephone network
US5500920A (en) * 1993-09-23 1996-03-19 Xerox Corporation Semantic co-occurrence filtering for speech recognition and signal transcription applications
US5526407A (en) * 1991-09-30 1996-06-11 Riverrun Technology Method and apparatus for managing information
US5553119A (en) * 1994-07-07 1996-09-03 Bell Atlantic Network Services, Inc. Intelligent recognition of speech signals using caller demographics
US5600756A (en) * 1994-05-11 1997-02-04 Sony Corporation Method of labelling takes in an audio editing system
US5619708A (en) * 1994-10-25 1997-04-08 Korteam International, Inc. System and method for generating database input forms
US5642518A (en) * 1993-06-18 1997-06-24 Hitachi, Ltd. Keyword assigning method and system therefor
US5704060A (en) * 1995-05-22 1997-12-30 Del Monte; Michael G. Text storage and retrieval system and method
US5734794A (en) * 1995-06-22 1998-03-31 White; Tom H. Method and system for voice-activated cell animation
US5787421A (en) * 1995-01-12 1998-07-28 International Business Machines Corporation System and method for information retrieval by using keywords associated with a given set of data elements and the frequency of each keyword as determined by the number of data elements attached to each keyword
US5809464A (en) * 1994-09-24 1998-09-15 Alcatel N.V. Apparatus for recording speech for subsequent text generation
US5822744A (en) * 1996-07-15 1998-10-13 Kesel; Brad Consumer comment reporting apparatus and method
US5832433A (en) * 1996-06-24 1998-11-03 Nynex Science And Technology, Inc. Speech synthesis method for operator assistance telecommunications calls comprising a plurality of text-to-speech (TTS) devices
US5835912A (en) * 1997-03-13 1998-11-10 The United States Of America As Represented By The National Security Agency Method of efficiency and flexibility storing, retrieving, and modifying data in any language representation
US5848389A (en) * 1995-04-07 1998-12-08 Sony Corporation Speech recognizing method and apparatus, and speech translating system
US5943395A (en) * 1997-05-30 1999-08-24 Northern Telecom Limited Telephone apparatus, systems, and processes to enhance access for TDD and/or TTY devices
US6032184A (en) * 1995-12-29 2000-02-29 Mci Worldcom, Inc. Integrated interface for Web based customer care and trouble management
US6115693A (en) * 1998-04-17 2000-09-05 Andersen Consulting Llp Quality center and method for a virtual sales and service center
US6189029B1 (en) * 1996-09-20 2001-02-13 Silicon Graphics, Inc. Web survey tool builder and result compiler
US6349290B1 (en) * 1998-06-30 2002-02-19 Citibank, N.A. Automated system and method for customized and personalized presentation of products and services of a financial institution
US6460036B1 (en) * 1994-11-29 2002-10-01 Pinpoint Incorporated System and method for providing customized electronic newspapers and target advertisements
US6862710B1 (en) * 1999-03-23 2005-03-01 Insightful Corporation Internet navigation using soft hyperlinks
US7065494B1 (en) * 1999-06-25 2006-06-20 Nicholas D. Evans Electronic customer service and rating system and method

Patent Citations (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4430726A (en) * 1981-06-18 1984-02-07 Bell Telephone Laboratories, Incorporated Dictation/transcription method and arrangement
US4908866A (en) * 1985-02-04 1990-03-13 Eric Goldwasser Speech transcribing system
US4841387A (en) * 1987-12-15 1989-06-20 Rindfuss Diane J Arrangement for recording and indexing information
US5136655A (en) * 1990-03-26 1992-08-04 Hewlett-Pacard Company Method and apparatus for indexing and retrieving audio-video data
US5303299A (en) * 1990-05-15 1994-04-12 Vcs Industries, Inc. Method for continuous recognition of alphanumeric strings spoken over a telephone network
US5526407A (en) * 1991-09-30 1996-06-11 Riverrun Technology Method and apparatus for managing information
US5642518A (en) * 1993-06-18 1997-06-24 Hitachi, Ltd. Keyword assigning method and system therefor
US5500920A (en) * 1993-09-23 1996-03-19 Xerox Corporation Semantic co-occurrence filtering for speech recognition and signal transcription applications
US5600756A (en) * 1994-05-11 1997-02-04 Sony Corporation Method of labelling takes in an audio editing system
US5553119A (en) * 1994-07-07 1996-09-03 Bell Atlantic Network Services, Inc. Intelligent recognition of speech signals using caller demographics
US5809464A (en) * 1994-09-24 1998-09-15 Alcatel N.V. Apparatus for recording speech for subsequent text generation
US5619708A (en) * 1994-10-25 1997-04-08 Korteam International, Inc. System and method for generating database input forms
US6460036B1 (en) * 1994-11-29 2002-10-01 Pinpoint Incorporated System and method for providing customized electronic newspapers and target advertisements
US5787421A (en) * 1995-01-12 1998-07-28 International Business Machines Corporation System and method for information retrieval by using keywords associated with a given set of data elements and the frequency of each keyword as determined by the number of data elements attached to each keyword
US5848389A (en) * 1995-04-07 1998-12-08 Sony Corporation Speech recognizing method and apparatus, and speech translating system
US5704060A (en) * 1995-05-22 1997-12-30 Del Monte; Michael G. Text storage and retrieval system and method
US5734794A (en) * 1995-06-22 1998-03-31 White; Tom H. Method and system for voice-activated cell animation
US6032184A (en) * 1995-12-29 2000-02-29 Mci Worldcom, Inc. Integrated interface for Web based customer care and trouble management
US5832433A (en) * 1996-06-24 1998-11-03 Nynex Science And Technology, Inc. Speech synthesis method for operator assistance telecommunications calls comprising a plurality of text-to-speech (TTS) devices
US5822744A (en) * 1996-07-15 1998-10-13 Kesel; Brad Consumer comment reporting apparatus and method
US6189029B1 (en) * 1996-09-20 2001-02-13 Silicon Graphics, Inc. Web survey tool builder and result compiler
US5835912A (en) * 1997-03-13 1998-11-10 The United States Of America As Represented By The National Security Agency Method of efficiency and flexibility storing, retrieving, and modifying data in any language representation
US5943395A (en) * 1997-05-30 1999-08-24 Northern Telecom Limited Telephone apparatus, systems, and processes to enhance access for TDD and/or TTY devices
US6115693A (en) * 1998-04-17 2000-09-05 Andersen Consulting Llp Quality center and method for a virtual sales and service center
US6349290B1 (en) * 1998-06-30 2002-02-19 Citibank, N.A. Automated system and method for customized and personalized presentation of products and services of a financial institution
US6862710B1 (en) * 1999-03-23 2005-03-01 Insightful Corporation Internet navigation using soft hyperlinks
US7065494B1 (en) * 1999-06-25 2006-06-20 Nicholas D. Evans Electronic customer service and rating system and method

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090225764A1 (en) * 2003-06-24 2009-09-10 International Business Machines Corporation Method for managing resources in a multi-channeled customer service environment
US8126135B2 (en) 2003-06-24 2012-02-28 International Business Machines Corporation Method for managing resources in a multi-channeled customer service environment
US7519173B2 (en) * 2003-06-24 2009-04-14 International Business Machines Corporation Method for managing resources in a multi-channeled customer service environment
US20040264670A1 (en) * 2003-06-24 2004-12-30 International Business Machines Corporation Method for managing resources in a multi-channeled customer service environment
US20080288348A1 (en) * 2007-05-15 2008-11-20 Microsoft Corporation Ranking online advertisements using retailer and product reputations
US8108359B1 (en) * 2007-12-14 2012-01-31 Symantec Corporation Methods and systems for tag-based object management
US20090276417A1 (en) * 2008-05-02 2009-11-05 Ido Shapira Method and system for database query term suggestion
US8832135B2 (en) * 2008-05-02 2014-09-09 Verint Systems, Ltd. Method and system for database query term suggestion
US20110264478A1 (en) * 2010-04-21 2011-10-27 Sanyo Electric Co., Ltd. Attachment device, information collection device, and method for obtaining information about reasons for return
US20140052842A1 (en) * 2012-08-17 2014-02-20 International Business Machines Corporation Measuring problems from social media discussions
US9824403B2 (en) * 2012-08-17 2017-11-21 International Business Machines Corporation Measuring problems from social media discussions
US9535902B1 (en) 2013-06-28 2017-01-03 Digital Reasoning Systems, Inc. Systems and methods for entity resolution using attributes from structured and unstructured data
US9697192B1 (en) 2013-06-28 2017-07-04 Digital Reasoning Systems, Inc. Systems and methods for construction, maintenance, and improvement of knowledge representations
US9633002B1 (en) 2014-06-27 2017-04-25 Digital Reasoning Systems, Inc. Systems and methods for coreference resolution using selective feature activation

Also Published As

Publication number Publication date
US7058625B2 (en) 2006-06-06
US20030041056A1 (en) 2003-02-27
US6510427B1 (en) 2003-01-21

Similar Documents

Publication Publication Date Title
US5832068A (en) Data processing system with real time priority updating of data records and dynamic record exclusion
US6615176B2 (en) Speech enabling labeless controls in an existing graphical user interface
US7673340B1 (en) System and method for analyzing system user behavior
US8805929B2 (en) Event-driven annotation techniques
US7392240B2 (en) System and method for searching and matching databases
US10313520B2 (en) Real-time compliance monitoring facility
US5357431A (en) Character string retrieval system using index and unit for making the index
US7779349B2 (en) Method for adapting a K-means text clustering to emerging data
US5909669A (en) System and method for generating a knowledge worker productivity assessment
US6683947B2 (en) Call center monitoring system
US6710822B1 (en) Signal processing method and image-voice processing apparatus for measuring similarities between signals
CN1030114C (en) Apparatus and method of Chinese speech characters/Chinese changing
US8239375B2 (en) Method of searching for personal information management (PIM) information and handheld electronic device employing the same
US20030171942A1 (en) Contact relationship management system and method
Brown et al. Open-vocabulary speech indexing for voice and video mail retrieval
US8131524B2 (en) Method and system for automating the creation of customer-centric interfaces
CA2493640C (en) Improvements in or relating to information provision for call centres
US6424971B1 (en) System and method for interactive classification and analysis of data
EP1656662B1 (en) System for and method of automated quality monitoring
US20030185379A1 (en) Managing communications in a call centre
US6823054B1 (en) Apparatus and method for analyzing an automated response system
US20020052870A1 (en) Indexing method and apparatus
US20040120472A1 (en) Voice response system
US5555325A (en) Data capture variable priority method and system for managing varying processing capacities
US6970554B1 (en) System and method for observing calls to a call center

Legal Events

Date Code Title Description
AS Assignment

Owner name: AT&T INTELLECTUAL PROPERTY I, L.P., NEVADA

Free format text: CHANGE OF NAME;ASSIGNOR:SBC PROPERTIES, L.P.;REEL/FRAME:021217/0664

Effective date: 20020626

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION