JP2018512634A - System and method for automated dynamic messaging management - Google Patents

Abstract

Systems and methods are provided for automated dynamic message management. In some embodiments, a data store is added with one or more knowledge sets and one or more leader data sets in response to user input. A campaign builder may be given to the user to create and start a campaign. A campaign is a series of messages designed to serve one or more purposes. The campaign builder allows the creation of a campaign by allowing the composition of a series of message templates with variable fields. The variable field corresponds to a class of data from the knowledge set and / or leader data. At the start of the campaign, the system categorizes responses using algorithms. These categorizations have a corresponding confidence level. If the confidence level is too low, manual user intervention may be required to determine which subsequent actions the system should perform. [Selection] Figure 1

Description

  The present invention relates to a system and method for generating and managing dynamic messaging campaigns. Such systems and methods provide marketers and sales personnel with more efficient tools for client management and outreach. Such systems and methods also allow for more productive sales activities, increased profits, and more efficient allocation of sales resources.

  Currently, sales departments are passive and active. Passive sales activities include making a general offer for the sale of products and / or services to the masses and waiting for customers to contact them for the first time. In contrast, aggressive sales techniques involve sellers working directly with consumers. The advantage of aggressive sales activities is that they can target customers more effectively and can influence purchase decisions more effectively. Aggressive sales techniques may include dive “investment” or follow-up with “leaders” in response to some advertisement or purchased from a marketing company. While the Kang Yuan story has some meaning, continued dialogue with established leaders is the most targeted and effective means of marketing.

  Aggressive sales techniques have worked for a long time in business transactions. Sellers have traditionally sold their products using human solicitations and flyers. In fact, to date, advertisements are usually sent to consumers by mail. If available, these mailed advertisements include a degree of customization, such as including the recipient name printed on the advertisement.

  As technology has advanced, aggressive sales techniques have evolved considerably. With the widespread use of telephones, telemarketing has become the center of active sales technology. This initially took the form of a prospective customer for the salesperson's “collusion” story, but recently “Robocall” has become even more popular due to the ability to reach a fairly large mass with little additional resource expenditure. It has become popular.

  As the Internet has become a more prominent feature of transactions, online advertising and email campaigns have been added to the sales sector's weapons as a way to involve potential consumers. Email marketing in particular has become a very effective and frequently used customer reach. For large customer groups, these emails are generally minimally advertised. For small customer groups, individual emails may still be created by sales representatives. However, this activity is often very time consuming (although more effective). In addition, sales representatives may typically be involved in only a limited number of these sales communications without using contact management software.

  Thus, it is clear that there is an urgent need for a dynamic messaging system that brings the benefits of machine automation to the benefits of personalized email sales support. Such dynamic messaging allows for more effective sales activities and marketing campaigns.

  To achieve the foregoing, according to the present invention, a system and method for automated dynamic message management is provided.

Such systems and methods allow marketers and salespeople to follow up with their leaders more efficiently via email (or other text) exchanges. Using artificial intelligence, users need to provide relatively minimal manual intervention until all objectives of message exchange are met.

  In some embodiments, a data store is added with one or more knowledge sets and one or more leader data sets in response to user input. Leader data includes information collected about the target consumer (or leader), while the knowledge set provides the contextual knowledge necessary for artificial intelligence to categorize any incoming response.

  A campaign builder may be given to the user to create and start a campaign. A campaign is a series of messages designed to serve one or more purposes. Information collected through these message exchanges is called insight. The campaign builder (campaign component) enables the creation of a campaign by allowing the composition of a series of message templates with variable fields. The variable field corresponds to a class of data from the knowledge set and / or leader data. These templates may be generated from scratch or imported from an existing campaign (or template library).

  At the start of the campaign, the system categorizes responses using algorithms. These categorizations have a corresponding confidence level. If the confidence level is too low, manual user intervention may be required to determine which subsequent actions the system should perform. This may be done via an action interface.

  In addition, in some embodiments, the user can provide an insight interface (for reviewing and updating generated insights), an AI training tool (for adjusting AI algorithms), a knowledge set A knowledge set interface (to create or update) and a statistics interface (to provide user information regarding campaign status) can be accessed.

  It should be noted that the various features of the present invention described above may be implemented alone or in combination. In the following detailed description of the invention, these and other features of the present invention will be described in more detail in conjunction with the following figures.

  In order that the present invention may be more clearly understood, several embodiments will now be described by way of example with reference to the accompanying drawings.

1 is an example of a logical diagram of a system according to some embodiments for generating and implementing a messaging campaign. FIG. FIG. 2 is an example of a logical diagram of a dynamic messaging server according to some embodiments. FIG. 3 is an example of a logic diagram of a user interface in a dynamic messaging server according to some embodiments. FIG. 2 is an example of a logic diagram of a message generator in a dynamic messaging server according to some embodiments. 2 is an example of a logic diagram of a message response system in a dynamic messaging server according to some embodiments. FIG. FIG. 3 is an example of a flow diagram for a dynamic message campaign according to some embodiments. FIG. 3 is an example of a flow diagram for a process for onboarding a user according to some embodiments. FIG. 3 is an example of a flow diagram for a process for building a campaign according to some embodiments. FIG. 3 is an example of a flow diagram for a process for generating a message template according to some embodiments. FIG. 3 is an example of a flow diagram for a process for implementing a campaign according to some embodiments. FIG. 3 is an example flow diagram for a process for creating and sending outgoing messages according to some embodiments. FIG. 3 is an example flow diagram for a process for processing received responses according to some embodiments. FIG. 6 is an example screenshot of an interface for building a campaign according to some embodiments. FIG. 6 is an example screenshot of an interface for building a campaign according to some embodiments. FIG. 6 is an example screenshot of an interface for building a campaign according to some embodiments. FIG. 6 is an example screenshot of an interface for building a campaign according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of an AI training interface of a management dashboard according to some embodiments. 6 is an example screenshot of a context management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a context management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a context management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a context management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a context management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an insight management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an insight management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an insight management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an insight management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a knowledge set management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a knowledge set management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a knowledge set management interface of a management dashboard according to some embodiments. 6 is an example screenshot of a knowledge set management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an action management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an action management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an action management interface of a management dashboard according to some embodiments. 6 is an example screenshot of an action management interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of a statistics interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of a statistics interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of a statistics interface of a management dashboard according to some embodiments. FIG. 6 is an example screenshot of a statistics interface of a management dashboard according to some embodiments. It is an example figure of the computer system which can embody this invention. It is an example figure of the computer system which can embody this invention.

  The present invention will now be described in detail in connection with some embodiments of the invention as illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments may be practiced without some or all of these specific details. In other instances, well known process steps and / or structures have not been described in detail in order not to unnecessarily obscure the present invention. The features and advantages of the embodiments may be better understood with reference to the following discussion and drawings.

  The aspects, features and advantages of exemplary embodiments of the present invention will become better understood with regard to the following description taken in conjunction with the accompanying drawings. As will be appreciated by those skilled in the art, the described embodiments of the invention provided herein are merely exemplary and not limiting and are provided by way of example only. All features disclosed in this description may be replaced with other features serving the same or similar purposes, unless expressly stated otherwise. Accordingly, many other embodiments of the variations are considered to fall within the scope of the invention and its equivalents as defined herein. Thus, for example, “do”, “do not”, “do it”, “do not do”, “must do”, “do not do”, “first”, “first” , “Next”, “after”, “previous”, “after”, “at the end”, “at the end”, etc., the use of absolute and / or sequential terms shall limit the scope of the invention It is not something to do. This is because the embodiments disclosed herein are merely exemplary.

  It should be noted that the term “user” is used to represent a user of a device that generates and manages a messaging campaign. Similarly, it is understood that the terms “participant”, “salesperson”, and “salesperson” are often used interchangeably with the term “user”.

  Similarly, the term “recipient” is used to indicate who receives the generated message. Other terms such as “consumer” and “leader” may be used interchangeably.

  Finally, the following discussion and accompanying examples are directed to the use of messaging systems primarily in the context of sales activities that involve developing sales leaders. Although sales activities are a common use case in the systems and methods disclosed herein, the messaging system described herein is not limited to sales activities. Indeed, the systems and methods disclosed herein can be used in a variety of situations and situations. For example, the disclosed messaging system may be useful in customer support settings, educational campaigns, funding, or other situations where multiple messages within a given context are required.

The following disclosure includes a series of subsections. These subsections are in no way intended to limit the scope of the present disclosure, but merely for clarity and readability. Accordingly, disclosure in one section may apply equally to processes or descriptions in other sections, where appropriate.
I. Definition

  The following systems and methods for dynamic messaging campaigns rely on user interaction interactions and enhance the artificial intelligence (AI) processing of received messages. The goal of a message campaign is to allow logical dialog interaction with recipients, in which case recipients are not necessarily aware that they are communicating with an automated machine rather than a human user. Not necessarily. This may be done most efficiently by document dialogs such as e-mail, text message, chat. However, given the advances in audio and video processing, it can be entirely assumed that it may be entirely possible to further include audio and video components in the dialog.

  To achieve such an interaction, the AI system is used within the AI platform in the messaging system to process the response and generate conclusions about the interaction. These conclusions include calculating confidence about the document context, insights, sentiments, and conclusions. Given that these terms are not very familiar outside the field of natural language processing, a series of definitions are given to clarify the terms.

  Accuracy-the calculated probability that the classification determined by AI is correct.

  (AI) algorithm—a method used to calculate the weight of a document in a particular category.

  Aspect—a specific AI algorithm. Example: Naive Bayes, sentiment.

  Trial-a single message in a series for a campaign.

  AI trainer—a term for tools used to classify documents whose aspect was not credible scoring.

  Campaign—A set of possible messaging (sending and receiving) messages that are to be sent to the leader through conversation in response to receipt and classification of responses (or lack thereof).

  Categorization-the process by which ideas and ideas are generally recognized, differentiated and understood into categories.

Category—A possible answer to the insights to which they belong. Example:
Insight: “Do you want to continue the message?” Has the categories: “Stop” and “Continue”.

  Classification-another word for categorization.

  Confidence-the calculated probability that the categorization is correct.

Context-a group of documents with some commonality.
Example: “What is a good phone number? "All documents collected by asking", "Messages sent from customers chatting with Bill for customer service".

  Document-A set of words in a specific order used to convey meaning.

  Hard rule-an AI algorithm that determines a category based on a single token. It can be seen that these tokens occur predominantly within these particular categories.

  Hard rule term-a token used by a hard rule aspect.

  Insight—A group of categories used to answer some question about a document. Examples: “What does this person mean?” “How does this person feel?” “Would you keep sending email to this person?”

  Knowledge set—a set of tokens whose associated category weights are used by the aspect during classification.

  Leader (Principal user)-A person who is placed in the system at a specific time under a specific campaign (a person present in the system).

  Leader (event) history-in the time sequence of their occurrence, notable information for the leader entering the system, messages sent to that leader, responses received, and alarm alerts sent .

  n-gram—indicates the number of words used to form a token. Example: The token “Yes, yes, is” is a trigram or 3 engrams.

  Normalization—Remove characters / tokens to reduce document complexity without changing the accuracy of classification.

  Question-A query included in a message that is designed to limit responses to a subset of the target language.

  Response-A document that can be received after sending a message to a leader.

  (Response) Action—A task that the system can perform for a given lead based on a response classification.

  Sentiment-an AI algorithm used to measure how strongly a category is itself represented in a document.

  Series—a subset of a campaign that is to be sent until a response is received for that subset. Based on the response classification, the system may continue for other series of messaging within the same campaign.

  Score-a set of classifications made by different aspects for different insights.

  (AI) Platform—A system that allows interaction with the AI algorithm and sets up, scores and modifies the AI algorithm as needed. The platform also includes code, databases, and servers that are used for this particular purpose.

  Token-one or more words used as a single unit to be associated with a category by assigning weights.

  Training set—a set of classified documents used to calculate a knowledge set.

  Weight-A numerical value assigned to a token or document for a category based on training for a particular algorithm (training, learning).

  Word-a combination of characters used to indicate meaning in a language.

Variableization-grouping words or sets of words into sings. Example: “Alex”, “Sarah”, and “Jill” can all be variableized into token names.
II. Dynamic messaging system

  For ease of discussion, FIG. 1 is an example of a logical diagram of a system for generating and implementing a messaging campaign, generally designated 100. In this example of the block diagram, a number of users 102a-n participating in the dynamic messaging system 108 via the network 106 are shown. Note that in some embodiments, the messaging campaign may be uniquely customized by each user 102a-n. In an alternative embodiment, the user may be part of a co-marketing department (or other joint group) and everyone may have common access to messaging campaigns. Users 102a-n can access the network from any number of suitable devices such as laptops and desktop computers, workstations, mobile devices, media centers and the like.

  Network 106 most commonly includes the Internet, but may include other networks such as, for example, a corporate WAN, a cellular network, a corporate local area network, or a combination thereof. The messaging server 108 may distribute the generated message to various message delivery platforms 112 for delivery to individual recipients. Message delivery platform 112 may include any suitable messaging platform. Most of the present disclosure focuses on email messaging, so in such embodiments, the message delivery platform 112 may include an email server (eg, gmail, Yahoo, hotmail, etc.). However, it should be appreciated that the system for messaging disclosed herein is not necessarily limited to email messaging. Indeed, under some embodiments of this messaging system, any messaging type can be envisaged. Accordingly, the message delivery platform 112 can easily include a social network interface, an instant messaging system, a text messaging (SMS) platform, or even an audio communication system. Although voice can be envisioned for a given messaging system, it is often desirable for a recipient to have a seamless experience in which automated messages are substantially indistinguishable from messages created by salespeople. Due to the inherent difficulty of generating automated speech that is realistically audible to humans (much less likely to mimic a particular salesperson), much of this disclosure is related to the generation of written text messages. match the focal point.

  One or more data sources 110 may be available to the messaging server 108 to provide user specific information, message template data, knowledge sets, insights, and lead information. These information types are described in more detail below.

  Proceeding further, FIG. 2 provides a more detailed view of the dynamic messaging server 108 according to some embodiments. The server consists of three main logical subsystems: a user interface 210, a message generator 220, and a message response system 230. User interface 210 may be utilized to access message generator 220 and message response system 230 to set up messaging campaigns and manage those campaigns throughout their lifecycle. At a minimum, user interface 210 includes APIs to allow user devices to access these subsystems. Alternatively, user interface 210 may include web-accessible messaging creation and messaging management tools, as discussed below in some of the accompanying screenshot examples.

  FIG. 3 provides a more detailed illustration of the user interface 210. User interface 210 includes a series of modules to enable message generator 220 and message response system 230 to perform the functions described above. These modules include a campaign builder 310 (campaign construction means), a campaign manager 320, an AI manager 330, an insight manager 340, and a knowledge base manager 350.

  The campaign builder 310 allows the user to define a campaign and allows the user to enter a message template for each series in the campaign. Knowledge sets and leader data may be associated with the campaigns so that the system can automatically achieve them when the campaigns are built. Leader data includes all information collected about the intended recipient, and the knowledge set includes a database from which AI can infer context and classify with respect to responses received from the recipient. Can be performed.

  The campaign manager 320 provides campaign activity information, status, and logs when the campaign is executed. Accordingly, the user 102a can keep track of the progress of the campaign and the progress of success, and the user can manually work as necessary. The campaign may be edited or otherwise modified using the campaign manager 320 as well.

  The AI manager 330 allows the user access to artificial intelligence training that analyzes responses received from recipients. One purpose of a given system and method is to allow very high throughput of message exchanges with recipients with relatively minimal user input. In order to do this correctly, natural language processing by AI is required, and AI must be trained correctly to make a proper guess and classification of response messages. The user may use the AI manager 330 to review documents that have been processed and classified by the AI.

  Insight manager 340 allows users to manage insights. As already discussed, insight is a group of categories used to answer some question about a document. For example, a question about a document may include “Does the leader want to buy a car next month?” Answering this question can be of direct and great importance to car sales agents. The specific categories that the AI system generates may be relevant for the determination of this question. These categories are “insights” to the question and may be edited or newly created by the insight manager 340.

  In a similar manner, knowledge base manager 350 allows a user to manage a knowledge set. As previously mentioned, a knowledge set is a set of tokens whose associated category weights are used by the aspect (AI algorithm) during classification. For example, the category may include “continue contact?” And the associated knowledge set token can include sentences such as “stop”, “do not contact”, “respond”, etc. . The knowledge base manager 350 allows the user to build a new knowledge set or edit an existing knowledge set.

  Proceeding to FIG. 4, an example of a logic diagram of the message generator 220 is provided. Message generator 220 utilizes context knowledge 440 and leader data 450 to generate an initial message. Message generator 220 includes a rule builder 410 that allows a user to define rules for messages. The rule creation interface allows a user to define variables to change data in a specific way after checking in a given situation. For example, when receiving a score from AI, if the insight is interpretation and the selected category is “good”, then the continuation messaging insight returns to “continuation”.

  Rule builder 410 may give the message a phrase that can be assumed based on the available leader data. Message builder 420 incorporates those possible phrases into a message template in which variables are specified to generate an outgoing message. This is provided to the message sender 430, which formats the outgoing message and provides it to the messaging platform for delivery to the appropriate recipients.

  FIG. 5 is an example of a logic diagram of the message response system 230. In this example of the system, contextual knowledge base 440 is utilized in combination with response data 550 received from the leader. Message receiver 520 receives response data 550 and provides it to AI interface and target modeler 530 for feedback. The AI interface 510 allows the AI platform to process responses for context, insight, sentiment, and associated confidence scores. Objectives may be updated by objective modeler 530 based on the classification generated by the AI leader.

Thereafter, the message receiver 520 can determine whether there are still unresolved further objectives or whether there has been a request to interrupt the message to the leader. The message receiver may deactivate a campaign for a given leader if there is a termination request or if all objectives have been met. Otherwise, a scheduler 540 may be used to help schedule the next step of the campaign.
III. Messaging method

  Having generally described a system for dynamic messaging campaigns, attention is now directed to the process used to generate and deliver customized media. In FIG. 6, an example of a flow diagram for a dynamic message campaign, indicated generally at 600, is provided. This process can be roughly divided into three parts: user onboarding (610), campaign generation (620), and campaign implementation (630). The following figures and related disclosures delve deeper into the details of these predetermined process steps.

  For example, FIG. 7 provides a more detailed discussion of the onboarding process indicated generally at 610. Initially, the user is given a set of authentication certificates (or the user generates a set of authentication certificates) (710). This allows subsequent authentication of the user by any known authentication method. This may include a username and password combination, biometric identification, device certificate, and the like.

  Next, the leader data associated with the user is imported or otherwise aggregated (720) to provide the system with a leader database for message generation. Similarly, contextual knowledge data may be added (730) so that it is relevant to the user. In many cases, there is a general knowledge data set that can be automatically associated with a new user, but it is sometimes desirable to have a knowledge set specific to a user's campaign that is not generally applicable. These more specialized knowledge sets may be imported or added directly by the user.

  Finally, users can configure their preferences and settings (740). This is as simple as selecting a dashboard layout and setting the required confidence threshold before alerting the user for manual intervention.

  Next, FIG. 8 is an example of a flow diagram for the process of building a campaign indicated generally at 620. The user starts a new campaign (810) by first describing the campaign. The campaign description includes a campaign name, description, industry type selection, and service type. Industry choices and service types may be utilized to ensure that the appropriate knowledge set is relied upon for response analysis.

  After the campaign is described, a message template in the campaign is generated (820). If a series is added (830), the campaign is reviewed and submitted (840). Otherwise, the next message in the template is generated (820). FIG. 9 provides further details of an example of this sub-process for generating a message template. First, the user is queried as to whether an existing campaign is available for the template or if a new template is desired.

If existing campaigns are used, new message templates are generated (920) by adding existing templates to these templates. The user is then given the opportunity to change the message template to better reflect the new campaign (930). Because the objectives of many campaigns can be similar, users tend to generate a library of campaigns that can be reused with or without modification in some situations. If that is possible, there is an advantage in saving time by reusing campaigns.

  However, if there is no appropriate campaign to be utilized, the user may instead choose to write the message template from scratch (940). When a message template is generated, most of the message is written by the user and variables are imported with respect to the area of the message that changes based on the leader data. Successful messages are designed to elicit responses that are easily classified. With higher classification accuracy, the system can operate longer without user intervention, thereby increasing campaign efficiency and increasing user workload.

  Once the campaign is built, it is ready to run the campaign. FIG. 10 is an example of a flow diagram of a process for implementing a campaign indicated generally at 630. Here, the leader data is uploaded (1010). Leader data may include any number of data types, but generally includes a leader name, contact information, contact date, items that the leader is interested in, and the like. Other data includes public comments provided by the leader to the leader provider, any items that the leader may have to deal with, and the date that the leader entered the leader provider's system. be able to. In many cases, the leader data is industry specific and individual users may have their own data that can be used.

  An appropriate delay period may elapse (1020) before the message is created and sent (1030). The waiting period is important so that the leader does not feel excessive stress and the user does not look too enthusiastic. In addition, this delay more accurately mimics human responses (not instant automatic messages).

  FIG. 11 gives a more detailed example of message creation and output. In this example of the flow diagram, a message in the series is selected based on which purpose is outstanding (1110). In general, the messages are given in a set order, but other messages may be more appropriate if the goals for a particular leader have already been met for a given series. Similarly, if the recipient did not respond as expected or did not respond at all, it may be desirable to have an alternative message template to treat the leader most effectively.

  After a message template is selected from the series, the leader data is parsed and a match in a variable field in the message template is added (1120). The added message is output 1130 to an appropriate messaging platform, which generally includes an email service as described above, but may include SMS services, instant messaging, social networks, and the like.

  Returning to FIG. 10, after the message has been output, the process waits for a response (1040). If no response is received (1050), the process determines whether the latency has timed out (1060). Allowing leaders to be left too long can result in missed opportunities, but frequent and persistent plaguing of leaders can adversely affect relationships. Therefore, the user may define this timeout period. In many cases, the timeout period varies from a few days to over a week. If there is no timeout event, the system continues to wait for a response (1050). However, it may be desirable to return to the delay period (1020) and send a follow-up message (1030) when sufficient time has passed without a response. In many cases, a reminder template designed for just such a situation is available.

  However, if a response is received, the process may continue (1070) while the response is being processed. This processing of responses is described in further detail in connection with FIG. In this sub-process, a response is first received (1210) and the document is cleaned (1220). Document cleaning may include a normalization process in which characters and tokens are removed to reduce document complexity without changing the intended classification. There are many steps in document cleaning. When first receiving a response, often the original message, HTML encoding for HTML-formatted responses, Enforced UTF-8 encoding to get additional symbols and other notations from other languages, and not to confuse AI Many elements need to be removed, including the signature for Only after all of this removal process has been done is the normalization process performed, including variable costs, stopword deletion, manual replacement, spelling correction, and punctuation, numbers, and unnecessary. Includes removal of other possible tokens.

  The normalized document is then fed to the AI platform for classification using the knowledge set (1230). As mentioned above, some examples are used to classify a given document, including hard rules, naive bayes, sentiment, neural nets, k-nearest neighbors, other vector-based algorithms, etc. There are many known algorithms that may be present. In some embodiments, multiple algorithms may be used simultaneously, after which a combination of algorithm results is used to perform the classification. The selected algorithm may be the algorithm with the highest confidence level in their classification or the algorithm that most closely agrees with each other. Responses to information messages may be categorized differently than responses to questions. The classification depends on the type of response received with each outgoing message. Classification may be combined with business logic in the objective model rule engine to generate an action set. Campaign objectives may be used to redefine actions that are collected and scheduled as they are updated. For example, the “Skip to Follow Up” action may be replaced with an information message that introduces the salesperson before proceeding to the “Series 3” purpose. In addition, the “not send email” or “stop message” classification should deactivate the leader at any time during the leader's lifecycle and remove the schedule.

  After the action is set, a determination is made whether there is an action conflict (1250). If such a conflict exists, a manual review may be required (1270). Otherwise, the action may be performed by the system (1260).

  Returning to FIG. 10, after the response has been processed, a determination is made whether to deactivate the leader (1075). Such deactivation may be determined as needed when the leader requests it. If so, the leader is deactivated (1090). Otherwise, the process continues by determining whether the campaign for a given leader has been completed (1080). A campaign may be completed when all objectives for a leader have been met, or when there are no longer messages in the series that can be applied to a given leader. When the campaign is completed, the leader may be deactivated as well (1090).

However, if the campaign is not yet complete, the process may return to the delay period before creating and sending (1030) the next message in the series (1020). This process is repeated in this manner until the leader requests deactivation or until all objectives are met.
IV. Example

  The following example includes a screenshot of an interface for building and managing a messaging campaign. Although a significant number of screenshot examples are given for this sales-driven example, note that the disclosed system and method for dynamic messaging is applicable for many purposes other than sales and marketing. Should. For example, educators can benefit greatly from such messaging capabilities. In addition, customer services, helplines, and information services can benefit greatly from the disclosed messaging campaign systems and methods.

  The following example also places considerable emphasis on email messaging. Email messaging can be particularly useful as a communication tool, but messages to be generated can be audio, video and animation, text messages, instant messages, forum posts, messaging within social media platforms (sending and receiving messages), or It can be entirely envisaged that any combination may be included. Thus, it is extremely important that the following examples clarify the messaging campaign system and method without unduly limiting their scope.

  FIGS. 13-16 are example screen shots of an interface for building a campaign according to some embodiments. In FIG. 13, indicated generally at 1300, the user is given a list of existing campaigns and the ability to create new campaigns. If selected, the user is redirected to the screen shown in FIG. Here, shown generally at 1400, the user is asked to name the campaign, have a dashboard identifier for the campaign, and enter a description for the campaign. As previously mentioned, the systems and methods disclosed herein allow a user to have a very large number of campaigns running simultaneously. This description allows the user to organize and keep the various campaigns organized and clear.

  In addition to providing specifications and descriptions for the campaign, the user can similarly select the industry and service type for a given campaign. These choices allow the appropriate knowledge set to be associated with the campaign, thereby allowing AI to more accurately classify any response. Knowledge sets are also used to gain momentum in marketing service sales and billing, so that customers can be charged accurately for the campaigns they are running.

  In FIG. 15, the next step in campaign generation is message template formation. This can be done from scratch or an existing campaign may be used to generate these messages. This screenshot 1500 gives the options presented to the user.

Next, as shown at 1600 in FIG. 16, a message series is provided to the user. Here, the series may be selected from a set of pull-down options. The series generally includes short questions to help guide responses. Each series is generally directed towards those who meet the goals for the leader. Examples of series and purposes are given in the examples in the table below.

  The delay in the series, and the message subject and message body may be entered. If appropriate, the user can incorporate variables into the message. These variables may be defined by the user or automatically added by the system using the leader data. To construct a message, possible phrases are collected for each template component in the template iteration. A single phrase can be randomly selected from the possible phrases for each template component. The selected phrase is then imported to obtain the outgoing message. The logic can be generic or data specific as desired for individual message components.

  Each series may include a number of message templates corresponding to multiple attempts to meet the objective. Thus, for example, if the leader fails to respond to the first message, a different subsequent message may be sent that attempts to answer the purpose.

  Variable substitution can be performed on a phrase basis or after the message is constructed. Post-message construction verification may be incorporated into the message construction class. All rule interactions may be maintained using a messaging rules engine.

  Now that we ’ve looked at the campaign building in great detail, let's focus on the management tools that are available to users. This allows AI management, knowledge set management, insight management, and review of campaign statistics. FIG. 17 provides a top level dashboard for AI management, indicated generally at 1700. At this top level, the user is allowed to select AI training or review of AI document classification. All tools are located under tabs on the left side of the screen.

  FIG. 18 shows an AI training dashboard, generally designated 1800. The training tool allows an AI developer who may be a user or include a messaging server administrator to train an AI algorithm. This tool is in the middle of how AI is taught new things. The top left of the frame contains the tool title and below it is a context drop down filter, which allows document filtering to be trained by document context. Below that is the previous message button, which redirects to the original previous document trained. On the right is a next message button that goes to the next oldest document where some training is available. The determination of which documents need training is determined by how confident the classification for each insight performed by the AI is.

  FIG. 19, generally designated 1900, illustrates the use of a search bar, which can search for a document with a given document ID or search for the length of text that may be present in the body of the document. To. FIG. 20 shows a score button, which displays the current AI classification in the document generally designated 2000. This score is divided by the insights (shaded lines) and the categories selected for those insights. The right side of the category container has a numerical value that is a confidence value in the selected category. The confidence value is the degree of confidence of AI in selecting that category for that insight. In this embodiment, the range of belief is 0 to 100 (100 is the most certain).

  The center of the frame includes the unique document ID, the context assigned to the document, and the document body. On the right side of the center frame, there is a link called “Display Normalized Document”. This indicates what AI will do to the document with respect to cleaning, parsing, and other preprocessing techniques before AI actually begins scoring.

  By highlighting a part of the document text, a category selection box is displayed, as can be seen at 2100 in FIG. The category selection box allows manual categorization of the highlighted text. After selecting a category, the selected phrase and category appear below the document, as can be seen at 2200 in FIG. To the right of these new containers is a knowledge set dropdown. To train the AI, the AI developer selects a knowledge set that is stored in the new training. This associates a document with a knowledge set not previously used by the AI for context generation.

  Proceeding further, FIG. 23 provides a dashboard for reviewing AI training, indicated generally at 2300. The review tool is used to review documents that AI was convinced of regarding one or more of the classification decisions. Both context and insight may be filtered over when reviewing the document by selecting the appropriate drop down located in the upper left of the frame. For each confident category selected for the document, the AI developer can select a “thumbs up” button (to accept the AI classification) or a “thumbs down” (to disagree with the AI classification). It may be selected whether AI classification is approved by pressing any of the buttons.

  When a category is marked as accurate or inaccurate, its category column is highlighted, as shown at 2400 in FIG. Once all confident categories in a document have been marked, the document may be removed from the display and the next document loaded. Then there is a submit button that allows the AI developer to submit any training that has been selected.

  Proceeding to FIG. 25, the API management tab includes tools for handling object creation and management, indicated generally at 2500. The context tool shown at 2600 in FIG. 26 is useful for context management. Upon initial loading of the system to a new AI developer / user, a list of existing contexts may be displayed. Similar to AI training, context can be searched using the search bar located in the upper right part of the screen. By pressing the “+” button in the upper right corner or frame, a creation dialog box is launched, as can be seen at 2700 in FIG. This allows the creation of new contexts.

  FIG. 28 provides details of the context indicated generally at 2800. A detailed display in any given context may be viewed by selecting any one of the contexts. The detailed display includes all related information regarding the context and an edit button (wrench icon) located at the upper right of the detailed display. When the edit button is selected, an edit dialog box is launched, as can be seen at 2900 in FIG. The edit dialog box allows editing information about the context.

  Proceeding to FIG. 30, an insight management screen shot generally indicated at 3000 is provided. Insight tools can help you manage your insights. As with the context, a list of existing insights may be displayed on the first load. Similarly, insights may be searched using a search bar located in the upper right of the frame.

  As can be seen at 3100 in FIG. 31, the create insight dialog box may be accessed by pressing the “+” button in the upper right corner or frame. In order to create an insight, it is also necessary to create a category associated with the insight. Selecting any one of the insights opens a detailed display for that insight, as can be seen at 3200 in FIG. The detailed display includes all relevant information about insight and includes an edit button (wrench icon) located in the upper right of the frame. In some embodiments, each insight has at least two categories. Clicking on the edit button results in an edit dialog allowing the editing of information for insight, as can be seen at 3300 in FIG.

  Proceeding to FIG. 34, an interface for knowledge set management, generally designated 3400, is provided. As with context and insight, a list of existing knowledge sets is displayed on the first load. Similarly, the knowledge set may be searched using a search bar located in the upper right of the frame. By pressing the “+” button in the upper right corner or frame, a knowledge set creation dialog box may be launched, as can be seen at 3500 in FIG. This allows the creation of a new knowledge set. In some embodiments, the knowledge set requires that an insight exists so that a newly created knowledge set can be tied to that insight.

  Selecting any one of the insights opens a detailed display in that knowledge set, as can be seen at 3600 in FIG. The detailed display includes all related information related to the knowledge set and an edit button (wrench icon) located at the upper right of the detailed display. Clicking on the edit button results in an edit dialog allowing editing of information about the knowledge set, as can be seen at 3700 in FIG.

  Proceeding to FIG. 38, an action management dashboard generally shown at 3800 is shown. One of the tools below the action dashboard is the resolution tool seen at 3900 in FIG. The resolution tool allows for manual intervention on any unresolved response that was deemed “uncertain” by the AI classification. This confident / unconfident decision is not only based on the confidence score returned from the AI, but also additional factors (for example, the client may be new, the campaign may be new, or for a new industry May not be included). Manual intervention in an uncertain decision allows a user, system administrator, AI developer, or other appropriate individual to access the document and enter actions appropriate to the leader's response. Depending on the embodiment, this manual intervention does not train AI directly. The system tracks what action was taken in each response, and this collected data may then be used for analysis for AI improvement.

  FIG. 40, generally designated 4000, gives the leader history from when the leader was entered into the system until the leader's campaign was completed. This history gives a list of all messages sent to a given leader.

  Proceeding to FIG. 41, actions taken with respect to a given campaign generally seen at 4100 may be reviewed. The action review tool is laid out in substantially the same manner as the action resolution tool described above. However, the action review tool does two things. That is, the action review tool allows 1) reviewing responses for actions that were deemed confident and did not require manual review, and 2) reviewing responses that were manually resolved by other users . This tool may be used by higher level users (such as AI developers) to collect accuracy data about other users, and also allows users to check the accuracy of automated actions to be performed by the system. To.

  Proceeding to FIG. 42, a statistical dashboard generally designated 4200 is illustrated. The statistics tab contains several analysis and reporting tools. This tool collects data from resolution actions, reviews actions, and reviews accuracy tools. The AI decision tool reports on some AI specific statistics, as can be seen at 4300 in FIG. In this particular example diagram, each graph spans a 7 day window. The first graph shows how accurately the system is on context by context criteria. The following graph shows the same based on insight. The last group of statistics shows how many responses need to be resolved manually by the user.

Finally, FIGS. 44 and 45 illustrate action statistics generally indicated at 4400 and 4500, respectively. Each box contains statistics about the actions taken and includes, for example, training by the user, action accuracy, action resolution summary, recommended action accuracy, and the like.
V. System embodiment

  46A and 46B illustrate a computer system 4600 that is suitable for practicing embodiments of the present invention. FIG. 46A illustrates one possible physical form of computer system 4600. Of course, the computer system 4600 may have many physical forms ranging from printed circuit boards, integrated circuits, or small handheld devices to large supercomputers. Computer system 4600 may include a monitor 4602, a display 4604, a housing 4606, a disk drive 4608, a keyboard 4610, and a mouse 4612. Disk 4614 is a computer readable medium used to transfer data to and from computer system 4600.

  FIG. 46B is an example of a block diagram for a computer system 4600. A wide variety of subsystems are attached to the system bus 4620. A processor 4622 (also referred to as a central processing unit, or CPU) is coupled to a storage device that includes a memory 4624. The memory 4624 includes random access memory (RAM) and read only memory (ROM). As is well known in the art, ROM acts to transfer data and instructions to the CPU in one direction, and RAM is commonly used to transfer data and instructions in a bidirectional manner. The Any of these types of memory may include any suitable form of computer readable media described below. Fixed disk 4626 may be coupled bi-directionally to processor 4622. That is, the fixed disk provides additional data storage capacity and may include any of the computer-readable media described below. Fixed disk 4626 may be used to store programs, data, and the like, and is generally a secondary storage medium (such as a hard disk) that is slower than the primary storage device. It will be appreciated that, where appropriate, information held in fixed disk 4626 may be incorporated in memory 4624 in a standard manner as virtual memory. Removable disk 4614 may take the form of any of the computer-readable media described below.

  The processor 4622 is also coupled to various input / output devices such as a display 4604, a keyboard 4610, a mouse 4612, and a speaker 4630. In general, input / output devices include video displays, trackballs, mice, keyboards, microphones, touch-sensitive displays, transducer card readers, magnetic or paper tape readers, tablets, styluses, voice or handwriting recognition devices, biometric readers, motion sensors, Either an electroencephalogram reader or another computer may be used. Processor 4622 may optionally be coupled to another computer or telecommunications network using network interface 4640. With such a network interface 4640, it is believed that the processor 4622 can receive information from the network or output information to the network in the process of performing the dynamic messaging described above. Furthermore, the method embodiments of the present invention may be performed solely by the processor 4622 or may be performed over a network, such as the Internet, with a remote CPU that shares part of the processing.

  In summary, the present invention provides a system and method for dynamic automated messaging driven by artificial intelligence. An advantage of such a system is the ability to provide a person-initiated electronic mail exchange without the required manual input. Such a system may be particularly useful in the context of sales and marketing, but may be used wherever email mass delivery is used.

  While this invention has been described with respect to several embodiments, there are alterations, modifications, substitutions, and alternative equivalents that fall within the scope of this invention. Although subsection titles have been given to aid in the description of the invention, these titles are merely illustrative and are not intended to limit the scope of the invention.

  It should also be noted that there are many alternative ways of implementing the method and apparatus of the present invention. Accordingly, the following appended claims are to be construed to include all such alterations, modifications, substitutions and alternative equivalents that fall within the true spirit and scope of the invention. It is intended to be

Claims (30)

A method for managing automated messaging campaigns in a computer-processed messaging system comprising:
Adding a data store with at least one knowledge set and at least one leader data set in response to user input;
Providing a campaign builder to the user, the campaign builder including a series of message templates having variable fields, the variable field further comprising: at least one of the at least one knowledge set and the at least one leader data set; Corresponding to a class of data from at least one of
Receiving input from the user to initiate a campaign via the campaign builder;
Receiving input from the user when the confidence for categorization of responses is below a threshold;
A method comprising: The campaign is automatically generated by adding the variable field in the message template with suitable data from at least one of the at least one knowledge set and the at least one leader data set. The method of claim 1 comprising a series of messages. The method of claim 2, wherein the response is a message sent from a leader in response to a message generated as part of the campaign. The method of claim 3, further comprising providing an artificial intelligence training tool to the user, wherein the artificial intelligence training tool is used to adjust an algorithm used to categorize the response. The method of claim 2, wherein the message is text. The method of claim 5, wherein the message is an email. The method of claim 4, further comprising providing an insight management tool to the user, the insight management tool allowing the user to review and update insight generated by the algorithm. The method of claim 1, further comprising providing a knowledge base management tool to the user, the knowledge base management tool allowing the user to create and update a knowledge set. Give the user an action interface for resolving actions when the next action in the campaign is unclear due to at least one of confidence below the threshold and conflict with the rules of the action The method of claim 1, further comprising a step. The method of claim 1, further comprising providing a statistical interface to the user, wherein the statistical interface provides user information regarding the campaign status. A method for processing a message exchange useful in connection with an artificial intelligence system in a computer-processed messaging system comprising:
Generating a first message by adding a variable field in the first message template with corresponding data from at least one of a knowledge set and a leader data set;
Receiving a response to the first message from a leader;
Categorizing the response using at least one artificial intelligence algorithm;
Generating a confidence value for the categorization;
Determining an action based on the categorization and the confidence value;
A method comprising: The method of claim 11, wherein the first message is a text message. The method of claim 11, wherein the first message is a message in a series of messages. The action includes at least one of asking for user input, proceeding to a second message in the series of messages, suspending messaging, and generating a follow-up message. The method described in 1. The method of claim 14, wherein each message in the series of messages has a purpose. 16. The method of claim 15, wherein the action proceeds to the second message if the response meets the purpose for the first message. The method of claim 14, wherein the action seeks user input if the confidence value is less than a threshold value. The method of claim 15, wherein the action generates a follow-up message if no response is received or if the purpose of the first message is not met. The at least one artificial intelligence algorithm compares n-grams in the response with the knowledge set, each n-gram being associated with at least one category having a confidence level and sufficient n associated with the category. The method of claim 11, wherein the presence of a gram strongly results in categorization, and how strongly the n-gram corresponds to the category determines the confidence value. The at least one artificial intelligence algorithm compares the n-gram in the response with a list of items overwhelmingly associated with a particular category, and if such an item is present in the n-gram, the item The method of claim 11, categorizing responses for the category associated with the. A method for setting a useful knowledge set and AI algorithm in connection with an automated messaging system in a computer-processed knowledge learning system comprising:
Receiving at least one training message;
Selecting a subsection of text from the at least one training message;
Selecting a knowledge set from a plurality of knowledge sets for the selected subsection of the text, each knowledge set including a probabilistic association between an item and a category;
Selecting an insight from a plurality of insights for the selected subsection of the text based on the relevance of items in the subsection of the text given the selected knowledge set;
Categorizing the training messages based on the insights;
Receiving one of approval or rejection of the categorization;
Updating the probability of relevance in response to a received approval or rejection;
A method comprising: The method of claim 21, wherein the training message is a text message. The method of claim 21, wherein the categorization includes a confidence value, and the confidence value is based on the probability of relevance. 24. The method of claim 23, wherein the training message is selected based on a low confidence value. The method of claim 21, further comprising selecting a context from a plurality of contexts for the training message, wherein each context is a group of documents having commonality. The method of claim 21, further comprising generating a new insight within the plurality of insights. 26. The method of claim 25, further comprising generating a new context within the plurality of contexts. The method of claim 21, further comprising generating a new knowledge set within the plurality of knowledge sets. 29. The method of claim 28, wherein each knowledge set is associated with at least one insight. 30. The method of claim 29, further comprising editing the probabilistic association in the at least one knowledge set.

Images