WO2019231405A1 - System and method for generating notifications related to user queries - Google Patents

System and method for generating notifications related to user queries Download PDF

Info

Publication number
WO2019231405A1
WO2019231405A1 PCT/SG2019/050285 SG2019050285W WO2019231405A1 WO 2019231405 A1 WO2019231405 A1 WO 2019231405A1 SG 2019050285 W SG2019050285 W SG 2019050285W WO 2019231405 A1 WO2019231405 A1 WO 2019231405A1
Authority
WO
WIPO (PCT)
Prior art keywords
queries
user
gesture
meaning
identified
Prior art date
Application number
PCT/SG2019/050285
Other languages
French (fr)
Inventor
Vivekkumar KAPADIA
Original Assignee
Kaha Pte. Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kaha Pte. Ltd. filed Critical Kaha Pte. Ltd.
Publication of WO2019231405A1 publication Critical patent/WO2019231405A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/254Fusion techniques of classification results, e.g. of results related to same input data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/30Semantic analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038Indexing scheme relating to G06F3/038
    • G06F2203/0381Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems

Definitions

  • the subject matter in general relates to information technology. More particularly, but not exclusively, the subject matter relates to a system and a method for notifying a user.
  • a system for generating notifications relating to user queries comprises a query input module configured to receive one or more queries associated with a user by verbal communication means and/or gesture means.
  • the system further comprises a natural language processing (NLP) engine configured to determine a meaning of the one or more queries (MOQ) received from verbal communication means.
  • the system further comprises a gesture engine configured to determine the meaning of the one or more queries (MOQ) received from gesture means.
  • the system further comprises a machine learning unit which receives the meaning of the one or more queries (MOQ) determined from at least one of the NLP and the gesture engine.
  • the machine learning unit may be configured to identify at least one data source, among the one or more data sources, required to resolve the one or more queries.
  • the machine learning unit may be configured to identify one or more trigger points based on the meaning of the query (MOQ).
  • the machine learning unit is configured to determine a logical relationship between one or more identified trigger points.
  • the system further comprises an execution unit in communication with the machine learning unit.
  • the execution unit may be configured to collect contextual data from the identified data sources, process incoming data, generate one or more trigger points pertaining to the associated data sources and evaluate logical relationship between the one or more trigger points to generate one or more events upon successful logical relationships.
  • the execution unit may be configured to communicate one or more events to an electronic device associated with the user.
  • the electronic device is configured to convert the events to appropriate user notifications.
  • a method for generating notifications relating to user queries comprises identifying at least one data source pertaining to the meaning of the query (MOQ).
  • the method further comprises of identifying one or more trigger points (TPs) based on the at least one identified data source.
  • the method further comprises determining a logical relationship (R) between the one or more trigger points identified.
  • the method further comprises collecting contextual data from the identified data sources.
  • the method further comprises processing of incoming data and generating one ormore trigger points pertaining to the associated data sources and identified trigger points.
  • the method further comprises evaluating the logical relationship between one or more trigger points to generate at least one event.
  • the method comprises generating one or more events pertaining to the one or more successful logical relationships.
  • the method further comprises delivering the events to an electronic device and converting the subject event into appropriate user notification.
  • FIG. 1 is an exemplary architecture of a system 100 illustrates the processing of user queries, in accordance with an embodiment.
  • FIG. 2 is an exemplary block diagram illustrating an execution unit 200 which receives basic configuration settings (one time) from the machine learning unit 116 and generates notifications, in accordance with an embodiment.
  • FIG. 3 is an exemplary architecture of a system 300 illustrating co-ordination between modules of the machine learning unit 116 and execution unit 200, in an embodiment of the present invention.
  • FIG. 4 illustrates an exemplary system architecture 400 of the system for generating events and notifications, in accordance with an embodiment.
  • FIG. 5 illustrates an exemplary method 500 illustrating a flowchart for receiving a user query, processing the query, identifying one or more data sources relating to meaning of the query (MOQ), identifying one or more trigger points relating to the identified data sources, and determining a logical relationship with respect to identified trigger points, in accordance with an embodiment.
  • FIG. 6 illustrates an exemplary method 600 illustrating a flowchart for supplying identified data to the execution unit from the machine learning unit, in accordance with an embodiment.
  • FIG. 7 illustrates an exemplary method 700 illustrating a flowchart for evaluating one or more trigger points, in accordance with an embodiment.
  • the terms“a” or“an” are used, as is common in patent documents, to include one or more than one.
  • the term“or” is used to refer to a non exclusive“or,” such that“A or B” includes“A but not B,”“B but not A,” and“A and B,” unless otherwise indicated.
  • FIG. 1 is an exemplary architecture of a system 100 illustrates the processing of user queries, in accordance with an embodiment.
  • the system 100 may include a query input module 102, an NLP engine 104, a voice to text converter 106, a command meaning unit 108, a gesture engine 110, a pattern recognition engine 112, a gesture meaning unit 114, a machine learning unit 116, a network 118, and an electronic device 120.
  • the NLP engine 104 may comprise the voice to text converter 106 and the command meaning unit 108.
  • the gesture engine 110 may comprise the pattern recognition engine 112 and the gesture meaning unit 114. Each component is described in further detail below.
  • the user inputs in the form of‘Queries’ are given to the system 100 through the electronic device 120 (for example, a smart wearable). Further, the user inputs one or more queries using verbal communication means and gesture means.
  • Verbal communication relates to inputs in the form of speech.
  • Gesture communication relates to inputs in the form of user actions.
  • the electronic device 120 may capture the subject query (inputted either verbally and/or through gesture means) and sending the query to the query input module 102 via a communication network 118.
  • the query input module 102 may be configured to receive the one or more queries received from the user through verbal means, through gesture means, or through a combination of both verbal means and gesture means. If the input may be a combination of both verbal means and gesture means, then the input is first split into both verbal means and gesture means, and the verbal means is fed to the NLP engine 104 and the gesture means is fed to the gesture engine 110. The NLP engine 104 and the gesture engine 110 both integrate to determine the meaning of the command/queries (MOQ).
  • the NLP engine 104 may be configured to determine a meaning of the one or more queries (MOQ) of the user, pertaining to verbal communication means.
  • the user may input the sounds for verbal communication means using a microphone. Vibrations are created in the air typically when users speak. Analog waves of these vibrations created are then translated into digital data using an analog-to-digital converter (ADC). The translation occurs by sampling the wave by taking precise measurements at frequency intervals. The digitalized sound is then filtered for removing unwanted noise and segregating the sound into separate frequency bands. Segregation mainly occurs because not all persons speak at the same speed, tone, ascent, so the sound must be adjusted matching to the sound samples stored in a memory of the NLP engine 104.
  • ADC analog-to-digital converter
  • the sound waves are then divided into multiple segments, which are then matched with phonemes of the respected language through the voice to text converter 106.
  • the command meaning unit 108 may be trained to compare related phonemes with other phonemes and vast database of words, sentences, phrases etc...
  • the command meaning unit 108 determines closest phonemes and prioritizes them according to the closest meaning pertaining to the user input.
  • the user repeats a word more than once, says some word(s) louder, says some word(s) softer, and the like, while providing their queries. This may be because, as an example, a specific word may carry more importance.
  • the command meaning unit 108 then creates priorities based on importance.
  • the command meaning unit 108 considers these factors with higher importance and priority. If a user repeats a sentence one or more times, then the command meaning unit 108 also considers this factor with high importance and priority.
  • the command meaning unit 108 may be trained to distinguish importance of every word, with respect to the user’s actions and data. In one example, the command meaning unit 108 may allocate priority for every word and maintains a priority table of words, for further processing.
  • the gesture engine 110 may be configured to determine the meaning of the one or more queries of the user, pertaining to gesture means. Gestures relate to the movement of body parts, while the user may be trying to express something, by way of one or more actions. Gestures may be captured (for example by a camera) and fed to the gesture engine 110 for recognition. The gesture engine 110 may be trained to identify gestures relating to facial expressions, hand and body movement and the like.
  • the one or more gestures may be captured by the camera, and then received by the gesture engine 110.
  • the pattern recognition engine 112 compares the captured gesture images with stored images pertaining to the gesture context in a database of the gesture engine 110.
  • the database has a certain number of images corresponding to each gesture. Apart from storing images, the database also stores commands, which interpret a meaning of gestures.
  • the gesture meaning unit 114 correlates the database image with commands and pattern recognition engine 112, to determine the command matching the database image for determining the meaning of the gesture inputted by the user.
  • Support vector machine (SVM) classification, K-Nearest Neighbour (KNN) are known techniques in the art for matching the input image (gesture image here) with images stored in the database of the gesture engine 110.
  • the user may give a gesture action‘thumbs up’.
  • the camera captures one or more images of this‘thumbs up’ gesture.
  • These captured images are then fed to the gesture engine 110, which then determines the image(s) stored in its database that may depict a thumb in an upwards position, to match with the gesture image(s) captured by the camera.
  • the pattern recognition engine 112 matches the database image (thumb in upward direction here).
  • the gesture meaning unit 114 may receive the matching image determined by the pattern recognition engine 112 and further determine command(s) relating to the meaning of the thumbs up gesture.
  • Commands relating to the thumbs up gesture typically means good job, great and nice, and the like as examples.
  • the gesture meaning unit 114 determines that the meaning of the gesture command is good/nice/great.
  • Command(s) may be referred to as meaning of the query (MOQ) or processed queries.
  • the machine learning unit 116 may comprise a data source identification block 122, a trigger point definition unit 124 and a relationship definition unit 126.
  • the data source identification block 122 may provide a set of data sources identified based on relevancy of MOQ.
  • a data source is source of the data.
  • the data source can be a file, a database on a DBMS, or even a live data feed.
  • the data may be stored in a database (internal data) or may be in other remote servers (external data), accessed over a network.
  • the system 100 may collect data from various data from internal data sources (of user), that may include but not limited to sensor modules, emails, calendars, social networks, messages, app configurations, activity information (GPS), etc...
  • the external data source may be any piece of data/dataset which is available in internet (structured or unstructured) or 3rd party server, and accessible through one or more networking standards.
  • One or more data sources are required to process the given query and further. The identification of one or more data sources is crucial because, the entire output of the system depends on the appropriate selection of data sources, relevant to the context.
  • the data source identification block 122 of the machine learning unit 116 may be configured to receive the one or more MOQ from the NLP engine 104 and the gesture engine 110.
  • the data source identification block 122 may be trained to identify one or more data sources that relate to the MOQ determined based on the user’s input.
  • the data source identification block 122 can infer the closest matching data source based on MOQ, and user’s current context.
  • the objective of the data source identification block 122 is to identify relevant data source from the all available set of data sources.
  • the user query is“Do I have to carry an umbrella tomorrow?”
  • the MOQ may be,‘Is it going to rain here, tomorrow’.
  • the identified data sources may be‘weather’,‘location’ and‘time’.
  • Rain and tomorrow may be the keywords present in the MOQ, which may also be identified by the machine learning unit 116.
  • Data source‘weather’ may be in relation with rain
  • data source‘location’ may be in relation with GPS
  • data source‘time’ may be in relation with tomorrow.
  • the word umbrella relates to rain and user’s present position relates to location and time. The meaning umbrella may be carried by the user, when there is a rain tomorrow.
  • the machine learning unit 116 may prioritize these keywords and select the data sources that may be based on the closest inference of the MOQs.
  • the trigger point definition unit 124 of machine learning system 116 may be configured to identify one or more trigger points (TPs) relating to the identified data source and MOQ received. Each data source may have one or more trigger points that are continuously checked.
  • the trigger points may be time based, user based, activity based, priority based and the like.
  • the example of identified trigger points may include, Rain on 20/04/2018 (TP1), at user location (TP2), time the user leaves office (TP3).
  • the trigger points may be automatically framed by the system 100.
  • the network 118 may be a wired network, a wireless network or a combination thereof. Furthermore, the network 118 may include a wide area network (WAN) (e.g., the WAN).
  • WAN wide area network
  • the network 118 may include a peer-to-peer network.
  • the network 118 may also include a mobile data network that may include third-generation
  • the electronic device 120 may include, but not limited to, a mobile device, a smart wearable, a wrist-watch, a personal computer (PC), a tablet, a smartphone, a personal digital assistant (PDA) and the like.
  • a mobile device a smart wearable, a wrist-watch, a personal computer (PC), a tablet, a smartphone, a personal digital assistant (PDA) and the like.
  • PC personal computer
  • PDA personal digital assistant
  • FIG. 2 is an exemplary block diagram illustrating an execution unit 200 which receives basic configuration settings (one time) from the machine learning unit 116 and generates notifications, in accordance with an embodiment.
  • the execution unit 200 may include data sources 204, a trigger point generation unit 206, and a logical sequence block 208. Each component is explained in further detail below.
  • Each component of the execution unit 200 receives a configuration from other components of the machine learning unit 116 (a data source identification block 122, a trigger point definition unit 124 and a relationship definition unit 126, respectively).
  • the data source identification block 122 supplies information relating selection and identification of data sources 204.
  • the trigger point definition unit 124 may also supply identified trigger points to the trigger point generation unit 206.
  • the relationship definition unit 126 may send possible identified logical relationship data to the logical sequence block 208.
  • the data sources 204 may be configured to collect data related to one or more queries associated with the user.
  • the data sources 204 may be differentiated into internal data or external data.
  • the internal data may only have user data.
  • the user data may include but not limited to emails, messages, calendar, schedules, social network messages, data received from user sensor modules (accessible by the user), in app configurations and the like.
  • the external data may be any curated information from the internet.
  • the one or more external data sources 204 may include, but not limited to weather data, GPS data, food data, news feeds, activity data, sports data and the like.
  • the data sources 204 may be identified pertaining to the query, and the data sources 204 may be configured to fetch relevant data.
  • the one or more data sources 204 may also generate live data streams.
  • the trigger point generation unit 206 may be configured for generating one or more trigger points pertaining to the associated data sources/streams.
  • the data sources 204 may be weather data, GPS data, and activity data.
  • the user may input a query stating,“Do I have to carry an umbrella tomorrow?”, for which weather data may be identified as one of the data source 204 and a trigger point TP 1 is generated if the condition (“Rain on 20/04/2018”) is matched in the incoming weather data source.
  • the trigger point TP2 is generated if the condition (“at user location”) is matched in the incoming GPS location data source.
  • the trigger point TP3 is generated if the condition (“when the user leaves for office”) is matched in the incoming user activity data source.
  • the logical sequence block 208 may be configured to receive all the generated trigger points.
  • the logical sequence block 208 may be configured to generate events based on the relationships between the trigger points and once relationship evaluation is successful, then logical sequence block 208 may create an event (R).
  • the event may be delivered to the electronic device 120 via the network 118.
  • the event may be converted to appropriate form of user notification by the electronic device 120.
  • the trigger points TP1, TP2, and TP3 may be received by the logical sequence block 208 from the trigger point generation unit 206.
  • the logical sequence block 208 may consider (TP2 & TP3), if TP1 had occurred in the past, to create a corresponding event.
  • the notification of the corresponding event may be delivered to the user in one or more forms, not limited to, audible alarm, synthesized voice, digital display, depending on the capability of the electronic device 120.
  • FIG. 3 is an exemplary architecture of a system 300 illustrating co-ordination between modules of the machine learning unit 116 and execution unit 200, in an embodiment of the present invention.
  • the NLP engine 104, the gesture engine 110, and the blocks (data source identification block 122, the trigger point definition unit 124, the relationship definition unit 126) of the machine learning unit 116 may be one time setting units. In one embodiment, the modules of machine learning unit 116 are considered as‘one time setting units’.
  • the meaning of the user query is determined through NLP engine 104 and gesture engine 110.
  • the machine learning unit 116 identifies one or more appropriate data sources relating to the meaning of the query (MOQ), identifies one or more trigger points relating to the one or more identified data sources, and determines logical relationships between the one or more identified trigger points.
  • the NLP engine 104 and gesture engine 110 supplies the meaning of the query (MOQ) to each of the modules of machine learning unit 116.
  • the modules of machine learning unit 116 supplies identified data sources, trigger points and logical relationships (computed between the trigger points).
  • the execution unit 200 receives identified data sources, trigger points and successful logical relationships (computed between the trigger points) from the modules of machine learning unit 116.
  • the data sources 204, the trigger point generation unit 206, and the logical sequence block 208 may be continuous execution units. In other words, the modules of execution unit 200 are described as‘continuous execution units’.
  • the execution unit 200 also collects contextual data from the identified data sources, continuously process the incoming contextual data/data streams and generates one or more trigger points pertaining to one or more data sources and identified trigger points.
  • the execution unit 200 further configured to evaluate the logical relationship between one or more generated trigger points, considering the identified data sources and logic previously configured by the machine learning unit 116. When the evaluation is successful, the execution unit 200 creates one or more event(s).
  • the execution unit 200 further delivers the one or more events to the user’s electronic device 120 through the network 118.
  • the electronic device 120 appropriately converts the subject events into user notifications.
  • FIG. 4 illustrates an exemplary system architecture 400 of the system for generating notifications, in accordance with an embodiment.
  • the system may include a processor 402, a memory 406, a communication unit 404, a I/O 408, and a bus 410. Each component is described in further detail below.
  • the processor 402 may include any suitable special-purpose or general- purpose computer, computing entity, or processing device including various computer hardware or software modules and may be configured to execute instructions stored on any applicable computer-readable storage media.
  • the processor 402 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other digital or analog circuitry configured to interpret and/or to execute program instructions and/or to process data.
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA Field-Programmable Gate Array
  • the memory 406 may include computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon.
  • Such computer- readable storage media may be any available media that may be accessed by a general-purpose or special-purpose computer, such as the processor 402.
  • Such computer-readable storage media may include tangible or non-transitory computer- readable storage media including Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices), or any other storage medium which may be used to carry or store desired program code in the form of computer-executable instructions or data structures and which may be accessed by a general- purpose or special-purpose computer.
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • CD-ROM Compact Disc Read-Only Memory
  • flash memory devices e.g., solid state memory devices
  • the communication unit 404 may be a wired network or a wireless network. Furthermore, the communication unit 404 may include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), or other interconnected data paths across which multiple devices and/or entities may communicate.
  • LAN local area network
  • WAN wide area network
  • the communication unit 404 may include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), or other interconnected data paths across which multiple devices and/or entities may communicate.
  • the I/O (input/output unit) 408 may provide communication between the processor 402, the memory 406, and the communication unit 404.
  • the processor 402, the memory 406, the communication unit 404, and the I/O 408 may all be interconnected via the bus 410.
  • the bus 410 may be a communication system that transfers data/information between each component.
  • FIG. 5 illustrates an exemplary method 500 illustrating a flowchart for receiving a user query, processing the query, identifying one or more data sources relating to meaning of the query (MOQ), identifying one or more trigger points relating to the identified data sources, and determining a logical relationship with respect to identified trigger points, in accordance with an embodiment.
  • the steps 502 - 506 may be executed by the NLP engine 104 or the gesture engine 110.
  • the steps 508 - 512 may be executed by the machine learning unit 116. Each step is explained in further detail below.
  • the user inputs one or more queries to the system 100 (of FIG. 1) via the electronic device 120 (for example, a smart wearable). Further, the user may input the one or more queries using verbal communication means and gesture means.
  • Verbal communication relates to inputs in the form of speech.
  • Gesture communication relates to inputs in the form of user actions.
  • the NLP engine 104 and the gesture engine 110 may be configured to receive the one or more queries that the user provided as inputs.
  • the NLP engine 104 receives the inputs pertaining to verbal communication means and the gesture engine 110 receives the inputs pertaining to gesture means.
  • the meaning of the query is determined.
  • the NLP engine 104 may be configured to determine a meaning of the one or more queries of the user, pertaining to verbal communication means.
  • the gesture engine 110 may also be configured to determine the meaning of the one or more queries (MOQ) of the user, pertaining to gesture means.
  • the MOQ is fed to the machine learning unit 116.
  • the machine learning unit 116 may be configured to identify one or more identified data sources relating to the MOQ.
  • the machine learning unit 116 may be configured to identify one or more trigger points based on the data sources identified pertaining to the user queries. Each data source may have one or more trigger points that are continuously checked. The trigger points may be time based, user based, activity based, priority based and the like.
  • the machine learning unit 116 may be configured to determine a logical relationship between the one or more identified trigger points, to generate one or more events.
  • FIG. 6 illustrates an exemplary method 600 illustrating a flowchart for supplying identified data to the execution unit from the machine learning unit, in accordance with an embodiment.
  • Step 602 may be executed by the data source identification block 122
  • step 604 may be executed by the trigger point definition unit 124
  • step 606 may be executed by the relationship definition unit 126.
  • the data source identification block 122 may be configured to supply information relating to selection and identification of the one or more data sources to the data source block 204.
  • the trigger point definition unit 124 may be configured to supply a set of identified trigger points to the trigger point generation unit 206.
  • the relationship definition unit 126 may be configured to supply possible logical relationship(s) data to the logical sequence block 208.
  • FIG. 7 illustrates an exemplary method 700 illustrating a flowchart for evaluating one or more trigger points, in accordance with an embodiment.
  • the steps 702 - 714 may be executed by the data sources 204, the trigger point generation unit 206, and the logical sequence block 208 of the execution unit 200 (relating to FIG. 2). Each step is explained in further detail below.
  • contextual data from the one or more data sources may be collected.
  • Data may be collected from various data from internal data sources (of user), that may include but not limited to sensor modules, emails, calendars, social networks, messages, app configurations, activity information (GPS), etc.
  • the external data source may be any piece of data/dataset which is available in internet (structured or unstructured) or 3rd party server, and accessible through one or more networking standards.
  • the trigger point generation unit 206 may be configured to process the incoming data and generate one or more trigger points pertaining to the associated data sources and identified trigger points.
  • the trigger point generation unit 206 may be configured for generating one or more trigger points pertaining to associated data streams.
  • the logical sequence block 208 may be configured to evaluate one or more logical relationships between the trigger points generated.
  • the success of the logical evaluation is checked. If the logical evaluation is not successful, then step 706 repeats where the logical relationships may be re-evaluated. If the logical evaluation is successful, then at step 710, one or more events are created.
  • the one or more events may be delivered to the electronic device 120 via the network 118.
  • the events are converted into appropriate user notifications by the electronic device 120
  • the notification of the corresponding event may be delivered to the user in one or more forms, not limited to, audible alarm, synthesized voice, digital display, depending on the capability of the electronic device 120.
  • the present invention as discussed in this document with respect to different embodiments will be advantageous at least for improving receiving constant notifications pertaining to user queries.
  • the system and method are also advantageous as the system is intelligent enough to generate events pertaining to user query(s), generate notifications forthe events, and frequency of notifications automatically.
  • the system and method are also advantageous as the system can receive, collect, organize, and segregate the events. Additional advantages not listed may be understood by a person ordinary skilled in the art in view of the embodiments disclosed above.
  • numerous specific details, examples, and scenarios are explained in order to facilitate a thorough understanding of the present disclosure. However, the embodiments of the disclosure may be practiced without such specific details.

Abstract

A system and method for generating notifications relating to user queries is disclosed. The system comprises a query input module configured to receive one or more queries associated with a user. The system further comprises an NLP engine configured to determine the query meaning pertaining to verbal communication means and a gesture engine configured to determine the query meaning pertaining to gesture means. The system further comprises a machine learning unit and execution unit configured to generate one or more notifications based one events generated pertaining to one or more queries associated with the user.

Description

SYSTEM AND METHOD FOR GENERATING NOTIFICATIONS RELATED TO
USER QUERIES BACKGROUND
Field of invention:
[001] The subject matter in general relates to information technology. More particularly, but not exclusively, the subject matter relates to a system and a method for notifying a user.
Discussion of related art:
[002] With the increase in usage of computing networks, for example the internet, people are provided with an ample amount of information from various sources. Due to the advancement in technology over the last few years, people rely on the internet as a search engine where they receive results based on their inputs. Machine learning systems are such systems that are executable on computing networks and machine learning systems provide answers to user doubts.
[003] Conventional systems have been proposed that answer user doubts/inputs using techniques such as natural language processing, information retrieval, knowledge representation and reasoning, algorithms and the like. However, there may be cases where users have multiple questions and may not receive proper answers to all their queries. There are no intelligent machine learning based system exists to attend such user queries. Further, the existing rule-based systems are limited in scope, utility and cost user’s time. The conventional system could not able to appropriately learn and understand the questions asked by the user and also takes long processing time to provide a suitable answer for such user queries. Further, the conventional systems relating to query processing may not be able to generate appropriate results and does not provide relevant alerts/announcements/notices to the users for each of their questions/queries. Therefore, users will not be notified irrespective of whether each of their question is answered or not answered.
[004] Thus, the conventional systems failed to solve the above problems resulting in improper understanding of user questions and inconsistent notifications pertaining to user inputs/questions to the system, which may not be desirable.
[005] In view of the foregoing discussion, there is a need to overcome the above problems. SUMMARY
[006] Accordingly, an improved technique to overcome the above problems is needed. To fulfill this need, a system and method for generating notifications relating to user queries is disclosed.
[007] In one embodiment, a system for generating notifications relating to user queries is disclosed. The system comprises a query input module configured to receive one or more queries associated with a user by verbal communication means and/or gesture means. The system further comprises a natural language processing (NLP) engine configured to determine a meaning of the one or more queries (MOQ) received from verbal communication means. The system further comprises a gesture engine configured to determine the meaning of the one or more queries (MOQ) received from gesture means. The system further comprises a machine learning unit which receives the meaning of the one or more queries (MOQ) determined from at least one of the NLP and the gesture engine. The machine learning unit may be configured to identify at least one data source, among the one or more data sources, required to resolve the one or more queries. In addition, the machine learning unit may be configured to identify one or more trigger points based on the meaning of the query (MOQ). The machine learning unit is configured to determine a logical relationship between one or more identified trigger points. The system further comprises an execution unit in communication with the machine learning unit. The execution unit may be configured to collect contextual data from the identified data sources, process incoming data, generate one or more trigger points pertaining to the associated data sources and evaluate logical relationship between the one or more trigger points to generate one or more events upon successful logical relationships. In addition, the execution unit may be configured to communicate one or more events to an electronic device associated with the user. The electronic device is configured to convert the events to appropriate user notifications.
[008] In another embodiment, a method for generating notifications relating to user queries is disclosed. The method comprises identifying at least one data source pertaining to the meaning of the query (MOQ). The method further comprises of identifying one or more trigger points (TPs) based on the at least one identified data source. The method further comprises determining a logical relationship (R) between the one or more trigger points identified. The method further comprises collecting contextual data from the identified data sources. The method further comprises processing of incoming data and generating one ormore trigger points pertaining to the associated data sources and identified trigger points. The method further comprises evaluating the logical relationship between one or more trigger points to generate at least one event. In addition, the method comprises generating one or more events pertaining to the one or more successful logical relationships. The method further comprises delivering the events to an electronic device and converting the subject event into appropriate user notification.
[009] Other objects, features, and advantages of the present invention will become apparent from the following detailed description. It should however be understood that the detailed description and specific examples, while indicating preferred embodiments of the present invention, are given by way of illustration only and various modifications may naturally be performed without deviating from the present invention.
BRIEF DESCRIPTION OF DIAGRAMS
[010] This disclosure is illustrated by way of example and not limitation in the accompanying figures. Elements illustrated in the figures are not necessarily drawn to scale, in which like references indicate similar elements and in which:
[Oil] FIG. 1 is an exemplary architecture of a system 100 illustrates the processing of user queries, in accordance with an embodiment.
[012] FIG. 2 is an exemplary block diagram illustrating an execution unit 200 which receives basic configuration settings (one time) from the machine learning unit 116 and generates notifications, in accordance with an embodiment.
[013] FIG. 3 is an exemplary architecture of a system 300 illustrating co-ordination between modules of the machine learning unit 116 and execution unit 200, in an embodiment of the present invention.
[014] FIG. 4 illustrates an exemplary system architecture 400 of the system for generating events and notifications, in accordance with an embodiment.
[015] FIG. 5 illustrates an exemplary method 500 illustrating a flowchart for receiving a user query, processing the query, identifying one or more data sources relating to meaning of the query (MOQ), identifying one or more trigger points relating to the identified data sources, and determining a logical relationship with respect to identified trigger points, in accordance with an embodiment. [016] FIG. 6 illustrates an exemplary method 600 illustrating a flowchart for supplying identified data to the execution unit from the machine learning unit, in accordance with an embodiment.
[017] FIG. 7 illustrates an exemplary method 700 illustrating a flowchart for evaluating one or more trigger points, in accordance with an embodiment.
DETAILED DESCRIPTION OF THE INVENTION
[018] The following detailed description includes references to the accompanying drawings, which form part of the detailed description. The drawings show illustrations in accordance with example embodiments. These example embodiments are described in enough details to enable those skilled in the art to practice the present subject matter. However, it will be apparent to one of ordinary skill in the art that the present invention may be practised without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. The embodiments can be combined, other embodiments can be utilized, or structural and logical changes can be made without departing from the scope of the invention. The following detailed description is, therefore, not to be taken as a limiting sense.
[019] In this document, the terms“a” or“an” are used, as is common in patent documents, to include one or more than one. In this document, the term“or” is used to refer to a non exclusive“or,” such that“A or B” includes“A but not B,”“B but not A,” and“A and B,” unless otherwise indicated.
[020] It should be understood that the capabilities of the invention described in the present disclosure and elements shown in the figures may be implemented in various forms of hardware, firmware, software, non-transitory computer readable medium or combinations thereof.
[021] FIG. 1 is an exemplary architecture of a system 100 illustrates the processing of user queries, in accordance with an embodiment. As shown, the system 100 may include a query input module 102, an NLP engine 104, a voice to text converter 106, a command meaning unit 108, a gesture engine 110, a pattern recognition engine 112, a gesture meaning unit 114, a machine learning unit 116, a network 118, and an electronic device 120. The NLP engine 104 may comprise the voice to text converter 106 and the command meaning unit 108. The gesture engine 110 may comprise the pattern recognition engine 112 and the gesture meaning unit 114. Each component is described in further detail below.
[022] In one embodiment, the user inputs in the form of‘Queries’ are given to the system 100 through the electronic device 120 (for example, a smart wearable). Further, the user inputs one or more queries using verbal communication means and gesture means. Verbal communication relates to inputs in the form of speech. Gesture communication relates to inputs in the form of user actions. The electronic device 120 may capture the subject query (inputted either verbally and/or through gesture means) and sending the query to the query input module 102 via a communication network 118.
[023] The query input module 102 may be configured to receive the one or more queries received from the user through verbal means, through gesture means, or through a combination of both verbal means and gesture means. If the input may be a combination of both verbal means and gesture means, then the input is first split into both verbal means and gesture means, and the verbal means is fed to the NLP engine 104 and the gesture means is fed to the gesture engine 110. The NLP engine 104 and the gesture engine 110 both integrate to determine the meaning of the command/queries (MOQ).
[024] Further, the NLP engine 104 may be configured to determine a meaning of the one or more queries (MOQ) of the user, pertaining to verbal communication means. The user may input the sounds for verbal communication means using a microphone. Vibrations are created in the air typically when users speak. Analog waves of these vibrations created are then translated into digital data using an analog-to-digital converter (ADC). The translation occurs by sampling the wave by taking precise measurements at frequency intervals. The digitalized sound is then filtered for removing unwanted noise and segregating the sound into separate frequency bands. Segregation mainly occurs because not all persons speak at the same speed, tone, ascent, so the sound must be adjusted matching to the sound samples stored in a memory of the NLP engine 104.
[025] The sound waves are then divided into multiple segments, which are then matched with phonemes of the respected language through the voice to text converter 106. The command meaning unit 108 may be trained to compare related phonemes with other phonemes and vast database of words, sentences, phrases etc... The command meaning unit 108 determines closest phonemes and prioritizes them according to the closest meaning pertaining to the user input. [026] Further, there may be cases where the user repeats a word more than once, says some word(s) louder, says some word(s) softer, and the like, while providing their queries. This may be because, as an example, a specific word may carry more importance. The command meaning unit 108 then creates priorities based on importance. For example, if the user repeats a word more than one or says some word(s) with a louder tone, then the command meaning unit 108 considers these factors with higher importance and priority. If a user repeats a sentence one or more times, then the command meaning unit 108 also considers this factor with high importance and priority. The command meaning unit 108 may be trained to distinguish importance of every word, with respect to the user’s actions and data. In one example, the command meaning unit 108 may allocate priority for every word and maintains a priority table of words, for further processing.
[027] Further, the gesture engine 110 may be configured to determine the meaning of the one or more queries of the user, pertaining to gesture means. Gestures relate to the movement of body parts, while the user may be trying to express something, by way of one or more actions. Gestures may be captured (for example by a camera) and fed to the gesture engine 110 for recognition. The gesture engine 110 may be trained to identify gestures relating to facial expressions, hand and body movement and the like.
[028] The one or more gestures may be captured by the camera, and then received by the gesture engine 110. The pattern recognition engine 112 then compares the captured gesture images with stored images pertaining to the gesture context in a database of the gesture engine 110. The database has a certain number of images corresponding to each gesture. Apart from storing images, the database also stores commands, which interpret a meaning of gestures. Once the gesture image is matched with the image stored in the database, the gesture meaning unit 114 correlates the database image with commands and pattern recognition engine 112, to determine the command matching the database image for determining the meaning of the gesture inputted by the user. Support vector machine (SVM) classification, K-Nearest Neighbour (KNN) are known techniques in the art for matching the input image (gesture image here) with images stored in the database of the gesture engine 110.
[029] For example, the user may give a gesture action‘thumbs up’. The camera captures one or more images of this‘thumbs up’ gesture. These captured images are then fed to the gesture engine 110, which then determines the image(s) stored in its database that may depict a thumb in an upwards position, to match with the gesture image(s) captured by the camera. The pattern recognition engine 112 then matches the database image (thumb in upward direction here). Once the match is determined, the gesture meaning unit 114 may receive the matching image determined by the pattern recognition engine 112 and further determine command(s) relating to the meaning of the thumbs up gesture. Commands relating to the thumbs up gesture typically means good job, great and nice, and the like as examples. The gesture meaning unit 114 then determines that the meaning of the gesture command is good/nice/great. Command(s) may be referred to as meaning of the query (MOQ) or processed queries.
[030] The machine learning unit 116 may comprise a data source identification block 122, a trigger point definition unit 124 and a relationship definition unit 126. The data source identification block 122 may provide a set of data sources identified based on relevancy of MOQ.
[031] Further, a data source is source of the data. The data source can be a file, a database on a DBMS, or even a live data feed. The data may be stored in a database (internal data) or may be in other remote servers (external data), accessed over a network. Also, the system 100 may collect data from various data from internal data sources (of user), that may include but not limited to sensor modules, emails, calendars, social networks, messages, app configurations, activity information (GPS), etc... The external data source may be any piece of data/dataset which is available in internet (structured or unstructured) or 3rd party server, and accessible through one or more networking standards. One or more data sources are required to process the given query and further. The identification of one or more data sources is crucial because, the entire output of the system depends on the appropriate selection of data sources, relevant to the context.
[032] Further, the data source identification block 122 of the machine learning unit 116 may be configured to receive the one or more MOQ from the NLP engine 104 and the gesture engine 110. The data source identification block 122 may be trained to identify one or more data sources that relate to the MOQ determined based on the user’s input. When the query is being processed, the data source identification block 122 can infer the closest matching data source based on MOQ, and user’s current context. In other words, the objective of the data source identification block 122 is to identify relevant data source from the all available set of data sources.
[033] For example, the user query is“Do I have to carry an umbrella tomorrow?”, the MOQ may be,‘Is it going to rain here, tomorrow’. Further, in this case the identified data sources may be‘weather’,‘location’ and‘time’. Rain and tomorrow may be the keywords present in the MOQ, which may also be identified by the machine learning unit 116. Data source‘weather’ may be in relation with rain, data source‘location’ may be in relation with GPS and data source‘time’ may be in relation with tomorrow. For example, the word umbrella relates to rain and user’s present position relates to location and time. The meaning umbrella may be carried by the user, when there is a rain tomorrow. The machine learning unit 116 may prioritize these keywords and select the data sources that may be based on the closest inference of the MOQs.
[034] Further, the trigger point definition unit 124 of machine learning system 116 may be configured to identify one or more trigger points (TPs) relating to the identified data source and MOQ received. Each data source may have one or more trigger points that are continuously checked. The trigger points may be time based, user based, activity based, priority based and the like. The example of identified trigger points may include, Rain on 20/04/2018 (TP1), at user location (TP2), time the user leaves office (TP3). The trigger points may be automatically framed by the system 100.
[035] Further, the relationship definition unit 126 of machine learning unit 116 may be configured to determine a logical relationship (R) between the one or more identified trigger points, to generate one or more events. Relating to the query“Do I have to carry an umbrella tomorrow?”, events may be generated. The events may be possible logical combination of one or more trigger points. In continuation to the above example, the event may be represented by Rl = TP1 & TP2 & TP3.
[036] The network 118 may be a wired network, a wireless network or a combination thereof. Furthermore, the network 118 may include a wide area network (WAN) (e.g., the
Internet), or other interconnected data paths across which multiple devices and/or entities may communicate. In some implementations, the network 118 may include a peer-to-peer network.
The network 118 may also include a mobile data network that may include third-generation
(3G), fourth-generation (4G), long-term evolution (LTE), long-term evolution advanced (LTE-
A), Voice -over-LTE (“VoLTE”) or any other mobile data network or combination of mobile data networks. The electronic device 120 may include, but not limited to, a mobile device, a smart wearable, a wrist-watch, a personal computer (PC), a tablet, a smartphone, a personal digital assistant (PDA) and the like.
[037] FIG. 2 is an exemplary block diagram illustrating an execution unit 200 which receives basic configuration settings (one time) from the machine learning unit 116 and generates notifications, in accordance with an embodiment. As shown, the execution unit 200 may include data sources 204, a trigger point generation unit 206, and a logical sequence block 208. Each component is explained in further detail below.
[038] Each component of the execution unit 200 receives a configuration from other components of the machine learning unit 116 (a data source identification block 122, a trigger point definition unit 124 and a relationship definition unit 126, respectively). In one embodiment, the data source identification block 122 supplies information relating selection and identification of data sources 204. The trigger point definition unit 124 may also supply identified trigger points to the trigger point generation unit 206. Further, the relationship definition unit 126 may send possible identified logical relationship data to the logical sequence block 208. Each component of execution unit 200 is explained in further detail below.
[039] In one embodiment, the data sources 204 may be configured to collect data related to one or more queries associated with the user. The data sources 204 may be differentiated into internal data or external data. The internal data may only have user data. The user data may include but not limited to emails, messages, calendar, schedules, social network messages, data received from user sensor modules (accessible by the user), in app configurations and the like. The external data may be any curated information from the internet.
[040] Further, the one or more external data sources 204 may include, but not limited to weather data, GPS data, food data, news feeds, activity data, sports data and the like. The data sources 204 may be identified pertaining to the query, and the data sources 204 may be configured to fetch relevant data. The one or more data sources 204 may also generate live data streams.
[041] Further, the trigger point generation unit 206 may be configured for generating one or more trigger points pertaining to the associated data sources/streams. For example, the data sources 204 may be weather data, GPS data, and activity data. The user may input a query stating,“Do I have to carry an umbrella tomorrow?”, for which weather data may be identified as one of the data source 204 and a trigger point TP 1 is generated if the condition (“Rain on 20/04/2018”) is matched in the incoming weather data source. The trigger point TP2 is generated if the condition (“at user location”) is matched in the incoming GPS location data source. Similarly, the trigger point TP3 is generated if the condition (“when the user leaves for office”) is matched in the incoming user activity data source.
[042] Further, the logical sequence block 208 may be configured to receive all the generated trigger points. The logical sequence block 208 may be configured to generate events based on the relationships between the trigger points and once relationship evaluation is successful, then logical sequence block 208 may create an event (R). The event may be delivered to the electronic device 120 via the network 118. The event may be converted to appropriate form of user notification by the electronic device 120. For example, the trigger points TP1, TP2, and TP3 may be received by the logical sequence block 208 from the trigger point generation unit 206. For example, the logical sequence block 208 may create an event when the logical relationship evaluation R = (TP1+TP2) XOR TP3 is successful. Further, the logical sequence block 208 may consider (TP2 & TP3), if TP1 had occurred in the past, to create a corresponding event. The notification of the corresponding event may be delivered to the user in one or more forms, not limited to, audible alarm, synthesized voice, digital display, depending on the capability of the electronic device 120.
[043] FIG. 3 is an exemplary architecture of a system 300 illustrating co-ordination between modules of the machine learning unit 116 and execution unit 200, in an embodiment of the present invention.
[044] The NLP engine 104, the gesture engine 110, and the blocks (data source identification block 122, the trigger point definition unit 124, the relationship definition unit 126) of the machine learning unit 116 may be one time setting units. In one embodiment, the modules of machine learning unit 116 are considered as‘one time setting units’. The meaning of the user query is determined through NLP engine 104 and gesture engine 110. The machine learning unit 116 identifies one or more appropriate data sources relating to the meaning of the query (MOQ), identifies one or more trigger points relating to the one or more identified data sources, and determines logical relationships between the one or more identified trigger points. As shown there, the NLP engine 104 and gesture engine 110 supplies the meaning of the query (MOQ) to each of the modules of machine learning unit 116. The modules of machine learning unit 116 supplies identified data sources, trigger points and logical relationships (computed between the trigger points).
[045] On the other hand, the execution unit 200 receives identified data sources, trigger points and successful logical relationships (computed between the trigger points) from the modules of machine learning unit 116. The data sources 204, the trigger point generation unit 206, and the logical sequence block 208 may be continuous execution units. In other words, the modules of execution unit 200 are described as‘continuous execution units’. As shown there, the execution unit 200 also collects contextual data from the identified data sources, continuously process the incoming contextual data/data streams and generates one or more trigger points pertaining to one or more data sources and identified trigger points. The execution unit 200 further configured to evaluate the logical relationship between one or more generated trigger points, considering the identified data sources and logic previously configured by the machine learning unit 116. When the evaluation is successful, the execution unit 200 creates one or more event(s).
[046] The execution unit 200 further delivers the one or more events to the user’s electronic device 120 through the network 118. The electronic device 120 appropriately converts the subject events into user notifications.
[047] FIG. 4 illustrates an exemplary system architecture 400 of the system for generating notifications, in accordance with an embodiment. As shown, the system may include a processor 402, a memory 406, a communication unit 404, a I/O 408, and a bus 410. Each component is described in further detail below.
[048] In general, the processor 402 may include any suitable special-purpose or general- purpose computer, computing entity, or processing device including various computer hardware or software modules and may be configured to execute instructions stored on any applicable computer-readable storage media. For example, the processor 402 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other digital or analog circuitry configured to interpret and/or to execute program instructions and/or to process data. [049] The memory 406 may include computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon. Such computer- readable storage media may be any available media that may be accessed by a general-purpose or special-purpose computer, such as the processor 402. By way of example, and not limitation, such computer-readable storage media may include tangible or non-transitory computer- readable storage media including Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices), or any other storage medium which may be used to carry or store desired program code in the form of computer-executable instructions or data structures and which may be accessed by a general- purpose or special-purpose computer.
[050] The communication unit 404 may be a wired network or a wireless network. Furthermore, the communication unit 404 may include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), or other interconnected data paths across which multiple devices and/or entities may communicate.
[051] The I/O (input/output unit) 408 may provide communication between the processor 402, the memory 406, and the communication unit 404. The processor 402, the memory 406, the communication unit 404, and the I/O 408 may all be interconnected via the bus 410. The bus 410 may be a communication system that transfers data/information between each component.
[052] FIG. 5 illustrates an exemplary method 500 illustrating a flowchart for receiving a user query, processing the query, identifying one or more data sources relating to meaning of the query (MOQ), identifying one or more trigger points relating to the identified data sources, and determining a logical relationship with respect to identified trigger points, in accordance with an embodiment. The steps 502 - 506 may be executed by the NLP engine 104 or the gesture engine 110. The steps 508 - 512 may be executed by the machine learning unit 116. Each step is explained in further detail below.
[053] At step 502, the user inputs one or more queries to the system 100 (of FIG. 1) via the electronic device 120 (for example, a smart wearable). Further, the user may input the one or more queries using verbal communication means and gesture means. Verbal communication relates to inputs in the form of speech. Gesture communication relates to inputs in the form of user actions.
[054] At step 504, the NLP engine 104 and the gesture engine 110 may be configured to receive the one or more queries that the user provided as inputs. The NLP engine 104 receives the inputs pertaining to verbal communication means and the gesture engine 110 receives the inputs pertaining to gesture means.
[055] At step 506, the meaning of the query (MOQ) is determined. The NLP engine 104 may be configured to determine a meaning of the one or more queries of the user, pertaining to verbal communication means. The gesture engine 110 may also be configured to determine the meaning of the one or more queries (MOQ) of the user, pertaining to gesture means. The MOQ is fed to the machine learning unit 116.
[056] At step 508, the machine learning unit 116 may be configured to identify one or more identified data sources relating to the MOQ.
[057] At step 510, the machine learning unit 116 may be configured to identify one or more trigger points based on the data sources identified pertaining to the user queries. Each data source may have one or more trigger points that are continuously checked. The trigger points may be time based, user based, activity based, priority based and the like.
[058] At step 512, the machine learning unit 116 may be configured to determine a logical relationship between the one or more identified trigger points, to generate one or more events. [059] FIG. 6 illustrates an exemplary method 600 illustrating a flowchart for supplying identified data to the execution unit from the machine learning unit, in accordance with an embodiment. Step 602 may be executed by the data source identification block 122, step 604 may be executed by the trigger point definition unit 124, and step 606 may be executed by the relationship definition unit 126. Each step is explained in further detail below. [060] At step 602, the data source identification block 122 may be configured to supply information relating to selection and identification of the one or more data sources to the data source block 204.
[061] At step 604, the trigger point definition unit 124 may be configured to supply a set of identified trigger points to the trigger point generation unit 206.
[062] At step 606, the relationship definition unit 126 may be configured to supply possible logical relationship(s) data to the logical sequence block 208.
[063] FIG. 7 illustrates an exemplary method 700 illustrating a flowchart for evaluating one or more trigger points, in accordance with an embodiment. The steps 702 - 714 may be executed by the data sources 204, the trigger point generation unit 206, and the logical sequence block 208 of the execution unit 200 (relating to FIG. 2). Each step is explained in further detail below.
[064] At step 702, contextual data from the one or more data sources may be collected. Data may be collected from various data from internal data sources (of user), that may include but not limited to sensor modules, emails, calendars, social networks, messages, app configurations, activity information (GPS), etc. The external data source may be any piece of data/dataset which is available in internet (structured or unstructured) or 3rd party server, and accessible through one or more networking standards. [065] At step 704, the trigger point generation unit 206 may be configured to process the incoming data and generate one or more trigger points pertaining to the associated data sources and identified trigger points. The trigger point generation unit 206 may be configured for generating one or more trigger points pertaining to associated data streams.
[066] At step 706, the logical sequence block 208 may be configured to evaluate one or more logical relationships between the trigger points generated. At step 708, the success of the logical evaluation is checked. If the logical evaluation is not successful, then step 706 repeats where the logical relationships may be re-evaluated. If the logical evaluation is successful, then at step 710, one or more events are created. The logical sequence block 208 may be configured to generate events based on the relationships between the trigger points and once relationship evaluation is successful, then it may create an event (R). For example, the logical sequence block 208 may create an event when the logical relationship evaluation R = (TP1+TP2) XOR TP3 is successful. Further, the logical sequence block 208 may consider (TP2 & TP3), if TP1 had occurred in the past, to create a corresponding event.
[067] At step 712, the one or more events may be delivered to the electronic device 120 via the network 118. At step 714, the events are converted into appropriate user notifications by the electronic device 120 The notification of the corresponding event may be delivered to the user in one or more forms, not limited to, audible alarm, synthesized voice, digital display, depending on the capability of the electronic device 120.
[068] It shall be noted that the processes described above are described as sequence of steps; this was done solely for the sake of illustration. Accordingly, it is understood that some steps may be added, some steps may be omitted, the order of the steps may be re-arranged, or some steps may be performed simultaneously.
CONCLUSION
[069] The forgoing disclosure overcomes the drawbacks of the conventional inventory management system. Thus, the present invention as discussed in this document with respect to different embodiments will be advantageous at least for improving receiving constant notifications pertaining to user queries. Furthermore, the system and method are also advantageous as the system is intelligent enough to generate events pertaining to user query(s), generate notifications forthe events, and frequency of notifications automatically. Furthermore, the system and method are also advantageous as the system can receive, collect, organize, and segregate the events. Additional advantages not listed may be understood by a person ordinary skilled in the art in view of the embodiments disclosed above. In the foregoing detailed description, numerous specific details, examples, and scenarios are explained in order to facilitate a thorough understanding of the present disclosure. However, the embodiments of the disclosure may be practiced without such specific details. Further, such examples and scenarios are provided for illustration, and are not intended to limit the disclosure in any way. Those of ordinary skill in the art, with the included descriptions, should be able to implement appropriate functionality without undue experimentation. Thus, the scope of the invention should be determined by the appended claims and their legal equivalents rather than by details, examples, and scenarios provided. Although embodiments have been described with reference to specific example embodiments, it will be evident that various combinations, modifications, additions, and omissions may be made to these embodiments without departing from the broader spirit and scope of the foregoing disclosure and appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation.

Claims

CLAIMS What is claimed is:
1. A system for generating notifications based on one or more queries associated with a user, the system comprising:
a query input module configured to receive one or more queries associated with a user, wherein the user inputs the one or more queries using at least one of a verbal communication means and a gesture means;
a natural processing engine (NLP) engine configured to determine a meaning of the one or more queries received through the verbal communication means;
a gesture engine configured to determine a meaning of the one or more queries received through the gesture means;
a machine learning unit in communication with one or more data sources, wherein the machine learning unit receives the meaning of the one or more queries determined from at least one of the NLP engine and the gesture engine, and wherein the machine learning unit is configured to:
identify one or more data sources required to resolve the one or more queries;
identify one or more trigger points based on the data sources identified; and determine logical relationship between the one or more identified trigger points; and an execution unit in communication with the machine learning unit, wherein the execution unit is configured to:
generate one or more triggers pertaining to the data sources and the identified trigger points;
evaluate the logical relationship between the one or more trigger points;
create one or more events when the logical relationship is successful; and
communicate the one or more events to an electronic device associated with the user.
2. The system of claim 1, wherein the NLP engine comprises:
a voice to text converter configured to transcribe spoken words by the user while speaking out the one or more queries, into text; and
a command meaning unit configured to determine the meaning of the one or more queries based on the text determined by the voice to text converter, wherein the command meaning unit prioritize one or more keywords based on current and ongoing activities of the user.
3. The system of claim 1, wherein the gesture engine comprises:
a pattern recognition engine configured to identify one or more gestures pertaining to an action performed by the user, with respect to the one or more queries; and
a gesture meaning unit configured to determine the meaning of the one or more queries, based on the gesture identified by the pattern recognition engine.
4. A method for generating notifications based on one or more queries associated with a user, the method comprising:
identifying at least one data source, among the one or more data sources, required to resolve the one or more queries;
identifying one or more trigger points based on the at least one data source identified; determining a logical relationship between the one or more trigger points based on a meaning of the one or more queries received, to generate one or more events;
communicating the one or more events to an electronic device associated with the user; and
generating one or more notifications pertaining to the one or more events generated.
5. The method of claim 4, wherein the one or more queries are received through verbal communication means, gesture means, or a combination of verbal communication means and gesture means.
6. The method of claim 4, further comprising:
identifying one or more gestures pertaining to an action performed by the user corresponding to the one or more queries; and
determining the meaning of the one or more queries, based on the gesture identified, wherein one or more keywords are prioritized based on current and ongoing activities of the user.
7. The method of claim 4, further comprising:
transcribing spoken words by the user while speaking out the one or more queries, into text; and
determining a meaning of the one or more queries based on the text determined.
8. The method of claim 4, further comprising:
continuously receiving data from the identified data source;
determining whether a trigger must be generated based on the trigger point corresponding to the identified data source;
checking whether the logical relationship between the one or more trigger points is met; and
generating the one or more events when the logical relationship is met.
PCT/SG2019/050285 2018-06-01 2019-06-01 System and method for generating notifications related to user queries WO2019231405A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SG10201804684QA SG10201804684QA (en) 2018-06-01 2018-06-01 System and method for generating notifications related to user queries
SG10201804684Q 2018-06-01

Publications (1)

Publication Number Publication Date
WO2019231405A1 true WO2019231405A1 (en) 2019-12-05

Family

ID=68699005

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2019/050285 WO2019231405A1 (en) 2018-06-01 2019-06-01 System and method for generating notifications related to user queries

Country Status (2)

Country Link
SG (1) SG10201804684QA (en)
WO (1) WO2019231405A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5175814A (en) * 1990-01-30 1992-12-29 Digital Equipment Corporation Direct manipulation interface for boolean information retrieval
US20140136013A1 (en) * 2012-11-15 2014-05-15 Sri International Vehicle personal assistant
US20170160813A1 (en) * 2015-12-07 2017-06-08 Sri International Vpa with integrated object recognition and facial expression recognition
CN107301168A (en) * 2017-06-01 2017-10-27 深圳市朗空亿科科技有限公司 Intelligent robot and its mood exchange method, system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5175814A (en) * 1990-01-30 1992-12-29 Digital Equipment Corporation Direct manipulation interface for boolean information retrieval
US20140136013A1 (en) * 2012-11-15 2014-05-15 Sri International Vehicle personal assistant
US20170160813A1 (en) * 2015-12-07 2017-06-08 Sri International Vpa with integrated object recognition and facial expression recognition
CN107301168A (en) * 2017-06-01 2017-10-27 深圳市朗空亿科科技有限公司 Intelligent robot and its mood exchange method, system

Also Published As

Publication number Publication date
SG10201804684QA (en) 2020-01-30

Similar Documents

Publication Publication Date Title
US10642577B2 (en) Search and knowledge base question answering for a voice user interface
CN107908803B (en) Question-answer interaction response method and device, storage medium and terminal
US20190272269A1 (en) Method and system of classification in a natural language user interface
US11488576B2 (en) Artificial intelligence apparatus for generating text or speech having content-based style and method for the same
US8170866B2 (en) System and method for increasing accuracy of searches based on communication network
US10498888B1 (en) Automatic call classification using machine learning
US9293133B2 (en) Improving voice communication over a network
US20190221208A1 (en) Method, user interface, and device for audio-based emoji input
US11355115B2 (en) Question answering for a voice user interface
US20230177798A1 (en) Relationship modeling and anomaly detection based on video data
US10958610B2 (en) Generating alerts based on predicted mood responses to received electronic messages
US11144279B1 (en) Memory retention system
KR102281504B1 (en) Voice sythesizer using artificial intelligence and operating method thereof
US11789695B2 (en) Automatic adjustment of muted response setting
US20200403816A1 (en) Utilizing volume-based speaker attribution to associate meeting attendees with digital meeting content
KR20210020656A (en) Apparatus for voice recognition using artificial intelligence and apparatus for the same
JP2021096847A (en) Recommending multimedia based on user utterance
KR20210044475A (en) Apparatus and method for determining object indicated by pronoun
US11417313B2 (en) Speech synthesizer using artificial intelligence, method of operating speech synthesizer and computer-readable recording medium
Yoon et al. Fear emotion classification in speech by acoustic and behavioral cues
US10923113B1 (en) Speechlet recommendation based on updating a confidence value
US10650055B2 (en) Data processing for continuous monitoring of sound data and advanced life arc presentation analysis
US10282417B2 (en) Conversational list management
WO2019231405A1 (en) System and method for generating notifications related to user queries
EP4020468A1 (en) System, method and apparatus for conversational guidance

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19810853

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19810853

Country of ref document: EP

Kind code of ref document: A1