US20190362217A1 - Always listening and active voice assistant and vehicle operation - Google Patents
Always listening and active voice assistant and vehicle operation Download PDFInfo
- Publication number
- US20190362217A1 US20190362217A1 US15/987,175 US201815987175A US2019362217A1 US 20190362217 A1 US20190362217 A1 US 20190362217A1 US 201815987175 A US201815987175 A US 201815987175A US 2019362217 A1 US2019362217 A1 US 2019362217A1
- Authority
- US
- United States
- Prior art keywords
- vehicle
- answer
- topic
- topics
- contexts
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/037—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
- B60R16/0373—Voice control
-
- G06F17/271—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G06N99/005—
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context
Definitions
- This disclosure relates to the operation of vehicles through active and always listening voice assistance.
- Query systems give answers to questions asked after invocation. For example, “Hey Ford®, what is the weather like today?” After the invocation, “Hey Ford®,” natural language processing and artificial methods are used to find an answer to the question. This cadence—where invocation is required prior to the question—may require more statements than necessary to provide the answer because conversations prior to invocation are ignored. Additionally, query systems that provide reverse cadence support may be ill-equipped to distinguish between forward cadences and reverse cadences.
- a vehicle includes a controller configured to select one of a group of topics for generating an answer to a question embedded within the group based on an operating parameter of the vehicle and a syntax of a phrase. The selection is responsive to input originating from utterances including a preceding topic and a following topic having a moniker therebetween that is associated with only one of the topics through the syntax.
- the vehicle may operate an interface to output the answer.
- a vehicle includes a controller configured to select one of a group of topics for generating an answer to a question embedded within the group based on an operating parameter of the vehicle and a syntax of a phrase. The selection is responsive to input originating from utterances including a preceding topic and a following topic having a moniker therebetween that is associated with only one of the topics through the syntax. The vehicle may be operated according to the answer.
- a method includes, by a controller, generating an answer to a question embedded within a group of topics based on a selection of one of the topics and an operating parameter of the vehicle and a syntax, and operate the vehicle according to the answer.
- the generation is responsive to input originating from utterances defining a group of phrases including a preceding topic and a following topic having a moniker therebetween associated with only one of the topics through syntax.
- FIG. 1 is a schematic of a vehicle having an infotainment system and associated communications capabilities
- FIG. 2 is a schematic of vehicle control systems and peripherals
- FIG. 3A is an algorithm always listening voice systems
- FIG. 3B is an algorithm for selecting contexts.
- a vehicle query system may provide answers and solutions related to a vehicle based on a combination of forward cadences, reverse cadences, vehicle operating parameters, and/or always-listening algorithms.
- Questions using a reverse cadence may include a tag question or a tag question phrase.
- the tag question phrase may be isolated to identify the reverse cadence.
- the tag question phrase would generally follow topics that require the answering service (i.e., a reverse cadence), while other interrogatories would generally precede topics that require the answering service (i.e., a forward cadence).
- Verbal utterances may be recorded and analyzed constantly to isolate topics and provide contexts. As the topics of potential questions are organized, the algorithm waits for an invocation such as, “Ford®, what do you think.”
- a tag question is any question that follows a statement as opposed to preceding a statement.
- a tag question phrase may include a moniker of whom the question is being asked.
- the always listening and tag question answering service provides a reverse cadence where the statement is made, the service is invocated, and then the answer is provided.
- the always listening and tag question answering service provides a forward cadence the service is invocated, where the statement is made, and then the answer is provided.
- the always listening service may also always search for answers to every topic so that the answer is readily available for presentation to the occupant.
- the previous conversation included at least three topics 1) the Detroit symphony orchestra is playing tonight in Detroit; 2) I bet we have enough fuel to get to Detroit; and 3) it is going to be very cold.
- the topics may be isolated based on syntactical, categorical, or other methods. Over time, the topics may be distilled and isolated to particular contexts.
- the contexts may be broad categories of topics of which answers may be required.
- the contexts may also be presented to vehicle occupants for selection. The selection may also be provided via machine learning such that topics are selected according to previously selected topics. Meaning, the occupant may select the topic selection made, and then the machine learning algorithm would update the preferred contexts automatically.
- the vehicle may provide an answer or indication to the occupant by operation of the vehicle or display of the answer.
- an always listening algorithm may be used to identify topics requiring answer service and automatically provide an answer to the topic after invocation by a tag question phrase.
- FIG. 1 illustrates an example system 100 including a vehicle 102 implementing an always listening answer retrieval algorithm.
- the vehicle 102 may include a vehicle computing system (VCS) 106 configured to communicate over a wide-area network using a telematics control unit (TCU) 120 A.
- VCS vehicle computing system
- TCU telematics control unit
- the TCU 120 A may have various modems 122 configured to communicate over respective communications paths and protocols.
- While an example system 100 is shown in FIG. 1 , the example components as illustrated are not intended to be limiting. Indeed, the system 100 may have more or fewer components, and additional or alternative components and/or implementations may be used.
- the vehicle 102 may include various types of automobile, crossover utility vehicle (CUV), sport utility vehicle (SUV), truck, recreational vehicle (RV), boat, plane or other mobile machine for transporting people or goods.
- the vehicle 102 may be powered by an internal combustion engine.
- the vehicle 102 may be a hybrid electric vehicle (HEV) powered by both an internal combustion engine and one or more electric motors, such as a series hybrid electric vehicle (SHEV), a parallel hybrid electrical vehicle (PHEV), or a parallel/series hybrid electric vehicle (PSHEV).
- SHEV series hybrid electric vehicle
- PHEV parallel hybrid electrical vehicle
- PSHEV parallel/series hybrid electric vehicle
- the capabilities of the vehicle 102 may correspondingly vary.
- vehicles 102 may have different capabilities with respect to passenger capacity, towing ability and capacity, and storage volume.
- the VCS 106 may be configured to support voice command and BLUETOOTH interfaces with the driver and driver carry-on devices, receive user input via various buttons or other controls, and provide vehicle status information to a driver or other vehicle 102 occupants.
- An example VCS 106 may be the SYNC® system provided by FORD MOTOR COMPANY of Dearborn, Mich.
- the VCS 106 may further include various types of computing apparatus in support of performance of the functions of the VCS 106 described herein.
- the VCS 106 may include one or more processors configured to execute computer instructions, and a storage medium on which the computer-executable instructions and/or data may be maintained.
- a computer-readable storage medium also referred to as a processor-readable medium or storage
- a processor receives instructions and/or data, e.g., from the storage, etc., to a memory and executes the instructions using the data, thereby performing one or more processes, including one or more of the processes described herein.
- Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and/or technologies, including, without limitation, and either alone or in combination, Java, C, C++, C#, Fortran, Pascal, Visual Basic, Python, Java Script, Perl, PL/SQL, etc.
- the VCS 106 may be configured to communicate with TCU 120 A.
- the TCU 120 A may include a plurality of modems 122 capable of packet-switch or circuit-switched signaling.
- the TCU 120 A may control the operation of the modems 122 such that a suitable communication path is used.
- the modems may be configured to communicate over a variety of communications paths.
- the paths may be configured with circuit-switched 130 , packet-switched 132 , 134 signaling, or combination thereof.
- Packet-switched communication 132 , 134 paths may be Internet Protocol (IP)-based or use packet-based switching to transfer information.
- IP Internet Protocol
- the packet-switched communication may be long-term evolution (LTE) communications.
- the circuit-switch 130 communication path may be SIGTRAN or another implement, carrying circuit-switched signaling information over IP.
- the underlying signaling information is, however, still formatted under the circuit-switched protocol.
- the VCS 106 may also receive input from human-machine interface (HMI) controls 108 configured to provide for occupant interaction with the vehicle 102 .
- HMI human-machine interface
- the VCS 106 may interface with one or more buttons or other HMI controls 108 configured to invoke functions on the VCS 106 (e.g., steering wheel audio buttons, a push-to-talk button, instrument panel controls, etc.).
- the VCS 106 may also drive or otherwise communicate with one or more displays 110 configured to provide visual output to vehicle occupants, e.g., by way of a video controller.
- the display 110 may be a touch screen further configured to receive user touch input via the video controller, while in other cases the display 110 may be a display only, without touch input capabilities.
- the display 110 may be a head unit display included in a center console area of the vehicle 102 cabin.
- the display 110 may be a screen of a gauge cluster of the vehicle 102 .
- the VCS 106 may be further configured to communicate with other components of the vehicle 102 via one or more in-vehicle networks 112 or vehicle buses 112 .
- the in-vehicle networks 112 may include one or more of a vehicle controller area network (CAN), an Ethernet network, and a media oriented system transfer (MOST), as some examples.
- the in-vehicle networks 112 may allow the VCS 106 to communicate with other vehicle 102 systems, such as a vehicle modem of the TCU 120 A (which may not be present in some configurations), a global positioning system (GPS) module 120 B configured to provide current vehicle 102 location and heading information, and various other vehicle ECUs configured to cooperate with the VCS 106 .
- GPS global positioning system
- the vehicle ECUs may include a powertrain control module (PCM) 120 C configured to provide control of engine operating components (e.g., idle control components, fuel delivery components, emissions control components, etc.) and monitoring of engine operating components (e.g., status of engine diagnostic codes); a body control module (BCM) 120 D configured to manage various power control functions such as exterior lighting, interior lighting, keyless entry, remote start, and point of access status verification (e.g., closure status of the hood, doors and/or trunk of the vehicle 102 ); a radio transceiver module (RCM) 120 E configured to communicate with key fobs or other local vehicle 102 devices; a climate control management (CCM) 120 F module configured to provide control and monitoring of heating and cooling system components (e.g., compressor clutch and blower fan control, temperature sensor information, etc.); and a battery control module (BACM) 120 G configured to monitor the state of charge or other parameters of the battery 104 of the vehicle 102 .
- PCM powertrain control module
- BCM body control module
- the VCS 106 may be configured to access the communications features of the TCU 120 A by communicating with the TCU 120 A over a vehicle bus 112 .
- the vehicle bus 112 may include a controller area network (CAN) bus, an Ethernet bus, or a MOST bus.
- the VCS 106 may communicate with the server 150 via a server modem 152 using the communications services of the modems 122 .
- the vehicle 102 may include an engine 113 , starter-generator 114 , battery 116 , and electrical loads 118 .
- the controller network 112 may connect to all of these vehicle systems through sensors (e.g., fuel level sensor 115 , oil sensor 117 ) or vehicle system controllers (e.g., 120 A, 120 B, 120 C, 120 D, 120 E, 120 F, 120 G).
- the controller network 112 may control the vehicle systems to provide autonomous control.
- the engine 113 may have a direct mechanical linkage to the starter-generator 114 .
- the starter-generator 114 may be electrically connected to the battery 116 and electrical loads 118 .
- the battery 116 may be connected to the electrical loads 118 .
- the VCS 106 may recognize that the vehicle occupants desire to travel to Detroit if they have enough gas.
- the VCS 106 may pull data from vehicle sensors (i.e., fuel level sensor 115 ) to determine the remaining fuel in the fuel tank.
- the VCS 106 may then request the anticipated fuel consumption for the vehicle's 102 current location to the symphony orchestra. Indeed, the vehicle 102 can listen to the occupants' conversation and upon request provide a response without further requiring the question to be re-asked or the provision of additional information.
- an algorithm 300 is shown.
- the algorithm 300 starts in step 302 .
- An implementation of the algorithm 300 may include additional or less steps, and the steps may be performed in a different order. The steps may also be performed simultaneously, at similar times, or sequentially.
- the VCS 106 or other processors collect verbal utterances.
- the verbal utterances may be sayings, statements, uttered words, or conversations available for capture by the microphone or array of microphones 124 by occupants in or near the vehicle.
- topics are identified within the verbal utterances. The topics may be identified based on any natural language processing algorithm.
- the topics are identified to later be associated with the question asked.
- the topics may be portions of a sentence or entire sentences.
- the topics may be formed by verb-noun associations or other grammatical, syntactical, or semantical associations.
- a moniker may be any word or saying associated with the vehicle (e.g., manufacturer, model, software provider, infotainment system provider, branding associated with the vehicle or vehicle manufacturer).
- the moniker may be part of a tag question phrase or a tag question.
- a context score is compared with a syntax score.
- the context score may be based on the topic score of identified topics in step 306 . Meaning, the confidence that a given statement, phrase, or word is associated with a particular topic and context may be determined based on a variety of natural language and machine learning methods. For example, the statement “the Detroit symphony orchestra is playing tonight in Detroit” includes two topics. One topic in the statement may be the Detroit symphony orchestra. A second topic may be the city of Detroit. The context of the statement may be musical performance events. Because the statement is directly associated with the context, the context confidence score may be labeled as high, qualitatively, or above 75, quantitatively.
- the statement “the Detroit art gallery is hosting anêt tonight in Detroit” includes two similar topics that may have some relation to the musical performance events context.
- the context score in this situation may be markedly lower because the topic is not specifically musical performance, instead it is related to artistic performances. Therefore, the context confidence score may be labeled as medium, qualitatively, or above 50, quantitatively. The highest context score may be compared with the syntax score.
- the syntax score may determine, syntactically, whether a tag question phrase exists and the moniker is part of the tag question phrase.
- the tag question phrase may simply be, “Ford®?”
- Ford® comes at the end of a statement as a tag question phrase—indicating a reverse cadence—it could also be the beginning of a forward cadence.
- the algorithm may determine whether the moniker Ford® is syntactically at the beginning or end of a question using known and unknown methods.
- the confidence score of whether a forward or reverse cadence is being used may be labeled qualitatively or quantitatively as described above.
- a phrase with the highest syntax score may be, “(topic).
- step 326 and step 328 non-interrogatory moniker utterances are filtered to prevent unintentional requests. If the moniker is not part of a tag question phrase (e.g., reverse cadence) or an active request (e.g., forward cadence), the algorithm returns to step 306 . If the moniker is part of a tag question phrase according to a syntactical confidence score being above a predetermined threshold—or above a threshold set relative to the context score—the algorithm proceeds to step 338 . If the moniker is not part of a tag question phrase according to a syntactical confidence score being above a predetermined threshold—or above a threshold set relative to the context score—the algorithm proceeds to step 328 .
- a tag question phrase e.g., reverse cadence
- an active request e.g., forward cadence
- sub-algorithm A 310 collects information to define contexts. Contexts may be categories of topics or other logical representations configured to represent classes of vehicle operating parameters. As shown in step 312 , sub-algorithm A 310 identifies contexts within verbal utterances. Context identification may include nutrition information generally, while topic identification is more narrowly tuned to a question about nutrition in a candy bar.
- the list of contexts may be narrowly tuned for the vehicle in step 314 such that generic information requests are not available (e.g., answers to arithmetic, pronunciation of words). Meaning, broad question retrieval abilities may optionally be narrowed by the manufacturer or occupant under the assumption that vehicle or travel related questions will be present.
- vehicle operating parameters are analyzed to provide contexts.
- the vehicle may be configured to make vehicle-specific parameter contexts available for the question-answer service. Meaning, contexts associated with oil life, fuel level, state of charge, climate status, engine temperature, or other vehicle parameters may be made available through contexts in step 316 .
- a machine algorithm or manufacturer may select the operating parameters available for answer retrieval, in step 318 .
- oil temperature may be an available vehicle parameter, but a machine learning algorithm may determine that the context should not be made available because questions are so infrequently asked about engine oil temperature.
- Vehicle parameter contexts may be given a stronger weight that the verbal utterance contexts.
- the contexts are presented to the user. Meaning, the user can further select which contexts it may desire to have answered by the answer service.
- the contexts may be presented using the HMI controls 108 or the display screen 110 .
- the contexts may be read by the system to the occupants and the occupants may provide confirmation of the proper context selection. For example, the vehicle may state, “Line 1: Weather.” The occupant may then verbally affirm that line one is the proper selection by saying, “One.”
- the contexts may be presented such that the most commonly used contexts are presented first.
- the contexts may also be presented in an order based on the verbal utterances already received and the frequency of the contexts being discussed.
- the weather context may be presented to the user as a primary option.
- the user context selections are received for use in step 338 .
- the contexts may be assigned weights to improve the answer service. For example, the heavily discussed weather context may be given a stronger weight than the sparsely discussed oil temperature. All other things being equal, the heavier weighted context will take topic selection precedence over the unweighted or less-weighted context in the topic selection process of step 338 .
- the selected contexts along with the identified topics are known.
- the algorithm may then recognize the topic to be determined based on the contexts.
- the topic selection may take into account the weights applied to the contexts that each topic resides in. For example, topics within the weather context may take precedence over the topics in the oil temperature context. Further, the topics syntactical and semantical strength may be weighted. For example, a confidence value of the topic may be determined based on the condition of the verbal utterance. Meaning, phrases that have grammatical coherence may fall within a stronger weighted context but be discounted because of the syntactical or semantical score. Additionally, proximity to the tag question phrase may be used to further weight the topic.
- topics falling directly before the tag question phrase may have its score doubled or multiplied by a factor. Meaning, a context having a low weight may be selected over a context having a high weight if the topic immediately precedes the tag question phrase and the syntactical and semantical scores are low for the weather topic.
- step 340 the selected topic having the highest confidence score is sent to the server 150 to be answered.
- the server 150 provides the highest likely answer through the statistical and machine learning algorithms therein. Any answering service may provide the answer, and the answer does not need to be relative a vehicle.
- the answer may be sent back to the vehicle and presented to the occupants in step 342 .
- the vehicle 102 may then automatically operate the vehicle based on the answer or prompt the user select a course of action in step 344 . For example, if the topic selected was “I bet we have enough fuel to get to Detroit” the vehicle may prepare a route to Detroit for the symphony and autonomously navigate the car to the destination.
- the VCS 106 determines whether the moniker is part of a forward cadence.
- the forward cadence determination may be based on the comparison of context and syntactic scores. For example, the syntactical score may be high when the moniker is accompanied by a salutation (e.g., “Hey Ford®”). If the moniker is associated with a forward cadence in step 328 , the VCS 106 will discard all utterances before invocation and store any utterances after invocation in step 330 .
- the VCS 106 sends the query request determined from the stored verbal utterances to the server 150 .
- the server 150 provides the highest likely answer through the statistical and machine learning algorithms therein.
- Any answering service may provide the answer, and the answer does not need to be relative a vehicle.
- the answer may be sent back to the vehicle and presented to the occupants in step 334 .
- the vehicle 102 may then automatically operate the vehicle based on the answer or prompt the user select a course of action in step 336 . For example, statement was, “Hey Ford®, do we have enough fuel to get to Detroit,” the vehicle may prepare a route to Detroit for the symphony and autonomously navigate the car to the destination upon an affirmative answer.
- These attributes may include, but are not limited to cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. As such, embodiments described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics are not outside the scope of the disclosure and may be desirable for particular applications.
Abstract
Description
- This disclosure relates to the operation of vehicles through active and always listening voice assistance.
- Query systems give answers to questions asked after invocation. For example, “Hey Ford®, what is the weather like today?” After the invocation, “Hey Ford®,” natural language processing and artificial methods are used to find an answer to the question. This cadence—where invocation is required prior to the question—may require more statements than necessary to provide the answer because conversations prior to invocation are ignored. Additionally, query systems that provide reverse cadence support may be ill-equipped to distinguish between forward cadences and reverse cadences.
- A vehicle includes a controller configured to select one of a group of topics for generating an answer to a question embedded within the group based on an operating parameter of the vehicle and a syntax of a phrase. The selection is responsive to input originating from utterances including a preceding topic and a following topic having a moniker therebetween that is associated with only one of the topics through the syntax. The vehicle may operate an interface to output the answer.
- A vehicle includes a controller configured to select one of a group of topics for generating an answer to a question embedded within the group based on an operating parameter of the vehicle and a syntax of a phrase. The selection is responsive to input originating from utterances including a preceding topic and a following topic having a moniker therebetween that is associated with only one of the topics through the syntax. The vehicle may be operated according to the answer.
- A method includes, by a controller, generating an answer to a question embedded within a group of topics based on a selection of one of the topics and an operating parameter of the vehicle and a syntax, and operate the vehicle according to the answer. The generation is responsive to input originating from utterances defining a group of phrases including a preceding topic and a following topic having a moniker therebetween associated with only one of the topics through syntax.
-
FIG. 1 is a schematic of a vehicle having an infotainment system and associated communications capabilities; -
FIG. 2 is a schematic of vehicle control systems and peripherals; -
FIG. 3A is an algorithm always listening voice systems; and -
FIG. 3B is an algorithm for selecting contexts. - Embodiments of the present disclosure are described herein. It is to be understood, however, that the disclosed embodiments are merely examples and other embodiments may take various and alternative forms. The figures are not necessarily to scale; some features could be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention. As those of ordinary skill in the art will understand, various features illustrated and described with reference to any one of the figures may be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combinations of features illustrated provide representative embodiments for typical applications. Various combinations and modifications of the features consistent with the teachings of this disclosure, however, could be desired for particular applications or implementations.
- Vehicle occupants carry conversations discussing events, circumstances, or issues of the past, present, and future. Instead of requiring forward-biased invocation similar to, “Hey Ford®, what is the temperature outside,” an algorithm may be implemented to distinguish between forward and reverse cadences. Indeed, a vehicle query system may provide answers and solutions related to a vehicle based on a combination of forward cadences, reverse cadences, vehicle operating parameters, and/or always-listening algorithms. Questions using a reverse cadence may include a tag question or a tag question phrase. The tag question phrase may be isolated to identify the reverse cadence. The tag question phrase would generally follow topics that require the answering service (i.e., a reverse cadence), while other interrogatories would generally precede topics that require the answering service (i.e., a forward cadence). Verbal utterances may be recorded and analyzed constantly to isolate topics and provide contexts. As the topics of potential questions are organized, the algorithm waits for an invocation such as, “Ford®, what do you think.” A tag question is any question that follows a statement as opposed to preceding a statement. A tag question phrase may include a moniker of whom the question is being asked. The always listening and tag question answering service provides a reverse cadence where the statement is made, the service is invocated, and then the answer is provided. The always listening and tag question answering service provides a forward cadence the service is invocated, where the statement is made, and then the answer is provided. The always listening service may also always search for answers to every topic so that the answer is readily available for presentation to the occupant.
- An inherent problem with providing forward and reverse answering services is that a conversation may include multiple answerable topics, and the occupants desired answer topic is not particularly clear. An example conversation is provided below:
-
- Suzy says, “The Detroit symphony orchestra is playing tonight in Detroit.” “Ford®, what do you think?” “I bet we have enough fuel to get to Detroit but it is going to be very cold.” (Reverse Cadence)
- Suzy says, “The Detroit symphony orchestra is playing tonight in Detroit.” “Hey Ford®, I bet we have enough fuel to get to Detroit but it is going to be very cold.” (Forward Cadence)
- The previous conversation included at least three topics 1) the Detroit symphony orchestra is playing tonight in Detroit; 2) I bet we have enough fuel to get to Detroit; and 3) it is going to be very cold. The topics may be isolated based on syntactical, categorical, or other methods. Over time, the topics may be distilled and isolated to particular contexts. The contexts may be broad categories of topics of which answers may be required. The contexts may also be presented to vehicle occupants for selection. The selection may also be provided via machine learning such that topics are selected according to previously selected topics. Meaning, the occupant may select the topic selection made, and then the machine learning algorithm would update the preferred contexts automatically. After the topic is selected based on the context, the vehicle may provide an answer or indication to the occupant by operation of the vehicle or display of the answer. Indeed, an always listening algorithm may be used to identify topics requiring answer service and automatically provide an answer to the topic after invocation by a tag question phrase.
-
FIG. 1 illustrates anexample system 100 including avehicle 102 implementing an always listening answer retrieval algorithm. Thevehicle 102 may include a vehicle computing system (VCS) 106 configured to communicate over a wide-area network using a telematics control unit (TCU) 120A. TheTCU 120A may havevarious modems 122 configured to communicate over respective communications paths and protocols. While anexample system 100 is shown inFIG. 1 , the example components as illustrated are not intended to be limiting. Indeed, thesystem 100 may have more or fewer components, and additional or alternative components and/or implementations may be used. - The
vehicle 102 may include various types of automobile, crossover utility vehicle (CUV), sport utility vehicle (SUV), truck, recreational vehicle (RV), boat, plane or other mobile machine for transporting people or goods. In many cases, thevehicle 102 may be powered by an internal combustion engine. As another possibility, thevehicle 102 may be a hybrid electric vehicle (HEV) powered by both an internal combustion engine and one or more electric motors, such as a series hybrid electric vehicle (SHEV), a parallel hybrid electrical vehicle (PHEV), or a parallel/series hybrid electric vehicle (PSHEV). As the type and configuration ofvehicle 102 may vary, the capabilities of thevehicle 102 may correspondingly vary. As some other possibilities,vehicles 102 may have different capabilities with respect to passenger capacity, towing ability and capacity, and storage volume. - The
VCS 106 may be configured to support voice command and BLUETOOTH interfaces with the driver and driver carry-on devices, receive user input via various buttons or other controls, and provide vehicle status information to a driver orother vehicle 102 occupants. Anexample VCS 106 may be the SYNC® system provided by FORD MOTOR COMPANY of Dearborn, Mich. - The
VCS 106 may further include various types of computing apparatus in support of performance of the functions of theVCS 106 described herein. In an example, theVCS 106 may include one or more processors configured to execute computer instructions, and a storage medium on which the computer-executable instructions and/or data may be maintained. A computer-readable storage medium (also referred to as a processor-readable medium or storage) includes any non-transitory (e.g., tangible) medium that participates in providing data (e.g., instructions) that may be read by a computer (e.g., by the processor(s)). In general, a processor receives instructions and/or data, e.g., from the storage, etc., to a memory and executes the instructions using the data, thereby performing one or more processes, including one or more of the processes described herein. Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and/or technologies, including, without limitation, and either alone or in combination, Java, C, C++, C#, Fortran, Pascal, Visual Basic, Python, Java Script, Perl, PL/SQL, etc. - The
VCS 106 may be configured to communicate withTCU 120A. TheTCU 120A may include a plurality ofmodems 122 capable of packet-switch or circuit-switched signaling. TheTCU 120A may control the operation of themodems 122 such that a suitable communication path is used. The modems may be configured to communicate over a variety of communications paths. The paths may be configured with circuit-switched 130, packet-switched 132, 134 signaling, or combination thereof. Packet-switchedcommunication 132, 134 paths may be Internet Protocol (IP)-based or use packet-based switching to transfer information. For example, the packet-switched communication may be long-term evolution (LTE) communications. In some circumstances the circuit-switch 130 communication path may be SIGTRAN or another implement, carrying circuit-switched signaling information over IP. The underlying signaling information is, however, still formatted under the circuit-switched protocol. - The
VCS 106 may also receive input from human-machine interface (HMI) controls 108 configured to provide for occupant interaction with thevehicle 102. For instance, theVCS 106 may interface with one or more buttons or other HMI controls 108 configured to invoke functions on the VCS 106 (e.g., steering wheel audio buttons, a push-to-talk button, instrument panel controls, etc.). TheVCS 106 may also drive or otherwise communicate with one ormore displays 110 configured to provide visual output to vehicle occupants, e.g., by way of a video controller. In some cases, thedisplay 110 may be a touch screen further configured to receive user touch input via the video controller, while in other cases thedisplay 110 may be a display only, without touch input capabilities. In an example, thedisplay 110 may be a head unit display included in a center console area of thevehicle 102 cabin. In another example, thedisplay 110 may be a screen of a gauge cluster of thevehicle 102. - The
VCS 106 may be further configured to communicate with other components of thevehicle 102 via one or more in-vehicle networks 112 orvehicle buses 112. The in-vehicle networks 112 may include one or more of a vehicle controller area network (CAN), an Ethernet network, and a media oriented system transfer (MOST), as some examples. The in-vehicle networks 112 may allow theVCS 106 to communicate withother vehicle 102 systems, such as a vehicle modem of theTCU 120A (which may not be present in some configurations), a global positioning system (GPS)module 120B configured to providecurrent vehicle 102 location and heading information, and various other vehicle ECUs configured to cooperate with theVCS 106. As some non-limiting possibilities, the vehicle ECUs may include a powertrain control module (PCM) 120C configured to provide control of engine operating components (e.g., idle control components, fuel delivery components, emissions control components, etc.) and monitoring of engine operating components (e.g., status of engine diagnostic codes); a body control module (BCM) 120D configured to manage various power control functions such as exterior lighting, interior lighting, keyless entry, remote start, and point of access status verification (e.g., closure status of the hood, doors and/or trunk of the vehicle 102); a radio transceiver module (RCM) 120E configured to communicate with key fobs or otherlocal vehicle 102 devices; a climate control management (CCM) 120F module configured to provide control and monitoring of heating and cooling system components (e.g., compressor clutch and blower fan control, temperature sensor information, etc.); and a battery control module (BACM) 120G configured to monitor the state of charge or other parameters of the battery 104 of thevehicle 102. - In an example, the
VCS 106 may be configured to access the communications features of theTCU 120A by communicating with theTCU 120A over avehicle bus 112. As some examples, thevehicle bus 112 may include a controller area network (CAN) bus, an Ethernet bus, or a MOST bus. In other examples, theVCS 106 may communicate with theserver 150 via aserver modem 152 using the communications services of themodems 122. - Referring to
FIG. 2 , thevehicle 102 may include anengine 113, starter-generator 114,battery 116, andelectrical loads 118. Thecontroller network 112 may connect to all of these vehicle systems through sensors (e.g.,fuel level sensor 115, oil sensor 117) or vehicle system controllers (e.g., 120A, 120B, 120C, 120D, 120E, 120F, 120G). For example, thecontroller network 112 may control the vehicle systems to provide autonomous control. Theengine 113 may have a direct mechanical linkage to the starter-generator 114. The starter-generator 114 may be electrically connected to thebattery 116 andelectrical loads 118. Thebattery 116 may be connected to the electrical loads 118. In response to overhearing the conversation mentioned above, theVCS 106 may recognize that the vehicle occupants desire to travel to Detroit if they have enough gas. TheVCS 106 may pull data from vehicle sensors (i.e., fuel level sensor 115) to determine the remaining fuel in the fuel tank. TheVCS 106 may then request the anticipated fuel consumption for the vehicle's 102 current location to the symphony orchestra. Indeed, thevehicle 102 can listen to the occupants' conversation and upon request provide a response without further requiring the question to be re-asked or the provision of additional information. - Referring to
FIGS. 3A-B , analgorithm 300 is shown. Thealgorithm 300 starts instep 302. An implementation of thealgorithm 300 may include additional or less steps, and the steps may be performed in a different order. The steps may also be performed simultaneously, at similar times, or sequentially. Instep 304, theVCS 106 or other processors collect verbal utterances. The verbal utterances may be sayings, statements, uttered words, or conversations available for capture by the microphone or array ofmicrophones 124 by occupants in or near the vehicle. Instep 306, topics are identified within the verbal utterances. The topics may be identified based on any natural language processing algorithm. Any part of speech may be used—or combination thereof—to determine the topics (e.g., nouns, verbs). The topics are identified to later be associated with the question asked. The topics may be portions of a sentence or entire sentences. The topics may be formed by verb-noun associations or other grammatical, syntactical, or semantical associations. - In
step 308, theVCS 106 waits for recognition of a moniker. A moniker may be any word or saying associated with the vehicle (e.g., manufacturer, model, software provider, infotainment system provider, branding associated with the vehicle or vehicle manufacturer). The moniker may be part of a tag question phrase or a tag question. - In
step 324, a context score is compared with a syntax score. The context score may be based on the topic score of identified topics instep 306. Meaning, the confidence that a given statement, phrase, or word is associated with a particular topic and context may be determined based on a variety of natural language and machine learning methods. For example, the statement “the Detroit symphony orchestra is playing tonight in Detroit” includes two topics. One topic in the statement may be the Detroit symphony orchestra. A second topic may be the city of Detroit. The context of the statement may be musical performance events. Because the statement is directly associated with the context, the context confidence score may be labeled as high, qualitatively, or above 75, quantitatively. Similarly, the statement “the Detroit art gallery is hosting an exposé tonight in Detroit” includes two similar topics that may have some relation to the musical performance events context. The context score in this situation may be markedly lower because the topic is not specifically musical performance, instead it is related to artistic performances. Therefore, the context confidence score may be labeled as medium, qualitatively, or above 50, quantitatively. The highest context score may be compared with the syntax score. - The syntax score may determine, syntactically, whether a tag question phrase exists and the moniker is part of the tag question phrase. For example, the tag question phrase may simply be, “Ford®?” Although Ford® comes at the end of a statement as a tag question phrase—indicating a reverse cadence—it could also be the beginning of a forward cadence. The algorithm may determine whether the moniker Ford® is syntactically at the beginning or end of a question using known and unknown methods. The confidence score of whether a forward or reverse cadence is being used may be labeled qualitatively or quantitatively as described above. A phrase with the highest syntax score may be, “(topic). What do you think, Ford®?” While a phrase with a low score may be, “Hey Ford®, (topic).” This is because a reverse cadence could almost never be associated with “Hey Ford®” preceding the topic, and a phrase like “What do you think, Ford®” leaves ambiguity as to whether the topic precedes or follows the phrase. Therefore, the syntax score is a representation of whether the cadence is forward or reverse.
- In
step 326 and step 328, non-interrogatory moniker utterances are filtered to prevent unintentional requests. If the moniker is not part of a tag question phrase (e.g., reverse cadence) or an active request (e.g., forward cadence), the algorithm returns to step 306. If the moniker is part of a tag question phrase according to a syntactical confidence score being above a predetermined threshold—or above a threshold set relative to the context score—the algorithm proceeds to step 338. If the moniker is not part of a tag question phrase according to a syntactical confidence score being above a predetermined threshold—or above a threshold set relative to the context score—the algorithm proceeds to step 328. - After a tag question phrase is detected in
step 326, the algorithm will select an already captured topic from the verbal utterances based on contexts, as defined insub-algorithm A 310.Sub-algorithm A 310 collects information to define contexts. Contexts may be categories of topics or other logical representations configured to represent classes of vehicle operating parameters. As shown instep 312,sub-algorithm A 310 identifies contexts within verbal utterances. Context identification may include nutrition information generally, while topic identification is more narrowly tuned to a question about nutrition in a candy bar. The list of contexts (e.g., nutrition, distance to destination, points of interest, weather) may be narrowly tuned for the vehicle instep 314 such that generic information requests are not available (e.g., answers to arithmetic, pronunciation of words). Meaning, broad question retrieval abilities may optionally be narrowed by the manufacturer or occupant under the assumption that vehicle or travel related questions will be present. - Similarly, in
step 316, vehicle operating parameters are analyzed to provide contexts. For example, the vehicle may be configured to make vehicle-specific parameter contexts available for the question-answer service. Meaning, contexts associated with oil life, fuel level, state of charge, climate status, engine temperature, or other vehicle parameters may be made available through contexts instep 316. A machine algorithm or manufacturer may select the operating parameters available for answer retrieval, instep 318. For example, oil temperature may be an available vehicle parameter, but a machine learning algorithm may determine that the context should not be made available because questions are so infrequently asked about engine oil temperature. Vehicle parameter contexts may be given a stronger weight that the verbal utterance contexts. - In
step 320 the contexts are presented to the user. Meaning, the user can further select which contexts it may desire to have answered by the answer service. The contexts may be presented using the HMI controls 108 or thedisplay screen 110. The contexts may be read by the system to the occupants and the occupants may provide confirmation of the proper context selection. For example, the vehicle may state, “Line 1: Weather.” The occupant may then verbally affirm that line one is the proper selection by saying, “One.” The contexts may be presented such that the most commonly used contexts are presented first. The contexts may also be presented in an order based on the verbal utterances already received and the frequency of the contexts being discussed. For example, if the weather in Detroit is a topic discussed along a road trip, the weather context may be presented to the user as a primary option. Instep 322 the user context selections are received for use instep 338. The contexts may be assigned weights to improve the answer service. For example, the heavily discussed weather context may be given a stronger weight than the sparsely discussed oil temperature. All other things being equal, the heavier weighted context will take topic selection precedence over the unweighted or less-weighted context in the topic selection process ofstep 338. - In
step 338, the selected contexts along with the identified topics are known. The algorithm may then recognize the topic to be determined based on the contexts. The topic selection may take into account the weights applied to the contexts that each topic resides in. For example, topics within the weather context may take precedence over the topics in the oil temperature context. Further, the topics syntactical and semantical strength may be weighted. For example, a confidence value of the topic may be determined based on the condition of the verbal utterance. Meaning, phrases that have grammatical coherence may fall within a stronger weighted context but be discounted because of the syntactical or semantical score. Additionally, proximity to the tag question phrase may be used to further weight the topic. For example, topics falling directly before the tag question phrase may have its score doubled or multiplied by a factor. Meaning, a context having a low weight may be selected over a context having a high weight if the topic immediately precedes the tag question phrase and the syntactical and semantical scores are low for the weather topic. - In
step 340, the selected topic having the highest confidence score is sent to theserver 150 to be answered. Theserver 150 provides the highest likely answer through the statistical and machine learning algorithms therein. Any answering service may provide the answer, and the answer does not need to be relative a vehicle. The answer may be sent back to the vehicle and presented to the occupants instep 342. Thevehicle 102 may then automatically operate the vehicle based on the answer or prompt the user select a course of action instep 344. For example, if the topic selected was “I bet we have enough fuel to get to Detroit” the vehicle may prepare a route to Detroit for the symphony and autonomously navigate the car to the destination. - In
step 328, theVCS 106 determines whether the moniker is part of a forward cadence. The forward cadence determination may be based on the comparison of context and syntactic scores. For example, the syntactical score may be high when the moniker is accompanied by a salutation (e.g., “Hey Ford®”). If the moniker is associated with a forward cadence instep 328, theVCS 106 will discard all utterances before invocation and store any utterances after invocation instep 330. Instep 332, theVCS 106 sends the query request determined from the stored verbal utterances to theserver 150. Theserver 150 provides the highest likely answer through the statistical and machine learning algorithms therein. Any answering service may provide the answer, and the answer does not need to be relative a vehicle. The answer may be sent back to the vehicle and presented to the occupants instep 334. Thevehicle 102 may then automatically operate the vehicle based on the answer or prompt the user select a course of action instep 336. For example, statement was, “Hey Ford®, do we have enough fuel to get to Detroit,” the vehicle may prepare a route to Detroit for the symphony and autonomously navigate the car to the destination upon an affirmative answer. - The words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the disclosure. As previously described, the features of various embodiments may be combined to form further embodiments of the invention that may not be explicitly described or illustrated. For example, an always listening algorithm may be used to identify topics requiring answer service and automatically provide an answer to the topic, which may not be based on an operating parameter of the vehicle. While various embodiments could have been described as providing advantages or being preferred over other embodiments or prior art implementations with respect to one or more desired characteristics, those of ordinary skill in the art recognize that one or more features or characteristics may be compromised to achieve desired overall system attributes, which depend on the specific application and implementation. These attributes may include, but are not limited to cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. As such, embodiments described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics are not outside the scope of the disclosure and may be desirable for particular applications.
Claims (20)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/987,175 US11704533B2 (en) | 2018-05-23 | 2018-05-23 | Always listening and active voice assistant and vehicle operation |
CN201910428175.5A CN110525363A (en) | 2018-05-23 | 2019-05-22 | Always it monitors and active voice assists and vehicle operating |
DE102019113681.4A DE102019113681A1 (en) | 2018-05-23 | 2019-05-22 | Always listening and active voice assistant and vehicle operation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/987,175 US11704533B2 (en) | 2018-05-23 | 2018-05-23 | Always listening and active voice assistant and vehicle operation |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190362217A1 true US20190362217A1 (en) | 2019-11-28 |
US11704533B2 US11704533B2 (en) | 2023-07-18 |
Family
ID=68499564
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/987,175 Active 2041-10-22 US11704533B2 (en) | 2018-05-23 | 2018-05-23 | Always listening and active voice assistant and vehicle operation |
Country Status (3)
Country | Link |
---|---|
US (1) | US11704533B2 (en) |
CN (1) | CN110525363A (en) |
DE (1) | DE102019113681A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11056113B2 (en) * | 2018-12-12 | 2021-07-06 | Hyundai Motor Company | Conversation guidance method of speech recognition system |
US20220128373A1 (en) * | 2020-10-26 | 2022-04-28 | Hyundai Motor Company | Vehicle and control method thereof |
Citations (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020010604A1 (en) * | 2000-06-09 | 2002-01-24 | David Block | Automated internet based interactive travel planning and reservation system |
US20100088254A1 (en) * | 2008-10-07 | 2010-04-08 | Yin-Pin Yang | Self-learning method for keyword based human machine interaction and portable navigation device |
US20100198590A1 (en) * | 1999-11-18 | 2010-08-05 | Onur Tackin | Voice and data exchange over a packet based network with voice detection |
US8009025B2 (en) * | 2003-11-20 | 2011-08-30 | Volvo Technology Corp | Method and system for interaction between a vehicle driver and a plurality of applications |
US20130185066A1 (en) * | 2012-01-17 | 2013-07-18 | GM Global Technology Operations LLC | Method and system for using vehicle sound information to enhance audio prompting |
US20140136187A1 (en) * | 2012-11-15 | 2014-05-15 | Sri International | Vehicle personal assistant |
US20150046418A1 (en) * | 2013-08-09 | 2015-02-12 | Microsoft Corporation | Personalized content tagging |
US9104537B1 (en) * | 2011-04-22 | 2015-08-11 | Angel A. Penilla | Methods and systems for generating setting recommendation to user accounts for registered vehicles via cloud systems and remotely applying settings |
US20150348551A1 (en) * | 2014-05-30 | 2015-12-03 | Apple Inc. | Multi-command single utterance input method |
US9250856B2 (en) * | 2014-04-21 | 2016-02-02 | Myine Electronics, Inc. | In-vehicle web presentation |
US20160159345A1 (en) * | 2014-12-08 | 2016-06-09 | Hyundai Motor Company | Vehicle and method of controlling the same |
US9396727B2 (en) * | 2013-07-10 | 2016-07-19 | GM Global Technology Operations LLC | Systems and methods for spoken dialog service arbitration |
US20160232221A1 (en) * | 2015-02-06 | 2016-08-11 | International Business Machines Corporation | Categorizing Questions in a Question Answering System |
US20160232890A1 (en) * | 2013-10-16 | 2016-08-11 | Semovox Gmbh | Voice control method and computer program product for performing the method |
US20170011742A1 (en) * | 2014-03-31 | 2017-01-12 | Mitsubishi Electric Corporation | Device and method for understanding user intent |
US20170068423A1 (en) * | 2015-09-08 | 2017-03-09 | Apple Inc. | Intelligent automated assistant in a media environment |
US20170109947A1 (en) * | 2015-10-19 | 2017-04-20 | Toyota Motor Engineering & Manufacturing North America, Inc. | Vehicle operational data acquisition responsive to vehicle occupant voice inputs |
US20170140757A1 (en) * | 2011-04-22 | 2017-05-18 | Angel A. Penilla | Methods and vehicles for processing voice commands and moderating vehicle response |
US20170162205A1 (en) * | 2015-12-07 | 2017-06-08 | Semiconductor Components Industries, Llc | Method and apparatus for a low power voice trigger device |
US9728188B1 (en) * | 2016-06-28 | 2017-08-08 | Amazon Technologies, Inc. | Methods and devices for ignoring similar audio being received by a system |
US20180061409A1 (en) * | 2016-08-29 | 2018-03-01 | Garmin Switzerland Gmbh | Automatic speech recognition (asr) utilizing gps and sensor data |
US20180096681A1 (en) * | 2016-10-03 | 2018-04-05 | Google Inc. | Task initiation using long-tail voice commands |
US20180182380A1 (en) * | 2016-12-28 | 2018-06-28 | Amazon Technologies, Inc. | Audio message extraction |
US20180232645A1 (en) * | 2017-02-14 | 2018-08-16 | Microsoft Technology Licensing, Llc | Alias resolving intelligent assistant computing device |
US10102844B1 (en) * | 2016-03-29 | 2018-10-16 | Amazon Technologies, Inc. | Systems and methods for providing natural responses to commands |
US20180350365A1 (en) * | 2017-05-30 | 2018-12-06 | Hyundai Motor Company | Vehicle-mounted voice recognition device, vehicle including the same, vehicle-mounted voice recognition system, and method for controlling the same |
US20190146491A1 (en) * | 2017-11-10 | 2019-05-16 | GM Global Technology Operations LLC | In-vehicle system to communicate with passengers |
US20190180740A1 (en) * | 2017-12-12 | 2019-06-13 | Amazon Technologies, Inc. | Architectures and topologies for vehicle-based, voice-controlled devices |
US10332513B1 (en) * | 2016-06-27 | 2019-06-25 | Amazon Technologies, Inc. | Voice enablement and disablement of speech processing functionality |
US20190237067A1 (en) * | 2018-01-31 | 2019-08-01 | Toyota Motor Engineering & Manufacturing North America, Inc. | Multi-channel voice recognition for a vehicle environment |
US20190279624A1 (en) * | 2018-03-09 | 2019-09-12 | International Business Machines Corporation | Voice Command Processing Without a Wake Word |
US10489393B1 (en) * | 2016-03-30 | 2019-11-26 | Amazon Technologies, Inc. | Quasi-semantic question answering |
US10515625B1 (en) * | 2017-08-31 | 2019-12-24 | Amazon Technologies, Inc. | Multi-modal natural language processing |
US10706846B1 (en) * | 2018-01-12 | 2020-07-07 | Amazon Technologies, Inc. | Question answering for a voice user interface |
US10733987B1 (en) * | 2017-09-26 | 2020-08-04 | Amazon Technologies, Inc. | System and methods for providing unplayed content |
US10762903B1 (en) * | 2017-11-07 | 2020-09-01 | Amazon Technologies, Inc. | Conversational recovery for voice user interface |
US10789948B1 (en) * | 2017-03-29 | 2020-09-29 | Amazon Technologies, Inc. | Accessory for a voice controlled device for output of supplementary content |
US10819667B2 (en) * | 2018-03-09 | 2020-10-27 | Cisco Technology, Inc. | Identification and logging of conversations using machine learning |
US10847149B1 (en) * | 2017-09-01 | 2020-11-24 | Amazon Technologies, Inc. | Speech-based attention span for voice user interface |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140195663A1 (en) | 2013-01-07 | 2014-07-10 | Sirius Xm Connected Vehicle Services Inc. | Method and System for Providing Cloud-Based Common Distribution Applications |
CN104349291A (en) | 2013-08-09 | 2015-02-11 | 富泰华工业(深圳)有限公司 | Electronic device and system and method for surfing Internet |
CN104363331A (en) | 2014-10-13 | 2015-02-18 | 惠州市德赛西威汽车电子有限公司 | Method and vehicular multimedia device allowing cellphone APP (application) startup by cellphone interconnection |
US20170262537A1 (en) | 2016-03-14 | 2017-09-14 | Amazon Technologies, Inc. | Audio scripts for various content |
-
2018
- 2018-05-23 US US15/987,175 patent/US11704533B2/en active Active
-
2019
- 2019-05-22 DE DE102019113681.4A patent/DE102019113681A1/en active Pending
- 2019-05-22 CN CN201910428175.5A patent/CN110525363A/en active Pending
Patent Citations (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100198590A1 (en) * | 1999-11-18 | 2010-08-05 | Onur Tackin | Voice and data exchange over a packet based network with voice detection |
US20020010604A1 (en) * | 2000-06-09 | 2002-01-24 | David Block | Automated internet based interactive travel planning and reservation system |
US8009025B2 (en) * | 2003-11-20 | 2011-08-30 | Volvo Technology Corp | Method and system for interaction between a vehicle driver and a plurality of applications |
US20100088254A1 (en) * | 2008-10-07 | 2010-04-08 | Yin-Pin Yang | Self-learning method for keyword based human machine interaction and portable navigation device |
US9104537B1 (en) * | 2011-04-22 | 2015-08-11 | Angel A. Penilla | Methods and systems for generating setting recommendation to user accounts for registered vehicles via cloud systems and remotely applying settings |
US20170140757A1 (en) * | 2011-04-22 | 2017-05-18 | Angel A. Penilla | Methods and vehicles for processing voice commands and moderating vehicle response |
US20130185066A1 (en) * | 2012-01-17 | 2013-07-18 | GM Global Technology Operations LLC | Method and system for using vehicle sound information to enhance audio prompting |
US20140136187A1 (en) * | 2012-11-15 | 2014-05-15 | Sri International | Vehicle personal assistant |
US9798799B2 (en) * | 2012-11-15 | 2017-10-24 | Sri International | Vehicle personal assistant that interprets spoken natural language input based upon vehicle context |
US9396727B2 (en) * | 2013-07-10 | 2016-07-19 | GM Global Technology Operations LLC | Systems and methods for spoken dialog service arbitration |
US20150046418A1 (en) * | 2013-08-09 | 2015-02-12 | Microsoft Corporation | Personalized content tagging |
US20160232890A1 (en) * | 2013-10-16 | 2016-08-11 | Semovox Gmbh | Voice control method and computer program product for performing the method |
US20170011742A1 (en) * | 2014-03-31 | 2017-01-12 | Mitsubishi Electric Corporation | Device and method for understanding user intent |
US9250856B2 (en) * | 2014-04-21 | 2016-02-02 | Myine Electronics, Inc. | In-vehicle web presentation |
US20150348551A1 (en) * | 2014-05-30 | 2015-12-03 | Apple Inc. | Multi-command single utterance input method |
US9966065B2 (en) * | 2014-05-30 | 2018-05-08 | Apple Inc. | Multi-command single utterance input method |
US20160159345A1 (en) * | 2014-12-08 | 2016-06-09 | Hyundai Motor Company | Vehicle and method of controlling the same |
US20160232221A1 (en) * | 2015-02-06 | 2016-08-11 | International Business Machines Corporation | Categorizing Questions in a Question Answering System |
US20170068423A1 (en) * | 2015-09-08 | 2017-03-09 | Apple Inc. | Intelligent automated assistant in a media environment |
US9875583B2 (en) * | 2015-10-19 | 2018-01-23 | Toyota Motor Engineering & Manufacturing North America, Inc. | Vehicle operational data acquisition responsive to vehicle occupant voice inputs |
US20170109947A1 (en) * | 2015-10-19 | 2017-04-20 | Toyota Motor Engineering & Manufacturing North America, Inc. | Vehicle operational data acquisition responsive to vehicle occupant voice inputs |
US20170162205A1 (en) * | 2015-12-07 | 2017-06-08 | Semiconductor Components Industries, Llc | Method and apparatus for a low power voice trigger device |
US10102844B1 (en) * | 2016-03-29 | 2018-10-16 | Amazon Technologies, Inc. | Systems and methods for providing natural responses to commands |
US10489393B1 (en) * | 2016-03-30 | 2019-11-26 | Amazon Technologies, Inc. | Quasi-semantic question answering |
US10332513B1 (en) * | 2016-06-27 | 2019-06-25 | Amazon Technologies, Inc. | Voice enablement and disablement of speech processing functionality |
US9728188B1 (en) * | 2016-06-28 | 2017-08-08 | Amazon Technologies, Inc. | Methods and devices for ignoring similar audio being received by a system |
US20180061409A1 (en) * | 2016-08-29 | 2018-03-01 | Garmin Switzerland Gmbh | Automatic speech recognition (asr) utilizing gps and sensor data |
US20180096681A1 (en) * | 2016-10-03 | 2018-04-05 | Google Inc. | Task initiation using long-tail voice commands |
US20180182380A1 (en) * | 2016-12-28 | 2018-06-28 | Amazon Technologies, Inc. | Audio message extraction |
US20180232645A1 (en) * | 2017-02-14 | 2018-08-16 | Microsoft Technology Licensing, Llc | Alias resolving intelligent assistant computing device |
US10789948B1 (en) * | 2017-03-29 | 2020-09-29 | Amazon Technologies, Inc. | Accessory for a voice controlled device for output of supplementary content |
US20180350365A1 (en) * | 2017-05-30 | 2018-12-06 | Hyundai Motor Company | Vehicle-mounted voice recognition device, vehicle including the same, vehicle-mounted voice recognition system, and method for controlling the same |
US10515625B1 (en) * | 2017-08-31 | 2019-12-24 | Amazon Technologies, Inc. | Multi-modal natural language processing |
US10847149B1 (en) * | 2017-09-01 | 2020-11-24 | Amazon Technologies, Inc. | Speech-based attention span for voice user interface |
US10733987B1 (en) * | 2017-09-26 | 2020-08-04 | Amazon Technologies, Inc. | System and methods for providing unplayed content |
US10762903B1 (en) * | 2017-11-07 | 2020-09-01 | Amazon Technologies, Inc. | Conversational recovery for voice user interface |
US20190146491A1 (en) * | 2017-11-10 | 2019-05-16 | GM Global Technology Operations LLC | In-vehicle system to communicate with passengers |
US20190180740A1 (en) * | 2017-12-12 | 2019-06-13 | Amazon Technologies, Inc. | Architectures and topologies for vehicle-based, voice-controlled devices |
US10706846B1 (en) * | 2018-01-12 | 2020-07-07 | Amazon Technologies, Inc. | Question answering for a voice user interface |
US20190237067A1 (en) * | 2018-01-31 | 2019-08-01 | Toyota Motor Engineering & Manufacturing North America, Inc. | Multi-channel voice recognition for a vehicle environment |
US20190279624A1 (en) * | 2018-03-09 | 2019-09-12 | International Business Machines Corporation | Voice Command Processing Without a Wake Word |
US10819667B2 (en) * | 2018-03-09 | 2020-10-27 | Cisco Technology, Inc. | Identification and logging of conversations using machine learning |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11056113B2 (en) * | 2018-12-12 | 2021-07-06 | Hyundai Motor Company | Conversation guidance method of speech recognition system |
US20220128373A1 (en) * | 2020-10-26 | 2022-04-28 | Hyundai Motor Company | Vehicle and control method thereof |
Also Published As
Publication number | Publication date |
---|---|
US11704533B2 (en) | 2023-07-18 |
DE102019113681A1 (en) | 2019-11-28 |
CN110525363A (en) | 2019-12-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105957522B (en) | Vehicle-mounted information entertainment identity recognition based on voice configuration file | |
US10351009B2 (en) | Electric vehicle display systems | |
US9798799B2 (en) | Vehicle personal assistant that interprets spoken natural language input based upon vehicle context | |
US9085303B2 (en) | Vehicle personal assistant | |
US10290300B2 (en) | Text rule multi-accent speech recognition with single acoustic model and automatic accent detection | |
US11120650B2 (en) | Method and system for sending vehicle health report | |
CN110660397A (en) | Dialogue system, vehicle, and method for controlling vehicle | |
US20190237069A1 (en) | Multilingual voice assistance support | |
US20190122661A1 (en) | System and method to detect cues in conversational speech | |
CN110648661A (en) | Dialogue system, vehicle, and method for controlling vehicle | |
US20190019516A1 (en) | Speech recognition user macros for improving vehicle grammars | |
US20190311713A1 (en) | System and method to fulfill a speech request | |
CN110033380A (en) | Based on the insurance guide system used | |
US11358603B2 (en) | Automated vehicle profile differentiation and learning | |
US11704533B2 (en) | Always listening and active voice assistant and vehicle operation | |
CN110503948A (en) | Conversational system and dialog process method | |
CN103802761A (en) | Method for activating a voice interaction with a passenger of a motor vehicle and voice interaction system for a vehicle | |
US20190362218A1 (en) | Always listening and active voice assistant and vehicle operation | |
JP2019127192A (en) | On-vehicle device | |
CN110503947A (en) | Conversational system, the vehicle including it and dialog process method | |
CN111724798A (en) | Vehicle-mounted device control system, vehicle-mounted device control apparatus, vehicle-mounted device control method, and storage medium | |
WO2020208397A1 (en) | Voice control of vehicle systems | |
CN112534499B (en) | Voice conversation device, voice conversation system, and method for controlling voice conversation device | |
US20210056656A1 (en) | Routing framework with location-wise rider flexibility in shared mobility service system | |
US11507346B1 (en) | Intelligent text and voice feedback for voice assistant |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FORD GLOBAL TECHNOLOGIES, LLC, MICHIGAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GANDIGA, SANDEEP RAJ;REEL/FRAME:045882/0593 Effective date: 20180508 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PRE-INTERVIEW COMMUNICATION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |