Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/28—Constructional details of speech recognition systems
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/16—Sound input; Sound output
  • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/28—Constructional details of speech recognition systems
  • G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04W—WIRELESS COMMUNICATION NETWORKS
  • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
  • H04W4/30—Services specially adapted for particular environments, situations or purposes
  • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
  • H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
  • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
  • B60R16/037—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
  • B60R16/0373—Voice control

Definitions

• the invention is related to a method for providing a functionality of a digital assistant in a vehicle according to the independent method claim. Further, the invention is related to a system for a vehicle for providing a functionality of a digital assistant in the vehicle according to the independent system claim. Furthermore, the invention is related to a vehicle comprising a corresponding system according to the independent device claim. Moreover, the invention is related to an electronic control device according to the further independent device claim as well as to a computer program product according to the independent product claim.
• Intelligent personal assistants are basically known.
• An Intelligent personal assistant is as a voice assistant that enables information to be retrieved, dialogs to be conducted and assistance services to be provided by means of communication in natural, human language by performing a speech analysis for speech recognition, interpreting this semantically, processing it logically and, as a result, formulating an answer through speech synthesis.
• Such applications are used especially on mobile devices like smartphones.
• Mobile devices do not have access to the applications running in a vehicle like a built-in application has. Voice control of in-vehicle facilities like the climatization is not possible.
• a mobile device often is not at hand (e.g. it is in the pocket) when the user wants to access their digital assistant while driving.
• a mobile device does not have an optimized microphone setup in the vehicle yielding to an error prone activation word recognition by the proxy and speech command recognition by the digital assistant.
• the response of the digital assistant is played back on the mobile device’s loudspeaker which is low in volume and interferes with the vehicle’s audio system.
• a mobile device has inferior radio signal reception and transmission characteristics compared to the vehicle’s radio signal transmission/reception characteristics. The user is distracted by handling the mobile device when using the digital assistant while driving.
• the aim of the present invention is to provide a method for providing an extended functionality of a digital assistant in a vehicle, with improved acoustic features, convenient use and improved safety during the use, preferably in an easy, simple and intuitive way. Also, the aim of the invention is to provide an improved system for a vehicle for providing a functionality of a digital assistant in the vehicle. Besides, the aim of the invention is to provide a vehicle comprising a corresponding system. Further, the aim of the invention is to provide an electronic control device and a computer program product for a corresponding method.
• embodiments of the invention provide a method for providing a functionality of a digital assistant in a vehicle with the features of the independent method claim.
• embodiments of the invention provide a system for a vehicle for providing a functionality of a digital assistant in the vehicle with the features of the independent system claim.
• embodiments of the invention provide a vehicle comprising a corresponding system with the features of the independent device claim.
• embodiments of the invention provide a corresponding electronic control device for a corresponding method with the features of the second independent device claim.
• embodiments of the invention provide a computer program product for a corresponding method with the features of the independent product claim. Details and features disclosed on individual aspects of the invention also apply to the other aspects of the invention and vice versa.
• embodiments of the invention provide a method for providing a functionality of a digital assistant in a vehicle, the method comprising: providing a recording device installed in the vehicle for recording a voice request (or command) of a user, providing a (special) proxy server for the digital assistant in an edge gateway of an electronic control device of the vehicle, transmitting the voice request of the user from the proxy server to a digital assistant server (in a backend device) for performing a voice analysis and providing an answer (or reply) as a result of the analysis, receiving the answer to the voice request from digital assistant server at the proxy server, transmitting the answer from the proxy server to a play back device installed in the vehicle.
• Embodiments of the method may be carried out in the given order or in a modified order. Individual actions of the method may be carried out simultaneously and/or repeatedly to allow a flowing process.
• a backend device within the meaning of the invention is a computer or a group of computers providing a high computational power, a large memory and easy access to updates as well as to a digital assistant software that may be used in the vehicle.
• the backend device within the meaning of the invention is the representative of the cloud.
• the idea is to provide a proxy server (or simply a proxy) that interfaces with a digital assistant in the cloud.
• the invention proposes to allocate the proxy to the edge gateway of an electronic control device of the vehicle and within the edge gateway of an electronic control device especially to the loT Edge runtime for the sake of easy update possibilities which is paramount if new skills are to be added to the proxy.
• loT Edge runtime is designed to support loT data traffic.
• the loT traffic is bulky, there are packets of huge size that can block transmission of voice commands for seconds. Therefore, it may be preferably provided that the traffic from the loT Edge runtime to the cloud shall permit to suspend the transmission of large loT information packets in favor of higher priority audio command data.
• the audio stream coming from the vehicle may be aggregated to larger packets by the proxy in the loT Edge runtime.
• the speech command reply may be directly sent to the element which plays back the speech command reply.
• the action command (so called “raw action command”) may be sent to the proxy and within the proxy further detailed according to the available skills and vehicle capabilities.
• the edge gateway of an electronic control device aggregates all incoming and outgoing data traffic streams and thus the proxy arranged there may prioritize the audio data over e.g. bulk data transfer.
• Speech data comes usually in small to very small data chunks, for example a coded audio data stream with 16kbit/s and a typical audio frame length of 20ms. Such a frame has 320 data bits/40 data bytes. However, the overhead of sending such a packet is already larger than the 40 data bytes. And response time matters but not to such extend of some 10s of ms.
• the invention may allow that said audio frames encompassing 100ms of speech are aggregated into one transmit frame.
• the speech reply from the digital assistant may be directly sent to the playback instance (usually the head device) installed in the vehicle, having improved sound characteristics.
• the action command (“raw action command”) may be further translated into a vehicle specific command sequence, for example machine language.
• the location of the proxy in the edge gateway of an electronic control device promotes faster access to all components in the vehicle.
• high priority is assigned to audio data of the voice request and/or of the answer among different data to be transmitted between the vehicle and the cloud.
• the voice requests of the user who wants to gain access to the digital assistant may be transmitted from the vehicle to the cloud prior to bulk data packets for other services which cause a network traffic jam between the vehicle and the backend device.
• the answers of the digital assistant server may be transmitted from the backend device to the vehicle prior to other data coming from the backend device.
• audio data of the voice request are aggregated into: a defined transmit frame, a determined length and/or a defined transmit format.
• the method comprises at least one of the following: play back of the answer for the user by the play back device, especially by a head device of the vehicle, and/or visualize the answer for the user by a display device installed in the vehicle.
• the play back device of the vehicle with improved acoustic characteristics may be used for the reply to the user command.
• the method comprises at least one of the following: providing an action command in the vehicle according to the answer from the digital assistant server, preferably by the proxy server for the digital assistant and/or by the digital assistant server.
• providing an action command in the vehicle preferably by the proxy server for the digital assistant and/or by the digital assistant server.
• the extended services and/or improved skills of the vehicle may be controlled using digital assistant functionality in an easy and convenient way by voice requests.
• An action command may be provided in the backend device and/or in the vehicle itself.
• the action command is transformed to: a specific vehicle command sequence, preferably by the proxy server for the digital assistant.
• an action command provided by the backend device may be translated into the language, which the corresponding device of the vehicle understands for controlling functions.
• the method comprises at least one of the following: verbalize an action command according to the answer from the digital assistant server for the user by the play back device, especially by a head device of the vehicle, and/or visualize of an action command according to the answer from the digital assistant server for the user by a display device installed in the vehicle.
• the method comprises at least one of the following: providing a confirmation possibility for the user to approve an action command in the vehicle according to the answer from the digital assistant server through using the recording device and/or a display device and/or an input device installed in the vehicle.
• the method comprises at least one of the following: execute an action command in the vehicle according to the answer from the digital assistant server, especially upon a confirmation from the user of the action command, preferably by an electronic control device of the vehicle.
• execute an action command in the vehicle according to the answer from the digital assistant server, especially upon a confirmation from the user of the action command, preferably by an electronic control device of the vehicle.
• the method comprises at least one of the following: provide at least one alternative action command according to the answer from the digital assistant server, especially upon a rejection from the user of the action command, preferably by the proxy server for the digital assistant, and/or invite the user to provide a new voice request by the play back device and/or a display device through using the recording device and/or a display device and/or an input device installed in the vehicle, especially upon a rejection from the user of the action command.
• fail back positions may be provided for the case that the user voice request was not interpreted correctly.
• the method may be executed for synchronization of a digital calendar of the user in the backend device 20, in the vehicle 10 and/or in a mobile device 30 of a user.
• enhanced use of the method may be provided.
• embodiments of the invention provide a system for a vehicle for providing a functionality of a digital assistant in the vehicle, comprising: a recording device installed in the vehicle for recording a voice request of a user, a special proxy server for the digital assistant in an edge gateway (of an electronic control device) of the vehicle, wherein the proxy server is configured for transmitting the voice request of the user from the vehicle to a digital assistant server (in a backend device) for performing a voice analysis and providing an answer as a result of the analysis, wherein the proxy server is configured for receiving the answer to the voice request from digital assistant server at the vehicle, and wherein the proxy server is configured for transmitting the answer from the proxy server to a play back device installed in the vehicle, and the play back device for verbalizing the answer for the user.
• the same advantages may be achieved as described above in the context of the inventive method. Full reference is made to these advantages in the present case.
• the proxy server is configured for providing: an action command in the vehicle according to the answer from the digital assistant server, and/or a transformation of an action command in the vehicle according to the answer from the digital assistant server into a specific vehicle command sequence, at least one alternative action command according to the answer from the digital assistant server, and/or especially upon a rejection from the user of the action command, an invitation for the user to provide a new voice request, especially upon a rejection from the user of the action command.
• the proxy server may be configured for synchronizing a digital calendar of the user in the backend device, in the vehicle and/or in a mobile device of a user, such as a smartphone.
• improved functionality of the digital assistant may be provided in the vehicle using different services and features of the vehicle.
• Embodiments of the invention provide, according to the third aspect, a vehicle comprising a corresponding system. With such a vehicle the same advantages may be achieved as with the system described above. Full reference is made to these advantages in the present case.
• embodiments of the invention provide an electronic control device comprising: a memory, in which a program code is stored, and a computing device, wherein when carrying out the program code by the computing device, a method is performed as described above.
• a memory in which a program code is stored
• a computing device wherein when carrying out the program code by the computing device, a method is performed as described above.
• the invention provides a computer program product comprising a program code for carrying out a method as described above.
• the computer program product comprises instructions which, when the program is executed by a computer, cause the computer to carry out embodiments of the method described above.
• the same advantages may be achieved as with corresponding embodiments of the method described above. Full reference is made to these advantages in the present case.
• Fig. 1 a schematic design of a system for a vehicle within the meaning of the invention configured for providing a functionality of a digital assistant (DA) in the vehicle.
• DA digital assistant
• Fig. 1 shows a system 100 for a vehicle 10 for providing a functionality of a digital assistant DA in the vehicle 10, a recording device 11 installed in the vehicle 10 for recording a voice request VR of a user, a special proxy server DAP for the digital assistant DA in an edge gateway GW of an electronic control device ECU of the vehicle 10, wherein the proxy server DAP is configured for transmitting the voice request VR of the user from the vehicle 10 to a digital assistant server DAS in a backend device 20 for performing a voice analysis and providing an answer AW as a result of the analysis, wherein the proxy server DAP is configured for receiving the answer AW to the voice request VR from digital assistant server DAS at the vehicle 10, and wherein the proxy server DAP is configured for transmitting the answer AW from the proxy server DAP to a play back device 12 installed in the vehicle 10, and the play back device 12 for verbalizing the answer AW for the user.
• the proxy server DAP is configured for transmitting the voice request VR of the user from the vehicle 10 to a digital assistant server DAS
• the proxy server DAP may be configured for providing: an action command ACT in the vehicle 10 according to the answer AW from the digital assistant server DAS, and/or a transformation of an action command ACT in the vehicle 10 according to the answer AW from the digital assistant server DAS into a specific vehicle command sequence, at least one alternative action command AACT according to the answer AW from the digital assistant server DAS, and/or especially upon a rejection from the user of the action command ACT, an invitation for the user to provide a new voice request VR, especially upon a rejection from the user of the action command ACT.
• the proxy server DAP may be configured for synchronizing a digital calendar of the user in the backend device 20, in the vehicle 10 and/or in a mobile device 30 of a user, such as a smartphone.
• a backend device 20 may be a cloud computer providing a high computational power, a large memory and easy access to updates as well as to a digital assistant software that may be used in the vehicle. The backend device 20 is thus the representative of the cloud.
• the idea is to provide a proxy server DAP (or simply a proxy) that interfaces with the digital assistant DA in the backend device 20 arranged in the cloud.
• the invention proposes to allocate the proxy to the edge gateway GW of an electronic control device ECU of the vehicle 10 and within the edge gateway GW of an electronic control device ECU especially to the loT Edge runtime for the sake of easy update possibilities which is paramount if new skills are to be added to the proxy.
• the loT traffic can be bulky, it may be preferably provided according to some embodiments of the invention that the traffic from the loT Edge runtime to the cloud suspends the transmission of large loT information packets in favor of higher priority audio data of the user voice request or commands.
• the audio stream coming from the vehicle may be aggregated to larger packets by the proxy server DAP in the loT Edge runtime.
• the voice request reply (that is answer from the backend device 20) may be directly sent to the element installed in the vehicle 10 which plays back the answer with better sound quality, such as head device as a possible play back device 12 of the vehicle 10.
• an action command (so called “raw action command”) corresponding to the answer to the voice request may be sent to the proxy server DAP and within the proxy server DAP further detailed according to the available skills and vehicle capabilities.
• the latency between spoken command and reaction played back on the audio system within the vehicle may be drastically reduced.
• the performance of the digital assistant functions may be preferably increased.
• the edge gateway GW of an electronic control device ECU aggregates all incoming and outgoing data traffic streams and thus the proxy arranged there may easily prioritize the audio data over e.g. bulk data transfer.
• the invention may allow that audio data may be aggregated into one transmit frame for better throughput of the audio data.
• the speech reply from the digital assistant may be directly sent to the playback instance (usually the head device) installed in the vehicle, having improved sound properties.
• the action command (“raw action command”) may be further translated into a vehicle specific command sequence, for providing extended functionality of the vehicle with the help of the digital assistant DA.
• the location of the proxy server DAP in the edge gateway GW of the electronic control device ECU provides faster access to all components in the vehicle.
• the vehicle 10 comprising a corresponding system 100 provides the third aspect of the invention.
• An electronic control device ECU comprising a memory, in which a program code is stored, and a computing device, wherein when carrying out the program code by the computing device, a method is performed as described above, provides the fourth aspect of the invention.
• the electronic control device ECU is configured in such a way to execute embodiments of the method as described above.
• a computer program product comprising a program code for carrying out a method as described above provides an aspect of the invention.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Computational Linguistics (AREA)
• Theoretical Computer Science (AREA)
• General Health & Medical Sciences (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• Signal Processing (AREA)
• Mobile Radio Communication Systems (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=77838855

Family Applications (1)

Country Status (2)

Family Cites Families (1)

• 2021
  • 2021-09-07 EP EP21773372.4A patent/EP4377953A1/de active Pending
  • 2021-09-07 WO PCT/EP2021/074599 patent/WO2023036401A1/en unknown

Also Published As

Similar Documents

Legal Events