EP3807803A1 - Device, system and method for causing an output device to provide information for voice command functionality - Google Patents
Device, system and method for causing an output device to provide information for voice command functionalityInfo
- Publication number
- EP3807803A1 EP3807803A1 EP19749915.5A EP19749915A EP3807803A1 EP 3807803 A1 EP3807803 A1 EP 3807803A1 EP 19749915 A EP19749915 A EP 19749915A EP 3807803 A1 EP3807803 A1 EP 3807803A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- textual
- suggested
- terms
- received
- term
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/1822—Parsing for meaning understanding
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/063—Training
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
- G10L15/187—Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/225—Feedback of the input speech
Definitions
- Voice command functionality at devices is becoming ubiquitous. However, when similar phrases are used to cause different functions to be activated, a voice recognition engine may return an incorrect command and/or activate an incorrect function.
- FIG. 1 depicts a system for causing an output device to provide information for voice command functionality in accordance with some embodiments.
- FIG. 2 depicts an example device for causing an output device to provide information for voice command functionality in accordance with some embodiments.
- FIG. 3 depicts a flowchart of a method for causing an output device to provide information for voice command functionality in accordance with some embodiments.
- FIG. 4 depicts an example graphic user interface which may be used to input a received textual term in accordance with some embodiments.
- FIG. 5 depicts generation of one or more suggested textual terms, related to the received textual term, that minimizes phonetic similarities with the one or more existing textual terms in accordance with some embodiments.
- FIG. 6 depicts causing an output device to provide an indication of the one or more suggested textual terms to use in place of the received textual term in accordance with some embodiments.
- FIG. 7 depicts acceptance of one of the one or more suggested textual terms to use in place of the received textual term and association with a given voice command functionality in accordance with some embodiments.
- FIG. 8 depicts receipt of an accepted suggested textual terms at a microphone of a communication device and automatic implementation of associated functionality in accordance with some embodiments.
- An aspect of the specification provides a device comprising: one or more of an input device and a communications unit; and a controller communicatively coupled to one or more of the input device and the communications unit, the controller configured to: when a received textual term, received via one or more of the input device and the communications unit, is phonetically similar to one or more existing textual terms used for activating functionality at a communication device using a voice recognition algorithm: generate one or more suggested textual terms, related to the received textual term, that minimizes phonetic similarities with the one or more existing textual terms; and cause an output device to provide an indication of the one or more suggested textual terms to use in place of the received textual term.
- Another aspect of the specification provides a method comprising:
- FIG. 1 depicts a system 100 for causing an output device to provide information for voice command functionality.
- the system 100 comprises a computing device 101, a provisioning device 103 and a
- the provisioning device 103 is configured to communicate with the computing device 101 via a communication network 107 (interchangeably referred to hereafter as the network 107), and further configured to communicate with the communication device 105, using respective wired and/or wireless communication links.
- the provisioning device 103 is generally configured to provision the communication device 105, as described in further detail below.
- Communication links are depicted in system 100 as arrows between the components of the system 100 and the network 107.
- the provisioning device 103 may be local to the communication device 105, however in other examples, the provisioning device 103 and the communication device 105 may also be in communication via the network 107.
- the computing device 101 may comprise one or more cloud computing devices and/or servers, and may be operated by a public service and/or emergency service entity, such as a police service entity, a fire service entity, an emergency medical service entity, and the like. However, the computing device 101 may be operated by another type of entity including, but not limited to, a commercial entity and/or business entity. In some examples, the computing device 101 may be optional.
- a public service and/or emergency service entity such as a police service entity, a fire service entity, an emergency medical service entity, and the like.
- the computing device 101 may be operated by another type of entity including, but not limited to, a commercial entity and/or business entity.
- the computing device 101 may be optional.
- the provisioning device 103 may comprise a computing device, including, but not limited to, a personal computer, a laptop computer, and the like.
- the provisioning device 103 may be generally configured to provision the communication device 105, for example to provision the communication device 105 at the beginning of a shift when the communication device 105 is checked out of a storage area by a user and the like.
- the provisioning device 103 may be used to provision channels and/or talkgroups at the communication device 105.
- the computing device 101 may be configured to provision the communication device 105 and/or the communication device 105 may be configured to provision itself; in these examples, provisioning device 103 may be optional.
- present examples are described with reference to the provisioning device 103 provisioning channels and/or talkgroups at the communication device 105
- the communication device 105 may generally comprise a radio and/or mobile device used by first responders, and the like, and may be generally configured to communicate using channels and/or talkgroups, and the like.
- other types of communication devices are within the scope of the present specification including, but not limited to, commercial and/or business-related communication devices.
- the communication device 105 is generally configured for voice command functionality.
- the communication device 105 generally comprises a microphone 133, and given functions of the communication device 105 may be activated upon receipt of voice at the microphone 133 that includes a voice command, the voice commands recognized using a voice recognition algorithm at the communication device 105.
- the communication device 105 comprise a radio
- voice command functionality may include, but are not limited to, activating one or more channels and/or talkgroups upon receipt of a name of the channel and/or talkgroup at the microphone 133 (e.g. the channel and/or talkgroup name may comprise a command for activating the associated channel and/or talkgroup).
- voice command functionality may be particularly pertinent in communication devices and/or radios used by public service personnel and/or first responders, such as police, fire fighters, emergency medical technicians and the like.
- a first responder may be attempting to rapidly initiate communications with other first responders and may be using voice recognition at the communication device 105 to quickly establish such communications by speaking a name of a channel and/or a talkgroup into the microphone 133.
- names of channels and/or talkgroups are similar, an unintended channel and/or talkgroup may be activated which may lead to confusion and/or poor communications in such emergency situations.
- confusion and/or poor communications may lead to a worsening of an emergency situation and/or result in a waste of processing resources, radio resources and/or bandwidth resources at the communication device 105 and/or in a
- one or more of the computing device 101, the provisioning device 103 and the communication device 105 is generally configured to cause an output device to provide information for voice command functionality, for example during a provisioning process; in particular, as described in more detail below, the information for voice command functionality may include providing alternative suggestions for channel and/or talkgroup names, when a received channel and/or talkgroup name is phonetically similar to one or more existing channel and/or talkgroup names.
- voice command functionality is described with respect to channel and/or talkgroup names of radios, techniques for causing an output device to provide information for voice command functionality as described herein may include any type of voice command functionality at any type of device that uses voice command functionality.
- alternative suggestions for voice command functionality may be provided at an output device of the communication device 105, such as a display screen 135 and/or a speaker 136; however, when the provisioning device 103 is being used to provision the communication device 105, the alternative suggestions may be provided at a display screen and/or speaker (not depicted) of the provisioning device 103.
- FIG. 2 sets forth a schematic diagram that illustrates an example of a communication and/or computing device 201
- the device 201 and/or the example device 201 may have a configuration similar to the example device 201.
- the example device 201 is described with respect to including certain components, it is understood that the example device 201 may be configured according to the functionality of a specific device and the configuration of the example device 201 may be adapted for a particular respective configuration of the devices 101, 103, 105.
- the computing device 101 may not include input devices and/or output devices (such as a display screen, speakers, microphones, keyboards and the like) and/or a Global Positioning (GPS) unit, and the provisioning device 103 may not include a GPS unit; other variations and configurations of the example device 201 adapted for functionality of one or more of the devices 101, 103, 105 are within the scope of the present specification.
- input devices and/or output devices such as a display screen, speakers, microphones, keyboards and the like
- GPS Global Positioning
- the example device 201 generally includes a communications unit 202, a processing unit 203, a Random Access Memory (RAM) 204, a display screen 205, an input device 206, one or more wireless transceivers 208, one or more wired and/or wireless input/output (I/O) interfaces 209, a combined modulator/demodulator 210, a code Read Only Memory (ROM) 212, a common data and address bus 217, a controller 220, a static memory 222 storing one or more applications 223 (which may include one or more machine learning algorithms) and one or more voice recognition algorithms 224, a speaker 228, an imaging device 232, a microphone 233 and a GPS unit 234.
- ROM Read Only Memory
- the one or more applications 223 will be interchangeably referred to hereafter as the application 223, though different applications 223 may be used for different modes of the example device 201, as described in further detail below.
- the one or more voice recognition algorithms 224 will be interchangeably referred to hereafter as the voice recognition algorithm 224, though different or more voice recognition algorithms 224 may be used for different modes of the example device 201; furthermore, the voice recognition algorithm 224 may be present when the example device 201 includes the communication device 105, however, the voice recognition algorithm 224 may not be present when the example device 201 includes the computing device 101 and/or the provisioning device 103.
- the voice recognition algorithm 224 includes any suitable voice recognition algorithm for recognizing given words in voice received at the microphone 233 (and/or at the microphone 133).
- the example device 201 is described hereafter in further detail. As shown in FIG. 2, the example device 201 includes the communications unit 202 coupled to the common data and address bus 217 of the processing unit 203. The example device 201 may also include the one or more input devices 206 (e.g., keypad, pointing device, touch-sensitive surface, etc.) and the display screen 205 (which, in some examples, may be a touch screen and thus also act as an input device 206), each coupled to be in communication with the processing unit 203.
- input devices 206 e.g., keypad, pointing device, touch-sensitive surface, etc.
- the display screen 205 which, in some examples, may be a touch screen and thus also act as an input device 206
- the speaker 228 may be present for reproducing audio that is decoded from voice or audio streams of calls received via the communications unit 202 for example to provide navigation instructions.
- the display screen 205 and/or the speaker 228 may be caused to provide an indication of one or more suggested textual terms to use in place of a received textual term as described in more detail below.
- the imaging device 232 is generally optional and, when present, may provide video (still or moving images) of an area in a field of view of the example device 201 for further processing by the processing unit 203 and/or for further transmission by the communications unit 202.
- the microphone 233 is generally configured for capturing audio from a user and/or other environmental or background audio that may be further processed by the processing unit 203 and/or is transmitted as voice or audio stream data, or as acoustical environment indications, by the communications unit 202. Furthermore, when the example device 201 is embodied as the communication device 105, the microphone 233 may capture voice commands to activate functionality of the communication device 105.
- the processing unit 203 may include the code Read Only Memory (ROM) 212 coupled to the common data and address bus 217 for storing data for initializing system components.
- the processing unit 203 may further include the controller 220 coupled, by the common data and address bus 217, to the Random- Access Memory (RAM) 204 and a static memory 222.
- ROM Read Only Memory
- RAM Random- Access Memory
- the communications unit 202 may include one or more wired and/or wireless input/output (I/O) interfaces 209 that are configurable to communicate with other devices.
- I/O input/output
- the communications unit 202 may include one or more wireless transceivers 208, such as a digital mobile radio (DMR) transceiver, a Project 25 (P25) transceiver, a terrestrial trunked radio (TETRA) transceiver, a Bluetooth transceiver, a Wi-Fi transceiver, for example operating in accordance with an IEEE 802.11 standard (e.g., 802.1 la, 802.1 lb, 802. l lg), an LTE transceiver, a Worldwide Interoperability for Microwave Access (WiMAX) transceiver, for example operating in accordance with an IEEE 802.16 standard, and/or another similar type of wireless transceiver configurable to communicate via a wireless radio network.
- DMR digital mobile radio
- P25 Project 25
- TETRA terrestrial trunked radio
- Bluetooth transceiver e.g., a Bluetooth transceiver
- Wi-Fi transceiver for example operating in accordance with an IEEE 802.11 standard (e.g
- the communications unit 202 may additionally or alternatively include one or more wireline transceivers 208, such as an Ethernet transceiver, a USB transceiver, or similar transceiver configurable to communicate via a twisted pair wire, a coaxial cable, a fiber-optic link, or a similar physical connection to a wireline network.
- the transceiver 208 is also coupled to a combined modulator/demodulator 210.
- the controller 220 may include ports (e.g. hardware ports) for coupling to the display screen 205, the input device 206, the imaging device 232, the speaker 228 and/or the microphone 233.
- the controller 220 includes one or more logic circuits, one or more processors, one or more microprocessors, one or more ASIC (application-specific integrated circuits) and one or more FPGA (field-programmable gate arrays), and/or another electronic device.
- the controller 220 and/or the example device 201 is not a generic controller and/or a generic device, but a device specifically configured to implement functionality for causing an output device to provide information for voice command functionality.
- the controller 220 and/or the example device 201 specifically comprises a computer executable engine configured to implement specific functionality for causing an output device to provide information for voice command functionality.
- the static memory 222 is a machine readable medium that stores machine readable instructions to implement one or more programs or applications.
- Example machine readable media include a non-volatile storage unit (e.g. Erasable Electronic Programmable Read Only Memory (“EEPROM”), Flash Memory) and/or a volatile storage unit (e.g. random-access memory (“RAM”)).
- EEPROM Erasable Electronic Programmable Read Only Memory
- RAM random-access memory
- programming instructions e.g., machine readable instructions
- that implement the functional teachings of the example device 201 as described herein are maintained, persistently, at the memory 222 and used by the controller 220 which makes appropriate utilization of volatile storage during the execution of such programming instructions.
- the memory 222 stores instructions corresponding to the application 223 that, when executed by the controller 220, enables the controller 220 to implement functionality for causing an output device to provide information for voice command functionality.
- the controller 220 executes the application 223, the controller 220 is enabled to: when a received textual term, received via one or more of the input device 206 and the communications unit 202, is phonetically similar to one or more existing textual terms used for activating functionality at a communication device (e.g. the communication device 105) using a voice recognition algorithm (e.g.
- the voice recognition algorithm 224) generate one or more suggested textual terms, related to the received textual term, that minimizes phonetic similarities with the one or more existing textual terms; and cause an output device to provide an indication of the one or more suggested textual terms to use in place of the received textual term.
- the application 223 may include one or more machine learning algorithms which may include, but are not limited to: a generalized linear regression algorithm; a random forest algorithm; a support vector machine algorithm; a gradient boosting regression algorithm; a decision tree algorithm; a generalized additive model; neural network algorithms; deep learning algorithms; evolutionary programming algorithms; Bayesian inference algorithms, reinforcement learning algorithms, and the like.
- machine learning algorithms may include, but are not limited to: a generalized linear regression algorithm; a random forest algorithm; a support vector machine algorithm; a gradient boosting regression algorithm; a decision tree algorithm; a generalized additive model; neural network algorithms; deep learning algorithms; evolutionary programming algorithms; Bayesian inference algorithms, reinforcement learning algorithms, and the like.
- the application 223 may include one or more confusability algorithms; such confusability algorithms may comprise one or more algorithms configured to determine when two terms are phonetically similar to each other.
- the controller 220 may be further configured to determine whether the received textual term is phonetically similar to the one or more existing textual terms using one or more of: at least one confusability algorithm; at least one algorithm configured to determine when two textual terms are phonetically similar to each other; and at least one machine learning algorithm.
- different applications 223 may correspond to different machine learning algorithms, and/or different modes of the example device 201. For example, different combinations of one or more different machine learning algorithms may be executed depending on a preconfigured and/or selected mode of the example device 201 [0043] Furthermore, the functionality of the system 100 and/or the example device 201 may be distributed among a plurality of devices of the system 100 including, but not limited to, one or more of the devices 101, 103, 105 and/or any other cloud based device of the system 100.
- FIG. 3 depicts a flowchart representative of a method 300 for causing an output device to provide information for voice command functionality.
- the operations of the method 300 of FIG. 3 correspond to machine readable instructions that are executed by, for example, the example computing device 201, and specifically by the controller 220 of the example computing device 201.
- the instructions represented by the blocks of FIG. 3 are stored at the memory 222 for example, as the application 223.
- the method 300 of FIG. 3 is one way in which the controller 220 and/or the example computing device 201 and/or the system 100 is configured.
- the following discussion of the method 300 of FIG. 3 will lead to a further understanding of the system 100, and its various components.
- the method 300 and/or the system 100 may be varied, and need not work exactly as discussed herein in conjunction with each other, and that such variations are within the scope of present examples.
- the method 300 of FIG. 3 need not be performed in the exact sequence as shown and likewise various blocks may be performed in parallel rather than in sequence. Accordingly, the elements of method 300 are referred to herein as“blocks” rather than“steps.” The method 300 of FIG. 3 may be implemented on variations of the system 100 of FIG. 1, as well.
- the method 300 may be performed at one or more of the devices of the system 100, for example at a combination of one or more of the devices 101, 103, 105.
- the provisioning device 103 may be configuring the
- the device 201 may be embodied as the provisioning device 103, the method 300 being performed at the provisioning device 103 to provision voice commands for activating channels at the communication device 105.
- the controller 220 of the example device 201 receives, a textual term (e.g. a received textual term) via one or more of an input device and a communications unit.
- a textual term e.g. a received textual term
- Any type of textual term is within the scope of present examples, including alphanumeric textual terms and/or any other suitable type of textual term.
- The_ received textual term may be received at the input device 206, for example via a user of the example device 201 typing at a keyboard, and the like, of the example device 201; however, the received textual term may be received at an input device of another device and transmitted to the example device 201, the received textual term being received via the communications unit 202.
- the received textual term may be received via an input device of the communications device 105 and transmitted to the example device 201 and/or the provisioning device 103.
- the display screen 135 may include a touch screen that may be used to input text at the communication device 105.
- a user of the communications device 105 may be attempting to generate a command to be used to voice activate given functionality at the communication device 105 (e.g. to automatically initiate radio communications on a given channel), and operate an input device of the communications device 105 to input a channel name (e.g. the received textual term).
- a textual term received at the block 302 may comprise input corresponding to typed textual input; however, the textual term received at the block 302 may comprises textual terms received via a microphone (e.g. the microphone 233) and converted to text using a speech-to-text engine, and the like.
- a microphone e.g. the microphone 233
- the controller 220 of the example device 201 determines whether the received textual term is phonetically similar to one or more existing textual terms used for activating functionality at a communication device (e.g. the communication device 105) using a voice recognition algorithm (e.g. the voice recognition algorithm 224). As described above, the controller 220 may be further configured to determine whether the received textual term is phonetically similar to the one or more existing textual terms using one or more of: at least one confusability algorithm; at least one algorithm configured to determine when two textual terms are phonetically similar to each other; and at least one machine learning algorithm.
- the terms“TUK” and “TOOSAWN” may be determined not to be phonetically similar as: the“k” sound in “TUK” and the sibilant“s” sound in“TOOSAWN” are not within a similarity threshold level; and as“TOOSAWN” has two syllables and“TUK” has one syllable.
- such algorithms may determine that the terms“TUK” and“TOOSAWN” have different consonants sounds and different sound patterns and such algorithms may determine the terms are not phonetically similar.
- a received textual term is phonetically similar to one or more existing textual terms used for activating functionality at a communication device using a voice recognition algorithm (e.g. a“YES” decision at the block 304)
- the controller 220 of the example device 201 generates one or more suggested textual terms, related to the received textual term, that minimizes phonetic similarities with the one or more existing textual terms.
- Generation of the one or more suggested textual terms may occur according to various modes and/or in various orders.
- the controller 220 may be further configured to generate the one or more suggested textual terms by one or more of: using geographic names related to the received textual term; using a location of an entity related to the device to prioritize the geographic names; expanding an abbreviation in the received textual term; expanding an acronym in the received textual term; expanding a public-safety-related acronym in received textual term; and generating a textual representation of a phonetic representation of a name indicated in the received textual term.
- expansions of public safety acronyms may be prioritized, for example when more than one acronym and/or abbreviation is in a received textual term.
- the acronym and/or abbreviation”CMD in a term“TUC Fire CMD” received as input at the block 302 to name a channel, may be expanded to“TUC Fire Command” as a suggested textual term.
- “CMD” is a public safety acronym and/or abbreviation
- “CMD” is expanded before the abbreviation“TUC”, which stands for“Tucson” (e.g. Arlington, Arizona).
- the controller 220 determines whether the expanded acronym and/or abbreviation minimizes phonetic similarities with the one or more existing textual terms, as described above; when the phonetic similarities are not minimized the expanded acronym and/or abbreviation may be discarded.
- geographic names related to a received textual term may be used as a suggested textual term. For example, for a term“TUC Fire CMD” received as input at the block 302 to name a channel, a suggested textual term of“Tucson Fire
- Command may be generated, when the controller 220 has access to a geographic location (e.g.“Tucson, Arizona”) associated with channel names being received.
- a geographic location e.g.“Tucson, Arizona”
- a location of an entity related to the example device 201 may be used to prioritize the geographic names. For example, for a term“Fire CMD” received as input at the block 302 to name a channel, a suggested textual term of“Tucson Fire Command” and“Arizona Fire Command” may be generated; however, when the example device 201 (e.g. and/or the communication device 105) is associated with the Arlington Fire Command, the location “Tucson” may be prioritized over the more general location“Arizona”.
- controller 220 may generate a textual
- a suggested textual term of“Toosawn Fire Command” as the term“Toosawn” is similar to how the name“Tucson” is pronounced (e.g. as compared to how the name“Tucson” is spelled).
- the controller 220 may be further configured to generate the one or more suggested textual terms by: when the one or more existing textual terms are from a first given list of textual terms, retrieving the one or more suggested textual terms from a second given list of textual terms; and when the one or more existing textual terms comprise channel names of a first zone, retrieving the one or more suggested textual terms from respective channel names of a second zone.
- the communication device 105 comprises a radio configured to communicate via channels the channels may be divided into zones and/or groups of channels. In general, in these examples, the communication device 105 may communicate over channels in zones, but only using one zone at a time (e.g.
- the channels in each zone may have names which may be the same or different as channel names of other zones.
- the controller 220 may retrieve the one or more suggested textual terms from respective channel names of a second zone; the one or more suggested textual terms may hence be the same as one or more of the respective channel names of the second zone.
- the controller 220 may retrieve the one or more suggested textual terms from a second given list of textual terms, for example which may be used as commands for activating functionality at a communication device using a voice recognition algorithm in a second mode.
- the controller 220 may be further configured to generate the one or more suggested textual terms by: retrieving the one or more suggested textual terms from a list of previously used textual terms no longer in use for activating the functionality at a communication device using the voice recognition algorithm. For example, a given textual term may have been used historically to activate functionality at the communication device 105 and/or may have been used to name a channel and/or talkgroup at the communication device 105, but may no longer be in use.
- a list of such previously used textual terms and/or channel names and/or talkgroup names that are no longer in use may be maintained at a memory of the system 100 and/or the memory 222, and the controller 220 may retrieve the one or more suggested textual terms from such a list.
- a cloud based device of the system 100 may maintain one or more lists of textual terms, for example a list of textual terms maintained and/or compiled by an entity with which the system 100 is associated; and the controller 220 may retrieve the one or more suggested textual terms from such a cloud based device.
- the example device 201 is embodied as the provisioning device 103 and/or the communication device 105
- the computing device 101 may maintain one or more lists of textual terms and the provisioning device 103 and/or the communication device 105 may retrieve the one or more suggested textual terms form the computing device 101.
- the controller 220 may generate the one or more suggested textual terms by one or more of prepending and appending one or more additional textual terms to the received textual term.
- a numerical order word such“first”,“second”, and the like, may prepended to the received textual term and/or a number“1”,“2”, and the like, may be appended to the received textual term.
- any suitable type of additional textual terms may be prepended and/or appended to the received textual term.
- the controller 220 may generate the one or more suggested textual terms according to various modes, and each of the modes may be implemented using the same application 223 and/or different applications 223.
- the modes may be implemented in a given order.
- the controller 220 may generate a suggested textual term by expanding an abbreviation.
- the controller 220 may determine whether the suggested textual term with the expanded abbreviation is phonetically similar to existing textual terms; if so, in a second mode, the controller 220 may then retrieve a suggested term from a second list and/or a list of channel names of a second zone, as described above.
- the controller 220 may determine whether the suggested textual term retrieved from the second list is phonetically similar to existing textual terms (e.g.
- the controller 220 may then retrieve a suggested term from list of previously used textual terms, as described above.
- the suggested textual term may be accepted and associated with given functionality of the computing device, as described hereafter.
- the controller 220 of the example device 201 causes an output device to provide an indication of the one or more suggested textual terms to use in place of the received textual term.
- the display screen 205 may be controlled to respectively output a visual indication of the one or more suggested textual terms and/or the speaker 228 may be controlled to respectively output an aural indication of the one or more suggested textual terms.
- a display screen and/or a speaker of the provisioning device 103 may be controlled to respectively output an indication of the one or more suggested textual terms.
- the display screen 135 and/or a speaker of the communication device l05 may be controlled to respectively output an indication of the one or more suggested textual terms.
- the controller 220 may cause an output device at the communication device 105 to provide an indication of the one or more suggested textual terms to use in place of the received textual term.
- the computing device 101 and the provisioning device 103 may receive a received textual term from the communication device 105, and generate the one or more suggested textual terms as described above.
- the one or more of the computing device 101 and the provisioning device 103 may cause an output device of the communication device 105 to provide an indication of the one or more suggested textual terms by transmitting the one or more suggested textual terms to the communication device 105.
- the communication device 105 may then output the one or more suggested textual terms at the display screen 135 and/or the speaker 136.
- the one or more suggested textual terms may be output at an output device as suggestions to use in place of the received textual term.
- the one suggested textual term may be automatically associated with given functionality of the communication device 105 (e.g. for automatic initiation of communications on a given channel) such that when the voice recognition algorithm 224 recognizes the one suggested textual term in voice received at the microphone 133 of the communication device 105, the given functionality is implemented at the communication device 105.
- the communication device 105 may automatically initiate communications on the channel having the name of the one suggested textual term.
- the controller 220 of the example device 201 may receive (e.g. via one or more of the input device 206 and the communications unit 202), an acceptance of one of the one or more suggested textual terms.
- the example device 201 may request a manual acceptance of the one suggested textual term.
- the example device 201 may request a manual selection of one of the suggested textual terms as part of the acceptance at the block 310. Such a manual acceptance (and/or selection) may occur using a user interface at the example device 201 and/or the communication device 105.
- the controller 220 of the example device 201 associates the accepted suggested textual term with the given functionality of the communication device 105.
- the block 312 occurs after the manual acceptance (and/or selection) at the block 310.
- the block 310 is optional and/or the acceptance of a suggested term occurs automatically, and the block 312 occurs automatically, as described above (e.g. when one suggested textual term is generated); in such examples, the block 310 and/or the block 312 may include a visual and/or aural notification of the accepted suggested textual being associated with the given functionality of the communication device 105; the visual and/or aural notification may occur at the communication device 105.
- a received textual term is not phonetically similar to one or more existing textual terms used for activating functionality at a communication device using a voice recognition algorithm (e.g. a “NO” decision at the block 304)
- the controller 220 of the example device 201 associates the received suggested textual term with the given functionality of the communication device 105.
- the block 314 may or may not include acceptance of the received textual term similar to acceptance of one of the one or more suggested textual terms at the block 310.
- FIG. 4 to FIG. 8 depict an example of the method 300, described with respect to voice activating of channels and/or using voice commands to initiate communication on channels.
- FIG. 4 depicts a graphic user interface 401 that may be provided at a display screen 205 of the example device 201 (e.g. the provisioning device 103).
- the graphic user interface 401 provides a list 403 of current channel names (e.g. in a column), for example for a first zone.
- the graphic user interface 401 may be further used to add new channel names, each of the channel names being a textual term which, when spoken by a user, may be used for activating functionality at the communication device 105 using a voice recognition algorithm, such as the voice recognition algorithm 224 and the like.
- a voice recognition algorithm such as the voice recognition algorithm 224 and the like.
- communications on the corresponding channel may be automatically initiated (e.g. using a radio of the communication device 105).
- a new channel name 405 is being received (e.g. the received channel name 405), for example at a field of the list 403 which corresponds to a given channel.
- the received channel name 405 may be received (e.g. at the block 302 of the method 300) using an input device 206 of the example device 201, such as the provisioning device 103, for example a virtual keyboard (not depicted) and the like, for example when the display screen 205 includes a touch screen.
- the received channel name 405 is“TUC Fire CMD”, which is compared (e.g.
- the received channel name 405 may be transmitted to one or more of the devices 101, 103.
- the received channel name 405“TUC Fire CMD” is phonetically similar to an existing channel name in the list 403, for example“TUK FIRE CMD”
- a second graphic user interface 407 is provided at the display screen 135 which indicates that a similarly conflict.
- the terms“TUC” and“TUK” are phonetically similar.
- a selectable option 409 to enter a new name is provided in the second graphic user interface 407, along with a selectable option 411 to play the new channel name (e.g. selection of the selectable option 411 will cause the communication device 105 to play the new name at the speaker 228 (e.g. of the provisioning device 103) using, for example, a text-to-speech algorithm).
- the second graphic user interface 407 further includes a selectable option 413 which, when selected, causes the suggested channel names to be generated (e.g. at the block 306 of the method 300).
- the selectable option 413 is further provided with a selectable option 415 to play a suggested channel name and selection of the selectable option 415 will cause the communication device 105 to play the new name at the speaker 228 (e.g. of the provisioning device 103) using, for example, a text-to-speech algorithm.
- the selectable options 411, 415 may be used to aurally compare an entered channel name with a suggested channel name.
- Each of the selectable options 409, 411, 413, 415 may comprise virtual buttons and the like.
- suggested channel names are generated (e.g. at the block 306 of the method 300).
- FIG. 5 depicts the example device 201 (e.g. the provisioning device 103) generating (e.g. using the application 223) suggested channel names that are related to the received channel name 405, that minimize phonetic similarities with the one or more existing channel names. While not all components of the example device 201 are depicted, they are understood to be nonetheless present.
- the example device 201 is in communication with an optional cloud based device 501 that stores an optional list 503 of channel names for a second zone.
- the channel names of the list 503 are not depicted, they may be similar or different to the channels names of the list 403. However, the list 503 may alternatively be stored at the memory 222 of the example device 201.
- the cloud based device 501 may comprise the computing device 101; however, the cloud based device 501 may be any cloud based device 501 that stores the list 503 of channel names for a second zone, and the like.
- FIG. 5 Also depicted in FIG. 5 is an optional list 505 of previously used channel names stored at the example device 201 (e.g. at the memory 222) but which may alternatively be stored at the cloud based device 501.
- the example device 201 (e.g. the provisioning device 103) generates a list 507 of one or more suggested channel names that are related to the received channel name 405, and that minimizes phonetic similarities with the one or more existing channel names.
- the suggested channel name“TUC FIRE Command” is generated by prioritizing expansion of the public-safety related abbreviation“CMD”.
- the suggested channel name“Tucson FIRE Command” is generated by expanding the abbreviations“TUC and“CMD”.
- the suggested channel name“Toosawn FIRE Command” is generated by expanding the abbreviation “CMD” and generating a textual representation of a phonetic representation of name indicated in the received channel name 405; in particular, a phonetic representation of the name“Tucson”, associated with the abbreviation“TUC” generated and in particular“Toosawn”.
- the suggested channel name“TUC FIRE Command 2” is generated by appending the number“2” to the received channel name 405.
- the suggested channel name“Second TUC FIRE Command” is generated by prepending the word“Second” to the received channel name 405.
- the suggested channel names in the list 507 may be retrieved from one or more of the cloud based device 501, the list 503 and the list 505.
- the suggested channel names of the list 507 indicate that suggested textual terms that are both related to a received textual term, and that minimize phonetic similarities with one or more existing textual terms, may include but are not limited to: words that indicate information similar to information in the received textual term but expressed using words that are phonetically different from existing textual terms; addition of words to the received textual term that are phonetically different from existing textual terms; and the like.
- the one or more suggested channel names are provided in the form of the list 507, and further while the one or more of the suggested channel names include five suggested channel names, the one or more suggested channel names may be provided in other formats and/or the one or more of the suggested channel names may include as few as one suggested channel name, fewer than five suggested channel names, and more than five suggested channel names.
- FIG. 6 is substantially similar to FIG. 4 with like elements having like numbers.
- the display screen 205 is being caused (e.g. at the block 308 of the method 300) to provide an indication 607 of the one or more of the suggested channel names in the list 507, the indication 607 comprising a plurality of selectable options each corresponding to one of the suggested channel names in the list 507.
- a hand 609 of a user is touching (e.g. at a touch screen of the display screen 205), the selectable option labelled“Toosawn Fire Command”, which causes an acceptance of the suggested channel name“Toosawn Fire Command” to be received (e.g. at the block 310 of the method 300).
- the accepted suggested channel name is then associated (e.g. at the block 312 of the method 300) with voice command functionality for activating the channel associated with the accepted suggested channel name“Toosawn Fire Command”.
- an accepted suggested name 705 of “Toosawn Fire Command” is associated with a respective corresponding channel in the list 403 of channel names which, when received in voice at the microphone 133 of the communication device 105, causes voice activating of the corresponding channel.
- the accepted suggested name 705 is provided in the same field of the list 403 where the received channel 405 was initially received; providing the accepted suggested name 705 in the same field of the list 403 where the received channel name 405 was initially received comprises an indication of an acceptance of the accepted suggested name 705 as well as an association of the accepted suggested name 705 with voice activating of a respective corresponding channel.
- a person of skill in the art understands that when the method 300 is being implemented at the provisioning device 103 to provision the communication device 105, when the provisioning device 103 associates a channel name with a channel, data for voice activating the channel using an accepted suggested channel name may be provided (e.g. transmitted) to the communication device 105.
- FIG. 8 attention is next directed to FIG. 8.
- the communication device 105 has been provisioned with an accepted suggested channel name“Toosawn Fire Command” that is associated with a respective corresponding channel.
- a user 809 such as a police officer, is using the communication device 105 after the accepted suggested channel name“Toosawn Fire Command” is associated with a respective corresponding channel.
- the user 809 may include the accepted suggested channel name 705,“Toosawn Fire Command”, in voice 811 that is received at the microphone 133 of the communication device 105.
- the communication device 105 automatically implements voice command functionality associated with the accepted suggested channel name 705; in particular, the communication device 105 initiates communications 813 over the respective corresponding channel associated with the accepted suggested channel name 705,“Toosawn Fire Command” (e.g. the arrow indicated at the communications 813 indicates a signal transmitted over the respective corresponding channel associated with the accepted suggested channel name 705,“Toosawn Fire Command”).
- the method 300 may be similarly applied to voice activating of talkgroups.
- the one or more suggested textual terms generated at the block 306 of the method 300 may comprises a name of one or more of a channel and a talkgroup to be activated when a voice recognition algorithm recognizes the one or more suggested textual terms in voice received at the microphone 133 of the communication device 105.
- the one or more suggested terms may be for any type of voice command functionality, including, but not limited to, voice activating of one or more of: applications, messaging, telephone calls, generating and/or editing documents, and the like.
- a device, system and method for causing an output device to provide information for voice command functionality When a received textual term is phonetically similar to one or more existing textual terms used for activating functionality at a communication device using a voice recognition algorithm, one or more suggested textual terms are generated, related to the received textual term.
- the one or more suggested textual terms may be generated in any suitable manner including expanding acronyms and/or abbreviations, related textual terms in lists and/or other zones, historical data, and the like.
- language of“at least one of X, Y, and Z” and“one or more of X, Y and Z” may be construed as X only, Y only, Z only, or any combination of two or more items X, Y, and Z (e.g., XYZ, XY, YZ, XZ, and the like). Similar logic may be applied for two or more items in any occurrence of“at least one ...” and“one or more...” language.
- relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
- the terms“a” and“an” are defined as one or more unless explicitly stated otherwise herein.
- the terms“substantially”,“essentially”,“approximately”,“about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%.
- the term“coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically.
- a device or structure that is“configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
- processors such as
- microprocessors digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein.
- FPGAs field programmable gate arrays
- unique stored program instructions including both software and firmware
- control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein.
- some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more application specific integrated circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic.
- ASICs application specific integrated circuits
- an embodiment may be implemented as a computer-readable storage medium having computer readable code stored thereon for programming a computer (e.g., comprising a processor) to perform a method as described and claimed herein.
- Examples of such computer-readable storage mediums include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Artificial Intelligence (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Telephonic Communication Services (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/042,056 US10672392B2 (en) | 2018-07-23 | 2018-07-23 | Device, system and method for causing an output device to provide information for voice command functionality |
| PCT/US2019/042542 WO2020023298A1 (en) | 2018-07-23 | 2019-07-19 | Device, system and method for causing an output device to provide information for voice command functionality |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| EP3807803A1 true EP3807803A1 (en) | 2021-04-21 |
| EP3807803B1 EP3807803B1 (en) | 2024-08-28 |
Family
ID=67544367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP19749915.5A Active EP3807803B1 (en) | 2018-07-23 | 2019-07-19 | Device, system and method for causing an output device to provide information for voice command functionality |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US10672392B2 (en) |
| EP (1) | EP3807803B1 (en) |
| CA (1) | CA3106472C (en) |
| WO (1) | WO2020023298A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11869494B2 (en) * | 2019-01-10 | 2024-01-09 | International Business Machines Corporation | Vowel based generation of phonetically distinguishable words |
| US10775990B1 (en) * | 2019-07-26 | 2020-09-15 | Slack Technologies, Inc. | Group-based communication system and apparatus configured to manage channel titles associated with group-based communication channels |
| US11176942B2 (en) * | 2019-11-26 | 2021-11-16 | Vui, Inc. | Multi-modal conversational agent platform |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3114703B2 (en) | 1998-07-02 | 2000-12-04 | 富士ゼロックス株式会社 | Bilingual sentence search device |
| US7299181B2 (en) * | 2004-06-30 | 2007-11-20 | Microsoft Corporation | Homonym processing in the context of voice-activated command systems |
| US8099287B2 (en) * | 2006-12-05 | 2012-01-17 | Nuance Communications, Inc. | Automatically providing a user with substitutes for potentially ambiguous user-defined speech commands |
| US8892443B2 (en) * | 2009-12-15 | 2014-11-18 | At&T Intellectual Property I, L.P. | System and method for combining geographic metadata in automatic speech recognition language and acoustic models |
| US8473289B2 (en) * | 2010-08-06 | 2013-06-25 | Google Inc. | Disambiguating input based on context |
| WO2012050800A1 (en) * | 2010-09-29 | 2012-04-19 | International Business Machines Corporation | Context-based disambiguation of acronyms and abbreviations |
| US8290772B1 (en) * | 2011-10-03 | 2012-10-16 | Google Inc. | Interactive text editing |
| US9137645B2 (en) * | 2012-05-31 | 2015-09-15 | Motorola Solutions, Inc. | Apparatus and method for dynamic call based user ID |
| US9858038B2 (en) * | 2013-02-01 | 2018-01-02 | Nuance Communications, Inc. | Correction menu enrichment with alternate choices and generation of choice lists in multi-pass recognition systems |
| US9245523B2 (en) | 2013-07-25 | 2016-01-26 | Nice-Systems Ltd | Method and apparatus for expansion of search queries on large vocabulary continuous speech recognition transcripts |
| US9495959B2 (en) * | 2014-02-27 | 2016-11-15 | Ford Global Technologies, Llc | Disambiguation of dynamic commands |
| US9378740B1 (en) | 2014-09-30 | 2016-06-28 | Amazon Technologies, Inc. | Command suggestions during automatic speech recognition |
| US10176219B2 (en) * | 2015-03-13 | 2019-01-08 | Microsoft Technology Licensing, Llc | Interactive reformulation of speech queries |
| US20170337923A1 (en) | 2016-05-19 | 2017-11-23 | Julia Komissarchik | System and methods for creating robust voice-based user interface |
| US10636423B2 (en) | 2018-02-21 | 2020-04-28 | Motorola Solutions, Inc. | System and method for managing speech recognition |
-
2018
- 2018-07-23 US US16/042,056 patent/US10672392B2/en active Active
-
2019
- 2019-07-19 EP EP19749915.5A patent/EP3807803B1/en active Active
- 2019-07-19 WO PCT/US2019/042542 patent/WO2020023298A1/en not_active Ceased
- 2019-07-19 CA CA3106472A patent/CA3106472C/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| US10672392B2 (en) | 2020-06-02 |
| WO2020023298A1 (en) | 2020-01-30 |
| CA3106472A1 (en) | 2020-01-30 |
| CA3106472C (en) | 2022-01-25 |
| US20200027447A1 (en) | 2020-01-23 |
| EP3807803B1 (en) | 2024-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10679605B2 (en) | Hands-free list-reading by intelligent automated assistant | |
| US10705794B2 (en) | Automatically adapting user interfaces for hands-free interaction | |
| KR102596446B1 (en) | Modality learning on mobile devices | |
| US10540521B2 (en) | Selective enforcement of privacy and confidentiality for optimization of voice applications | |
| CN110110319B (en) | Word level correction of speech input | |
| KR102084646B1 (en) | Device for recognizing voice and method for recognizing voice | |
| US8364487B2 (en) | Speech recognition system with display information | |
| EP3005668B1 (en) | Application gateway for providing different user interfaces for limited distraction and non-limited distraction contexts | |
| US20140035823A1 (en) | Dynamic Context-Based Language Determination | |
| US20110276327A1 (en) | Voice-to-expressive text | |
| CA3106472C (en) | Device, system and method for causing an output device to provide information for voice command functionality | |
| AU2015210460A1 (en) | Speech recognition repair using contextual information | |
| EP2092514A2 (en) | Content selection using speech recognition | |
| WO2012138917A2 (en) | Gesture-activated input using audio recognition | |
| WO2014197737A1 (en) | Automatically adapting user interfaces for hands-free interaction | |
| US20170364512A1 (en) | Electronic device for providing translation service and method thereof | |
| US20130300666A1 (en) | Voice keyboard | |
| JP2018072560A (en) | Information processing system, information processor, and information processing method | |
| KR20180066513A (en) | Automatic interpretation method and apparatus, and machine translation method | |
| CN110827815A (en) | A speech recognition method, terminal, system and computer storage medium | |
| US20210241755A1 (en) | Information-processing device and information-processing method | |
| US20230040219A1 (en) | System and method for hands-free multi-lingual online communication | |
| US12437759B2 (en) | Dialogue management method, dialogue management system, and computer-readable recording medium | |
| KR102496398B1 (en) | A voice-to-text conversion device paired with a user device and method therefor | |
| CN118541751A (en) | Method and system for providing speech synthesis service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20210108 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| AX | Request for extension of the european patent |
Extension state: BA ME |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20220801 |
|
| REG | Reference to a national code |
Ref legal event code: R079 Ipc: G06F0040400000 Ref country code: DE Ref legal event code: R079 Ref document number: 602019057894 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G06F0040200000 Ipc: G06F0040400000 |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
| RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 40/40 20200101AFI20240327BHEP |
|
| INTG | Intention to grant announced |
Effective date: 20240412 |
|
| GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
| GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
| P01 | Opt-out of the competence of the unified patent court (upc) registered |
Free format text: CASE NUMBER: APP_39307/2024 Effective date: 20240702 |
|
| AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602019057894 Country of ref document: DE |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
| REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241128 |
|
| REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1718764 Country of ref document: AT Kind code of ref document: T Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241230 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241129 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241228 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241128 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241128 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241230 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241128 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241228 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241129 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602019057894 Country of ref document: DE |
|
| PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20250619 Year of fee payment: 7 |
|
| 26N | No opposition filed |
Effective date: 20250530 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240828 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20250620 Year of fee payment: 7 |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: H13 Free format text: ST27 STATUS EVENT CODE: U-0-0-H10-H13 (AS PROVIDED BY THE NATIONAL OFFICE) Effective date: 20260224 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20250719 |
|
| REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20250731 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20250731 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20250731 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20250731 |