Classifications

• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
  • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
  • G09B5/00—Electrically-operated educational appliances
  • G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied

Definitions

• the present invention relates to devices for enabling communication between persons, and more especially to allow communication between a person with a speech and / or hearing disability with another person having all his speaking abilities or having also a communication disability.
• Handicap - Incapabilities - Addiction HFD
• the invention relates to devices for simplifying communication or triggering warning signals for persons diminished with less mobility or less autonomy (children, elderly, disabled, sick), or having just had an accident, to another entity with all its ability to speak.
• Impaired people can usually communicate easily with healthy people close to them. When they have an accident, triggering a call for help signal is usually impossible, either because of the lack of mobility induced by the accident, or because of a lack of awareness. The absence of an alarm in the first moments after the accident generally results in a drastic aggravation of the situation, which can lead to complications until the death of the person.
• US 2004/0073432 A1 discloses a portable device for communication by a disabled user of speech, comprising:,, *
• a touch-screen input text interface capable of generating a stream of text data transmitted from a message entered by the user on the text input interface; a speech synthesis device receiving the stream of text data transmitted from the text input interface, and transforming it into a stream of transmitted audio signals,
• a loudspeaker audio output interface receiving the stream of emitted audio signals, and transforming it into an acoustic wave stream emitted image of the transmitted and audible text data stream in the immediate environment of the portable device.
• the device described in this document is a remote access to the Internet provided with communication aids.
• the term "webpad” used indicates that it is a personal computer tablet (tablet PC) having substantially an A4 page size and a weight of 1 to 2 kilograms.
• This type of tablet PC (“webpad”) is widespread especially in hotels in the United States to access various services (Internet, shopping, information) by the TV.
• the tablet PC exchanges the messages by wireless local connection with an external telephone or with an internet connection device.
• Several devices are then necessary, and the set described is not portable.
• Such a device (“webpad") is too heavy and cumbersome to be permanently available, especially when the user is a handicapped person or diminished who moves, or who is standing. The use of both hands is necessary, and you need a support plan.
• the invention provides a portable device for communication by a disabled user of speech, comprising:
• a text input interface capable of generating a stream of transmitted text data image of a message entered by the user on the text input interface; a speech synthesis device receiving the stream of text; transmitted text data from the text input interface, and transforming it into a stream of transmitted audio signals,
• an audio output interface receiving the stream of transmitted audio signals, and transforming it into a stream of acoustic waves emitted image of the transmitted and audible text data stream in the immediate environment of the portable device, the device being implemented on a cellular phone hardware base with digital personal assistant (PDA) features and an open operating system, with a touch screen, a computer architecture and a digital signal processing body.
• PDA digital personal assistant
• cellular phone hardware base incorporating digital personal assistant (PDA) functions, also called a smartphone, which can also be defined as a digital personal assistant (PDA) with the means and functions of a cellular phone (also called communicating PDA), designates a portable electronic device, integrated in a housing holding in the hand of the user, comprising a computer architecture (processors, memories, input-output, software), the electronic circuits of a cellular telephone , with a touch screen and a digital signal processing body.
• PDA digital personal assistant
• This hardware base completely solves the mobility requirement of the device without compromising the ergonomics and ease of use.
• the voice synthesis device may advantageously comprise embedded speech synthesis software.
• an on-board voice synthesizer of a type dedicated to the automotive market or global positioning systems can be used (Loquendo Automotive Solution from Loquendo, or Acapela Onboard from Acapela Group). In this way, the cost of development and production of such a communication device is particularly reduced, and the volume is also reduced.
• the invention proposes, in a preferred embodiment, that the text input interface is connected to the speech synthesis device by an automatic processing module constituting a programming interface (AP), including a parameterizable and programmable interface routine for adaptation to different input interface modules.
• API programming interface
• a user may have access to the interface routine, for example to adapt a new input interface module compatible with a particular handicap, or to delete an unnecessary input interface module.
• the text input interface may advantageously comprise several text input modules, each of which is capable of generating a flow of text data transmitted from a solicitation of a nature distinct from the user, and which transmit the stream of text data transmitted to the speech synthesis device.
• the text entry interface comprises at least two of the input modules, preferably all of the input modules of the family comprising:
• a second pictogram input module each generating, on manual request from the user, an image text data stream of a pre-recorded word or group of words or phrases,
• a fifth sensory glove input module provided with sensors and a motion decoder associating a word or a phoneme at each position of the user's hand (s). wearing the sensory gloves,. . .
• a sixth input electroacoustic transducer input module and an automatic speech recognition device for transforming speech acoustic signals emitted by the user into a text data stream.
• a person with a disability or a diminished person can usefully choose to communicate using one or the other of the input modules, which he will choose according to the communication stage considered. to say according to the message to transmit, or according to the circumstances (handicap, accident ).
• the third input module can advantageously use the phonemes of the BOREL-MAISONNY method or the Spoken Speech Language (PLC). In this way, the disabled person can use his knowledge of these particular methods that are very widely used.
• the device according to the invention furthermore uses the radiofrequency transmitter of the hardware base, which receives the stream of audio signals emitted from the speech synthesis device and which transmits as radio waves.
• the disabled person can communicate with a distant person.
• the device thus determined thus makes it possible to send messages to a remote person, who will receive them with a standard receiving device.
• the radiofrequency transmitter may be of the mobile phone type, for transmission and reception according to the GSM standard.
• the device can advantageously use the radio frequency receiver, which receives radio waves conveying reception audio signals and extracts a stream of reception audio signals.
• An electroacoustic receiving transducer is then used to receive the stream of receiving audio signals and transform it into an audible reception sound wave stream in the immediate environment of the portable device.
• Such a device is then suitable for use by a person with a speech disability, but not disabled by hearing.
• the device further comprises an automatic speech recognition module, which receives the stream of reception audio signals and transforms it into a stream of text data sent to display means for displaying the image text data of the reception audio signal stream.
• the device may include an automatic speech recognition module, which receives the stream of reception audio signals and transforms it into a data stream for animating the face of an avatar to allow reading lip.
• an audio switching device which receives the stream of audio signals from the speech synthesis device and selectively transmits it:
• VoIP voice over IP
• the input electroacoustic transducer adapted to receive the voice of a local interlocutor and transform it into a stream of input audio signals; the audio switcher receives on the one hand the stream of input audio signals from the electroacoustic transducer input, and / or on the other hand the stream of reception audio signals from the radio frequency receiver, and transmits them in sequence to the automatic speech recognition module for displaying the input message or the reception message.
• the automatic speech recognition module for displaying the input message or the reception message.
• FIG. 1 schematically illustrates a portable device according to a simplified embodiment of the invention
• FIG. 2 illustrates a. schematic structure of a portable device according to a second embodiment of the invention
• FIG. 3 is a diagrammatic view of the detail of the device of FIG. 2;
• FIGS. 5 and 6 illustrate two embodiments of the invention.
• FIG. 7 illustrates the architecture of a communicating smartphone or PDA hardware base.
• the device according to the invention is implemented on a cellular telephone hardware base integrating digital personal assistant (PDA) functions, of smartphone or PDA communicating type, and equipped with an open operating system .
• PDA digital personal assistant
• Such a system comprises in series a touch screen and a digital signal processing body (for example DSP) easily configurable to constitute the speech synthesis device and all or part of the text input interface 1.
• Figure 7 illustrates schematically the architecture of such a hardware base type smartphone or PDA communicating.
• the cellular telephone hardware base 30 comprises a radio frequency transmitter-receiver circuit 8 and a subset of signal management 32.
• the signal management subsystem 32 a first signal management subassembly 32.
• processor 33 constituting, with suitable software implemented in the processor 33 or stored in a memory 39, a digital signal processing body, for example of the DSP type, capable of processing the telecommunication signals at high speed.
• the first processor 33 communicates with the radiofrequency transceiver 8 and manages simple interfaces such as an audio interface 34, light-emitting diodes 35, vibrating transducers 36, a telephone keyboard 37, a memory card 38 of SIM type, a memory 39 dedicated.
• the digital personal assistant hardware base 40 comprises a second processor 41 which, with appropriate software, constitutes a management computer architecture and communicates with more complex interfaces such as an LCD screen 42, a touch screen 43, a flash memory 44, an audio interface 45, and optionally with an interface circuit 46 to a communication circuit 47 of WiFi type.
• this hardware base comprises two distinct processors 33 and
• the first processor 33 being dedicated to the specific management of telecommunication signals
• the second processor 41 being dedicated to the management of complex peripherals. Sufficient speed is thus achieved for the processing of the telecommunication signals, and sufficient management power to manage the touch screen, the input.
• the portable device comprises a text input interface 1, which the user can request to generate a stream of transmitted text data 2 which is the image of a message that the user wishes to enter on the text input interface 1.
• the transmitted text data stream 2 is sent to a speech synthesis device 3, which transforms it into a stream of transmitted audio signals 4
• An audio output interface 5 receives the transmitted audio signal stream 4, and transforms it into an emitted acoustic wave stream 6 which is the image of the transmitted text data stream 2 and which is audible in the immediate environment of the portable device.
• a speech disabled user can enter a message in the text entry interface 1, by means other than speech, for example by a manual action, and the device transforms this solicitation into a stream of transmitted acoustic waves 6, that a speaker can hear directly in the immediate environment of the portable device.
• the text input interface 1 may for example be implemented in the form of the touch screen 43 (FIG. 7).
• the voice synthesis function 3 can be performed by the first processor 33 and an associated program, for example an embedded voice synthesis software of a type dedicated to the automotive market or global positioning systems (GPS).
• the audio output interface 5 may comprise a local amplifier 5a which supplies a loudspeaker 5b.
• the input interface 1 is connected to the speech synthesis device 3 by an automatic processing module comprising a parameterizable and programmable interface subprogram for adaptation to different modules of the interface.
• This programmable automatic and programmable interface subroutine processing module constitutes an open programming interface 7 (Open API), which offers the possibility of developing a new text input means without having to modify the architecture of the device nor to appeal to the manufacturer. This makes it easy to adapt the device to various handicaps or various problems, which force the user to use such or such means to enter the messages in the text entry interface 1 or to trigger alarms. This is also useful when a handicap or a sum of handicaps are not taken into account by the standard device.
• Open API open programming interface 7
• the text input interface 1 comprises several text input modules 1a, 1b, 1c, 1d, 1e and 1f, each of which is capable of generating a flow. of transmitted text data 2 from a solicitation of a nature distinct from the user, and which transmits the transmitted text data stream 2 to the speech synthesis device 3 via the programming interface 7.
• the first text input module 1a may for example be an alphanumeric keyboard, allowing the input of text, numbers and punctuation.
• the keyboard can be virtual, in the form of the touch screen 43 ( Figure 7), or in the form of a keyboard projected on a surface through visible or physical radiation.
• the second input module 1b may comprise a series of pictograms each generating, on manual request from the user, a flow of text image data of a word or a group of words or a pre-recorded sentence.
• a pictogram can be associated with a phrase in ready-made text like "Hello, my name is Fanny, I am mute and speak with a voice synthesizer", or like "Hello, my name is Fanny, I live at 48 rue Vendians and I need ... ".
• the pictogram provides ease for repetitive phrases. The user can create, modify or delete at will a pictogram and its associated sentence.
• a sentence may have variable elements depending on external parameters. For example, if it is 8 pm, the presentation pictogram will say “Good evening ", instead of saying “Hello !”.
• the third input module 1c may include phoneme, space and punctuation keys. This module allows the creation of phonetic or correctly spelled text from a selection of phonemes, from the touch screen. In order to accelerate the learning of these phonemes, it is advantageous to choose existing phonemes such as those of the BOREL-MAISONNY method or the Completed Spoken Language (PLC), or any other sentence decomposition into phonemes. In order for the text to be interpreted correctly with the appropriate intonation, phonemes are enriched with symbols to create the space between words and punctuation. The concatenation of phonemes, spaces and punctuation, gives a sentence exploitable by a vocal synthesis.
• PLC Completed Spoken Language
• the fourth input module 1d can be a handwriting recognition screen. This handwriting is transformed into a string of characters interpretable by a computer system.
• the fifth input module 1e may comprise one or two sensory gloves, provided with a sensor and a motion decoder associating a word or a phoneme at each position of the user's hands or wearing the sensory gloves or gloves.
• This module allows you to create the sentence using the gestures specific to each sign language. It allows, for example, a user accustomed to using the BOREL-MAISONNY method, to build his sentence using his usual sign language.
• One or two sensory gloves, according to the sign language used, allows the motion decoder to associate a phoneme or a word with each position of the hands.
• Sensory gloves can be replaced by a video camera and appropriate signal processing, or by breaking wave beams positioned in all three dimensions. The text is enriched with new signs to create the space between words and punctuation.
• the sixth input module 1f may include an input electroacoustic transducer and an automatic speech recognition device, for transforming speech acoustic signals emitted by the user into a text data stream 2.
• This module may allow to correct inaudible sentences for the purpose of re-education or conversation: this is a device for a deaf person who wishes to express himself orally. The user speaks in the device.
• the automatic speech recognition device transcribes the dictated sentence into text, which is corrected automatically if possible or manually with the help of the user who dictated the sentence.
• the device can inculcate a rhythm to speech thanks to an integrated metronome (visual or auditory).
• the programming interface 7 allows a third party developer to transmit his text to the system without having to go through one. predefined entries by the application ...
• This abstraction layer provides flexibility in product development. Indeed, text input modules. 1a-1f can be developed with total independence, by different teams, with different techniques. In addition, opening this layer to independent developers allows you to create new text entry modules that are tailored to particular disabilities without conflicting with the original application.
• the speech synthesis device 3 can comprise two successive modules that can be distinguished, namely a text generator module 3a and a speech synthesis module itself 3b.
• the text generator module 3a prepares the text for speech synthesis. For example, it can inject standard vocal changes into the speech synthesis, by playing on the timbre, the attack, the speech speed and the sound volume, in order to personalize the synthesized voice.
• the speech synthesis module itself 3b transforms the data stream as text into a stream of transmitted audio signals 4.
• the device further uses the radio frequency transmitter 8, which receives the stream of transmitted audio signals 4 from the voice synthesis device 3 and which transmits it in the form of radio waves 9.
• the radiofrequency transmitter 8 incorporates a radio frequency receiver, which receives radio waves conveying reception audio signals and extracts a stream of reception audio signals 10.
• audio reception signals 10 is sent to an electroacoustic receiving transducer, for example constituted by the amplifier 5a and the loudspeaker 5b, which transforms it into an audible reception acoustic wave stream 6 in the immediate environment of the device portable.
• applications can be implemented on the device to monitor in real time its use in one of its modes of operation.
• the device can be used as a television remote control. The prolonged non-use of this device results in requests for actions towards the user (erasing errors, answering questions). An unsatisfactory response to these stimuli results in the triggering of alarm systems via the radiofrequency module 8.
• applications can be implemented on the device in order to follow in real time the sound environment of the diminished person to automatically detect a sound or a series of sounds.
• the device can be used to analyze the ambient noise in real time in order to detect help calls launched to the system.Use is to enable diminished people who have fallen and are still conscious to make a call.
• the system analyzes the ambient noise in real time.When an abnormal noise is detected in the person's environment, this results in requests for actions to the user (erasing errors, responses Unsatisfactory response to these stimuli results in the triggering of alarm systems via the radio frequency module 8.
• the device of FIG. 2 further comprises an audio switching device 11, which receives the stream of audio signals 4 coming from the speech synthesis device 3 and which selectively transmits it to the local amplifier 5a of the audio output interface, and / or to the radiofrequency transmitter 8, and / or to a voice over IP module 12 which transposes the stream of audio signals 4 to be exploitable by an IP network, and which transmits the transposed stream to the radiofrequency transmitter 8.
• FIG. 3 is now considered , which illustrates in more detail the operating mode of the audio switching device 11.
• the audio switching device 1 1 distributes the audio stream 4 from the speech synthesis device 3 to the devices concerned. According to the context of use (noted 13) of the device by the user, the audio switching device 11 makes the decision to feed one or more voice outputs.
• the audio switching device 1 1 redirects the audio stream 4 on the radio frequency transmitter 8, and simultaneously on the amplifier local 5a.
• the audio switching device 11 redirects the audio stream 4 only to the local amplifier 5a.
• voice over IP (VoIP) mode without amplified listening, the audio switcher device 11 redirects the audio stream 4 to the voice over IP module 12 only.
• VoIP voice over IP
• the device can also be adapted so that a diminished user, for example aged or injured, can emit an audio message locally or remotely, the message being perceived by the interlocutor as a request for help or a call for help generated automatically by the device.
• the user speech handicapped or decreased can receive back a response in voice form, 'directly as in the devices commonly used by healthy people.
• Figure 4 illustrates an improvement of the previous device, allowing use by a disabled user of hearing.
• the radiofrequency transmitter 8 the voice over IP device 12, the audio switching device 11.
• An electroacoustic input transducer 14 is used, able to receive the voice of a local interlocutor and to transform it in an input audio signal stream 15, transmitted to the audio switching device 11.
• the audio switching device 11 thus receives, on the one hand, the stream of audio input signals coming from the electroacoustic input transducer 14, and on the other hand the stream of audio reception signals 10 coming from the radio frequency transmitter 8 or the device
• the audio switching device 11 transmits them in sequence to an automatic speech recognition module 16, which transforms them into a stream of text data 17 transmitted to a display 18 which ensures the display of the message. input contained in the input audio signal stream 15 or the reception message contained in the reception audio signal stream 10.
• the invention described above has been implemented and carried out on an iPAQ hp5540 (brand of use) under Pocket PC 2003 (brand of use).
• This iPAQ brand of use
• This iPAQ has a WiFi interface for communicating voice over VoIP over a wireless Internet network.
• the demonstration uses a SaySo speech synthesis engine (brand of use) provided by the company ELAN.
• the sentence entry is carried out with a virtual keyboard containing the phonemes of the BOREL-MAISONNY method. Space key and punctuation keys main were added to obtain a synthetic voice, understandable, pleasant, human. See the drawing of Figure 5.
• PiPAQ brand of use
• Skype software WIN CE brand name
• the invention may provide automatic detection means for the protective case, described below.
• the device incorporates an algorithm that includes a sequence for listening to the sounds generated by the portable device, and that analyzes the amplitude of the high frequencies of the sounds received by comparing it with a given threshold. If this amplitude is lower than the given threshold, then the device deduces that there is presence of a protection, mechanical, and the algorithm modifies accordingly the treatments applied to the audio stream to correct the influence of the mechanical protection on the sounds emitted. It will be understood that such sound correction means can be used independently of the other means previously described.

Landscapes

• Engineering & Computer Science (AREA)
• Business, Economics & Management (AREA)
• Physics & Mathematics (AREA)
• Educational Administration (AREA)
• Educational Technology (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• General Health & Medical Sciences (AREA)
• Telephonic Communication Services (AREA)
• Telephone Function (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=34979245

Family Applications (1)

Country Status (5)

Families Citing this family (20)

Family Cites Families (10)

• 2005
  • 2005-03-31 FR FR0503386A patent/FR2884023B1/fr not_active Expired - Fee Related
• 2006
  • 2006-03-31 CA CA2602633A patent/CA2602633C/fr not_active Expired - Fee Related
  • 2006-03-31 WO PCT/FR2006/000707 patent/WO2006103358A1/fr not_active Application Discontinuation
  • 2006-03-31 US US11/909,775 patent/US8082152B2/en not_active Expired - Fee Related
  • 2006-03-31 EP EP06726156A patent/EP1864270A1/de not_active Ceased

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events