WO2019237427A1

WO2019237427A1 - Method, apparatus and system for assisting hearing-impaired people, and augmented reality glasses

Info

Publication number: WO2019237427A1
Application number: PCT/CN2018/092812
Authority: WO
Inventors: 苏进; 张志扬; 李琦; 杨莉; 苏卓然
Original assignee: 北京佳珥医学科技有限公司
Priority date: 2018-06-11
Filing date: 2018-06-26
Publication date: 2019-12-19
Also published as: CN108962254A

Abstract

Embodiments of the present invention relate to the technical field of augmented reality. Provided are a method, apparatus and system for assisting hearing-impaired people, and augmented reality glasses. The method comprises: receiving the voice of at least one sound source; recognizing the voice of each of the at least one sound source, so as to convert the voice of each of the at least one sound source into a text expressed by using a preset target language; and displaying the text obtained by converting the voice of each of the at least one sound source. The apparatus comprises: a voice receiving module, a voice recognition module, and a display module. The system comprises the apparatus and a client. The augmented reality glasses comprise the apparatus. Thus, hearing-impaired people can understand the content of the talk of others, thereby implementing communication between the hearing-impaired people and the others.

Description

Method, device and system for assisting the hearing impaired and augmented reality glasses

Technical field

The present invention relates to the field of augmented reality technology, and in particular, to a method, device, and system for assisting the hearing impaired, and augmented reality glasses.

Background technique

Augmented Reality (AR) technology is a technology that calculates the position and angle of an image in real time, superimposes the corresponding image, video, and 3D model on the image, and then fuses the virtual world with the real world. The AR client can combine real-time image recognition of the offline environment of the user with the pictures stored directly in its local image recognition, and according to the pre-configured display of the identified offline targets in the real scene The effect is enhanced to display the corresponding display data. With the development of technology, the application of augmented reality technology is widespread, but for the hearing impaired, augmented reality technology has not helped them well.

At present, there are two main ways for hearing impaired people and hearing people to communicate: sign language interpreters or hearing aids. However, these two communication channels have certain problems for the hearing impaired.

Summary of the Invention

An object of the present invention is to provide a method, a device, and a system for assisting a hearing-impaired person, and augmented reality glasses, which can make the hearing-impaired person understand the content of another person's speech.

In order to achieve the above object, an aspect of the present invention provides a method for assisting a hearing impaired person, the method comprising: receiving a voice of at least one sound source; identifying a voice of each of the at least one sound source to Converting the speech of each sound source in the at least one sound source into text expressed in a first preset target language; and displaying the text converted by the speech of each sound source in the at least one sound source.

Optionally, the method further includes: receiving text; converting the received text into speech expressed in a second preset language; and playing the converted speech.

Optionally, before the receiving the voice of at least one sound source and / or the received text, the method further includes: receiving a response to the first preset target language and / or the second preset target language. set up.

Optionally, the method further comprises: determining location information of the hearing impaired person; and sending the location information to a mobile terminal and / or client, so that the mobile terminal and / or client obtains the location in real time information.

Optionally, before sending the location information to a mobile terminal and / or client, the method further includes: receiving a setting for a contact, wherein the mobile terminal and / or client is a contact with the selected one A mobile terminal and / or client corresponding to a person.

Accordingly, another aspect of the present invention provides a device for assisting a hearing impaired person, the device comprising: a voice receiving module for receiving voice of at least one sound source; a voice recognition module for identifying the at least one A voice of each sound source in the sound source to convert the voice of each sound source in the at least one sound source into text expressed in a first preset target language; and a display module for displaying the at least one Text converted from the voice of each source in the sound source.

Optionally, the device further includes: a text receiving module for receiving text; a text conversion module for converting the received text into speech expressed in a second preset language; and a voice playback module for playing the text Transformed Voice.

Optionally, the device further includes a language setting module configured to receive the first preset target before the voice receiving module receives the voice of at least one sound source and / or the text receiving module receives the text. Language and / or the setting of the second preset target language.

Optionally, the display module is a near-eye display.

Optionally, the near-eye display is a see-through near-eye display.

Optionally, the device further includes: a positioning module for determining position information of the hearing impaired; and a communication module for sending the position information to a mobile terminal and / or a client, so that the mobile terminal And / or the client obtains the location information in real time.

Optionally, the device further includes: a contact setting module configured to receive the setting of the contact before the communication module sends the location information to the mobile terminal and / or the client, wherein the mobile terminal And / or the client is a mobile terminal and / or client corresponding to the selected contact.

In addition, another aspect of the present invention provides an augmented reality glasses, which includes the above-mentioned device.

In addition, another aspect of the present invention provides a system for assisting a hearing impaired person, the system including the device described above, and a client.

In addition, another aspect of the present invention provides a machine-readable storage medium, where the machine-readable storage medium stores instructions, and the instructions are used to cause a machine to execute the foregoing method.

Through the above technical solution, the speech of each sound source in at least one sound source is converted into text and the converted text of each sound source is displayed. In this way, the hearing impaired can understand the content of other people's speech by reading the words, thereby achieving hearing impairment Communication and communication between people and others.

Other features and advantages of the present invention will be described in detail in the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the description. Together with the following specific implementations, the drawings are used to explain the embodiments of the present invention, but not to limit the embodiments of the present invention. In the drawings:

1 is a flowchart of a method for assisting a hearing-impaired person according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating an example of displaying text converted by each sound source of at least one sound source according to another embodiment of the present invention; FIG.

FIG. 3 is an exemplary diagram using arrows to indicate directions according to another embodiment of the present invention; FIG.

FIG. 4 is an exemplary diagram of an orientation provided by another embodiment of the present invention; FIG.

FIG. 5 is an exemplary diagram showing the position of a sound source and the text converted by voice according to another embodiment of the present invention; FIG.

FIG. 6 is an exemplary diagram showing the positions of multiple sound sources and the text converted by voice according to another embodiment of the present invention; FIG.

7 is a flowchart of a method for assisting a hearing-impaired person according to another embodiment of the present invention; and

FIG. 8 is a structural block diagram of a device for assisting a hearing impaired person according to another embodiment of the present invention.

Reference Signs

1 voice receiving module 2 voice recognition module

3 display module

detailed description

The specific implementations of the embodiments of the present invention will be described in detail below with reference to the drawings. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the embodiments of the present invention, and are not intended to limit the embodiments of the present invention.

An aspect of an embodiment of the present invention provides a method for assisting a hearing impaired person. FIG. 1 is a flowchart of a method for assisting a hearing impaired person according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps.

In step S10, the voice of at least one sound source is received.

In step S11, the speech of each sound source in the at least one sound source is identified, so as to convert the speech of each sound source in the at least one sound source into a text expressed in a first preset target language. For example, speech recognition technology is used to convert speech into text.

In step S12, the text converted by the voice of each of the at least one sound source is displayed. An example of displaying text converted by each sound source of at least one sound source may be shown in FIG. 2, and the sound source may be displayed in an orderly manner in the sound source in FIG. 2. In addition, the sound sources may be displayed in the order of the left and right of the sound sources, or displayed in other arrangements, which is not limited. In addition, when displaying the speech-transformed text of a certain sound source, if all the text cannot be displayed in one line, the text may be displayed in a new line or scrolled.

The speech of each sound source in at least one sound source is converted into text and the text converted by each sound source is displayed. In this way, the hearing impaired person can understand the content of other people's speech by reading the words, and realizes the hearing loss Communication and exchange. In addition, with the method described in the embodiment of the present invention, the user's operation experience is extremely light, and he / she can “hear” information beyond his ability without operating a technical system at all. In addition, it should be noted that the method for assisting the hearing impaired provided by the embodiment of the present invention is not only applicable to hearing impaired persons, but also applicable to ordinary persons.

Optionally, in the embodiment of the present invention, there may be many ways to display text. For example, the preset foreground color and the preset background color are used for display, and the preset foreground color and the preset background color are different colors. For example, the preset foreground color is white and the preset background color is black to display white characters on a black background; or the preset foreground color is black and the preset background color is white to display black characters on a white background. For another example, the preset foreground color is white, the preset background color is green, and the white text on a green background is displayed; or the preset foreground color is green, the preset background color is white, and the green text on a white background is displayed. In this way, the user can distinguish the text more clearly. For example, the way to display text may also be to display the text corresponding to different sound sources by alternately changing the color of the preset foreground color and the preset background color, that is, the preset foreground color and the preset background color are different colors. The order of voices, when the sound sources corresponding to the received adjacent voices are different sound sources, the preset foreground color and the preset background color are alternately changed; when the sound sources corresponding to the received adjacent voices are the same sound source, The preset foreground and background colors do not change colors. The colors of the preset foreground color and the preset background color can be limited according to actual conditions. For example, the preset foreground color is white, the preset background color is black, and the white text on a black background is displayed; or the preset foreground color is black. , The preset background color is white, and black characters on a white background are displayed; for example, the preset foreground color is white, the preset background color is green, and white characters on a green background are displayed; or the preset foreground color is green, and the preset background color is white Displays green text on a white background. The following exemplarily uses the preset foreground color to be white and the preset background color to be green to introduce alternately changing the preset foreground color and the preset background color to display text corresponding to different sound sources. If a certain voice (named as the first voice, the name is only for narrative purposes, without limitation) corresponds to the first sound source, and the text converted by the first voice is displayed in white on a green background; according to the order of receiving voices, the first The next sound of the voice (named second sound) corresponds to a different sound source from the first sound source (named second sound source corresponds to the second sound source), and is used when displaying the text converted by the second voice Green text on a white background; according to the order of the received speech, the sound source corresponding to the next speech of the second speech (named the third speech) is the second sound source, and the text converted from the third speech is still displayed in green on a white background Word; according to the order of received speech, the sound source corresponding to the next speech of the third speech (named the fourth speech) and the second sound source are different sound sources (the sound source corresponding to the fourth speech is named the third sound source, The third sound source can be the first sound source or other sound sources, as long as it is not the second sound source.) When displaying the text converted by the fourth voice, use white text on a green background. Correspondence of received voice All the information is displayed, and the information corresponding to the voice includes the text converted by the voice.

Optionally, in the embodiment of the present invention, the method for assisting the hearing impaired further includes: determining the position of each sound source in the at least one sound source based on the received speech of each sound source in the at least one sound source. . For each sound source in the at least one sound source, the displayed content includes the position in addition to the text converted by the voice.

Wherein, determining the position of the sound source may be based on the time when the voice emitted from the sound source is received. For example, a voice receiving module for receiving voice of at least one sound source includes a plurality of voice acquisition modules, the plurality of voice acquisition modules are disposed at different positions, and the plurality of voice acquisition modules receives voices from the same sound source. Time is different. For each sound source in the at least one sound source, the position of the sound source is determined according to the difference in the time when the speech arrives at the multiple speech collection modules, that is, according to the time difference between the time when the speech reaches the multiple speech collection modules. Optionally, in the embodiment of the present invention, the voice collection module may be a microphone, and the voice receiving module may be a microphone array. For example, the microphone array may include 2, 4, 6, 7, or 8 microphones.

Optionally, in the embodiment of the present invention, the reference point of the azimuth may be set according to the actual situation, for example, it may be the position where the voice receiving module is located. Specifically, it may be any one of the plurality of voice acquisition modules, or may be an intermediate position of the plurality of voice acquisition modules. In addition, when the speech receiving module is worn by the hearing-impaired person or is not far from the hearing-impaired person, the speech-receiving module is used as the reference point, which is actually the hearing-impaired person as the reference point. Azimuth knows where the sound source is relative to itself.

Optionally, in the embodiment of the present invention, the orientation may include a direction and / or a distance. Optionally, in the embodiment of the present invention, an arrow may be used to indicate the direction. The arrow is located in an area delimited by a circle. The starting point of the arrow is the origin of the circle. The origin is equivalent to the position of the hearing impaired and the arrow deviates. The vertical axis passing through the circle is at an angle. As shown in FIG. 3, the vertical dotted line in the figure is the vertical axis passing through the circle. In addition, the horizontal axis of the circle is used as a reference. As shown by the horizontal dashed line in FIG. 3, when the arrow is located above the horizontal axis, it means that the sound source is in front of the hearing impaired; when the arrow is located below the horizontal axis, Indicates that the sound source is behind the hearing impaired. For example, taking the direction example shown in 3 as an example, the sound source indicated by the arrow is in front of the hearing impaired person. In addition, the way in which the direction of the sound source is indicated by arrows can also be interpreted as the direction is indicated by a clock. The circle represents the dial, and the vertical axis located in the upper half of the horizontal axis of the circle indicates the 12 o'clock direction. According to the angle at which the arrow deviates from 12 o'clock, it is determined that the sound source is about the clock direction. Taking the direction example shown in FIG. 3 as an example, the sound source indicated by the arrow is about 10 o'clock. In addition, in a case where the azimuth includes a direction and a distance, the azimuth can be expressed using an example as shown in FIG. 4. It should be noted that the position of the display distance can be set according to the actual situation, and there is no limitation on this. In addition, in a case where the azimuth includes only the distance, only the distance may be displayed. In particular, when it is determined that the sound source comes from the hearing impaired person based on the received voice, when an arrow is used to indicate the direction of the sound source, "O" or "●" is displayed at the center of the circle to indicate the direction of the sound source. In addition, in the embodiment of the present invention, text may also be used to describe the orientation. For example, taking the orientation shown in FIG. 4 as an example, the text "direction is ten o'clock and distance is 50 cm" may be displayed.

In the case where the displayed content includes the text of orientation and speech conversion, for each sound source, an example of the displayed content may be as shown in FIG. 5. In addition, it should be noted that FIG. 5 only shows by way of example the position of the area where the orientation and the speech-transformed text are displayed. The position of the two display areas can be selected according to the actual situation. For the two display areas, The position is not limited. In addition, when displaying the position of each sound source and the text converted by speech in at least one sound source, the example shown in FIG. 5 may be used for display. In addition, when displaying the positions of multiple sound sources and the text converted by speech, the sound sources can be displayed in a sequence of up and down, as shown in FIG. 6. In addition, the sound sources may be displayed in the order of the left and right of the sound sources, or displayed in other arrangements, which is not limited.

Optionally, in the embodiment of the present invention, the manner of displaying the orientation and text simultaneously may also be displayed by using the above-mentioned manner of displaying text. Compared to displaying only the text, displaying the orientation simultaneously means that a sound source is provided at the corresponding text. The orientation and basic principle are the same, and will not be repeated here.

FIG. 6 is a flowchart of a method for assisting a hearing impaired person according to another embodiment of the present invention. The difference from the method shown in FIG. 1 is that the method shown in FIG. 6 further includes the following content.

In step S73, a character is received. Among them, there are many ways for the hearing impaired to enter text. For example, connecting a keyboard enables a hearing impaired person to enter text through the keyboard. For example, by connecting an interactive interface, a hearing impaired person can enter text through the interactive interface. In addition, you can connect to the client, and the hearing impaired can enter text through the client. Optionally, the client may be a mobile APP.

In step S74, the received text is converted into speech expressed in a second preset language, for example, text-to-speech conversion is achieved by using TTS technology.

In step S75, the converted voice is played.

In this way, when the hearing-impaired person does not have the pronunciation ability or the pronunciation ability is limited, the hearing-impaired person can input text to express his meaning and communicate with others.

It should be noted that steps S73 to S75 may also be performed before steps S70 to S72, and this is not limited.

Optionally, in the embodiment of the present invention, before receiving voice and / or text from at least one sound source, the method further includes: receiving settings for the first preset target language and / or the second preset target language. set. In this embodiment, the “hearing impaired person” may not be a person with limited hearing ability, but may be a “first-view equal hearing impaired person” who does not understand the language of others Of people who are different in their language are "second-view parity hearing impaired." Set the first preset target language used by the “first-view hearing-impaired person” to convert the voice received from at least one sound source into text expressed in the first preset target language. "Understand what the other person is talking to by looking at the converted text. Set a second preset target language used by others who communicate with the “second-view hearing-impaired person”, and convert the text entered by the “second-view hearing-impaired person” into an expression in the second preset target language Voice, others can understand the meaning of "as a second hearing impaired" by listening to the voice. In this way, communication between the “persons with hearing impairment” and others is realized.

Optionally, in the embodiment of the present invention, the method for assisting the hearing impaired may further include the following: determining the position information of the hearing impaired; and sending the position information to the mobile terminal and / or the client, so that the mobile The terminal and / or the client obtains the location information in real time. In this way, the contact person related to the hearing impaired person can obtain the position information of the hearing impaired person in real time to confirm whether he is safe, and can find him as soon as possible when a situation occurs. Among them, in the embodiment of the present invention, the position information of the hearing-impaired person can be determined in real time through GPS positioning technology.

Optionally, in the embodiment of the present invention, before sending the location information to the mobile terminal and / or the client, the method further includes: receiving a setting of a contact, wherein the mobile terminal and / or the client are connected with the selected The mobile terminal and / or client corresponding to the specified contact. When there are many contacts related to the hearing impaired, in different situations, some contacts can appear beside the hearing impaired in time to help the hearing impaired, thereby, When sending the hearing-impaired person's location information, the hearing-impaired person's location information can be sent directly to the contacts who can appear in time, so that when the hearing-impaired person has difficulty, he can reach the hearing-impaired person's location as soon as possible, helping the hearing-impaired person to solve the problem . In addition, the correspondence relationship between the hearing-impaired persons and the mobile terminals and / or clients used by them can be set in advance.

In addition, in the embodiment of the present invention, the method for assisting the hearing impaired may further include the following: according to the order of receiving voices, recording the position and / or text corresponding to each sound source, and combining the position and / Or text is stored locally or in the cloud to further help the hearing impaired to remember and share afterwards.

Accordingly, another aspect of the embodiments of the present invention provides a device for assisting a hearing impaired person. FIG. 8 is a device for assisting the hearing impaired according to another embodiment of the present invention. As shown in FIG. 8, the device includes a voice receiving module 1, a voice recognition module 2, and a display module 3. The voice receiving module 1 is configured to receive voice of at least one sound source. The voice recognition module 2 is configured to recognize the voice of each sound source in the at least one sound source, so as to convert the voice of each sound source in the at least one sound source into a text expressed in a first preset target language. The display module 3 is configured to display text converted by the voice of each sound source in the at least one sound source.

The speech of each sound source in at least one sound source is converted into text and the converted text of each sound source is displayed. In this way, the hearing impaired person can understand the content of other people's speech by reading the words, thus achieving the hearing impaired person and others. Communication and exchange. In addition, with the method described in the embodiment of the present invention, the user's operation experience is extremely light, and he / she can “hear” information beyond his ability without operating a technical system at all. In addition, it should be noted that the method for assisting the hearing impaired provided by the embodiment of the present invention is not only applicable to hearing impaired persons, but also applicable to ordinary persons.

Optionally, in the embodiment of the present invention, the device for assisting the hearing impaired further includes a determining module, which is configured to determine at least one sound source based on the voice of each sound source of the received at least one sound source. The position of each sound source in. The display module is further configured to display the position of each sound source in the at least one sound source.

Optionally, in the embodiment of the present invention, the device for assisting the hearing impaired further includes a text receiving module for receiving text; a text conversion module for converting the received text into a second preset language Expressed speech; and a speech playback module for playing the converted speech.

Optionally, in the embodiment of the present invention, the device further includes: a language setting module, configured to receive the first preset before the voice receiving module receives the voice of the at least one sound source and / or the text receiving module receives the text. Setting of the target language and / or the second preset target language.

Optionally, in the embodiment of the present invention, the display module may be a near-eye display. The distance between the near-eye display and the eyeball may be less than 2 cm. In addition, the near-eye display may include a see-through near-eye display or a non-see-through near-eye display. In this way, it is possible to present in front of the eyes the text converted by each sound source, or the position of each sound source, and the text converted by the sound. Preferably, in the embodiment of the present invention, the display module may be a see-through near-eye display. In this way, while not affecting the hearing-impaired person from observing other things, the hearing-impaired person can understand the speech-translated text of each sound source or the position of each sound source and the speech-translated text by watching "subtitles".

Optionally, in the embodiment of the present invention, the device further includes: a positioning module for determining the location information of the hearing impaired; and a communication module for sending the location information to the mobile terminal and / or the client so that the mobile The terminal and / or the client obtains the location information in real time.

Optionally, in the embodiment of the present invention, the device further includes: a contact setting module, configured to receive the setting of the contact before the communication module sends the location information to the mobile terminal and / or the client, The mobile terminal and / or client is a mobile terminal and / or client corresponding to the selected contact.

In addition, in the embodiment of the present invention, the device for assisting the hearing impaired further includes a storage module. The storage module is used to record the position and text corresponding to each sound source according to the order of receiving voices, to further help the hearing impaired to remember and share afterwards. Wherein, the storage module records the orientation and text corresponding to each sound source, which may be storing the orientation and text corresponding to each sound source on the local end or in the cloud.

The specific working principle and benefits of the device for assisting the hearing impaired provided by the embodiments of the present invention are similar to the specific working principle and benefits of the method for assisting the hearing impaired provided by the embodiments of the present invention, and will not be repeated here.

In addition, another aspect of the embodiments of the present invention provides a system for assisting the hearing impaired, the system includes: the device described in the above embodiments and a client. The client can receive text input by the user; and / or can receive location information of the hearing impaired.

In addition, another aspect of the embodiments of the present invention provides an augmented reality glasses. The augmented reality glasses include the devices described in the above embodiments.

The augmented reality glasses include an electronic circuit system that supports the operation of the device described in the foregoing embodiment. The electronic circuit system includes a power supply, a processor, a network connection, and other modules, as well as a voice receiving module, a text receiving module, and a voice playing module. In addition, the electronic circuit system may further include an externally visible human-machine interface module and buttons and / or a touch control panel. The processor includes the determination module, the speech recognition module, and the text conversion module described in the foregoing embodiments. The human-machine interface module includes a display module. The processor can also perform offline speech recognition locally, or online speech recognition in the cloud via a network connection.

Optionally, in the embodiment of the present invention, the touch control panel, buttons, and / or voice receiving module may be provided on the glasses or glasses accessories of the augmented reality glasses, for example, on the temples, frames, or lenses. Optionally, in the embodiment of the present invention, the voice receiving module may be disposed on the frame, on the same temple or on a different temple, or at a position close to the ears (both ears or monaural), to reach the pole. Try to fit the ear. For example, when the voice receiving module is a microphone array and the microphone sub-array includes two microphones, the two microphones are respectively disposed on two frames, or are disposed on different positions of the same temple, or are respectively disposed on two On the temples. When the number of microphones included in the microphone array is greater than two, a plurality of microphones may also be respectively arranged on the frame and / or the temple according to the actual situation. In addition, when using a microphone array, there is a difference in the time and intensity of speech reaching each microphone in the microphone array. By calculating the difference, a clearer sound that is more convenient to process can be obtained. In addition, compared to using a single microphone or a noise reduction microphone, the use of a microphone array is of great significance. Using a microphone array does not require the distance of the sound source from the voice receiving module. In addition, the use of the microphone array can adapt to various distances and can meet the requirements in most communication scenarios. The distance refers to the distance between the sound source and the microphone array. For example, it can meet the requirements of the following communication scenarios: two people talk individually, the distance between the sound source and the voice receiving module is between 50cm and 1m; in a multi-person group conversation, the sound source is between the distance of 1m and 2m from the voice receiving module; conference , The distance between the sound source and the voice receiving module is 3m; in class, the distance between the sound source and the voice receiving module is 3m to 5m, and so on.

In addition, in the case where the display module is a near-eye display, it is realized that the text converted by each sound source or the position of each sound source and the text converted by the sound are presented in front of the eyes. Among them, the near-eye display may be transparent or non-transparent. Further, in the case that the near-eye display is a see-through near-eye display, it does not affect the hearing-impaired person's observation of the display scene, and through the graphical instructions superimposed on the real scene, the hearing-impaired person can see each The text converted by a sound source or the position of each sound source and the text converted by a sound source make hearing-impaired people understand the voice information heard while watching "subtitles" or get similar information while understanding the voice information heard. To ordinary people's perception of location. In addition, in order to avoid distraction of the hearing impaired, the near-eye display may be a monochromatic display, which uses a preset background color and a preset foreground color to display the text or orientation and text corresponding to the sound source. In addition, the near-eye display can also be a color display. The background color and foreground color are alternately displayed to display the text or orientation and text corresponding to different sound sources. For the specific conversion method, refer to the content described in the foregoing embodiment. Fully avoid the deafness of the hearing impaired, so that the hearing impaired can focus on the content itself; meanwhile, the hearing impaired can conduct normal real-world communication without the discomfort of being interrupted and the need to change the focus of attention.

In addition, another aspect of the embodiments of the present invention further provides a machine-readable storage medium, where the machine-readable storage medium stores instructions, and the instructions are used to cause a machine to execute the method described in the foregoing embodiments.

In summary, the sound of each sound source in at least one sound source is converted into text and the converted text of each sound source is displayed. In this way, the hearing impaired can understand the content of other people's speech by reading the words, thereby achieving hearing impairment Communication and communication between people and others. The text input by the hearing impaired is converted into speech and the converted speech is played. In this way, when the hearing impaired has no pronunciation ability or the pronunciation ability is limited, the hearing impaired person can express his meaning by entering text and communicate with others. In addition, the received speech is converted into words expressed in the language used by the "deaf hearing impaired" and / or the words entered by the "deaf hearing impaired" are converted into others who communicate with the "deaf The speech expressed in the language used thus realizes the communication between the "persons with hearing impairment" and others.

The optional implementations of the embodiments of the present invention have been described above in detail with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the foregoing implementations. Within the scope of the technical concept of the embodiments of the present invention, the embodiments of the present invention The technical solution of the present invention performs various simple modifications, and these simple modifications all belong to the protection scope of the embodiments of the present invention.

In addition, it should be noted that the specific technical features described in the foregoing specific embodiments can be combined in any suitable manner without conflict. In order to avoid unnecessary repetition, the embodiments of the present invention do not separately describe various possible combinations.

Those skilled in the art can understand that all or part of the steps in the method of the above embodiments can be completed by a program instructing related hardware. The program is stored in a storage medium and includes a number of instructions to enable a microcontroller, a chip, or a processor. (processor) executes all or part of the steps of the method described in each embodiment of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .

In addition, various combinations of the embodiments of the present invention can also be arbitrarily combined, as long as it does not violate the idea of the embodiment of the present invention, it should also be regarded as the content disclosed in the embodiment of the present invention.

Claims

A method for assisting the hearing impaired, characterized in that the method includes:

Receiving voice from at least one sound source;

Identifying the speech of each sound source in the at least one sound source to convert the speech of each sound source in the at least one sound source into text expressed in a first preset target language; and

Display text converted by the voice of each of the at least one sound source.
The method according to claim 1, further comprising:

Receive text

Converting received text into speech expressed in a second preset language; and

Play the converted voice.
The method according to claim 2, wherein before the receiving the voice of at least one sound source and / or the received text, the method further comprises: receiving a response to the first preset target language and / or Setting of the second preset target language.
The method according to any one of claims 1-3, further comprising:

Determining the location information of the hearing impaired; and

Sending the location information to a mobile terminal and / or a client, so that the mobile terminal and / or the client obtains the location information in real time.
The method according to claim 4, wherein before sending the location information to a mobile terminal and / or a client, the method further comprises: receiving a setting for a contact, wherein the mobile terminal and / or The client is a mobile terminal and / or client corresponding to the selected contact.
A device for assisting the hearing impaired, characterized in that the device includes:

A voice receiving module, configured to receive voice of at least one sound source;

A voice recognition module, configured to recognize the voice of each of the at least one sound source, so as to convert the voice of each of the at least one sound source into a text expressed in a first preset target language; as well as

A display module, configured to display text converted by the voice of each of the at least one sound source.
The device according to claim 6, further comprising:

Text receiving module for receiving text;

A text conversion module for converting the received text into speech expressed in a second preset language; and

The voice playback module is used to play the converted voice.
The apparatus according to claim 7, further comprising:

A language setting module configured to receive the first preset target language and / or the second preset language before the voice receiving module receives the voice of at least one sound source and / or the text receiving module receives the text Set the target language setting.
The device according to any one of claims 6 to 8, wherein the display module is a near-eye display.
The device according to claim 9, wherein the near-eye display is a see-through near-eye display.
The device according to claim 6, further comprising:

A positioning module for determining position information of the hearing impaired person; and

A communication module is configured to send the location information to a mobile terminal and / or a client, so that the mobile terminal and / or the client obtains the location information in real time.
The apparatus according to claim 11, further comprising:

A contact setting module, configured to receive a setting of a contact before the communication module sends the location information to a mobile terminal and / or a client, wherein the mobile terminal and / or the client The mobile terminal and / or client corresponding to the specified contact.
An augmented reality glasses, characterized in that the augmented reality glasses include the device according to any one of claims 6-12.
A system for assisting the hearing impaired, characterized in that the system includes:

The device of any one of claims 6-12; and

Client.
A machine-readable storage medium has instructions stored on the machine-readable storage medium, which are used to cause a machine to perform the method according to any one of claims 1-5.