CN114550430A - Character reminding method and device based on AR technology - Google Patents
Character reminding method and device based on AR technology Download PDFInfo
- Publication number
- CN114550430A CN114550430A CN202210447780.9A CN202210447780A CN114550430A CN 114550430 A CN114550430 A CN 114550430A CN 202210447780 A CN202210447780 A CN 202210447780A CN 114550430 A CN114550430 A CN 114550430A
- Authority
- CN
- China
- Prior art keywords
- information
- keywords
- keyword
- audio information
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000005516 engineering process Methods 0.000 title claims abstract description 55
- 238000004590 computer program Methods 0.000 claims description 15
- 230000015654 memory Effects 0.000 claims description 10
- 208000032041 Hearing impaired Diseases 0.000 abstract description 6
- 230000000694 effects Effects 0.000 abstract description 6
- 230000007613 environmental effect Effects 0.000 abstract description 4
- 239000012141 concentrate Substances 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 239000011521 glass Substances 0.000 description 4
- 238000001514 detection method Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/24—Reminder alarms, e.g. anti-loss alarms
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B7/00—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00
- G08B7/06—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00 using electric transmission, e.g. involving audible and visible signalling through the use of sound and light sources
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Emergency Management (AREA)
- Computational Linguistics (AREA)
- Business, Economics & Management (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Optics & Photonics (AREA)
- Telephone Function (AREA)
Abstract
The invention provides a character reminding method and device based on AR technology. Wherein, the method comprises the following steps: collecting audio information through a microphone; carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information; performing keyword recognition on the obtained text information based on a preset keyword; and displaying the text information through a display screen of the head-mounted AR equipment, wherein the display form of the keywords is different from the display form of other texts which are not the keywords. The method and the device can directly display the characters corresponding to the voice on the display screen of the head-mounted AR device, so that a hearing-impaired person does not need to concentrate the eyes on a mobile phone when communicating with other people, other activities of the user cannot be influenced, the voice and the environmental sound can be distinguished in a noisy environment, and the user can quickly pay attention to key information in the information displayed on the display screen of the head-mounted AR device.
Description
Technical Field
The invention relates to the technical field of Augmented Reality (AR for short), in particular to a character reminding method and device based on AR technology.
Background
At present, when a hearing-impaired person communicates with other people, the hearing-impaired person often uses an App mobile phone with a voice-to-text function, and the function of the mobile phone can convert voice into corresponding text in real time, so that the daily communication requirement of the hearing-impaired person is met.
However, the voice is converted into the text through the App carried by the mobile phone, and the user needs to always focus on the mobile phone to check the converted text content on the mobile phone, so that the user cannot see the expression of the speaker, the surrounding environment and the like at the same time, and other activities of the user are affected. In a noisy environment, speech and environmental sound exist simultaneously, text contents obtained through speech recognition and conversion are relatively disordered, and when the contents are browsed, key information is difficult to notice at the first time.
Disclosure of Invention
The invention provides a character reminding method and device based on AR technology, which are used for solving the defects that in the prior art, the App carried by a mobile phone converts voice into characters, a user needs to concentrate on the mobile phone all the time, other activities of the user are influenced, and key information is difficult to notice in a noisy environment at the first time.
In a first aspect, the present invention provides a text reminding method based on an AR technology, including:
collecting audio information through a microphone;
carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
performing keyword recognition on the obtained text information based on preset keywords;
and displaying the text information through a display screen of the head-mounted AR equipment, wherein the display form of the keywords is different from the display form of other texts which are not the keywords.
According to the AR technology-based character reminding method provided by the invention, the keyword recognition is carried out on the obtained character information based on the preset keywords, and the method comprises the following steps:
matching the obtained text information with a preset keyword, and judging whether the text information is the keyword or not;
if the text information is not the keyword, determining to display the text information in a first display form;
and if the text information is the keyword, determining to display the text information in a second display form.
According to the AR technology-based character reminding method provided by the invention, the keyword recognition is carried out on the obtained character information based on the preset keywords, and the method comprises the following steps:
performing keyword recognition on the obtained text information through the head-mounted AR equipment based on a preset keyword;
further comprising:
the head-mounted AR equipment is connected with intelligent equipment, and keywords set by the intelligent equipment are received;
and storing the received keywords, or updating the stored keywords by the received keywords to obtain the preset keywords.
According to the AR technology-based character reminding method provided by the invention, the step of carrying out voice recognition on the collected audio information to obtain the character information corresponding to the voice information in the audio information comprises the following steps:
performing voice recognition on the collected audio information through the head-mounted AR equipment to obtain character information corresponding to the voice information in the audio information; or,
and carrying out voice recognition on the collected audio information through the intelligent equipment to obtain character information corresponding to the voice information in the audio information.
According to the character reminding method based on the AR technology, the method for collecting the audio information through the microphone comprises the following steps:
collecting audio information through a microphone provided by the head-mounted AR device; or,
and acquiring audio information through a microphone arranged on the intelligent equipment.
According to the character reminding method based on the AR technology, which is provided by the invention, the audio information is collected through the microphone, and the method comprises the following steps:
collecting audio information through a microphone arranged in an intelligent device connected with the head-mounted AR device;
the voice recognition is carried out on the collected audio information to obtain the character information corresponding to the voice information in the audio information, and the method comprises the following steps:
carrying out voice recognition on the collected audio information through the intelligent equipment to obtain character information corresponding to the voice information in the audio information;
the method for identifying the keywords of the obtained text information based on the preset keywords comprises the following steps:
performing keyword recognition on the obtained text information through the intelligent equipment based on preset keywords;
further comprising:
setting keywords through the intelligent equipment;
and storing the set keywords, or updating the stored keywords by the set keywords to obtain preset keywords.
According to the character reminding method based on the AR technology provided by the invention, the method further comprises the following steps:
and when the keywords are displayed on the display screen of the head-mounted AR device, the vibration motor arranged on the head-mounted AR device vibrates.
In a second aspect, the present invention further provides a text reminding device based on AR technology, including:
the audio acquisition module is used for acquiring audio information through a microphone;
the voice recognition module is used for carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
the keyword identification module is used for carrying out keyword identification on the obtained character information based on preset keywords;
and the character display module is arranged on the head-wearing AR equipment and is used for displaying the character information through a display screen of the head-wearing AR equipment, wherein the display form of the keyword is different from that of other characters which are not the keyword.
In a third aspect, the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the steps of any one of the above-mentioned character reminding methods based on AR technology when executing the program.
In a fourth aspect, the invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the above-mentioned text reminding method based on AR technology.
In a fifth aspect, the invention further provides a computer program product, on which a computer program is stored, and the computer program, when being executed by a processor, implements the steps of the above-mentioned text reminding method based on AR technology.
According to the character reminding method and device based on the AR technology, the audio information is collected through the microphone, the voice recognition is carried out on the collected audio information, the character information corresponding to the voice information in the audio information is obtained, the character information obtained through the voice recognition is displayed through the head-wearing AR equipment, the characters corresponding to the voice can be directly displayed on the display screen of the head-wearing AR equipment, a hearing-impaired person can communicate with other people without concentrating the eyes on a mobile phone, the expression of the speaker, the surrounding environment and the like can be seen at the same time, and other activities of the user cannot be influenced; by carrying out keyword identification on the obtained text information based on the preset keywords, the display form of the keywords is different from that of other non-keyword texts, so that the voice and the environmental sound can be distinguished in a noisy environment, and a user can quickly pay attention to the key information in the information displayed on the display screen of the head-mounted AR device.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow diagram of a text reminding method based on AR technology according to the present invention;
FIG. 2 is a flowchart illustrating an embodiment of a text reminding method based on AR technology according to the present invention;
FIG. 3 is a flowchart illustrating another embodiment of a text reminding method based on AR technology according to the present invention;
FIG. 4 is a flowchart illustrating a text reminding method based on AR technique according to another embodiment of the present invention;
fig. 5 is a schematic flow chart of an application scenario of the word reminding method based on the AR technology according to the present invention;
FIG. 6 is a schematic diagram of a structure of the text reminding device based on AR technology according to the present invention;
fig. 7 is a schematic diagram of a composition structure of the electronic device provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The text reminding method based on the AR technology of the invention is described below with reference to FIGS. 1-5.
Referring to fig. 1, fig. 1 is a schematic flow chart of a text reminding method based on AR technology according to the present invention, where the text reminding method based on AR technology shown in fig. 1 can be executed by a text reminding device based on AR technology, and as shown in fig. 1, the text reminding method based on AR technology at least includes:
audio information is collected 101 by a microphone.
In the embodiment of the present invention, the audio information may be acquired by setting a microphone on the head-mounted AR device, and the embodiment of the present invention does not limit the type of the head-mounted AR device and the type of the microphone set on the head-mounted AR device, for example, the head-mounted AR device may be an AR helmet, an AR glasses, or the like, and the microphone set on the head-mounted AR device may be an electret microphone or the like with a small volume; or audio information may be acquired by a smart device provided with a microphone, and the type of the smart device is not limited in the embodiment of the present invention, for example, the smart device may be a wearable smart device such as a smart phone, a tablet computer, and a smart watch.
In the embodiment of the invention, a switch can be arranged on the head-wearing AR equipment, and the microphone is started through the switch arranged on the head-wearing AR equipment to start to collect the audio information of the surrounding environment; or, an Application program (App for short) may be set in the smart device, and the microphone is started by starting the App set in the smart device, so as to start to acquire the audio information of the surrounding environment. The embodiment of the present invention does not limit the implementation manner of starting the microphone to collect the audio information of the surrounding environment.
And 102, carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information.
In the embodiment of the present invention, after the audio information is collected by the microphone, speech recognition may be performed on the collected audio information to obtain text information corresponding to the speech information in the audio information, and an implementation manner of performing speech recognition on the audio information in the embodiment of the present invention is not limited, for example, speech recognition may be performed on the audio information by using a gaussian Mixture Model-Hidden Markov Model (GMM-HMM), a Long Short Term Memory network (Long Short-Term Memory, LSTM), a Transformer Model, and the like.
In the embodiment of the present invention, after the audio information is collected by the microphone, preprocessing such as filtering, a/D conversion, pre-emphasis (Preemphasis), endpoint detection, and the like may be performed on the audio information collected by the microphone. For example, the collected audio information may be filtered by a filter circuit to suppress interference in the audio information, the filtered analog audio information may be converted into digital audio information by an a/D conversion circuit, a high-frequency portion of the digital audio information after the a/D conversion may be compensated by a pre-emphasis circuit, and a start point and an end point of speech information in the compensated audio information may be determined by an end point detection circuit.
And 103, performing keyword recognition on the obtained character information based on preset keywords.
In the embodiment of the present invention, the preset keyword may be determined according to a specific application scenario, which is not limited in the embodiment of the present invention, for example, in an application scenario of roll calling, the preset keyword may be a name of a user, and in an application scenario of queuing and calling, the preset keyword may be a serial number queued by the user. The setting mode of the keywords is not limited in the embodiments of the present invention, for example, the keywords may be input through an intelligent device connected to the head-mounted AR device, or the keywords stored in the head-mounted AR device or the intelligent device may also be used.
In the embodiment of the invention, after the collected audio information is subjected to voice recognition to obtain the corresponding text information, the text information obtained by voice recognition can be subjected to keyword recognition according to the preset keyword, so that different processing can be respectively carried out when the text information obtained by voice recognition is the preset keyword and is not the preset keyword according to the result of the keyword recognition. The implementation manner of performing keyword recognition on the text information is not limited in the embodiments of the present invention, and for example, the obtained text information may be matched with a preset keyword to determine whether the text information is the keyword, and if the text information is not the keyword, it is determined that the text information is displayed in the first display form, and if the text information is the keyword, it is determined that the text information is displayed in the second display form.
And 104, displaying the text information through a display screen of the head-mounted AR device, wherein the display form of the keywords is different from that of the texts without the keywords.
In the embodiment of the present invention, after performing keyword recognition on text information obtained by voice recognition according to a preset keyword, the text information obtained by voice recognition may be displayed through a display screen of a head-mounted AR device, and the display screen of the head-mounted AR device may adopt different display forms for the text information of the keyword and the non-keyword.
Optionally, for the keyword, the display screen of the head-mounted AR device may not only prompt through the change of the text color, but also make the keyword effectively displayed on the display screen of the head-mounted AR device. Optionally, the head-mounted AR device may further be provided with a vibration motor, and when the keywords are displayed on the display screen of the head-mounted AR device, the vibration motor may also vibrate to enhance the effect of the prompt.
According to the character reminding method based on the AR technology, provided by the embodiment of the invention, the audio information is collected through the microphone, the voice recognition is carried out on the collected audio information, the character information corresponding to the voice information in the audio information is obtained, the character information obtained through the voice recognition is displayed through the head-wearing AR equipment, and the characters corresponding to the voice can be directly displayed on the display screen of the head-wearing AR equipment, so that a hearing-impaired person can see the expression of the speaker, the surrounding environment and the like at the same time without concentrating the eyes on the mobile phone when communicating with other people, and other activities of the user cannot be influenced; by carrying out keyword identification on the obtained character information based on preset keywords, the display form of the keywords is different from that of other non-keyword characters, the voice and the environmental sound can be distinguished in a noisy environment, and a user can quickly pay attention to the key information in the information displayed on the display screen of the head-mounted AR equipment.
Referring to fig. 2, fig. 2 is a schematic flow chart of an embodiment of the text reminding method based on the AR technology, as shown in fig. 2, the text reminding method based on the AR technology at least includes:
audio information is collected 201 by a microphone provided by the head mounted AR device.
202, performing voice recognition on the collected audio information through the head-wearing AR device to obtain character information corresponding to the voice information in the audio information.
And 203, performing keyword recognition on the obtained character information based on a preset keyword through the head-mounted AR device.
And 204, displaying the text information through a display screen of the head-mounted AR device, wherein the display form of the keywords is different from that of the texts without the keywords.
In this embodiment, the collection of the audio information, the speech recognition of the audio information, the keyword recognition of the text information obtained by the speech recognition, and the display of the text information obtained by the speech recognition are all completed by the head-mounted AR device. For the collection of the audio information, the speech recognition of the audio information, the keyword recognition of the text information obtained by the speech recognition, and the description of the text information obtained by the speech recognition, reference may be made to the descriptions of 101, 102, 103, and 104 in fig. 1, so that the description will not be repeated here.
In this embodiment, can be connected with smart machine through wear-type AR equipment, receive the keyword that the user passes through smart machine setting, if wear-type AR equipment does not have the storage of keyword this moment, can save received keyword, accomplish the setting to the keyword, obtain preset keyword, if wear-type AR equipment has stored keyword this moment, can update the keyword that has stored with received keyword, the completion is to the setting of keyword, obtain preset keyword. The connection mode of the head-mounted AR device and the intelligent device is not limited in the embodiment of the invention, for example, the head-mounted AR device and the intelligent device can be connected through a data line, a Bluetooth connection or a wifi connection.
The embodiment of the invention does not limit the implementation mode of the keywords set by the user through the smart device, for example, the user may input the keywords through the virtual keyboard of the smart phone, or the user may also recognize the keywords through the scanning function of the smart phone, or the user may also recognize the keywords through the gesture recognition function of the smart wristband, or the user may also input the keywords through the virtual keyboard of the head-mounted AR device through the smart wristband.
Referring to fig. 3, fig. 3 is a schematic flow chart of another embodiment of the text reminding method based on the AR technology, as shown in fig. 3, the text reminding method based on the AR technology at least includes:
and 301, collecting audio information through a microphone arranged on the intelligent equipment.
And 302, performing voice recognition on the collected audio information through the intelligent equipment to obtain character information corresponding to the voice information in the audio information.
303, performing keyword recognition on the obtained text information based on a preset keyword through the head-mounted AR device.
And 304, displaying the text information through the display screen of the head-mounted AR device, wherein the display form of the keywords is different from that of the texts without the keywords.
In this embodiment, the collection of the audio information and the voice recognition of the audio information are completed by the smart device, and the keyword recognition of the text information obtained by the voice recognition and the display of the text information obtained by the voice recognition are completed by the head-mounted AR device. For the collection of the audio information, the speech recognition of the audio information, the keyword recognition of the text information obtained by the speech recognition, and the description of the text information obtained by the speech recognition, reference may be made to the descriptions of 101, 102, 103, and 104 in fig. 1, so that the description will not be repeated here.
In this embodiment, can be connected with the smart device through head-mounted AR device, receive the text message that smart device speech recognition obtained, carry out keyword recognition to the text message that receives. The head-mounted AR equipment can also receive keywords set by a user through the intelligent equipment, if the keywords are not stored in the head-mounted AR equipment at the moment, the received keywords can be stored, the setting of the keywords is completed, the preset keywords are obtained, and if the keywords are stored in the head-mounted AR equipment at the moment, the stored keywords can be updated through the received keywords, the setting of the keywords is completed, and the preset keywords are obtained. The embodiment of the invention does not limit the connection mode of the head-mounted AR device and the intelligent device, for example, the head-mounted AR device and the intelligent device can be connected through a data line, a Bluetooth connection or a wifi connection.
Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a text reminding method based on AR technology according to another embodiment of the present invention, as shown in fig. 4, the text reminding method based on AR technology at least includes:
401, audio information is collected through a microphone provided by a smart device connected to the head-mounted AR device.
And 402, carrying out voice recognition on the collected audio information through the intelligent equipment to obtain character information corresponding to the voice information in the audio information.
And 403, performing keyword recognition on the obtained text information based on preset keywords through the intelligent device.
And 404, displaying the text information through a display screen of the head-mounted AR device, wherein the display form of the keywords is different from that of the texts without the keywords.
In this embodiment, the collection of the audio information, the speech recognition of the audio information, and the keyword recognition of the text information obtained by the speech recognition are completed by the smart device, and the display of the text information obtained by the speech recognition is completed by the head-mounted AR device. For the collection of the audio information, the speech recognition of the audio information, the keyword recognition of the text information obtained by the speech recognition, and the description of the text information obtained by the speech recognition, reference may be made to the descriptions of 101, 102, 103, and 104 in fig. 1, so that the description will not be repeated here.
In this embodiment, the intelligent device may be connected to the head-mounted AR device, and the text information obtained by the speech recognition and the result of the keyword recognition are sent or transmitted to the head-mounted AR device, and the text information obtained by the speech recognition is displayed by the head-mounted AR device according to the result of the keyword recognition. The user can set up the keyword through smart machine, if smart machine did not have the storage of keyword this moment, can save the keyword that sets up, accomplishes the setting to the keyword, obtains preset keyword, if smart machine had stored the keyword this moment, can update the keyword that has stored with the keyword that sets up, accomplishes the setting to the keyword, obtains preset keyword. The connection mode of the head-mounted AR device and the intelligent device is not limited in the embodiment of the invention, for example, the head-mounted AR device and the intelligent device can be connected through a data line, a Bluetooth connection or a wifi connection.
Please refer to fig. 5, fig. 5 is a schematic flow diagram of an application scenario of the AR technology-based text reminding method according to the present invention, as shown in fig. 5, a mobile phone is connected to AR glasses via a data line or bluetooth, an App on the mobile phone is started, a microphone on the mobile phone is started to start collecting audio information of a surrounding environment, a keyword is set by the mobile phone, the mobile phone performs voice recognition on the audio information collected by the microphone to obtain text information corresponding to the voice information in the audio information, performs keyword recognition on the text information obtained by voice recognition according to the set keyword to determine whether the text information obtained by voice recognition is the keyword, if the text information obtained by voice recognition is not the keyword, it is determined that the text information is displayed in a first display form, if the text information obtained by voice recognition is the keyword, it is determined that the text information is displayed in a second display form, and sending the text information obtained by voice recognition and the display form of the text information to AR glasses, displaying the non-keyword text in a white font through a display screen of the AR glasses, and displaying the keyword in a yellow font.
The following describes the text reminding device based on the AR technology provided by the present invention, and the text reminding device based on the AR technology described below and the text reminding method based on the AR technology described above can be referred to correspondingly.
Referring to fig. 6, fig. 6 is a schematic diagram illustrating a structure of a text reminding device based on AR technology according to the present invention, the text reminding device based on AR technology shown in fig. 6 can be used to execute the text reminding method based on AR technology shown in fig. 1, and as shown in fig. 6, the text reminding device based on AR technology at least includes:
and the audio acquisition module 610 is used for acquiring audio information through the microphone.
And the voice recognition module 620 is configured to perform voice recognition on the acquired audio information to obtain text information corresponding to the voice information in the audio information.
The keyword recognition module 630 is configured to perform keyword recognition on the obtained text information based on a preset keyword.
And the character display module 640 is arranged on the head-mounted AR device and used for displaying character information through a display screen of the head-mounted AR device, wherein the display form of the keywords is different from that of the characters of other non-keywords.
Optionally, the keyword recognition module 630 includes:
and the judging unit is used for matching the obtained text information with a preset keyword and judging whether the text information is the keyword or not.
And the first processing unit is used for determining to display the character information in a first display form according to the judgment result of the judging unit if the character information is not a keyword.
And the second processing unit is used for determining to display the character information in a second display form according to the judgment result of the judging unit if the character information is the key word.
Optionally, the keyword recognition module 630 is disposed in the head-mounted AR device, and configured to perform keyword recognition on the obtained text information based on a preset keyword.
This characters reminding device based on AR technique still includes:
and the communication module is used for connecting the head-mounted AR equipment with the intelligent equipment and receiving the keywords set by the intelligent equipment.
And the storage module is used for storing the received keywords or updating the stored keywords by the received keywords to obtain preset keywords.
Optionally, the speech recognition module 620 is disposed in the head-mounted AR device, and is configured to perform speech recognition on the acquired audio information to obtain text information corresponding to the speech information in the audio information; or,
the voice recognition module 620 is disposed in the intelligent device and configured to perform voice recognition on the collected audio information to obtain text information corresponding to the voice information in the audio information.
Optionally, the audio collecting module 610 is disposed in the head-mounted AR device, and is configured to collect audio information through a microphone disposed in the head-mounted AR device; or,
the audio collecting module 610 is arranged in the intelligent device and used for collecting audio information through a microphone arranged in the intelligent device.
Optionally, the audio collecting module 610 is disposed in a smart device connected to the head-mounted AR device, and is configured to collect audio information through a microphone disposed in the smart device.
The voice recognition module 620 is disposed in the intelligent device and configured to perform voice recognition on the collected audio information to obtain text information corresponding to the voice information in the audio information.
The keyword recognition module 630 is disposed in the intelligent device, and configured to perform keyword recognition on the obtained text information based on a preset keyword.
This characters reminding device based on AR technique still includes:
and the input module is used for setting keywords for the intelligent equipment.
And the storage module is used for storing the set keywords or updating the stored keywords by the set keywords to obtain preset keywords.
Optionally, the word reminding device based on the AR technology further includes:
and the vibration module is used for generating vibration through a vibration motor arranged on the head-mounted AR equipment while displaying the keywords through a display screen of the head-mounted AR equipment.
Fig. 7 illustrates a physical structure diagram of an electronic device, and as shown in fig. 7, the electronic device may include: a processor (processor)710, a communication interface (communication interface)720, a memory (memory)730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. Processor 710 may call logic instructions in memory 730 to perform a method of text alerting based on AR technology, the method comprising:
collecting audio information through a microphone;
carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
performing keyword recognition on the obtained text information based on preset keywords;
and displaying the text information through a display screen of the head-mounted AR equipment, wherein the display form of the keywords is different from the display form of other texts which are not the keywords.
In addition, the logic instructions in the memory 730 can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In another aspect, the present invention further provides a computer program product, where the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the method for text reminding based on AR technology provided in the foregoing embodiments, where the method includes:
collecting audio information through a microphone;
carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
performing keyword recognition on the obtained text information based on preset keywords;
and displaying the text information through a display screen of the head-mounted AR equipment, wherein the display form of the keywords is different from the display form of other texts which are not the keywords.
In another aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented by a processor to execute the method for text reminding based on AR technology provided in the foregoing embodiments, and the method includes:
collecting audio information through a microphone;
carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
performing keyword recognition on the obtained text information based on preset keywords;
and displaying the text information through a display screen of the head-mounted AR equipment, wherein the display form of the keywords is different from the display form of other texts which are not the keywords.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A character reminding method based on AR technology is characterized by comprising the following steps:
collecting audio information through a microphone;
carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
performing keyword recognition on the obtained text information based on a preset keyword;
and displaying the text information through a display screen of the head-mounted AR equipment, wherein the display form of the keywords is different from the display form of other texts which are not the keywords.
2. The AR technology-based text reminding method according to claim 1, wherein the step of performing keyword recognition on the obtained text information based on preset keywords comprises:
matching the obtained text information with a preset keyword, and judging whether the text information is the keyword or not;
if the text information is not the keyword, determining to display the text information in a first display form;
and if the text information is the keyword, determining to display the text information in a second display form.
3. The AR technology-based text reminding method according to claim 1 or 2, wherein the step of performing keyword recognition on the obtained text information based on preset keywords comprises:
performing keyword recognition on the obtained text information through the head-mounted AR equipment based on a preset keyword;
further comprising:
the head-mounted AR equipment is connected with intelligent equipment, and keywords set by the intelligent equipment are received;
and storing the received keywords, or updating the stored keywords by the received keywords to obtain the preset keywords.
4. The AR technology-based text reminding method of claim 3, wherein the performing speech recognition on the collected audio information to obtain text information corresponding to the speech information in the audio information comprises:
performing voice recognition on the collected audio information through the head-mounted AR equipment to obtain character information corresponding to the voice information in the audio information; or,
and carrying out voice recognition on the collected audio information through the intelligent equipment to obtain character information corresponding to the voice information in the audio information.
5. The AR technology-based text reminding method of claim 4, wherein the collecting audio information through a microphone comprises:
collecting audio information through a microphone provided by the head-mounted AR device; or,
and acquiring audio information through a microphone arranged on the intelligent equipment.
6. The AR technology-based text reminding method according to claim 1 or 2, wherein the collecting audio information through a microphone comprises:
collecting audio information through a microphone arranged in an intelligent device connected with the head-mounted AR device;
the voice recognition is carried out on the collected audio information to obtain the character information corresponding to the voice information in the audio information, and the method comprises the following steps:
carrying out voice recognition on the collected audio information through the intelligent equipment to obtain character information corresponding to the voice information in the audio information;
the method for identifying the keywords of the obtained text information based on the preset keywords comprises the following steps:
performing keyword recognition on the obtained text information through the intelligent equipment based on preset keywords;
further comprising:
setting keywords through the intelligent equipment;
and storing the set keywords, or updating the stored keywords by the set keywords to obtain preset keywords.
7. The AR technology-based text reminding method of claim 1, further comprising:
and when the keywords are displayed on the display screen of the head-mounted AR device, the vibration motor arranged on the head-mounted AR device vibrates.
8. A word reminding device based on AR technique, its characterized in that includes:
the audio acquisition module is used for acquiring audio information through a microphone;
the voice recognition module is used for carrying out voice recognition on the collected audio information to obtain character information corresponding to the voice information in the audio information;
the keyword identification module is used for carrying out keyword identification on the obtained text information based on preset keywords;
and the character display module is arranged on the head-wearing AR equipment and is used for displaying the character information through a display screen of the head-wearing AR equipment, wherein the display form of the keyword is different from that of other characters which are not the keyword.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the AR technology-based text reminding method according to any one of claims 1 to 7 when executing the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the AR technology-based text reminding method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210447780.9A CN114550430A (en) | 2022-04-27 | 2022-04-27 | Character reminding method and device based on AR technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210447780.9A CN114550430A (en) | 2022-04-27 | 2022-04-27 | Character reminding method and device based on AR technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114550430A true CN114550430A (en) | 2022-05-27 |
Family
ID=81667451
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210447780.9A Pending CN114550430A (en) | 2022-04-27 | 2022-04-27 | Character reminding method and device based on AR technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114550430A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103869471A (en) * | 2014-01-09 | 2014-06-18 | 盈诺飞微电子(上海)有限公司 | Head voice recognition projector and system |
| CN104951435A (en) * | 2015-07-28 | 2015-09-30 | 陈包容 | Method and device for displaying keywords intelligently during chatting process |
| US20170243520A1 (en) * | 2014-11-12 | 2017-08-24 | Fujitsu Limited | Wearable device, display control method, and computer-readable recording medium |
| CN107995101A (en) * | 2017-11-30 | 2018-05-04 | 上海掌门科技有限公司 | A kind of method and apparatus for being used to switching to speech message into text message |
| CN108762494A (en) * | 2018-05-16 | 2018-11-06 | 北京小米移动软件有限公司 | Show the method, apparatus and storage medium of information |
| CN108962254A (en) * | 2018-06-11 | 2018-12-07 | 北京佳珥医学科技有限公司 | For assisting the methods, devices and systems and augmented reality glasses of hearing-impaired people |
| CN109032545A (en) * | 2018-06-11 | 2018-12-18 | 北京佳珥医学科技有限公司 | For providing the method and apparatus and augmented reality glasses of sound source information |
| US20190068529A1 (en) * | 2017-08-31 | 2019-02-28 | Daqri, Llc | Directional augmented reality system |
| CN110146988A (en) * | 2019-05-15 | 2019-08-20 | 东北大学 | A kind of wear-type augmented reality glasses system and its implementation |
| CN110730939A (en) * | 2017-11-29 | 2020-01-24 | 深圳市柔宇科技有限公司 | Information prompting method, device and equipment for head-mounted display |
| CN111358066A (en) * | 2020-03-10 | 2020-07-03 | 中国人民解放军陆军军医大学第一附属医院 | Protective clothing based on speech recognition |
| CN111526247A (en) * | 2020-04-16 | 2020-08-11 | 广州三星通信技术研究有限公司 | Method and device for displaying voice text |
| CN111751995A (en) * | 2020-06-11 | 2020-10-09 | 重庆工业职业技术学院 | Sound visualization monocular head-mounted AR (augmented reality) glasses device and implementation method thereof |
| CN212781941U (en) * | 2020-08-28 | 2021-03-23 | 国网河南省电力公司检修公司 | AR auxiliary system of operation and maintenance operation of transformer substation |
| CN113345440A (en) * | 2021-06-08 | 2021-09-03 | 北京有竹居网络技术有限公司 | Signal processing method, device and equipment and Augmented Reality (AR) system |
| CN113763940A (en) * | 2021-08-11 | 2021-12-07 | 北京乐驾科技有限公司 | Voice information processing method and system for AR glasses |
-
2022
- 2022-04-27 CN CN202210447780.9A patent/CN114550430A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103869471A (en) * | 2014-01-09 | 2014-06-18 | 盈诺飞微电子(上海)有限公司 | Head voice recognition projector and system |
| US20170243520A1 (en) * | 2014-11-12 | 2017-08-24 | Fujitsu Limited | Wearable device, display control method, and computer-readable recording medium |
| CN104951435A (en) * | 2015-07-28 | 2015-09-30 | 陈包容 | Method and device for displaying keywords intelligently during chatting process |
| US20190068529A1 (en) * | 2017-08-31 | 2019-02-28 | Daqri, Llc | Directional augmented reality system |
| CN110730939A (en) * | 2017-11-29 | 2020-01-24 | 深圳市柔宇科技有限公司 | Information prompting method, device and equipment for head-mounted display |
| CN107995101A (en) * | 2017-11-30 | 2018-05-04 | 上海掌门科技有限公司 | A kind of method and apparatus for being used to switching to speech message into text message |
| CN108762494A (en) * | 2018-05-16 | 2018-11-06 | 北京小米移动软件有限公司 | Show the method, apparatus and storage medium of information |
| CN108962254A (en) * | 2018-06-11 | 2018-12-07 | 北京佳珥医学科技有限公司 | For assisting the methods, devices and systems and augmented reality glasses of hearing-impaired people |
| CN109032545A (en) * | 2018-06-11 | 2018-12-18 | 北京佳珥医学科技有限公司 | For providing the method and apparatus and augmented reality glasses of sound source information |
| CN110146988A (en) * | 2019-05-15 | 2019-08-20 | 东北大学 | A kind of wear-type augmented reality glasses system and its implementation |
| CN111358066A (en) * | 2020-03-10 | 2020-07-03 | 中国人民解放军陆军军医大学第一附属医院 | Protective clothing based on speech recognition |
| CN111526247A (en) * | 2020-04-16 | 2020-08-11 | 广州三星通信技术研究有限公司 | Method and device for displaying voice text |
| CN111751995A (en) * | 2020-06-11 | 2020-10-09 | 重庆工业职业技术学院 | Sound visualization monocular head-mounted AR (augmented reality) glasses device and implementation method thereof |
| CN212781941U (en) * | 2020-08-28 | 2021-03-23 | 国网河南省电力公司检修公司 | AR auxiliary system of operation and maintenance operation of transformer substation |
| CN113345440A (en) * | 2021-06-08 | 2021-09-03 | 北京有竹居网络技术有限公司 | Signal processing method, device and equipment and Augmented Reality (AR) system |
| CN113763940A (en) * | 2021-08-11 | 2021-12-07 | 北京乐驾科技有限公司 | Voice information processing method and system for AR glasses |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021184837A1 (en) | Fraudulent call identification method and device, storage medium, and terminal | |
| JP6400445B2 (en) | Conversation analyzer, conversation analysis system, conversation analysis method, and conversation analysis program | |
| JP6392374B2 (en) | Head mounted display system and method for operating head mounted display device | |
| CN109040471A (en) | Emotive advisory method, apparatus, mobile terminal and storage medium | |
| CN105957514A (en) | Portable deaf-mute communication equipment | |
| WO2016203805A1 (en) | Information processing device, information processing system, information processing method, and program | |
| CN108766416B (en) | Speech recognition method and related product | |
| CN111063355A (en) | Conference record generation method and recording terminal | |
| CN113033245A (en) | Function adjusting method and device, storage medium and electronic equipment | |
| CN112420049A (en) | Data processing method, device and storage medium | |
| US10216732B2 (en) | Information presentation method, non-transitory recording medium storing thereon computer program, and information presentation system | |
| CN111261139A (en) | Character personification broadcasting method and system | |
| CN115101053A (en) | Emotion recognition-based conversation processing method and device, terminal and storage medium | |
| CN111046223A (en) | Voice assisting method, terminal, server and system for visually impaired | |
| CN111970295B (en) | Multi-terminal-based call transaction management method and device | |
| CN105812535A (en) | Method of recording speech communication information and terminal | |
| CN109361804B (en) | Incoming call processing method and mobile terminal | |
| CN114550430A (en) | Character reminding method and device based on AR technology | |
| WO2015156443A1 (en) | Cartoon-type mobile personal secretary service system | |
| CN103929532A (en) | Information processing method and electronic equipment | |
| CN114844985A (en) | Data quality inspection method, device, equipment and storage medium | |
| CN111683174B (en) | Incoming call processing method, device and system | |
| CN115019788A (en) | Voice interaction method, system, terminal equipment and storage medium | |
| CN116935851A (en) | Method and device for voice conversion, voice conversion system and storage medium | |
| CN114049875A (en) | TTS (text to speech) broadcasting method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220527 |
|
| RJ01 | Rejection of invention patent application after publication |