WO2022048455A1

WO2022048455A1 - Barrier free information access system and method employing augmented reality technology

Info

Publication number: WO2022048455A1
Application number: PCT/CN2021/113662
Authority: WO
Inventors: 姚登峰; 赵启超; 阿布都克力木·阿布力孜
Original assignee: 语惠科技(南京)有限公司
Priority date: 2020-09-04
Filing date: 2021-08-20
Publication date: 2022-03-10
Also published as: CN112002186A; CN112002186B

Abstract

A barrier free information access system and method employing augmented reality technology. The system comprises a computing processing terminal, and a barrier free information access terminal in a communication connection with the computing processing terminal. The computing processing terminal acquires requested data of a user, performs processing thereof to obtain HUD information, and sends same to the barrier free information access terminal. The barrier free information access terminal employs augmented reality technology to superimpose the HUD information on the real environment where the user is located and display same. Augmented reality technology is employed to enhance the capability of a barrier free information access terminal, thereby solving the problem of poor user experience. The invention enables a user to receive information in an immersive manner, such that the user can view relevant information without lowering the head, thereby increasing the safety of the user.

Description

An information barrier-free system and method based on augmented reality technology

technical field

The invention relates to the technical field of wearable smart devices, in particular to an information barrier-free system based on augmented reality technology.

Background technique

At present, China has entered the information age, but there are still difficulties for most people to receive information. For example, the hearing-impaired groups have hearing impairments, and the elderly due to changes in hearing, vision, memory, etc., these groups have weakened ability to receive information and decreased sensitivity to external things, resulting in communication difficulties. For the able-bodied group, due to their limited education, when they travel abroad, engage in foreign trade and other foreign affairs, they may not understand foreign languages, which may lead to obstacles to receiving information. Even when traveling abroad, they need map navigation information prompts. At a certain stage of life, able-bodied people may encounter obstacles to obtaining information, such as loss of temporary hearing due to long-term wearing of headphones, gradual loss of hearing due to certain reasons, and encounters in noisy environments such as underground coal mines and aircraft carriers. hearing problems. Compared with able-bodied people, people with disabilities rely more on the Internet. People with hearing impairments are inconvenient to live, and most of them rely on Internet products for food and clothing. Therefore, the barrier-free design and optimization of application products gives everyone the right to receive and choose information.

At present, there are smart phones, tablets, notebook computers, desktop computers, embedded terminals and other application APP or software focusing on solving such problems. The idea is to use human visual sensitivity to convert auditory information into visual information to achieve defect compensation. functions, such as voice recognition, machine translation, map navigation and other APPs and software. However, these APPs or software have shortcomings such as poor interactive experience. People usually use electronic devices such as mobile phones, notebook computers, tablets, and embedded terminals to read speech-recognized text, map navigation interfaces, and other information, which cannot be integrated with the real world. , users need to look down at electronic devices such as mobile phones, laptops, tablets, embedded terminals, etc., while looking up at the surrounding environment, which has a great impact on cervical vertebra compression production and directly poses a huge threat to physical health. According to authoritative data surveys, As early as 2016, a domestic study conducted a survey on 2,000 patients with cervical spondylosis. The data showed that compared with people under the age of 30, patients with cervical spondylosis accounted for nearly 40%. The younger age of cervical spondylosis has evolved into a trend and is increasing at an incredible rate. The rapid development of electronic products has brought great convenience to people's lives. People with bowed heads can be seen everywhere in public transportation, subways, streets and alleys. Chinese people spend an average of more than 3 hours on mobile phones and 4 hours on computers every day. That is to say, for at least 7 full hours, you will keep your head down or lean forward, which is unimaginable for the compression of the spine. When we bow our head, the cervical spine will bend forward, and the normal physiological curve of the cervical spine is forcibly changed, thus losing the "natural curve". This kind of bowing posture, which forcibly changes the original physiological curvature of the cervical spine, increases the burden on the cervical spine. The lower the head, the greater the burden on the cervical spine. Generally, the average weight of the human brain is about 5kg, but when the head and neck are bent forward at 15° and 30°, the neck will increase by 24 and 36 pounds, while at 60°, the cervical spine will bear up to 54 pounds, which is equivalent to An 8-year-old child is riding around his neck. If we keep our head down for a long time, our cervical vertebra will be under extra pressure, resulting in cervical vertebra strain. Accumulation to a certain extent will cause cervical spondylosis symptoms such as cervical vertebra deformation, nerve compression, chest tightness and dizziness. In severe cases, it will lead to paralysis. Spinal cord decompression is performed without surgical treatment, so this traditional method causes poor experience and has a greater impact on human health.

There is another kind of solution that can also solve this kind of problem, namely pure virtual reality (VR) technology, users can wear VR glasses to read the text after speech recognition, map navigation interface and other information, but this pure virtual reality (VR) technology The real world information is shielded, and only the required virtual world information is displayed. Users cannot make autonomous behaviors in the real world, such as the need to take notes while reading information; in addition, this pure virtual reality (VR) technology equipment is relatively It is bulky and needs to be worn for a long time. If the user is in a pure virtual reality (VR) environment for a long time, it will also cause discomfort such as dizziness, which goes against the original design of the smart device and cannot achieve good human-computer interaction.

SUMMARY OF THE INVENTION

In view of the current problem of difficulty in receiving information without barriers, the present invention provides an information barrier-free system based on augmented reality technology, which enables information barrier-free terminals through augmented reality technology to solve the problem of poor user experience, especially for special groups, such as For hearing-impaired people, the present invention can perfectly solve the problem of barrier-free reception of their information, so that users can communicate with the surrounding people and the environment in time. It enables users to obtain and utilize information in an equal, convenient and barrier-free manner under any circumstances, so as to bridge the digital divide and help users including special groups to integrate into the information society. The present invention can realize the user's immersive reception of information, so that the user can see the relevant information without lowering his head, thereby improving the user's own safety factor.

The present invention first provides an information barrier-free system based on augmented reality technology, which is characterized by comprising a computing processing terminal and an information barrier-free terminal that communicate with each other;

The computing and processing terminal includes a sports health management module, a speech recognition and machine translation module, and a map navigation module;

The sports health management module is used to process the user's sports health data to obtain the user's sports health status;

The voice recognition and machine translation module is used to identify the voice data of the user's surrounding environment, and determine whether it is an abnormal voice signal. If it is an abnormal voice signal, it will directly generate a corresponding prompt text sequence; Attributive language voice signal, if it is a set language voice signal, it is directly converted into a set language text sequence; if it is not a set language voice signal, it is judged whether it is a foreign language, if so, the voice data is recognized as a foreign language text sequence , and translated into the set language text sequence;

The map navigation module is used for planning a navigation route according to the user's sports health data, the user's geographic location data, and the user-specified starting point and destination data;

The computing processing terminal converts the data processing results of the sports health management module, the speech recognition and machine translation module, and the map navigation module into HUD information that can be displayed by the information barrier-free terminal;

The information barrier-free terminal superimposes and displays the HUD information and the real environment where the user is located based on the augmented reality technology.

Preferably, the user's exercise health data is acquired through a motion sensor provided by the computing processing terminal and/or a motion sensor provided by the information barrier-free terminal.

Preferably, the user geographic location data is acquired through a network positioning chip and/or a satellite navigation and positioning chip provided in the computing and processing terminal.

Preferably, when the voice recognition and machine translation module recognizes that the voice data is an abnormal voice signal, the computing and processing terminal generates corresponding prompting HUD information and sends it to the information barrier-free terminal, and issues an instruction to the information barrier-free terminal to turn on the vibration motor reminder and Alert users.

Preferably, when the sports health management module and the map navigation module detect that the user is performing dangerous sports or is in a dangerous area, the computing and processing terminal generates corresponding prompt HUD information and sends it to the information barrier-free terminal, and sends out The instruction turns on the vibration motor to remind and alert the user to the information barrier-free terminal.

Preferably, the information barrier-free terminal is augmented reality glasses, including an optical display module, a communication module, a motion sensor, an embedded processor, and a vibration motor, and the optical display module is responsible for linking the HUD information with the user's location. The real environment is superimposed and displayed, the communication module is responsible for communicating with the computing processing terminal, the motion sensor is responsible for collecting user health and motion data, the embedded processor is responsible for data processing, and the vibration motor is used to remind and warn the user.

Preferably, the barrier-free terminal is configured with a proximity sensor for implementing gesture control operations.

Another object of the present invention is to provide a speech recognition method based on the above-mentioned information barrier-free system, which is characterized in that it comprises the following steps:

S1. The user selects a speech recognition and machine translation module, and the computing and processing terminal collects the speech data of the user's surrounding environment through a microphone;

S2. The voice recognition and machine translation module recognizes the voice data of the user's surrounding environment, and judges whether it is an abnormal voice signal. If so, it directly generates a corresponding prompt text sequence, and enters S4; if it is not an abnormal voice signal, judge whether it is an abnormal voice signal. Set the language, if it is, it will be directly converted to the text sequence of the set language, and go to S4; if the voice signal is not in the set language, judge whether it is a foreign language, and if so, go to S3;

S3. The speech recognition and machine translation module recognizes the speech data as a foreign language text sequence, and translates it into a set language text sequence;

S4. The computing processing terminal further processes the obtained text sequence, and converts it into HUD information that the information barrier-free terminal prompts the user;

S5, the computing processing terminal sends the HUD information to the information barrier-free terminal;

S6. The information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;

S7. When a set state trigger event occurs, the information barrier-free terminal reminds and warns the user through a vibration motor.

Another object of the present invention is to provide a map navigation method based on the above-mentioned information barrier-free system, which is characterized in that it comprises the following steps:

S1, the user selects the map navigation module, specifies the starting point and end point information, the computing and processing terminal and/or the information barrier-free terminal collects the user's exercise health data through the motion sensor, and the computing and processing terminal collects the user's geographic location data through the positioning module;

S2. The map navigation module calculates the user's current movement state according to the user's exercise health data and the user's geographic location data, including movement mode, movement speed, and positioning data;

S3, the map navigation module performs the navigation route planning from the starting point to the end point in the specified navigation mode according to the current motion state of the user; the navigation mode includes but is not limited to avoiding retrograde, avoiding underground passages, and avoiding overpasses;

S4. The computing and processing terminal further processes the obtained map navigation information, and converts it into HUD information that the information barrier-free terminal prompts the user;

S7. When a set state trigger event occurs, the information barrier-free terminal reminds the user through a vibration motor.

Another object of the present invention is to provide a sports health management method based on the above-mentioned information barrier-free system, which is characterized by comprising the following steps:

S1. The computing processing terminal and/or the information barrier-free terminal automatically collects the user's exercise health data through the motion sensor or according to the user's instruction;

S2. The sports health management module calculates the current exercise state of the user according to the collected exercise health data of the user, including exercise mode, exercise speed, exercise mileage, and energy consumption;

S3. The computing and processing terminal further processes the obtained user's current motion state data, and converts it into HUD data that the information barrier-free terminal prompts the user;

S4. The computing processing terminal sends the HUD information to the information barrier-free terminal;

S5. The information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;

S6. When a set state trigger event occurs, the information barrier-free terminal reminds the user through a vibration motor.

Beneficial effects of the present invention:

1. Solve the problem of receiving information for special populations

The present invention improves life for healthy people, and changes fate for special groups. For example, it is of great help to special groups in terms of information acquisition, convenient travel, and communication. Innovative technologies such as voice interaction, the Internet of Things, and smart hardware have provided new solutions for them to improve their lives. According to the exemplary embodiment of the present invention, voice recognition text information, navigation HUD information, and sports health information are displayed in the information barrier-free terminal, which replaces other terminals such as mobile phones and solves the disadvantage of poor interactivity of existing terminals. At the same time, although the existing terminals based on augmented reality technology have voice recognition and map navigation functions, there are also some problems. For example, these terminals are usually designed for healthy people and young people, not for special groups such as hearing-impaired people. These terminals are generally Equipped with headphones to transmit audio and other information through the headphones, it cannot remind the hearing-impaired people, the elderly, and other special groups in need. However, the present invention is equipped with a vibration motor in the information barrier-free terminal, regardless of healthy people, hearing-impaired people or elderly people who are not accustomed to using earphones, can perfectly solve their information receiving problems, and it is convenient for users to communicate with the surrounding people and the environment in time. Communication helps to enable special groups to equally enjoy the right to a better life brought by technology.

2. Good user experience;

According to the exemplary embodiment of the present invention, the information barrier-free terminal can take the speaking situation of the other party, the actual road condition, and its own health as one of the factors for the information barrier-free terminal to prompt information, which not only helps the hearing-impaired people and other groups receive information barrier-free It can also help groups such as hearing people to understand the information more accurately and ensure the integrity of the information received without barriers. According to another exemplary embodiment of the present invention, the terminal can simultaneously display the dialogue speech recognition text, which effectively reduces the difficulty of receiving information, facilitates communication with the other party, and improves the user experience. Simultaneously displays real-time road condition pictures with various visual effects, which effectively reduces the difficulty of navigation and facilitates the improvement of user experience in many scenarios such as business trips and overseas travel. The invention optimizes ergonomic design based on long-term research on hearing-impaired users. The physiological characteristics and psychological characteristics of the hearing-impaired group are fully considered, so that the use process and method of the present invention conform to the usage habits of the hearing-impaired people. The user experience of the present invention is better, and the human-computer interaction friendliness is achieved for all groups such as hearing-impaired people and hearing people.

3. High user participation;

The information barrier-free terminal adopts enhanced display technology to superimpose voice data, navigation data, sports health data and the real environment to display the real environment. Users can obtain information such as speech, real-time road conditions, navigation, health data and other information without obstacles, with a great sense of immersion and participation. . According to an exemplary embodiment of the present invention, the information barrier-free terminal uses the augmented vision glasses as a carrier, combined with the motion sensor chip of the augmented reality glasses, so that the information is more comprehensive, and the user obtains the content beyond the general amount of information, which helps to better understand the information content to better achieve information accessibility goals. These are the scenes that are used the most every day by groups such as the hearing-impaired, so that they can receive and obtain information without barriers, thus making users highly engaged.

4. Facilitate effective management and control;

By calculating and processing the terminal's map navigation module, voice recognition module, and equipped with motion sensors and vibration motors on the information barrier-free terminal, the user's status, gait and other information during use can be obtained, which can effectively analyze the user's location information and exercise health. , and achieve real-time feedback and alert users when key goals are reached. For example, when it is recognized that the hearing-impaired group is crossing the road and other dangerous occasions, it will automatically remind the hearing-impaired group to pay attention to safety, so as to achieve effective control. , When the horn reminds, the voice recognition module of the computing and processing terminal can automatically give warning information and vibrate in real time.

5. Increase user efficiency:

Due to the adoption of augmented reality technology, users can enter the immersive learning or work process well, abandon the influence of the external environment, and facilitate users to devote themselves to the work of receiving information and focus on the content of the received information, which greatly improves the The quality and effectiveness of study or work. For example, groups such as the hearing-impaired people cannot offer dangerous experimental courses such as chemistry due to hearing impairment. According to an exemplary embodiment of the present invention, the information barrier-free terminal can provide information such as safety warnings in real time in combination with the content of the chemical experimental course, reminding the hearing-impaired people to follow The security steps are completed step by step to immerse users in the course and achieve the goal of increasing user efficiency.

Description of drawings

FIG. 1 is a structural block diagram of an information barrier-free system based on an augmented reality technology according to an embodiment of the present invention.

Fig. 2 is a flow chart of the speech recognition method based on the information barrier-free system of the present invention.

Fig. 3 is a flow chart of a map navigation method based on the information barrier-free system of the present invention.

Fig. 4 is a flow chart of the health management method based on the information barrier-free system of the present invention.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings.

This specific embodiment is only an explanation of the present invention, and it does not limit the present invention. Those skilled in the art can make modifications without creative contribution to the present embodiment as required after reading this specification, but as long as the rights of the present invention are used All claims are protected by patent law.

Example 1

As shown in FIG. 1 , the information barrier-free system includes a computing processing terminal, and an information barrier-free terminal communicatively connected to the computing processing terminal.

The computing and processing terminal carrier is any kind of smartphone, tablet, notebook computer, desktop computer, embedded terminal, etc. The hardware is equipped with wireless connection chips such as processor, microphone, Bluetooth connection chip, WLAN, etc. or supports the three major operators 5G/ 4G+/4G/3G/2G and other network standard chips, as well as navigation and positioning chips that support Beidou, GPS, Galileo, GLONASS, etc., and configure the corresponding first data processing module, AGPS assisted positioning module, data network positioning module, WIFI network positioning module , speech recognition and machine translation module, map navigation module, sports health management module, communication module, etc.

The information barrier-free terminal uses enhanced display glasses as a carrier, including optical display modules, wireless connection chips such as WLAN and Bluetooth, high-precision motion sensors, proximity sensors, power management chips, vibration motors, and miniature embedded processing in the carrier. hardware, and configure the corresponding second data processing module and communication module. The communication module may be a WIFI communication module, a Bluetooth communication module or a USB communication module.

The computing and processing terminal relies on its own hardware device to collect the required information, and after being processed by the built-in module, the information is sent to the information barrier-free terminal through the communication module, so as to superimpose the information with the real world and achieve the purpose of information barrier-free communication. .

Specifically, the information barrier-free system based on augmented reality technology provided by this embodiment includes a first data processing module and a second data processing module that communicate with each other, as well as a sports health management module, a speech recognition and machine translation module, and a map navigation module. . The first data processing module, sports health management module, speech recognition and machine translation module, and map navigation module are arranged in the computing processing terminal, and the second data processing module is arranged in the information barrier-free terminal. The sports health management module processes the user's sports health data to obtain the user's sports health status. The speech recognition and machine translation module is divided into two modules: speech recognition and machine translation. The speech recognition module recognizes the voice data of the user's surrounding environment. When the voice data is in the set language, it directly converts the voice data into a text sequence of the set language. Otherwise, the voice data is Input the machine translation module to translate the text sequence into the set language. The map navigation module performs navigation route planning according to the user's sports health data, the user's geographic location data, and the user-specified starting point and destination data. The first data processing module is responsible for converting the data processing results of the sports health management module, the speech recognition and machine translation module, and the map navigation module into HUD information that can be displayed by the information barrier-free terminal, and the second data processing module is responsible for receiving the first data processing module. For the HUD information sent, the optical module of the control information barrier-free terminal superimposes and displays the HUD information and the real environment where the user is located based on the augmented reality technology. The second data processing module is also responsible for sending the user's sports health data collected by the information barrier-free terminal to the sports health management module of the computing processing terminal for management.

The speech recognition module can be directly implemented using existing technologies. It can either conduct supporting technology research and development based on the onboard computing processing terminal processor, or directly call third-party solutions, such as the underlying technology of iFLYTEK, to transmit the collected voice information to the computer. The iFLYTEK server, after the iFLYTEK server completes the recognition, returns the recognized text, and then sends it back to the computing processing terminal. After the computing processing terminal is processed, it is sent to the information barrier-free terminal.

The machine translation module can be directly implemented using existing technologies. It can not only conduct supporting technology research and development according to the onboard computing processing terminal processor, but also directly call third-party solutions, such as Sogou, Baidu, Alibaba, Google and other companies' underlying machine translation technology. , the computing and processing terminal transmits the collected voice information or text information to the Sogou, Baidu, Alibaba or Google server. After the Sogou, Baidu, Alibaba or Google server completes the translation, it returns the translated text, and then sends it back to the computing and processing terminal , the processing terminal is processed and then sent to the information barrier-free terminal.

The sports health management module can be directly implemented using existing technologies. It can either conduct supporting research and development based on the sports health monitoring hardware on board, or directly purchase third-party solutions.

The map navigation module can be directly implemented using existing technologies, such as calling the underlying technologies of any navigation such as AutoNavi, Baidu, Tencent, and Google, and using their services.

At present, computing and processing terminals such as mobile phones have their own motion sensors and display screens, which can collect simple exercise health data, such as how many steps the user has taken, how much energy they consume, and how many kilometers they have walked. And perform data display, so in the present invention, the computing and processing terminal is not only a data acquisition device, a data processing device, but also a data display device.

The exercise health data obtained by the computing and processing terminal is not enough to judge the user's fine state. The information barrier-free terminal of the present invention is also equipped with a motion sensor to detect the user's own state, such as whether to bend over, whether to lie on his back, etc. By configuring high-precision motion sensors (such as 3-axis acceleration sensor, 3-axis gyroscope sensor, 3-axis geomagnetic sensor) to collect motion-related position, displacement, velocity, acceleration, vibration displacement, amplitude, wave propagation and other data, it is also possible to detect For more complex motion states of the human body, such as whether the user's exercise mode is running, walking, cycling, or swimming, the detection function is more refined, more comprehensive, and more powerful. In the present invention, the information barrier-free terminal is not only a data collection device, but also a data processing device, and also a data display device.

In this embodiment, the information barrier-free terminal carrier is augmented reality glasses, and its important component is an optical display module, which is responsible for projecting information on the terminal lens or directly displaying information on the terminal lens, so that it is superimposed in the real world, thereby being viewed by Perceived by the user's senses, to obtain a sensory experience beyond reality. In the terminal, the optimized text sequence or navigation interface overlaps with the real environment, and can coexist on the same lens. Therefore, when using, you only need to wear the augmented reality glasses, and then point your sight to the scene to be detected, so as to browse the real environment and the important information displayed at the same time, which is very convenient to use, and you do not need to look down at the calculation during the whole process Handling equipment frees hands and head.

The computing processing terminal relies on the built-in processor to optimize the information processed by the sports health management module, speech recognition and machine translation module, and map navigation module, and convert it into HUD information to adapt to augmented reality technology. The Chinese name of HUD is head-up display, also known as head-up display system. The information barrier-free terminal projects information such as speed, navigation, and text onto the photoelectric display module on the carrier, onto the lens or in front of the lens, or directly on the lens. form an image. In this way, users can see navigation and speed information without turning their heads or bowing their heads. Take the text sequence after speech recognition or machine translation as an example. When the text sequence is displayed on the computing and processing terminal, it is generally displayed in full screen or in a designated display window. When it is transmitted to the information barrier-free terminal, in order not to interfere with the user's observation In the world, it cannot be displayed on the screen on the glasses lens in full screen, and certain optimizations should be made, such as selecting a certain area under the lens, specifically displaying 1-2 lines of subtitles, and streaming the text sequence, instead of displaying 1-2 in the center of the screen. Line subtitles, or display all text sequences on the screen full screen. Setting 2-line subtitles is convenient for displaying both foreign language and Chinese text sequences in foreign language environment. The specific optimization method essentially belongs to interface optimization, belongs to the prior art, and is not within the scope of the present invention. A proximity sensor is a sensor that detects the distance of an object. The proximity sensor is applied to the information barrier-free terminal, which can realize some gesture control operations of the user. For example, when the user approaches the lens of the augmented reality glasses, the augmented reality glasses can automatically reduce the screen brightness. , or directly close the information display and automatically lock the screen to prevent misoperation. When the hand is left, the screen automatically adjusts the screen brightness according to the brightness of the outside environment, or restores the information display and automatically unlocks.

In order to improve the convenience of information transmission, control buttons can be set on the information barrier-free terminal carrier, which is convenient for users to control the state of the light display module. The state of the connection computing processing terminal is used to re-establish the connection when the computing processing device is replaced. The working state is used to enable the function of the light display module. The off state is used to turn off the function of the light display module, save power consumption, prolong the use time of the glasses, and thus improve the durability of the information barrier-free terminal. The instruction to control the optical display module can also be issued through the computing processing terminal.

Example 2

Based on the above information barrier-free system, this embodiment provides a speech recognition method. In this embodiment, the speech recognition and machine translation module is divided into two modules for description. As shown in Figure 2, the method includes the following steps:

S1. Power on the system, determine whether the information barrier-free terminal is normally connected with the computing processing terminal through the communication module, and establish a data interaction relationship. If the connection is normal, enter S2;

S2, the user selects the voice recognition module through the computing and processing terminal, and the computing and processing terminal collects the voice signals of the surrounding environment required by the user according to the user's requirements;

S3. The voice recognition module of the computing and processing terminal determines whether the voice signal of the user's surrounding environment is the daily car horn, siren, fire alarm and other alarm sounds, telephone ringing, door ringing, door knocking, dog barking, and gunshots. , thunder, explosion and other abnormal voice signals, if so, directly generate the corresponding prompt text sequence, and enter S5; If it is judged to be a Chinese language, it is directly recognized as a Chinese character sequence, and goes to S5; if it is not the set language voice signal, it is judged whether it is a foreign language, and if so, go to S4;

S4. The speech recognition module of the computing and processing terminal directly recognizes the collected foreign language speech signal as a foreign language word sequence, and invokes the machine translation module to translate the recognized foreign language word sequence into a Chinese word sequence;

S5. The computing processing terminal further processes the obtained text sequence and converts it into HUD information that can be displayed by the information barrier-free terminal;

S6, the computing processing terminal sends the processed and transformed speech recognition data to the information barrier-free terminal through the communication module;

S7. The information barrier-free terminal processor issues a control command to the optical display module, and outputs the received data through the optical display module for the user to browse.

This method can set the vibrating motor reminder function. When an abnormal sound is detected by the trigger, the information barrier-free terminal can provide the vibrating function through the vibrating motor, and display red prompt information on the lens mounted on the terminal to remind the user to pay attention to the sound of car horns and sirens. , fire alarm and other alarm sounds, telephone ringing, door ringing, door knocking, dog barking, gunshots, thunder, explosions and other information.

If set in advance by the user, the speech recognition module of the computing processing terminal can locally save the data such as the speech and Chinese character sequences obtained in steps S3 and S4 as notes or other files.

Example 3

Based on the above information barrier-free system, the present embodiment provides a map navigation method, as shown in FIG. 3 , the method includes the following steps:

S1. The system is powered on to determine whether the information barrier-free terminal is normally connected to the computing processing terminal through the network transmission part, and establishes a data interaction relationship. If the connection is normal, enter S2;

S2. The user selects the map navigation module through the computing and processing terminal, and specifies the starting and ending point information. The computing and processing terminal collects the user's exercise health data and the geographic location information of the user's surrounding environment according to the user's requirements, including but not limited to GPS, Beidou and other satellites. Positioning signal, WIFI signal, GPRS data, 4G or 5G and other base station positioning data;

S3. The map navigation module of the computing processing terminal calculates the current motion state of the user, including the motion mode (driving, walking, cycling, etc.), motion speed, positioning data, etc., according to the collected motion health data and geographic location information of the user;

S4. The map navigation module of the computing and processing terminal automatically decides the choice of factors such as avoiding retrograde, avoiding underground passages, and avoiding overpasses according to the state of the user, and calculates the route planning from the starting point to the ending point in the specified navigation mode;

S5. The computing and processing terminal further processes the aggregated map navigation information, and converts it into HUD information that the information barrier-free terminal prompts the user;

S6. The computing processing terminal sends the processed and transformed HUD information that prompts the user to the information barrier-free terminal through the network transmission unit;

S7. The information barrier-free terminal outputs the received data through augmented reality glasses for the user to browse.

This method can set the vibrating motor reminder function. When triggering state events such as approaching the end point, need to turn, time is up, the information barrier-free terminal can provide vibration function through the vibration motor, which is used to remind the user that the end point is reached, the turn needs to be turned, and the alarm clock is called. wake up and other information. Especially when it is detected that the user is in a dangerous area such as a road, the operating frequency and duration of the vibration motor can be adjusted to warn the user.

Example 4

Based on the above information barrier-free system, this embodiment provides a sports health management method, as shown in FIG. 4 , the method includes the following steps:

S2. The computing and processing terminal automatically collects the user's sports health data through the built-in motion sensor or according to the user's choice, and can also separately collect or synchronously collect and upload through the built-in motion sensor of the information barrier-free terminal;

S3. The computing and processing terminal exports the user's own health data through the speed measurement and calculation function, including but not limited to its own walking speed or other speeds such as driving and cycling, the number of steps, the number of kilometers walked, pulse, heart rate and other sports health data. If the user If no further request is made, enter S5;

S4. The user selects the sports health management module through the computing and processing terminal, and specifies functions such as running counting and cycling mileage counting. The user's sports health data, and export the running count, cycling mileage and other sports health data required by the user;

S5. The computing and processing terminal further processes the obtained sports health data and converts it into HUD information that can be displayed by the information barrier-free terminal;

S6. The computing and processing terminal sends the processed and transformed sports health data to the information barrier-free terminal through the network transmission unit;

This method can set the vibrating motor reminder function. When the user's exercise amount reaches the target, the user reaches the end point, the user is engaged in dangerous sports, the user's state is detected and other state events are triggered, the HUD information generated by the computing and processing terminal will be sent to the information barrier. The terminal displays and sends an instruction to the information barrier-free terminal to turn on the vibration motor to remind and warn the user.

Claims

An information barrier-free system based on augmented reality technology, characterized in that it includes a computing processing terminal and an information barrier-free terminal that communicate with each other;

The computing and processing terminal includes a sports health management module, a speech recognition and machine translation module, and a map navigation module;

The sports health management module is used to process the user's sports health data to obtain the user's sports health status;

The voice recognition and machine translation module is used to identify the voice data of the user's surrounding environment, and determine whether it is an abnormal voice signal. If it is an abnormal voice signal, it will directly generate a corresponding prompt text sequence; Attributive language voice signal, if it is a set language voice signal, it is directly converted into a set language text sequence; if it is not a set language voice signal, it is judged whether it is a foreign language, if so, the voice data is recognized as a foreign language text sequence , and translated into the set language text sequence;

The map navigation module is used for planning a navigation route according to the user's sports health data, the user's geographic location data, and the user-specified starting point and destination data;

The computing processing terminal converts the data processing results of the sports health management module, the speech recognition and machine translation module, and the map navigation module into HUD information that can be displayed by the information barrier-free terminal;

The information barrier-free terminal superimposes and displays the HUD information and the real environment where the user is located based on the augmented reality technology.
The information barrier-free system based on augmented reality technology according to claim 1, wherein the user's exercise health data is acquired through a motion sensor provided by a computing processing terminal and/or a motion sensor provided by the information barrier-free terminal .
The information barrier-free system based on augmented reality technology according to claim 1, wherein the user's geographic location data is obtained through a network positioning chip and/or a satellite navigation and positioning chip provided in the computing and processing terminal.
The information barrier-free system based on augmented reality technology according to claim 1, wherein when the speech recognition and machine translation module recognizes that the speech data is an abnormal speech signal, the computing and processing terminal generates corresponding prompt HUD information and sends it to the information barrier-free terminal, and issue an instruction to the information barrier-free terminal to turn on the vibration motor to remind and warn the user.
The information barrier-free system based on augmented reality technology according to claim 1, wherein when the sports health management module and the map navigation module detect that the user is performing a dangerous exercise or is in a dangerous area, the computing The processing terminal generates corresponding prompt HUD information and sends it to the information barrier-free terminal, and issues an instruction to the information barrier-free terminal to turn on the vibration motor to remind and warn the user.
The information barrier-free system based on augmented reality technology according to claim 1, wherein the information barrier-free terminal is augmented reality glasses, including an optical display module, a communication module, a motion sensor, an embedded processor, and a vibration motor. , the optical display module is responsible for superimposing and displaying the HUD information and the real environment where the user is located, the communication module is responsible for communicating with the computing processing terminal, the motion sensor is responsible for collecting the user's health and motion data, and the embedded The processor is responsible for data processing, and the vibration motor is used to remind and warn the user.
The information barrier-free system based on augmented reality technology according to claim 6, characterized in that the barrier-free terminal is equipped with a proximity sensor, which is used to realize gesture control operation.
A speech recognition method based on the described information barrier-free system of claim 1, is characterized in that comprising the steps:

S1. The user selects a speech recognition and machine translation module, and the computing and processing terminal collects the speech data of the user's surrounding environment through a microphone;

S2. The voice recognition and machine translation module recognizes the voice data of the user's surrounding environment, and judges whether it is an abnormal voice signal. If so, it directly generates a corresponding prompt text sequence, and enters S4; if it is not an abnormal voice signal, judge whether it is a set Attributive language, if it is, it is directly converted into the set language text sequence, and goes to S4; if the voice signal is not the set language, it is judged whether it is a foreign language, if it is, then go to S3;

S3. The speech recognition and machine translation module recognizes the speech data as a foreign language text sequence, and translates it into a set language text sequence;

S4. The computing processing terminal further processes the obtained text sequence, and converts it into HUD information that the information barrier-free terminal prompts the user;

S5, the computing processing terminal sends the HUD information to the information barrier-free terminal;

S6. The information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;

S7. When a set state trigger event occurs, the information barrier-free terminal reminds and warns the user through a vibration motor.
A map navigation method based on the described information barrier-free system of claim 1, is characterized in that comprising the steps:

S1, the user selects the map navigation module, specifies the starting point and end point information, the computing and processing terminal and/or the information barrier-free terminal collects the user's exercise health data through the motion sensor, and the computing and processing terminal collects the user's geographic location data through the positioning module;

S2. The map navigation module calculates the user's current movement state according to the user's exercise health data and the user's geographic location data, including the movement mode, movement speed, and positioning data;

S3, the map navigation module performs the navigation route planning from the starting point to the end point in the specified navigation mode according to the current motion state of the user; the navigation mode includes but is not limited to avoiding retrograde, avoiding underground passages, and avoiding overpasses;

S4. The computing and processing terminal further processes the obtained map navigation information, and converts it into HUD information that the information barrier-free terminal prompts the user;

S5, the computing processing terminal sends the HUD information to the information barrier-free terminal;

S6. The information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;

S7. When a set state trigger event occurs, the information barrier-free terminal reminds and warns the user through a vibration motor.
A kind of sports health management method based on the described information barrier-free system of claim 1, is characterized in that comprising the steps:

S1. The computing processing terminal and/or the information barrier-free terminal automatically collects the user's exercise health data through the motion sensor or according to the user's instruction;

S2. The exercise health management module calculates the current exercise state of the user according to the collected exercise health data of the user, including exercise mode, exercise speed, exercise mileage, and energy consumption;

S3. The computing and processing terminal further processes the obtained user's current motion state data, and converts it into HUD data that the information barrier-free terminal prompts the user;

S4, the computing processing terminal sends the HUD information to the information barrier-free terminal;

S5. The information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;

S6. When a set state trigger event occurs, the information barrier-free terminal reminds and warns the user through a vibration motor.