WO2022048455A1 - Système et procédé d'accès à des informations sans barrière utilisant une technologie de réalité augmentée - Google Patents

Système et procédé d'accès à des informations sans barrière utilisant une technologie de réalité augmentée Download PDF

Info

Publication number
WO2022048455A1
WO2022048455A1 PCT/CN2021/113662 CN2021113662W WO2022048455A1 WO 2022048455 A1 WO2022048455 A1 WO 2022048455A1 CN 2021113662 W CN2021113662 W CN 2021113662W WO 2022048455 A1 WO2022048455 A1 WO 2022048455A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
information
barrier
data
terminal
Prior art date
Application number
PCT/CN2021/113662
Other languages
English (en)
Chinese (zh)
Inventor
姚登峰
赵启超
阿布都克力木·阿布力孜
Original Assignee
语惠科技(南京)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 语惠科技(南京)有限公司 filed Critical 语惠科技(南京)有限公司
Publication of WO2022048455A1 publication Critical patent/WO2022048455A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/009—Teaching or communicating with deaf persons
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C23/00—Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type

Definitions

  • the invention relates to the technical field of wearable smart devices, in particular to an information barrier-free system based on augmented reality technology.
  • able-bodied people may encounter obstacles to obtaining information, such as loss of temporary hearing due to long-term wearing of headphones, gradual loss of hearing due to certain reasons, and encounters in noisy environments such as underground coal mines and aircraft carriers. hearing problems.
  • people with disabilities rely more on the Internet. People with hearing impairments are inconvenient to live, and most of them rely on Internet products for food and clothing. Therefore, the barrier-free design and optimization of application products gives everyone the right to receive and choose information.
  • the average weight of the human brain is about 5kg, but when the head and neck are bent forward at 15° and 30°, the neck will increase by 24 and 36 pounds, while at 60°, the cervical spine will bear up to 54 pounds, which is equivalent to An 8-year-old child is riding around his neck. If we keep our head down for a long time, our cervical vertebra will be under extra pressure, resulting in cervical vertebra strain. Accumulation to a certain extent will cause cervical spondylosis symptoms such as cervical vertebra deformation, nerve compression, chest tightness and dizziness. In severe cases, it will lead to paralysis. Spinal cord decompression is performed without surgical treatment, so this traditional method causes poor experience and has a greater impact on human health.
  • VR virtual reality
  • users can wear VR glasses to read the text after speech recognition, map navigation interface and other information, but this pure virtual reality (VR) technology
  • This pure virtual reality (VR) technology The real world information is shielded, and only the required virtual world information is displayed. Users cannot make autonomous behaviors in the real world, such as the need to take notes while reading information; in addition, this pure virtual reality (VR) technology equipment is relatively It is bulky and needs to be worn for a long time. If the user is in a pure virtual reality (VR) environment for a long time, it will also cause discomfort such as dizziness, which goes against the original design of the smart device and cannot achieve good human-computer interaction.
  • VR virtual reality
  • the present invention provides an information barrier-free system based on augmented reality technology, which enables information barrier-free terminals through augmented reality technology to solve the problem of poor user experience, especially for special groups, such as For hearing-impaired people, the present invention can perfectly solve the problem of barrier-free reception of their information, so that users can communicate with the surrounding people and the environment in time. It enables users to obtain and utilize information in an equal, convenient and barrier-free manner under any circumstances, so as to bridge the digital divide and help users including special groups to integrate into the information society.
  • the present invention can realize the user's immersive reception of information, so that the user can see the relevant information without lowering his head, thereby improving the user's own safety factor.
  • the present invention first provides an information barrier-free system based on augmented reality technology, which is characterized by comprising a computing processing terminal and an information barrier-free terminal that communicate with each other;
  • the computing and processing terminal includes a sports health management module, a speech recognition and machine translation module, and a map navigation module;
  • the sports health management module is used to process the user's sports health data to obtain the user's sports health status
  • the voice recognition and machine translation module is used to identify the voice data of the user's surrounding environment, and determine whether it is an abnormal voice signal. If it is an abnormal voice signal, it will directly generate a corresponding prompt text sequence; Attributive language voice signal, if it is a set language voice signal, it is directly converted into a set language text sequence; if it is not a set language voice signal, it is judged whether it is a foreign language, if so, the voice data is recognized as a foreign language text sequence , and translated into the set language text sequence;
  • the map navigation module is used for planning a navigation route according to the user's sports health data, the user's geographic location data, and the user-specified starting point and destination data;
  • the computing processing terminal converts the data processing results of the sports health management module, the speech recognition and machine translation module, and the map navigation module into HUD information that can be displayed by the information barrier-free terminal;
  • the information barrier-free terminal superimposes and displays the HUD information and the real environment where the user is located based on the augmented reality technology.
  • the user's exercise health data is acquired through a motion sensor provided by the computing processing terminal and/or a motion sensor provided by the information barrier-free terminal.
  • the user geographic location data is acquired through a network positioning chip and/or a satellite navigation and positioning chip provided in the computing and processing terminal.
  • the computing and processing terminal when the voice recognition and machine translation module recognizes that the voice data is an abnormal voice signal, the computing and processing terminal generates corresponding prompting HUD information and sends it to the information barrier-free terminal, and issues an instruction to the information barrier-free terminal to turn on the vibration motor reminder and Alert users.
  • the computing and processing terminal when the sports health management module and the map navigation module detect that the user is performing dangerous sports or is in a dangerous area, the computing and processing terminal generates corresponding prompt HUD information and sends it to the information barrier-free terminal, and sends out The instruction turns on the vibration motor to remind and alert the user to the information barrier-free terminal.
  • the information barrier-free terminal is augmented reality glasses, including an optical display module, a communication module, a motion sensor, an embedded processor, and a vibration motor, and the optical display module is responsible for linking the HUD information with the user's location.
  • the communication module is responsible for communicating with the computing processing terminal
  • the motion sensor is responsible for collecting user health and motion data
  • the embedded processor is responsible for data processing
  • the vibration motor is used to remind and warn the user.
  • the barrier-free terminal is configured with a proximity sensor for implementing gesture control operations.
  • Another object of the present invention is to provide a speech recognition method based on the above-mentioned information barrier-free system, which is characterized in that it comprises the following steps:
  • the user selects a speech recognition and machine translation module, and the computing and processing terminal collects the speech data of the user's surrounding environment through a microphone;
  • the voice recognition and machine translation module recognizes the voice data of the user's surrounding environment, and judges whether it is an abnormal voice signal. If so, it directly generates a corresponding prompt text sequence, and enters S4; if it is not an abnormal voice signal, judge whether it is an abnormal voice signal. Set the language, if it is, it will be directly converted to the text sequence of the set language, and go to S4; if the voice signal is not in the set language, judge whether it is a foreign language, and if so, go to S3;
  • the speech recognition and machine translation module recognizes the speech data as a foreign language text sequence, and translates it into a set language text sequence
  • the computing processing terminal further processes the obtained text sequence, and converts it into HUD information that the information barrier-free terminal prompts the user;
  • the computing processing terminal sends the HUD information to the information barrier-free terminal;
  • the information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;
  • the information barrier-free terminal reminds and warns the user through a vibration motor.
  • Another object of the present invention is to provide a map navigation method based on the above-mentioned information barrier-free system, which is characterized in that it comprises the following steps:
  • the user selects the map navigation module, specifies the starting point and end point information
  • the computing and processing terminal and/or the information barrier-free terminal collects the user's exercise health data through the motion sensor
  • the computing and processing terminal collects the user's geographic location data through the positioning module
  • the map navigation module calculates the user's current movement state according to the user's exercise health data and the user's geographic location data, including movement mode, movement speed, and positioning data;
  • the map navigation module performs the navigation route planning from the starting point to the end point in the specified navigation mode according to the current motion state of the user;
  • the navigation mode includes but is not limited to avoiding retrograde, avoiding underground passages, and avoiding overpasses;
  • the computing and processing terminal further processes the obtained map navigation information, and converts it into HUD information that the information barrier-free terminal prompts the user;
  • the computing processing terminal sends the HUD information to the information barrier-free terminal;
  • the information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;
  • the information barrier-free terminal reminds the user through a vibration motor.
  • Another object of the present invention is to provide a sports health management method based on the above-mentioned information barrier-free system, which is characterized by comprising the following steps:
  • the computing processing terminal and/or the information barrier-free terminal automatically collects the user's exercise health data through the motion sensor or according to the user's instruction;
  • the sports health management module calculates the current exercise state of the user according to the collected exercise health data of the user, including exercise mode, exercise speed, exercise mileage, and energy consumption;
  • the computing and processing terminal further processes the obtained user's current motion state data, and converts it into HUD data that the information barrier-free terminal prompts the user;
  • the computing processing terminal sends the HUD information to the information barrier-free terminal;
  • the information barrier-free terminal receives the HUD information, and uses the augmented reality glasses to superimpose and display it with the real environment where the user is located;
  • the information barrier-free terminal reminds the user through a vibration motor.
  • the present invention improves life for healthy people, and changes fate for special groups. For example, it is of great help to special groups in terms of information acquisition, convenient travel, and communication.
  • innovative technologies such as voice interaction, the Internet of Things, and smart hardware have provided new solutions for them to improve their lives.
  • voice recognition text information, navigation HUD information, and sports health information are displayed in the information barrier-free terminal, which replaces other terminals such as mobile phones and solves the disadvantage of poor interactivity of existing terminals.
  • the existing terminals based on augmented reality technology have voice recognition and map navigation functions, there are also some problems. For example, these terminals are usually designed for healthy people and young people, not for special groups such as hearing-impaired people.
  • These terminals are generally Equipped with headphones to transmit audio and other information through the headphones, it cannot remind the hearing-impaired people, the elderly, and other special groups in need.
  • the present invention is equipped with a vibration motor in the information barrier-free terminal, regardless of healthy people, hearing-impaired people or elderly people who are not accustomed to using earphones, can perfectly solve their information receiving problems, and it is convenient for users to communicate with the surrounding people and the environment in time. Communication helps to enable special groups to equally enjoy the right to a better life brought by technology.
  • the information barrier-free terminal can take the speaking situation of the other party, the actual road condition, and its own health as one of the factors for the information barrier-free terminal to prompt information, which not only helps the hearing-impaired people and other groups receive information barrier-free It can also help groups such as hearing people to understand the information more accurately and ensure the integrity of the information received without barriers.
  • the terminal can simultaneously display the dialogue speech recognition text, which effectively reduces the difficulty of receiving information, facilitates communication with the other party, and improves the user experience. Simultaneously displays real-time road condition pictures with various visual effects, which effectively reduces the difficulty of navigation and facilitates the improvement of user experience in many scenarios such as business trips and overseas travel.
  • the invention optimizes ergonomic design based on long-term research on hearing-impaired users.
  • the physiological characteristics and psychological characteristics of the hearing-impaired group are fully considered, so that the use process and method of the present invention conform to the usage habits of the hearing-impaired people.
  • the user experience of the present invention is better, and the human-computer interaction friendliness is achieved for all groups such as hearing-impaired people and hearing people.
  • the information barrier-free terminal adopts enhanced display technology to superimpose voice data, navigation data, sports health data and the real environment to display the real environment. Users can obtain information such as speech, real-time road conditions, navigation, health data and other information without obstacles, with a great sense of immersion and participation. .
  • the information barrier-free terminal uses the augmented vision glasses as a carrier, combined with the motion sensor chip of the augmented reality glasses, so that the information is more comprehensive, and the user obtains the content beyond the general amount of information, which helps to better understand the information content to better achieve information accessibility goals.
  • the user's status, gait and other information during use can be obtained, which can effectively analyze the user's location information and exercise health. , and achieve real-time feedback and alert users when key goals are reached. For example, when it is recognized that the hearing-impaired group is crossing the road and other dangerous occasions, it will automatically remind the hearing-impaired group to pay attention to safety, so as to achieve effective control. , When the horn reminds, the voice recognition module of the computing and processing terminal can automatically give warning information and vibrate in real time.
  • the information barrier-free terminal can provide information such as safety warnings in real time in combination with the content of the chemical experimental course, reminding the hearing-impaired people to follow
  • the security steps are completed step by step to immerse users in the course and achieve the goal of increasing user efficiency.
  • FIG. 1 is a structural block diagram of an information barrier-free system based on an augmented reality technology according to an embodiment of the present invention.
  • Fig. 2 is a flow chart of the speech recognition method based on the information barrier-free system of the present invention.
  • Fig. 3 is a flow chart of a map navigation method based on the information barrier-free system of the present invention.
  • Fig. 4 is a flow chart of the health management method based on the information barrier-free system of the present invention.
  • the information barrier-free system includes a computing processing terminal, and an information barrier-free terminal communicatively connected to the computing processing terminal.
  • the computing and processing terminal carrier is any kind of smartphone, tablet, notebook computer, desktop computer, embedded terminal, etc.
  • the hardware is equipped with wireless connection chips such as processor, microphone, Bluetooth connection chip, WLAN, etc. or supports the three major operators 5G/ 4G+/4G/3G/2G and other network standard chips, as well as navigation and positioning chips that support Beidou, GPS, Galileo, GLONASS, etc., and configure the corresponding first data processing module, AGPS assisted positioning module, data network positioning module, WIFI network positioning module , speech recognition and machine translation module, map navigation module, sports health management module, communication module, etc.
  • the information barrier-free terminal uses enhanced display glasses as a carrier, including optical display modules, wireless connection chips such as WLAN and Bluetooth, high-precision motion sensors, proximity sensors, power management chips, vibration motors, and miniature embedded processing in the carrier. hardware, and configure the corresponding second data processing module and communication module.
  • the communication module may be a WIFI communication module, a Bluetooth communication module or a USB communication module.
  • the computing and processing terminal relies on its own hardware device to collect the required information, and after being processed by the built-in module, the information is sent to the information barrier-free terminal through the communication module, so as to superimpose the information with the real world and achieve the purpose of information barrier-free communication. .
  • the information barrier-free system based on augmented reality technology includes a first data processing module and a second data processing module that communicate with each other, as well as a sports health management module, a speech recognition and machine translation module, and a map navigation module.
  • the first data processing module, sports health management module, speech recognition and machine translation module, and map navigation module are arranged in the computing processing terminal, and the second data processing module is arranged in the information barrier-free terminal.
  • the sports health management module processes the user's sports health data to obtain the user's sports health status.
  • the speech recognition and machine translation module is divided into two modules: speech recognition and machine translation. The speech recognition module recognizes the voice data of the user's surrounding environment.
  • the voice data When the voice data is in the set language, it directly converts the voice data into a text sequence of the set language. Otherwise, the voice data is Input the machine translation module to translate the text sequence into the set language.
  • the map navigation module performs navigation route planning according to the user's sports health data, the user's geographic location data, and the user-specified starting point and destination data.
  • the first data processing module is responsible for converting the data processing results of the sports health management module, the speech recognition and machine translation module, and the map navigation module into HUD information that can be displayed by the information barrier-free terminal, and the second data processing module is responsible for receiving the first data processing module.
  • the optical module of the control information barrier-free terminal superimposes and displays the HUD information and the real environment where the user is located based on the augmented reality technology.
  • the second data processing module is also responsible for sending the user's sports health data collected by the information barrier-free terminal to the sports health management module of the computing processing terminal for management.
  • the speech recognition module can be directly implemented using existing technologies. It can either conduct supporting technology research and development based on the onboard computing processing terminal processor, or directly call third-party solutions, such as the underlying technology of iFLYTEK, to transmit the collected voice information to the computer.
  • the iFLYTEK server after the iFLYTEK server completes the recognition, returns the recognized text, and then sends it back to the computing processing terminal. After the computing processing terminal is processed, it is sent to the information barrier-free terminal.
  • the machine translation module can be directly implemented using existing technologies. It can not only conduct supporting technology research and development according to the onboard computing processing terminal processor, but also directly call third-party solutions, such as Sogou, Baidu, Facebook, Google and other companies' underlying machine translation technology.
  • the computing and processing terminal transmits the collected voice information or text information to the Sogou, Baidu, Facebook or Google server. After the Sogou, Baidu, Facebook or Google server completes the translation, it returns the translated text, and then sends it back to the computing and processing terminal , the processing terminal is processed and then sent to the information barrier-free terminal.
  • the sports health management module can be directly implemented using existing technologies. It can either conduct supporting research and development based on the sports health monitoring hardware on board, or directly purchase third-party solutions.
  • the map navigation module can be directly implemented using existing technologies, such as calling the underlying technologies of any navigation such as AutoNavi, Baidu, Tencent, and Google, and using their services.
  • computing and processing terminals such as mobile phones have their own motion sensors and display screens, which can collect simple exercise health data, such as how many steps the user has taken, how much energy they consume, and how many kilometers they have walked. And perform data display, so in the present invention, the computing and processing terminal is not only a data acquisition device, a data processing device, but also a data display device.
  • the exercise health data obtained by the computing and processing terminal is not enough to judge the user's fine state.
  • the information barrier-free terminal of the present invention is also equipped with a motion sensor to detect the user's own state, such as whether to bend over, whether to lie on his back, etc.
  • a motion sensor to detect the user's own state, such as whether to bend over, whether to lie on his back, etc.
  • high-precision motion sensors such as 3-axis acceleration sensor, 3-axis gyroscope sensor, 3-axis geomagnetic sensor
  • the detection function is more refined, more comprehensive, and more powerful.
  • the information barrier-free terminal is not only a data collection device, but also a data processing device, and also a data display device.
  • the information barrier-free terminal carrier is augmented reality glasses
  • its important component is an optical display module, which is responsible for projecting information on the terminal lens or directly displaying information on the terminal lens, so that it is superimposed in the real world, thereby being viewed by Perceived by the user's senses, to obtain a sensory experience beyond reality.
  • the optimized text sequence or navigation interface overlaps with the real environment, and can coexist on the same lens. Therefore, when using, you only need to wear the augmented reality glasses, and then point your sight to the scene to be detected, so as to browse the real environment and the important information displayed at the same time, which is very convenient to use, and you do not need to look down at the calculation during the whole process Handling equipment frees hands and head.
  • the computing processing terminal relies on the built-in processor to optimize the information processed by the sports health management module, speech recognition and machine translation module, and map navigation module, and convert it into HUD information to adapt to augmented reality technology.
  • the Chinese name of HUD is head-up display, also known as head-up display system.
  • the information barrier-free terminal projects information such as speed, navigation, and text onto the photoelectric display module on the carrier, onto the lens or in front of the lens, or directly on the lens. form an image. In this way, users can see navigation and speed information without turning their heads or bowing their heads. Take the text sequence after speech recognition or machine translation as an example. When the text sequence is displayed on the computing and processing terminal, it is generally displayed in full screen or in a designated display window.
  • a proximity sensor is a sensor that detects the distance of an object.
  • the proximity sensor is applied to the information barrier-free terminal, which can realize some gesture control operations of the user. For example, when the user approaches the lens of the augmented reality glasses, the augmented reality glasses can automatically reduce the screen brightness. , or directly close the information display and automatically lock the screen to prevent misoperation. When the hand is left, the screen automatically adjusts the screen brightness according to the brightness of the outside environment, or restores the information display and automatically unlocks.
  • control buttons can be set on the information barrier-free terminal carrier, which is convenient for users to control the state of the light display module.
  • the state of the connection computing processing terminal is used to re-establish the connection when the computing processing device is replaced.
  • the working state is used to enable the function of the light display module.
  • the off state is used to turn off the function of the light display module, save power consumption, prolong the use time of the glasses, and thus improve the durability of the information barrier-free terminal.
  • the instruction to control the optical display module can also be issued through the computing processing terminal.
  • this embodiment provides a speech recognition method.
  • the speech recognition and machine translation module is divided into two modules for description. As shown in Figure 2, the method includes the following steps:
  • the user selects the voice recognition module through the computing and processing terminal, and the computing and processing terminal collects the voice signals of the surrounding environment required by the user according to the user's requirements;
  • the voice recognition module of the computing and processing terminal determines whether the voice signal of the user's surrounding environment is the daily car horn, siren, fire alarm and other alarm sounds, telephone ringing, door ringing, door knocking, dog barking, and gunshots. , thunder, explosion and other abnormal voice signals, if so, directly generate the corresponding prompt text sequence, and enter S5; If it is judged to be a Chinese language, it is directly recognized as a Chinese character sequence, and goes to S5; if it is not the set language voice signal, it is judged whether it is a foreign language, and if so, go to S4;
  • the speech recognition module of the computing and processing terminal directly recognizes the collected foreign language speech signal as a foreign language word sequence, and invokes the machine translation module to translate the recognized foreign language word sequence into a Chinese word sequence;
  • the computing processing terminal further processes the obtained text sequence and converts it into HUD information that can be displayed by the information barrier-free terminal;
  • the computing processing terminal sends the processed and transformed speech recognition data to the information barrier-free terminal through the communication module;
  • the information barrier-free terminal processor issues a control command to the optical display module, and outputs the received data through the optical display module for the user to browse.
  • This method can set the vibrating motor reminder function.
  • the information barrier-free terminal can provide the vibrating function through the vibrating motor, and display red prompt information on the lens mounted on the terminal to remind the user to pay attention to the sound of car horns and sirens. , fire alarm and other alarm sounds, telephone ringing, door ringing, door knocking, dog barking, gunshots, thunder, explosions and other information.
  • the speech recognition module of the computing processing terminal can locally save the data such as the speech and Chinese character sequences obtained in steps S3 and S4 as notes or other files.
  • the present embodiment provides a map navigation method, as shown in FIG. 3 , the method includes the following steps:
  • the system is powered on to determine whether the information barrier-free terminal is normally connected to the computing processing terminal through the network transmission part, and establishes a data interaction relationship. If the connection is normal, enter S2;
  • the user selects the map navigation module through the computing and processing terminal, and specifies the starting and ending point information.
  • the computing and processing terminal collects the user's exercise health data and the geographic location information of the user's surrounding environment according to the user's requirements, including but not limited to GPS, Beidou and other satellites. Positioning signal, WIFI signal, GPRS data, 4G or 5G and other base station positioning data;
  • the map navigation module of the computing processing terminal calculates the current motion state of the user, including the motion mode (driving, walking, cycling, etc.), motion speed, positioning data, etc., according to the collected motion health data and geographic location information of the user;
  • the map navigation module of the computing and processing terminal automatically decides the choice of factors such as avoiding retrograde, avoiding underground passages, and avoiding overpasses according to the state of the user, and calculates the route planning from the starting point to the ending point in the specified navigation mode;
  • the computing and processing terminal further processes the aggregated map navigation information, and converts it into HUD information that the information barrier-free terminal prompts the user;
  • the computing processing terminal sends the processed and transformed HUD information that prompts the user to the information barrier-free terminal through the network transmission unit;
  • the information barrier-free terminal outputs the received data through augmented reality glasses for the user to browse.
  • This method can set the vibrating motor reminder function.
  • the information barrier-free terminal can provide vibration function through the vibration motor, which is used to remind the user that the end point is reached, the turn needs to be turned, and the alarm clock is called. wake up and other information.
  • the operating frequency and duration of the vibration motor can be adjusted to warn the user.
  • this embodiment provides a sports health management method, as shown in FIG. 4 , the method includes the following steps:
  • the system is powered on to determine whether the information barrier-free terminal is normally connected to the computing processing terminal through the network transmission part, and establishes a data interaction relationship. If the connection is normal, enter S2;
  • the computing and processing terminal automatically collects the user's sports health data through the built-in motion sensor or according to the user's choice, and can also separately collect or synchronously collect and upload through the built-in motion sensor of the information barrier-free terminal;
  • the computing and processing terminal exports the user's own health data through the speed measurement and calculation function, including but not limited to its own walking speed or other speeds such as driving and cycling, the number of steps, the number of kilometers walked, pulse, heart rate and other sports health data. If the user If no further request is made, enter S5;
  • the user selects the sports health management module through the computing and processing terminal, and specifies functions such as running counting and cycling mileage counting.
  • the user's sports health data and export the running count, cycling mileage and other sports health data required by the user;
  • the computing and processing terminal further processes the obtained sports health data and converts it into HUD information that can be displayed by the information barrier-free terminal;
  • the computing and processing terminal sends the processed and transformed sports health data to the information barrier-free terminal through the network transmission unit;
  • the information barrier-free terminal outputs the received data through augmented reality glasses for the user to browse.
  • This method can set the vibrating motor reminder function.
  • the user's exercise amount reaches the target, the user reaches the end point, the user is engaged in dangerous sports, the user's state is detected and other state events are triggered, the HUD information generated by the computing and processing terminal will be sent to the information barrier.
  • the terminal displays and sends an instruction to the information barrier-free terminal to turn on the vibration motor to remind and warn the user.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Navigation (AREA)

Abstract

L'invention concerne un système et un procédé d'accès à des informations sans barrière utilisant une technologie de réalité augmentée. Le système comprend un terminal de traitement informatique et un terminal d'accès à des informations sans barrière dans une connexion de communication avec le terminal de traitement informatique. Le terminal de traitement informatique acquiert des données demandées d'un utilisateur, effectue un traitement de celles-ci pour obtenir des informations de visualisation tête haute (HUD), et les envoie au terminal d'accès à des informations sans barrière. Le terminal d'accès à des informations sans barrière utilise une technologie de réalité augmentée afin de superposer les informations HUD sur l'environnement réel où l'utilisateur est situé et les afficher. Une technologie de réalité augmentée est utilisée afin d'améliorer la capacité d'un terminal d'accès à des informations sans barrière, ce qui permet de résoudre le problème d'une mauvaise expérience utilisateur. L'invention permet à un utilisateur de recevoir des informations d'une manière immersive, de telle sorte que l'utilisateur peut visualiser des informations pertinentes sans abaisser la tête, ce qui permet d'augmenter la sécurité de l'utilisateur.
PCT/CN2021/113662 2020-09-04 2021-08-20 Système et procédé d'accès à des informations sans barrière utilisant une technologie de réalité augmentée WO2022048455A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010918613.9A CN112002186B (zh) 2020-09-04 2020-09-04 一种基于增强现实技术的信息无障碍系统及方法
CN202010918613.9 2020-09-04

Publications (1)

Publication Number Publication Date
WO2022048455A1 true WO2022048455A1 (fr) 2022-03-10

Family

ID=73468282

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/113662 WO2022048455A1 (fr) 2020-09-04 2021-08-20 Système et procédé d'accès à des informations sans barrière utilisant une technologie de réalité augmentée

Country Status (2)

Country Link
CN (1) CN112002186B (fr)
WO (1) WO2022048455A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116208918A (zh) * 2023-02-16 2023-06-02 涂宇恒 一种野外作业信息终端及野外作业指挥系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112002186B (zh) * 2020-09-04 2022-05-06 语惠科技(南京)有限公司 一种基于增强现实技术的信息无障碍系统及方法

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101483806A (zh) * 2009-02-24 2009-07-15 南京师范大学 面向视障人群的户外导盲服务系统及方法
CN102973395A (zh) * 2012-11-30 2013-03-20 中国舰船研究设计中心 一种多功能智能导盲方法、处理器及其装置
CN103919663A (zh) * 2014-03-31 2014-07-16 浙江大学 盲人户外环境感知方法
CN104983511A (zh) * 2015-05-18 2015-10-21 上海交通大学 针对全盲视觉障碍者的语音帮助智能眼镜系统
CN105444775A (zh) * 2015-12-31 2016-03-30 歌尔科技有限公司 一种增强现实的导航系统、头戴设备和导航方法
CN105975615A (zh) * 2016-05-19 2016-09-28 哈尔滨亿时代数码科技开发有限公司 一种网站信息无障碍化处理方法
CN108099790A (zh) * 2017-12-05 2018-06-01 华南理工大学 基于增强现实平视显示与多屏语音交互的驾驶辅助系统
CN108168540A (zh) * 2017-12-22 2018-06-15 福建中金在线信息科技有限公司 一种智能眼镜导航方法、装置及智能眼镜
CN109963144A (zh) * 2019-03-28 2019-07-02 重庆长安汽车股份有限公司 一种基于ar-hud的车载识别系统
US20190252080A1 (en) * 2017-12-30 2019-08-15 Ifgcure Holdings, Llc Virtual reality and augmented reality group therapy for treating mental health, substance abuse, and developmental disorders
CN110544368A (zh) * 2018-05-31 2019-12-06 深圳疆程技术有限公司 一种疲劳驾驶增强现实预警装置及预警方法
EP3189655B1 (fr) * 2014-09-03 2020-02-05 Aira Tech Corporation Procédé et système mis en oeuvre par un ordinateur pour fournir une assistance à distance pour des utilisateurs ayant une maladie visuelle
WO2020032469A1 (fr) * 2018-08-08 2020-02-13 김보언 Procédé, dispositif et programme de fourniture d'événement de réalité virtuelle
CN110795608A (zh) * 2018-08-02 2020-02-14 声音猎手公司 可视地呈现与自然语言对话相关的信息
CN112002186A (zh) * 2020-09-04 2020-11-27 语惠科技(南京)有限公司 一种基于增强现实技术的信息无障碍系统及方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5459179B2 (ja) * 2010-05-14 2014-04-02 カシオ計算機株式会社 移動状態判別装置、移動状態判別方法およびプログラム
CN104111466A (zh) * 2013-04-18 2014-10-22 三星电子（中国）研发中心 一种位置提醒方法及装置
EP3191795A1 (fr) * 2014-09-08 2017-07-19 InvenSense, Inc. Procédé et appareil permettant d'utiliser une navigation portative améliorée assistée par des informations cartographiques
CN104952447B (zh) * 2015-04-30 2020-03-27 深圳市全球锁安防系统工程有限公司 一种老龄人安康服务智能穿戴设备及语音识别方法
CN106441315A (zh) * 2016-09-23 2017-02-22 百度在线网络技术（北京）有限公司 信息提供方法和装置
CN106534523A (zh) * 2016-10-28 2017-03-22 维沃移动通信有限公司 一种移动终端危险预警的方法及移动终端
DE102016015410A1 (de) * 2016-12-22 2018-06-28 Preh Car Connect Gmbh Verfahren und Navigationseinrichtung zum Berechnen einer Ausweichroute
CN106897562B (zh) * 2017-02-28 2022-04-29 镇江市高等专科学校 一种旅游随身健康终端的工作方法
CN208850075U (zh) * 2018-10-10 2019-05-10 格物感知（深圳）科技有限公司 一种穿戴式定位导航管理系统
CN109598003B (zh) * 2018-12-17 2022-12-16 深圳市传智科技有限公司 一种ar翻译装置及方法
CN109919127B (zh) * 2019-03-20 2021-12-17 胡海明 一种哑语语言转换系统

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101483806A (zh) * 2009-02-24 2009-07-15 南京师范大学 面向视障人群的户外导盲服务系统及方法
CN102973395A (zh) * 2012-11-30 2013-03-20 中国舰船研究设计中心 一种多功能智能导盲方法、处理器及其装置
CN103919663A (zh) * 2014-03-31 2014-07-16 浙江大学 盲人户外环境感知方法
EP3189655B1 (fr) * 2014-09-03 2020-02-05 Aira Tech Corporation Procédé et système mis en oeuvre par un ordinateur pour fournir une assistance à distance pour des utilisateurs ayant une maladie visuelle
CN104983511A (zh) * 2015-05-18 2015-10-21 上海交通大学 针对全盲视觉障碍者的语音帮助智能眼镜系统
CN105444775A (zh) * 2015-12-31 2016-03-30 歌尔科技有限公司 一种增强现实的导航系统、头戴设备和导航方法
CN105975615A (zh) * 2016-05-19 2016-09-28 哈尔滨亿时代数码科技开发有限公司 一种网站信息无障碍化处理方法
CN108099790A (zh) * 2017-12-05 2018-06-01 华南理工大学 基于增强现实平视显示与多屏语音交互的驾驶辅助系统
CN108168540A (zh) * 2017-12-22 2018-06-15 福建中金在线信息科技有限公司 一种智能眼镜导航方法、装置及智能眼镜
US20190252080A1 (en) * 2017-12-30 2019-08-15 Ifgcure Holdings, Llc Virtual reality and augmented reality group therapy for treating mental health, substance abuse, and developmental disorders
CN110544368A (zh) * 2018-05-31 2019-12-06 深圳疆程技术有限公司 一种疲劳驾驶增强现实预警装置及预警方法
CN110795608A (zh) * 2018-08-02 2020-02-14 声音猎手公司 可视地呈现与自然语言对话相关的信息
WO2020032469A1 (fr) * 2018-08-08 2020-02-13 김보언 Procédé, dispositif et programme de fourniture d'événement de réalité virtuelle
CN109963144A (zh) * 2019-03-28 2019-07-02 重庆长安汽车股份有限公司 一种基于ar-hud的车载识别系统
CN112002186A (zh) * 2020-09-04 2020-11-27 语惠科技(南京)有限公司 一种基于增强现实技术的信息无障碍系统及方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116208918A (zh) * 2023-02-16 2023-06-02 涂宇恒 一种野外作业信息终端及野外作业指挥系统
CN116208918B (zh) * 2023-02-16 2023-12-26 涂宇恒 一种野外作业信息终端及野外作业指挥系统

Also Published As

Publication number Publication date
CN112002186B (zh) 2022-05-06
CN112002186A (zh) 2020-11-27

Similar Documents

Publication Publication Date Title
WO2022048455A1 (fr) Système et procédé d'accès à des informations sans barrière utilisant une technologie de réalité augmentée
CN105894733B (zh) 驾驶员监控系统
KR102450803B1 (ko) 양방향 수화 번역 장치 및 장치가 수행하는 양방향 수화 번역 방법
US9101459B2 (en) Apparatus and method for hierarchical object identification using a camera on glasses
CN104983511A (zh) 针对全盲视觉障碍者的语音帮助智能眼镜系统
US20150379896A1 (en) Intelligent eyewear and control method thereof
CN109087485B (zh) 驾驶提醒方法、装置、智能眼镜及存储介质
EP3416034A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
CN104473717A (zh) 针对全盲人群的可穿戴式引导设备
US20020128541A1 (en) Visual displaying device for virtual reality with a built-in biofeedback sensor
WO2017092396A1 (fr) Système et procédé d'interaction de réalité virtuelle
KR20090036183A (ko) 카메라센서로 찍은 문서 이미지를 시각장애인에게 음성으로알려주는 방법 및 장치
CN108052682A (zh) 一种智能查询机系统
CN111508609A (zh) 健康状况的风险预测方法、装置、计算机设备及存储介质
WO2022037398A1 (fr) Procédé, dispositif et système de commande audio
CN111479148B (zh) 可穿戴设备、眼镜终端、处理终端、数据交互方法与介质
CN109938974A (zh) 一种视障人士用便携式智能导盲设备及导盲方法
KR101659033B1 (ko) 이동 단말기 및 이의 제어 방법
CN112561113A (zh) 危险场景预警方法及终端设备
CN105824137A (zh) 可视化智能眼镜
CN215529127U (zh) 一种具备视障人辅助功能的智能穿戴设备
KR100934390B1 (ko) 상황인지 정보 기반의 멀티미디어 장치 및 그 서비스 방법
CN210606226U (zh) 一种双模式聋哑人交流设备
CN210690949U (zh) 一种vr设备
US20190073183A1 (en) Information processing apparatus, information processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21863522

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21863522

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21863522

Country of ref document: EP

Kind code of ref document: A1