WO2014014145A1 - Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande - Google Patents

Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande Download PDF

Info

Publication number
WO2014014145A1
WO2014014145A1 PCT/KR2012/005768 KR2012005768W WO2014014145A1 WO 2014014145 A1 WO2014014145 A1 WO 2014014145A1 KR 2012005768 W KR2012005768 W KR 2012005768W WO 2014014145 A1 WO2014014145 A1 WO 2014014145A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
user
hmd
direction change
display device
Prior art date
Application number
PCT/KR2012/005768
Other languages
English (en)
Korean (ko)
Inventor
박효림
Original Assignee
엘지전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 엘지전자 주식회사 filed Critical 엘지전자 주식회사
Priority to PCT/KR2012/005768 priority Critical patent/WO2014014145A1/fr
Publication of WO2014014145A1 publication Critical patent/WO2014014145A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3626—Details of the output of route guidance instructions
    • G01C21/365—Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment

Definitions

  • the present invention relates to a head mounted display device using a brain wave signal and a control method thereof, and more particularly, to obtain direction change information intended by a user using an brain wave signal measured by the head mounted display device,
  • the present invention relates to a head mounted display device and a method of controlling the same, which provide a different method of providing navigation information.
  • Head mounted displays (hereinafter, referred to as 'HMD') refer to various image display devices that are worn on the head like glasses to view an image. According to the trend of light weight and miniaturization of digital devices, various wearable computers are being developed, and HMDs are also widely used.
  • the HMD can be combined with augmented reality technology and N screen technology to provide users with a variety of conveniences beyond simple display functions.
  • the HMD may perform a communication function such as a telephone connection, a text transmission, a mail transmission and reception, a navigation function such as a directions, and the like.
  • brainwave signal is the flow of electricity generated when a signal is transmitted between the nerves in the nervous system.
  • EEG signals vary depending on the state of mind and body and are used as important indicators of brain activity. Medical, meditation, education, brain development, etc. are also increasingly diversified in the fields where EEG signals are used.
  • An embodiment of the present invention is to provide a HMD and a control method for acquiring direction change information intended by a user using an EEG signal and controlling the output navigation information based on the direction change information.
  • another embodiment of the present invention is to provide a HMD and a control method for providing navigation information only when the user needs.
  • another embodiment of the present invention is to provide an HMD and a control method capable of confirming an accuracy of a direction in which a user wants to go by using an EEG signal without a separate manipulation process.
  • a control method of a head mounted display apparatus using an EEG signal includes the steps of: receiving GPS (Global Positioning System) information, outputting navigation information using the received GPS information; Acquiring direction change information intended by a user, and the direction change information is information obtained by using an EEG signal of the user measured by the head mounted display device, included in the direction change information and the navigation information. Determining whether the route guidance information is identical, and outputting a notification when the determination result is not identical.
  • GPS Global Positioning System
  • a head mounted display (HMD) using an EEG signal includes a communication unit for receiving GPS (Global Positioning System) information and a graphic image of navigation information using the received GPS information. And a controller configured to control the HMD operation by acquiring the direction change information intended by the user by using the EEG sensor and the EEG signal sensing the EEG signal of the user.
  • the display apparatus may determine whether the direction change information and the path guide information included in the navigation information match, and, if the determination result does not match, output the notice.
  • the HMD can acquire direction change information intended by the user by using an EEG signal, thereby performing a function corresponding to the user's intention.
  • the HMD can control the navigation information output from the HMD based on the direction change information intended by the user, thereby increasing the usability of the HMD.
  • the HMD can increase user convenience by providing navigation information only when the user needs it.
  • the HMD enables the user to check the accuracy of the direction in which the user wants to go by using an EEG signal without a separate manipulation process. Accordingly, the user can learn and confirm the path so that the user can easily make a decision about the change of direction.
  • FIG. 1 is a diagram illustrating a user wearing HMD and HMD according to an embodiment of the present invention.
  • FIG. 2 is a functional block diagram of an HMD according to an embodiment of the present invention.
  • FIG. 3 is a diagram illustrating embodiments of dividing an EEG signal of a user measured by a HMD according to a wavelength according to the present invention.
  • FIG. 4 is a diagram illustrating embodiments of dividing an EEG signal of a user measured by an HMD according to a user's state according to the present invention.
  • FIG. 5 is a diagram illustrating an embodiment of displaying navigation information in an HMD according to the present invention.
  • FIG. 6 is a diagram illustrating another embodiment of displaying navigation information in an HMD according to the present invention.
  • FIG. 7 is a diagram illustrating an embodiment of navigation information displayed when the route guidance information of the navigation and the direction change information of the user do not match in the HMD according to the present invention.
  • FIG. 8 is a diagram illustrating an embodiment of navigation information displayed when the route guidance information of the navigation and the user's redirection information match in the HMD according to the present invention.
  • FIG. 9 is a view showing another embodiment of the navigation information displayed when the route guidance information of the navigation and the user's direction switching information in the HMD according to the present invention.
  • FIG. 10 is a view showing another embodiment of the navigation information displayed when the route guidance information of the navigation and the user's direction switching information in the HMD according to the present invention.
  • FIG. 11 is a diagram illustrating an embodiment of a range of comparing route guidance information of a navigation and direction change information of a user in an HMD according to the present invention.
  • FIG. 12 is a diagram illustrating an embodiment of checking whether a direction intended by a user matches navigation information in an HMD according to the present invention.
  • FIG. 13 is a diagram illustrating an embodiment of a usage menu provided to a user by an HMD according to the present invention.
  • FIG. 14 is a view showing an embodiment in which the HMD according to the present invention is used in conjunction with the navigation of the vehicle.
  • FIG. 15 is a view showing another embodiment in which the HMD according to the present invention is used in conjunction with the navigation of a vehicle.
  • 16 is a flowchart of a control method of an HMD according to an embodiment of the present invention.
  • FIG. 1 is a diagram illustrating a user wearing HMD and HMD according to an embodiment of the present invention.
  • the user is one of the wearable computers that can view an image by wearing the HMD 100 on the head like glasses.
  • the HMD 100 includes an EEG sensor in order to sense the EEG signal of the user, and accordingly needs a portion 110 in contact with the user's head. This will be described in detail with reference to FIG. 2.
  • FIG. 2 is a functional block diagram of an HMD according to an embodiment of the present invention.
  • FIG. 2 is an embodiment for describing the present invention, and some component modules may be deleted or new component modules may be added according to the needs of those skilled in the art. .
  • the HMD 200 may include a communication unit 210, a display unit 220, an audio output unit 230, an EEG sensor 240, and a controller ( 250).
  • the communication unit 210 may include various types of wired or wireless networks that enable pairing or communication with external devices.
  • the wireless network that can be used includes NFC (Near Field Communication), Zigbee, infrared communication, Bluetooth (Bluetooth), Wi-Fi, etc.
  • NFC Near Field Communication
  • Zigbee Zigbee
  • infrared communication Bluetooth (Bluetooth), Wi-Fi, etc.
  • Bluetooth Bluetooth
  • Wi-Fi Wi-Fi
  • the communication unit 210 may include a GPS receiver for receiving Global Positioning System (GPS) information.
  • GPS Global Positioning System
  • the HMD 200 receives signals from three or more GPS satellites to determine the location of the satellite and the HMD 200.
  • the HMD 200 may obtain a distance between the satellite and the HMD 200 by measuring the time difference between the signal transmitted from the satellite and the signal received from the communication unit 210.
  • the transmitted signal contains information about the position of the satellite.
  • the HMD 200 receives distances from at least three satellites and location information of each satellite, and calculates the position of the HMD 200 using a method such as triangulation.
  • the display unit 220 may display a user interface of the HMD 200 itself, a response to a user input, or the like, and may display various contents received by the communication unit 210. Accordingly, the display unit 220 may output a graphic image of the navigation information using the GPS information received from the communication unit 210.
  • the audio output unit 230 includes audio output means such as speakers and earphones.
  • the audio output unit 230 may output a voice for route guidance based on route guidance information included in the navigation information.
  • the EEG sensor 240 senses a user's EEG signal, and may include an electrode unit, a signal conversion unit, an amplification unit, and a converter. Since the EEG signal is a bioelectrical signal due to the activity of brain neurons (neurons), it can be measured by the potential difference between two different points on the surface of the head.
  • the electrode unit is attached to the surface of the user's head and measures the electrical signal generated by the activity of the brain.
  • the electrode unit may include at least four electrodes including at least two measuring electrodes Active and a reference electrode REF and a ground electrode GND attached to the head.
  • the input impedance is increased through the amplification unit for accurate signal magnitude detection. That is, the reference potential of the sensor and the reference potential of the human body may be matched through the amplification unit.
  • the amplified electrical signal is converted into an EEG signal through a signal conversion unit.
  • the measured EEG signal may show a wave form that vibrates in a very complex pattern, and may use power spectrum analysis to classify according to frequency. Power spectral analysis assumes that EEG is a linear combination of simple oscillations that oscillate at a particular frequency, and breaks down each frequency component in this signal and displays its magnitude.
  • FIG. 3 is a diagram illustrating embodiments of dividing an EEG signal of a user measured by a HMD according to a wavelength according to the present invention. Power spectral analysis shows the magnitude of each frequency component in the EEG signal.
  • the EEG signal is artificially delta wave ( ⁇ wave: 0.2 ⁇ 3.99 Hz), theta wave ( ⁇ wave: 4 ⁇ 7.99 Hz), alpha wave ( ⁇ wave: 8) according to the range of the oscillating frequency ⁇ 12.99 Hz), beta wave ( ⁇ wave: 13 ⁇ 29.99 Hz), gamma wave ( ⁇ wave: 30 ⁇ 50 Hz) can be divided into.
  • Delta waves have a frequency of 0.2 to 4 Hz and an amplitude of 20 to 200 V, and appear mainly in deep sleep or in newborns. And in the delta wave state it is known to produce a large amount of growth hormone.
  • Theta waves have frequencies between 4 and 8 Hz and amplitudes between 20 and 100 V and appear before they are emotionally stable or sleep. So theta waves are often called the boundary between perception and dreams.
  • Alpha waves have frequencies from 8 to 13 Hz and amplitudes from 20 to 60 V. They appear in a relaxed state, such as meditation, to help relieve stress and improve concentration. When we close our eyes and relax, our brains produce alpha waves, and our brains become alpha.
  • Beta waves have a frequency of 13 to 30 Hz and an amplitude of 2 to 20 V. They appear when you are active, such as in tension or excitement. It helps to improve athleticism and is an EEG signal when consciousness is awake.
  • gamma waves have a frequency of 30-50 Hz and an amplitude of 2-20 V, and appear mainly when excited.
  • FIG. 4 is a diagram illustrating embodiments of dividing an EEG signal of a user measured by an HMD according to a user's state according to the present invention.
  • the EEG signal may vary according to the user's state, and the power spectral distribution shows a slightly different pattern for each measurement site of the head surface.
  • the occipital lobe has a primary visual cortex, which is responsible for the primary visual information processing
  • the parietal lobe has a somatosensory cortex, which is responsible for processing motion and sensory related information. Therefore, since the HMD according to the present invention outputs navigation information reflecting a direction intended by the user, the EEG sensor may be located at the head of the user's head.
  • the converter converts the amplified analog EEG signal into a digital EEG signal. Accordingly, the controller 250 acquires the direction change information intended by the user using the sensed EEG signal to control the HMD operation. That is, the controller 250 determines whether the direction change information and the route guidance information included in the navigation information match, and controls the output information to be different according to the determination result.
  • the EMD signal generated when the user recognizes the direction should be patterned to construct a database. This will be described again with reference to FIG. 7.
  • FIG. 5 is a diagram illustrating an embodiment of displaying navigation information in an HMD according to the present invention, in which a starting point 511 and a destination 513 are displayed on a real space viewed by the user while the user wears the HMD.
  • the navigation map 510 is displayed.
  • the navigation information may include at least one of current location information, electronic map information, and route guidance information, and a graphic image corresponding to the current location information, electronic map information, and route guidance information. And sound.
  • the HMD can use the GPS information received from the communication unit as the current location information, and can obtain the current location information through the gyro sensor included in the HMD 200 or the BECON existing on the road.
  • Electronic map information is a facility that stores road maps that include road shapes and traffic restrictions, background maps that are graphic images in vector format for display around roads, and latitude and longitude coordinates of facilities for finding and searching for specific locations by name or address. It can include a DB.
  • Route guidance information is information for setting a point to go through the facility DB of the map, and find the road closest to the set point to guide the route.
  • the route guidance information includes rotation guidance information necessary at the current position to move to the found route.
  • the route guide information includes direction related information for guiding a change of direction such as a left turn right turn in the course of moving a route.
  • the HMD according to the present invention may store navigation information in a separate storage unit. Alternatively, necessary information and updated information may be received from an external device as needed.
  • FIG. 6 is a diagram illustrating another embodiment of displaying navigation information in an HMD according to the present invention.
  • the HMD displays the route guidance information 610 small on the upper right side as shown in FIG. 6.
  • the HMD is a device worn and used by a user, and is particularly related to the eyes of the user. Therefore, when the user obscures the view while wearing the HMD, it may interfere with the user's activity. Therefore, the HMD according to the present invention is to provide an HMD that can minimize the disturbance of the user's view.
  • the HMD can minimize the interference of the user's view by displaying only the information required by the user and not displaying the information that the user knows.
  • FIG. 7 is a diagram illustrating an embodiment of navigation information displayed when the route guidance information of the navigation and the direction change information of the user do not match in the HMD according to the present invention.
  • the HMD displays route guide information 710 according to a route set by a user.
  • the route guide information includes direction related information for guiding rotation or direction change.
  • the route guide information is to turn left after walking the crosswalk.
  • the HMD determines that there is a change of direction from the path guide information included in the navigation information
  • the HMD acquires the direction change information intended by the user and determines whether the direction change information and the path guide information match.
  • the redirection information is information obtained by using an EEG signal of the user measured by the HMD.
  • the EEG signal of the user is measured, and the pattern of the measured EEG signal is compared with the patterns stored in the database to find the EEG signal having a similar pattern. Since the database stores the EEG signal pattern generated when the user thinks of the direction, the HMD can obtain direction change information about the direction intended by the user when the EMD finds an EEG signal having a similar pattern.
  • the HMD can connect to an already built database via a network and obtain necessary information without having to directly build a database.
  • a database of EEG signals of a user wearing the HMD in the HMD is constructed is described as an embodiment.
  • the HMD first measures an EEG signal generated when the user recognizes a direction in order to construct a database by storing the change direction information according to the pattern of the EEG signal according to the user.
  • the user may extract the pattern of the EEG signal generated when thinking of the specific direction and store the pattern of the EEG signal corresponding to the direction.
  • the HMD may use an induced potential analysis method, which is one of classical analysis methods for analyzing an EEG signal.
  • An induction potential is a waveform obtained by repeatedly presenting a stimulus containing specific information and then obtaining only electrical activity of the brain related to this stimulus treatment. Therefore, the HMD suggests a direction stimulus so that a user can repeatedly think about a specific direction such as right and left.
  • the HMD stores and averages the EEG signals measured based on a time point at which the directional stimulus is presented, thereby eliminating the electrical activity portion of the brain that is not related to the stimulus, and can measure only the EEG signals that are commonly involved in the stimulus process. In this case, when the EEG signal is measured together when not thinking about the direction, the EEG signal generated when the direction is recognized can be measured more efficiently.
  • the HMD may construct a database by patterning the EEG signal for each direction and storing the change of direction information.
  • the HMD may have a BCI (Brain-Comuter Interface), and whenever a user wearing the HMD is changed, the HMD may set a database of EEG signals for the changed direction of the user through the BCI.
  • BCI Brain-Comuter Interface
  • the HMD may be patterned to obtain direction switching information intended by the user through an EEG signal measured by the user using a database in which EEG signals are stored for each direction.
  • the HMD determines whether the obtained direction information and the route guidance information included in the navigation information match.
  • the HMD when the direction 720 according to the direction change information intended by the user is different from the direction 710 presented in the navigation information, the HMD outputs a notice.
  • the HMD can increase the size of the graphic image 730 of the route guidance information so that the user can properly recognize the direction.
  • the HMD may output knots using at least one of audio, graphic image, lighting, and vibration.
  • the HMD may change the attribute information outputting the audio or graphic image corresponding to the navigation information to output the light, to emit light from a means capable of outputting a light such as an LED attached to the HMD, or the vibration state of the mobile phone.
  • the method of outputting knots in the HMD is not limited to a specific method.
  • FIG. 8 is a diagram illustrating an embodiment of navigation information displayed when the route guidance information of the navigation and the user's redirection information match in the HMD according to the present invention.
  • the HMD has the same direction 810 as the path guide information included in the navigation information and a direction 820 according to the direction change information acquired using the EEG signal of the user. If so, the navigation information can be set to the hiding state as shown in FIG.
  • the hiding state is a state in which the navigation information is not output to the user, and as shown in FIG. 8B, the hiding state may not display a graphic image or output audio.
  • the HMD maintains the navigation information in a heighted state and outputs the user information when the user needs the navigation information.
  • the HMD determines whether or not the direction change information and the route guidance information match each other in a predetermined time unit, and if so, maintains the hiring state. Information can be output.
  • the HMD may release the hiding state and output the navigation information when there is no direction change information acquired in a predetermined time unit.
  • the reason why the navigation information is set to the hiding state in the HMD is to provide unnecessary information because the user knows the path.
  • FIG. 9 is a view showing another embodiment of the navigation information displayed when the route guidance information of the navigation and the user's direction switching information in the HMD according to the present invention.
  • the HMD has the same direction 910 as presented by the route guidance information included in the navigation information and a direction 920 according to the direction change information acquired using the EEG signal of the user. If so, the navigation information may be output by changing the visual or audio attribute of the graphic image corresponding to the route guidance information.
  • the visual attributes of the graphic image corresponding to the route guidance information may include all of the elements necessary for display, including transparency, size, color, saturation, and the like.
  • the audio attribute corresponding to the path guide information may include all of the elements necessary for outputting audio, including the size of the audio to be output, whether to mute. Accordingly, the route guidance information 930 may be displayed by increasing the transparency of the graphic image displayed as shown in FIG. 9B.
  • One embodiment according to FIG. 9 is particularly useful when the user is large or dependent on navigation information. That is, while providing the navigation information continuously, the user can not easily recognize the navigation information on the path that the user knows well, thereby reducing the interference of the user.
  • FIG. 10 is a view showing another embodiment of the navigation information displayed when the route guidance information of the navigation and the user's direction switching information in the HMD according to the present invention.
  • the HMD may set the navigation information to the hiding state.
  • the HMD may separately indicate 1010 that the user is in the hiding state, as shown in FIG. 10, so that the user may know that the user is in the hiding state.
  • the HMD maintains the hiding state of the navigation information
  • the navigation information is not output, but internally, the navigation information is continuously updated according to the movement of the user. Therefore, when the path is known to the user, the user may stop the navigation function to reduce power consumption.
  • the HMD sets the navigation information to the hiding state, the user may not recognize whether the navigation function is being executed and may not stop the function. Therefore, the embodiment of FIG. 10 may be usefully used in this case.
  • the HMD according to the present invention can determine whether the route guidance information within a predetermined distance range from the current location of the user matches the direction change information. That is, according to the navigation information, there will be several times when the direction of the user must be changed in the path of the user, and only the direction change information intended by the user can be compared with the direction guidance information that is intended for the user at a close distance.
  • the reason for determining whether the HMD matches the route guidance information and the user's redirection information is to confirm whether the user correctly recognizes the direction in which the user is heading.
  • the direction change information obtained using the EEG signal of the user is information about the direction change existing at the closest distance. Therefore, the HMD does not acquire the direction change information of the user when the direction is not changed in the path guide information within the predetermined distance range or does not compare with the path guide information even if obtained.
  • FIG. 11 is a diagram illustrating an embodiment of a range of comparing route guidance information of a navigation and direction change information of a user in an HMD according to the present invention.
  • the display unit of the HMD can display the navigation map 1110.
  • the HMD determines a range 1120 in which the user's intended direction change information and route guidance information coincide with each other in the route of the navigation map 1110.
  • the proximity distance may be 5 m, for example.
  • the HMD according to the present invention determines whether the path guidance information within a preset time range from the current time matches the direction change information. That is, according to the navigation information, there will be several times when the user needs to change directions in the user's path, and only the user's intended direction change information and route guidance information can be compared with respect to the direction change in the near time.
  • the HMD does not acquire the direction change information of the user when the direction is not changed in the path guide information within the predetermined time range or does not compare with the path guide information even if obtained.
  • the direction change information may not be changed in the route guide information within a predetermined distance or within a predetermined time range, but the direction change information may be obtained from the user. That is, it is a case in which the user incorrectly recognizes the direction, which will be described with reference to FIGS. 12 and 14 to 15.
  • FIG. 12 is a diagram illustrating an embodiment of checking whether a direction intended by a user matches navigation information in an HMD according to the present invention.
  • the HMD may acquire the direction change information that is incorrectly recognized by the user as the direction change even if the direction is not changed in the path guide information within the predetermined distance or the predetermined time range. As shown in FIG. 12A, the HMD may receive the direction change information 1210 from the user even though the direction is not changed within a predetermined range of distance in the path guide information. That is, the direction change information 1210 is information based on the EEG signal generated by determining that the user should change direction by confusing the path.
  • the HMD outputs a message 1220 indicating that the direction change information currently recognized by the user is wrong, and changes the display property so that the user can easily identify the route guidance information to go.
  • the route guide information 1230 may be output.
  • the HMD is not an optimal route, but if the destination can be reached even if it follows the direction change information intended by the user, the HMD outputs a message 1240 indicating that the route is possible, and resets the route guidance information that the user should go.
  • the guide information 1250 may be output.
  • FIG. 13 is a diagram illustrating an embodiment of a usage menu provided to a user by an HMD according to the present invention.
  • the HMD includes a current location menu 1310 for searching for a current location, a path search menu 1320 for searching for a path from the current location to a destination, and an EEG signal for generating direction change information of the user.
  • Registration menu (1330), voice setting menu (1340) to set whether to use the voice guidance when the route guidance, public transport connection menu (1350) available when using the route and auto linkage menu (1360) used in conjunction with the car's navigation system And the like can be provided to the user.
  • HMD has various functions and can be used in various fields.
  • the user can wear it is often used with other devices.
  • the HMD does not need to store the navigation information and is useful because only the direction change information according to the EEG signal of the user is obtained.
  • the navigation information may be received from a vehicle as necessary.
  • FIG. 14 is a view showing an embodiment in which the HMD according to the present invention is used in conjunction with the navigation of the vehicle.
  • the HMD is used in conjunction with the navigation of the car, as shown in Figure 14 when the user tries to exit the highway by car, the exit is particularly useful when there is a continuous confusion to the user.
  • the HMD may acquire direction change information according to an EEG signal of the user in a predetermined time unit. As shown in FIG. 14A, when the user turns direction information to the right 1430a, the HMD compares the route guidance information with the direction change information. At this time, there are two exits at a close distance, and since the exit directions are all right, it is often determined that only the direction change information and the route guidance information are the same.
  • the HMD may compare the route guidance information in the direction 1420a with the direction change information generated when the user thinks that the user should go to the right side 1430a by judging the exit in the direction 1410a. As a result, the HMD does not output a separate note, but after the user enters an exit in the direction of 1410a, the HMD does not know that the path is wrong. Therefore, the user can not go back, there is a problem that must go back a long way.
  • the HMD according to the present invention determines whether a path having the same direction is continuously present within a predetermined distance range from a point in time or a location at which the user intends to change direction information. As a result of the determination, when the path having the same direction exists continuously and the user needs to go to the next direction other than this time, as shown in (b) of FIG. can do.
  • the audio output can provide navigation information to the user more clearly.
  • the preset distance range should be shorter than that when the user walks. That is, in order to determine whether the paths having the same direction continuously exist within the preset distance range, the preset distance range should be short. Therefore, the HMD can set the distance range differently when the user uses the navigation information in the walking state and when the user uses the navigation information in association with the vehicle.
  • the HMD displays the direction change information and the route guidance information. It can be determined to compare and compare. However, even in this case, since there is a path having the same direction within a predetermined distance range, as shown in FIG. 14C, the HMD generates a message 1440 indicating that the direction change information and the path guidance information intended by the user match. It can be displayed separately.
  • the HMD may output the navigation information by changing the display property or the audio property of the path guide information.
  • the degree to which the display property is changed may vary depending on whether the direction change information of the user and the path guide information match.
  • the HMD according to the present invention has an advantage of reducing confusion before the user enters a specific path, and accordingly, proceeds to the designated path. That is, unlike a conventional navigation system in which a user enters a wrong path compared to current location information after the user enters a path, the HMD according to the present invention can perform a notice in advance. In addition, the HMD according to the present invention reduces the case where the user goes the wrong path, so that the user can reach the destination in a short time.
  • FIG. 15 is a view showing another embodiment in which the HMD according to the present invention is used in conjunction with the navigation of a vehicle.
  • the HMD according to the present invention determines whether the direction change information intended by the user and the path guide information included in the navigation information match. At this time, the determination time can be implemented in two forms.
  • the first method is to compare the route guidance information with each turn information obtained through the EEG sensor in the HMD.
  • the second method when there is a change of direction within a predetermined distance or a predetermined time range from the current position according to the route guide information, the direction change information is obtained and compared with the route guide information.
  • FIG. 15 is a diagram illustrating an embodiment of the first method, and according to the first method, has an effect of reducing confusion of a user in an HMD.
  • the HMD obtains the change direction information to change to the right direction 1520 from the user.
  • the HMD follows the route guidance information, there is no route to change the direction within the preset distance range 1510.
  • the HMD may determine that the user thinks about a direction change that is not within the preset distance range 1510 but is present, and may extract route guidance information in which the direction change exists within the closest time or the closest distance range. .
  • the HMD compares the extracted route guidance information with the change of direction information. If the result of the comparison does not match, the HMD outputs a message 1540 indicating that the user incorrectly determines the route as shown in FIG. 15B, and changes and displays the display attribute of the route guide information 1550.
  • the display attribute includes all elements that can be identified by the user, including the size, color, and transparency of the graphic image for the route guidance information.
  • the HMD When the comparison result is matched, the HMD outputs a message 1550 indicating that the user is properly determining the path as shown in FIG. 15C.
  • the HMD also sets the navigation information to the hiding state because the user knows the path. Inform. Accordingly, the HMD sets the route guidance information 1570 that was displayed after the preset time passes and no longer displays the route guide information 1570.
  • the HMD according to the present invention may vary the degree to which the display property is changed depending on whether the user's direction change information and the path guide information match.
  • the present invention determines whether the current location or the distance close to the current location, and the route guidance information to be at the current time or at a time close to the current time and the user's redirection information is matched, and the HMD that outputs the navigation information according to the determination result To provide.
  • the HMD according to the present invention can output the result by comparing the path guidance information at a distance and the direction change information intended by the user as set as described with reference to FIGS. 14 and 15. Therefore, the user can test in advance whether the path that he thinks about the path that he is confused with has an advantage of reducing the error.
  • FIG. 16 is a flowchart of a control method of an HMD according to an embodiment of the present invention. A control method of an HMD according to an embodiment of the present invention will be described with reference to FIG. 16.
  • the HMD receives GPS information (S1610). GPS can be received through a GPS receiver embedded in the HMD, and GPS information can be received from a separate external device using a wired or wireless network.
  • the HMD outputs navigation information using the received GPS information (S1620).
  • the navigation information may include at least one of current location information, electronic map information, and route guidance information, and may include graphic images and audio corresponding thereto. Therefore, when the HMD outputs navigation information to the user, the HMD can display a graphic image and output audio as described with reference to FIGS. 5 to 6.
  • the HMD acquires direction change information intended by the user (S1630).
  • the direction change information is information generated by using the EEG signal of the user measured by the HMD, and is the same as described with reference to FIGS. 3 to 4 and 7. That is, the HMD may measure an EEG signal generated when the user recognizes a direction, store the measured EEG signal, and pattern the EEG signal for each direction to construct a database of the direction change information of the user. Accordingly, when the HMD measures the EEG signal of the user through the EEG sensor, the HMD can acquire the direction change information using the database.
  • the database of the user's redirection information may be built and stored using the HMD's storage unit and controller, and may already be built on an external storage medium.
  • the HMD determines whether the direction guidance information and the route guidance information included in the navigation information match (S1640). As described with reference to FIG. 11, the HMD determines whether a match is within a preset distance range from a current position, or determines whether a match is within a preset time range.
  • the HMD outputs a notice (S1650).
  • the HMD outputs knots using at least one of audio, graphical images, lighting, and vibration. That is, the HMD changes the attribute information in which the audio or graphic image corresponding to the navigation information is output, outputs the light, or emits light from a means capable of outputting a light such as an LED attached to the HMD, or like the vibration state of the mobile phone. You can output knots to users by setting all or part of them to be vibrated.
  • the HMD sets the navigation information to the hiding state (S1660). Since the determination result means that the user knows the route, the HMD does not output unnecessary information, thereby improving user convenience.
  • the HMD should output navigation information when the user requires it. Accordingly, the HMD may release the hiding state if the direction change information and the route guidance information match each other in a predetermined time unit, and release the hiding state. You can release it to output navigation information back to the user.
  • the HMD according to the present invention can set or release the navigation information to a hiding state according to a user's need, thereby increasing user convenience.
  • the head mounted display device and the method of controlling the same according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments are all or all of the embodiments so that various modifications can be made. Some may be optionally combined.
  • the method for controlling the head mounted display device of the present invention can be implemented as a processor-readable code on a processor-readable recording medium provided in the network device.
  • the processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet. .
  • the processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.
  • the present invention relates to a head mounted display device using an EEG signal. Therefore, the present invention has industrial applicability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Dermatology (AREA)
  • Biomedical Technology (AREA)
  • Optics & Photonics (AREA)
  • Navigation (AREA)

Abstract

La présente invention concerne un dispositif d'affichage monté sur tête utilisant des signaux d'onde cérébrale et un procédé pour sa commande et, de manière plus spécifique, un dispositif d'affichage monté sur tête et un procédé pour sa commande, des informations sur un changement de direction prévu par un utilisateur étant acquises à l'aide d'un signal d'onde cérébrale mesuré par le dispositif d'affichage monté sur tête, et un procédé de fourniture d'informations de changements d'informations de navigation selon les informations de changement de direction.
PCT/KR2012/005768 2012-07-19 2012-07-19 Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande WO2014014145A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/KR2012/005768 WO2014014145A1 (fr) 2012-07-19 2012-07-19 Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/KR2012/005768 WO2014014145A1 (fr) 2012-07-19 2012-07-19 Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande

Publications (1)

Publication Number Publication Date
WO2014014145A1 true WO2014014145A1 (fr) 2014-01-23

Family

ID=49948962

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/005768 WO2014014145A1 (fr) 2012-07-19 2012-07-19 Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande

Country Status (1)

Country Link
WO (1) WO2014014145A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105938391A (zh) * 2015-03-06 2016-09-14 松下电器（美国）知识产权公司 穿戴终端和穿戴终端的控制方法
CN106200006A (zh) * 2016-08-29 2016-12-07 周光磊 脑电波控制的智能ar眼镜设备
KR20200046075A (ko) * 2017-09-27 2020-05-06 애플 인크. 공간적 오디오 내비게이션

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07182595A (ja) * 1993-12-22 1995-07-21 Matsushita Electric Ind Co Ltd 危険動作通知機能付きナビゲーション装置
JP2003247847A (ja) * 2002-02-22 2003-09-05 Mitsubishi Electric Corp 脳波信号を用いたナビゲーション装置及びナビゲーション制御方法
JP2005037181A (ja) * 2003-07-17 2005-02-10 Pioneer Electronic Corp ナビゲーション装置、サーバ装置、ナビゲーションシステム、及びナビゲーション方法
KR20090047889A (ko) * 2007-11-08 2009-05-13 한국전자통신연구원 투명 디스플레이를 이용한 증강 현실 구현 방법, 장치 및시스템

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07182595A (ja) * 1993-12-22 1995-07-21 Matsushita Electric Ind Co Ltd 危険動作通知機能付きナビゲーション装置
JP2003247847A (ja) * 2002-02-22 2003-09-05 Mitsubishi Electric Corp 脳波信号を用いたナビゲーション装置及びナビゲーション制御方法
JP2005037181A (ja) * 2003-07-17 2005-02-10 Pioneer Electronic Corp ナビゲーション装置、サーバ装置、ナビゲーションシステム、及びナビゲーション方法
KR20090047889A (ko) * 2007-11-08 2009-05-13 한국전자통신연구원 투명 디스플레이를 이용한 증강 현실 구현 방법, 장치 및시스템

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105938391A (zh) * 2015-03-06 2016-09-14 松下电器（美国）知识产权公司 穿戴终端和穿戴终端的控制方法
CN106200006A (zh) * 2016-08-29 2016-12-07 周光磊 脑电波控制的智能ar眼镜设备
KR20200046075A (ko) * 2017-09-27 2020-05-06 애플 인크. 공간적 오디오 내비게이션
KR102545721B1 (ko) 2017-09-27 2023-06-21 애플 인크. 공간적 오디오 내비게이션
US11709068B2 (en) 2017-09-27 2023-07-25 Apple Inc. Spatial audio navigation

Similar Documents

Publication Publication Date Title
Kim et al. Navigating visually impaired travelers in a large train station using smartphone and bluetooth low energy
US10598506B2 (en) Audio navigation using short range bilateral earpieces
CN104127302B (zh) 一种视障人士行路安全导航方法
Heuten et al. Tactile wayfinder: a non-visual support system for wayfinding
WO2014014145A1 (fr) Dispositif d'affichage monté sur tête utilisant un signal d'onde cérébrale et procédé pour sa commande
WO2019066401A1 (fr) Dispositif électronique et procédé de gestion de géorepérage de celui-ci
US20180106639A1 (en) Travel information providing method, travel information providing apparatus, and non-transitory recording medium
WO2016167230A1 (fr) Système de navigation portable, bague/montre/bracelet et programme
WO2015064892A1 (fr) Procédé permettant d'accentuer un point d'intérêt d'un utilisateur de navigation pour chaque période de temps et serveur de guidage d'itinéraire
KR20170114356A (ko) 전자 장치 및 그 제어 방법
US10430572B2 (en) Information processing system that recognizes a user, storage medium, and information processing method
KR20150069124A (ko) 시각보조기와 이를 이용한 시각장애자용 길 안내 시스템
US20230273034A1 (en) Information processing apparatus and non-transitory computer readable medium storing program
WO2017135533A1 (fr) Appareil d'affichage d'image utilisant une image rémanente d'une source lumineuse, procédé pour afficher une image, et support d'enregistrement pour enregistrer un programme associé
EP1033557A3 (fr) Dispositif de navigation
Henze et al. Non-intrusive somatosensory navigation support for blind pedestrians
JP2018093503A (ja) 音声コンテンツ再生イヤホン、方法、および、プログラム
KR20110028902A (ko) 휴대 단말기에서 네비게이션과 관련된 정보를 제공하는 방법 및 그 휴대 단말기
KR20080057651A (ko) 청각장애인을 위한 소리 인식 보조 시스템 및 소리 인식방법
WO2008061539A1 (fr) Procédé et appareil permettant de transmettre des instructions de navigation à un utilisateur
CN105716600B (zh) 行人导航系统以及方法
WO2019139409A1 (fr) Procédé et dispositif électronique de correction et de génération de données relatives à l'air extérieur en fonction d'un mouvement
Seto et al. A navigation system for the visually impaired using colored navigation lines and RFID tags
KR20160072985A (ko) 스마트 글래스를 이용한 도착시간 알림 시스템 및 그 방법
KR20160013657A (ko) 청각장애인을 위한 손목시계

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12881238

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12881238

Country of ref document: EP

Kind code of ref document: A1