JP6445118B2 - Wearable terminal, method and system - Google Patents

Description

  Embodiments described herein relate generally to a wearable terminal, a method, and a system.

  In recent years, glasses-type wearable terminals have been actively used. For example, at a work site, an operator wears a glasses-type wearable terminal, and a work procedure or the like is displayed on the lens surface. Thereby, it is not necessary for the operator to refer to the paper manual during the operation, and it is possible to efficiently perform an unfamiliar or complicated operation without resting the hand of the operation.

  The glasses-type wearable terminal is shared by many workers, and it is necessary to associate the ID of the terminal with the ID of the worker who wears it in order to leave a record of the work. As an example of this, there is a technique for authenticating a user of a head-mounted display device. A head-mounted display device that allows a user to view a virtual image and an outside scene at the same time, photographing at least a part of the user's viewing direction with the head-mounted display device mounted, and The user is a valid user by using the imaging unit to be acquired, the trajectory acquiring unit for acquiring the user's motion trajectory from the motion captured by the imaging unit, and the trajectory acquired by the trajectory acquiring unit. There has been proposed a head-mounted display device including an authentication processing unit that authenticates whether or not.

  According to this apparatus, the authentication processing unit authenticates whether or not the user is a valid user using the user's movement trajectory, compared with authentication using a character string or an image, It is difficult to copy information that is the key to authentication. As a result, in the head-mounted display device, it is possible to realize authentication that reduces the risk of impersonation and further improves security.

JP 2014-52942 A

  The conventional authentication technology for the head-mounted display device has a problem that it is necessary to separately provide a photographing unit for photographing the user's field of view.

  An object of the present invention is to provide a wearable terminal, method and system for identifying a wearer with a simple configuration.

According to the embodiment, the wearable terminal that can be worn on the head includes first detection means that outputs first information related to the movement of the wearable terminal, second detection means that outputs second information related to the position of the wearable terminal, and Based on the first information output from the first detection means and the second information output from the second detection means, a specifying means for specifying the wearer of the wearable terminal, and a display screen positioned on the line of sight of the wearer The specifying unit displays information on the wearer candidate on the display screen, and specifies the wearer based on a user operation .

It is a perspective view which shows an example of the wearable terminal of embodiment. It is the figure which shows the front view and cross-sectional structure which show an example of a wearable terminal. It is a figure which shows an example of the position detection of a wearable terminal. It is a figure which shows the principle of position detection of a wearable terminal. It is a figure which shows the signal waveform at the time of the position detection of a wearable terminal. It is a figure which shows an example of the system containing a wearable terminal and a communication server. It is a block diagram which shows the electric constitution of a wearable terminal. It is a flowchart which shows the flow of a process of the wearable terminal in the case of wearer determination. It is a flowchart which shows the flow of a process of the communication server in the case of wearer determination. It is a figure which shows the other example of the system containing a wearable terminal and a communication server. It is a flowchart which shows the flow of the telephone call process in the system of FIG.

  Hereinafter, embodiments will be described with reference to the drawings.

  Wearable terminals include head-mounted types (including glasses, goggles, helmets, etc., which may be collectively referred to as glasses), wristbands, pendants, and the like. Here, an embodiment of a glasses-type wearable terminal will be described. Glasses-type wearable devices include a type that allows you to see the scenery ahead of your line of sight through a transparent lens, and a type that is called a head-mounted display where the view is blocked and you cannot see the scenery. Explain the type that can be seen.

  FIG. 1 is a perspective view of a glasses-type wearable terminal (hereinafter simply referred to as a wearable terminal) 10, FIG. 2 (a) is a front view, and FIG. 2 (b) is a diagram showing a cross-sectional structure viewed from above.

  The wearable terminal 10 has substantially the same shape as normal glasses, but here, the projection device 12 is attached to the outside of the temple on the right eye side. Glasses 14 and 16 are fitted in the frame. The glass 14 on the left eye side is a normal transparent glass so that the user can see the scenery. The right-eye glass 16 is at least partially a screen 16. The screen 16 allows the user to see the image projected by the projection device 12. The screen 16 is transparent when the projection device 12 does not project an image, and the user can view the scenery through the glass (screen) 16 on the right eye side.

  The projection device 12 includes a power supply unit 22 and a control unit 24 as electronic components. The power supply unit 22 may include a button-type battery, a rechargeable battery, a secondary battery capable of non-contact power supply, and the like. Alternatively, the projection apparatus 12 may be supplied with power from an external power supply via a power supply line without incorporating a power supply. The control unit 24 communicates with a server and other electronic devices via a network described later, and transmits and receives information. This communication may be either wired or wireless. For wireless, Bluetooth (registered trademark), ZigBee (registered trademark), short-range wireless communication such as UWB, medium-range wireless communication such as WiFi (registered trademark), 3G / 4G, WiMAX (registered trademark) depending on the usage environment Any of long-distance wireless communications such as the above may be used.

  The projection device 12 further includes a light source 28, a display unit 30, a prism 32, a lens group 34, and the like as optical components. The light source 28 may be a dimming white LED light source including a plurality of, for example, three LEDs (Light Emitting Diodes) whose emission colors are different from each other and whose output light amounts can be changed independently. According to the dimmable white LED light source, even when the use environment of the wearable terminal 10 is, for example, in a clean room where illumination mainly using orange is used, the emission color is changed according to the use environment. And a clear projected image can be obtained. Furthermore, according to the dimming white LED light source, it is possible to output a display color that is easy for the user to see, and compared with the case where the user outputs a display color that is difficult to see, eye fatigue, migraines associated therewith, etc. It is possible to avoid the occurrence of a factor that hinders the user.

  The display unit 30 is, for example, a reflective LCD (Liquid Crystal Display) module. Based on display control by the control unit 24, a predetermined text, image, or the like (hereinafter, what the display unit 30 displays is referred to as a display image). May be collectively referred to). Non-parallel light emitted from the light source 28 (divergent light, sometimes referred to as divergent light) is reflected by the half mirror surface 32a of the prism 32 and illuminates the display image of the display unit 30. The reflected light of the display unit 30 passes through the half mirror surface 32a as light corresponding to the display image (sometimes referred to as image light), is emitted from the emission surface 32c, and is projected through the lens group 34 to a predetermined size. An image is projected onto the screen 16.

  The screen 16 includes a near-side transparent refractor 42, a Fresnel lens-shaped half mirror surface 44, and a back-side transparent refractor 46. A part of the image light reaching the Fresnel lens half mirror surface 44 is reflected by the Fresnel lens half mirror surface 44 to form a virtual image (projected image) corresponding to the display image of the display unit 30 several meters ahead. Note that the screen 16 can partially transmit the scenery ahead of the user's line of sight wearing the wearable terminal 10, and the screen 16 is configured to display a scenery that can be visually recognized by the user together with the projected image. Also good.

  Part of the image light (diverged light) emitted from the light source 28 and passed through the half mirror surface 32a is totally reflected by the total reflection surface 32b, refracted by the emission surface 32c, and leaked light that is divergent light from the light source 28. 50. The leaked light 50 is emitted in a direction different from that of the screen 16 through an opening or a gap (guide portion) 52 formed on the front surface of the projection device 12.

  The wearable terminal 10 includes a speaker 54A, an earphone jack 54B, a microphone jack 56, a slide switch 57, a rotary switch 58, and the like at a predetermined position of the projection device 12, for example, a bottom surface portion. A hands-free microphone (not shown) is connected to the microphone jack 56 to collect user's voice. The slide switch 57 can adjust the brightness, color tone, and the like of the projection image of the projection device 12, for example. The rotary switch 58 can adjust the projection angle of the projection image, for example. By making it possible to set different adjustment amounts by different operations such as the slide switch 57 and the rotary switch 58, the user can adjust the projection image by blind touch while viewing the projection image. For example, by operating the slide switch 57, it is possible to provide a projected image with display brightness and color tone that suits the user's preference. By operating the rotary switch 58, the projection angle can be adjusted so as to display an image at an optimal position in accordance with the shape and size of the user's head. Of course, the objects to be adjusted by the slide switch 57 and the rotary switch 58 may be reversed, or the positions of the slide switch 57 and the rotary switch 58 may be reversed. You may assign to two types of operation of one operation member.

  The selection by these switches 57 and 58 may be carried out in practice and error while only looking at the projection image. However, in order to increase the efficiency of adjustment, a menu screen is projected and items are selected on the screen. You may adjust. When the display unit 30 displays the menu screen, the menu screen is projected on the screen 16.

  Further, the menu item may be selected by a touch operation, not by the operation of the switches 57 and 58. For this reason, a touch pad 55 is also provided outside the projection device 12. When the display unit 30 displays a menu or the like and touches a position in the touch pad 55 corresponding to the display position of the item in the menu, a user operation can be input easily and efficiently.

  A camera 59 is provided outside the center of the front, and an image ahead of the user's line of sight (either a still image or a moving image is acceptable) can be taken. Although not shown, a camera is provided on the inner side of the front center (a position corresponding to the arrangement position of the camera 59) so as to face the user's face so that the user's eyeball can be photographed and the user's iris can be detected. Also good. The iris can be used for user authentication.

  By using the leaked light 50 from the wearable terminal 10, the state of the wearable terminal 10, that is, the state of the user can be detected. The principle of detecting the state of the wearable terminal will be described with reference to FIGS. Here, the state includes a position and a movement of the position.

  An example of the usage example of the wearable terminal is shown in FIG. For example, an arbitrary number of work spaces or product shelves A01 to Axy (x and y are both positive integers), B01 to Bxy in a work area 60 such as a factory parts yard, a mail order company's product warehouse, or a retail delivery department , C01 to Cxy are arranged. The work space or the product shelf may be, for example, a factory work table, a manufacturing apparatus in a production line, a school desk, a seating position in a conference room, or the like.

  In the work area 60, at least one optical sensor 62-1 to 62-n (n is a positive integer) is arranged. The optical sensors 62-1 to 62-n are wearable terminals 10-1 to 10-m (m is a positive integer) position (x, y, z), number, change in position (movement), change in orientation, and the like. Can be detected individually by the detection method shown in FIGS. By detecting changes in position, number, movement, and orientation of the wearable terminals 10-1 to 10-m. It is possible to recognize the positions, movements, and the like of an arbitrary number of users wearing the wearable terminals 10-1 to 10-m.

  The user can move freely in the work area 60. The user performs a predetermined work in a work space 64 that is a predetermined position, for example, a station (cart), a container according to the work, or a movable table. The work space 64 is not movable and may be a fixed desk, a seating position thereof, or the like.

  As shown in FIGS. 3 and 4, the detection system includes one or more wearable terminals 10 and one or more optical sensors 62. The optical sensor 62 has a function of detecting the leaked light 50 and a communication function of transmitting the detection result to a server or the like. This communication function may be either wired or wireless, similar to the communication function of wearable terminal 10. For wireless, Bluetooth (registered trademark), ZigBee (registered trademark), short-range wireless communication such as UWB, medium-range wireless communication such as WiFi (registered trademark), 3G / 4G, WiMAX (registered trademark) depending on the usage environment Any of long-distance wireless communications such as the above may be used. Although the embodiment described below includes various units and modules having a communication function, the communication functions of these units and modules may be either wired or wireless as well. For wireless, Bluetooth (registered trademark), ZigBee (registered trademark), short-range wireless communication such as UWB, medium-range wireless communication such as WiFi (registered trademark), 3G / 4G, WiMAX (registered trademark) depending on the usage environment Any of long-distance wireless communications such as the above may be used.

  In order to be able to identify the wearable terminal 10 from the leaked light 50 received by the optical sensor 62, the wearable terminal 10 uses the information including the terminal identification information (hereinafter also referred to as terminal ID) to detect the leaked light 50. Modulate intermittently. A typical example of the modulation method is a chopper type modulation method that reduces the light emission amount to zero. Here, a modulation method that can secure a light emission amount equal to or larger than a predetermined amount even when the light emission amount is low is adopted. Thereby, the burden with respect to a user's eyes can be reduced. As a modulation method, for example, a DSV (Digital Sum Value) free modulation method (that is, a modulation method in which a DSV of a modulation signal is always calculated, a bit inversion code can be inserted as appropriate, and a DC component is zero) is employed. The change in the amount of light emission in a relatively long range is suppressed, the change in the amount of light emission can always be made macroscopically, and the burden on the user's eyes is further reduced. Since the human eye can recognize a change of about 0.02 seconds, an effect of reducing the burden on the user's eyes by setting the reference frequency of the modulation to 10 Hz or more, for example, 20 Hz or more, more preferably 60 Hz or more. Is also born. On the other hand, since the LED used for the light source 28 has internal impedance and connection capacitance, the accurate modulation frequency is preferably less than 100 MHz, preferably 10 MHz or less. Therefore, the modulation frequency of the light source 28 used in the detection system of the embodiment is preferably 10 Hz to 100 MHz, and more preferably 10 Hz to 10 MHz.

  Since the diverging light leakage light 50 from the light source 28 is used, the amount of light detected by the optical sensor 62 changes according to the distance between the wearable terminal 10 and the optical sensor 62. If this phenomenon is used, the distance between the wearable terminal 10 and the optical sensor 62 or the orientation of the wearable terminal 10 with respect to the optical sensor 62 can be obtained. Since the position (including height) of the optical sensor 62 is fixed, if the distance between the optical sensor 62 and the wearable terminal 10 is known, the position (x, y, z) of the wearable terminal 10 can be detected.

  Furthermore, since the diverging light leakage light 50 from the light source 28 is used, the leakage light 50 can be detected in a relatively wide range. As a result, only by installing a relatively small number of optical sensors 62-1 to 62-n, the position of the wearable terminals 10-1 to 10-m in the work area 60, the distance between the wearable terminal 10 and the optical sensor 62, The direction of wearable terminals 10-1 to 10-m or the direction of wearable terminal 10 with respect to optical sensor 62 can be detected. Thereby, the installation cost required in order to install a detection system can be reduced.

  The light amount information of the leakage light 50 detected by the optical sensor 62 is transmitted from the optical sensor 62 to a server described later at a predetermined timing. The server analyzes the collected information from the optical sensor 62. Thereby, arbitrary wearable terminals 10-1 to 10-m, that is, the position and state of the user can be detected.

  FIG. 4 is a schematic diagram illustrating a specific usage example of the system that recognizes the wearable terminal according to the embodiment. Assume a situation in which there are three users wearing wearable terminals 10-1 to 10-3 around four optical sensors 62-1 to 60-4. The leaked light 50 from the wearable terminals 10-1 and 10-2 is detected by the optical sensors 62-1 to 60-4. Each of the optical sensors 62-1 to 60-4 performs AD (Analog-Digital) conversion on the light amount of the leaked light 50 detected in each of them, and uses, for example, short-range wireless communication at predetermined timing as light amount information corresponding to the light amount. To the server.

  Wearable terminal 10-1 moves to optical sensor 62-1 according to the movement of the user, while wearable terminal 10-2 is temporarily oriented according to the user's arbitrary operation, for example, swinging (head turning). Suppose that The change of the detection information at this time is shown in FIG.

  In FIG. 5, the example which used the intermittent time change system as a modulation system of each leaked light 50 of wearable terminal 10-1 to 10-3 is shown. That is, the ID modulation period is shifted in each of the wearable terminals 10-1 to 10-3.

  As shown in FIGS. 5A, 5 </ b> B, and 5 </ b> C, the ID modulation period of the wearable terminal is intermittently set for the first to third wearable terminals 10-1 to 10-3, and the others This period is a non-modulation period. Within each ID modulation period, the synchronization signal SYNC and the terminal IDs of wearable terminals 10-1 to 10-3 constitute one set (corresponding one-to-one), and the set is a plurality of times (as shown in FIG. 5). If there are four sensors, a multiple of 4) is repeated.

  As soon as the first wearable terminal 10-1 enters the non-modulation period, the ID modulation period of the second wearable terminal 10-2 starts. Similarly, the ID modulation period of the third wearable terminal 10-3 starts at the same time as the second wearable terminal 10-2 enters the non-modulation period.

  In the ID modulation period of the second wearable terminal 10-2 and the ID modulation period of the third wearable terminal 10-3, the synchronization signal SYNC and the terminal IDs of the wearable terminals 10-2 and 10-3 are repeatedly modulated. Is done. Thus, the terminal ID can be detected by placing the terminal ID of the wearable terminal 10 in the modulated signal.

  In the above example, the modulation timings of the wearable terminals 10-1 to 10-3 are time-division (intermittent). However, for example, all the wearable terminals 10-1 to 10-3 may be continuously modulated, and the modulation reference frequencies of the wearable terminals 10-1 to 10-3 may be changed. Moreover, each frequency spectrum characteristic at the time of spectrum spreading may be changed.

  As shown in FIGS. 5D, 5E, 5F, and 5G, the information communication period from the optical sensors 62-1 to 62-4 is finely divided in each ID modulation period.

  As shown in FIG. 4, at the initial time, part of the leaked light from the wearable terminal 10-1 reaches the optical sensor 62-4. Therefore, at the initial time, as shown in FIG. 5 (k), the optical sensor 62-4 detects the leaked light from the wearable terminal 10-1. However, as the wearable terminal 10-1 moves toward the optical sensor 62-1, the modulation signal amplitude of the leaked light from the wearable terminal 10-1 detected by the optical sensor 62-4 decreases. On the other hand, as shown in FIG. 5 (h), the modulation signal amplitude of the leaked light from the wearable terminal 10-1 detected by the optical sensor 62-1 increases with time. In this way, by comparing the time change of the amplitude of the modulation signal detected by the optical sensors 62-1 to 62-n, the time change (moving state) of the position of the wearable terminal 10-1 to 10-m that is the detection target. Can be detected.

  On the other hand, since the wearable terminal 10-2 is directed to the optical sensor 62-3 at the initial time, the modulation signal amplitude obtained from the leaked light is detected by the optical sensor 62-3 from the detection value of the optical sensor 62-2. The detected value of is larger. Thereafter, for example, it is assumed that the second user swings his head and temporarily faces the optical sensor 62-2. Then, the detection output of the wearable terminal 10-2 output by the optical sensor 62-2 temporarily increases and then decreases as shown in FIG. 5 (i). On the other hand, as shown in FIG. 5 (j), the detection output of the wearable terminal 10-2 output by the optical sensor 62-3 increases after being temporarily reduced.

  In this way, by comparing the time change of the modulation signal amplitude detected by the optical sensor 62, the time change in the direction of the wearable terminals 10-1 to 10-m that are detection targets can also be estimated.

  The above detection example is an example in the case of movement or swing as the user's movement. However, the present invention is not limited to this, and various other actions of the user may be used. For example, the leakage light may be temporarily blocked by moving the user's hand, twisting the upper body (body), or the like. In this case, in all the optical sensors 62-1 to 60-4, the modulation signal amplitude temporarily decreases in the same time zone. Thus, by comparing the relevance of the change in the modulation signal amplitude of all the optical sensors 62-1 to 60-4, it is possible to identify different behavior patterns of the user.

  By using the above method, it is possible not only to detect the user's behavior but also to recognize the user's intention.

  Note that a beacon may be used as a method of detecting the position (x, y, z) of the wearable terminal 10. In the above-described example, the light whose terminal identification information is modulated is emitted from a large number of wearable terminals 10 to a large number of optical sensors 60, and the information received by the large number of optical sensors 60 is compared to determine the position of the wearable terminal 10, Although the state is detected, a large number of position information transmitters are arranged in the work area 60, and a beacon according to the arrangement position is transmitted from the transmitter by short-range wireless communication such as RF-ID having a reach distance of several meters, for example. Then, the wearable terminal 10 that has received it can be considered to be at substantially the same position as the position of the transmitter. Furthermore, the position of the wearable terminal can be detected using GPS. The position detection does not need to be based on only one type of method, and the detection accuracy can be improved by using a plurality of methods in combination.

  FIG. 6 shows an example of the entire system using a wearable terminal. The system includes one or more LANs 102 and one wide area network 104. The wide area network 104 may be a network for each factory, building, or company, or the Internet. The LAN 102 may be a unit such as a factory building, a department, a building floor, a company sales office, or the like. Each LAN 102 includes a plurality of wearable terminals 10, an administrator terminal 202, a plurality of optical sensors 62, an access point 112, a plurality of position information transmitters 113, a plurality of cameras 114, and the like. Wearable terminal 10, administrator terminal 202, optical sensor 62, position information transmitter 113, camera 114, and the like are connected to access point 112 wirelessly. The LAN 102 is connected to the communication server 106 via the wide area network 104. The position information transmitter 113 transmits a beacon according to the position by short-range wireless communication such as RF-ID having a reach distance of several meters. The camera 114 captures an area of the LAN 102, and the user's action can be known by analyzing the image. For example, the wearing user can be identified by storing a standard image for each user and comparing it with an image when the user attaches or removes the wearable terminal 10. In the case where a large number of cameras cannot be arranged, a camera 114 that can photograph a wide range of users with a single photographing direction can be arranged.

  The communication server 106 includes an information search unit 120, a position / information management unit 122, a wearer estimation unit 124, a schedule management unit 126, and the like.

  The position / state management unit 122 collects and manages information on the position and state of the wearable terminal 10 and the administrator terminal 202. Further, the information is extracted from the information search unit 120. The schedule management unit 126 has a function for managing and providing schedule information and attendance records of employees who wear terminals. The schedule information includes the work schedule of the employee (from what time to what time in which area he / she will be engaged in work, etc.), the time of departure / working, and the work record (from what time to what time in which area he / she is working. Etc.). Further, the information is extracted from the information search unit 120. The information search unit 120 has a function for searching and processing information held by the position / state management unit 122 and the schedule management unit 126. The wearer estimation unit 124 may determine who the wearer is by correlating existing feature data of the managed employee with the given feature data of the user.

  FIG. 7 shows an example of the electrical configuration of the wearable terminal 10. The wearable terminal 10 includes a CPU 140, a system controller 142, a main memory 144, a storage device 146, a microphone 148, a speaker 54, a projection processing unit 150 (controlling the light source 28 and the display unit 30), a camera 59, a wireless communication device 152, a motion A sensor 154, a line-of-sight detection sensor 156, a gesture sensor 158, a touch pad 55, a vibrator 68, a position information receiver 159, a GPS module 155, and the like are provided.

  The CPU 140 is a processor that controls the operation of various modules in the wearable terminal 10, and executes a computer program loaded into the main memory 144 from the storage device 146 formed of a nonvolatile semiconductor memory such as an SSD or a flash array. These programs include an operating system (OS) and various application programs. The CPU 140 executes various application programs and communicates with the communication server 106 via the network using the wireless communication device 152, and performs the following processing, for example. For example, the CPU 140 performs various controls such as performing voice input using the microphone 148, sending voice data to the communication server 106, and generating sound from a stereo earphone or speaker 54 (not shown) connected to the earphone jack 54B. . If a stereo speaker is required, a speaker may be provided on the left eye temple, although not shown in FIGS.

  The system controller 142 is a device that connects between the local bus of the CPU 140 and various components. The microphone 148 is connected to the microphone jack 56 and collects voice or environmental sound emitted by the user. By recognizing the voice uttered by the user or analyzing the environmental sound, the user's behavior can be estimated and the user can be identified. For example, the wearing user can be specified by storing the standard voice for each user and comparing it with the voice generated by the wearer. Further, by analyzing the environmental sound, it is possible to identify the work place where the wearer is located, and to narrow down the users who are in the place from the user's action schedule. The speaker 54 outputs an alarm for alerting the user. The projection processing unit 150 outputs an image signal to the display unit 30 and lights the light source 28 to project the image of the display unit 30 on the screen 16. This image can include not only still images but also moving images. The wireless communication device 152 has a wireless LAN function, for example, and connects the wearable terminal 10 and the access point 112 wirelessly.

  The motion sensor 154 is a sensor that integrates a three-axis acceleration, a three-axis gyro, and a three-axis geomagnetic sensor. The motion sensor 154 can detect the movement of the head of the user who uses the wearable terminal 10, and as a result, determines the direction in which the face is facing. To do. The line-of-sight detection sensor 156 is provided toward the user's face inside the center of the frame of the glasses, photographs the user's eyeball, and detects the movement of the line of sight. Further, the line-of-sight detection sensor 156 may be capable of detecting the user's iris. The gesture sensor 158 is a sensor that determines a gesture caused by finger movement. Specifically, it is a generic term for sensors that determine a user's gesture by analyzing the movement of a finger on the touch pad 55 provided on the projection device 12 or an image of the camera 59. The vibrator 68 vibrates the temple of the wearable terminal 10 by vibrating the projection device 12, and conveys some information to the user. The position information receiver 159 receives a beacon including position information transmitted from a plurality of position information transmitters 113 arranged in the area of the LAN 102 using near field communication such as RF-ID. Since it is short-range wireless communication, the position of the transmitter and the receiver (wearable terminal) can be regarded as almost the same. The GPS module 155 detects the position (x, y, z) of the wearable terminal 10. By combining this detection result, the detection result of the position information receiver 159, and the detection result of the optical sensor 62 in FIG. 3, the position of the user and its change can be detected more accurately.

  Next, determination of who is the wearable terminal user by the wearable terminal 10 and the communication server 106 configured as described above will be described. Here, the usage environment of the wearable terminal 10 is assumed to be a work site where there are a large number of work objects, a large number of work processes, and a large number of workers are involved. Assume that an administrator who supervises the worker is arranged in a predetermined area, for example, each area of the LAN 102, and the administrator uses the administrator terminal 202. The administrator terminal 202 may have the same configuration as the wearable terminal 10, but since the administrator does not need to move, the administrator terminal 202 may have the same configuration as a normal personal computer or tablet, and detailed description of the administrator terminal 202 is omitted. It is not necessary for all workers at the work site to wear the wearable terminal 10. Therefore, it is not necessary to prepare all the wearable terminals 10 for all the workers, and only a predetermined number is prepared, and a shared wearable terminal in which a necessary worker is available is attached. The system needs to identify the user when wearing. The reason for this is to notify a specific user of something by the projected image of the screen 16, sound from the speaker 54, vibration of the vibrator 68, or the like, or based on the wearing state of the wearable terminal 10. For managing behavior.

  FIG. 8 is a flowchart showing the process of the wearable terminal 10, and FIG. 9 is a flowchart showing the process flow of the communication server 106.

  In the wearable terminal 10, the CPU 140 determines whether the power is turned on in block B12. When it is on, the CPU 140 collects one or more outputs of the motion sensor 154, the line-of-sight detection sensor 156, and the gesture sensor 158 in block B14. The CPU 140 also collects the output of the built-in camera 59 and / or microphone 148 in block B16.

  In block B20, the CPU 140 performs an arithmetic process on these outputs, and calculates a feature amount indicating the movement of the wearable terminal 10, in other words, the behavior of the user carrying the wearable terminal 10. The feature amount is, for example, a frequency component calculated from time-series data within a certain time with acceleration and angular velocity obtained from the motion sensor 154. In the obtained frequency component, a component important for processing to be estimated may be subjected to various signal processing such as enhancement. In the estimation process, time-series data in which feature amounts for a certain time are collected is used for analysis. Based on the output of the motion sensor 154, a feature amount representing the movement of the user's hand can be calculated. Based on the output of the line-of-sight detection sensor 156, a feature amount representing the movement of the user's line of sight can be calculated. Based on the output of the gesture sensor 158, a feature value representing the movement of the user's hand can be calculated. By analyzing the image captured by the built-in camera 59, a feature value representing the movement of the body including the user's face can be calculated. Similarly, by recognizing the environmental sound collected by the microphone 148, the feature amount representing the user's movement can be calculated. For example, it is possible to know the wearing state of the wearable terminal based on the friction sound between the temple and the skin or the hair when the wearable terminal 10 is put on the face or removed from the face. Furthermore, the user can be specified by recognizing the voice uttered by the user collected by the microphone 148. Note that all of these outputs are not necessarily required for the calculation of the feature amount, and signals used for the calculation can be selected as appropriate.

  As described above, as long as the power is turned on, the wearable terminal 10 constantly calculates a feature amount indicating the movement of the wearable terminal 10, in other words, the action of the user who owns the wearable terminal 10. When the wearable terminal 10 is put on the face or removed from the face, that is, when the wearable terminal 10 is attached / detached, an individual's habit is likely to appear in the user's behavior. Therefore, the user is specified based on a certain period or a certain number of feature amounts after the start of attachment / detachment is detected.

  In block B <b> 22, the CPU 140 determines whether or not the calculated feature value has a high correlation with the feature value at the time of attachment / removal obtained and stored in advance. If not, the determination process of block B12 is executed again. When it is determined that the correlation is high, a window process for a certain period of time for specifying the user is started. Here, feature quantities are calculated multiple times within a certain time, and a user is specified based on the plurality of feature quantities. However, a predetermined number of feature quantities are calculated instead of window processing for a certain period of time. It may be a window processing of variable time until it is done.

  In block B24, the CPU 140 collects one or more outputs of the motion sensor 154, the line-of-sight detection sensor 156, the gesture sensor 158, the built-in camera 59, and the microphone 148 in the same manner as the blocks B14, B16, and B20. Then, one feature amount representing the user's behavior is calculated by a predetermined calculation. The CPU 140 accumulates this one feature amount in block B26. In block B28, the CPU 140 determines whether or not a predetermined time has elapsed from the start of block B24. If NO, the data collection / feature amount calculation process of block B24 is executed again, and the feature amounts are accumulated one after another.

  When the predetermined time has elapsed, the CPU 140 transmits to the communication server 102 the feature value at the time of attachment / detachment determined in block B22 and the plurality of feature values accumulated in block B26 in block B32.

In the communication server 106, as shown in FIG. 9, the position / state management unit 122 receives the outputs of a large number of optical sensors 62 in block B44, detects the positions and states of a large number of wearable terminals 10, and Table 1 Write to the position / state management table as shown in. The position / state management table includes items of terminal ID, date and time, position, state, feature amount, and user ID. In block B44, only the terminal ID, date / time, and position are written. Each time block B44 is repeated, the time is updated, and the position / state management table is updated accordingly.

The information search unit 120 waits in block B46 to receive the feature amount transmitted from any of the wearable terminals 10 in block B32. Until the feature value from the wearable terminal 10 is received, the process of block B44 is executed again. When the information is received, the information search unit 120 writes the received feature amount in the position / state management table. Here, it is assumed that the feature amount is transmitted from the wearable terminal with terminal ID = 02. Table 2 shows a position / state management table at the stage where the feature amount of the wearable terminal with terminal ID = 02 is written.

  In block B48, the position / state management unit 122 inputs the output image of the camera 114 including the position (X1, Y1) of the wearable terminal with the terminal ID = 02 among the several cameras 114 in the area as the subject of photographing. , Analyze the image. By this analysis, the presence or absence of a user near the position (X1, Y1) can be known. When it is determined from the image that the user is not present despite the user position (X1, Y1) being detected from the output of the optical sensor 62, the detection result by the optical sensor 62 can be determined to be an error. When such an error is found, the position / state management unit 122 changes the corresponding position information indefinitely. Thus, since the position of the wearable terminal is detected by combining the outputs of the plurality of sensors, the accuracy is improved.

In block B52, the information search unit 120 generates a non-wearer list, which is a list of users who have no wear state, as shown in Table 3, from the position / state management table in the position / state management unit 122. The list and the feature quantity received in block B46 are sent to the wearer estimation unit 124. All wearable terminals are not worn by all users in an undefined initial state.

In block B54, the information search unit 120 reads out the user-specific schedule table as shown in Table 4 from the schedule management unit 126, and refers to this to engage in work of workers who are not working or other places. A worker who is a bag and has no bag in this place is identified, and as shown in Table 4, the state of such a worker is changed to absent.

  As a result, the list of non-wearing users can be narrowed down, so that the amount of subsequent correlation calculation is reduced and the calculation time is shortened. Furthermore, the accuracy of mounting / non-mounting determination is improved.

The wearer estimation unit 124 determines, based on the feature amount received in block B46 and the unmounted wearer list, which of the unattached users has the highest correlation with the standard feature amount at the time of attachment / detachment of the user. Examine and extract wearer candidates. The number of candidates to be extracted is not limited to one, but may be multiple. Standard feature values of all workers when the wearable terminal is attached / detached are checked in advance, and a standard feature table for each user as shown in Table 5 is stored in the wearer estimation unit 124. The standard feature amount is obtained by attaching or removing the wearable terminal to all workers, and one or more of the motion sensor 154, the gaze detection sensor 156, the gesture sensor 158, the built-in camera 59, and the microphone 148. The output is obtained by performing the same operation as the operation executed in block B24 by the CPU 140 of the wearable terminal.

  Since the correlation calculation is not intended for all users and is limited to users who are determined not to be worn, the user can be specified with high accuracy in a short time. In block B58, the wearer estimation unit 124 transmits the specific result as a candidate to the wearable terminal 10 having the terminal ID = 02 that transmitted the feature amount.

In wearable terminal 10 with terminal ID = 02, as shown in FIG. 8, CPU 140 determines whether a candidate has been received from communication server 106 in block B <b> 34. When the candidate is received, the CPU 140 displays the candidate on the display unit 30, emits the light source 28, and displays the candidate on the screen 16 in block B <b> 36. For example, the display example may be a display requesting confirmation from the user such as “You are Mr. XX?”. On the other hand, as shown in block B38, the user makes a high / no decision. Specific examples of the determination operation include shaking the face “vertically / horizontally”, inputting “yes / no” from the microphone 148, touching / tapping the touch pad 55 once (high) / 2 times (no), 1 There are finger gesture operations such as touching with two fingers (high) / two fingers (no). The gesture operation can also be recognized by analyzing an image taken by the camera 59. When there are a plurality of candidates, a display such as “Are you (1) AA, (2) BB, or (3) CC?” Is performed and a selection operation is performed. For the selection operation, the face is shaken as many times as there are choices, the number of choices is input from the microphone 148, the touchpad 55 is touched / tapped as many times as there are choices, and the number of fingers corresponding to the choices is selected. Touch with, finger gesture operation etc. The processing result of the determination operation is transmitted from the wearable terminal 10 to the communication server 106.

  In the wearable terminal 10, after the determination operation is performed, if not finished, the process of block B <b> 12 is repeated.

As illustrated in FIG. 9, in the communication server 106, the wearer estimation unit 124 determines whether or not the processing result of the determination operation is transmitted from the wearable terminal 10 in block B <b> 62. When the processing result is received, in block B64, as shown in Table 6, the state management unit 122 writes the wearer's user ID of the wearable terminal with the terminal ID = 02, in the position / state management table, and the position / state management table Update.

  The communication server 106 repeatedly executes the above process.

  As described above, when the user who uses the wearable terminal is identified by calculating the correlation between the feature amount representing the operation of the wearable terminal 10 and the standard feature amount at the time of wearing of each user, Excluding the determined user, only the correlation with the feature amount of the non-wearing user is calculated. For this reason, the amount of calculation for obtaining the correlation is reduced, the calculation time is shortened, and the accuracy specified by the user is improved.

  In the embodiment, the feature amounts are obtained based on outputs of a large number of types of sensors, cameras, and microphones. However, it is not necessary to use all of them, and any one or a plurality can be selected as appropriate according to the situation. Although the user specifying process is performed not only at the time of attachment but also at the time of removal, it may be performed only at the time of attachment. The window processing for user identification may be performed for a certain period of time until a certain number of feature quantities are obtained.

  The feature quantity for estimation is obtained by the wearable terminal and the correlation calculation is performed on the server side. However, the correlation calculation is performed by the wearable terminal, and the server may be unnecessary.

  FIG. 10 shows an example of the entire system according to another embodiment. In addition to the configuration of FIG. 6, the LAN 102 is also connected to a network 104 via an access point 112 for a device 115 on which an operator works. The apparatus 115 may be anything, but here, it is a manufacturing facility that requires periodic inspection and maintenance, and may occasionally break down, and the worker can recover.

  The communication server 106 includes a voice command processing unit 128, a call processing unit 130, and the like in addition to the configuration of FIG.

  The voice command processing unit 128 converts the voice uttered by the user into a command for executing a corresponding service on the communication server 106. The input data to the voice command processing unit 128 may be voice data itself collected from a microphone or text data after voice recognition of input voice, and this point is not particularly limited. The call processing unit 130 has a host function for realizing an instant messaging function and a voice call function between terminals.

  Since the electrical configuration of wearable terminal 10 is the same as in FIG. 7, illustration is omitted, but in addition to the processing described in FIG. 7, CPU 140 performs voice input using microphone 148 to communicate voice data. An electronic manual is stored based on the processing result of the voice command processing unit 128 of the communication server 106, sent to the voice command processing unit 128 of the server 106, sounded from a stereo earphone or speaker 54 (not shown) connected to the earphone jack 54 B It is also possible to perform various controls such as downloading to the device 146 and generating an image image displayed on the display unit 30 and projected onto the screen 16 while expanding a part thereof on the main memory 144. .

  An example of use of the system of FIG. 10 will be described. This system includes a communication server 106, and has a communication function such as an instant messaging function between users of wearable terminals or between a user of wearable terminals and an administrator. In a normal instant messaging function, when a person who wants to start a call issues a call start request, a call start screen is displayed on the screen of the terminal of the other party who wants to call. However, if the call start screen is suddenly displayed on the screen 16 of the wearable terminal of the worker who is performing the recovery work of the failed device 115, the worker's view may be lost, and the worker is exposed to danger. there is a possibility. Furthermore, while the operator is on the phone, the restoration work is forced to be interrupted, and it takes time for the device 115 to recover, resulting in an undesirable situation such as a reduction in the operating rate of the device 115. Therefore, in this embodiment, efficient notification is performed without obstructing the worker's view by avoiding sudden display of information on the screen of the wearable terminal of the worker who is performing the work. Specifically, a call start notification sound is generated on the wearable terminal of the worker who wants to be notified, and the call start screen is displayed on the screen of the wearable terminal of the worker only when the worker responds positively to it. Is done. The notification sound is generated from a stereo earphone (not shown) connected to the earphone jack 54B. Although the notification sound is a simple sound, for example, the generation position in the three-dimensional space such as the upper right and the left side can be controlled, and the generation position is determined according to the notification situation. For this reason, since the worker can recognize the content of the notification based on the mere sound generation position, the worker can receive the notification without interruption of concentration even during the work. If the worker hears the message sound instead of the notification sound, the worker may be distracted by the message, and there is a risk that the operation will be interrupted or the hand may be caught in the device 115.

  FIG. 11 is a flowchart showing the flow of each process of the administrator terminal 202, the communication server 106, and the wearable terminal 10 when the administrator wants to talk to the worker.

  In the administrator terminal 202, the administrator activates the instant messaging application in block B102, and requests a call with the worker X in block B104.

  In the communication server 106, the call processing unit 130 receives a call request in block B112. Here, as shown in FIG. 9, the communication server 106 always receives the output of the optical sensor 62 and manages the position / state of the wearable terminal 10 in real time. Furthermore, the communication server 106 also manages the position of the administrator terminal 202. For this reason, the administrator terminal 202 may have a GPS function and be configured to be able to detect latitude, longitude, and altitude. In block B114, the call processing unit 130 detects the positions and states of the administrator terminal 202 and the wearable terminal of the user X from the position / state management unit 122. In block B116, the call processing unit 130 transmits a call request to the wearable terminal of the user X and also transmits location information of the administrator terminal.

  In the wearable terminal of the user X, when a call request is received at block B122, the CPU 140 detects the orientation of the operator's face based on the output of the motion sensor 154 or the like at block B124. In block B126, the CPU 140 calculates a position on the space to be reproduced as the stereophonic sound of the notification sound generated using the stereo earphone based on the position (latitude, longitude, altitude) of the administrator and the face orientation of the user X. To do. Here, the position in space is calculated so that a notification sound is generated from the callee (user X) to the position of the caller (manager) in the actual direction. In block B128, the CPU 140 determines the volume of the notification sound. The volume may be determined by the distance between the caller and the callee, or may be determined by a setting on the instant messaging application that determines what the caller is for the caller.

  The subsequent processing is repeated until the caller cancels the call processing or performs the call processing. In block B132, the CPU 140 determines whether or not the administrator cancels the call processing. If not, the CPU 140 generates a notification sound from the stereo earphone at block B134. When the speaker 54 is a stereo speaker, it may be generated from the speaker 54.

  Since the worker knows the position of the manager, when the notification sound is heard from the direction, the manager knows that he / she wants to give a notice to himself / herself. The worker determines whether to respond to the notification in consideration of a situation where the work can be interrupted or a situation where it is not dangerous to respond to the notification. If it is possible to respond, the operator is required to turn his face or look at the manager. Thus, since it can respond only by the movement of the face, it can respond easily to the notification while continuing the work without resting hands.

  In block B136, the CPU 140 detects the orientation of the operator's face based on the output of the motion sensor 154, and in block B138, determines whether the operator's face is facing the administrator. If so, a user interface for initiating a call is projected onto the screen 16 at block B146.

  If it is determined in block B138 that the operator's face is not facing the manager's position, the CPU 140 detects the direction of the operator's line of sight based on the output of the line-of-sight detection sensor 156 in block B142, and the block In B144, it is determined whether or not the operator's line of sight faces the manager. If so, a user interface for initiating a call is projected onto the screen 16 at block B146.

  In this way, the call start screen is not suddenly displayed on the screen 16 of the wearable terminal of the worker who is performing the work, but before that, the worker is notified by a sound that a call request has been made. The call start screen is displayed on the screen 16 only when it is determined that the call can be started and a response operation is performed. For this reason, the worker's view is not suddenly lost, and the worker is not exposed to danger or the work is not interrupted. Note that the operator's response operation to the notification sound may depend on the orientation of the face and line of sight regardless of the orientation of the face or line of sight shown in FIG. Shake the face “vertically / horizontally”, input “yes / no” from the microphone 148, touch / tap the touch pad 55 once (high) / 2 times (no), one finger (high) / Touching with two fingers (no good), finger gesture operation, etc. may be used.

  In block B148, the CPU 140 determines whether or not the operator has performed an operation for starting a call on the user interface projected on the screen 16. When it is determined that a call start operation has been performed, the call is started. Note that the determination in block B148 may be omitted, and the call may be started immediately when the user interface is displayed.

  Although not shown in the flowchart, in the iterative processing from when the notification sound is generated until it responds, the orientation of the wearable terminal is sequentially detected, and the position in the space where the notification sound is reproduced as stereophonic sound is periodically determined. The notification sound may be repeatedly generated.

  We explained that the sound field position is calculated directly based on the positional relationship between the callers, but in addition to this, a method for associating the caller's position information with the sound field position in units of areas, The position of the sound field may be determined based on the setting information, and it may be possible to determine which area or from whom the notification is sent.

  In addition to a call, in order to search for a person or device that meets certain conditions, the information search unit 120 of the communication server 106 can be inquired, and the search result candidate can be presented to the worker as the position of the sound.

  In addition to the method performed by the wearer, the method for inquiring to the information search unit 120 is detected by the position / state management unit 122 of the communication server 106 when the device 115 is in an abnormal state. The position information of the workers to be communicated and the workers to be communicated is collected, and the position information of the device 114 in an abnormal state, the position information and the contents of the abnormal status are sent to the schedule management unit 126 and the position / state management unit 122. On the other hand, the result of searching for candidate workers that satisfy a certain condition can be presented to the worker as the position of the sound.

  Further, a candidate who satisfies a certain condition may be inquired about the work status from the communication server, and the result may be collected and presented to a person who should convey the information.

  According to the second embodiment, when a caller wears a wearable terminal and uses both hands to communicate with a worker who is performing a dangerous work, information is not displayed on the display on the wearable terminal suddenly. In addition, the sound field based on the positional relationship with the caller and the setting information is determined, and notification is made using three-dimensional sound. In this way, the display function of the wearable terminal can be used safely because it does not suddenly take away the field of view of a worker who may be performing dangerous work.

  In addition, although the sound was demonstrated as a prior notification method, it is not restricted to this, You may notify by the image of a screen, if it is a grade which does not block a visual field. For example, it is possible to change the color of a part of the projected image to be displayed, to partially change the color, or to blink. The glasses may be vibrated by the vibrator 68. Furthermore, the prior notification is not limited to one type, and the type of sound, tone color, melody, etc. may be changed according to the degree of urgency.

  The glasses-type wearable terminal has been described as an embodiment, but other head-mounted types such as a goggle type and a helmet type may be used, and the present invention is applicable to a wristband type and a pendant type. For example, in the case of a helmet type or goggles type, the projection device 12 and the camera 59 can be attached to a helmet or goggles, and a normal glasses user can also be used. Furthermore, if the helmet type is used, the speaker 54 can be attached to the inside of the helmet, so that a clearer sound can be heard, the microphone can be attached to the helmet, and the position can be adjusted. improves.

  The present invention can also be applied to wearable terminals other than the head-mounted type.

  Furthermore, the present invention can be applied to a small and light portable device that is not wearable and always with the user, such as a notebook computer, a tablet computer, and a smartphone.

  The function sharing between the wearable terminal and the communication server is not limited as described above, and part of what is described as the function of the wearable terminal may be realized as the function of the communication server, or described as the function of the communication server. A part of the device may be realized as a wearable terminal function.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  DESCRIPTION OF SYMBOLS 10 ... Wearable terminal, 12 ... Projection apparatus, 16 ... Screen, 54 ... Speaker, 55 ... Patch pad, 59 ... Camera, 62 ... Optical sensor, 106 ... Communication server, 114 ... Camera, 148 ... Microphone, 152 ... Wireless communication device 202: Administrator terminal.

Claims (16)

A wearable terminal that can be worn on the head,
First detection means for outputting first information relating to movement of the wearable terminal;
Second detection means for outputting second information relating to the position of the wearable terminal;
Identifying means for identifying a wearer of the wearable terminal based on the first information output from the first detection means and the second information output from the second detection means;
A display screen located on the line of sight of the wearer;
Comprising
The identification unit is a wearable terminal that displays information on the wearer candidate on the display screen and identifies the wearer based on a user operation . The user operation is at least one of determination of a motion detected by the first detection unit, operation of a touch sensor or a switch provided in the wearable terminal, and recognition of an input of a microphone provided in the wearable terminal. wearable device according to claim 1, further comprising a. The specifying unit is configured to identify the wearer based on the first information output from the first detection unit, the second information output from the second detection unit, and an action schedule of a non-wearer of the wearable terminal. claim 1 or claim 2 Symbol placement of wearable device to identify. The identification unit obtains a correlation between information about movement when attaching the wearable terminal of a plurality of people or information about movement when removing the wearable terminal, and the first information output from the first detection unit. claim 2 or claim 3 Symbol placement of wearable terminals. Said first detecting means and the second detecting means, the three-axis acceleration sensor, 3-axis gyro, 3-axis geomagnetic sensor, according to claim 1, comprising a microphone, at least one of a camera, according to claim 2, claim 3 or claim 4. Symbol placement of the wearable terminal. Detecting the movement of a wearable terminal that can be worn on the head;
Detecting the position of the wearable terminal;
Identifying a wearer of the wearable terminal based on the detected movement and the position;
Comprising
Identifying the wearer is
Displaying information on the wearer candidate on a display screen provided on the wearer's line of sight in the wearable terminal;
Identifying the wearer based on a user operation;
A method comprising: The user operation includes at least one of determination of movement of the wearable terminal, operation of a touch sensor or a switch provided in the wearable terminal, and recognition of input of a microphone provided in the wearable terminal. 6. The method according to 6 . Wherein identifying the wearer, and said detected motion, and said detected positions, comprising at identifying the wearer based on the non-wearer behavior schedule of the wearable device according to claim 6 or 7 Symbol mounting method claim. The identification of the wearer is performed by obtaining a correlation between information on movement when attaching the wearable terminal of a plurality of persons or information on movement when removing the wearable terminal and information on the detected movement. 7 or claim 8 Symbol mounting method. Detecting the movement of the wearable terminal and detecting the position of the wearable terminal,
The wearable 3-axis acceleration sensor provided in the terminal, the triaxial gyro, 3-axis geomagnetic sensor, a microphone, claim 6 using at least one of a camera, according to claim 7, claim 8 or claim 9 Symbol mounting method. A wearable device,
First detection means for outputting first information relating to movement of the wearable terminal;
Second detection means for outputting second information relating to the position of the wearable terminal;
Identifying means for identifying a wearer of the wearable terminal based on the first information output from the first detection means and the second information output from the second detection means;
The specifying unit displays information on the wearer candidate on the wearable terminal, and specifies the wearer based on a user operation . The user operation includes at least one of determination of the movement detected by the first detection unit, operation of a touch sensor or a switch provided in the wearable terminal, and recognition of an input of a microphone provided in the wearable terminal. The system according to claim 11 , further comprising: The specifying unit is configured to identify the wearer based on the first information output from the first detection unit, the second information output from the second detection unit, and an action schedule of a non-wearer of the wearable terminal. claim 11 or claim 12 Symbol mounting system to identify. The specifying means is:
Determining whether the first information output from the first detection means is correlated with information about movement when attaching the wearable terminal or information about movement when removing the wearable terminal;
When it is determined that there is a correlation, a correlation between the first information output from the first detection unit and information regarding movement when attaching a wearable terminal or information regarding movement when removing the wearable terminal for a plurality of non-wearers is obtained. 11. claim 12 or claim 13 Symbol mounting system. The wearable terminal includes at least one of a three-axis acceleration sensor, a three-axis gyro, a three-axis geomagnetic sensor, a microphone, and a camera.
The first detection means detects the movement of the wearable terminal based on the output of at least one of the triaxial acceleration sensor, the triaxial gyro, the triaxial geomagnetic sensor, the microphone, and the camera,
The second detection means detects the position of the wearable terminal based on the output of at least one of the three-axis acceleration sensor, the three-axis gyro, the three-axis geomagnetic sensor, the microphone, and the camera. 12., claim 13 or claim 14 Symbol mounting system. The wearable device is obtained is mounted on the head, according to claim 11 having a display screen located on the wearer's line of sight, according to claim 12, claim 13, claim 14 or claim 15 Symbol mounting system.

Images