JP2023076619A - Wearable terminal, system, and display method - Google Patents

Abstract

To provide wearable terminal, a system, and a display method that can easily adjust a display position of an image.SOLUTION: A projection device includes a lens that projects divergent light emitted from a display to the front of a user, a first user operation unit, and a second user operation unit. The first user operation unit can adjust the projection angle of the projection device through a rotation operation. The user rotates the first user operation unit while visually observing a virtual image to adjust the projection angle, and thereby can adjust a display position of the virtual image according to the shape or the size of the head of the user. The virtual image is changed according to a change in the direction of the face of the user detected by a motion sensor. The user can adjust the luminance or the hue of the virtual image through an operation other than the rotation operation of the second user operation unit.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to wearable terminals, systems, and display methods.

2. Description of the Related Art In a manufacturing site or a manufacturing factory having a large number of manufacturing apparatuses, the operating rate of the manufacturing apparatuses is a major factor that affects the production volume of products. Failure of manufacturing equipment that could have been avoided due to inadequate regular maintenance and inspections, or failure to respond to sudden failures of manufacturing equipment, resulting in prolonged inoperability of manufacturing equipment. leads to lower operating rates and lower product yields. Therefore, it is required to shorten the stop time of the manufacturing equipment as much as possible. Since maintenance, inspection, and repair work differs for each manufacturing apparatus, workers sometimes refer to manuals or checklists (hereinafter collectively referred to as checklists) describing work procedures at each stage of work. 2. Description of the Related Art In recent years, wearable terminals have been widely used at manufacturing sites. For example, a worker wears glasses-type wearable terminals at a manufacturing site, and a checklist is displayed on the lens surface. As a result, the worker does not need to refer to the paper checklist during work, and can efficiently perform even unfamiliar or complicated work without resting the work.

However, since it is necessary to check the checklist in order to grasp the certainty of the work performed at each stage, although the reference is made electronically on the screen, a paper checklist is still prepared and the work at each stage is completed. Then, the worker stops working and writes. For this reason, the manufacturing equipment must be stopped, leading to a decrease in production volume. Furthermore, after returning to the office, the workers create work reports based on this checklist. Creating this work report is troublesome for the worker.

A system has been developed that uses a camera-equipped head-mounted display to assist operators in their work. One example is a medical equipment management system that assists operators in handling used and contaminated medical equipment such as endoscopes and tools such as scalpels and forceps.

This system uses a head-mounted camera that captures the field of view of an operator handling medical equipment or instruments, and an image of the operator's field of vision that is captured during exemplary work when handling medical equipment or instruments as standard images. A first determination means for comparing an image taken by a camera with a standard image read from the storage means, and determining whether or not a predetermined work is normally performed based on the degree of similarity between the two images. , information output means for outputting information indicating a warning or instruction based on the determination result of the determining means, and output means for issuing a warning or instruction to the operator based on the information indicating the warning or instruction.

JP 2009-279193 A

This system automatically recognizes the operator's work by comparing the contents of the operator's work handling the medical equipment photographed by a camera with standard images prepared in advance. However, in this method, simple pattern matching between images results in low recognition accuracy, so complex image processing such as extraction of characteristic portions from images is required. Therefore, the automatic recognition process becomes highly complicated and takes a long time.

An object of the present invention is to provide a wearable terminal, system, and display method that can easily adjust the display position of an image.

According to an embodiment, a wearable terminal includes a display that displays a display image; a projection device that projects the display image in front of a user to form a virtual image corresponding to the display image in front of the user; a motion sensor capable of detecting a movement of the user's head and determining a change in face orientation. The projection device includes a lens that projects divergent light emitted from the display in front of the user, a first user operation section, and a second user operation section. The first user operation unit can adjust the projection angle of the projection device by rotating. The user can adjust the display position of the virtual image according to the shape or size of the user's head by adjusting the projection angle by rotating the first user operation unit while viewing the virtual image. . The virtual image is changed according to a change in orientation of the user's face detected by the motion sensor. The user can adjust the brightness or color tone of the virtual image by an operation other than the rotating operation of the second user operation unit.

It is a perspective view which shows an example of the wearable terminal of embodiment. It is a figure which shows the front view and cross-sectional structure which show an example of a wearable terminal. It is a figure which shows an example of the position detection of a wearable terminal. It is a figure which shows the principle of the position detection of a wearable terminal. FIG. 10 is a diagram showing signal waveforms when detecting the position of the wearable terminal; It is a figure which shows an example of the system containing a wearable terminal and an information management server. 2 is a block diagram showing an electrical configuration of a wearable terminal; FIG. FIG. 4 is a diagram showing an example of a sensor that detects the state of the device; It is a figure which shows an example of the sensor which detects a user's action. 1 is a schematic diagram showing an example of a usage environment of a system; FIG. 1 is an exploded perspective view showing the structure of an example of a sensor used in the system for detecting user behavior; FIG. FIG. 11 is an exploded perspective view showing the structure of another example of a sensor for detecting user behavior used in the system; It is a figure which shows an example of the work procedure displayed by a system, and the work record produced by a system.

Embodiments will be described below with reference to the drawings.

Wearable terminals include head-mounted types (which include glasses type, goggle type, helmet type, etc., but these are sometimes collectively referred to as glasses type), wristband types, pendant types, and the like. , here, an embodiment of a glasses-type wearable terminal will be described. There are two types of glasses-type wearable terminals: a type that allows you to see the scenery in front of your line of sight through transparent lenses, and a type called a head-mounted display that blocks your view and prevents you from seeing the scenery. describes the type that is visible.

1 is a perspective view of a glasses-type wearable terminal (hereinafter simply referred to as a wearable terminal) 10, FIG. 2(a) is a front view, and FIG. 2(b) is a view showing a cross-sectional structure seen from above.

The wearable terminal 10 has almost the same shape as ordinary glasses, but here a projection device 12 is attached to the outside of the temple on the right eye side. Glasses 14, 16 are fitted in the frame. The left eye glass 14 is a normal clear glass so that the user can see the scenery. At least a part of the glass 16 on the right eye side serves as a screen 16 . The screen 16 allows the user to view the image being projected by the projection device 12 . The screen 16 is transparent when the projection device 12 is not projecting an image, allowing the user to see the scenery through the glass (screen) 16 on the right eye side.

The projection device 12 includes a power supply section 22 and a control section 24 as electronic components. The power supply unit 22 can include a button type battery, a rechargeable battery, a secondary battery capable of non-contact power supply, and the like. Alternatively, power may be supplied to the projection device 12 from an external power supply through a power supply line without a built-in power supply. The control unit 24 communicates with a server and other electronic devices via a network, which will be described later, to transmit and receive information. This communication may be either wired or wireless. In the case of wireless, short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark), UWB, etc., medium-range wireless communication such as WiFi (registered trademark), 3G/4G, WiMAX (registered trademark), depending on the usage environment Any of the long-range wireless communications such as

The projection device 12 further includes a light source 28, a display section 30, a prism 32, a lens group 34, etc. as optical components. The light source 28 may be a dimming type white LED light source including a plurality of, for example, three LEDs (Light Emitting Diodes) whose emission colors are different from each other and whose output light amounts can be independently changed. According to the dimmable white LED light source, even when the use environment of the wearable terminal 10 is, for example, in a clean room where orange-based lighting is often used, the emission color can be changed according to the use environment. and a clear projected image can be obtained. Furthermore, according to the dimmable white LED light source, it is possible to output a display color that is easy for the user to see. It is possible to avoid the occurrence of factors that hinder users of

The display unit 30 is, for example, a reflective LCD (Liquid Crystal Display) module, and based on display control by the control unit 24, displays predetermined text, images, etc. (hereinafter, what the display unit 30 displays is called a display image. may be collectively referred to). Non-parallel light (divergent light, hereinafter also referred to as divergent light) emitted from the light source 28 is reflected by the half mirror surface 32 a of the prism 32 to illuminate the display image on the display section 30 . Reflected light from the display unit 30 passes through the half-mirror surface 32a as light corresponding to a display image (sometimes referred to as image light), is emitted from the exit surface 32c, and is projected through the lens group 34 to a predetermined size. It is projected onto the screen 16 as an image.

The screen 16 has a front side transparent refractor 42 , a Fresnel lens type half mirror surface 44 and a back side transparent refractor 46 . Part of the image light reaching the Fresnel lens half mirror surface 44 is reflected by the Fresnel lens half mirror surface 44 to form a virtual image (projection image) corresponding to the display image on the display unit 30 several meters ahead. The screen 16 can also partially transmit the scenery ahead of the line of sight of the user wearing the wearable terminal 10, and the screen 16 is configured to display the scenery visible to the user together with the projected image. good too.

A part of the image light (divergent light) emitted from the light source 28 and passed through the half mirror surface 32a is totally reflected by the total reflection surface 32b and refracted by the output surface 32c to be divergent light leaked from the light source 28. 50. Leaked light 50 is emitted in a direction different from the screen 16 through an opening or gap (guide portion) 52 formed in the front of the projection device 12 .

The wearable terminal 10 has a speaker 54A, an earphone jack 54B, a microphone jack 56, a slide switch 57, a rotary switch 58, etc. at predetermined positions of the projection device 12, for example, on the bottom surface. A hands-free microphone (not shown) is connected to the microphone jack 56 to collect the voice of the user. The slide switch 57 can adjust the brightness, color tone, etc. of the projection image of the projection device 12, for example. The rotary switch 58 can adjust, for example, the projection angle of the projected image. By enabling different adjustment amounts to be set by different operations such as the slide switch 57 and the rotary switch 58, the user can adjust the projected image by blind touch while viewing the projected image. For example, by operating the slide switch 57, it is possible to provide a projected image with display brightness and color tone that suit the user's preference. By operating the rotary switch 58, the projection angle can be adjusted so that the image is displayed at the optimum position according to the shape and size of the user's head. It goes without saying that the objects to be adjusted by the slide switch 57 and the rotary switch 58 may be reversed. You may assign to two types of operation of one operation member.

Selection by these switches 57 and 58 may be made by trial and error while viewing only the projected image. may be adjusted. The menu screen is projected on the screen 16 by displaying the menu screen on the display unit 30 .

Further, the selection of menu items may be made by touch operation instead of operation of switches 57 and 58 . Therefore, a touch pad 55 is also provided on the outside of the projection device 12 . By displaying a menu or the like on the display unit 30 and touching a position within the touch pad 55 corresponding to the display position of an item in the menu, user operation can be input simply and efficiently.

A camera 59 is provided outside the front central portion, and can capture an image (either a still image or a moving image) ahead of the user's line of sight. Although not shown, a camera is provided facing the user's face inside the center of the front (position corresponding to the arrangement position of the camera 59) to photograph the user's eyeballs and detect the user's iris. good too. The iris can be used for user authentication.

By using the leakage light 50 from the wearable terminal 10, the state of the wearable terminal 10, that is, the state of the user can be detected. The principle of detecting the state of the wearable terminal will be described with reference to FIGS. 3, 4, and 5. FIG. Here, the state includes position, movement of position, and the like.

FIG. 3 shows an example of usage of a wearable terminal. For example, an arbitrary number of work spaces or product shelves A01 to Axy (both x and y are positive integers), B01 to Bxy in a work area 60 such as a parts yard of a factory, a product warehouse of a mail-order company, or a delivery department of a retailer. , C01 to Cxy are arranged. A work space or shelf may be, for example, a work table in a factory, manufacturing equipment in a production line, a desk in a school, a seating position in a conference room, or the like.

At least one optical sensor 62-1 to 62-n (n is a positive integer) is arranged in the work area 60. FIG. The optical sensors 62-1 to 62-n detect the position (x, y, z) of the wearable terminals 10-1 to 10-m (m is a positive integer), the number, the change in position (movement), the change in orientation, etc. can be individually detected by the detection method shown in FIGS. By detecting the position, number, movement, change in direction, etc. of the wearable terminals 10-1 to 10-m. It is possible to recognize the positions, movements, and other states of any number of users wearing the wearable terminals 10-1 to 10-m.

The user can freely move within the work area 60 . A user performs a predetermined task in a predetermined position, such as a work space 64 such as a station (cart), a corresponding container, or a movable table. Note that the work space 64 is not movable, and may be a fixed desk or a sitting position.

As shown in FIGS. 3 and 4 , the sensing system includes one or more wearable terminals 10 and one or more optical sensors 62 . The optical sensor 62 has a function of detecting the leakage light 50 and a communication function of transmitting the detection result to a server or the like. This communication function, like the communication function of the wearable terminal 10, may be wired or wireless. In the case of wireless communication, any of short-range wireless communication such as Bluetooth, ZigBee and UWB, medium-range wireless communication such as WiFi, and long-distance wireless communication such as 3G/4G and WiMAX may be used depending on the usage environment. Although the embodiments described below have various units and modules with communication capabilities, the communication capabilities of these units and modules may likewise be either wired or wireless. In the case of wireless communication, any of short-range wireless communication such as Bluetooth, ZigBee and UWB, medium-range wireless communication such as WiFi, and long-distance wireless communication such as 3G/4G and WiMAX may be used depending on the usage environment.

In order to identify the wearable terminal 10 from the leaked light 50 received by the optical sensor 62, the wearable terminal 10 detects the leaked light 50 using information including terminal identification information (identification, hereinafter sometimes referred to as a terminal ID). Modulate intermittently. A typical example of the modulation method is a chopper type modulation method that reduces the amount of light emission to zero. Here, a modulation method that can secure a predetermined amount or more of light emission even when the amount of light emission is low is adopted. This reduces the burden on the user's eyes. As a modulation method, for example, DSV (Digital
Sum Value)-free modulation method (i.e., a modulation method that always calculates the DSV of the modulated signal and allows the bit-reversal code to be inserted as appropriate to set the DC component to zero) is adopted, the light emission amount in a relatively long range The change is suppressed, and macroscopically, the change in the light emission amount can always be zero, further reducing the burden on the user's eyes. Since the human eye can perceive a change of about 0.02 seconds, setting the reference frequency of the modulation to 10 Hz or more, for example 20 Hz or more, more preferably 60 Hz or more has the effect of reducing the burden on the user's eyes. is also born. On the other hand, since the LED used as the light source 28 has an internal impedance and a connection capacitance, it is desirable that the modulation frequency with good precision is less than 100 MHz, preferably 10 MHz or less. Therefore, the modulation frequency of the light source 28 used in the detection system of the embodiment is preferably in the range of 10 Hz to 100 MHz, preferably 10 Hz to 10 MHz.

Since leaked light 50 of divergent light from light source 28 is used, the amount of light detected by optical sensor 62 changes according to the distance between wearable terminal 10 and optical sensor 62 . Using this phenomenon, the distance between the wearable terminal 10 and the optical sensor 62 or the orientation of the wearable terminal 10 with respect to the optical sensor 62 can be obtained. Since the position (including the height) of optical sensor 62 is fixed, the position (x, y, z) of wearable terminal 10 can be detected if the distance between optical sensor 62 and wearable terminal 10 is known.

Furthermore, since the leaked light 50 of divergent light from the light source 28 is used, the leaked light 50 can be detected in a relatively wide range. As a result, by installing a relatively small number of optical sensors 62-1 to 62-n, the positions of wearable terminals 10-1 to 10-m in work area 60, the distances between wearable terminal 10 and optical sensors 62, The directions of the wearable terminals 10-1 to 10-m or the orientation of the wearable terminal 10 with respect to the optical sensor 62 can be detected. This reduces the equipment costs required to install the detection system.

Light amount information of the leaked light 50 detected by the optical sensor 62 is transmitted from the optical sensor 62 to a server described later at a predetermined timing. The server analyzes the collected information from the light sensor 62 . As a result, the positions and states of arbitrary wearable terminals 10-1 to 10-m, ie, users, can be detected.

FIG. 4 is a schematic diagram illustrating a specific usage example of the system for recognizing wearable terminals according to the embodiment. Assume that there are three users wearing wearable terminals 10-1 to 10-3 around four optical sensors 62-1 to 60-4. Leaked light 50 from wearable terminals 10-1 and 10-2 is detected by optical sensors 62-1 to 60-4. The optical sensors 62-1 to 60-4 convert the light amount of the leaked light 50 detected by each of them into AD (Analog-Digital), and output light amount information corresponding to the light amount at a predetermined timing, for example, short-range wireless communication. to the server.

While wearable terminal 10-1 moves to optical sensor 62-1 in accordance with the movement of the user, the direction of wearable terminal 10-2 is temporarily changed in response to arbitrary movements of the user, for example, swinging of the head (turning of the head). Suppose that it changed to FIG. 5 shows changes in the detected information at this time.

FIG. 5 shows an example in which an intermittent time-varying method is used as the modulation method for the leaked light 50 of each of the wearable terminals 10-1 to 10-3. That is, the ID modulation periods are shifted in each of wearable terminals 10-1 to 10-3.

As shown in FIGS. 5(a), (b), and (c), for the first to third wearable terminals 10-1 to 10-3, the ID modulation period of the wearable terminal is intermittently set, and the other is a non-modulation period. Within each ID modulation period, the synchronization signal SYNC and the terminal IDs of the wearable terminals 10-1 to 10-3 constitute one set (one-to-one correspondence), and the set is repeated multiple times (as shown in FIG. 5). (multiple of 4 times if there are 4 sensors in ).

The ID modulation period of the second wearable terminal 10-2 starts at the same time when the first wearable terminal 10-1 enters the non-modulation period. Similarly, when the second wearable terminal 10-2 enters the non-modulation period, the ID modulation period of the third wearable terminal 10-3 starts.

During the ID modulation period of the second wearable terminal 10-2 and the ID modulation period of the third wearable terminal 10-3, the synchronization signal SYNC and the terminal IDs of the wearable terminals 10-2 and 10-3 are repeatedly modulated. be done. Thus, the terminal ID can be detected by including the terminal ID of the wearable terminal 10 in the modulated signal.

In the above example, the modulation timing of each of wearable terminals 10-1 to 10-3 is time-divided (intermittent). However, for example, all wearable terminals 10-1 to 10-3 may be continuously modulated, and the modulation reference frequency of each of wearable terminals 10-1 to 10-3 may be changed. Moreover, each frequency spectrum characteristic at the time of spread spectrum may be changed.

As shown in FIGS. 5(d), (e), (f), and (g), the information communication period from the optical sensors 62-1 to 62-4 is finely divided in each ID modulation period.

As shown in FIG. 4, at an early stage, part of leaked light from wearable terminal 10-1 reaches optical sensor 62-4. Therefore, at the initial time, as shown in FIG. 5(k), the optical sensor 62-4 detects leaked light from the wearable terminal 10-1. However, as the wearable terminal 10-1 moves toward the optical sensor 62-1, the modulated signal amplitude of the leaked light from the wearable terminal 10-1 detected by the optical sensor 62-4 decreases. On the other hand, as shown in FIG. 5(h), the modulated signal amplitude of the leaked light from the wearable terminal 10-1 detected by the optical sensor 62-1 increases over time. In this way, by comparing the temporal changes in the modulated signal amplitudes detected by the optical sensors 62-1 to 62-n, the temporal changes in the position (moving state) of the wearable terminals 10-1 to 10-m to be detected can be determined. can be detected.

On the other hand, since wearable terminal 10-2 faces optical sensor 62-3 at the initial time point, the modulated signal amplitude obtained from leaked light is is larger. After that, for example, the second user shakes his head and temporarily faces the optical sensor 62-2. Then, the detection output of the wearable terminal 10-2 output by the optical sensor 62-2 temporarily increases and then decreases, as shown in FIG. 5(i). On the other hand, the detection output of the wearable terminal 10-2 output by the optical sensor 62-3 temporarily decreases and then increases, as shown in FIG. 5(j).

In this way, by comparing temporal changes in modulated signal amplitude detected by the optical sensor 62, temporal changes in orientation of the wearable terminals 10-1 to 10-m to be detected can also be estimated.

The above detection example is an example of a case where the user's motion is movement or head swing. However, it is not limited to this, and various other behaviors of the user may be used. For example, leaked light may be temporarily blocked by movement of the user's hand, twisting of the upper body (body), or the like. In this case, all the optical sensors 62-1 to 60-4 commonly experience a temporary reduction in modulation signal amplitude during the same time period. In this way, by comparing the relevance of changes in modulated signal amplitudes of all optical sensors 62-1 to 60-4, different user behavior patterns can be identified.

By using the above method, it is possible to recognize not only the behavior of the user but also the intention of the user.

A beacon may be used as a method of detecting the position (x, y, z) of the wearable terminal 10 . In the above example, light modulated with terminal identification information is emitted from multiple wearable terminals 10 to multiple optical sensors 60, and the information received by multiple optical sensors 60 is compared and processed to determine the position of wearable terminal 10, Although the state is detected, a large number of location information transmitters are placed in the work area 60, and beacons corresponding to the placement positions are transmitted from the transmitters, for example, by short-range wireless communication such as RF-ID with a range of several meters. Then, the wearable terminal 10 that received it can be considered to be at approximately the same position as the transmitter. Furthermore, GPS can be used to detect the position of the wearable terminal. Position detection does not need to be based on only one type of method, and multiple methods can be used together to improve detection accuracy.

FIG. 6 shows an example of an entire system using wearable terminals. Here, a system constructed at a manufacturing site of a manufacturing plant will be described. Multiple wearable terminals 10, multiple optical sensors 62 shown in FIG. 3, one or multiple administrator terminals 104, multiple manufacturing apparatuses 106, one or multiple cameras 114, and an information management server 116 are networked. 102. The network 102 may be, for example, a unit such as a factory building, a department, a building floor, or a business office of a company, or may be a network for each factory, building, or company, or may be the Internet. If there are multiple manufacturing sites in a factory, the network for each site in FIG. 6 may constitute a LAN, and multiple LANs may be connected to the network for the entire factory. Network 102 may be a wireless network or a wired network.

Although there are many workers at the manufacturing site, not all workers need to wear the wearable terminal 10 . Therefore, it is not necessary to prepare wearable terminals 10 for all workers, and only a predetermined number of wearable terminals 10 are prepared, and necessary workers wear available shared wearable terminals. The system needs to identify the user when the user wears the wearable terminal. The reason for this is to display the work procedure of a particular manufacturing apparatus to a user who uses the manufacturing apparatus, and to create a work report based on the behavior of the user. There are various specific methods, but when the wearable terminal 10 is attached or detached, the user may input his or her own ID and terminal ID from a terminal (not shown). Input is not limited to key input, voice input from a microphone, or scan input with a bar code may be used. Furthermore, since the behavior of the user when wearing the terminal is likely to be a habit, the behavior of the user at this time may be detected and estimated. The feature quantity indicating the behavior of the user can be obtained from the acceleration and angular velocity of the wearable terminal 10, the movements of the user's face, hands and fingers, and the environmental sounds collected by the microphone. For example, when the wearable terminal 10 is put on or removed from the face, the wearable state of the wearable terminal can be known based on the frictional sound between the temples and the skin or hair.

At least one manager who supervises workers is placed at the manufacturing site, and the manager uses a manager terminal 104 . The administrator terminal 104 may have the same configuration as the wearable terminal 10, but since the administrator does not need to move, the administrator terminal 104 may have the same configuration as a normal personal computer or tablet, and detailed description of the administrator terminal 104 is omitted.

Each manufacturing device 106 is equipped with a device status sensor 108 and a user behavior sensor 110 . These sensors 108 , 110 also have communication capabilities and are connected to the network 102 .

The camera 114 constantly captures moving images of users in the manufacturing site. User behavior can be estimated by analyzing this image. For example, by storing a standard image for each user in advance and comparing an image when the wearable terminal 10 is attached or removed by the user with the standard image, the wearing user can be identified. If it is not possible to arrange a number of cameras capable of covering the entire manufacturing site at the same time, several cameras 114 with variable angles and capable of photographing a wide range of users may be arranged.

The information management server 116 includes a control section 118, a communication section 120, a position management section 122, a user behavior management section 124, an apparatus state management section 126, and the like. The communication function of the sensors 108 and 110, the communication function of the administrator terminal 104, the communication function of the camera 114, and the communication function of the communication unit 120 may be wired or wireless, like the communication function of the wearable terminal 10. . In the case of wireless communication, any of short-range wireless communication such as Bluetooth, ZigBee and UWB, medium-range wireless communication such as WiFi, and long-distance wireless communication such as 3G/4G and WiMAX may be used depending on the usage environment.

The position management unit 122 collects information about the positions of the wearable terminal 10 and the administrator terminal 104 based on the outputs of the optical sensor 62 and various sensors of the wearable terminal 10 and the administrator terminal 104 at regular time intervals. Further, the location management unit 122 identifies the user of the wearable terminal 10 or administrator terminal 104 and manages the terminal ID, user ID, and location of the wearable terminal 10 or administrator terminal 104 .

The user behavior management unit 124 periodically collects information on the behavior and state of the user of the wearable terminal 10 based on the outputs of the optical sensor 62, various sensors of the wearable terminal 10, and the user behavior sensor 110 of the manufacturing apparatus 106, It manages the terminal ID, user ID, and action/state of the wearable terminal 10 . The equipment status management unit 126 periodically collects and manages information about the status of the manufacturing equipment based on the output of the equipment status sensor 108 of the manufacturing equipment 106 . When the state of the device changes, the device state sensor 108 may notify the device state management unit 126 and collect information about the state of the manufacturing device.

When the device status management unit 126 detects an abnormality in the device, the information management server 116 notifies the administrator terminal 104 of the location information and status of the corresponding manufacturing device. At the same time, it judges the status of the worker, extracts a worker candidate who can most efficiently deal with the device in which the abnormality has occurred, and presents it to the manager terminal 104 .

The outline of this embodiment is to automatically generate a task checklist and present it to the user, and automatically check off the corresponding items in the checklist as the user performs the task. Therefore, information obtained from a plurality of sensors 108 and 110 connected to the network 102 or various sensors of the terminals 10 and 104 is integrated and processed by the information management server 116, and the actions of individual workers are automatically determined. Guess/Recognize. Then, the information management server 116 generates a work procedure (checklist) based on the result, and supports automatic input (automatic entry) to corresponding parts in the checklist. The information management server 116 automatically generates a work report when automatic input (automatic entry) is completed up to the last item in the work checklist.

The contents of the work checklist differ for each manufacturing apparatus to be maintained. Furthermore, the content of the work checklist differs depending on the location of the defect within the manufacturing apparatus. Therefore, the information management server 116 collects manufacturing equipment-related information obtained from the equipment status sensors 108 related to the manufacturing equipment requiring maintenance, and automatically estimates/recognizes the defect occurrence location of the manufacturing equipment. The information management server 116 automatically identifies the wearable terminal 10 worn by the worker performing maintenance, and displays the maintenance content on the terminal 10 in the form of a work checklist.

FIG. 7 shows an example of the electrical configuration of the wearable terminal 10. As shown in FIG. The wearable terminal 10 includes a CPU 140, a system controller 142, a main memory 144, a storage device 146, a microphone 148, a speaker 54, a projection processing unit 150 (which controls the light source 28 and the display unit 30), a camera 59, a wireless communication device 152, motion It includes a sensor 154, a line-of-sight detection sensor 156, a gesture sensor 158, a touch pad 55, a vibrator 68, a position information receiver 159, a GPS module 155, and the like.

The CPU 140 is a processor that controls the operation of various modules in the wearable terminal 10, and executes computer programs that are loaded into the main memory 144 from a storage device 146 that is a non-volatile semiconductor memory such as an SSD or flash array. These programs include an operating system (OS) and various application programs. The CPU 140 executes various application programs and communicates with the information management server 116 via the network 102 using the wireless communication device 152 to perform, for example, the following processes. For example, the CPU 140 inputs voice using the microphone 148 and sends voice data to the information management server 116 , takes an image using the camera 59 and sends image data to the information management server 116 , uses the motion sensor 154 , line-of-sight detection sensor 156, gesture sensor 158, touch pad 55, and position information receiver 159 to send input data to information management server 116, or output sound from stereo earphones or speakers 54 (not shown) connected to earphone jack 54B. Various controls such as ringing and vibrating the vibrator 68 are performed. Although the speaker 54 is assumed to be a monaural speaker, if a stereo speaker is required, a speaker may also be provided on the temple on the left eye side, though not shown in FIGS.

The system controller 142 is a device that connects between the local bus of the CPU 140 and various components. A microphone 148 is connected to the microphone jack 56 and collects user-generated speech or environmental sounds. By recognizing the voice uttered by the user or analyzing the environmental sounds, the behavior of the user can be estimated and the user can be identified. For example, by pre-storing a standard voice for each user and comparing the voice produced by the wearer with the standard voice, the wearer can be identified. Also, by analyzing the environmental sound, it is possible to identify the work place where the wearer is located. A speaker 54 outputs an alarm or the like to attract the user's attention. The projection processing unit 150 outputs an image signal to the display unit 30 and lights the light source 28 to project the image of the display unit 30 onto the screen 16 . This image can include not only still images but also moving images. The wireless communication device 152 has, for example, a wireless LAN function, and wirelessly connects the wearable terminal 10 and the access point 112 .

The motion sensor 154 is a sensor that integrates a 3-axis acceleration sensor, a 3-axis gyro sensor, and a 3-axis geomagnetic sensor, and detects the movement of the head of the user using the wearable terminal 10, and as a result, determines the direction in which the face is facing. do. The state of the worker may be detected using a microphone 148, a barometer, or the like. The state of the worker includes not only moving and resting, but also contents of work, progress of work, and the like. Using motion sensor 154, altitude by air pressure, etc., it is determined whether or not the feature amount obtained from these detection results matches the feature amount of each process obtained in advance from the worker or the like. can determine which steps are being performed or completed. In addition, even if it is determined whether or not the feature amount of the environmental sound input from the microphone 148 matches the feature amount of the environmental sound characteristic to each work process acquired in advance, which process of the plurality of processes is executed. It can be determined whether it is running or completed.

The line-of-sight detection sensor 156 is provided facing the user's face inside the center of the frame of the eyeglasses, photographs the user's eyeballs, and detects movement of the line of sight. Furthermore, the line-of-sight detection sensor 156 may be capable of detecting the user's iris. The gesture sensor 158 is a sensor that discriminates a gesture by moving a finger. Specifically, a sensor that discriminates a user's gesture by analyzing finger movements on the touch pad 55 provided on the projection device 12 and hand and finger movements in an image captured by the camera 59 . be. The vibrator 68 vibrates the temple of the wearable terminal 10 by vibrating the projection device 12, and conveys some information to the user. The location information receiver 159 receives beacons including location information transmitted from a plurality of location information transmitters 113 arranged within the area of the LAN 102 using short-range wireless communication such as RF-ID. Since it is short-range wireless communication, it can be assumed that the positions of the transmitter and the receiver (wearable terminal) are almost the same. GPS module 155 detects the position (x, y, z) of wearable terminal 10 . By synthesizing this detection result, the detection result of the position information receiver 159, and the detection result of the optical sensor 62 in FIG. 3, the user's position and its change can be detected more accurately.

The display unit 30 displays instructions from the manager terminal 104 and the information management server 116, incoming calls, and the worker's work status detected by the motion sensor 154 or the like. This display image is displayed on the screen 16 by the projection processing unit 150 .

A voice call with the outside is possible using the microphone 148 and the speaker 54 .

The administrator terminal 104 may have the same configuration as the wearable terminal 10, or may be a normal personal computer or tablet. The electrical configuration of a normal personal computer or tablet is equivalent to the wearable terminal 10 omitting the projection processing unit 150, camera 59, motion sensor 154, line-of-sight detection sensor 156, gesture sensor 158, and the like. The position of the administrator terminal 104 is detected by GPS.

An example of the device state sensor 108 attached to the manufacturing device 106 will be described with reference to FIG. FIG. 8(a) shows the mounting position with respect to the device, and FIG. 8(b) shows the configuration of the sensor 108. FIG. Conventionally, whenever a problem occurs in a manufacturing apparatus, a worker inspects and repairs the defective portion of the manufacturing apparatus and investigates the cause of the problem. As a result, the maintenance time of the manufacturing equipment (the non-operating period of the manufacturing equipment) is lengthened, resulting in a decrease in product productivity. In this embodiment, the information management server 116 collects and integrates information related to the state of the equipment obtained from the equipment state sensor 108 connected to the network 102, and automatically estimates or recognizes the defective part of the manufacturing equipment. conduct. As a result, it becomes possible to automatically diagnose defective points in the manufacturing equipment, thereby significantly shortening the maintenance time of the manufacturing equipment (the non-operating period of the manufacturing equipment) and preventing a decrease in product productivity.

The device state sensor 108 is composed of an acceleration sensor 108 a and a wireless communication device 108 b , and an acceleration signal detected by the acceleration sensor 108 a is transmitted to the information management server 116 via the wireless communication device 108 b and network 102 . The machine status sensor 108 is provided with adhesive or fixed parts for easy attachment to existing manufacturing equipment. An adhesive layer may be formed in advance on the adhesion portion, or an adhesive may be applied at the time of adhesion. Alternatively, the equipment state sensor 108 may be attached to the existing manufacturing equipment by screwing the fixing portion to the existing manufacturing equipment.

Automatic diagnosis of a defective portion of a manufacturing apparatus requires automatic collection processing of the operation status of each part of the manufacturing apparatus. Therefore, purchasing a new manufacturing device or buying a new one requires a huge amount of money. However, in this embodiment, it is only necessary to additionally fix sensor terminals, which can be purchased at a very low cost, to each part in the existing manufacturing apparatus. As a result, an environment for automatically diagnosing defective locations can be added at a very low cost while maintaining the existing equipment environment.

As shown in FIG. 8A, the apparatus state sensor 108 is fixed to, for example, a portion of the moving belt 136 or a portion of the movable arm 134 or movable shaft 132 that sandwiches the article. Then, when the place to be moved in the normal state stops, it can be determined that a problem has occurred in the movable portion.

The control unit 118 in the information management server 116 preliminarily stores a maintenance procedure guide, which is a procedure manual for repair, maintenance, and inspection methods for each defect occurrence location for various manufacturing equipment, and based on the above automatic diagnosis result. Generate an optimal work checklist.

The acceleration detection method has been described as an example of the device state sensor 108 in FIG. Not limited to this, any physical quantity or chemical quantity such as the temperature or the amount of conduction current may be used for extracting the defect location. Further, it is also possible to automatically diagnose defective locations in the manufacturing apparatus by comparing images captured by a camera or by comparing environmental sounds collected by a microphone.

When a defective manufacturing apparatus is detected by the method described in FIG. 8, the information management server 116 automatically selects a worker to maintain the manufacturing apparatus, and the worker terminal 103 worn by the worker is selected. display maintenance procedures or derived work checklists on The information management server 116 selects, for example, a worker who (i) is located near the defective manufacturing apparatus, (ii) can interrupt the ongoing work, and (iii) can perform the maintenance work. . According to this, it is possible to minimize the worker's travel time loss.

As a method of most efficiently searching for a worker located near the defective manufacturing apparatus, in this embodiment, as shown in FIG. A wireless communication device 106 b is attached to a portion of the manufacturing equipment 106 . As described with reference to FIG. 5 , leaked light 50 emitted from wearable terminal 10 includes terminal ID information of terminal 10 . Therefore, when the information contained in the leaked light 50 detected by the optical sensor 106a is transmitted to the information management server 116 via the wireless communication device 106b and the network 102, any wearable terminal near the defective manufacturing apparatus 10, that is, the information management server 116 can recognize which worker is located. Based on the information, the information management server 116 selects a worker who will maintain the target manufacturing apparatus, and transmits the work checklist to the wearable terminal 10 of the worker. A work checklist is displayed on the screen 16 of the terminal 10, as shown in FIG. 13(a). For convenience of explanation, the illustration is simplified in FIG. 13(a), but the actual work checklist is as follows.

・Put the luggage in the cart ・Close the valve ・Turn off the ON/OFF switch ・Turn off the first lighting switch ・Turn off the third lighting switch In this way, the leakage light 50 emitted from the wearable terminal 10 is detected in real time. Since /collection/aggregation is performed, it is possible to identify workers who are located near the manufacturing apparatus 106 to be maintained very easily and accurately. As a result, it is possible to save the worker's travel time, shorten the maintenance period, and prevent a decrease in manufacturing efficiency.

Another example of a method of recognizing a worker located near the target manufacturing apparatus is a method of using a camera 114 installed in advance near the manufacturing apparatus 106 . An imaging device 114 a in the camera 114 takes an image of the vicinity of the manufacturing apparatus 106 , and the result is transmitted to the information management server 116 via the wireless communication device 14 b and the network 102 . The information management server 116 may analyze the received images and automatically identify the workers in them.

An example of a worker working according to a work checklist will be described with reference to FIG. When the work checklist as shown in FIG. 13(a) is displayed on the screen 16, the worker starts work. If the entire checklist is displayed on the screen 16 at once and the characters are too small to see, only one or a few steps are displayed, the work progress is automatically recognized sequentially, and when the work of each step is completed, it is displayed accordingly. The checklist may be continuously updated in real time. A worker at the work site in FIG. 10( a ) puts a load 162 in a cart 164 according to the work checklist, closes the valve 170 (or rotates the handle 170 by a specified angle), and turns off the on/off switch 172 . , the first light switch 176 and the third light switch 180 are turned off. In this embodiment, the behavior of the worker is automatically recognized/identified in real time by the user behavior sensor 110 attached to the manufacturing apparatus 106, and the work completion time is automatically added to the work checklist (see FIG. 13(b)). is written to When the final work step is completed, a work report is automatically generated within the information management server 116 and its contents are displayed on the manager's terminal 104 . A work report (FIG. 13(b)) is a work checklist (FIG. 13(a)) with completion times entered therein.

The method of automatic recognition/discrimination of worker activity by the user activity sensor 110 can use any detection technique or combination thereof. For example, an action image of the worker captured by the camera 114 or 59 may be analyzed to automatically recognize/identify the action of the worker. However, when using image analysis captured by the camera 114 for automatic recognition/discrimination of worker behavior, the worker behavior may be hidden behind the image in some situations. Alternatively, voice recognition technology may be used. The input of the specific voice is detected by the microphone 148 by setting the worker to utter a specific voice every time each item of the work checklist (maintenance work procedure) displayed on the wearable terminal 10 is completed. to provide automatic recognition/discrimination of worker activity. Alternatively, the microphone 148 or the built-in microphone in the device state sensor 108 may be used to detect environmental sounds generated when a specific task is performed, thereby automatically recognizing/identifying the worker's actions. In addition, there is also a method for automatically recognizing/identifying worker actions by identifying predetermined worker gestures. As a method for recognizing the gesture of the worker, an action image of the worker captured by the cameras 59 and 114 may be analyzed. The results detected by the optical sensor 106a attached to the manufacturing apparatus 106 may be compared.

A pair of a light-emitting part 166a and a light-receiving part 166b is installed in the opening of the cart 164, and detects light shielding when the cargo 162 passes through the opening of the cart 164, and automatically detects loading and unloading of the cargo 162.例文帳に追加FIG. 10(b) shows the signal characteristics detected by the light-receiving part 166b when a package is put in and taken out of the cart 164. FIG. The vertical axis indicates the amount of light detected by the light receiving section, and the horizontal axis indicates the passage of time. While the cargo 162 passes through the opening of the cart 164, the amount of light detected by the light receiving unit is reduced. The method for detecting the loading and unloading of the luggage 162 with respect to the cart 164 is not limited to the method using light as described above, and any other method can be used.

A detection example for automatically recognizing/identifying in real time "valve closed", "on/off switch off", and "illumination switch off" will be described. In general, in order to perform maintenance (maintenance, inspection, repair) of a manufacturing apparatus, it is necessary for an operator to directly touch a predetermined location in the manufacturing apparatus. Utilizing this feature, this embodiment detects the contact of the worker with a predetermined location in the manufacturing apparatus and reflects it in the automatic recognition/identification of the worker's behavior. This method is very easy to detect and ensures high automatic recognition/identification accuracy. In the example of FIG. 10(a), the valve 170 is fitted with a contact sensor 168, and the on/off switch 172 and the light switch board 174 are also fitted with transparent contact sensors. Light switch board 174 includes first, second and third light switches 176 , 178 , 180 .

A contact sensor, which is an example of the user behavior sensor 110, includes a wireless communication function (for example, short-range wireless communication) and a worker contact status detection function. Any contact detectable element such as a piezoelectric element, a photointerrupter, or an acceleration sensor (gyro sensor) can be used to detect the contact state. This contact sensor can be attached to equipment such as existing manufacturing equipment and is very inexpensive. Therefore, by attaching this contact sensor (user behavior sensor 110) to an existing manufacturing apparatus, a short-range wireless communication network environment can be additionally constructed at a very low cost while maintaining the existing infrastructure.

An example of the user behavior sensor 110 is shown in FIGS. 11 and 12. FIG. FIG. 11 shows the user action sensor 110 attached to the existing infrastructure on/off switch 172 or the user action sensor 110 (contact sensor 168) attached to the valve 170, and FIG. 12 attaches to the existing infrastructure lighting switchboard 174. A user behavior sensor 110 is shown.

As shown in FIG. 11, the user behavior sensor 110 includes an adhesive layer 202 on the bottom, on which a control and communication circuit 204 and a solar cell 206 are sequentially formed. A transparent conductive layer 208 , a transparent intermediate layer 210 , a transparent conductive layer 212 , and a transparent uneven layer 214 are sequentially laminated on the solar cell 206 . The control/communication circuit 204 has a wireless communication function (near-field wireless communication) and a worker contact detection function. A solar battery 206 is used as a driving power supply for both functions. If a battery is used as the drive power source, it will take time and effort to replace the battery. In addition, if an external power supply connected by an electric wire is used as the drive power supply, the electric wire interferes with contact by the operator. However, if the solar cell 206 is used, the user action sensor 110 can be used for a long period of time without the need for battery replacement and without interfering with the contact of the operator.

By stacking the control/communication circuit 204 for short-range wireless communication and control under the solar cell 206, the power generation efficiency of the solar cell 206 is increased and the planar size of the user behavior sensor 110 is reduced.

In order to use the solar cell 206, ambient light must be able to illuminate the solar cell 206. FIG. On the other hand, it is desirable to arrange the portion that detects user contact on the surface of the user behavior sensor 110 . As a method to satisfy both requirements at the same time, the part that detects contact is made transparent and a capacitive detection method is adopted. In order to detect the touch or pressure of an operator using changes in capacitance, a transparent elastic transparent intermediate layer 210 (for example, a sheet made of a transparent organic material) is placed between two transparent layers. A structure sandwiched between conductive layers 208 and 212 (for example, transparent organic material sheets) can be employed. An AC voltage 216 is applied between the two transparent conductive layers 208 and 212 to resonate the transparent conductive layers 208 and 212 . When the operator touches the surface of the user behavior sensor 110, the capacitance changes, so that the AC resonance condition changes. The operator's contact can be detected by detecting a change in this AC resonance condition. It should be noted that any element that can irradiate the solar cell 206 with at least part of the ambient light in the user behavior sensor 110 and that can detect contact or pressure is not limited to such a capacitive detection method. may

As an example, the transparent layer on the surface of the user behavior sensor 110 is provided with fine unevenness. This is partly to prevent the surface from slipping, but by recording information in Braille on the uneven surface, it is easier for people with visual impairments to use.

As a method of fixing the user behavior sensor 110 to a part of the existing manufacturing apparatus, any fixing method such as screwing may be adopted, but direct adhesion or sticking/adhering can save space. This adhesion or adhesion/attachment method is not limited to direct adhesion using an adhesive, but may be a method using an adhesive sheet or adhesive tape. For the on/off switch 172 and the light switchboard 174, an adhesive layer 202 with double-sided tape properties is available, and for the bulb 170, an adhesive layer 202 consisting of a transparent adhesive tape layer is available.

FIG. 12 shows the user activity sensor 110 attached to the light switchboard 174. FIG. In the lighting switch board 174, characters such as "lighting 1", "lighting 2", and "lighting 3" are written on the surfaces of the first, second and third light switches 176, 178 and 180, so that the user can It is desirable that these characters remain visible even when the activity sensor 110 is attached. Therefore, it is desirable that the top of the light switches 176, 178, 180 be transparent. In addition, it is necessary to detect the contact status of the plurality of lighting switches 176, 178, 180 independently and individually. On the other hand, the lighting switch board 174 has an extra space 182 where the lighting switches 176, 178 and 180 are not installed. In order to adapt to such a situation, the user behavior sensor 110 shown in FIG. 12 has a transparent sheet 208 , a transparent intermediate layer 210 , a transparent sheet 212 and a transparent concave-convex layer 214 laminated in order on the adhesive layer 202 . Transparent sheets 208, 212 correspond to transparent conductive layers 208, 212 of FIG. 11, transparent sheet 208 includes three transparent conductive regions 208a, 208b, 208c and transparent sheet 210 includes three transparent conductive regions 210a, 210b, 210c. including. Transparent conductive regions 208 a , 210 a are provided at the position of the first light switch 176 , transparent conductive regions 208 b , 210 b at the position of the second light switch 178 , and transparent conductive regions 208 c , 210 c at the position of the third light switch 180 . An alternating voltage 216 is applied between the transparent sheets 208 and 212 . By dividing the transparent sheet into three areas corresponding to the three lighting switches in this way, the contact statuses of the three lighting switches can be detected independently. Braille information can also be formed on the transparent uneven layer 214 on the surface.

A control circuit 204a and a communication circuit 204b are formed on the transparent concavo-convex layer 214 in the spare space 182 where the lighting switch is not installed, and the solar cell 206 is formed thereon. Since the solar cell 206 is arranged at the top, power generation efficiency is high. Also, since the control circuit 204a, the communication circuit 204b, and the solar cell 206 are positioned vertically, the planar size of the user behavior sensor 110 is reduced.

According to the embodiment, the conditions of the wearable terminal and the manufacturing apparatus are detected, and based on the detection results, near the manufacturing apparatus that requires work such as maintenance, inspection, and repair, a worker capable of performing the work is identified. Significant information can be provided to the worker by displaying the work procedure on the wearable terminal. In addition, based on the results of detecting the status of the wearable terminal and the manufacturing equipment, it determines the completion of each step of a series of tasks, and automatically creates a work report that records the performance of the work, reducing the amount of work required by the worker. can save a lot. By attaching a contact sensor to the manufacturing equipment, the completion of the work can be detected. Therefore, it is possible to estimate/recognise the actions of the worker in a very simple and inexpensive manner with high accuracy without modifying the existing manufacturing equipment. becomes possible.

In the above description, the maintenance of manufacturing equipment is taken as an example. However, the present embodiment is not limited to this, and may monitor user's actions corresponding to other purposes and display work contents accordingly. Further, as a means for monitoring the behavior of the user, the contact sensors are provided at locations where the user may come into contact. However, other sensors may be used.

Although the glasses-type wearable terminal has been described as an embodiment, the present invention can also be applied to other head-mounted terminals such as goggles and helmets, as well as wristbands, pendants, and the like. For example, if a helmet type or goggle type is used, the projection device 12 and the camera 59 can be attached to the helmet or goggles, and can be used by ordinary eyeglass users. Furthermore, if the helmet type is used, the speaker 54 can be attached to the inside of the helmet, so that a clearer sound can be heard. improves.

The types of sensors that detect the states of the manufacturing apparatus and wearable terminal are not limited to those described above, and various sensors can be used as appropriate.

The present invention can also be applied to wearable terminals other than head-mounted terminals. The present invention can also be applied to non-wearable, portable, small and lightweight electronic devices that are always with users, such as notebook personal computers, tablet personal computers, and smartphones.

The sharing of functions between the wearable terminal and the information management server is not limited to the above description, and some of the functions described as the functions of the wearable terminal may be implemented as the functions of the information management server, or the functions of the information management server. You may implement|achieve a part of what was demonstrated as a function of a wearable terminal.

It should be noted that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying constituent elements without departing from the scope of the present invention at the implementation stage. Also, various inventions can be formed by appropriate combinations of the plurality of constituent elements disclosed in the above embodiments. For example, some components may be omitted from all components shown in the embodiments. Furthermore, constituent elements of different embodiments may be combined as appropriate.

DESCRIPTION OF SYMBOLS 10... Wearable terminal, 12... Projection device, 16... Screen, 54... Speaker, 55... Patch pad, 59... Camera, 62... Optical sensor, 104... Administrator terminal, 106... Manufacturing apparatus, 108... Apparatus state sensor, 110 ... User action sensor, 114 ... Camera, 116 ... Information management server, 148 ... Microphone, 152 ... Wireless communication device.

Claims (3)

a display for displaying a display image;
a projection device that forms a virtual image corresponding to the display image in front of the user by projecting the display image in front of the user;
a motion sensor capable of detecting a movement of the user's head and determining a change in face orientation;
A wearable terminal comprising
The projection device includes a lens that projects divergent light emitted from the display in front of the user, a first user operation section, and a second user operation section,
The first user operation unit is capable of adjusting a projection angle of the projection device by rotating,
The user can adjust the display position of the virtual image according to the shape or size of the user's head by adjusting the projection angle by rotating the first user operation unit while viewing the virtual image. ,
the virtual image is changed in accordance with a change in orientation of the user's face detected by the motion sensor;
The wearable terminal, wherein the user can adjust the luminance or color tone of the virtual image by an operation other than the rotating operation of the second user operation unit. a server that stores images;
a display connected to the server for displaying a display image based on the image transmitted from the server; and a virtual image corresponding to the display image formed in front of the user by projecting the display image in front of the user. A wearable terminal comprising a projection device and a motion sensor capable of detecting a movement of the user's head and determining a change in face orientation;
A system comprising
The projection device includes a lens that projects divergent light emitted from the display in front of the user, a first user operation section, and a second user operation section,
The first user operation unit is capable of adjusting a projection angle of the projection device by rotating,
The user can adjust the display position of the virtual image according to the shape or size of the user's head by adjusting the projection angle by rotating the first user operation unit while viewing the virtual image. ,
the virtual image is changed in accordance with a change in orientation of the user's face detected by the motion sensor;
The system, wherein the user can adjust the brightness or color tone of the virtual image by an operation other than the rotating operation of the second user operation unit. a display for displaying a display image; a projection device for forming a virtual image corresponding to the display image in front of the user by projecting the display image in front of the user; A display method for a wearable terminal comprising a motion sensor capable of determining a change in orientation of the wearable terminal,
The projection device includes a lens that projects divergent light emitted from the display in front of the user, a first user operation section, and a second user operation section,
The first user operation unit is capable of adjusting a projection angle of the projection device by rotating,
The user can adjust the display position of the virtual image according to the shape or size of the user's head by adjusting the projection angle by rotating the first user operation unit while viewing the virtual image. ,
the virtual image is changed in accordance with a change in orientation of the user's face detected by the motion sensor;
The display method, wherein the user can adjust the brightness or color tone of the virtual image by an operation other than the rotating operation of the second user operation unit.

Images