KR20160027813A - Glass type terminal - Google Patents

Abstract

The present invention provides a glass type terminal comprising: a fixing part installed in the head of a user; a camera part combined with the fixing part to obtain the image information for an object located around a user; a storage part for storing defect information of eyes of the user; a control part for producing the notification information supplementing the defect of eyes of the user based on the image information for an object obtained by the camera part; and an output part for informing the user of the notification information, thereby making up the defect depending on the type and degree of defects of the visually impaired person.

Description

[0001] GLASS TYPE TERMINAL [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a glass-type terminal that can be worn on a human body in the field of mobile terminals, and relates to a glass-type terminal for a visually impaired person and a control method thereof.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can directly carry the mobile terminal. In this specification, a wearable device, particularly, a glass-type terminal will be described among mobile terminals.

The functions of glass type terminals are diversified. For example, there are data and voice communication, photographing and video shooting through a camera, voice recording, music file playback through a speaker system, and outputting an image or video on a display unit. Some terminals are equipped with an electronic game play function or a multimedia player function. In particular, modern mobile terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

In recent years, there have been many devices and interfaces for functions for supplementing defects of the user's body.

In particular, there are apparatuses that compensate for defects of the user's eyes, but existing apparatuses have substantially the same functions as the eyes of the user, and there is a limit to complementing the defects.

Since the wearable device must be worn on the wearer's body, the weight and volume of the wearable device can be the biggest problem. To solve these problems, miniaturization of the battery and minimization of power consumption become an issue. Therefore, it is necessary to reduce power consumption for unused portions.

The present invention provides a notification function of an object approaching the periphery in order to compensate a defect of a user who has a visual defect in the above-described glass-type terminal. Another object is to store visual defect information of a user and to supplement the function corresponding thereto. Furthermore, it has various execution modes, and power consumption can be adjusted accordingly.

According to an aspect of the present invention, there is provided an image capturing apparatus comprising: a fixing unit mounted on a head of a user; a camera unit coupled to the fixing unit and configured to acquire image information about an object positioned around the user; A storage unit for storing a visual acuity value, a viewing angle value, and a hue abnormality information; a visual acuity value determining unit for determining whether the visual acuity value and the viewing angle value are in an abnormal range and, if there is a visual acuity value or a viewing angle value corresponding to the abnormal range, A control unit for generating image information as notification information, a storage unit for storing a visual acuity value of a user's eye, a viewing angle value, and color hue abnormality information, and an output unit for informing the user of the generated notification information.

According to another aspect of the present invention, there is provided a display device including a display unit for outputting an image of an object, an illuminance sensor for measuring an illuminance value of the surroundings, and the image information includes an infrared image emitted by the object, And an infrared camera for obtaining an infrared image, wherein the control unit controls the display unit to output an infrared image obtained through the infrared camera when the measured illuminance value is equal to or less than a first reference illuminance value Type terminal.

According to another aspect of the present invention, the image information includes a character image, the notification information includes a voice signal, and when the reading mode is executed by the user, Converts the converted character data to the recognizable character data, and converts the converted character data into the voice signal.

Effects of the mobile terminal and the control method according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, there is an advantage that it can be supplemented according to the type and degree of defects of a user having a visual defect.

According to at least one embodiment of the present invention, when the illuminance value measured by the illuminance sensor is equal to or lower than the first reference illuminance value, an infrared mode is executed and an image obtained through the infrared camera can be output to the display unit There are advantages.

Further scope of applicability of the present invention will become apparent from the following detailed description. It should be understood, however, that the detailed description and specific examples, such as the preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art.

1 is a block diagram illustrating a glass-type terminal according to the present invention.
2 is a perspective view illustrating an example of a glass-type terminal according to an embodiment of the present invention.
FIGS. 3A, 3B, and 3C show an embodiment of a process of notifying a user of a risk of collision with a specific object.
FIG. 4 illustrates an embodiment of sensing and informing a protruding portion or a depressed portion when a sudden walking is encountered.
Figure 5 is an embodiment related to the present invention including an inner camera.
6A is a view of an object viewed by a user with a narrow viewing angle.
FIG. 6B is a view showing an embodiment showing a corrected display for a user with a narrow viewing angle.
7A illustrates an embodiment of a display provided to a user having a color vision abnormality.
FIG. 7B shows an embodiment of a speech signal provided to a user having a color vision abnormality.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

In this specification, a wearable device, particularly, a glass-type terminal will be described among mobile terminals.

However, the configuration according to the embodiments described herein may be applied to other mobile terminals such as a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal computer (PDA) digital assistants, portable multimedia players (PMPs), navigation, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, But also to an HMD (head mounted display).

1 is a block diagram illustrating a glass-type terminal according to the present invention.

The glass type terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, 190), and the like. The components shown in FIG. 1 are not essential for implementing a mobile terminal, so that the glass-like terminal 100 described herein can have more or fewer components than the components listed above.

The wireless communication unit 110 may be connected between the glass type terminal 100 and the wireless communication system or between the glass type terminal 100 and another mobile terminal or between the glass type terminal 100 and the external server 100. [ Lt; RTI ID = 0.0 > wireless < / RTI > In addition, the wireless communication unit 110 may include one or more modules for connecting the glass-like terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera unit 121 or a video input unit for inputting a video signal, a microphone 122 for inputting an audio signal, or an audio input unit, a user input unit 123 for receiving information from a user, For example, a touch key, a mechanical key, etc.). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include at least one sensor for sensing at least one of the information in the glass-type terminal 100, the surrounding environment information surrounding the glass-type terminal 100, and the user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 226, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the glass-type terminal 100 disclosed in the present specification can combine and utilize information sensed by at least two of the sensors.

The output unit 150 includes at least one of a display unit 151, an acoustic output unit 152, a haptic module 153, and a light output unit 154 to generate an output related to visual, auditory, can do. The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. The touch screen functions as a user input unit 123 that provides an input interface between the glass-like terminal 100 and a user, and can provide an output interface between the glass-like terminal 100 and a user.

The interface unit 160 serves as a path for connecting various types of external devices connected to the glass- The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the mobile terminal 100, corresponding to the connection of the external device to the interface unit 160, it is possible to perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the glass-like terminal 100. The memory 170 may store a plurality of application programs or applications that are driven by the mobile terminal 100, data for operation of the glass-like terminal 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. At least some of these application programs may exist on the glass-like terminal 100 at the time of shipment for basic functions of the glass-like terminal 100 (e.g., phone call incoming, outgoing call, message reception, have. Meanwhile, the application program may be stored in the memory 170, installed on the glass-like terminal 100, and operated by the control unit 180 to perform the operation (or function) of the glass-like terminal.

In addition to the operations associated with the application program, the control unit 180 typically controls the overall operation of the glass- The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1 in order to drive an application program stored in the memory 170. FIG. In addition, the controller 180 can operate at least two of the components included in the glass-based terminal 100 in combination with each other for driving the application program.

Under the control of the controller 180, the power supply unit 190 receives external power and internal power, and supplies power to the components included in the glass-type terminal 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with one another to implement a method of operation, control, or control of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the glass-like terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170. [

Hereinafter, the components listed above will be described in more detail with reference to FIG. 1 before explaining various embodiments implemented through the glass-type terminal 100 as described above.

The position information module 115 is a module for obtaining the position (or current position) of the glass-like terminal, and representative examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, a glass-type terminal can acquire the position of a glass-type terminal using a signal transmitted from a GPS satellite by using a GPS module. As another example, a glass-based terminal can utilize a Wi-Fi module to acquire the position of a glass-based terminal based on information of a wireless access point (AP) that transmits or receives a wireless signal with a Wi-Fi module have. Optionally, the location information module 115 may substitute or additionally perform any of the other modules of the wireless communication unit 110 to obtain data regarding the location of the glass-like terminal. The location information module 115 is a module used to obtain the location (or current location) of the glass-like terminal, and is not limited to a module that directly calculates or obtains the location of the glass-like terminal.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For inputting image information, the glass- One or more camera units 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. [ A plurality of cameras 121 provided in the glass-type terminal 100 may be arranged to form a matrix structure. Through the camera 121 having the matrix structure, the glass- A plurality of pieces of image information having a focus can be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes the external acoustic signal into electrical voice data. The processed voice data can be utilized variously according to functions (or application programs being executed) being performed in the glass-like terminal 100. [ Meanwhile, the microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 123 is for receiving information from the user. When information is inputted through the user input unit 123, the control unit 180 controls the operation of the glass- have. The user input unit 123 may include a mechanical input unit (or a mechanical key such as a button located on the front, rear or side of the glass-like terminal 100, a dome switch, , Jog switches, etc.) and touch-type input means. For example, the touch-type input means may comprise a virtual key, a soft key or a visual key displayed on the touch screen through software processing, The virtual key or the visual key can be displayed on the touch screen with various forms. For example, the virtual key or the visual key can be displayed on the touch screen, ), An icon, a video, or a combination thereof.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a corresponding sensing signal. The control unit 180 may control driving or operation of the glass-like terminal 100 or perform data processing, function or operation related to the application program installed in the glass-based terminal 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity of the detection surface, without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner area of the mobile terminal or in proximity to the touch screen, which is covered by the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 141 can be configured to detect the proximity of the object with a change of the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity with no object touching the touch screen is referred to as "proximity touch & The act of actually touching an object on the screen is called a "contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 141 can detect a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, have. Meanwhile, the control unit 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and further provides visual information corresponding to the processed data It can be output on the touch screen. Further, the control unit 180 can control the glass-like terminal 100 so that different operations or data (or information) are processed according to whether the touch to the same point on the touch screen is a proximity touch or a contact touch have.

The touch sensor uses a touch (or touch input) applied to the touch screen (or the display unit 151) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, Detection.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like. Here, the touch controller may be a separate component from the control unit 180, and may be the control unit 180 itself.

On the other hand, the control unit 180 may perform different controls or perform the same control according to the type of the touch object touching the touch screen (or a touch key provided on the touch screen). Depending on the type of the touch object, whether to perform different controls or to perform the same control can be determined according to the current state of the glass-type terminal 100 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 180 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera 121 includes at least one of a camera sensor (for example, a CCD, a CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to sense a touch of the sensing object with respect to the three-dimensional stereoscopic image. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 151 displays (outputs) information processed in the glass-like terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the glass-type terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information have.

Also, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

In the stereoscopic display unit, a three-dimensional display system such as a stereoscopic system (glasses system), an autostereoscopic system (no-glasses system), and a projection system (holographic system) can be applied.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 152 also outputs sound signals related to functions performed in the glass-like terminal 100 (for example, call signal reception sound, message reception sound, and the like). The audio output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 153 may be vibration. The intensity and pattern of the vibration generated in the haptic module 153 can be controlled by the user's selection or the setting of the control unit. For example, the haptic module 153 may synthesize and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 153 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 153 can transmit the tactile effect through the direct contact, and the tactile effect can be felt by the user through the muscles of the finger or arm. The haptic module 153 may include two or more haptic modules 153 according to the configuration of the glass-

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the glass type terminal 100. Examples of events that occur in the glass-like terminal 100 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output by the optical output unit 154 is implemented as the glass-type terminal emits light of a single color or a plurality of colors to the front or rear surface. The signal output may be terminated by the glass-like terminal detecting the event confirmation of the user.

The interface unit 160 serves as a path for communication with all external devices connected to the glass- The interface unit 160 receives data from an external device or supplies power to each component in the glass-type terminal 100, or allows data in the glass-type terminal 100 to be transmitted to an external device. For example, a port for connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, an audio I / O port, a video I / O port, an earphone port, and the like may be included in the interface unit 160.

The identification module is a chip for storing various kinds of information for authenticating the use right of the glass type terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the glass-like terminal 100 through the interface unit 160. [

When the glass-type terminal 100 is connected to an external cradle, the interface unit 160 may be a path through which the power from the cradle is supplied to the glass-like terminal 100, Type command signal may be transmitted to the glass-type terminal 100. [0050] FIG. Various command signals input from the cradle or the power source can be operated as a signal for recognizing that the glass-like terminal 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 170 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read memory, a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and / or an optical disk. The glass-like terminal 100 may be operated in association with a web storage that performs a storage function of the memory 170 on the Internet.

Meanwhile, as described above, the control unit 180 controls the operations related to the application program and the general operation of the glass-type terminal 100. For example, when the state of the glass-like terminal satisfies a set condition, the controller 180 may execute or release a lock state for restricting input of a user's control command to applications.

In addition, the control unit 180 performs control and processing related to voice communication, data communication, video call, or the like, or performs pattern recognition processing to recognize handwriting input or drawing input performed on the touch screen as characters and images, respectively . Further, the control unit 180 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the glass-like terminal 100 according to the present invention.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 includes a battery, the battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like.

Also, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of the interface unit 160 to which an external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission apparatus and a magnetic resonance coupling method based on an electromagnetic resonance phenomenon Power can be delivered.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

The wearable device can be made to be able to exchange (or interlock) data with other mobile terminals. The short range communication module 114 can detect (or recognize) a wearable device capable of communicating around the mobile terminal. Further, when the detected wearable device is a device authenticated to communicate with the mobile terminal, the controller 180 may transmit at least a part of the data processed in the mobile terminal to the wearable device through the short distance communication module 114. [ Therefore, the user can use the data processed in the mobile terminal through the wearable device. For example, when a telephone is received in the mobile terminal, it is possible to make a telephone call through the wearable device or to confirm the received message through the wearable device when the message is received in the mobile terminal.

2 is a perspective view illustrating an example of a glass-like terminal 100 according to an embodiment of the present invention.

The glass-type terminal 100 is configured to be worn on the user's head, and a fixing unit (a case, a housing, etc.) for the glass-type terminal 100 can be provided. The fixing part 101 may be formed of a flexible material so as to be easily worn.

The fixing part (101) is supported on the head part, and a space for mounting various parts is provided. As shown in the figure, electronic parts such as the control module 180, the sound output modules 1521 and 1522, and the like may be mounted on the fixed portion. A lens 103 covering at least one of the left eye and the right eye may be detachably attached to the fixing portion 101. [

The control module 180 controls various electronic components included in the mobile terminal 100. The control module 180 can be understood as a configuration corresponding to the control unit 180 described above. This figure illustrates that the control module 180 is installed on the fixing portion 101 on one side of the head. However, the position of the control module 180 is not limited thereto.

The display unit 151 may be implemented as a head mounted display (HMD). The HMD type refers to a display method that is mounted on a head and displays an image directly in front of the user's eyes. When the user wears the glass-type mobile terminal 100, the display unit 151 may be disposed to correspond to at least one of the left and right eyes so that the user can directly provide an image in front of the user's eyes. In this figure, the display unit 151 is located at a portion corresponding to the right eye so that an image can be output toward the user's right eye.

The display unit 151 can project an image to the user's eye using a prism. Further, the prism may be formed to be transmissive so that the user can view the projected image and the general view of the front (the range that the user views through the eyes) together.

As described above, the image output through the display unit 151 can be seen overlapping with the general view. The mobile terminal 100 can provide an Augmented Reality (AR) in which a virtual image is superimposed on a real image or a background and displayed as a single image using the characteristics of the display.

The glass-type mobile terminal 100 may include user input units 123a and 123b operated to receive control commands. The user input units 123a and 123b can be employed in any tactile manner, such as a touch or a push, in a tactile manner. In this figure, the push buttons and the touch input type user input portions 123a and 123b are provided in the fixed portion and the control module 180, respectively.

In addition, the glass-type terminal 100 may be provided with a microphone (not shown) for receiving sound and processing the sound into electrical voice data and an acoustic output module 152 for outputting sound. The sound output module 152 may be configured to transmit sound in a general sound output mode or a bone conduction mode. When the sound output module 152 is implemented in a bone conduction manner, when the user wears the mobile terminal 100, the sound output module 152 is brought into close contact with the head and vibrates the skull to transmit sound.

The camera unit 121 can be coupled to the fixing unit 101 mounted on the user's head. Since the camera 121 is located adjacent to the eye, a scene viewed by the user can be obtained as an image.

Although the camera 121 is provided in the control module 180 in this figure, it is not limited thereto. The camera 121 may be installed in the fixing part 101.

The camera unit 121 acquires image information about an object located around the user. Here, the term " surrounding " may include a front side or a side side, and in some cases, a rear side. The image information may include a visual image or position information, and the visual image may include a still image or a moving image.

The camera unit 121 may include a distance sensor 1217. This may be coupled to the camera lens unit itself or may be mounted on the fixing unit 101 in a separate configuration (not shown). The position information may be acquired by the distance detection sensor 1217. [ The distance sensor 1217 measures a distance from a user to an object by calculating a time when a signal such as an ultrasonic wave or a laser reaches a specific object located in a direction in which the sensor is facing and arrives at the sensor again.

The user may have visual defects. The ability of the eye is judged based on visual acuity, which is the eye's ability to recognize the shape of an object, and how wide the angle can be perceived based on the center of the eye.

The visual acuity value, the viewing angle value, and the hue abnormality information, which are visual information of the user, can be stored in the storage unit 171 (see FIG. 1) on the glass-like terminal 100. When the visual acuity value and the viewing angle value are in an abnormal range, the user can provide notification information corresponding to the visual acuity value and the viewing angle value, and a criterion for executing a power saving mode for some or all of the functions to prevent unnecessary power wasting .

The user visual information may be input through another mobile terminal associated with the glass- The user can input his or her own numerical value according to the visual acuity value, the viewing angle value, and the color-blindness information item. In the case of visual acuity setting, it is possible to display a screen for designating grade / arc according to the degree of failure grade. Such visual acuity may be set to be subdivided for near vision, primordial or astigmatism.

In addition to a method of directly inputting each item of visual information by a user, a data-rendered image or a predetermined data code or the like, such as a QR code containing information on previously measured visual information of the user, 171, Fig. 1). When the information is input in the form of an image like the QR code, the camera unit 121 of the glass-like terminal 100 receives the image without using the mobile terminal interlocked with the glass- It may be stored in the storage unit 171 (see FIG. 1) by itself.

The control unit 180 can generate notification information corresponding to the visual information of the user based on the image information about the object acquired by the camera unit 121. [ The notification information may be an image or a voice signal, or a signal such as vibration. The notification information may be a purpose for providing a user with a specific guidance, or may be used for notifying risk information in a specific dangerous situation.

The camera unit 121 can be zoomed in / out according to the degree of visual acuity of the user so that a basic zoom state can be set. For example, in the case of nearsightedness in which a distant place is not easily seen, the camera unit 121 can zoom / photograph / analyze mainly a distant place.

An inner camera for recognizing the user's eyes may be provided as needed, and the inner camera may determine the position, direction, etc. of the user's eyes or eyes. The display unit 151 or the inner camera for recognizing the user's eyes may not be operated in the case of blindness with no visual acuity at all.

FIGS. 3A, 3B, and 3C show an embodiment of a process of notifying a user of a risk of collision with a specific object.

3A shows a plurality of objects 300, cameras 1213 and 1214 existing on the glass-like terminal 100, and distance sensors 1213a and 1214a provided in the respective camera units.

FIG. 3B illustrates the determination of the position by measuring the distance to one of the plurality of objects in FIG. 3A.

FIG. 3C shows the determination of the position by measuring the distance when the position of the object to be measured shown in FIG. 3B is changed.

It is possible to generate a warning signal by judging whether or not a specific object may hit the user. The camera unit 121 (see FIG. 2) recognizes the surrounding object 300. This can be done by the distance detection sensor 1217 (see FIG. 2). 2) includes at least two cameras (referred to as a first camera 1213 and a second camera 1214) present at different positions on the fixed part, Sensor (referred to as a first distance detection sensor 1213a, a second distance detection sensor 1214a). A distance value (referred to as a first distance value) from the first distance detection sensor 1213a to an object and a distance value (referred to as a second distance value) from the second distance detection sensor 1214a to the object are measured , A point that simultaneously satisfies the first distance value 402 and the second distance value 403 by the control unit can be calculated as the position of the object. Further, when the first distance value 402 or the second distance value 403 satisfies the first reference distance value or less and the direction of change of the position of the calculated object is directed to the user, have.

Or when a dangerous situation occurs on the right side of the user, a voice signal is outputted through the right sound output unit 1521 (see FIG. 2), and when the left sound output occurs through the left sound output unit 1522 You can also tell the risk by separating left and right as you print.

FIG. 4 shows an embodiment of sensing and notifying when a sudden protrusion (not shown) or a depression 302 is encountered during a walking. When the distance detection sensor (referred to as third distance detection sensor 1216a) senses the lower front end of the user and the detected distance 405 immediately after is increased to a certain value or more , It is possible to generate a notification signal indicating that the depression 302 is present in front of the user. On the other hand, if the distance is suddenly reduced, it is possible to inform (not shown) through a specific signal that there is a protrusion in front of the user.

In addition, it is also possible to perform a function of distinguishing between a vehicle and a roadway through the camera unit 121. Generally, the roadway has a lane in a chain line pattern. Such a pattern image is recognized by the camera, and a danger signal may be notified (not shown) when the user is heading to the driveway.

Figure 5 is an embodiment related to the present invention including an inner camera.

The glass-type terminal 100 may include an inner camera 1215 that can recognize the user's eyes. The inner camera 1215 may be provided with two or more such that it can recognize the right eye or the left eye, respectively.

The inner camera 1215 can recognize where the user's line of sight is pointing.

It may be necessary to grasp where the user's gaze is directed as needed. This is the case, for example, when sufficient enlargement is required for the point, or when text expansion through text recognition or output in the form of a voice.

 The direction of the user's gaze is directed to the direction in which the front camera 1211 of the glass-type terminal 100 faces the angle 406 with respect to the direction in which the pupil 202 of the user is facing the glass- You can judge by adding. This allows intensive analysis or display of the object or area 301 to which the user's pupil is pointing.

This function can be applied even when there is a discomfort of the body other than the eye. For example, when a neck joint or body can not be rotated or deformed at will, a specific object or area can be identified through rotation of the eyes only. This confirmation can be confirmed through an augmented reality method of a method that is displayed semi-transparently on the display unit 151 (see Fig. 2), or an object can be grasped through a voice signal.

The inner camera 1215 can recognize whether or not the user's eyes 201 are wound.

The power saving mode can be executed for some or all of the functions by determining whether the eyes of the user are closed or not, and when the eyes are closed, it is unnecessary to recognize the surrounding situation. Or it may be judged as a sleeping state or a disorder of the eye, so that a higher level of perception of the surrounding state can be performed for the eye.

The inner camera 1215 may recognize whether the pupil of the user is enlarged or not and indirectly recognize the ambient illumination. It is possible to recognize that the pupil is enlarged and judge that the ambient illuminance is low, or to recognize that the pupil has been reduced and judge that the ambient illuminance is high.

The zoom in / out of the camera unit 121 can be determined according to the degree of defect of the input viewing angle 401 of the user. 6A is a view of an object 300 seen by a user with a narrow viewing angle 401. FIG. The narrower the viewing angle 401 becomes, the narrower the angle of view in which a wide area can be seen. FIG. 6B is a view showing an embodiment showing a corrected display for a user with a narrow viewing angle 401. FIG. Since the viewing angle 401 is narrow, the camera unit 121 (see FIG. 2) can photograph a target object viewed by the user in a zoomed-out state to provide a corrected display with a wider viewing angle. The corrected display may be presented in an augmented reality state (not shown) that may be presented in a translucent manner and viewed with the object 300 by the actual field of view.

7A illustrates an embodiment of a display provided to a user having a color vision abnormality. If the visual information is color-blind error information for a specific color, information that complements the generated information can be generated and provided to the user. Hue hue refers to a state in which the cone cells of the eye are abnormal and have no ability to distinguish or discriminate against a specific color or all colors. In this case, the color information of the image area of the object 300 can be grasped. The recognized information can be displayed on the display unit 151 in the form of a text 1511 for the corresponding color name.

FIG. 7B shows an embodiment of a speech signal provided to a user having a color vision abnormality. When a specific condition is satisfied, the color may be guided through voice. For example, in the case of a vehicle driver or a pedestrian signal, a green signal may be sensed to indicate that it is a green signal in text or voice. If it is changed to a red signal, it can be notified to the user by text or voice.

Since the glass-type terminal 100 is worn on the user's body, weight reduction of the product is a major factor, and it is difficult to sufficiently provide a battery for power supply. Therefore, it is necessary to operate unnecessary configuration or unused functions in the power saving mode so as to consume minimum power.

2) for detecting the front and the side of the camera unit 121 (see FIG. 2), and a forward camera provided to the front of the fixing unit 101 (see FIG. 2) Side camera for sensing the side of the user. Among these lateral cameras, the left eye camera assists the left eye of the user, and the right eye camera assists the right eye of the user.

The inputted defect information of the user's eyes can be stored in the input and storage unit 171 (see FIG. 1) by distinguishing the left eye and the right eye. The power saving mode of the left eye camera is executed when the value of the defect information for the left eye is in the normal value range and the power saving mode of the right eye camera is executed when the value of the defect information for the right eye is in the normal value range have.

In the power saving mode, a method of preventing the camera unit 121 (see FIG. 2) from recognizing an external object, a method of operating a sensor included in the camera unit 121 (see FIG. 2) And a software internal method in which the control unit does not perform an unnecessary operation on the distance value, the speed value, or the like, thereby saving power consumption.

In the power saving mode, the degree of power conservation or the object may be changed depending on the mode according to various situations as well as the division according to the left eye.

The indoor and outdoor modes indicate the difference in power saving mode depending on the degree of risk. In outdoor mode, there may be a sudden external risk, so we can obtain the most sensitive information without considering power consumption. In the indoor mode, since the relative risk is low, only the method of not recognizing the external object can be performed. Or the brightness of the display unit is reduced due to the fact that the illuminance is relatively low in the room, so that the power can be saved.

The glass-type terminal 100 (see FIG. 2) may further include an illuminance sensor 142 (see FIG. 1) for measuring a surrounding illuminance value. When the illuminance value measured by the illuminance sensor 142 (see FIG. 1) becomes equal to or less than the first reference illuminance value, the automatic mode can enter the night mode.

 When entering the night mode, the risk probability increases, so that the infrared camera can be additionally driven in addition to the case in the outdoor mode. The image acquired through the infrared camera can be displayed through the display unit.

In the reading mode, a book or other text character image located on the front side is recognized through a front camera provided on the front of the fixing unit, and converted into character data recognizable by a computer, thereby generating notification information. Such notification information may be an image signal obtained by enlarging a character image, or may be a signal converted into character voice information.

The various modes can be automatically searched and switched on the basis of certain conditions of the surrounding conditions, and manually selected by the user.

Hereinafter, embodiments related to a control method that can be implemented in a mobile terminal configured as above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present invention described above can be implemented as computer readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). In addition, the computer may include a control unit 180 of the terminal. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

100: glass-type terminal 101:
110: wireless communication unit 120: input unit
121: camera 1211: front camera
1213: first camera 1213a: first distance detection sensor
1214: second camera 1214a: second distance detection sensor
1215: Inner camera 1216a: Third distance sensing sensor
1217: Distance sensor 140:
142: illuminance sensor 150: output section
151: Display unit 1521: Right sound output unit
1521: Left sound output unit 160: Interface unit
170: memory 180:
190: Power supply unit 200: User
201: user's eyes 203: user's eyes
300: object
301: an object or area to which the user's pupil is pointing
401: Viewing Angle 402: First Distance Value
403: second distance value 404: distance detected immediately before
405: Distance detected immediately after
406: an angle with respect to the direction in which the user's pupil is directed with respect to the glass-

Claims (10)

A fixing part mounted on the user's head;
A camera unit coupled to the fixed unit and configured to acquire image information about an object located around the user;
A storage unit that stores the visual acuity value, the viewing angle value, and the hue error information of the user eye;
A controller for determining whether the visual acuity value and the viewing angle value are in an abnormal range and generating the visual information as notification information when the visual acuity value or the viewing angle value corresponding to the abnormal range or the color hue abnormal information exists; And
And an output unit for informing the user of the generated notification information. The method according to claim 1,
Wherein the camera unit includes a first camera and a second camera,
Wherein the first camera comprises a first distance sensing sensor,
The second camera having a second distance sensing sensor,
Wherein the image information comprises a first distance value from the first distance sensing sensor to the object and a second distance value from the second distance sensing sensor to the object,
Wherein,
Calculating a position of the object through the first and second distance values when the visual acuity value or the viewing angle value corresponds to the abnormal range,
Wherein the notification information is generated when the first and second distance values satisfy a first reference distance value or less and the direction of change of the position of the object faces the user. The method according to claim 1,
The camera unit includes:
And a third distance detection sensor coupled to the fixed portion and sensing a distance from the fixed portion to a front lower end of the fixed portion,
Wherein,
Wherein the notification information is generated when the visual acuity value or the viewing angle value corresponds to the abnormal range and the change in the third distance value measured by the third distance detection sensor is equal to or greater than a second reference distance value. The method according to claim 1,
The camera unit includes:
Further comprising an inner camera coupled to the fixed portion to sense a direction of a pupil of the left or right eye of the user,
Wherein when the visual acuity value falls within the abnormal range, the object is located in the direction of the pupil of the left eye or the right eye. The method according to claim 1,
And a display unit for outputting an image of the image information,
Wherein,
Wherein the control unit controls the display unit to display an image of the image information obtained by zooming out the camera unit when the viewing angle value is in an abnormal range. The method according to claim 1,
When a specific color corresponds to the hue or hue,
Wherein,
Wherein the camera unit recognizes the specific color and generates the notification information. The method according to claim 1,
Wherein the camera unit includes a left side camera for acquiring image information of a left peripheral object of the user or a right side camera for acquiring image information of a right peripheral object of a user,
Wherein,
When the left eye visual field value and the left visual field angle value of the user's visual information are in the normal range and there is no color darkness abnormality,
The power saving mode of the left side room camera is executed,
Wherein the power saving mode of the right side camera is executed when the right eye vision value and the right eye view angle value of the user's visual information are within a normal range and there is no color vision abnormality. The method according to claim 1,
The camera unit may include a distance sensor,
Wherein the image information includes a distance value from the distance sensor to the object,
Wherein,
And a controller that calculates a speed value based on the distance value generated by the distance detection sensor and reduces the brightness of the display unit when the distance value satisfies the third reference distance value or less and the speed value satisfies the first reference speed value or less, Mode is executed. The method according to claim 1,
A display unit for outputting an image of the object;
An illuminance sensor for measuring a surrounding illuminance value; And
Wherein the image information includes an infrared image emitted by the object,
Wherein the camera unit further comprises an infrared camera for acquiring the infrared image,
Wherein,
Wherein the control unit controls the display unit to output an infrared image obtained through the infrared camera when the measured illuminance value is equal to or less than the first reference illuminance value. The method according to claim 1,
Wherein the image information includes a character image,
Wherein the notification information includes a voice signal,
Wherein,
Wherein when the reading mode is executed by the user, the character image is converted into computer-recognizable character data, and the converted character data is converted into the voice signal.

Images