JP5955348B2 - Wearable device having privacy protection function of captured image, control method thereof, and image sharing system - Google Patents

Description

  The present invention relates to a wearable device, and more specifically, includes a display that is mounted on a user's head, captures an image visually recognized by the user, and displays it as a virtual image in front of the user's eyes. The present invention relates to a wearable device, a control method thereof, and an image sharing system that implement a privacy protection function by processing, converting the image quality into an image quality that is difficult to identify, and sharing with an external device.

  In recent years, there is an increasing demand for wearable devices that can be used by being attached to a user's body. In particular, a wearable device having a function of shooting an image visually recognized by the user and displaying it as a virtual image in front of the user's eyes, such as a head-mounted display (HMD) mounted on the head, is used by the user himself / herself. At the same time as visual recognition, various communication means are used to provide a function for exchanging information between users wearing wearable devices, thereby developing a form for use as a tool for communication.

For example, Patent Documents 1 to 3 disclose conventional HMDs.
In Patent Document 1 (Japanese Patent Laid-Open No. 5-303053), an external field image (referred to as an external image) viewed by a user is superimposed on a display image of an image display device, and these images are simultaneously displayed by the user. An HMD including an optical system configured to be visually recognized is disclosed. This type of HMD is called a see-through HMD.
In Patent Document 2 (Japanese Patent Laid-Open No. 2011-69849), when a see-through type HMD is used, a display image based on image information (or content information) is presented on top of an external image so that the user can In order to avoid the difficulty of distinguishing between an image and a display image, a see-through type HMD provided with display effect adding means such as adding a frame to the display image is disclosed.

  Patent Document 3 (Japanese Patent Laid-Open No. 2010-213214) discloses a means for storing data of an image captured using an HMD equipped with a camera in association with content information such as a travel guide displayed by a display means. An HMD with the following is disclosed. Patent Document 3 describes that the HMD detects a user's line-of-sight direction and captures the user's visual field range as a still image according to the detection result.

  A technique of monitoring an image visually recognized by the user and storing the image desired by the user as a still image, such as the HMD described in Patent Document 3, is useful for personal possession as a personal record.

However, when the image visually recognized by the user is shared with others as a communication tool described above, privacy protection is an important issue.
In particular, in an HMD having a conventional imaging function, an image of a person existing in the field of view regardless of the user's intention when the user using the HMD shares the image that the user views with other users. It also includes privacy problems.

  On the other hand, when using the wearable device as a communication tool, it is important to reduce the communication load when transmitting / receiving various data to / from other users and sharing information. In particular, since the image data has a large data size, the delay increases in a wireless communication environment with a low communication speed, which may be a factor that hinders real-time communication.

Japanese Patent Laid-Open No. 5-303053 JP 2011-69849 A JP 2010-213214 A

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a display that is mounted on a user's head, captures an image visually recognized by the user, and displays it as a virtual image in front of the user's eyes. And a wearable device that converts a captured image into an image quality that is difficult to identify in real time and realizes a privacy protection function, an image processing control method thereof, and an image sharing system.

  A wearable device according to an aspect of the present invention made to achieve the above object is worn on a user's head, and an image of image light based on image data is visually recognized by the user together with a visual field image of the outside world. A wearable device having a display that captures an image viewed by a user, a distance measuring unit that measures a viewing distance from a user's eye to a gazing point, and the camera unit. An image projection unit that projects image light onto a user's eye as a virtual image visually recognized at the viewing distance based on the image data and viewing distance measurement data corresponding to the image data, and a wearable device for controlling the wearable device to function as a wearable device A storage unit for storing various programs and various data, and image data captured by the camera unit in real time. An image processing unit for processing, a wireless communication unit for transmitting / receiving data to / from an external device, and a control unit for controlling the operation of each of the components, wherein the control unit captures image data captured by the camera unit. When transmitting to the external device via the wireless communication unit, the captured image data is converted into low gradation image data that is difficult to identify in real time by the image processing unit and transmitted to the external device. It is characterized by controlling to.

The process of converting the captured image data into low gradation image data by the image processing unit may be a binarization process or a ternary process.
When the external device is a wearable device having the same configuration as the wearable device worn by the user, the visual distance measurement data corresponding to the image data can be transmitted together with the image data captured by the camera unit.
The image processing unit further includes means for determining whether or not the photographed image data includes an image identified as a person, and the control unit transmits the photographed image data to an external device. If the image processing unit determines that the captured image data includes an image identified as a person, the image processing unit converts the captured image data into low-resolution image data. Control may be performed to transmit to the external device.
The wearable device further includes a visual field variation detection unit that detects a variation in the visual field direction of the user, and the control unit confirms a detection signal output from the visual field variation detection unit and detects a first visual field variation. When the period until receiving the second detection signal after detecting the next visual field variation after receiving the detection signal exceeds a preset time, the image visually recognized by the user when the preset time is reached Can be captured as a still image, and image data of the captured still image can be stored in the storage unit or the external device.

  In order to achieve the above object, a wearable device control method according to an aspect of the present invention is mounted on a user's head, and an image based on image light based on image data is displayed to the user together with a visual field image of the outside world. A method for controlling a wearable device having a display that is visually recognized by the camera, the step of taking an image that is visually recognized by the user with the camera unit, and the measurement of the viewing distance from the user's eye to the gazing point by the distance measuring unit. And projecting image light onto a user's eye as a virtual image visually recognized at the viewing distance based on image data photographed by the camera unit by the image projecting unit and viewing distance measurement data corresponding to the image data. Transmitting image data captured by the camera unit to an external device via a wireless communication unit, and the camera The step of transmitting the image data captured in step S3 to the external device via the wireless communication unit includes the step of converting the captured image data into low-gradation image data that is difficult to identify in real time by the image processing unit. It is characterized by that.

The process of converting the captured image data into low gradation image data may be a binarization process or a ternary process.
The step of transmitting image data captured by the camera unit to an external device via a wireless communication unit is performed when the external device is a wearable device having the same configuration as a wearable device worn by a user. The method may include transmitting the measurement data of the viewing distance corresponding to the image together with the image data.
The control method of the wearable device further includes a step of detecting a change in the visual field direction of the user, and the step of detecting the fluctuation in the visual field direction of the user confirms a detection signal output from the visual field fluctuation detection unit. A step of measuring a period until receiving a second signal detecting the next visual field variation after receiving the first signal detecting the visual field variation, and when the measurement time exceeds a preset time The image data corresponding to the image visually recognized by the user when the preset time is reached can be stored in the storage unit or the external device as image data of a still image.

  In order to achieve the above object, an image sharing system according to an aspect of the present invention is an image sharing system including a plurality of the wearable devices and further including an external data server connected to the plurality of wearable devices. The wearable device transmits / receives image data obtained by capturing an image viewed by a user with a camera unit mounted on the wearable device and measurement data of a viewing distance corresponding to the image data to the external data server via a communication network. The external data server includes: a communication unit that transmits / receives data to / from the plurality of wearable devices; a storage unit that stores data received from the wearable devices; and a plurality of wearable devices From our first wearable device The received image data and viewing distance measurement data are received and stored in the storage unit, and the first stored in the storage unit in response to a request from a second wearable device among the plurality of wearable devices. A control unit that transmits image data and viewing distance data of the wearable device to the second wearable device that has made the request.

According to the wearable device and its control method of the present invention, by capturing an image visually recognized by the user, converting the captured image data into low-gradation image data that is difficult to identify in real time and outputting the converted image data externally Privacy can be protected and communication speed can be improved.
In addition, according to the image sharing system including a plurality of wearable devices of the present invention and further comprising an external storage device connected to the plurality of wearable devices, real-time between users through a wireless network in a state where privacy is protected It is possible to provide a communication tool and means capable of exchanging information.

1 is a perspective view illustrating a wearable device according to an embodiment of the present invention. 1 is a block diagram schematically showing a configuration of a wearable device according to an embodiment of the present invention. It is a figure which shows schematically the structure of the image projection part of the wearable device by one Embodiment of this invention. 3 is a flowchart illustrating a wearable device control method according to the first exemplary embodiment of the present invention. 5 is a flowchart illustrating a method for controlling a wearable device according to a second embodiment of the present invention. 1 is a diagram illustrating a configuration of an image sharing system using a wearable device according to an embodiment of the present invention. 3 is a flowchart illustrating a control method of an image sharing system using a wearable device according to an embodiment of the present invention. 6 is a flowchart illustrating a control method of an image sharing system using a wearable device according to another embodiment of the present invention. 6 is a flowchart illustrating a control method of an image sharing system using a wearable device according to still another embodiment of the present invention.

  Hereinafter, specific examples of embodiments for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a perspective view illustrating a wearable device according to an embodiment of the present invention. FIG. 2 is a block diagram schematically showing the configuration of a wearable device according to an embodiment of the present invention.

  Referring to FIGS. 1 and 2, the wearable device 100 includes an image projection unit 10, a camera unit 20, a microphone 30, an earphone 40, and a touch switch 50. Further, a control unit 70, a visual field variation detection unit 81, a wireless communication unit 82, an audio processing unit 83, and a storage unit 84 are built in the wearable device housing 60, and a battery is installed in the wearable device housing 90. 85.

  The wearable device 100 captures an external image visually recognized by the user with the camera unit 20 and sends the captured image data to the control unit 70. The control unit 70 processes the received image data in accordance with an instruction from the user, and controls the image projection unit 10 to project image light based on the image data onto the user's eyes.

The image projection unit 10 constitutes the display of the wearable device 100, has an autofocus function, adjusts the projection distance of the virtual image, and projects the virtual image at the position where the user is gazing. Thereby, the user can visually recognize the virtual image as if it floated in the air. Details of this function will be described later.
The image projection unit 10 can project image data based on the content information received through the wireless communication unit 82 onto the user's eyes.

  Referring to FIG. 2, the control unit 70 includes a CPU 71 that controls each component for general operation of the wearable device 100, an image processing unit 72 that performs image conversion of image data and performs processing such as data compression, and An input / output control unit 73 that determines the type of input / output data and controls connection switching between a corresponding interface and a component is provided.

  A CPU (Central Processing Unit) 71 controls power supply to each component of the wearable device 100 and executes an initialization process. When the initialization process is completed, a user interface for various data processing for each component is provided. And performs a function selected based on a touch event generated by the user, and controls transmission / reception of input / output data.

The image processing unit 72 is a dedicated processor (personal identification by image analysis of image data, data conversion processing to a low gradation image such as binarization processing or other types of low resolution images, and data compression / decompression processing) For example, a DSP) is provided.
The input / output control unit 73 controls input / output of image data, audio data, and various content data, and transmits them to corresponding components based on user instructions.

  The visual field fluctuation detection unit 81 includes a gyro sensor and an acceleration sensor (not shown), and detects fluctuations in the visual field direction of the user. The visual field fluctuation detection unit 81 has a visual field fluctuation of the user when the angular velocity value detected by the gyro sensor and / or the acceleration value detected by the acceleration sensor changes more than a predetermined threshold with respect to a certain reference value. The detection signal is sent to the control unit 70.

  When the control unit 70 receives the detection signal from the visual field fluctuation detection unit 81, the control unit 70 sends a command signal for activating the autofocus function to the image projection unit 10. That is, the autofocus function is not always activated, and is controlled so that it is activated only when a change in the visual field direction is detected. Thereby, it becomes possible to suppress the power consumption of the battery 85 provided in the wearable device 100.

  The wireless communication unit 82 performs formation of a communication channel for a voice call, formation of a communication channel for transmitting image data or character data, and the like under the control of the control unit 70. That is, the wireless communication unit 82 forms a voice call channel, a character data communication channel, an image data communication channel, and the like between mobile communication systems. For the operation of the wireless communication unit 82, the wearable device 100 controls to output various information such as communication destination list information to the image projection unit 10. The wireless communication unit 82 is configured by Bluetooth (registered trademark), WiFi (registered trademark), or the like.

  The audio processing unit 83 outputs audio data transmitted / received during a call, audio data included in the received message, audio data by audio file playback received from an external storage device, and the like to the earphone 40, and the user's voice or Other audio signals are collected through the microphone 30.

  The touch switch 50 is disposed on the frame side portion of the wearable device 100, and the user touches a specific area of the touch switch 50 with a specific operation in order to instruct a specific image or a specific function to be output to the image projection unit 10. If so, support to generate the corresponding touch event.

  The storage unit 84 is configured by a flash memory (flash EEPROM), a RAM, and the like (not shown), and includes various application programs necessary for the functional operation according to the present embodiment, a menu map for operating the touch switch 50, and the like. Here, the menu map can use various forms. That is, the menu map may be a menu map for operation control of the currently activated application program, or a menu map having various menus provided by the wearable device 100 as a list. The storage unit 84 includes a program area and a data area.

The program area includes an OS (Operating System) for starting up the wearable device 100 and operating each of the above-described components, an application program for executing various functions, for example, an application program for supporting a call function, and an Internet server A web browser for connecting to the computer, an MP3 application program for reproducing audio data, an image output application program for displaying still images, a moving image compression / reproduction application program, and the like are stored.
In particular, the program area of this embodiment includes an image conversion processing application program.

  The image conversion processing application program, when setting privacy protection for a captured image by a selection signal based on a menu or a touch action, applies image conversion processing (for example, binarization or ternization) to image data captured by the camera unit 20 in advance. And a routine for performing gradation reduction processing such as digitization or other resolution reduction processing. For example, a sharing mode (a mode for transmitting / receiving image data to / from other wearable device users) and a private mode (a mode for transmitting image data to an external device) may be set as the use mode of the wearable device for which privacy protection is set. it can.

  The data area is an area for storing various data generated by using the wearable device 100, and stores menu map information, at least one icon corresponding to various functions, or various contents. The data area temporarily stores image data captured by the camera unit 20, image data received from the outside, various content information data, and the like. In particular, the data area of the present embodiment functions as a buffer memory for image data when image data captured by the camera unit 20 is converted by the image processing unit 72, and is activated when an application program that supports a communication function is activated. The converted data is sent to the wireless communication unit 82.

  Next, the image projection unit 10 of the present embodiment will be described in more detail with reference to FIG.

  The image projection unit 10 includes a display unit 11 that displays image data, an optical module 12 that guides image light emitted from the display unit 11 to the user's eye E, and a distance measurement unit 13 that measures the user's “viewing distance”. Prepare. Here, the “viewing distance” means a distance from the user's eye E to the gazing point.

Although not shown, the display unit 11 includes a display element and a light source, converts image data into image light, and outputs the image light to the optical module 12.
The display element is constituted by, for example, a transmissive liquid crystal display element or a reflective liquid crystal display element in which pixels that are regions through which light passes are arranged in a matrix. The light source emits light of each color of R (red), G (green), and B (blue), and includes, for example, an LED (light emitting diode). The light source is preferably composed of an RGB integrated LED panel, but may emit monochromatic light or white light.

The optical module 12 is an optical system that guides image light from the display unit 11 to the user's eye E. The optical module 12 includes a prism 14, a correction lens 15, and a projection distance adjustment unit 16 that adjusts the projection distance of the virtual image by adjusting the focal position of the correction lens.
The prism 14 is a light guide member that functions as means for bending the optical path of the image light from the display unit 11 at a right angle. The prism 14 may be configured by a single prism or may be configured by combining a plurality of prisms.

  Although not shown, the ranging unit 13 includes a ranging sensor that measures the refractive power of the user's eye E, and the user's eye E is focused based on the refractive power obtained from the ranging sensor. Specify the position of the viewpoint. Thereby, the distance measuring unit 13 measures the distance (viewing distance) from the user's eye E to the gazing point. The distance measuring unit 13 sends data related to the measured viewing distance to the control unit 70.

  The control unit 70 controls the projection distance adjustment unit 16 of the optical module 12 based on the data regarding the viewing distance. That is, the projection distance adjustment unit 16 receives a control signal corresponding to the data regarding the viewing distance from the control unit 70 and moves the physical position of the correction lens 15 back and forth, thereby condensing the image light on the prism 14. To adjust the projection distance of the virtual image. Note that the method by which the projection distance adjustment unit 16 adjusts the projection distance of the virtual image is not limited to the above, and a known method can be used as appropriate.

  Hereinafter, a method for controlling the wearable device 100 according to an embodiment of the present invention will be described in more detail.

  FIG. 4 is a flowchart illustrating a wearable device control method according to the first embodiment of the present invention.

  Referring to FIG. 4, first, when the user wears wearable device 100 (step S100), a sensor (not shown) included in wearable device 100 senses the wear and power is supplied, and the supplied power is used. Then, the control unit 70 of the wearable device 100 completes activation and outputs an initial image or menu image to the image projection unit 10 according to preset schedule information (step S110). Thereafter, the control unit 70 confirms whether a specific input event by the user is generated for the touch switch 50, for example.

When the input event is generated, the control unit 70 checks whether or not the corresponding input event is an input event that instructs the camera unit 20 to start photographing (step S120).
If the input event for activating the camera unit 20 is not generated, the control unit 70 returns to step S120 and performs control so as to check whether an input event is generated.
Note that step S120 for confirming the start of activation of the camera unit may be omitted, and the camera unit 20 may be operated in response to the start of power supply in step S110.

On the other hand, when an input event for instructing camera start is generated in step S120, the control unit 70 activates the camera unit 20, and the camera unit 20 captures and captures image data obtained by capturing an outside scene image visually recognized by the user. To send. At the same time, the control unit 70 activates the distance measuring unit 13 and performs control so as to measure the viewing distance to the gazing point visually recognized by the user. The distance measuring unit 13 sends data relating to the measured viewing distance to the control unit 70 (step S130).
Note that “visual recognition” means that the user recognizes the image light based on the image displayed on the display unit and the aspect in which the image light based on the outside scene is imaged on the user's eyes (on the retina) and the user recognizes the image. Including two aspects.

  The control unit 70 sends the image data from the camera unit 20 and the data related to the viewing distance from the distance measuring unit 13 to the image projecting unit 10, and the image projecting unit 10 provides the viewing distance to the user's eyes based on the two data. A virtual image corresponding to is projected (step S140).

  Further, the control unit 70 confirms the signal output from the visual field fluctuation detection unit 81 (step S150). Thereby, a user's gaze state is judged. That is, after receiving the first detection signal informing the visual field variation from the visual field variation detection unit 81, the period until receiving the second detection signal informing the next visual field variation exceeds a preset time (determination time). If so, it is determined that the user is watching the direction (step S160).

When the preset determination time has elapsed, the control unit 70 stores the image data corresponding to the image that the user is viewing when the determination time is reached in the storage unit 84 as a still image (step S170). At this time, the control unit 70 may project an object for allowing the user to recognize the elapse of the gaze determination time from the image projection unit 10 to the user's eyes.
The determination time may be, for example, 3 to 6 seconds, and object display for the user to recognize may be started, for example, 0.5 to 2 seconds after receiving the first detection signal.

Thereafter, the control unit 70 confirms whether or not a specific input event by the user is generated (step S180). When there is no occurrence of an input event for instructing the end of the operation of the camera unit 20, the control unit 70 branches before step S130 and performs control so that the subsequent steps are repeated.
On the other hand, when an input event for ending the operation of the camera unit 20 occurs, the control unit 70 stops the operation of the camera unit 20, branches before step S120, and controls to repeat the subsequent steps.

  FIG. 5 is a flowchart illustrating a wearable device control method according to the second exemplary embodiment of the present invention. In the present embodiment, an example of a control method after a user wears a wearable device and activates the camera unit will be described.

  Referring to FIG. 5, first, when the user wears the wearable device 100 and activates the camera unit 20 to start shooting an external image (step S210), the control unit 70 touches a specific input event by the user, for example, Whether or not it is generated for the switch 50 is confirmed.

  When the input event is generated, the control unit 70 confirms whether or not the corresponding input event is an input event for selecting to output the captured image in the private mode (step S220). Here, the private mode is a mode in which image data captured by the camera unit 20 is subjected to image processing in real time and converted into an image (for example, a binary image) in which a person or place in the image is difficult to identify.

  If it is not an input event for selecting the private mode, the control unit 70 branches to step S240, and sends the captured image data and data relating to the viewing distance to the image projection unit 10. The image projection unit 10 projects a virtual image corresponding to the viewing distance on the user's eye based on the two data (step S250). As a result, the image captured by the image projection unit 10 is projected as it is onto the user's eyes (hereinafter referred to as the normal mode).

  On the other hand, when an input event for selecting the private mode occurs in step S220, the control unit 70 controls the image processing unit 72 to perform image conversion (for example, binarization processing) on the captured image data in real time. (Step S230).

  Then, the control unit 70 sends the image data (for example, binary image data) processed by the image processing unit 72 and data relating to the viewing distance to the image projection unit 10 (step S240). Therefore, in this case, the binary image is projected as a virtual image on the user's eyes.

  The control unit 70 confirms whether a specific input event by the user is generated even during camera shooting. When the input event is generated, the control unit 70 checks whether or not the corresponding input event is an input event for selecting a mode for externally outputting a captured image (step S260).

  Here, the mode in which the captured image is output to the outside is a mode in which the control unit 70 controls the input / output control unit 73 to select a connection means with an external device based on a user instruction, and the wireless communication unit 82 is selected. In this mode, the connection means is operated to establish a communication channel with the external device, and image data captured by the user is transmitted to the external device. As a means for connecting to an external device, Bluetooth (registered trademark) or WiFi (registered trademark) communication provided in the wireless communication unit 82 is used. The external devices to be connected are a smartphone, a tablet terminal, a personal computer, and a data (or Web) server. Data sharing with a wearable device of the same type as the wearable device worn by the user, that is, the wearable device having the same configuration and the same function will be described later.

  If the input event for selecting the external output is not generated, the control unit 70 branches to step S280 and controls to repeat the subsequent steps.

  On the other hand, when an input event for selecting an external output occurs in step S260, the control unit 70 checks whether or not the private mode is automatically selected regardless of a user instruction (step S265).

  If it is confirmed in step 265 that the private mode is selected, the control unit 70 controls the input / output control unit 73 to select a connection means with an external device based on the user's instruction, and perform wireless communication. The image processed image data is transmitted to the external device through the unit 82 (step S270).

On the other hand, if the private mode is not selected, the control unit 70 branches before step S230 and starts the image conversion process (for example, binarization process) of the image data captured by controlling the image processing unit 72. (Switch to private mode) and execute the following steps. As a result, even if a person appears in the captured image, the image data that can be output to the outside is limited to that converted into low-gradation image data (for example, binary image data) that is difficult to identify individuals. Therefore, privacy is protected. Further, since the image data is compressed by image conversion processing such as binarization, the communication speed is improved.
Note that step S265 may be omitted, and setting may be made so that image data corresponding to the mode (normal mode or private mode) selected by the user in step S220 can be transmitted to the external device.

Thereafter, the control unit 70 checks whether or not an input event for releasing the private mode is generated (step S280). If there is no occurrence of an input event for releasing the private mode, control branches to before step S230 and the subsequent steps are repeated.
In addition, when an input event for releasing the private mode occurs, the control unit 70 stops the image conversion process (step S290), branches before step S220, and repeats the subsequent steps. To do.

  In the present embodiment, the flow for instructing whether or not the user selects the private mode when selecting the external output of the image data is shown, but the present invention is not limited to this. For example, when an input event for selecting external output of image data occurs in step S260 of FIG. 5, in step S265, the control unit 70 includes an image identified as a person in the captured image data. Whether or not the image is identified using the image analysis function of the image processing unit 72 and it is determined that an image identified as a person is included in the photographed image data is automatically processed without a user instruction. A setting for controlling the image data to be converted into low gradation image data by the unit 72 can be applied.

  Hereinafter, a configuration of an image sharing system using a wearable device according to an embodiment of the present invention and a control method thereof will be described in detail.

  FIG. 6 is a diagram illustrating a configuration of an image sharing system using a wearable device according to an embodiment of the present invention.

  Referring to FIG. 6, wearable devices (WD2) 102 and (WD3) 103 of the same type as wearable device (WD1) 101 worn by user 1 can perform short-range wireless communication (for example, Bluetooth (registered trademark) communication). It exists in the range 200. Each of the wearable devices (WD1 to WD3) 101 to 103 includes a wireless communication unit that supports short-range wireless communication. The wearable device 101 (WD1) automatically searches for these external wearable devices, and the user 1 designates a desired connection partner and performs pairing (link key exchange and establishment of master-slave communication), thereby directly transmitting / receiving (sharing) data to / from one or more designated wearable devices.

On the other hand, the wearable device (WD1) 101 connects to a nearby access point AP using WiFi communication or the like to the wearable device (WD4) 104 existing outside the short-range wireless communication range, and the Internet line or the like. The communication network 400 is used for communication.
Similarly, wearable device (WD1) 101 communicates with database server (or Web server) 300 or the like connected to communication network 400 such as an Internet line using WiFi communication.

  The database server 300 includes a control unit 310, a storage unit 320, and a communication unit 330. In response to a request from the wearable device (WD1) 101, the database server 300 stores various information and content data stored in the storage unit 320 as a wearable device ( WD1) 101, and receives image data and various data transmitted from wearable device 101 (WD1), stores them in storage unit 320, and stores them from other wearable devices (for example, WD4) permitted by user 1. In response to the request, the user 1 transfers the data stored in the storage unit 320 to the wearable device (WD4) 104.

  FIG. 7 is a flowchart illustrating a method for controlling an image sharing system using a wearable device according to an embodiment of the present invention.

  6 and 7, first, when the user 1 wears the wearable device (WD1) 101 and activates the camera unit 20 to start capturing an external image (step S310), the control unit of the wearable device 101 is started. 70 confirms whether a specific input event by the user 1 is generated for the touch switch 50, for example.

  When the input event is generated, the control unit 70 confirms whether or not the corresponding input event is an input event for selecting to output the captured image in the sharing mode (step S320).

  Here, the shared mode is to establish a communication channel with the wearable device (WD2) 102 and / or (WD3) 103 of the same type existing within the short-range wireless communication range by operating the wireless communication unit 82, 1 is a mode in which image data being photographed is transmitted in real time to a wearable device to be connected and a voice call is made.

  If the input event for selecting to output the captured image in the sharing mode is not generated, the control unit 70 returns to the step before step S320 and performs control so that the subsequent steps are repeated.

  On the other hand, when an input event for selecting to output the captured image in the sharing mode occurs in step S320, the control unit 70 proceeds to step S330 and confirms whether to apply the private mode to the image data.

When the private mode is not applied, the control unit 70 branches to step S350, and sends the captured image data and data relating to the viewing distance to the image projection unit 10.
The image projecting unit 10 projects a virtual image corresponding to the viewing distance onto the eyes of the user 1 based on the received image data and data regarding the viewing distance (step S360). Thereby, the image visually recognized by the user 1 is directly projected as a virtual image on the eyes of the user 1.

  On the other hand, when an input event for instructing private mode application occurs in step S330, the control unit 70 controls the image processing unit 72 to perform image conversion (for example, binarization processing) in real time (step binarization processing). S340).

  Then, the control unit 70 sends the image data (binary image data) image-processed by the image processing unit 72 and data regarding the viewing distance to the image projection unit 10 (step S350). Accordingly, the binary image is projected as a virtual image on the eyes of the user 1 (step S360).

  Further, the control unit 70 sends the image data sent to the image projection unit 10 and the data related to the viewing distance to the wireless communication unit 82 (step S355).

  The wireless communication unit 82 uses the Bluetooth (registered trademark) function to send image data and data regarding the viewing distance to the external wearable device (WD2) 102 and / or (WD3) 103 existing within the short-range wireless communication range. Transmit (step S370).

  Here, since the wearable device (WD2) 102 worn by the user 2 has the same configuration as the wearable device (WD1) worn by the user 1, based on the two received data, An image visually recognized by the user 1 is projected as a virtual image at the same viewing distance as the user 1 on the eyes of the user 2 wearing the wearable device (WD2). Therefore, the user 1 and the user 2 can virtually share the same outside scene image.

  In the present embodiment, the control method for selecting whether to perform image conversion processing by the user 1 is shown for the image data to be shared. However, the present invention is not limited to this. From the viewpoint of privacy protection and communication speed, it is desirable to use a binary image. In this case, whether or not a person is included in the photographed image is inspected using the image analysis function of the image processing unit 72 and photographed. If it is determined that a person is included in the image, it is possible to program to automatically perform image conversion processing regardless of a user instruction.

Thereafter, the control unit 70 confirms whether or not an input event for releasing the sharing mode is generated (step 380). If there is no input event for releasing the sharing mode, the process branches to before step S330, and the subsequent steps are repeated.
When an input event for canceling the sharing mode occurs, the control unit 70 controls to stop image data transmission to an external wearable device, branch before step S320, and repeat the subsequent steps. .

  FIG. 8 is a flowchart illustrating a control method of an image sharing system using a wearable device according to another embodiment of the present invention.

  Referring to FIGS. 8 and 6, first, when the user 1 wears the wearable device (WD1) 101, power is supplied, and the control unit 70 of the wearable device (WD1) 101 starts up using the supplied power. After completion, an image corresponding to the operation mode is output to the image projection unit 10. Thereafter, the control unit 70 confirms whether or not an input event due to a call from an external user is received by the wireless communication unit 82. For example, the Bluetooth (registered trademark) communication function is activated, and a search from an external wearable device is set to a possible state.

  When an input event is generated by a call from the external wearable device (WD2) 102 to the wireless communication unit 82 (step S410), the control unit 70 checks whether or not to respond to the corresponding call (step S420). .

  When the user 1 instructs not to answer the corresponding call, the control unit 70 controls the wireless communication unit 82 not to answer, returns to step S420, and repeats the subsequent steps.

  On the other hand, when the user 1 is permitted to answer the corresponding call in step S420, the control unit 70 controls the wireless communication unit 82 to establish a communication channel with the external wearable device (WD2) 102, Various data transmitted from the external wearable device (WD2) 102 is received.

  The control unit 70 of the wearable device (WD1) 101 analyzes the contents of various data transmitted from the external wearable device (WD2) 102 and performs control to execute corresponding processing (step S430).

  When image data is included in the various received data, the control unit 70 sends the received image data to the image projection unit 10. At the same time, the control unit 70 sends the received data relating to the viewing distance to the image projection unit 10 (step S440).

  The image projection unit 10 displays a virtual image at a position corresponding to the distance visually recognized by the user 2 wearing the external wearable device (WD2) 102 on the eyes of the user 1 based on the received image data and data on the viewing distance. Then, the image light is projected (step S450).

  In addition, when the received various data includes audio data, the control unit 70 sends the corresponding audio data to the audio processing unit 83 (step S460).

  The audio processing unit 83 converts the audio data into an analog signal and reproduces it to the ear of the user 1 through the earphone 40 (step S470).

Thereafter, the control unit 70 confirms whether or not a specific input event by the user 1 is generated. If there is no occurrence of an input event for ending communication with the wearable device (WD2) 102, control is performed so as to branch before step S430 and repeat the subsequent steps.
Further, when an input event for instructing the end of communication occurs, the control unit 70 controls to stop communication by the wireless communication unit 82, branch before step S420, and repeat the subsequent steps.

  FIG. 9 is a flowchart illustrating a control method of an image sharing system using a wearable device according to still another embodiment of the present invention.

  Referring to FIG. 9 and FIG. 6, first, when the user 1 wears the wearable device (WD1) 101 and operates the camera unit 20 to capture an external image (step S510), the control unit 70 is controlled by the user 1. It is confirmed whether or not a specific input event is generated for the touch switch 50, for example.

  When the input event is generated, the control unit 70 confirms whether or not the input event is an input event for selecting a mode for transferring a captured image via an external server (step S520). Here, the mode in which the captured image is transferred via the external server is a mode in which the image data captured by the user 1 is transferred to the wearable device (WD4) 104 existing outside the short-range wireless communication range. It is.

  If the input event for selecting the mode for transferring the captured image via the external server has not occurred, the control unit 70 returns to step S520 and controls to repeat the subsequent steps.

  On the other hand, when an input event for selecting a mode for transferring a captured image via an external server occurs in step S520, the control unit 70 proceeds to step S530 and confirms whether to apply the private mode to the image data.

When the private mode is not applied, the control unit 70 branches to step S550, and sends the captured image data and data related to the viewing distance to the image projection unit 10.
The image projection unit 10 projects a virtual image corresponding to the viewing distance onto the eyes of the user 1 based on the image data and the data related to the viewing distance (step S560). Thereby, the image visually recognized by the user 1 is projected as it is onto the eyes of the user 1.

  On the other hand, when an input event for instructing private mode application occurs in step S530, the control unit 70 controls the image processing unit 72 to perform image conversion (for example, binarization processing) in real time (step binarization processing). S540).

  Then, the control unit 70 sends the image data (binary image data) processed by the image processing unit 72 and data relating to the viewing distance to the image projection unit 10 (step S550). Accordingly, the binary image is projected as a virtual image on the eyes of the user 1 (step S560).

  Further, the control unit 70 sends the image data sent to the image projection unit 10 and the data related to the viewing distance to the wireless communication unit 82 (step S555).

  The wireless communication unit 82 connects to a nearby access point AP using a WiFi (registered trademark) function, and communicates with the wearable device (WD4) 104 and the external data server 300 via a communication network 400 such as an Internet line. Establish.

Next, wearable device (WD1) 101 transmits image data and data relating to the viewing distance to the external data server (step S570).
The external data server 300 stores the data transmitted from the wearable device (WD1) 101 in the storage unit 320, and the wearable device (WD4) 104 downloads the stored data from the server 300, thereby transferring the data. Is completed.
Here, wearable devices (WD1) 101, (WD4) 104, and external data server 300 each have a MAC address and communicate based on the TCP / IP protocol.

  Further, since the wearable device (WD4) 104 worn by the user 4 has the same configuration as the wearable device (WD1) worn by the user 1, the received image data and data relating to the viewing distance are included. Based on this, an image visually recognized by the user 1 is projected as a virtual image at the same viewing distance as the user 1 on the eyes of the user 4 wearing the wearable device (WD4). Therefore, the user 1 and the user 4 can virtually share the same outside scene image.

Thereafter, the control unit 70 checks whether or not a mode release input event to be transferred via the external data server is generated (step S580). If there is no occurrence of an input event for canceling the mode to be transferred via the external data server, control is performed so as to branch before step S530 and repeat the subsequent steps.
On the other hand, when an input event for canceling the mode to be transferred via the external server occurs, control is performed so that image data transmission to the external data server is stopped and branched before step S520, and the subsequent steps are repeated. .

  As mentioned above, although embodiment of this invention was described in detail, referring drawings, this invention is not limited to the above-mentioned embodiment, In the range which does not deviate from the technical scope of this invention, it changes variously. It is possible to implement.

DESCRIPTION OF SYMBOLS 10 Image projection part 11 Display part 12 Optical module 13 Distance measurement part 14 Prism 15 Correction lens 16 Projection distance adjustment part 20 Camera part 30 Microphone 40 Earphone 50 Touch switch 60 Case 70, 310 Control part 71 CPU
72 Image processing unit 73 Input / output control unit 81 Field of view fluctuation detection unit 82 Wireless communication unit 83 Audio processing unit 84, 320 Storage unit 85 Battery 90 Case 100, 101, 102, 103, 104 Wearable device 200 Short-range wireless communication is possible Within 300 range Data server 330 Communication unit 400 Communication network

Claims (7)

A wearable device that includes a display that is mounted on a user's head and allows the user to view an image of image light based on image data together with a visual field image of the outside world.
A camera unit that captures an external image visually recognized by the user;
A distance measuring unit that measures a visual distance that is a distance from the user's eye to the gazing point;
Image data obtained by photographing the external image viewed by the user based on image data obtained by photographing the external image visually recognized by the user by the camera unit and measurement data of the viewing distance of the user measured corresponding to the photographed image data. An image projection unit that projects the image light on the user's eyes as a virtual image visually recognized at the measured viewing distance of the user;
A storage unit for storing a program and various data for controlling the wearable device to function as a wearable device;
An image processing unit that performs real-time image processing on image data obtained by capturing an external image visually recognized by the user with the camera unit;
A wireless communication unit that transmits and receives data by forming a voice call channel, a character data communication channel, and an image data communication channel with an external device having a function of exchanging information between users ;
A control unit for controlling the operation of each of the components,
The image processing unit
And further comprising means for determining whether or not an image identified as a person is included in image data obtained by capturing an external image visually recognized by the user by the camera unit,
The controller is
The image data obtained by photographing the external image visually recognized by the user by the camera unit, when transmitting said at least one to the user specified in the external device via the wireless communication unit, and the captured by the image processing unit When it is determined that the image data includes an image that is identified as a person, the image processing unit automatically converts the captured image data into a low gradation that is difficult to identify in real time by the image processing unit. A wearable device that controls to convert to image data and transmit the image data to the external device.   The process of converting image data obtained by capturing an external image visually recognized by a user with the camera unit into low-gradation image data with the image processing unit is a binarization process or a ternary process. The wearable device according to Item 1.   When the external device is a wearable device having the same configuration as the wearable device worn by the user, the user's view measured corresponding to the image data together with image data obtained by capturing an external image visually recognized by the user by the camera unit. The wearable device according to claim 1, wherein distance measurement data is transmitted. A control method for a wearable device comprising a display that is mounted on a user's head and allows the user to view an image of image light based on image data together with a visual field image of the outside world.
Photographing an external image visually recognized by the user with the camera unit;
Measuring a viewing distance, which is a distance from the user's eye to the gazing point, by a distance measuring unit;
Based on the image data obtained by photographing the external image visually recognized by the user by the camera unit by the image projection unit and the measurement data of the viewing distance of the user measured corresponding to the photographed image data, the external image viewed by the user is obtained. Projecting image light from the captured image data onto the user's eyes as a virtual image viewed at the measured user viewing distance;
Transmitting image data obtained by capturing an external image visually recognized by the user by the camera unit to at least one designated by a user in an external device having a function of exchanging information between users via a wireless communication unit; Have
The image data obtained by photographing the external image visually recognized by the user by the camera unit, wherein the step of transmitting at least one designated by the user in the external device via the wireless communication unit,
Further comprising the step of determining whether or not an image identified as a person is included in the image data obtained by capturing an external image visually recognized by the user by the camera unit;
When it is determined that the photographed image data includes an image identified as a person,
The method of the wearable device, characterized in that it comprises the step of personally identifiable in real time and transmits the converted to the image data of the hard low-gradation image data obtained by automatically the shot regardless of the user's instruction by the image processing unit .   5. The process of converting image data obtained by capturing an external image visually recognized by the user with the camera unit into low-gradation image data is a binarization process or a ternary process. Wearable device control method. Transmitting the image data obtained by photographing the external image visually recognized by the user by the camera unit, at least one of the external device user to specify via the wireless communication unit,
When the external device is a wearable device having the same configuration as the wearable device worn by the user, the user's view measured corresponding to the image data together with image data obtained by capturing an external image visually recognized by the user by the camera unit. The wearable device according to claim 4, further comprising a step of transmitting distance measurement data. An image sharing system comprising a plurality of wearable devices according to claim 3, further comprising an external data server connected to the plurality of wearable devices,
The wearable device is:
Image data obtained by capturing an external image visually recognized by a user with a camera unit mounted on the wearable device, and measurement data of a user's viewing distance measured corresponding to the image data are transmitted to the external data server via a communication network. A wireless communication unit for transmitting and receiving,
The external data server is
A communication unit for transmitting and receiving data to and from the plurality of wearable devices;
A storage unit for storing data received from the wearable device;
Image data obtained by processing image data obtained by capturing an external image visually recognized by the user of the first wearable device transmitted from the first wearable device among the plurality of wearable devices by the image processing unit of the first wearable device And the measurement data of the user's viewing distance measured corresponding to the captured image data is received and stored in the storage unit, and in response to a request from a second wearable device among the plurality of wearable devices Corresponding to the image data processed by the image processing unit of the first wearable device and the captured image data of the image data captured by the user of the first wearable device stored in the storage unit and viewed by the user The measurement data of the measured viewing distance of the user is sent to the second wearable device that has made the request. Image sharing system, characterized in that it comprises a control unit, the to be transmitted to the scan.

Images