KR20190048241A - Wearable device and image processing method thereof - Google Patents

Abstract

A transmitting wearable device and an image processing method thereof are disclosed. The image processing method of the wearable device comprises the steps of: (a) photographing a real world that a user views through a front camera; (b) photographing eyes of the user through a rear camera to generate an eye image and checking gaze information for the user by using the eye image; (c) performing focus blur on the remaining region except a partial region according to the gaze information from the photographed image to generate an augmented reality image; and (d) overlapping the photographed image and the augmented reality image to output the overlapped image.

Description

[0001] Wearable device and image processing method thereof [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a wearable eyeglass device capable of being transparent and an image processing method thereof.

Transmissible wearable devices such as 'Google Glass', 'Epson Moverio', and 'Microsoft Hololens' are being actively developed. Through the transmission type eyeglass wearable device, the user can simultaneously view images reproduced in a real-world and a display device. Studies on augmented reality technology for matching images or information to a display device have recently been actively conducted.

When a user wearing a transmission type eyeglass device views a reproduced image on a display device, the real world beyond the glasses necessarily blends.

The present invention provides a wearable eyeglass device and a method of processing the same.

In addition, the present invention provides a wearable eyeglass device and its image processing method capable of superimposing an image photographed by a wearable device on a wearer's device so that a real-world scene viewed by a user is perceived in a different manner.

In addition, the present invention can be applied to a portable information terminal, which can focus on only a desired area or a user's point of view, and blur the remaining area according to a depth, A wearable device of the eyeglass type, and an image processing method thereof.

According to an aspect of the present invention, there is provided an image processing method of a wearable wearable device that is transparent.

According to an embodiment of the present invention, there is provided a method of processing an image of a wearable device, comprising the steps of: (a) photographing a real world viewed by a user through a camera; (b) generating a depth map for the photographed image; (c) generating an augmented image by focus blurring an area other than a partial area corresponding to a user's viewpoint or a user's selected position using the photographed image and depth map; And (d) superimposing the augmented image on the photographed image and outputting the augmented image.

The step (b) may be generated through a depth sensor or may be generated by depth estimation through stereo matching from a plurality of photographed images.

The step (c) includes the steps of: generating an augmented image by converting the photographed image according to a user's viewpoint; Generating a target blur map for the enhancement image using the depth map, the target blur map including a blur value for each pixel; And adjusting the focus by applying the target blur map to a region other than a partial region corresponding to the user's viewpoint or the user's selected position among the augmented images.

The blur value for each pixel is calculated using the following equation,

는 실세계 객체의 블러값을 나타내며,

는 객체의 깊이 값을 나타내고,

는 각 픽셀별 블러값을 결정하기 위한 함수를 나타낸다. here,

Represents a blur value of a real-world object,

Represents the depth value of the object,

Represents a function for determining a blur value for each pixel.

According to another embodiment of the present invention, there is provided a method of processing an image of a wearable device, comprising the steps of: (a) photographing a real world viewed by a user through a front camera; (b) capturing an eye of the user through a rear camera to generate a snow image, and confirming the user's gaze information using the snow image; (c) generating an augmented image by focus blurring a remaining region of the photographed image except a partial region corresponding to the visual information; And (d) superimposing the augmented image on the photographed image and outputting the augmented image.

The step (c) includes the steps of: converting the photographed image according to the visual information to generate an augmented image; Generating a target blur map for the photographed image by applying different blur values according to distances based on the gaze information, the target blur map including a blur value for each pixel box; And adjusting the focus by applying the target blur map to a region other than a partial region corresponding to the gaze information among the enhancement images.

According to another aspect of the present invention, there is provided a transmissive spectacles-type wearable apparatus.

According to an embodiment of the present invention, there is provided a wearable apparatus comprising: a camera for photographing a real world viewed by a user; A map generator for generating a depth map of the photographed image; An augmented image generating unit for generating an augmented image by focus blurring the remaining region excluding a partial region corresponding to a user's viewpoint or a user's selected position using the photographed image and depth map; And a display unit for superimposing the augmented image on the photographed image and outputting the superimposed image.

The augmented image generation unit generates a target blur map for the augmented image using the depth map after converting the photographed image according to a user's viewpoint to generate an augmented image, Wherein the object blur map includes a blur value for each pixel, wherein the blur value for each pixel is determined by applying a depth value of each pixel May include different blur values depending on the < / RTI >

According to another embodiment of the present invention, there is provided a wearable apparatus comprising: a front camera which photographs a real world viewed by a user; A rear camera for photographing the user's eyes and generating an eye image; A line-of-sight obtaining unit for obtaining line-of-sight information of the user using the eye image; An augmented image generation unit for generating an augmented image by focus blurring an area other than a part of the photographed image according to the sight line information; And a display unit for superimposing the augmented image on the photographed image and outputting the superimposed image.

Wherein the augmented image generating unit generates the augmented image by converting the photographed image according to the sight information and applying a blur value corresponding to the distance on the basis of the sight information, target blur map, and adjusts the focus by applying the target blur map to a remaining area of the augmented image except a part of the area according to the sight line information.

The wearable device and the image processing method of the present invention can be used to superimpose the images photographed by the wearable device according to the viewpoint of the user so that the real world scene viewed by the user can be perceived in a different manner have.

That is, according to the present invention, a user can focus on an area coinciding with a desired position or a user's view, and blur according to a depth of the remaining area, thereby concentrating only the area desired by the user, and making the undesired area invisible.

1 is a perspective view of a wearable device according to an embodiment of the present invention;
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic view showing an internal configuration of a wearable device according to an embodiment of the present invention; FIG.
2 is a flowchart illustrating an image processing method of a wearable device according to an embodiment of the present invention.
3 is a view illustrating an example of superimposing a video image of a real-world scene on a display of a wearable device according to an embodiment of the present invention.
FIGS. 4 and 5 are diagrams for explaining a correlation between a distance of an object and a focus blur according to an embodiment of the present invention; FIG.
6 illustrates an example of an experiment designed to determine refocusing parameters so that a refocusing effect can be felt from an actual superimposed image according to an embodiment of the present invention.
FIG. 7 is a flowchart illustrating an image processing method of a wearable device according to another embodiment of the present invention. FIG.
8 is a view schematically showing an internal configuration of a wearable device according to another embodiment of the present invention.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this specification, the terms " comprising ", or " comprising " and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps. Also, the terms " part, " " module, " and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

The present invention relates to a method and apparatus for generating a refocused image (hereinafter, referred to as an augmented image) suitable for a user's viewpoint using an image captured in a real-world scene and a depth map through a transparent eyeglass wearable device, It can be superimposed on the display and output. Accordingly, the present invention can concentrate only the specific area desired by the user and make the remaining areas invisible.

A wearable device according to an embodiment of the present invention is a transmissible eyeglass-type device, and is a device that a user can wear on his face. In order to facilitate understanding and explanation, the present invention is limited to a transmissible spectacular apparatus. However, it is natural that the present invention is equally applicable to a device capable of providing a user with a real-world image and an augmented image superimposed on each other. Although the present invention is limited to wearable devices, the present invention can be equally applied to not only wearable devices but also portable electronic devices.

A wearable device according to an embodiment of the present invention can mount a small camera (image sensor) on a frame of a wearable device to take a real-world image. The photographed image is processed through an internal processor, and then the real world image and the augmented image are superimposed over the wearable device and output through the display unit of the wearable device.

In addition, the wearable apparatus according to an embodiment of the present invention may include a camera on the front face and / or the rear face of the frame, respectively. Accordingly, the wearable apparatus can photograph the real world in the direction in which the wearer is looking through the camera provided on the front face of the frame. At the same time, the wearable apparatus can photograph the eyes of the user through the camera provided on the rear side of the frame, and then track the user's gaze to acquire gaze information. In addition, a battery for power supply may be incorporated in a part of the frame of the wearable device. Hereinafter, the internal structure of the wearable device will be described in more detail with reference to FIG.

FIG. 1 is a view schematically showing an internal configuration of a wearable device according to an embodiment of the present invention.

1, a wearable device 100 according to an exemplary embodiment of the present invention includes a camera 210, a display unit 215, a map generating unit 220, an augmented image generating unit 225, a memory 230, And a processor 235.

The camera 210 is a means for capturing an image.

For example, the camera 210 can capture an image in a direction in which the user wearing the wearable device 100 is looking.

In FIG. 1, it is assumed that there is only one camera 210, which is mainly described. However, the wearable device 100 according to an embodiment of the present invention may include a plurality of cameras. When there are a plurality of cameras as described above, the front camera 210 is located at the front of the frame of the wearable device 100, and can photograph an image in a direction in which the user views the camera. The rear camera 210 is located behind the front camera 210 and is located behind the frame of the wearable device 100. The rear camera 210 can be used to photograph a user's eyes and track a user's gaze.

The display unit 215 is a means for superimposing and displaying the image (real world image) photographed through the camera 210 through the control of the processor 235 and the augmented image.

The map generating unit 220 is a means for generating a depth map for the photographed image.

When the wearable device 100 includes a depth sensor, the map generation unit 220 may generate a depth map using the depth value obtained through the depth sensor. However, if the wearable device 100 does not have a depth sensor, a depth map may be generated using a plurality of images.

The augmented image generating unit 225 is means for generating an augmented image by focus blurring a remaining region except for a partial region corresponding to a user's viewpoint or a user's selected position using the photographed image and the depth map.

For example, the augmented image generation unit 225 generates a target blur map for the augmented image using the depth map after converting the photographed image according to the user's viewpoint, The object blur map may be applied to an area other than a partial area corresponding to the viewpoint of the user or the user's selection position in the augmented image to finally adjust the focus to generate an augmented image. This will be more clearly understood from the description of FIG.

The memory 230 is a means for storing various algorithms / commands necessary for superimposing and displaying an image photographed and augmented image in the wearable device according to an embodiment of the present invention, various data derived from the process, and the like.

The processor 235 may include means for controlling internal components (e.g., camera 210, display portion 215, memory 230, etc.) of the wearable device 100 in accordance with an embodiment of the present invention to be.

In addition, the processor 235 performs a plurality of commands stored in the memory 230 so as to focus on a part of the photographed image and blur the remaining area, thereby providing a real-world image and an augmented image in a superimposed manner can do. This will be described in more detail below with reference to FIG.

FIG. 2 is a flowchart illustrating an image processing method of a wearable device according to an embodiment of the present invention. FIG. 3 is an example of superimposing a video image of a real-world scene according to an embodiment of the present invention on a display of a wearable device. 4 and 5 are diagrams for explaining a correlation between a distance of an object and a focus blur according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an example of an experiment designed to determine refocusing parameters so that a refocusing effect can be felt from an actual superimposed image according to an example.

In step 310, the wearable device 100 photographs a scene viewed by a wearer wearing the wearable device 100 through a camera provided in the wearable device 100. [

In step 315, the wearable device 100 generates a depth map for the photographed image.

At this time, when the wearable apparatus 100 is provided with a depth sensor, a depth map for an image photographed through the depth sensor can be generated.

However, if the wearable device 100 is not provided with a depth sensor, the wearable device 100 may generate a depth map using a plurality of images for the same scene. For example, the wearable device 100 can generate a depth map using an algorithm such as stereo matching or depth from motion, which estimates depth from a plurality of images. Since the method of estimating the depth by using a plurality of images and generating the depth map is a known matter, a further explanation will be omitted.

In step 320, the wearable device 100 converts the image photographed by the camera to fit the viewpoint of the user. In this way, the image converted according to the user's viewpoint will be referred to as an augmented image in order to facilitate understanding and explanation.

In general, even if a scene is photographed through the wearable device 100 worn by a user, it is necessary to correct the viewpoint difference because the viewpoint of the user is different from the viewpoint of the camera lens.

3 shows an example of superimposing an image of a real world scene on a display of a wearable device. When the image of the real-world scene is superimposed on the display of the wearable device, the real-world scene and the augmented image can be matched when the position of the augmented image and color / brightness correction are accurate and the spatial resolution is sufficiently high.

According to an embodiment of the present invention, in the process of superimposing the real-world image photographed by the wearable device 100 at the user's viewpoint, the image using the image refocusing technique is superimposed to allow the user to recognize that the real world is refocused .

The correlation between the distance of the object and the focus blur will be described with reference to FIGS. 4 and 5. FIG. The camera focal length will be briefly described.

거리에 위치해 있는 경우, 렌즈를 거처 초점 거리(

)만큼 떨어진 이미지 평면(image plane)에 맺히게 된다. 이때, 도 4에서 보여지는 바와 같이 피사체의 거리가

보다 멀리 떨어지거나 도 5에서 보여지는 바와 같이 피사체의 거리가

보다 가까운 경우 객체는 이미지 평면상에 하나의 점으로 맺히지 않고 C를 반지름으로 갖는 착란원(circle of confusion)안에 맺히게 된다. 그로 인해 defocus blur가 발생된다. 이러한 블러의 정도는 객체와 제1 거리(

)가 클수록 커지는 특성이 갖는다. When the subject (object, point) moves from the lens (radius r)

If it is located at a distance,

) In the image plane (image plane). At this time, as shown in FIG. 4,

As shown in FIG. 5, the distance of the object

The object is not formed as a single point on the image plane but is formed in a circle of confusion with C as a radius. This causes defocus blur. The degree of blur is defined as the distance between the object and the first distance

) Is larger.

According to the basic principle of the lens, the focal length F can be expressed by Equation (1). Further, the radius (C) of the confusion circle based on the focal length F can be calculated as shown in Equation (2).

Where u denotes the distance between the object and the lens, v denotes the distance between the lens and the image plane, and F denotes the focal length of the lens.

는 렌즈와 이미지 평면까지의 거리를 나타내며, u는 렌즈와 객체까지의 거리를 나타낸다. 이러한 포커스 블러(focus blur)는 경우에 따라 도 6에서 보여지는 바와 같이 의도적으로 적용되어 미적 효과를 주거나 일부 영역에 집중을 위한 목적으로 활용이 가능하다.here,

Represents the distance from the lens to the image plane, and u represents the distance from the lens to the object. Such a focus blur may be intentionally applied as shown in FIG. 6 depending on the case, so that the focus blur may be used for aesthetic effect or for focusing on a certain area.

In an embodiment of the present invention, such a focus blur may be used to focus on a region where the user's gaze (viewpoint) is located or a specific region designated by the user.

Accordingly, the wearable device 100 according to an embodiment of the present invention can generate augmented image by converting the photographed image according to the user's viewpoint.

To this end, in an embodiment of the present invention, it is necessary to convert the image photographed by the camera to fit the viewpoint of the user. The user's viewpoint can be recognized by using a camera to capture a user's eye image and then tracking the gaze. When the image photographed according to the user's viewpoint is corrected, a pre-generated calibration matrix can be used.

In another example, the wearable device 100 may convert an image photographed according to a user's viewpoint using the generated depth map.

In step 325, the wearable device 100 generates a target blur map for the augmented image.

)를 포함한다. 이때, 목적 블러 맵은 사용자의 시점에 상응하여 자동으로 생성되거나 사용자의 입력을 통해 생성될 수 있다. Here, the target blur map is a blur parameter (

). At this time, the object blur map may be generated automatically according to the user's viewpoint or may be generated through the user's input.

=0), 나머지 영역은 깊이 값에 따라 블러 처리할 수 있다. For example, if the user recognizes the line of sight he or she is looking at and does not blur the position,

= 0), and the remaining area can be blurred according to the depth value.

Referring to FIG. 6, an example of an experiment designed to determine a refocusing parameter so that a refocusing effect can be sensed from an actually superimposed image will be described.

Referring to FIG. 6, it is possible to superimpose a refocused image on a clean object part of a real object's real object and a blurred object of a real world, and not to enhance an image on a real-world blurred object.

When the augmented image is superimposed on a clean object in the real world, it is possible to find a refocusing parameter that allows the same object to be recognized when looking at an actual blurred object (left / right in Fig. 6).

This can be expressed by the following equation (3).

는 실세계 객체의 블러값을 나타내며,

는 객체의 깊이 값을 나타내고,

는 각 픽셀별 블러값을 결정하기 위한 함수를 나타낸다. here,

Represents a blur value of a real-world object,

Represents the depth value of the object,

Represents a function for determining a blur value for each pixel.

))으로 표현될 수 있다. 다양한 깊이와 실세계 블러(

)에 대하여 획득된 증강 영상 블러(

) 값을 이용하여 함수를 생성할 수 있다. 또한, 매칭 함수 생성시 데이터 피팅(data fitting)을 이용하거나 기계 학습(machine learning) 기법이 이용될 수도 있다. In general, the blur is the standard deviation of the Gaussian blur

)). ≪ / RTI > Various depth and real-world blur (

) Obtained for the augmented image blur (

) Value can be used to create a function. In addition, data fitting or machine learning techniques may be used in generating the matching function.

The above-described object blur map may be generated automatically or may be generated through the user's input.

In step 330, the wearable device 100 uses the target blur map to re-adjust the focus by applying the target blur map to the rest of the regions other than the partial region corresponding to the user's viewpoint or the user's selected position in the augmented image.

) 나머지 영역은 깊이 값에 따라 블러값을 다르게 적용하여 블러 처리할 수 있다.For example, the wearable device 100 does not blur some areas according to the viewpoint of the user or the user's selected position

) The remaining area can be blurred by applying a different blur value according to the depth value.

Thus, according to an embodiment of the present invention, only the region of the photographed image that matches the viewpoint of the user can be emphasized and the remaining region can be blurred.

In another example, the wearable device 100 may process only a region that is important for the purpose of advertisement or the like in the photographed image, regardless of the viewpoint of the user, and blur the remaining region.

In step 335, the wearable device 100 can display the superimposed and displayed enhanced images.

7 is a flowchart illustrating an image processing method of a wearable device according to another embodiment of the present invention.

In step 810, the wearable device 800 shoots an image of a real-world scene viewed by the user through the front camera.

In step 815, the wearable device 800 photographs the eyes of the user through the rear camera and generates eye images.

In step 820, the wearable device 800 analyzes the eye image and recognizes the eyes of the user by recognizing the iris.

The method of tracking / recognizing the eyes through the iris recognition of a person is obvious to those skilled in the art, so a separate explanation will be omitted.

In step 825, the wearable device 800 generates an augmented image by focus blurring an area other than a partial area corresponding to the user's gaze among the photographed images.

For example, the wearable device 800 can generate augmented image by converting the photographed image to fit the recognized line of sight. Then, the wearable device 800 can generate the target blur map for the photographed image by applying the blur value differently according to the distance based on the line of sight.

That is, the wearable device 800 may generate the target blur map for the photographed image by applying the blur value so that the blur value increases as the distance increases, based on the position of the sight line (point of view) determined for the photographed image.

As described above, the wearable device 800 can adjust the focus by applying the target blur map to the remaining area excluding the partial area according to the sight line information in the augmented image after generating the target blur map.

In step 830, the wearable device 800 can superimpose and display the re-adjusted augmented image.

8 is a view schematically showing an internal configuration of a wearable device according to another embodiment of the present invention.

8, a wearable device 800 according to another embodiment of the present invention includes a front camera 910, a rear camera 915, a line acquiring unit 920, a display unit 925, 930, a memory 935, and a processor 940.

The front camera 910 is located in front of the frame of the wearable device 800 and can photograph a scene of a real world that the user sees.

The rear camera 915 is located behind the frame of the wearable device 800 and is a means for photographing the eyes of the user and generating eye images.

The line-of-sight acquiring unit 920 is means for acquiring line-of-sight information by recognizing the iris of the user using the eye image generated through the rear-view camera 915.

The display unit 925 is a means for superimposing and displaying the image (real world image) photographed through the camera (?) Through the control of the processor 940 and the augmented image.

The augmented-image generating unit 930 is a means for generating an augmented image by focus-blurring a remaining region of the photographed image except a partial region corresponding to the sighted information.

For example, the augmented image generation unit 930 generates an augmented image by converting the photographed image according to sight line information, and applies a blur value corresponding to the distance on the basis of the sight line information, A target blur map can be generated. Then, the augmented image generating unit 930 may adjust the focus by applying the target blur map to a remaining region of the augmented image except the partial region corresponding to the sight information.

The memory 935 is a means for storing various algorithms / commands necessary for superimposing and displaying the images photographed in the wearable device and the enhanced images according to an embodiment of the present invention, various data derived from the process, and the like.

The processor 940 may include internal components of the wearable device 800 (e.g., front camera 910, back camera 915, line of sight acquisition 920, display portion 920, 925), the augmented image generating unit 930, the memory 935, and the like).

FIG. 9 is a view showing an example of superimposing and displaying a re-adjusted augmented image on an image photographed with respect to a real-world scene through a wearable spectacle type wearable device according to an embodiment of the present invention. As shown in FIG. 9, by re-adjusting the focus to a user's viewpoint or a specific area desired by the user, the viewpoint of the user or a specific area desired by the user can be highlighted and displayed. In the present specification, only the user's viewpoint or the focus of a specific area desired by the user is readjusted and superimposed and output. However, the present invention can be similarly applied to a case where visual effects such as a different color are applied to the corresponding area.

The apparatus and method according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a computer-readable medium may be those specially designed and constructed for the present invention or may be known and available to those of ordinary skill in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The embodiments of the present invention have been described above. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (12)

An image processing method of a wearable apparatus,
(a) photographing a real world viewed by a user through a camera;
(b) generating a depth map for the photographed image;
(c) generating an augmented image by focus blurring an area other than a partial area corresponding to a user's viewpoint or a user's selected position using the photographed image and depth map; And
(d) superimposing the augmented image on the photographed image and outputting it.
The method according to claim 1,
Wherein the step (b) is performed through a depth sensor.
The method according to claim 1,
Wherein the step (b) is performed through depth estimation through stereo matching from a plurality of photographed images.
The method according to claim 1,
The step (c)
Generating an augmented image by converting the photographed image according to a user's viewpoint;
Generating a target blur map for the enhancement image using the depth map, the target blur map including a blur value for each pixel; And
And adjusting the focus by applying the target blur map to a region other than a partial region corresponding to the user's viewpoint or the user's selected position among the augmented images.
5. The method of claim 4,
Wherein the blur value for each pixel is calculated using the following equation.

here,

Represents a blur value of a real-world object,

Represents the depth value of the object,

Represents a function for determining a blur value for each pixel.
An image processing method of a wearable apparatus,
(a) photographing a real world viewed by a user through a front camera;
(b) capturing an eye of the user through a rear camera to generate a snow image, and confirming the user's gaze information using the snow image;
(c) generating an augmented image by focus blurring a remaining region of the photographed image except a partial region corresponding to the visual information; And
(d) superimposing the augmented image on the photographed image and outputting it.
The method according to claim 6,
The step (c)
Converting the photographed image according to the visual information to generate an augmented image;
Generating a target blur map for the photographed image by applying different blur values according to distances based on the gaze information, the target blur map including a blur value for each pixel box; And
And adjusting the focus by applying the target blur map to a remaining area of the augmented image except for a part of the area according to the sight line information.
8. A computer-readable recording medium having recorded thereon a program code for performing the method according to any one of claims 1 to 7.
In a wearable device,
A camera for photographing a real world viewed by a user;
A map generator for generating a depth map of the photographed image;
An augmented image generating unit for generating an augmented image by focus blurring the remaining region excluding a partial region corresponding to a user's viewpoint or a user's selected position using the photographed image and depth map; And
And a display unit for superimposing the augmented image on the photographed image and outputting the superimposed image.
10. The method of claim 9,
Wherein the augmented image generating unit comprises:
Generating a target blur map for the augmented image by using the depth map after generating the augmented image by converting the photographed image according to the user's viewpoint and generating a target blur map for the augmented image using the depth map, The focus is adjusted by applying the target blur map to a region other than a partial region according to a user's selection position,
Wherein the object blur map includes a blur value for each pixel and a different blur value depending on a depth value of each pixel.
In a wearable device,
A front camera that captures the real world the user is looking at;
A rear camera for photographing the user's eyes and generating an eye image;
A line-of-sight obtaining unit for obtaining line-of-sight information of the user using the eye image;
An augmented image generation unit for generating an augmented image by focus blurring an area other than a part of the photographed image according to the sight line information; And
And a display unit for superimposing the augmented image on the photographed image and outputting the superimposed image.
The apparatus of claim 11, wherein the augmented-
And generates a target blur map for the photographed image by applying a blur value corresponding to the distance on the basis of the gaze information, by converting the photographed image according to the gaze information to generate an augmented image, And adjusts the focus by applying the target blur map to a remaining area of the augmented image except for a part of the area corresponding to the sight line information.

Images