KR102086779B1 - Focus setting modul and method, virtual video playback system including the same - Google Patents

Abstract

The present invention relates to a virtual image reproducing system for inducing a user's gaze to a region intended by a director by automatically applying a focus effect to a 360 image when a predetermined requirement is satisfied in reproducing a 360 image. A storage unit for storing the 360 image source for which the execution requirement of the focus is set; a reproducing unit for reproducing the 360 image source stored in the storage unit for displaying the 360 image source on the display; It includes a focus applying unit for applying the focus to the 360 image source.

Description

FOCUS SETTING MODUL AND METHOD, VIRTUAL VIDEO PLAYBACK SYSTEM INCLUDING THE SAME}

The present invention relates to a focus setting module and a method, and a virtual video reproducing system including the same. More specifically, a director intends to automatically apply a focus effect to a 360 video when a predetermined requirement is satisfied in the playback of a 360 video. The present invention relates to a virtual image reproducing system that draws a user's attention to a region.

Since the advent of panoramic technology capable of generating 360 images, cameras capable of capturing 360 images, and head-worn displays (hereinafter referred to as VR devices), the content market for virtual images is rapidly developing.

The conventional 360 image content allows the viewer to experience a virtual space by giving the freedom to view the pre-rendered image according to the direction of the viewer.

Most 360 images are shot with Deep Focus, where objects can be seen clearly regardless of distance.

However, the degree of freedom of the 360 image showing the viewer's direction is not able to control the situation in which the viewer is looking in a different direction from the important scene intended by the director. I couldn't prevent the situation where I didn't understand the content flow because I didn't see facial expressions, major events, etc.).

On the other hand, US Patent 9292973 has been proposed a technique for adjusting the focus area on the image corresponding to the gaze by tracking the gaze of the user using the VR device. However, the US Patent 9292973 is because the focus is set in the direction of the user's gaze can not solve the problem that the audience does not see the main scene prepared by the conventional director.

In addition, U.S. Patent No. 9292973 uses a method of effecting or physically modifying an optical component separately configured in addition to a display to adjust focus according to a user's gaze. However, this method has complicated the structure of the VR device and the system for controlling the VR device, which causes a problem that the weight or volume of the VR device increases or the price of the VR device increases.

Published Patent Publication 10-2017-0122688

Accordingly, the present invention has been proposed to solve the conventional problems as described above, and an object of the present invention is that the director intends to automatically apply a focus effect to a 360 image when a predetermined requirement is satisfied in the reproduction of the 360 image. An object of the present invention is to provide a virtual image reproducing system that draws a user's attention to one area.

In order to achieve the above object, the focus setting module according to the technical spirit of the present invention includes a focus area setting unit for setting a focus area and an execution requirement of focus in a 360 image source; And a focus effect setting unit for setting a detail effect including the type and intensity of the focus.

In addition, the execution requirement of the focus may include a time setting to which the focus is applied during a playback time of a 360 image source or an execution region setting to which the focus is applied when the user's field of view enters.

In addition, the focus setting method according to the technical idea of the present invention to achieve the above object comprises the steps of setting the focus area and the execution requirements of the focus in the 360 image source; The focus effect setting unit may include setting a detailed effect including the type and intensity of the focus.

In addition, in order to achieve the above object, the virtual image reproducing system according to the technical concept of the present invention includes a storage unit for storing a 360 image source in which a focus area and a focus execution requirement are set; A playback unit for playing back the 360 image source stored in the storage unit such that the 360 image source is displayed on a display; And a focus applying unit configured to apply the focus to the 360 image source reproduced when the execution requirement of the focus is satisfied.

In addition, the execution requirement of the focus may include a time setting to which the focus is applied during a playback time of a 360 image source or an execution region setting to which the focus is applied when the user's field of view enters.

The apparatus may further include a focus setting module configured to set the focus area, an execution requirement of the focus, and a detail effect in the 360 image source.

The display unit may further include an inquiry unit including a display unit displaying the 360 image source reproduced by the reproduction unit and a view tracker which tracks the viewing direction of the user.

The focus may be gradually applied from an area farthest from the focus area of the 360 image source to an edge of the focus area.

According to a focus setting module and method according to the present invention, and a virtual video reproduction system including the same,

First, the 360 effect allows the viewer to naturally emphasize the subject of interest through the focus effect in the 360 image where the viewer can view the image in the desired direction, allowing timely information and storytelling in the 360 image.

Second, as the audience's understanding of content is improved by the focus effect of the 360 image, the viewer can be more immersed in the 360 image content.

1 is a block diagram showing a focus setting module and a virtual video playback system according to a first embodiment of the present invention.
2 is a block diagram showing a virtual video playback system according to a second embodiment of the present invention.
3 is a block diagram showing a virtual video playback system according to a third embodiment of the present invention.
4 is a flow chart of a focus setting method according to an embodiment of the present invention.
5 is a flowchart of a virtual image playback method according to an embodiment of the present invention.
6 is an exemplary diagram of a 360 image source that is a video.
7 is a view showing that a focus area and a focus application time are set in a 360 image source which is a video.
8 is a view showing that a plurality of focusing times are set on a 360 image source which is a moving picture, a 360 image source is played, and a defocus effect is applied to an image when a playback time reaches a focusing time.
9 is an illustration of a 360 image source in a photographic state.
10 is a view showing that a focus region and an execution region are set in a 360 image source in a photo state.
FIG. 11 is a view illustrating a state in which a viewer's view of a 360 image source in a photo state is located outside the execution region and thus no focus effect is applied to the 360 image source.
12 is a view illustrating a state in which a field of view of a viewer watching a 360 image source in a photographic state is located in an execution region and an out focus effect is applied to the 360 image source.

With reference to the accompanying drawings will be described in detail with respect to the focus setting module and method according to embodiments of the present invention, and a virtual image playback system including the same. As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific disclosed form, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

In addition, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

The 360 image source may include content such as dynamic movies, dramas, animations, and games in addition to static pictures and photographs. The 360 image source can be viewed using a flat-panel TV or monitor device or a VR device worn on the head.

Optically, the focus refers to a point where a straight line extending the ray after passing through meets the optical axis when a ray parallel to the optical axis is incident on the optical system. As the image is blurry in the unfocused area, the viewer pays attention to the focus area that is clearly visible.

Instead of applying the focus effect to the 360 image source optically, the embodiment of the present invention applies a focus-like effect through post-processing to a 360 image source already taken in a deep focus state.

It is to be understood that the area to which the focus set in the following description is not applied only to the corresponding area, but most other areas located outside the area are blurred.

1 is a block diagram illustrating a focus setting module 140 and a virtual image reproducing system 100 according to a first embodiment of the present invention.

Referring to FIG. 1, the first embodiment of the present invention is characterized in that the focus setting module 140, the virtual image reproducing system 100, and the inquiry unit 160 are separately configured and electrically connected to each other.

The focus setting module 140 according to the first embodiment of the present invention includes a focus area setting unit 142 for setting a focus area and execution requirements of a focus in a 360 image source, and a detail effect of focus including the type and intensity of the focus. It includes a focus effect setting unit 144 for setting.

The focus area is set in the area intended by the director in the 360 image. For example, the focus area may be one in which focus is set on the face area of the actor to emphasize the emotional state of the actor. The focus area setting may be performed by designating an area to be clearly seen in the 360 image source. 7 and 10 are performed by selecting only the focus area as an ellipse, it may also include a function for accurately selecting the outline of the subject.

In addition, a plurality of focus areas may be set in one image. In-focus and out-focus using a physical optical system cannot focus on subjects having different distances from the photographing apparatus together, but when applying a focus effect through post-processing as in this embodiment, simultaneously to subjects having different distances It can give a focus effect.

The execution requirement of the focus includes setting the time when the focus is applied during the reproduction time of the 360 image source when the 360 image source is a moving image. A plurality of focuses different from each other during the playback time may be set in the 360 image source configured as a moving image (see FIGS. 6 to 8).

In addition, the execution requirement of the focus includes setting an execution region to which the focus is applied when the viewer's view is entered when the viewer's view of the 360 image source can be tracked (see FIGS. 9 to 12). When setting the execution region, the time requirement for the viewer's view to stay in the execution region may also be set in order for the focus effect to be applied. For example, the viewer may have a visual field of more than 2 seconds to apply the focus effect.

The type of focus set by the focus effect setting unit 144 is a form of a blur effect formed inside or outside the focus area due to the application of the focus. The type of focus is largely divided into an focus that blurs the outside of the selected area and an infocus that blurs only the inside of the selected area (both FIGS. 8 and 12 are out of focus). It is also possible to form a patterned blur in detail. The pattern of the blur may include a blur having a weaker or stronger intensity of the blur, a blur having a wavy pattern, and the like closer to the edge of the focus area.

The intensity of the focus is the intensity of the blur effect. The stronger the focus is, the darker the image of the blur is processed.

In addition, the focus effect setting unit 144 may set a range of focus. The focus range is a range in which blur processing is performed. In addition to the focus area, blur processing, blur processing of only a separately selected area, and blur processing up to a distance selected from the edge of the focus area may be selected.

In addition, the focus effect setting unit 144 may set a condition for removing focus. The focus may be set to remove the focus n seconds after the application of the focus, or to remove the focus if the viewer's field of view is out of the execution region.

In addition, the focus effect setting unit 144 may also set the focus animation. The focus animation is to animate the focus at the moment when the focus is applied to the 360 image source, when the focus is displayed, and when the focus is removed. For example, the focus may be gradually applied from the farthest area to the edge of the focus area among the 360 image sources.

Meanwhile, the virtual image reproducing system 100 according to the first embodiment of the present invention includes a storage unit 122 storing a 360 image source in which a focus area and a focus execution requirement are set, and a 360 image source to be displayed on a display. And a reproducing unit 124 for reproducing the 360 image source stored in the storage unit 122 and a focus applying unit 126 for applying the focus to the reproducing 360 image source when the execution requirement of the focus is satisfied.

The storage unit 122 of this embodiment stores a 360 image source that is focused using the focus setting module 140 of this embodiment. The 360 image source is preferably stored in a file format including data such as an image and a video. The file format may be a conventionally known format such as * .jpg, * .gif or * .avi.

The playback unit 124 plays back the file of the 360 image source stored in the storage unit 122. When the 360 image source is an image or a video, the corresponding file is loaded and reproduced in a manner that can be projected from the display device. If the 360 video source is content that can be dynamically controlled by the audience, such as an electronic game, the ability to run an electronic game application is also included.

The focus applying unit 126 applies focus to the 360 image source that is being played when the execution requirements of the focus are satisfied, such as reaching the time when the focus is applied during the playback of the 360 image source and entering the view of the viewer in the execution region to which the focus is applied. . The focus applied to the 360 image source is applied according to the detail effect of the focus set by the focus effect setting unit 144.

In addition, the focus applying unit 126 removes the focus applied to the 360 image source when the focus removal condition is satisfied.

The focus effect setting unit 144 and the focus applying unit 126 may be implemented by using a game engine such as unity or unreal.

2 is a block diagram showing a virtual video playback system 100 according to a second embodiment of the present invention. The virtual image reproducing system 100 according to the second embodiment of the present invention further includes a focus setting module 140 for setting a focus area, an execution requirement of a focus, and a detail effect of a focus in a 360 image source. The focus setting module 140 of the second embodiment has the same function as the focus setting module 140 of the first embodiment.

3 is a block diagram showing a virtual video playback system 100 according to a third embodiment of the present invention. In addition to the focus setting module 140, the virtual image reproducing system 100 according to the third embodiment of the present invention further includes an inquiry unit 160 that enables the viewer to watch a 360 image source reproduced by the reproduction unit 124. do. The inquiry unit 160 may be a flat panel TV or monitor device or a VR device worn on the head. In particular, when the inquiry unit 160 is implemented as a VR device, the inquiry unit 160 may display the display unit 162 on which the 360 image source played by the playback unit 124 is displayed, and the field of view that tracks the viewing direction of the viewer. The tracking unit 164 is included.

 Tracking of the viewing direction of the viewer may be performed in various ways. One method may be to track the head movement of the audience to identify the direction the front of the head is facing. Alternatively, by tracking the head posture of the viewer and the position of the pupil of the viewer, the position of the field of view viewed by the audience, or the direction of the field of view may be identified.

Next, a focus setting method and a virtual image reproduction method according to an embodiment of the present invention will be described.

4 is a flowchart illustrating a focus setting method according to an embodiment of the present invention. Referring to FIG. 4, in the focus setting method of this embodiment, a 360 image source is first loaded into the focus setting module 140 (S141), and the focus area setting unit 142 executes the focus area and the focus on the 360 image source. Setting requirements (S142), the focus effect setting unit 144 includes the step of setting the detailed effect of the focus, including the type of focus, the intensity (S144).

5 is a flowchart of a virtual image playback method according to an embodiment of the present invention. Referring to FIG. 5, in the virtual image reproducing method of the present embodiment, the reproducing unit 124 reproduces the 360 image source stored in the storage unit 122 (S122), and the focus applying unit 126 corresponds to the focus execution requirement. Monitoring (S124), if the focus execution requirements, the focus applying unit 126 includes applying a focus effect to the 360 image source (S126).

Focus setting and application processes in the case where the 360 image source is a video and the execution requirement of the focus is the time setting to which the focus is applied during the playback time of the 360 image source will be described with reference to the drawings. Referring to FIG. 6, a 360 image source, which is a video, is loaded into the focus setting module 140. Referring to FIG. 7, the focus area setting unit 142 sets a focus application time which is a requirement for applying a focus area and focus. At this time, the focus effect setting unit 144 also sets the detail effect of the focus. Referring to FIG. 8, when the playback unit 124 plays a 360 image source and the playback time coincides with the focus application time, the focus application unit 126 applies the focus effect to the preset focus area.

In addition, the focus setting in the case where the query unit 160 includes the field tracker 164 and the execution requirement of the focus is the execution region setting to which the focus is applied when the viewer's field of view enters, and The application process will be described based on the drawings. Referring to FIG. 9, a 360 image source is first loaded into the focus setting module 140. Referring to FIG. 10, the focus area setting unit 142 sets a focus area and an execution area that is an execution requirement for focus. At this time, the focus effect setting unit 144 also sets the detail effect of the focus. Referring to FIG. 11, when the playback unit 124 plays the 360 image source, if the viewer's field of view is out of the execution region, the focus effect is not applied. Referring to FIG. 12, when the viewer's field of view enters the execution region, a focus effect is applied.

Although preferred embodiments of the present invention have been described above, the present invention may use various changes, modifications, and equivalents. It is clear that the present invention can be applied in the same manner by appropriately modifying the above embodiments. Therefore, the above description does not limit the scope of the present invention as defined by the limitations of the following claims.

100: virtual video playback system 122: storage unit
124: playback unit 126: focus application unit
140: focus setting module 142: focus area setting unit
144: focus effect setting unit 160: inquiry unit
162: display unit 164: field tracking unit

Claims (8)

A focus area setting unit for setting a focus area and an execution requirement of focus in the 360 image source;
It includes a focus effect setting unit for setting a detail effect including the type and intensity of the focus,
The execution requirement of the focus includes a time setting to which the focus is applied during a playback time of a 360 image source or an execution region setting to which the focus is applied when the viewer's view enters, and the focus is the focus among the 360 image sources. The focus setting module is gradually applied to the edge of the focus area from the area farthest from the area, and the focus area is set in the area intended by the director among 360 images. delete Setting, by the focus area setting unit, a focus area and an execution requirement of focus in the 360 image source;
A focus effect setting unit sets a detailed effect including the type and intensity of the focus;
In setting the execution requirement of the focus, the execution requirement of the focus includes a time setting to which the focus is applied during a playback time of a 360 image source or an execution region setting to which the focus is applied when the viewer's view enters, Focus is gradually applied to the edge of the focus area from the area farthest from the focus area of the 360 image source, the focus area is set to a region intended by the director of the 360 image. A storage unit for storing a 360 image source having a focus area and an execution requirement for focus;
A playback unit for playing back the 360 image source stored in the storage unit such that the 360 image source is displayed on a display;
And a focus applying unit configured to apply the focus to the 360 image source reproduced when the execution requirement of the focus is satisfied.
The execution requirement of the focus includes a time setting to which the focus is applied during a playback time of a 360 image source or an execution region setting to which the focus is applied when the viewer's view enters, and the focus is the focus among the 360 image sources. The image is gradually applied from the farthest region to the edge of the focus region, wherein the focus region is set in the region intended by the director of the 360 image. delete The method of claim 4, wherein
And a focus setting module configured to set the focus area, an execution requirement of the focus, and a detail effect in the 360 image source. The method of claim 6,
And an inquiry unit including a display unit for displaying the 360 image source reproduced by the reproduction unit and a view tracking unit for tracking the viewing direction of the viewer. delete

Images