KR102334503B1 - Apparatus and method for reproducing multi-focused image - Google Patents

Abstract

A multifocal image reproducing apparatus is disclosed. A multifocal image reproducing apparatus according to an embodiment of the present invention includes: a photographing unit for photographing an image using a plurality of cameras; Eye-tracking sensor unit for detecting the user's gaze; A representative image is selected from among the images photographed by the camera according to a predetermined criterion, and one of the images photographed by each camera (one of a plurality of cameras or) is selected as a playback image based on the detection result of the eye tracking sensor unit. a control unit to select; and a display unit for outputting the representative image first, and sequentially outputting the reproduced image when the reproduced image is selected, wherein the photographing unit allows each camera to specify a different background or object, Focus and shoot.

Description

Multifocal image reproducing apparatus and method {APPARATUS AND METHOD FOR REPRODUCING MULTI-FOCUSED IMAGE}

An embodiment of the present invention relates to a multifocal image reproducing apparatus and method for providing an image of a focus suitable for a user's gaze.

The currently used video playback uses only a single-focused video. In the case of providing performance images of singers, images taken from various angles are provided using multiple cameras, but only one image captured at a specific angle is provided with a single focus.

However, according to the user's taste, there is a tendency to focus on various objects for each user and watch the video. For example, in the case of a video in which a group singer composed of a plurality of people performs a performance, the singer with interest may be different depending on the viewer.

1 is a diagram illustrating the prior art (refer to Korean Patent Application Laid-Open No. 10-2014-0033531, "Mobile terminal and its control method, and a recording medium therefor", 2014.03.19.) providing an image with a different focus. It is a drawing.

Referring to FIG. 1 , a method of obtaining a multi-focusing image having different focal lengths and providing an image having the maximum sharpness with respect to an object to which a user's touch input is applied is disclosed.

However, the prior art has a disadvantage in that the reaction speed is slow in that the user changes the focus through a touch using a hand.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a multifocal image reproducing apparatus and method that can respond more conveniently and quickly.

Multifocal image reproducing apparatus according to an embodiment of the present invention for achieving the above object includes: a photographing unit for photographing an image using a plurality of cameras; Eye-tracking sensor unit for detecting the user's gaze; A representative image is selected from among the images photographed by the camera according to a predetermined criterion, and one of the images photographed by each camera (one of a plurality of cameras or) is selected as a playback image based on the detection result of the eye tracking sensor unit. a control unit to select; and a display unit for outputting the representative image first, and sequentially outputting the reproduced image when the reproduced image is selected, wherein the photographing unit allows each camera to specify a different background or object, Focus and shoot.

Preferably, the photographing unit includes: an identification unit for identifying each of a background and an object in all images captured by each camera; a priority setting unit for setting priorities with respect to the identified background and objects; and an allocator for allocating a background or object to each camera based on the priority, wherein each camera focuses on the assigned background or object to photograph.

Preferably, the rank setting unit sets the rank based on the degree of movement of the background or object.

Preferably, the control unit analyzes the background or object toward which the user's gaze is directed based on the detection result of the eye tracking sensor unit, and converts the image of the camera captured by focusing on the background or object toward which the user's gaze is directed as a playback image. choose

Preferably, the photographing unit includes an image storage unit for storing an image for each camera, the stored image includes metadata related to a background or sharpness of an object, and the control unit is based on the detection result of the eye tracking sensor unit to analyze the object to which the user's gaze is directed, and select a stored image based on metadata regarding the sharpness of the object to which the user's gaze is directed.

Preferably, the images stored for each camera in the image storage unit are temporally synchronized and stored, and the display unit outputs the images by synchronizing the playback time.

A multifocal image reproducing method according to an embodiment of the present invention for achieving the above object includes: photographing an image using a plurality of cameras; detecting the user's gaze; selecting a representative image from among images photographed by a plurality of cameras according to a predetermined criterion, and selecting one of the images photographed by each camera as a reproduction image based on a detection result of the user's gaze; and outputting the representative image first, outputting the reproduced image sequentially when the reproduced image is selected, and photographing the image using the plurality of cameras, each camera specifies a different background or object, Shoot by focusing on a specific background or object.

Preferably, the step of taking an image by using the plurality of cameras may include: identifying each of a background and an object from all images captured by each camera; setting priorities for the identified backgrounds and objects; and allocating a background or object to each camera based on the priority, wherein each camera focuses on the assigned background or object to take a picture.

Preferably, the selecting of the representative image and the playback image is performed by analyzing a background or object to which the user's gaze is directed based on the user's gaze detection result, and focusing on the background or object to which the user's gaze is directed. Select the camera video as the playback video.

A multifocal image reproducing apparatus and method according to an embodiment of the present invention may detect a human gaze and provide an image in which a focus suitable for the human gaze is formed.

1 is a flowchart illustrating a conventional technique for providing another image.
2 is a block diagram illustrating a multifocal image reproducing apparatus according to an embodiment of the present invention.
3A to 3C are diagrams illustrating an operation method of a multifocal image reproducing apparatus according to an embodiment of the present invention.
4 is a block diagram illustrating a detailed configuration of the photographing unit of FIG. 2 .
5 is a block diagram illustrating a multifocal image reproducing apparatus according to another embodiment of the present invention.
6 is a flowchart illustrating a method for reproducing a multifocal image according to an embodiment of the present invention.
7 is a flowchart illustrating a step of selecting a representative image and a reproduced image according to an embodiment of the present invention.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating a multifocal image reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 2 , the multifocal image reproducing apparatus 100 according to an embodiment of the present invention includes a photographing unit 110 , an eye tracking sensor unit 120 , a control unit 130 , and a display unit 140 .

The photographing unit 110 includes a plurality of cameras. The plurality of cameras may face different directions, but they may all face the same direction. Even if they all face the same direction, multifocal images, that is, a plurality of images having different focal lengths, may be generated by each camera specifying different backgrounds or objects and focusing on the specified backgrounds or objects to take pictures. Here, the image includes a moving image and a still image (still shot).

The multifocal image reproducing apparatus 100 may be a smartphone. In the case of a smartphone, a plurality of cameras may be located on the back of the smartphone and all face in the same direction.

The eye tracking sensor unit 120 detects the user's gaze. By detecting the user's gaze, it is possible to analyze what object of the image the user is looking at.

The control unit 130 selects a representative image from among the images captured by the cameras according to a predetermined criterion, and based on the detection result of the eye tracking sensor unit 120 (one of the plurality of cameras or) images captured by each camera. Select one of them as the playback image. As the image selected by the controller 130, an image captured in real time by the camera or an image captured and stored may be selected.

A predetermined criterion for selecting the representative image may be preset to select one image from among a plurality of cameras or to select an image captured by a specific camera as the representative image. Alternatively, from among all the temporally synchronized and stored videos, an image captured by focusing on an object with the greatest movement in a specific playback time period may be selected as the representative image. Here, the specific playback time zone may correspond to a time period in which the playback of the video starts, that is, the first part of the video.

In addition, the reproduced image analyzes the object the user is looking at based on the detection result of the eye-tracking sensor unit 120, and selects the image captured with the focus on the object as the reproduced image.

The display unit 140 outputs the representative image first, and sequentially outputs the reproduced image when the reproduced image is selected.

When the multifocal image reproducing apparatus 100 starts to operate, the display unit 140 first plays a representative image, and at the same time, the eye tracking sensor unit 120 detects the user's gaze and analyzes the object viewed by the user. Thus, when the object viewed by the user is analyzed, an image of the object the user is looking at is outputted from then, and images of the moving object are sequentially output whenever the user's gaze is moved.

3A to 3C are diagrams illustrating an operation method of a multifocal image reproducing apparatus according to an embodiment of the present invention.

3A is a diagram illustrating a method in which the multifocal image reproducing apparatus 100 captures a multifocal image.

Referring to FIG. 3A , the photographing unit 110 of the multifocal image reproducing apparatus 100 includes four cameras 1 , 2 , 3 and 4 . The cameras 1 to 3 may capture an image with focus on the objects 1 to 3, respectively, and the camera 4 may generate a multifocal image by capturing a background.

3B and 3C are diagrams illustrating a method for a user to reproduce a multifocal image in the multifocal image reproducing apparatus 100 .

Referring to FIG. 3B , when the object 2 is observed in the reproduced image, the eye tracking sensor unit 120 detects this point, and the control unit 130 selects the image of the camera 2 photographed with the object 2 as the focus. let it play

And when the eye tracking sensor unit 120 detects that the user's gaze has moved to the object 1, the control unit selects and reproduces the image of the camera 1 photographed with the object 1 as a focus.

4 is a block diagram illustrating a detailed configuration of the photographing unit of FIG. 2 .

Referring to FIG. 4 , the photographing unit 110 includes an identification unit 112 , a ranking setting unit 114 , and an allocating unit 116 .

The identification unit 112 distinguishes and identifies each of the background and at least one object in all images captured by each camera.

One image may have multiple objects. For example, when the user is photographing three group singers performing on stage, when photographing each singer in an image, each singer may be identified and a background may be identified.

The priority setting unit 114 sets priorities with respect to the identified background and objects.

In the previous example, you can set priorities for three group singers and their backgrounds. The priority may be set according to the distance from the camera, or the priority may be set based on the degree of movement in a specific playback time among all videos synchronized and stored in time. Alternatively, it may be set by a user input such as a screen touch.

The allocator 116 allocates a background or an object to each camera based on the priority. Here, each camera takes a picture by focusing on the assigned background or object.

If the performance of the cameras included in one image reproducing apparatus 100 is different, it may be an important issue to assign which object to which camera. For example, you can assign a camera with high performance and high resolution by setting high priority to the most popular singer among group singers, and assign a lower priority to the singer with the highest performance or resolution when it is less important, such as a background. We will assign it to the falling camera.

5 is a block diagram illustrating a multifocal image reproducing apparatus according to another embodiment of the present invention.

Referring to FIG. 5 , the multifocal image reproducing apparatus 100 according to another embodiment of the present invention includes a photographing unit 110 , an eye tracking sensor unit 120 , a control unit 130 , and a display unit 140 . The unit 110 includes an image storage unit 118 . Here, the description of the components described above will be omitted and only the image storage unit 118 will be described.

The image storage unit 118 is included in the photographing unit 110 , and images captured by each camera are stored. In addition, the stored image may include metadata regarding the sharpness of the background or object.

Here, the information on the sharpness may be stored in synchronization with time. For example, it may include information about the sharpness of a corresponding object in a specific playback time period.

In addition, the controller 130 analyzes an object to which the user's gaze is directed based on the detection result of the eye tracking sensor unit 120 and selects a stored image based on metadata regarding the sharpness of the object to which the user's gaze is directed.

In addition, the stored image may be temporally synchronized and stored. In this case, the display unit 140 may output the image based on the synchronized time.

6 is a flowchart illustrating a method for reproducing a multifocal image according to an embodiment of the present invention.

2 and 6 , in step 610 , the photographing unit 110 captures an image using a plurality of cameras.

2 and 6 , in step 620 , the eye tracking sensor unit 120 detects the user's gaze.

2 and 6 , in step 630, the controller 130 selects a representative image from among images captured by a plurality of cameras according to a predetermined criterion, and based on the detection result of the user's gaze, Select one of the recorded images as the playback image.

Here, as another embodiment of the step of selecting the representative image and the playback image, the background or object to which the user's gaze is directed is analyzed based on the user's gaze detection result, and the background or object to which the user's gaze is directed is focused and photographed. You can select the video from the camera as the playback video.

Referring to FIGS. 2 and 6 , in step 640 , the display unit 140 outputs a representative image first, and sequentially outputs a reproduced image when a reproduced image is selected.

In the step of photographing an image using a plurality of cameras, each camera specifies a different background or object, and focuses on the specified background or object to photograph.

7 is a flowchart illustrating a step of selecting a representative image and a reproduced image according to an embodiment of the present invention.

The step of taking an image using a plurality of cameras,

4 and 7 , in step 710 , the identification unit 112 identifies a background and an object from all images captured by each camera.

4 and 7 , in step 720, the priority setting unit 114 sets a priority for the identified background and object.

4 and 7 , in step 730 , the allocator 116 allocates a background or an object to each camera based on priority.

Here, each camera takes a picture by focusing on the assigned background or object.

So far, the present invention has been looked at with respect to preferred embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (5)

In the multifocal video reproducing apparatus,
a photographing unit in which each of the plurality of cameras specifies a different background or object, and captures an image by focusing on the specified background or object;
an eye-tracking sensor unit positioned to face the user in the multifocal image reproducing apparatus facing the user and sensing the user's gaze;
a control unit that selects a representative image from among the images captured by the camera according to a predetermined criterion, and selects one of the images captured by the respective cameras as a playback image based on a detection result of the eye tracking sensor unit; and
A display unit for outputting the representative image first, and sequentially outputting the reproduced image when the reproduced image is selected,
the filming unit
an identification unit that identifies each of the background and the object in all images captured by each camera;
For the identified background and object, set according to the distance from the camera, or based on the degree of motion in a specific playback time among all videos stored in synchronization with time, or prioritize by user input including screen touch according to camera performance Rank setting unit to set; and
Allocating unit for allocating a background or object to each camera based on the priority; Including, each camera is to focus on the assigned background or object to shoot,
Including an image storage unit for storing images for each camera, the stored images include metadata about the clarity of the background or object,
The metadata may be stored in synchronization with time, and include information about the sharpness of the object in a specific playback time period,
the control unit
Analyzes an object to which the user's gaze is directed based on the detection result of the eye tracking sensor unit, selects a stored image based on metadata about the sharpness of the object to which the user's gaze is directed,
the display unit
Outputs the stored image based on the synchronized time,
The rank setting unit
Set the ranking based on the degree of movement of the background or object,
The image stored for each camera in the image storage unit is temporally synchronized and stored, a multifocal image reproducing apparatus. delete delete delete delete

Images