KR20160022329A - Apparatus and method for navigation digital object using gaze information of user - Google Patents

Abstract

In the following, disclosed are an apparatus and a method for searching a digital object. The method for searching the digital object according to an embodiment of the present invention may include the steps of: collecting eye gaze information and gesture information of a user; enabling or disabling searching of the digital object on the basis of the gesture information; determining a current block on the basis of the eye gaze information, in case searching of the digital object is enabled; comparing the current block with a prior block; determining at least one of information quality of the current block and a zoom rate as to zoom-in of the current block on the basis of eye gaze maintaining time, in case the current block is equal to the prior block; and determining at least one of the zoom rate as to zoom-out to the current block from the prior block, the information quality of the current block, and the information quality of the prior block, in case the current block is different from the prior block.

Description

TECHNICAL FIELD [0001] The present invention relates to a digital object search apparatus and method using a user's gaze information,

An embodiment of the present invention relates to a digital object search apparatus and method, and more particularly, to a digital object search apparatus for zooming in / zooming out according to a user's sight line information, performing quality rendering, ≪ / RTI >

Recently, with the development of digital devices, users use digital objects such as photographs, movies, and music very frequently. In general, a user can process digital objects using a user interface such as a keyboard or a mouse. However, in the case of a large wall display in which a user can not input a desired command through a keyboard, a mouse, or the like, it is difficult to process a digital object. Also, in the case of a network-based digital TV such as IPTV, it is difficult for a user to use a widget for use of EPG and news / weather / traffic information / financial information in hundreds or more of channels using a remote controller.

In order to solve such a problem, a method of adjusting the real-time output quality of a digital object by tracking a user's behavior and reflecting it is attracting attention. However, a technique for efficiently performing such a method has not yet been developed.

A digital object searching method comprising: collecting gaze information and gesture information of a user; Activating or deactivating a search for a digital object based on the gesture information; Determining, if the search of the digital object is activated, a current block based on the sight line information; Comparing the current block with a previous block; Determining at least one of a zoom rate for a zoom in of the current block and an information quality of the current block based on a line retention time when the current block and the previous block are identical; And determining at least one of a zoom ratio for zooming out from the previous block to the current block, an information quality of the current block, and an information quality of the previous block when the current block and the previous block are different from each other A method of searching for a digital object can be provided.

The digital object includes a plurality of blocks, the current block corresponds to a block currently viewed by the user, and the previous block may correspond to a block previously viewed by the user.

Wherein the information quality of the current block includes at least one of a rendering quality of the current block and an additional information intensity of the current block, and the information quality of the previous block includes at least one of a rendering quality of the previous block, And may include at least one.

A digital object searching method comprising: collecting gaze information and gesture information of a user; Activating a search for a digital object based on the gesture information; Determining a current block based on the line of sight information; Comparing the current block with a previous block; And determining at least one of a zoom ratio for zooming in of the current block and an information quality of the current block if the current block is identical to the previous block.

A digital object searching method comprising: collecting gaze information and gesture information of a user; Activating a search for a digital object based on the gesture information; Determining a current block based on the line of sight information; Comparing the current block with a previous block; And determining at least one of a zoom ratio for zooming out from the previous block to the current block, an information quality of the current block, and an information quality of the previous block when the current block and the previous block are different from each other A method of searching for a digital object can be provided.

The digital object includes a plurality of blocks, the current block corresponds to a block currently viewed by the user, and the previous block may correspond to a block previously viewed by the user.

An apparatus for searching digital objects, comprising: an information collecting unit for collecting gaze information and gesture information of a user; An object search unit for activating or deactivating a search for a digital object based on the gesture information; And when the search of the digital object is activated, determining a current block based on the visual information, comparing the current block with a previous block, and if the current block and the previous block are identical, A zoom ratio for zooming in on the current block and an information quality of the current block are determined, and when the current block is different from the previous block, a zoom for zooming out from the previous block to the current block, The information quality of the current block, and the information quality of the previous block, based on the information quality of the current block.

The digital object includes a plurality of blocks, the current block corresponds to a block currently viewed by the user, and the previous block may correspond to a block previously viewed by the user.

Wherein the information quality of the current block includes at least one of a rendering quality of the current block and an additional information intensity of the current block, and the information quality of the previous block includes at least one of a rendering quality of the previous block, And may include at least one.

According to an embodiment of the present invention, a natural Zoomable User Ineration can be provided without a separate additional device by searching a digital object by determining a zoom ratio for zooming in / zooming out through the user's sight line information and gesture information .

According to an embodiment of the present invention, by improving the rendering quality at the current block center through the user's gaze information, it is possible to increase the output quality of the digital object around the user's gaze and improve the immersion feeling and usability of the digital object have.

According to an embodiment of the present invention, the user can acquire the information of the component displayed on the digital object without additional equipment by providing additional information previously stored in the digital object through the user's sight line information.

According to an embodiment of the present invention, it is possible to adjust the position of an object (channel or widget) according to the sight line information of a user, output a small area close to a user, have. According to an embodiment of the present invention, when viewing a screen in a diagonal direction on a large screen, it is difficult to secure a view, so that the output UI can follow the user's position and output.

1 is a view for explaining the operation of a digital object search apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating a detailed configuration of a digital object according to an exemplary embodiment of the present invention.
3 is a flowchart illustrating an example of performing zoom in / zoom out operations for a digital object based on eye line information according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating an example of zooming in / zooming out a digital object based on gaze information and gesture information according to an embodiment of the present invention.
5 is a diagram illustrating an example of searching for a digital object by zooming in or zooming out according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating an example of searching for a digital object according to blocks in accordance with a rendering quality, based on visual line information according to an exemplary embodiment of the present invention.
FIG. 7 is a diagram illustrating an example of determining a rendering quality for a digital object differently for each block according to an exemplary embodiment of the present invention. Referring to FIG.
8 is a flowchart illustrating an example of searching for a digital object by calculating the intensity of additional information for a digital object based on visual line information according to an embodiment of the present invention.
9 is a diagram illustrating an example of calculating the intensity of additional information according to a user's gaze position according to an embodiment of the present invention.
10 is a diagram illustrating an example in which additional information is displayed according to a user's line of sight according to an embodiment of the present invention.
11 is a diagram illustrating an example of searching for a channel according to a user's gaze position according to an embodiment of the present invention.
12 is a diagram illustrating an example of determining a user interface according to a location of a user according to an exemplary embodiment of the present invention.
13 is a diagram illustrating an example in which a user interface according to an embodiment of the present invention is output according to a user's location.
14 is a flowchart illustrating an example of changing an output position and an output angle of a digital object according to a user's position information and visual information according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. Like reference symbols in the drawings denote like elements. A digital object search method according to an embodiment of the present invention can be performed by a digital object search apparatus.

1 is a view for explaining the operation of a digital object search apparatus according to an embodiment of the present invention.

The user 100 using various digital objects such as photographs, moving pictures, music, etc., can track the gaze information 103 and the gesture information 104 such as a hand gesture by using the digital object searching apparatus 101 according to an embodiment of the present invention. So that the output quality of the digital object around the visual line can be increased. Then, the user 100 can improve the immersion feeling and usability with respect to the digital object, and the load on the processing of the digital object can be reduced. The digital object search device 101 may be located inside or outside the display 102.

The digital object search apparatus 101 according to an embodiment of the present invention can perform the following operations.

(1) Visual Attentive Zoomable User Interaction (ZUI)

When the user 100 reproduces the digital object, the digitalobject search apparatus 101 zooms in on the point where the user 100 is located in the digital object, and zooms out Out.

(2) Visual Attentive Rendering

When the user 100 reproduces the digital object, the digitalobject search apparatus 101 renders the resolution of the point at which the user 100 is located in the digital object at High Quality and renders it at Normal Quality when the line of sight is out of the digital object have. When the user 100 reproduces the visualized music, the volume and the decoding quality of the spot where the user 100 is located are increased on the music reproduction screen. When the user 100 is out of sight, Quality can be reduced.

(3) Visual Attentive Annotation

When the user 100 reproduces the digital object, the digitalobject search apparatus 100 can output the additional information adjacent to the point where the user 100 is located in the digital object. For example, the digital object search apparatus 100 may output additional information in the form of images, audio, or text stored at a point adjacent to a point where the user 100 is located.

(4) Motion Parallax UI

The position and direction of the user interface object can be adjusted to output a view such that the user 100 views the actual 3D object. According to the position of the user 100, it is possible to provide an optimized view to the user 100 by adjusting the projection of the user interface (UI) so that a small area near the screen and a large area far are displayed. Or when the user 100 sees a large screen in a diagonal direction and the widget is displayed on the opposite side of the user 100 on the large screen, the user 100 has difficulty in securing a view for the widget, The output UI can also be moved and output.

2 is a block diagram illustrating a detailed configuration of a digital object according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the digital object search apparatus 101 may include an information collecting unit 201, an information determining unit 202, and an object searching unit 203.

The information collecting unit 201 may collect the user's sight line information 209 and the user's gesture information 210 with respect to the digital object 208. [ The gaze information may include a user's gaze position on a 2D screen, blinking of the eyes, and user's position information. At this time, the digital object 208 may be divided into a plurality of blocks. The user's line of sight changes very rapidly. In this case, errors may increase in the line of sight information 209 collected by the information collection unit 201. Accordingly, the digital object search device 101 can filter the line of sight of the user, which is finely shaken by dividing the digital object 208 into a plurality of blocks, block by block. A block ID may be assigned to each of a plurality of blocks.

The information determination unit 202 can determine the search information for the digital object 208 using the collected sight line information 209 and the gesture information 210. [ For example, the information determination unit 202 may include a zoom ratio calculation unit 204, a rendering quality calculation unit 205, an additional information intensity calculation unit 206, and a motion parallax calculation unit 207.

The zoom ratio calculation unit 204 can calculate the zoom ratio for the digital object 208 using the sight line information 209 and the gesture information 210. [ More specifically, the zoom ratio calculation unit 204 calculates a zoom ratio for zooming in or zooming out of the digital object 208 based on the ID of the block included in the line of sight information 209 and the eye- Can be calculated.

In this case, if the IDs of the blocks to be examined by the user during the gaze hold time are the same, the zoom ratio calculation unit 204 can calculate the zoom ratio for the zoom in of the digital object 208 using the gaze hold time. When the ID of the block to be examined by the user is different during the eye-gaze holding time, the zoom-ratio calculating unit 204 may calculate the zoom ratio for zooming out the digital object 208 using the eye-gaze holding time.

For example, the zoom ratio calculation unit 204 may calculate a zoom ratio for zooming in the digital object 208 when the eye-gaze holding time exceeds a preset threshold time.

The rendering quality calculator 205 may calculate the rendering quality for the digital object 208 using the user's sight line information 209 and the gesture information 210. [ For example, the rendering quality calculator 205 may calculate the rendering quality of the digital object 208 based on the ID of the block included in the line of sight information and the line retention time at which the user strikes the block. The rendering quality can be determined to decrease around the point where the user's gaze is located.

If the IDs of the blocks to be looked at by the user are the same during the eye retention time, the rendering quality calculator 205 can calculate the rendering quality for the digital object 208 using the eye retention time. On the other hand, if the ID of the block to be looked at by the user is different during the eyes-keeping time, the rendering quality calculator 205 can initialize the digital object 208.

Specifically, if the IDs of the blocks to be looked at by the user during the gaze hold time are the same, the rendering quality calculator 205 increases the rendering quality of the block with respect to the eye-gaze-keeping time and decreases the rendering quality with respect to the neighboring blocks of the block . At this time, the rendering quality calculation unit 205 may sequentially reduce the rendering quality from the neighboring block adjacent to the block to be examined by the user.

The additional information intensity calculation unit 206 may calculate the intensity of the additional information with respect to the digital object 208 using the sight line information 209 and the gesture information 210. [ At this time, the additional information may be in the form of text, audio, image, etc. stored at a specific point of the digital object 208. For example, the additional information intensity calculation unit 206 calculates the additional information intensity calculation unit 206 based on (a) the distance between the gaze position and the side information of the user, (b) the distance between the gaze position and the gaze position, c) calculate the intensity of the additional information for the digital object 208 using at least one of the distances of the eye position on the heat map 212 representing the user's interest.

If at least one additional information is stored in the digital object 208, the additional information having the greatest intensity of the additional information may be output to the gaze position of the user.

The motion parallax calculation unit 207 can calculate the 3D position information and the rotation information of the digital object using the user's sight line information 209 and the gesture information 210. [

The object search unit 203 can search the digital object 208 using the search information for the digital object 208. [ For example, when a digital object 208 such as an image or a moving picture is reproduced on the display 213, the object searching unit 203 searches for the digital object 208 to be reproduced based on the user's sight line information 208 .

In one example, the object search unit 203 can zoom in / out on the digital object 208 based on the zoom ratio. The object search unit 203 can change the rendering quality of the digital object 208 based on the rendering quality centered on the line of sight of the user. In addition, the object searching section 203 can search for additional information about the digital object 208 based on the intensity of the additional information centered on the line of sight of the user. In addition, the object searching unit 203 can search a digital object by changing an output position and an output angle of the digital object according to the user's position information and the sight line information 210. [

3 is a flowchart illustrating an example of performing zoom in / zoom out operations for a digital object based on eye line information according to an embodiment of the present invention.

The digital object search apparatus can collect sight line information (S301). Then, the digitalobject search apparatus can determine whether the user blinks through the sight line information (S302). At this time, blinking of eyes may be a criterion for judging whether to hold or release the search of the digital object.

If the user blinks, the digital object search apparatus can determine whether the search of the digital object is enabled before blinking (S303). If the search for the digital object is not enabled, the digitalobject search apparatus can determine whether the search for the digitalobject is the same as the block ID when the search for the digitalobject is disabled (S307).

If they are the same, the digitalobject search apparatus can change the search for the digital object to Enable (S308). Conversely, if they are not the same, the digitalobject search apparatus can update the screen for the digital object (S314). At this time, the search for the digital object is enabled means that the search for the digital object is performed, and the search for the digital object is Disable means that the search for the digital object is stopped.

In step S303, if the digital object search is enabled, the digital object search device can change the digital object search to Disable (S305). Then, the digitalobject search apparatus can store the ID of the block when the search for the digital object is disabled (S306). At this time, the ID of the block is related to the block in which the user's line of sight is located. After that, the digitalobject search apparatus can update the screen for the digital object (S314).

If the user does not blink in step S302, the digitalobject search apparatus can determine whether the search of the digital object is enabled (S304). At this time, if the search for the digital object is not enabled, the digitalobject search apparatus can update the screen for the digital object (S314). If the search for the digital object is enabled, the digitalobject search apparatus can determine whether the block (current block) in which the user's line of sight is located is the same as the previous block in which the user's line of sight is located (S309) .

At this time, if the IDs of the current block and the previous block are the same, the digital object search device judges whether or not the user's line of sight exceeds the line-of-sight holding time (T) gazing at the current block and the preset threshold time (S310). In step S310, if the gaze holding time T does not exceed the preset threshold time Th, the digitalobject search apparatus can update the screen for the digital object (S314).

Conversely, if the visual-line holding time T exceeds the preset threshold time Th, the digital-object search apparatus can calculate the zoom magnification for the zoom-in in the current block (S311). If the ID of the current block differs from the ID of the previous block in step S309, the digitalobject search apparatus may calculate the zoom magnification for zooming out from the previous block to the current block (S312). At this time, the zoom magnification can be determined using the zoom adjustment coefficient K and the eye-gaze holding time T. [

Then, the digitalobject search apparatus searches for a digital object based on the calculated zoom magnification (S313), and can update the screen (S314). If the user continues to gaze at the current block, the digital object search apparatus can zoom in on the current block according to the zoom magnification during the sight retention time. If the user moves his or her gaze from the previous block to the current block, the digital object search apparatus can zoom out according to the zoom magnification during the sight retention time around the current block.

After the screen is updated, the digitalobject search apparatus can repeat the process of collecting gaze information again (S301).

FIG. 4 is a flowchart illustrating an example of zooming in / zooming out a digital object based on gaze information and gesture information according to an embodiment of the present invention.

In this case, FIG. 3 illustrates a process of determining whether to search for a digital object based on the user's sight line information, and performing a zoom in / zoom out operation. 4 illustrates a process of determining whether to search for a digital object based on the gesture information of the user and performing a zoom in / zoom out operation based on the user's sight line information.

The digital object search apparatus can collect gaze information and gesture information (S401). At this time, the gesture information may include a hand motion of the user. The digital object search apparatus can determine whether the user holds his / her hand through the gesture information (S403). If the user holds the hand, the digitalobject search apparatus can change the search for the digital object to Disable (S404).

If the user does not hold his / her hand, the digital object search device can determine whether the user has opened his / her hand (S406). If the user opens his / her hand, it can be determined whether the ID of the block to be examined by the user is the same as the block ID when the search for the digital object is Disable (S407). If the block IDs are the same, the digitalobject search apparatus can change the search for the digital object to Enable (S408). The remaining steps can be referred to the description of FIG.

As a result, the digital object search device can perform or stop the search for the digital object based on whether the user's eyes are blinking or whether the user grasps or opens the hand.

According to one embodiment of the present invention, the gesture of the hand is not limited to grasping or stretching operations, but may be changed to other operations. In addition, according to one embodiment of the present invention, the gesture operation is not limited to a hand, but may also include a gesture operation for other body parts of the user.

5 is a diagram illustrating an example of searching for a digital object by zooming in or zooming out according to an embodiment of the present invention.

FIG. 5 shows that the searching of the digital object is enabled by blinking of the user's eyes or hand operation. If the search of the digital object is Disable, the operation described in FIG. 5 can be performed if the search of the digital object is enabled by blinking of the user's eyes or hand operation.

<Zoom In>

The frame 501 and the frame 502 constitute the same digital object. It is assumed that at time T1, the user gazes at the block 503 among the frames 501 divided into a plurality of blocks. It is assumed that the user's line of sight is located in the block 504 at the same position as the block 503 of the frame 501 at a time T2 after a lapse of a predetermined time. Blocks 503 and 504 represent the same block ID.

If the user's gaze holding time exceeds a preset threshold time, the digital object search device can calculate the zoom magnification using the gaze holding time and the zoom adjustment coefficient. The digital object search device can then perform a zoom in on the block 504 of the frame 502 based on the zoom magnification. Then, the image or moving image displayed in the frame 502 may be enlarged around the block 504.

<Zoom Out>

The frame 505 and the frame 506 constitute the same digital object. At time T1, it is assumed that the user gazes at the block 507 among the frames 505 divided into a plurality of blocks. It is assumed that the user's line of sight is located at the block 508 of the frame 506 after a predetermined time has elapsed.

Then, the digital object search apparatus can calculate the zoom magnification by using the eye holding time and the zoom adjustment coefficient. In this case, the gaze holding time may mean the time when the gaze moves from the block 507 to the block 508. [ The digital object search device may then perform a zoom out from block 507 of frame 505 to block 508 of frame 506 based on the zoom magnification. Then, images displayed during the progress from the frame 505 to the frame 506 can be reduced.

FIG. 6 is a flowchart illustrating an example of searching for a digital object according to blocks in accordance with a rendering quality, based on visual line information according to an exemplary embodiment of the present invention.

The digital object search apparatus can collect sight line information of the user (S601). Then, the digital object search apparatus can determine whether the search for the digital object is enabled (S602). If the search for the digital object is Disable, the digitalobject search apparatus can perform a screen update (S608).

If the search for the digital object is enabled, the digitalobject search apparatus can determine whether the ID of the block in which the user's line of sight is located and the previous block ID are the same (S603). If the IDs of the blocks are not the same, the digitalobject search apparatus can initialize the previous block that the user has examined (S606). That is, the resolution of the previous block and the like can be restored to the original state.

If the IDs of the blocks are the same, the digital object search apparatus can calculate the rendering quality of the current block that the user is looking at (S604). Then, the digital object search apparatus can calculate the rendering quality of a neighboring block of the current block (S605). At this time, the rendering quality of the current block may be larger than the rendering quality of the neighboring block. For example, the rendering quality can be determined through the eye-retention time and the rendering adjustment factor.

When the rendering quality is calculated, the digitalobject search apparatus can search for the digital object (S607). Then, the digital object search device can update the screen of the digital object.

FIG. 7 is a diagram illustrating an example of determining a rendering quality for a digital object differently for each block according to an exemplary embodiment of the present invention. Referring to FIG.

7 shows one frame of a plurality of frames constituting a digital object. At this time, the frame can be divided into a plurality of blocks. If the user stares any one of the plurality of blocks, the digital object search apparatus can increase the rendering quality of the block. That is, the quality (resolution, sound quality, volume, etc.) of the block can be improved. In addition, the rendering quality of surrounding blocks around the block to be examined by the user can be sequentially degraded in quality.

A block (current block) indicating "3 " in Fig. 7 means a block currently being examined by the user. In this case, 3 means rendering quality. Then, it can be seen that the digital object search apparatus reduces the rendering quality as it goes around the current block. That is, the quality is improved around the block to be examined by the user, and the user can immerse the digital object around the block being examined.

8 is a flowchart illustrating an example of searching for a digital object by calculating the intensity of additional information for a digital object based on visual line information according to an embodiment of the present invention.

The digital object search device can collect the user's sight line information (S801). Then, the digital object search device can determine whether the search for the digital object is enabled (S802). If the search for the digital object is Disable, the digitalobject search apparatus can update the screen (S811).

In contrast, when the search for the digital object is Enable, the digitalobject search apparatus can calculate the distance (X _Dist , Y _Dist ) between the user's gaze position and the additional information (S803). Then, the digital object search apparatus can calculate the distance D _Dist of the user's gaze position on the depth map (S804). At this time, the depth map may mean a map showing the distance between the user and the component included in the digital object. Further, the digital object search apparatus can calculate the distance (I _Dist ) of the user's gaze position on the heat map (S805). At this time, the heat map may indicate a map indicating an area where the user is interested.

Then, the digitalobject search apparatus can calculate the intensity of the additional information using the calculated distance (S806). At this time, the intensity of the additional information may be a criterion for determining which one of the at least one additional information stored in the digital object is to be displayed with respect to the position that the user is looking at.

The digital object search apparatus may set the text additional information according to the intensity of the additional information (S807), set the video additional information (S808), or set the audio additional information (S809). For example, in the case of text, the digital object search apparatus can adjust the size, font, or color of the text. In the case of video, the digital object search device can adjust the size, resolution, and volume of the video. Further, in the case of audio, the digital object search apparatus can adjust the audio volume and the frequency rate.

The digitalobject search apparatus can search for a digital object to display additional information (S801). Then, the digitalobject search apparatus can update the screen (S811).

9 is a diagram illustrating an example of calculating the intensity of additional information according to a user's gaze position according to an embodiment of the present invention.

The digital object search apparatus can calculate the distance (X _Dist , Y _Dist ) between the user's gaze position and the side information in the digital object 901. The digital object search apparatus can calculate the distance (D _Dist ) of the user's gaze position on the depth map for the digital object 901. At this time, the depth map may mean a map showing the distance between the user and the component included in the digital object.

For example, the digital object search apparatus can calculate the distance (X _Dist , Y _Dist ) between the user's gaze position and the side information according to Equation (1) below. Then, the digital object search apparatus can calculate the distance (D _Dist ) of the user's gaze position on the depth map for the digital object 901 according to Equation (2) below.

In this case, Annot X and Annot Y respectively represent the X and Y components of the additional information, and CurX and Cur Y represent the X and Y components of the user's eye position, respectively.

In this case, Depth Map (x, y) represents the component of the additional information on the depth map, and CurZ represents the position of the user's gaze on the depth map. At this time, CurZ may be a value to which the zoom ratio is applied.

Further, the digital object search device can calculate the distance (IDist) of the user's gaze position on the heat map 902. [ At this time, the heat map may indicate a map indicating an area where the user is interested. For example, the digital object search apparatus can calculate the distance (IDist) of the user's gaze position on the heat map 902 according to Equation (3) below.

For example, the intensity (D) of the additional information may be determined according to the following equation (4).

10 is a diagram illustrating an example in which additional information is displayed according to a user's line of sight according to an embodiment of the present invention.

10, it is assumed that additional information is stored in the digital object at points A, B, C, D, and E. At this time, when the user's gaze position is present at a specific point on the digital object, the digital object search apparatus can calculate the intensity of the additional information about the user's gaze position. The intensity of the additional information can be determined according to equations (1), (2), (3) and (4) in FIG. That is, the intensity of the additional information is related to the distance between the position of the additional information stored in advance and the gaze position of the user.

The intensity of the additional information stored at the position D closest to the user's gaze position can be determined to the greatest in FIG. Then, the additional information stored at the D position can be displayed in text, video or audio type. For example, when the digital object is a map, the digital object search device can display the detailed information of the restaurant D closest to the user's gaze position in text, audio, or video form. As a result, the user can acquire additional information previously stored by gazing at a specific point in the digital object.

11 is a diagram illustrating an example of searching for a channel according to a user's gaze position according to an embodiment of the present invention.

Referring to FIG. 11, a process of changing broadcast channels, which are digital objects in 3D form according to a user's gaze, is illustrated. That is, according to the present invention, 3D position information and rotation information of a digital object are calculated using the user's sight line information and gesture information. Then, the 3D position information and the rotation information of the digital object are determined centering on the gaze position of the user, so that the digital object can be searched.

According to FIG. 11, the user can search a large number of channels only from the current position without a separate operation device such as a remote controller. In Fig. 11, when the user gazes to the left side of the screen, a plurality of channels can be displayed at the position of the line of sight gazing at the user. When the user moves the line of sight from left to right of the screen, the channel displayed on the screen can also move according to the user's line of sight.

12 is a diagram illustrating an example of determining a user interface according to a location of a user according to an exemplary embodiment of the present invention.

Referring to FIG. 12, the digital object displayed on the screen can be largely output in the direction of the user, and can be adjusted so as to be output in a direction opposite to the user. In particular, in the case of a large-sized screen, a far-away digital object can be displayed larger than when it is located close to the user, since it is difficult for the user to visually recognize.

Therefore, the digital object search apparatus adjusts the projection of the user interface to change the position and the rotation property of the digital object so that the digital object is largely output in the direction opposite to the user in the direction of the user according to the position of the user, view can be provided. That is, according to FIG. 12, a user can provide a perspetive view according to the position where the user views the screen.

13 is a diagram illustrating an example in which a user interface according to an embodiment of the present invention is output according to a user's location.

Referring to FIG. 13, the digital object can be moved and output according to the position of the user. That is, when the user is located on the left side of the screen, the outputted digital object also moves to the left, and when the user is positioned on the right side of the screen, the outputted digital object can also move to the right.

In particular, since it is difficult for a user to recognize a digital object when the user views the screen in a diagonal direction on a large screen, the digitalobject search apparatus according to an embodiment of the present invention can move and output a digital object have. Also, when the distance between the user and the screen increases, the digital object displayed on the screen may be displayed in an increased size by the increased distance.

14 is a flowchart illustrating an example of changing an output position and an output angle of a digital object according to a user's position information and visual information according to an embodiment of the present invention.

The digital object search device may collect the location information of the user (S1401). At this time, the digital object search apparatus can determine whether the digital object follows the user (first mode) as shown in FIG. 13 (S1402). If the mode is the first mode, the digital object search apparatus may change the position of the digital object according to the position of the user (S1403).

Conversely, when the mode is not the first mode, the digitalobject search apparatus can determine whether the digital object is output in the direction of the user (second mode) as shown in FIG. If the mode is the second mode, the digital object search apparatus can change the position and rotation property of the digital object so that the digital object is displayed in a direction in which the user is located and is output in a direction opposite to the direction in which the user is located (S1405).

If the mode is not the second mode, the digitalobject search apparatus can determine whether the digital object is output in accordance with the user's line of sight (third mode) as shown in FIG. In the third mode, the digital object search apparatus can change the position and rotation attribute of the digital object output on the screen according to the viewpoint of the user's gaze.

After the step S1403, the step S1405, the step S1406 and the step S1407, the screen displaying the digital object can be updated.

In addition, the digital object search method according to an exemplary embodiment of the present invention includes a computer readable medium including program instructions for performing various computer-implemented operations. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The media may be program instructions that are specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium for transmitting a signal specifying a program command, a data structure, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. Various modifications and variations are possible in light of the above teachings. Accordingly, it is to be understood that one embodiment of the present invention should be understood only by the appended claims, and all equivalent or equivalent variations thereof are included in the scope of the present invention.

100: User
101: Digital object search device
102: Display
103: sight line information
104: Gesture information

Claims (11)

A digital object search method comprising:
Collecting glance information and gesture information of a user;
Activating or deactivating a search for a digital object based on the gesture information;
When the search of the digital object is activated,
Determining a current block based on the line of sight information;
Comparing the current block with a previous block;
Determining at least one of a zoom rate for a zoom in of the current block and an information quality of the current block based on a line retention time when the current block and the previous block are identical; And
Determining at least one of a zoom ratio for zooming out from the previous block to the current block, an information quality of the current block, and an information quality of the previous block when the current block is different from the previous block
And searching the digital object.
The method according to claim 1,
Wherein the digital object includes a plurality of blocks,
Wherein the current block corresponds to a block currently viewed by the user,
Wherein the previous block corresponds to a block previously viewed by the user,
A method for searching digital objects.
The method according to claim 1,
Wherein the information quality of the current block includes at least one of a rendering quality of the current block and an additional information intensity of the current block,
Wherein the information quality of the previous block includes at least one of a rendering quality of the previous block and an additional information intensity of the previous block.
A digital object search method comprising:
Collecting glance information and gesture information of a user;
Activating a search for a digital object based on the gesture information;
Determining a current block based on the line of sight information;
Comparing the current block with a previous block; And
Determining at least one of a zoom ratio for zooming in of the current block and an information quality of the current block if the current block is identical to the previous block
And searching the digital object.
A digital object search method comprising:
Collecting glance information and gesture information of a user;
Activating a search for a digital object based on the gesture information;
Determining a current block based on the line of sight information;
Comparing the current block with a previous block; And
Determining at least one of a zoom ratio for zooming out from the previous block to the current block, an information quality of the current block, and an information quality of the previous block when the current block is different from the previous block
And searching the digital object.
6. The method of claim 5,
Wherein the digital object includes a plurality of blocks,
Wherein the current block corresponds to a block currently viewed by the user,
Wherein the previous block corresponds to a block previously viewed by the user,
A method for searching digital objects.
A computer-readable recording medium having recorded thereon a program for executing the method according to any one of claims 1 to 6.
A computer program stored on a medium for executing the method of any one of claims 1 to 6 in combination with hardware.
A digital object search apparatus comprising:
An information collecting unit for collecting visual information of a user and gesture information;
An object search unit for activating or deactivating a search for a digital object based on the gesture information; And
When the search of the digital object is activated,
Determines a current block based on the visual information,
Comparing the current block with a previous block,
Determining at least one of a zoom ratio for a zoom in of the current block and an information quality of the current block based on a line retention time when the current block and the previous block are the same,
The information quality of the current block, and the information quality of the previous block when the current block differs from the previous block, a zoom ratio for zooming out from the previous block to the current block, an information quality of the current block,
The digital object search apparatus comprising:
10. The method of claim 9,
Wherein the digital object includes a plurality of blocks,
Wherein the current block corresponds to a block currently viewed by the user,
Wherein the previous block corresponds to a block previously viewed by the user,
Digital object search device.
10. The method of claim 9,
Wherein the information quality of the current block includes at least one of a rendering quality of the current block and an additional information intensity of the current block,
Wherein the information quality of the previous block includes at least one of a rendering quality of the previous block and an additional information intensity of the previous block.

Images