KR20160072015A - Device and method for arranging contents displayed on the screen - Google Patents

Abstract

According to an embodiment of the present invention, provided is a method for a device arranging content displayed on a screen. The method includes the following steps: detecting a gaze of a user for a screen of a device; generating preferable area information related to a preferable area that a user prefers of the whole screen area of the device, based on the detected gaze of the user; selecting at least one among a plurality of contents displayed on the screen of the device; and rearranging the plurality of contents displayed on the screen of the device by changing a display position of the selected content based on the generated preferable area information.

Description

TECHNICAL FIELD [0001] The present invention relates to a device and a method for arranging contents displayed on a screen,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device and a method for arranging content displayed on a screen, and more particularly, to a device and a method for arranging content on a screen based on information sensed by a user using the device.

The Internet makes it possible for users to find and obtain information in astronomical amounts of content. As a result, the user can retrieve a tremendous amount of content through the device. The web browser of the device can display the content according to the format of the web page, and the web page can display various contents that are displayed to the user in a format that can be read by the user.

Conventional devices have provided a fixed display of content. In the layouts of the fixed website, the contents can be displayed in a predetermined area on the display screen. Also, the content displayed in the web page may include those related to the content and those not related to the content.

Existing devices lacked the ability to arrange content according to user information and relevance between the contents. In addition, the ability to display content of high interest by the user was insufficient on the screen area where the user frequently gazed.

Accordingly, the user has spent a lot of time searching for the desired content on the display screen, and the content has become excessive due to the fixed arrangement of the contents.

Thus, there is a growing need for a technology that can dynamically rearrange content on a display screen in a device.

According to an embodiment, a device capable of effectively arranging positions of contents displayed on a screen of a device based on a user's gaze and a method of arranging contents using the device can be provided.

According to one embodiment, there is provided a method for rearranging content displayed on a screen, the method comprising the steps of: sensing a user's gaze on a screen of the device; Selecting at least one of a plurality of contents displayed on a screen of the device based on the generated preference information; selecting a display position of the selected content based on the generated preference information; And rearranging the plurality of contents displayed on the screen of the device by changing the content of the content.

In addition, the step of selecting the content may select the content to be recommended to the user based on the content usage history information of the user.

In addition, the content usage history information includes information such as the number of times the user looked at the content, the amount of time the user took the content, the frequency of use of the content, the user using the content, the storage location of the content, the time using the content, History and history of interruption, reservation of use of content, registration of the item of interest in the content, and information on the available time of the content.

Also, the content usage history information includes information about a facial expression and a posture while the user looks at the content, and the step of selecting the content includes a step of selecting the content based on the expression and the attitude of the user taking the content Lt; / RTI >

In addition, the step of selecting the content may select the content to be recommended to the user based on at least one of the content attribute information of the content and the user attribute information for the user and the content usage history information of the user.

Here, the content attribute information includes information on the kind and genre of the content to be watched by the user, and the step of selecting the content may be to select the content based on the kind and genre of the content to be watched by the user .

In addition, the user attribute information includes information on the user's usage rights and the location of the user with respect to the content, and the step of selecting the content may be to select the content based on the usage rights of the user and the location of the user have.

Also, the preference area information may include at least one of the size, shape, position, and priority of the preference area.

In addition, the preference area information may be generated based on at least one of a path of a perceived line of sight of a user, a size of a time when a user's line of sight is sensed, and a number of times a line of sight of the user is sensed.

Also, the preference area may be set by the user.

The step of rearranging the content may further include the steps of: determining a priority of a plurality of contents displayed on the screen; selecting a content among the plurality of contents based on the preference information and the priority of the plurality of contents; , And changing the location of the selected content.

In addition, the step of changing the position of the selected content may move the selected content to the preference area and move the unselected content to a screen area other than the preference area when the non-selected content is located in the preference area have.

Also, the step of rearranging the content may include removing at least some of the non-selected content from among the plurality of contents on the screen.

Further, the step of rearranging the plurality of contents may change the shape of the selected content based on the preference area information.

The step of reordering the plurality of contents further includes the steps of: acquiring at least one new content, determining a location where the selected content and at least one new content are arranged based on the preference area information, And rearranging the selected content and at least one new content.

The step of reordering the plurality of contents further includes the steps of: determining a priority of the plurality of contents; changing at least a part of the plurality of contents into new contents based on the determined priority; And changing the content corresponding to a predetermined rank or lower to new content if the content is less than a preset rank.

The method may further include changing a background color of the preference area based on the preference area information.

In addition, the step of sensing the user's gaze may detect the user's gaze based on at least one of the expanded state of the user's pupil, the direction of the user's gaze, the presence of the user's eyes, and the user's retina pattern.

The step of detecting the user's gaze may include displaying a cursor corresponding to the user's gaze on the screen of the device and displaying the path of the cursor to be changed in accordance with the movement of the user's gaze .

The method may further include changing a background color of the content meeting the movement path along the movement path of the cursor.

Here, the content may include at least one of text, a hyperlink, an image, an icon, a UI, an audio file, an image file, an advertisement, and a web page object.

According to an embodiment, there is provided a device for rearranging content displayed on a screen, the device comprising: a sensing unit for sensing a user's gaze on a screen of the device;

A display unit for displaying a plurality of contents; and a display unit for generating preference region information related to a user's preferred region out of the entire screen region of the device based on the sensed line of sight of the user, selecting at least one of the plurality of contents, And a control unit for rearranging the plurality of contents by changing the display position of the selected content based on the generated preference area information.

Further, the control unit can select the content to be recommended to the user based on the content usage history information of the user.

Here, the content usage history information includes information such as the number of times the user looked at the content, the amount of time the user took the content, the frequency of use of the content, the user using the content, the storage location of the content, the time using the content, History and history of interruption, reservation of use of content, registration of the item of interest in the content, and information on the available time of the content.

Also, the content usage history information may include information on a facial expression and posture while the user looks at the content, and the control unit may select the content based on the expression and attitude of the user who is looking at the content.

In addition, the control unit may select a content to be recommended to the user based on at least one of the content attribute information of the content and the user attribute information for the user and the content usage history information of the user.

In addition, the content attribute information may include information on the type and genre of the content to be watched by the user, and the control unit may select the content based on the type and genre of the content to be watched by the user.

In addition, the user attribute information includes information on the user's usage rights and the location of the user with respect to the content, and the control unit may select the content based on the usage rights of the user and the location of the user.

The preference region information may include at least one of a size, a shape, a position, and a priority of the preference region.

In addition, the preference area information may be generated based on at least one of a path of a perceived line of sight of a user, a size of a time when a user's line of sight is sensed, and a number of times a line of sight of the user is sensed.

Also, the preference area may be set by the user.

The control unit may determine a priority order of the plurality of contents displayed on the screen, select a content from the plurality of contents based on the preference area information and the priority order of the plurality of contents, change the position of the selected content .

In addition, the controller may move the selected content to the preferred area to change the position of the selected content, move the unselected content to a screen area other than the preferred area if the non-selected content is located in the preferred area, .

In addition, the control unit may remove at least a part of the non-selected content from the plurality of contents on the screen.

Further, the control unit may change the shape of the selected content based on the preference area information.

Also, the control unit may acquire at least one new content, determine a location where the selected content and at least one new content are arranged based on the preference area information, and determine the selected content and at least one new content in the determined location Can be arranged.

In addition, the control unit may determine the priority of the plurality of contents, change at least a part of the plurality of contents into new contents based on the determined priority, and when the priority of the contents falls below a preset rank, The content corresponding to a predetermined rank or lower can be changed to new content.

The control unit may change the background color of the preference area based on the preference area information.

In addition, the sensing unit may sense the user's gaze based on at least one of the expanded state of the user's pupil, the direction of the user's gaze, the presence of the user's eyes, and the user's retina pattern.

Further, the sensing unit may display a cursor corresponding to the user's gaze on the screen of the device, and may display the movement path of the cursor to be changed in accordance with the movement of the user's gaze.

In addition, the sensing unit may change the background color of the content meeting the movement path along the movement path of the cursor.

Here, the content may include at least one of text, a hyperlink, an image, an icon, a UI, an audio file, an image file, an advertisement, and a web page object.

A computer-readable recording medium having recorded thereon a program for causing a computer to execute one of the methods according to one embodiment.

According to an exemplary embodiment, information on a user's preferred region can be generated according to a user's gaze. Furthermore, content can be efficiently arranged through the priority information on the user's preference area.

According to an embodiment, by rearranging the content in consideration of the attitude of the user, the fatigue of the user staring at the device can be reduced.

According to an exemplary embodiment, the content can be rearranged based on the content usage history including information on the user's gaze, thereby providing a customized content arrangement screen to the user.

1 schematically illustrates a device for rearranging content displayed on a screen, according to an embodiment.
2 is a flowchart illustrating a method of rearranging content displayed on a screen of a device according to an exemplary embodiment.
3A and 3B are diagrams illustrating an embodiment of a device for rearranging a Kent tent, according to one embodiment.
4 is a flow diagram illustrating a method for a device to select content, according to one embodiment.
5A and 5B are views showing an embodiment in which a plurality of contents are rearranged on a screen of a device according to an embodiment.
6 is a flowchart illustrating a method of rearranging content when new content is added according to an exemplary embodiment.
7 is a diagram showing an embodiment in which new content is added according to an embodiment.
8 is a flowchart illustrating an operation of a device for detecting a user's gaze according to an exemplary embodiment of the present invention.
Figs. 9A and 9B are diagrams showing, in accordance with an embodiment, a device distinguishing and displaying contents recognized by a user by displaying a path of a user's line of sight. Fig.
10 is a flowchart illustrating a method of rearranging content on a Web browser screen according to an exemplary embodiment.
11 is a flowchart illustrating a method of rearranging content in a web browser for real-time processing according to an exemplary embodiment of the present invention.
12 is a block diagram illustrating a configuration of rearranging contents of a web page in a web browser according to an embodiment.
13 and 14 are functional block diagrams showing the configuration of a device according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

1 schematically illustrates a device for rearranging content displayed on a screen, according to an embodiment.

The device 100 can arrange the content displayed on the screen of the device 100 based on the line of sight of the user looking at the screen of the device 100. [ The device 100 may be, for example, a notebook computer, but this is merely an example, and the device 100 may be implemented with various electronic devices including a display. For example, the device 100 may be a TV, a tablet PC, a digital camera, a camcorder, a mobile terminal device, a tablet PC, a desktop, an electronic book terminal, a digital broadcast terminal, a PDA (Personal Digital Assistants) Navigation, MP3 players, wearable devices, and the like. In addition, the device 100 may be stationary or mobile.

As shown in FIG. 1, the device 100 may sense the user's gaze on the screen of the device 100. [ The device 100 can determine a view point 10 corresponding to the user's gaze toward the screen of the device 100. [ In addition, the viewpoint 10 can be displayed on the screen.

The device 100 can generate preference region information related to the preference region preferred by the user among the entire screen region of the device based on the user's gaze.

The preference area may mean a screen area preferred by the user in the full screen of the device 100. [ In addition, the preference area may refer to a screen area in which the user frequently looks up the entire screen of the device 100. [ The preference region may be an area such as an upper end portion, a lower end portion, a middle portion, and a side portion of a screen of the device 100, but is not limited thereto. In addition, the preference area on the screen of the device 100 can be changed.

In addition, the preference area may be a part of the entire screen of the device 100, or may be all of the entire screen area. Here, the number of the partial regions is not limited.

The device 100 can select at least one of a plurality of contents displayed on the screen of the device. Here, the selected content may be content to be rearranged on the display upon sensing the user's gaze. For example, the content selected may be the one with the highest interest of the user. The content with the highest interest of the user can be selected to be moved to the user's top priority area. According to one embodiment, the device 100 may move the selected content to the top preference area.

The device 100 can rearrange a plurality of contents displayed on the screen of the device 100 by changing the display position of the selected content based on the generated preference area information.

That is, the device 100 may determine each preference area in which a plurality of content should be located, and then rearrange the content to match the preference area.

Accordingly, the device 100 can rearrange the content on the screen of the device 100 based on the user ' s gaze toward the content displayed on the screen of the device 100. [ Accordingly, the device 100 can display the content of interest to the user in the preference area, and the user can conveniently look at the content displayed in the preference area.

2 is a flowchart illustrating a method of rearranging content displayed on a screen of a device according to an exemplary embodiment.

2, the device 100 may rearrange the content displayed on the screen according to the preference of the user, thereby increasing the convenience of the user viewing the content.

In step S210, the device 100 can detect the user's gaze on the screen of the device 100. [ The device 100 may sense a user's gaze using a camera, a light sensing unit, or the like.

Here, the device 100 can recognize the user's gaze using the camera and track the user's gaze. The device 100 may monitor a user viewing a screen of the device 100 using a camera. As an example of a camera according to one embodiment, it may include at least one of a web camera, a mobile phone camera, and a digital single-lens reflex camera (DSLR). However, the present invention is not limited thereto. The device 100 may monitor the user's gazing pattern by tracking various characteristics of user behavior using the camera. An example of a user's characteristics recognized by a camera according to one embodiment may include at least one of a user's gaze, facial expression, emotions, presence of eyes, eye position, flicker rate, retina pattern, and pupil dilation . But is not limited thereto.

In step S220, the device 100 can generate preference area information related to the preference area preferred by the user, among the entire screen area of the device 100, based on the detected user's gaze.

The device 100 can generate preference area information related to a preference area preferred by the user on the basis of the number of times the user's gaze looks at the area on the screen, the amount of time taken to look at the area on the screen, . The preference area information may refer to an area where a user particularly frequently looks for on the screen of the device 100. [ The preference region information may be generated differently depending on the user. The preference region information can be generated in real time. The preference region information may be transmitted to the server and stored. The preference area information may be stored in a storage device within the device 100. [

The device 100 can measure the number of times that the user's line of sight sensed in the display screen area is sensed. The device 100 can determine the preferred region according to the number of times the user's sight line is sensed. The preference area may be set based on a screen area having the highest probability of being displayed to the user among the screens of the device 100. [

In one embodiment, the device 100 may measure the number of times the areas within the screen of the device 100 are sensed by the user. The device 100 may determine the priority of the areas on the display screen according to the number of times the user's line of sight is sensed. The region determined to be the highest priority may mean the region that is most frequently looked up by the user. But is not limited thereto. Areas that are not taken by the user on the display screen may be of low priority. In one embodiment, the preference area is an area that the user tends to see frequently, and the preference area may not be limited to the upper area but may include one of the middle area, the left area, the right area and the lower area. Further, the preference area may include a highlighted area that indicates an unreadable state of the content on the display screen.

In step S230, the device 100 can select at least one of a plurality of contents displayed on the screen of the device 100. [ The device 100 can select the content that needs to be rearranged among a plurality of contents. Device 100 may select all of the plurality of content.

The device 100 can select the content based on the content usage history information of the user. The device 100 may select content based on the number of times the user has stared at the content, the time to stare, the amount of time to stare, the frequency of use, and the like.

The device 100 can select a user-preferred content according to a preset reference. The content may be selected to move to a preference area with a high preference, or to move to a preference area with a low preference.

The content usage history information includes information such as the number of times the user has looked at the content, the amount of time the user took the content, the frequency of use of the content, the user using the content, the storage location of the content, the time using the content, A history, a use reservation of the content, a registration of the interested item of the content, and information on the available time of the content.

For example, when the device 100 detects that the number of times the user has taken a particular content is greater than a preset number, the device 100 may determine that the user is interested in the specific content. Accordingly, the device 100 can set the specific content as the content of interest of the user. The device 100 may select specific content to rearrange the particular content into a preferred region of the user if the particular content is not located in a preferred region of the user.

  As another example, device 100 may measure the amount of time a user has taken a particular content. The device 100 can set the specific content as the content of interest of the user when the time when the user has looked at the specific content exceeds a preset time. The device 100 may select specific content to rearrange the particular content into a preferred region of the user if the particular content is not located in a preferred region of the user.

In addition, the device 100 can select the content to be rearranged using the content attribute information or the user attribute information for the user.

Here, the content attribute information may include information on the type and genre of the content watched by the user. The device 100 can select a contact based on the type and genre of the content that the user has looked at.

For example, the device 100 may set the sports content as user preference content if the content with a high frequency of user's viewing is sports content. The device 100 may select the sports content among a plurality of contents and move the sports content to the preferred area. The device 100 may store the sport content as preferred content.

As another example, the device 100 may select the content based on the genre of the content. For example, the device 100 may select a particular genre of movie content to arrange content if the user's content of interest is a particular genre of movie content.

The user attribute information may include information about the user's rights to the content and the location of the user. The device 100 can select the content based on the usage rights of the user and the location of the user.

For example, the device 100 may select the content based on the usage rights of the user. If the user does not have the right to use the specific content, the user may not select the specific content. The device 100 can select the content from among the contents for which the user is authorized to use the content.

In addition, the device 100 may select the content based on the location of the user. For example, the device 100 may select content primarily used by the company if the location of the user is recognized as " company ". In addition, the device 100 may map the selected content to the company and store the selected content in the device 100 or the server.

In step S240, the device 100 can rearrange a plurality of contents displayed on the screen of the device 100 by changing the display position of the selected content based on the generated preference area information.

For example, the device 100 may move content of a user's preference to a preference area frequently viewed by the user. In addition, the device 100 may move content with a low user preference to an area that the user does not often see. The preference region frequently viewed by the user can be determined based on the user's gaze.

3A and 3B are diagrams illustrating an embodiment of a device for rearranging a Kent tent, according to one embodiment.

3A is a diagram showing that the device 100 displays a plurality of contents on a display screen. As shown in FIG. 3A, the device can display a plurality of contents according to an initial rule set in the device. At this time, the device 100 can display the content without considering the preference area information by the user's line of sight.

The device 100 can sense the user's gaze of the content. The device 100 may generate a content priority using the number of times the user's gaze is applied to the plurality of contents c1 to c6, the time at which the user's gaze is sensed, and the pupil expansion of the user. The device 100 may set the contents c1, c3, c4, and c5 as the contents of interest of the user according to the analysis of the user's gaze of the user to be sensed. The device 100 may select to move the content c1, c3, c4, c5 to the user's preferred area.

In addition, the device 100 can set a preference area of the user. The device 100 can set a screen area that the user's line of sight has. In FIG. 3A, the device 100 can set a preference area, which is mainly affected by a user's line of sight, as a screen area at the top of the entire screen area of the device 100. [

In FIG. 3B, the device 100 may move the user's preferred content to the user's preferred area. In FIG. 3B, the device 100 may move the selected content to a screen area of the upper end set as the user's preferred area.

For example, in FIG. 3B, the device 100 can move the selected content c1, c3, c4, and c5 to be positioned in the upper screen area of the entire screen area. In FIG. 3B, it can be seen that the user's preferred content is located on the user's preferred area.

In addition, the device 100 may add content related to the user ' s preferred content. The device 100 can determine the user's preferred content through analysis of the user's gaze. For example, the device 100 may set the priority of a specific content higher when a user's gaze is detected over a predetermined number of times in the specific content. The device 100 may add content associated with a particular high priority content to the user's preferences. As shown in FIG. 3B, the device 100 can determine c1 as a content having a high priority of the user. The device 100 can add and arrange c1, which is a content related to the content of the user's primary interest, in the course of rearranging the plurality of contents (c1 to c6). Alternatively, the device 100 may remove c2 and then arrange further by adding c1.

That is, the device 100 can arrange content according to the user's preference in an area set as a preference region of the priority of the entire screen of the user device 100. [ The device 100 may provide an effect that allows the user to easily recognize the preferred content.

4 is a flow diagram illustrating a method for a device to select content, according to one embodiment.

Referring to FIG. 4, the device 100 may select content that reflects the user's preferences, in order to rearrange a plurality of content.

In step S410, the device 100 can determine the priority of a plurality of contents displayed on the screen of the device 100. [

The device 100 can select the content to be repositioned based on the content usage history information of the user. In addition, the device 100 can select the content to be recommended to the user based on the content usage history information of the user.

Here, the content usage history information includes information such as the number of times the user looked at the content, the amount of time the user took the content, the frequency of use of the content, the user using the content, the storage location of the content, the time using the content, And at least one of an interrupt history, a reservation for use of content, a registration of an item of interest in the content, and information on the available time of the content.

Also, the content usage history information may include contents related to the user's attitude to look at the content and the content while the user is watching the content. The device 100 may select the content based on the look and attitude of the user staring at the content.

According to one embodiment, the device 100 may rearrange content on the screen based on a change in the user's posture. The device 100 can determine the user's posture based on the distance from the screen of the device 100 to the user's face. The device 100 may rearrange the content based on the determined posture of the user. The device 100 may also change the attributes of the content such as, for example, the location of the content, the display direction, the display size, the color, the font size of the text included in the content, and the color of the text.

According to one embodiment, the device 100 may rearrange the content on the screen based on the user's facial expression. The device 100 can read the user's facial expression by sensing the face of the user. The device 100 can determine the emotional state of the user from the facial expression of the read user. For example, emotional states may include happiness, sadness, surprise, fear, nausea, anger, and the like. But is not limited thereto. The device 100 may recommend the content based on the emotion of the user.

For example, the device 100 may determine through an analysis of the user's gaze that the content the user is interested in is an image file characterized by humor. The device 100 may determine that the user's emotional state is " happy " or " joyous " through the user's gaze detected in the image file. The device 100 may recommend content related to " pleasure " and " happiness " based on the emotional state of the user. The device 100 may select the recommended content and move it to the user's preferred area. In addition, the device 100 can acquire new content and arrange it in the user's preferred area.

In addition, the device 100 may determine the priority of the content in order to select the content. For example, the device 100 may determine the priority of the content by analyzing the number of times the user has stared at the content and the amount of time the user has stared at the content. For example, the device 100 can determine the content of which the user's gaze is less than the predetermined number of times and the user's gaze of the plurality of content arranged in the entire screen area. The device 100 may prioritize a plurality of content.

In step S420, the device 100 can select the content from the plurality of contents based on the preference area information and the priority order of the plurality of contents. At this time, the device 100 can select the content to rearrange the content according to the priority of the content. For example, the device 100 may choose to move specific content if the particular content is high in content priority, but not in a high priority preference area. On the other hand, when the priority of the specific content is low, the device 100 may not select the specific content when the specific content is located in the low priority region.

That is, the device 100 may select the content to rearrange the content so that the user's preferred content matches the preferred screen area.

In step S420, the device 100 may change the location of the selected content. The device 100 may change the location of the content so that the content is arranged according to the user's preference area information.

Here, the preference region information may be set by the user. The user can preset his or her favorite area in the entire screen area of the device 100. [ The device 100 may receive input from the user for the preferred region and update the preferred region information.

The preference region information may be generated based on at least one of a path of a detected user's gaze, an amount of time a user's gaze is sensed, and a number of times a user's gaze is sensed.

In addition, the preference region information may include information on at least one of size, shape, position, and priority of the preference region.

The device 100 may remove at least some of the non-selected content from the plurality of content items on the screen in order to change the position of the content. For example, when a content having a low priority is located in a preference area having a high priority in the screen of the device 100, the device 100 can remove the low priority content on the screen. The device 100 may move the content having a higher priority to the preference area where the removed content is located. Alternatively, the device 100 may add and arrange new content.

According to one embodiment, the device 100 may move the selected content to a preference area with a higher priority and move the non-selected content to a screen area other than the preference preference area. The device 100 may replace the content according to the priority of the preference area.

5A and 5B are views showing an embodiment in which a plurality of contents are rearranged on a screen of a device according to an embodiment.

5A is a diagram showing a screen before the device 100 rearranges a plurality of contents. 5A, the device 100 includes a cricket content 505, a game content 510, a movie content 515, a fashion content 520, a music content 525, and a political content 530 on the screen. . ≪ / RTI >

Further, the device 100 can generate preference area information based on the user's line of sight. In Fig. 5A, the preferred region is set to the upper region 535, the middle region 540, and the lower region 545 based on the user's line of sight. The device 100 can determine the priority of the preference area based on the user's gaze. For example, the preference of the preference region may be determined in the order of the top region 535, the middle region 540, and the bottom region 545.

The device 100 can select the content to be changed from the plurality of contents based on the content usage history information of the user. According to one embodiment, the device 100 may determine the priority of a plurality of content according to a user's preference.

The device 100 may determine the priority of the content based on the category of the content. The device 100 may be configured such that the device 100 has priority in order of the movie content 515, the political content 530, the fashion content 520, the music content 525 and the game content 510, for example. You can decide.

FIG. 5B is a diagram showing a device after rearranging the content based on the preference region information and the priority of the content. FIG.

5B, the device 100 displays the movie content 515, the political content 530, and the fashion content 520 having a high content priority in the top area 535 having a high priority according to the preference area information, Lt; / RTI > At this time, the device 100 can change the size and shape of the content arranged on the basis of the attributes such as the size and shape of the upper region 535, and the like.

In addition, the device 100 may arrange the cricket content 505 to be located in the intermediate region 540 based on the preference region information and the priority of the content.

The device 100 may also arrange the relatively low priority music content 525 and the game content 510 to be in the bottom region 545 of the screen.

Accordingly, the device 100 can arrange the plurality of contents appropriately based on the preference area information about the screen, thereby providing a screen including the arranged content based on the user's preference. Accordingly, the device 100 can efficiently arrange the contents. In addition, the device 100 can provide the user with an effect that can be read to the user without missing high-content-content.

 6 is a flowchart illustrating a method of rearranging content when new content is added according to an exemplary embodiment.

According to the method of FIG. 6, the content that is newly added according to an embodiment can be arranged according to preference.

In step S610, the device 100 may receive at least one new content. The new content may be selected based on at least one of the content usage history information of the user, the user attribute information for the user, and the content attribute information of the content.

The new content may mean content added to a new screen that appears as the screen is scrolled according to the movement of the user's gaze. In one embodiment, the device 100 may allow a user to move the scroll bar on the screen of the device 100 according to a change in the user's gaze. For example, the device 100 may move the scroll bar of the web page according to the position of the user's line of sight. In addition, the device 100 can adjust the user's line of sight on the screen on which the web browser is displayed. The device 100 can adjust the scrolling of the web page according to the movement of the user's gaze and the position of the user's gaze.

The new content may include the content added by the user's input. In addition, the device 100 may recommend new content as it analyzes the user's gaze, and may additionally display the new content on the screen of the device 100.

In step S620, the device 100 may determine, based on the preference region information, the location where the selected content and the at least one new content are arranged.

The device 100 can determine the content located on the existing screen and the location information on the new content. The device 100 can determine the location of the contents according to the preference area information determined based on the user's gaze.

According to one embodiment, the device 100 can continue to track and analyze regions of the display screen on which the user's attention is focused as the user searches for a web page. The device 100 can determine the line of sight of the user. In addition, the device 100 may determine priorities of regions of the screen having a high number of times that the user's line of sight is sensed. Areas of the screen where the number of times the user's gaze is highly susceptible may include areas that the user can easily observe. The device 100 can also keep track of the user's sitting position and generate preference area information based on the user's sitting position.

The device 100 may include information about a user's preference of the content, information on recognition based on the user's gaze and information on the non-recognized content, information on a region where the user's gaze is detected a high number of times, Can be stored. In addition, the device 100 may include information on user preferences of the content, information pertaining to recognition and non-perceived content according to the user's gaze, information on areas where the number of times the user's gaze is detected is high, The information may be transmitted to the server for storage on the server.

 In addition, the device 100 may assign a priority step to areas on the display screen based on the preference area information. In addition, the device 100 may determine the arrangement position of the content based on the content priority determined based on the content usage history of the user.

Here, the selected content may mean selected content among a plurality of existing content, and may include all of a plurality of content from at least one content.

In step S630, the device 100 may rearrange the selected content and at least one new content on the determined screen location.

The device 100 may determine the priority of the selected content and the new content. The device 100 may determine the priority of the content based on the content usage history of the user, the content attribute information, and the user attribute information.

The device 100 may change at least some of the plurality of content into new content based on the determined priority of the content. Among the plurality of contents, the content to be changed may include content having a lower priority of the content. For example, the device 100 may display new content by replacing the low priority content of the content with new content. As a result, the device 100 can provide an effect of additionally confirming the contents of high interest to the user.

The device 100 may remove at least some of the plurality of content on the screen based on the determined priority of the content. The device 100 can change the shape of the content that has not been removed and display it on the removed screen. For example, the device 100 can control to display a higher priority content on the screen after the lower priority content is removed.

7 is a diagram showing an embodiment in which new content is added according to an embodiment.

As shown in FIG. 7, the device 100 may add new content on a screen where a plurality of contents are displayed.

The device 100 can display a plurality of contents (contents A to F) by adding a new window 710 on the screen where the window 705 including the existing contents is displayed.

The plurality of contents included in the new window 710 may include contents that are related to existing contents. Or may contain content of new content. But is not limited thereto.

For example, the device 100 may arrange the movie content, which is a content having a high content priority, to be located in a high-priority preference region. Device 100 may additionally display content A or content F associated with high priority movie content. For example, content A may refer to the latest film ranking, and content B may include movie star interviews and the like.

Alternatively, the device 100 may retrieve the user's past information about the user's content priority associated with the currently displayed content from the store or server. The device 100 may add new content based on information about the user's interest in the currently displayed content. Alternatively, the device 100 may rearrange the currently displayed content based on information about the user's interest in the currently displayed content.

The device 100 may display content that meets the needs of the user by additionally presenting content relevant to the content of interest to the user according to the content priority.

8 is a flowchart illustrating an operation of a device for detecting a user's gaze according to an exemplary embodiment of the present invention.

In step S810, the device 100 can detect the user's gaze.

In addition, the device 100 can display the viewpoint of the user on the screen of the device 100 by analyzing the user's gaze. Here, the user's viewpoint may refer to one area of the display screen which is in contact with the user's sight line. In addition, the device 100 may keep track of the content of the user's gaze and the area of the screen being looked at by the user. In addition, the device 100 can determine the user's interest in the content by monitoring the time spent in retrieving the content, facial expressions during that time, and pupil dilation. The device 100 can track the distance from the display screen to the user's eyes. The device 100 can also sense in real time the attitude of the user staring at the device 100 based on the tracked distance.

In addition, the device 100 may continue to track and analyze regions of the display screen on which the user's gaze is sensed while the user is searching for a web page. The device 100 can find the path of the user's viewpoint. Accordingly, the device 100 can generate preference region information including a priority as an area frequently visited by the user. In addition, the device 100 may store information on the preference region information, the path of the user's viewpoint, and the content located on the path of the viewpoint.

In step S820, the device 100 may display a cursor corresponding to the user's line of sight on the screen of the device 100. [ The device 100 can display the viewpoint of the user corresponding to the user's gaze as a cursor on the screen. At this time, the shape of the cursor is not limited. The user can recognize which area of the screen his / her gaze is located due to the cursor.

According to another embodiment, the device 100 may display a user's signal by an input device such as a mouse as a cursor, instead of the user's gaze. The type of the input device is not limited.

In S830, the device 100 can display the movement path of the cursor to be changed in accordance with the movement of the user's gaze.

The device 100 can prevent the cursor corresponding to the user's gaze from being removed even if the user's gaze is moved. That is, the device 100 can display the movement path of the cursor according to the movement of the user's line of sight by point, line, plane, or the like. The display method of the cursor is not limited.

The user can clearly recognize the moving direction of the user's gaze as the moving path of the cursor is displayed.

In S840, the device 100 can change the background color of the preference area included in the movement path of the cursor.

The device 100 can display the content to be encountered with the movement path of the cursor. The device 100 can determine that the content to be encountered with the path of movement of the cursor has been recognized by the user. The device 100 can display the content already read by distinguishing it from the unread content by changing the background color of the content recognized by the user.

According to another embodiment, the device 100 can change not only the background image of the preferred region included in the movement path of the cursor, but also the color of the border of the preferred region, the shape of the content, and the like.

FIGS. 9A and 9B are diagrams showing, in accordance with an embodiment, the device 100 distinguishing and displaying the contents recognized by the user by displaying the path of the user's line of sight.

In FIG. 9A, the device 100 can display the viewpoint 10 corresponding to the user's line of sight on the screen of the device 100. FIG. Here, the device 100 can display the rearranged contents based on the preference area information and the priority of the data. The manner of arrangement of the contents in Figs. 9A and 9B is not limited.

As shown in FIG. 9A, the device 100 can display the movement path 15 of the point 10 at which the user's eyes move together as the line moves, using lines. The device 100 may determine the content to be encountered with the movement path 15 of the viewpoint 10 as the content read by the user. In addition, the device 100 can use the content read by the user's gaze as information for determining the priority of the content. In addition, the device 100 may generate and store information about the movement path 15 of the viewpoint 10. [ In addition, the device 100 can map and store the information about the movement path 15 of the viewpoint 10 and the content of the movement path 15.

In FIG. 9B, the device grasps the user's line of sight and displays the read content by distinguishing the determined content from other content.

The device 100 may change the background color of the content meeting with the movement path 15 of the viewpoint 10. [ For example, the device 100 may shade the content determined by the read content to a predetermined color. The process in which the device 100 shadows the content read in a predetermined color can be referred to as shading of the content. Further, the device 100 may remove the shaded content from the screen. In addition, the device 100 may replace the shaded content with new content. Where new content may include content that is relevant to the content being read.

According to one embodiment, the device 100 may dynamically arrange the content to fit the user's gaze by analyzing the user's gaze. In addition, it is possible to provide the effect of allowing the user to more easily identify the content by separately displaying the read contents among the arranged contents.

10 to 12 are views illustrating an example of rearranging content on the screen by the device 100 displaying a web page according to an embodiment.

10 is a flowchart illustrating a method of rearranging content on a Web browser screen according to an exemplary embodiment.

More specifically, FIG. 10 is a sequence diagram illustrating a method for rearranging content in a web browser with pre-processed information for user observing behavior according to an embodiment.

Here, a web browser may include an application program used to search for information in the Internet network. It is also called a browser or an Internet browser. This allows users to perform various activities such as viewing images, sending and receiving e-mails, uploading and downloading various data. Typical functions include, but are not limited to, opening a web page, providing a recently visited internet address (URL) and favorite, and storing a web page.

The content may be displayed on the device 100 screen, such as, for example, a desktop of the device 100, a home screen, a lock screen, and the like. In addition, the content may be displayed within a window displayed on the screen of the device 100. For example, the content may be displayed within a web page displayed on a web browser. For example, the content may be displayed in the entire area or in some areas within the partitioned layout within the web page.

Examples of content content include, but are not limited to, text, icons, images, video, audio files, interactive apps, widgets, and interactive text. However, the present invention is not limited thereto.

In step S1010, the device 100 may analyze the user's gazing pattern. The user's gazing pattern may be, for example, a pattern as to when, where, in which position the user stared at the screen. For example, the user's gazing pattern may include a moving path at a time point at which a user's gaze is directed on the screen as the user's gaze moves. The information about the user's gazing pattern includes information about user interest in a plurality of contents, recognition between a plurality of contents and information about non-recognized contents, information on an area that a user can easily observe on a display screen, Information about the area in which the user's gaze is most detected, and information about the sitting position of the user while searching the web page in the web browser.

In step S1020, the device 100 may retrieve the content from the server. The server may be a server that provides content to the web page of the device 100. The server may store content based on the user ' s content priority. In addition, the server may recommend the content based on content priority to the device 100.

In step S1030, the device 100 can design a screen layout. The device 100 may design the screen layout to rearrange the contents. Designing the screen layout may mean changing the location, shape, background color, etc. of the contents displayed on the screen of the device 100. For example, you can change the layout of the window to rearrange the content displayed in a particular window.

The device 100 may rearrange the contents on the screen of the device 100 taking into consideration the user's gazing pattern and the priority of the contents on the screen. The device 100 may reorder the content in real time. The device 100 can dynamically generate a web page layout according to the user's gaze. The device 100 may arrange a plurality of content in the generated web page layout.

11 is a flowchart illustrating a method of rearranging content in a web browser for real-time processing according to an exemplary embodiment of the present invention.

In step S1110, the device 100 may analyze the user's gazing pattern. The information on the user's gazing pattern may include a moving path to the user's viewpoint moving on the screen according to the movement of the user's gaze. The information about the user's gazing pattern includes information about user interest in a plurality of contents, recognition between a plurality of contents and information about non-recognized contents, information on an area that a user can easily observe on a display screen, Information about the area in which the user's gaze is most detected, and information about the sitting position of the user while searching the web page in the web browser.

In step S1120, the device 100 can retrieve a plurality of contents from the server. Examples of content include, but are not limited to, text, video, audio files, interactive apps, widgets, and interactive text. But is not limited thereto.

The device 100 may receive the content to be displayed on the screen of the device 100 from the server. The device 100 may receive new content to be displayed on the screen of the device 100 from the server. In addition, the device 100 may receive new content to be displayed on the screen of the device 100 from the server.

In step S1130, the device 100 may retrieve the non-recognized content. The non-recognized content may refer to content that is not directed by the user's gaze. In addition, the non-recognized content may include content that is not read by the user.

The device 100 may search for non-recognized content to distinguish and display user-read content and unread content.

In step S1140, the device 100 can design a screen layout. The device 100 may include information about user interest in the content, information about the recognition and non-aware content among the plurality of content, information about the easy-to-observe area on the display screen, The layout of the screen can be designed based on the information on the sitting position of the user. That is, the device 100 can dynamically generate a web page layout based on the user's attitude and gaze.

Device 100 may rearrange the content in the generated layout. Device 100 may rearrange content based on content priority.

For example, in one embodiment, the device 100 may arrange the content according to the priority of the preference area on the display screen. For example, the content read by the user may be moved onto a lower priority preference area.

The device 100 may change at least one of a position, a direction, an image size, a color, a text font size, and a font color of a text of a plurality of contents according to a sitting posture of the user. In addition, the device 100 may change at least one of a font size and a color of a plurality of contents based on a distance of a user's face from a display screen. In another embodiment of the present invention, the content may be changed based on at least one of the user's sitting attitude and the user's interest.

In addition, the device 100 can display the content determined as the highest priority on the preference region determined as the highest priority based on the user's gaze. Accordingly, the user can preferentially recognize the high priority contents.

12 is a block diagram illustrating a configuration of rearranging contents of a web page in a web browser according to an embodiment. The configurations disclosed in Fig. 12 may be those included in the device 100. Fig.

12, in order to display a web page on a display screen, a web browser of the device 100 may generate a web page from a language file HTML file 405 for hypertext technology and a plurality of contents. The first HTML parser 415a may change the HTML file 405 and construct a first document object model (DOM) tree 420a. The DOM 430 may refer to an object presentation of the HTML file 405. The DOM 430 may be modified by a controller (PROCESSOR) 110 of the device 100.

The control unit 110 can continue to track and analyze areas of the display screen concentrated by the user while the user searches for a web page. The control unit 110 can determine the observation patterns of the user. Accordingly, the control unit 110 can determine the areas on the display screen to be examined by the user. Furthermore, the control unit 110 can determine the screen areas having a high probability of being applied to the user. Areas with a high probability of being seen by the user may include areas that the user is likely to observe.

Also, the control unit 110 can keep track of the sitting position of the user. The control unit 110 transmits information on user interest, information on recognition and non-recognition contents, information on easy-to-observe areas, and information on a user's sitting posture to an adaptive layout design module (ALDF) 435 .

The adaptive layout design module 435 interacts with the controller 110 and can transmit information in real time. The control unit 110 may change the first DOM tree 420a while the first DOM tree 420a is configured to display the first content according to the priority of the preference region. Here, the first content may include the highest priority content. But is not limited thereto.

To form a modified layout of the first DOM tree 420a, the adaptive layout design module 435 identifies the node elements in the first DOM tree 420a for representing the contents on the display preference area. According to one embodiment, a second content having a lower content priority may be located on the preference area of the highest priority. In this case, the adaptive layout design module 435 may change the first DOM tree 420a and replace the second content with the first content. Also, the second content can be moved onto the lower priority region.

Style sheet 410 may store information about the appearance of the current web page in real time.

The second HTML parser 415b may modify the stylesheet 410 and obtain a second document object model (DOM) tree 420b.

While the HTML parsers 415a and 415b constitute the DOM trees 420a and 420b, the web browser may configure the renderer 450. [

The renderer 450 may be generated from a process called attachment 425. [ The renderer 450 may refer to a tree with a plurality of content arranged in a presentation order. The renderer 450 may be a visual representation of a web page.

The device 100 may perform the layout process 445 to determine information related to the location and size of the multiple content on the web page.

The paint process 455 may arrange a number of content within a web page.

The display process 460 may display the modified web page on the display screen.

According to one embodiment, the device 100 may include a URL ranking module, although it is not shown in the figures.

The URL ranking module may include a browser history module, a classification module, an additional window module, a URL analysis module, a URL data retrieval module, a content recommendation module, and a rules module. The URL ranking module may sort URLs among a plurality of content in a web page in order according to user interest, content priority, priority of a preference region, and predefined rules. Here, data items within a web page may represent URLs. Examples of data items representing URLs within a web page may include, but are not limited to, text data. URLs within web pages can point to data locations within servers connected to a web browser over the Internet. The data locations within the servers may store at least one of a web page, a media file, an image, an executable program file, and a contact information file.

The URL data retrieval module can identify the URLs in the web page. In addition, the URL data retrieval module may pre-populate URLs within the web page in the device 100. In order to pre-install the URLs, the URL data retrieval module can perform three processes. First, the URL data retrieval module can access the data stored in the data locations shown by the URLs. Second, the URL data retrieval module may retrieve the date of publication of data stored within the locations specified by the URLs. Third, the URL data retrieval module can retrieve metadata information related to URLs. Here, examples of metadata information associated with URLs may include the date of publication of the URL, the expiration time, and the data author. The URL data retrieval module can transmit the data indicated by the URL, the date of publication, and metadata information related to the URL to the URL analysis module.

The URL analysis module may retrieve historical information about interest related to the user ' s content from the storage. The URL analysis module may analyze data items representing URLs within a web page. In addition, the URL analysis module may determine user interest in the data items due to historical information about user interest. Moreover, the URL analysis module can analyze the data stored in the data locations indicated by the URL. The URL analysis module may also determine user interest in the data stored in editor locations based on historical information about user interest. The URL analysis module may analyze the metadata information associated with the URLs. The URL analysis module may determine user interest in the metadata associated with the URLs. Further, the URL analysis module determines the level of conformance to the URLs, which includes user interest in the data item, user interest in the data stored in the data location indicated by the URL, date of publication of the data stored in the data location, Based on at least one of the user's interest in the associated metadata. The URL analysis module may send information about the conformance level of the URL to the classification module.

The classification module can operate in cooperation with the rule module. This is to classify the URLs based on the suitability of the URLs and predefined rules. Predefined rules can be stored in the rule module. The predefined rules may provide guidelines for evaluating URLs according to a number of ranking parameters. Examples of ranking parameters may include the suitability of the URL, the date of publication of the data stored in the data location indicated by the URL, and the user's interest in the data items associated with the URL. In one embodiment of the present invention, predefined rules may be assigned by the user. In another embodiment of the present invention, the predefined rules may be integrated with a web browser.

In one embodiment, the URL data retrieval module may access data in the first and second web pages. The URL analysis module can analyze the data in the first and second web pages. The URL analysis module can assign a higher priority to the first URL compared to the second URL. The URL analysis module may send information about the conformance levels of the first and second URLs to the classification module.

The classification module works in conjunction with the rules module to determine the priority of the first and second URLs based on the suitability of the URLs and predefined rules. The predefined rules can provide a guideline for evaluating the first and second URLs in order of decreasing the conformance level of the first and second URLs. For example, the classification module may assign a higher ranking to the first URL than the second URL.

According to one embodiment, the predefined rules may be specific to the start of a web page in a web browser. Further, the weighting levels may be assigned to ranking parameters. The weighting levels of the ranking parameters may indicate the level of influence of the ranking parameters when determining the sequence of the URL. In one embodiment, the weighting levels may be assigned by the user. In another embodiment, the weighting level may be assigned by a web browser. In other embodiments, the weighting levels may be specific to the start of a web page in a web browser.

The classification module may send information about the ranking of the URL to the content recommendation module and the adaptive layout design module 435.

In addition, the content recommendation module may have a blocking rank for the URL. The content recommendation module may select the first, second, and third sets of URLs according to predefined rules within the rules module. The content recommendation module may allow a user to determine user priorities for selection of URLs. The content recommendation module may select the first set of URLs from the start of the web page in the web browser. The content recommendation module may also select a second set of URLs from the first web page listed in the browsing history module. The content recommendation module may also select a third set of URLs from a second web page listed as a URL at the start of a web page in the web browser. The content recommendation module may send information for the first, second and third sets of URLs to the additional window module.

The additional window module can launch additional windows in the web browser and list the first, second and third URL sets on additional windows. In one embodiment, the additional window may be a dialog box of a web browser. In another embodiment, the additional window may be a toolbar in a web browser. In another embodiment, the additional window module may fuse the additional window to the web page in the web browser. In another embodiment, the additional window module may display the additional window on a lower priority preference area in the web page in the web browser. In another embodiment, the user may prevent additional windows from being executed.

Consequently, the device 100 may adjust and change the layout of the web page based on changes in the user's line of sight and sitting posture. Thus, the user can easily recognize a large number of contents.

According to one embodiment, it is possible to change at least one of a position, a direction, an image size, a color, a text font size, and a font color of a text of a plurality of contents according to a sitting posture of the user. Further, in another embodiment, the device 100 may change at least one of the font size and color of a plurality of content based on the distance of the user's face from the display screen.

13 and 14 are functional block diagrams showing the configuration of a device according to an embodiment.

As shown in FIG. 13, the device 100 may include a control unit 110, a display unit 120, and a sensing unit 130. However, not all of the components shown in Fig. 20 are essential components of the device 100. Fig. The device 100 may be implemented by more components than the components shown in Fig. 20, or the device 100 may be implemented by fewer components than those shown in Fig.

14, the device 100 according to some embodiments includes a power supply unit 140, a communication unit 150, a power supply unit 150, A storage unit 160, an input / output unit 170, a video processing unit 180, an audio processing unit 190, and an audio output unit 195.

The display unit 120 may convert a video signal, a data signal, an OSD signal, a control signal, and the like processed by the controller 110 to generate a driving signal. The display unit 120 may be implemented as a PDP, an LCD, an OLED, a flexible display, or a 3D display. Also, the display unit 120 may be configured as a touch screen and used as an input device in addition to the output device.

According to one embodiment, the display unit 120 may display a plurality of contents. Also, the display unit 120 can display a user's viewpoint set according to the user's gaze. Also, the display unit 120 can display the content on the preference area generated based on the preference area information set based on the user's line of sight.

The control unit 110 generates preference area information related to the preference area preferred by the user among the entire screen area of the device 100 based on the sensed line of sight of the user and selects at least one of the plurality of contents, By changing the display position of the selected content based on the preference area information, a plurality of contents can be rearranged.

Also, the control unit 110 can select the content to be recommended to the user based on the content usage history information of the user.

Here, the content usage history information includes information such as the number of times the user looked at the content, the amount of time the user took the content, the frequency of use of the content, the user using the content, the storage location of the content, the time using the content, History and history of interruption, reservation of use of content, registration of the item of interest in the content, and information on the available time of the content.

In addition, the content usage history information includes information on a facial expression and posture while the user looks at the content, and the control unit 110 may select the content based on the expression and attitude of the user who is looking at the content have.

In addition, the control unit 110 can select a content to be recommended to the user based on at least one of the content attribute information of the content and the user attribute information for the user and the content usage history information of the user.

In addition, the content attribute information includes information on the type and genre of the content watched by the user, and the control unit 110 may select the content based on the type and genre of the content watched by the user.

In addition, the user attribute information includes information on the user's usage rights and the location of the user with respect to the content, and the control unit 110 may select the content based on the usage rights of the user and the location of the user.

The preference region information may include at least one of a size, a shape, a position, and a priority of the preference region.

In addition, the preference area information may be generated based on at least one of a path of a perceived line of sight of a user, a size of a time when a user's line of sight is sensed, and a number of times a line of sight of the user is sensed.

Also, the preference area may be set by the user.

In addition, the control unit 110 determines a priority order of a plurality of contents displayed on the screen, selects a content among a plurality of contents based on the preference region information and the priority order of a plurality of contents, You can change the location.

In addition, the controller 110 moves the selected content to the preferred region to change the position of the selected content, and when the non-selected content among the plurality of contents is located in the preferred region, Area.

In addition, the control unit 110 may remove at least some of the unselected contents from among the number of contents on the screen.

Further, the control unit 110 can change the shape of the selected content based on the preference area information.

In addition, the control unit 110 may acquire at least one new content, determine a location where the selected content and at least one new content are arranged, based on the preference area information, You can rearrange the content.

In addition, the control unit 110 may determine the priority of the plurality of contents, change at least a part of the plurality of contents into new contents based on the determined priority, , The content corresponding to a predetermined rank or lower can be changed to new content.

Also, the control unit 110 can change the background color of the preference area based on the preference area information.

The control unit 110 may include embedded electronic circuitry for processing and functional control of the device 100. In addition, the control unit 110 may include a control module and a calculation module. The computing module may perform mathematical and logical decisions on the controller 110. The control module may control the association module and the communication unit 150.

The sensing unit 130 may sense a user's relative position with respect to the device 100. More specifically, the sensing unit 130 may photograph a user to detect the position of the user.

In addition, the sensing unit 130 may sense the user's gaze based on at least one of the expanded state of the user's pupil, the direction of the user's gaze, the presence of the user's eyes, and the retina pattern of the user.

In addition, the sensing unit 130 may display a cursor corresponding to the user's gaze on the screen of the device 100, and may display a cursor movement path changed according to the movement of the user's gaze.

In addition, the sensing unit 130 may change the background color of the content meeting the movement path along the movement path of the cursor.

Here, the content may include at least one of text, a hyperlink, an image, an icon, a UI, an audio file, an image file, an advertisement, and a web page object.

Referring to FIG. 14, the sensing unit 130 may include a camera unit 131 and a light receiving unit 132.

The camera section 131 can photograph the user in the camera recognition range. In addition, the camera unit 131 can receive an image corresponding to the motion of the user including the gesture in the camera recognition range. For example, the recognition range of the camera unit 131 may be within 0.1 to 5 meters from the camera unit 131 to the user. The user motion may include, for example, a user's body part, such as a user's face, a facial expression, a hand, a fist, a finger, or a motion of a part of a user. The camera unit 131 converts the received image into an electric signal under the control of the controller 110 and outputs the converted electric signal to the controller 110. [

According to one embodiment, the camera unit 131 photographs the face of the user including the user's pupil so that the control unit 110 can use the device 100 to identify the user's eyes, And outputs the image to the control unit 110. [

In addition, the camera unit 131 may include a plurality of camera modules. The control unit 110 may activate at least one of the plurality of cameras included in the camera unit 131 based on the current form of the device 100. [

The camera unit 131 may analyze the user's gaze while searching for a web page. In one embodiment, the camera unit 131 may include at least one of a web camera, a mobile phone camera, and a digital single lens reflex (DSLF) camera. The camera unit 131 can monitor a user's gazing pattern by tracking a plurality of characteristics of a user's behavior.

According to one embodiment, an example of a number of characteristics of user behavior may include at least one of a user's gaze, gaze, emotion, eye presence, eye position, blink rate of an eyelid, iris pattern, and pupil dilation . But is not limited thereto. In addition, the camera unit 131 may transmit information on user behavior to the control unit 110 and the storage unit 160.

In addition, the camera unit 131 may be composed of a lens (not shown) and an image sensor (not shown). The camera unit 131 can support optical zoom or digital zoom using a plurality of lenses and image processing. The recognition range of the camera unit 131 can be variously set according to the angle of the camera and the surrounding environment conditions. When the camera unit 131 is composed of a plurality of cameras, it is possible to receive a three-dimensional still image or a three-dimensional motion using a plurality of cameras.

The camera unit 131 may be integrated with the device 100 or separately. A separate device (not shown) including the separated camera unit 131 may be electrically connected to the device 100 through the communication unit 150 or the input / output unit 170.

The power supply unit 140 may supply power to the internal components of the device 100 from an external power source under the control of the controller 110. [ In addition, the power supply unit 140 can supply power to the internal components from one or more batteries (not shown) located inside the device 100 under the control of the controller 110. [

The light receiving unit 132 may receive an optical signal corresponding to a user input (e.g., touch, press, touch gesture, voice, or motion) to the device 100. [ The control signal can be extracted from the received optical signal under the control of the control unit 110. [

The communication unit 150 can connect the device 100 to an external device (for example, an audio device or the like) under the control of the control unit 110. [ The control unit 110 can transmit / receive images to / from an external device connected through the communication unit 150, download an application from an external device, or perform web browsing. The communication unit 150 may include one of a wireless LAN 151, a Bluetooth 152, and a wired Ethernet (Ethernet) 153 according to the performance and structure of the device 100. The communication unit 150 may include a combination of a wireless LAN 151, a Bluetooth 152, and a wired Ethernet (Ethernet) 153. The communication unit 150 can receive an image control signal from an external device under the control of the control unit 110. [ The control signal may be implemented as a Bluetooth type, an RF signal type, or a WiFi type.

The communication unit 150 may further include a near field communication (for example, NFC (near field communication), not shown), BLE (bluetooth low energy, not shown) in addition to Bluetooth.

The storage unit 160 includes an image receiving module, a display transformation detecting module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, a light receiving module, a display control module, an audio control module, , A power control module, a power control module of an external device connected to the wireless (e.g., Bluetooth), a voice database (DB), or a motion database (DB). Unlisted modules and a database of the storage unit 160 are provided in the device 100 for controlling image reception, a volume control function, a communication control function, a voice recognition function, a motion recognition function, a light reception control function, a display control function , An audio control function, an external input control function, a power control function, or a power control function of an external device connected to a wireless (for example, Bluetooth).

The storage unit 160 may store the preference area information determined based on the user's gaze and the movement path of the viewpoint according to the movement of the gaze. The storage unit 160 may analyze the movement of the line of sight and map the movement path of the content and the line of sight. The storage unit 160 may store the priority of the content and the priority of the preferred region.

The control unit 110 may perform the respective functions using the software stored in the storage unit 160. [

Storage 160 may include both non-volatile and volatile media. Also, the storage unit 160 may include optical or magnetic disks. Volatile media may include dynamic memory, such as memory.

The storage 160 may be any type of storage medium such as a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, CD-ROM, any other optical media, punch cards, paper tapes, PROM, EPROM, FLASH-EPROM, any other memory stick or cartridge.

The input / output unit 170 may receive video (e.g., moving picture), audio (e.g., audio, music, etc.) and additional information (e.g., a moving picture) from the outside of the device 100 under the control of the control unit 110 EPG, etc.). The input / output unit 170 includes one of a high-definition multimedia interface port 171, a component jack 172, a PC port 173, and a USB port 174 . ≪ / RTI > The input / output unit 170 may include a combination of an HDMI port 171, a component jack 172, a PC port 173, and a USB port 174.

It will be readily understood by those skilled in the art that the configuration and operation of the input / output unit 170 can be variously implemented according to the embodiments of the present invention.

The video processing unit 180 may perform processing on the video data rendered by the device 100. [ The video processing unit 180 may perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, resolution conversion, and the like on the video data.

The audio processing unit 190 may perform processing for audio data. In the audio processing unit 190, various processes such as decoding or amplification of audio data, noise filtering, and the like may be performed. Meanwhile, the audio processing unit 190 may include a plurality of audio processing modules for processing audio corresponding to a plurality of images.

The audio output unit 195 may output the audio included in the content under the control of the control unit 110. [ The audio output unit 195 may output audio (e.g., voice, sound) input through the communication unit 150 or the input / output unit 170. [ The audio output unit 195 may output audio stored in the storage unit 160 under the control of the control unit 110. [ The audio output unit 195 may include at least one of a speaker 196, a headphone output terminal 197 or a Sony / Philips Digital Interface (S / PDIF) 198. The audio output unit 195 A speaker 196, a headphone output terminal 197, and an S / PDIF output terminal 198. [

According to one embodiment, the device 100 may receive signals from external input devices. For example, it is possible to input the preference region information by the user from the input devices. Examples of the input device may include a touch pad, a keyboard, a touch screen, an electronic pen, a microphone, and a camera. But is not limited thereto.

According to one embodiment, the device 100 can provide a significant effect of dynamically rearranging the content on a web page according to user interest. Further, according to one embodiment, the device 100 may enhance the appearance of the content on the web page. Furthermore, the present invention can reduce the probability that a user skips relevant content when searching through a web page. In addition, the present invention can reduce the probability of skipping content having a high preference in use when searching through a web page. Also, according to one embodiment, the device 100 may analyze multi-user interactions to analyze user behavior. Examples of user interactions may include, but are not limited to, the number of blinks, facial expressions and emotions of the user. In analyzing user behavior, using a plurality of behaviors of a user may result in higher accuracy than using only the user's gaze.

In addition, according to one embodiment, the device 100 may perform automatic retrieval of new content from a server. In addition, the present invention can provide ergonomic support for a long time during the search. Moreover, the present invention can provide cognitive support to the user during retrieval. In addition, the present invention can adaptively support the support of people with disabilities and help them to efficiently search for content.

According to one embodiment, the device 100 may change the presentation of the content according to a change in the user's posture. According to one embodiment, the device 100 may support adaptive and dynamic web page layout design. Further, according to one embodiment, the device 100 may enable a user to locate specific content within a web page.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

Claims (43)

A method for arranging content displayed on a screen by a device,
Sensing a user's gaze on a screen of the device;
Generating preference region information related to a preference region preferred by the user among the entire screen regions of the device based on the sensed line of sight of the user;
Selecting at least one of a plurality of contents displayed on a screen of the device;
Rearranging the plurality of contents displayed on the screen of the device by changing a display position of the selected content based on the generated preference area information;
/ RTI >
The method according to claim 1,
Wherein the step of selecting the content comprises:
And selecting the content to be recommended to the user based on the content usage history information of the user.
3. The method of claim 2,
The content usage history information includes:
A usage time of the content, a user using the content, a storage location of the content, a time using the content, a usage time of the content, a usage time of the content, A completion history and an abort history, a reservation of use of the content, a registration of a favorite item of the content, and information on an available time of the content.
The method according to claim 2, wherein
Wherein the content usage history information includes information on a user's facial expression and posture while the user is gazing at the content,
Wherein the step of selecting the content comprises:
And selecting the content based on the user ' s look and attitude gazing at the content.
The method according to claim 1,
Wherein the step of selecting the content comprises:
Selecting content to be recommended to the user based on at least one of the content attribute information of the content and the user attribute information for the user and the content usage history information of the user.
6. The method of claim 5,
Wherein the content attribute information includes information on a type and a genre of the content watched by the user,
Wherein the step of selecting the content comprises:
And the content is selected based on the type and genre of the content that the user has looked at.
6. The method of claim 5,
Wherein the user attribute information comprises:
A usage right of the user for the content, and information about a location of the user,
Wherein the step of selecting the content comprises:
And selecting the content based on the usage rights of the user and the location of the user.
The method according to claim 1,
Wherein the preference region information includes:
Wherein the preference region comprises at least one of a size, a shape, a position, and a priority of the preference region.
The method according to claim 1,
Wherein the preference region information includes:
A path of the detected user's gaze, a magnitude of a time at which the user's gaze is sensed, and a number of times the gaze of the user is sensed.
The method according to claim 1,
Wherein the preference area is set by the user.
The method according to claim 1,
The step of rearranging the content comprises:
Determining a priority of a plurality of contents displayed on the screen;
Selecting a content among the plurality of contents based on the preference region information and the priority of the plurality of contents; And
Changing a position of the selected content;
/ RTI >
12. The method of claim 11,
Wherein changing the location of the selected content comprises:
Moving the selected content to the preference area,
And moving the unselected content to a screen area outside the preference area when the non-selected content among the plurality of contents is located in the preference area.
The method according to claim 1,
The step of rearranging the content comprises:
Removing at least some of the unselected contents from the plurality of contents on the screen;
/ RTI >
The method according to claim 1,
The step of rearranging the plurality of contents comprises:
And changing a display shape of the selected content based on the preference area information.
The method according to claim 1,
The step of rearranging the plurality of contents comprises:
Obtaining at least one new content;
Determining a location where the selected content and the at least one new content are arranged based on the preference region information;
Rearranging the selected content and the at least one new content in the determined location; ≪ / RTI > 16. The method of claim 15,
The step of rearranging the plurality of contents comprises:
Determining a priority of the plurality of contents;
Changing at least a portion of the plurality of contents into the new content based on the determined priority;
If the priority of the content corresponds to a predetermined rank or less, changing the content corresponding to the predetermined rank or less to new content. The method according to claim 1,
And changing the background color of the preference area based on the preference area information.
The method according to claim 1,
Wherein the step of sensing the user's gaze comprises:
Wherein the user's gaze is sensed based on at least one of an expanded state of the user's pupil, an orientation of the user's gaze, an existence of the user's eye, and a retina pattern of the user.
The method according to claim 1,
Wherein the step of sensing the user's gaze comprises:
Displaying a cursor corresponding to the user's gaze on the screen of the device; And
Displaying a movement path of a cursor to be changed according to movement of the user's gaze;
/ RTI >
20. The method of claim 19,
Further comprising changing a background color of the content meeting the movement path along the movement path of the cursor.
The method according to claim 1,
Wherein the content comprises at least one of text, a hyperlink, an image, an icon, a UI, an audio file, an image file, an advertisement and a web page object.
A device for arranging content displayed on a screen,
A sensing unit for sensing a user's gaze on a screen of the device;
A display unit for displaying a plurality of contents; And
Generating preference area information related to a preference area preferred by the user from the entire screen area of the device based on the sensed line of sight of the user, selecting at least one of the plurality of contents, And a control unit for rearranging the plurality of contents by changing a display position of the selected content based on the selected content.
23. The method of claim 22,
Wherein,
And to select the content to recommend to the user based on the content usage history information of the user.
24. The method of claim 23,
The content usage history information includes:
A usage time of the content, a user using the content, a storage location of the content, a time using the content, a usage time of the content, a usage time of the content, A completion history and an abort history, a reservation of use of the content, a registration of a favorite item of the content, and information on an available time of the content.
The method of claim 23, wherein
Wherein the content usage history information includes information about a facial expression and a posture while the user looks at the content,
Wherein,
And select the content based on the user ' s look and attitude gazing at the content.
23. The method of claim 22,
Wherein,
And selects content to be recommended to the user based on at least one of the content attribute information of the content and the user attribute information for the user and the content usage history information of the user.
27. The method of claim 26,
Wherein the content attribute information includes information on a type and a genre of the content to be watched by the user,
Wherein,
And selects the content based on a type and a genre of the content to be watched by the user.
27. The method of claim 26,
Wherein the user attribute information comprises:
A usage right of the user for the content, and information about a location of the user,
Wherein,
And based on the usage rights of the user and the location of the user.
23. The method of claim 22,
Wherein the preference region information includes:
Wherein the preference region comprises at least one of a size, a shape, a position, and a priority of the preference region.
23. The method of claim 22,
Wherein the preference region information includes:
Wherein the device is generated based on at least one of a path of the sensed user's gaze, a size of the time when the user's gaze is sensed, and a number of times the gaze of the user is sensed.
23. The method of claim 22,
Wherein the preference region comprises:
Lt; RTI ID = 0.0 > user. ≪ / RTI >
23. The method of claim 22,
Wherein,
Determining a priority of a plurality of contents displayed on the screen, selecting a content among the plurality of contents based on the preference region information and the priority order of the plurality of contents, changing a position of the selected content Device. 33. The method of claim 32,
Wherein the control unit is configured to, in order to change the position of the selected content,
Moving the selected content to the preference area,
And moves the unselected content to a screen area other than the preference area when the non-selected content among the plurality of contents is located in the preference area.
23. The method of claim 22,
Wherein,
And removes at least some of the unselected content from the plurality of content on the screen. 23. The method of claim 22,
Wherein,
And changes the shape of the selected content based on the preference region information. 23. The method of claim 22,
Wherein,
Determining, based on the preference region information, a location at which the selected content and the at least one new content are arranged, and selecting the selected content and the at least one new content at the determined location Devices that are rearranged. 37. The method of claim 36,
Wherein,
A priority determination unit configured to determine a priority of the plurality of contents and change at least a part of the plurality of contents to the new content based on the determined priority and if the priority of the content falls below a predetermined rank, And changes the content corresponding to the predetermined rank or lower to new content. 23. The method of claim 22,
Wherein the control unit changes the background color of the preference area based on the preference area information.
23. The method of claim 22,
The sensing unit includes:
Wherein the user's gaze is sensed based on at least one of an expanded state of the user's pupil, a direction of the user's gaze, an existence of the user's eye, and a retina pattern of the user.
23. The method of claim 22,
The sensing unit includes:
A cursor corresponding to the user's gaze is displayed on the screen of the device,
And displays a movement path of the cursor to be changed in accordance with the movement of the user's gaze.
41. The method of claim 40,
And changes the background color of the content meeting the movement path along the movement path of the cursor.
23. The method of claim 22,
Wherein the content comprises at least one of text, a hyperlink, an image, an icon, a UI, an audio file, an image file, an advertisement and a web page object.
A computer-readable recording medium storing a program for causing a computer to execute the method according to any one of claims 1 to 21.

Images