KR102366677B1 - Apparatus and Method for User Interaction thereof - Google Patents

Abstract

The present invention provides a method of interaction with consumer products capable of digital TV (DTV), mobile, and 3D (3-dimensional) display. In particular, the present invention relates to a system and method for user interaction with the Web in 3D space through the above-described consumer product. Additionally, the present invention provides a plurality of user interaction scenarios in which different users simultaneously search through different content with one device without substantial interference, and enables interaction between screens of a plurality of users.

Description

Electronic device and method for user interaction thereof {Apparatus and Method for User Interaction thereof}

A user interaction system and method therefor, and more particularly, to a user interface system and method provided in a 3D form in a web space.

The following background discussion contains information useful for understanding the present invention. Any information provided herein is not an admission that it is prior art or related information to the claimed subject matter. Moreover, it is not an admission that any specific publication or implied reference is prior art or related information of the claimed subject matter.

Various 3D technologies on the market today can display content in 3D with or without a glass cover. Most of the content displayed in 3D is video and some graphics.

PCT Publication No. WO 2012/082971 discloses a system and method for activation or interacting by a user with a 3D object displayed on a 3D computer display by a user's gesture that can be combined with the user's gaze on the 3D computer display teach This public document teaches features of capturing a user's gaze by a head frame included in at least one of an endo camera and an exo camera worn by the user, and recognizing the user's gestures from a plurality of gestures. On the other hand, this document only describes interactions with 3D objects. Moreover, this document does not teach or imply real-time conversion from 2D objects to 3D objects. Additionally, this document also does not cover the real-time conversion of standard web content to 3D. Also, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

US Publication No. 2003/0197737 discloses a 2D/3D web browser for displaying both 2D and 3D information in Windows. In particular, a web browser for searching a web site providing 2D or 3D information is disclosed. A 2D/3D web browser includes software for requesting access information to a web site; and a display for displaying the information provided to the website responding to the request on a display screen having a 2D/3D switchable window and a 3D window, wherein if the information is 3D content, the information is a 2D/3D switchable window and It is displayed in the combined area of the 3D window, and if the information is 2D content, the information is displayed in the 2D/3D switchable window. This document refers to converting and displaying 2D web content in 3D virtual space. It refers to technologies such as OpenGL/WebGL. On the other hand, this document does not teach or imply real-world 3D scenarios in which objects may appear different from their depth in real 3D space. For example, stereoscopic 3D content is viewed on a DTV with or without 3D glasses. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

US Publication No. 2012/0174038 discloses a system and method for navigating and selecting content using an interactive virtual sphere. According to the document, the system contains a content database accessible through the content server. The content database is configured to store a first plurality of content units, and an aggregator connected with the content database, the aggregator is configured to update the first plurality of content units. The system also includes a spherical graphics rendering module and a content navigation module. The spherical graphics rendering module is configured to render a visual representation corresponding to the second plurality of content units on the surface of the virtual sphere. The navigation module content provides a user interface for interacting with the virtual sphere. The spherical graphics rendering module is configured to access the content database and select at least one of the second plurality of content units in response to an input received by the user interface. On the other hand, this document does not teach or imply real-world 3D scenarios in which objects may appear different from their depth in real 3D space. For example, stereoscopic 3D content is viewed on a DTV with or without 3D glasses. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

US Publication No. 2012/0192115 discloses a system and method for interactive projection on the side of a three-dimensional shape and playback of an associated media segment. On the other hand, this document does not teach or imply real-world 3D scenarios in which objects may appear different from their depth in real 3D space. For example, stereoscopic 3D content is viewed on a DTV with or without 3D glasses. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

US Patent No. 8,035,635 discloses a system for the location of content elements in virtual 3D space. According to this document, a 2D base plane is specified in 3D space, and virtual content is located in 3D space with at least one page lying parallel to the base plane. A viewpoint and a viewing ray are selected in 3D space to show 3D space with a viewing ray that is neither parallel nor perpendicular to the base plane. A 2D surface is placed in 3D space so that the 2D surface intersects a view ray between the viewpoint and the base plane. At least a portion of the 3D space is projected onto the 2D surface by a perspective projection determined by the viewing ray. On the other hand, this document does not teach or imply real-world 3D scenarios in which objects may appear different from their depth in real 3D space. For example, stereoscopic 3D content is viewed on a DTV with or without 3D glasses. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

US Patent No. 8,436,891 discloses an interactive hyperlinked 3D video insertion and an interactive method thereof. This document relates to the metadata of multimedia content and teaches rendering and interaction with the metadata content in a virtual 3D space. On the other hand, this document does not teach or imply real-world 3D scenarios in which objects may appear different from their depth in real 3D space. For example, stereoscopic 3D content is viewed on a DTV with or without 3D glasses. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

US Patent Application Publication No. 2012/0127156 discloses a method and system for representing and retrieving items in a tilted 3D space. In the tilted viewing mode, groups of items at different successive levels of the hierarchy are learned to be displayed in each searchable parade viewed at different depths into the viewing area of the tilted 3D space. Users can search horizontally across each searchable parade, up and down the hierarchy within a tilted 3D space. Combining the horizontal search through the parade displayed in the visible area of the tilted 3D space and the ability to search the hierarchy up and down according to the depth of the 3D space, the user can navigate the entire hierarchy of the tilted 3D space. can On the other hand, this document does not teach or imply real-world 3D scenarios in which objects may appear different from their depth in real 3D space. For example, stereoscopic 3D content is viewed on a DTV with or without 3D glasses. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

PCT Publication No. 2013/183801 discloses a 3D web page rendering method and a terminal using the same. The rendering method includes loading source text including 3D object depth information constituting one or more 3D web pages, and analyzing the source text to include a DOM tree (document object model tree) and depth information Steps to create a style rule, create a render tree based on the DOM tree and style rules, perform layout on the render tree, information obtained by performing the layout and 3D object Drawing the left eye and right eye pages by applying a 3D factor including the above position, size, difference, shape, and arrangement, merging the left eye and right eye pages, and merging the left eye and right eye pages in a 3D browser to display includes steps. On the other hand, this document does not teach or imply rendering of reference 2D web content available on the Internet. Furthermore, this document does not teach or imply real-time interaction with objects in 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

EP application number 2591467 discloses a method for converting 2D content data into 3D content data in a 3D TV. The method includes receiving 3D video content and 2D user interface content data through a 2D to 3D content conversion module, the difference data defining separation of left and right eyes for 3D rendering of the 2D user interface content data. determining a movement, generating 3D user interface content data at a 3D depth of a display based on the received 2D user interface content data and the determined movement, and displaying 3D video content of a display of a 3D TV include On the other hand, this document does not teach or imply steps for displaying 3D standard web content. Furthermore, this document does not teach or imply real-time interaction with objects in real 3D space. Moreover, this document does not address the multi-user scenario where different users are simultaneously browsing different content on a single device.

Therefore, not much is done to display web content in 3D. Also, the system/method that can be used to interact with 3D objects shown on the web is not generic. Moreover, systems/methods available to handle multiple user scenarios simultaneously retrieving different content from a single device are not common.

Therefore, the present invention is to provide a system and method for enabling user interaction with the web in 3D space.

According to an embodiment of the present invention for achieving the above object, an interaction method of an electronic device includes: displaying web content including at least one object on a first screen; detecting selection of at least one object by the first user; and displaying the selected object or information linked to the selected object in a 3D (3-dimensional) form on a second screen; may be an interaction method comprising.

In addition, the step of displaying in 3D form is an interaction of displaying the selected object or information linked to the selected object in 2D form, and converting the selected object in 2D form or information linked to the selected object in 2D form into 3D form could be a way

In addition, the first screen and the second screen may be an interaction method in which a sense of depth is formed.

In addition, it may be an interaction method further comprising the step of retrieving information linked to the object selected from the information source (source).

And, detecting a user interaction with the second screen; and displaying the result of the interaction on a third screen.

In addition, detecting selection of the additional object by the first user on the first screen; and an interaction method of displaying the additionally selected object or information linked to the additionally selected object on the fourth screen in a 3D form.

And, when the electronic device is in a plurality of user modes, the screen by the first user may be an interaction method characterized in that only the first user is shown.

In addition, detecting an object selection by a second user on the first screen; and displaying the object selected by the second user or information linked to the object selected by the second user on the fifth screen in 3D form; may be an interaction method further comprising.

Also, when the mode of the electronic device is switched to the sharing mode, it may be an electronic device interaction method that enables interaction between the screen corresponding to the first user and the screen corresponding to the second user.

In addition, the object may be an interaction method characterized in that it is one of an image, video, text, hyperlink, attachment, and multimedia.

According to another embodiment of the present invention, an electronic device includes: a display unit for displaying web content including at least one object on a first screen; a sensing unit configured to sense selection of at least one object by the first user; and a processor controlling the display unit to display the selected object or information linked to the selected object in a 3D form on the second screen when selection of at least one object is sensed through the sensing unit.

In addition, the processor may control the display unit to display the selected object or information linked to the selected object in 2D form, and convert the selected object or information linked to the selected object in 2D form to 3D form to be displayed. .

In addition, the processor may control the display unit to display the first screen and the second screen so as to form a sense of depth.

It is also possible to retrieve information linked to the selected object from the information source.

In addition, the processor may control the sensing unit to detect the user interaction with the second screen, and may control the display unit to display the result of the interaction on the third screen.

In addition, the processor controls the sensing unit to detect the selection of the additional object by the first user on the first screen, and controls the display unit to display the additionally selected object or information linked to the additionally selected object on the fourth screen in 3D form. can

In addition, when the electronic device is in a plurality of user modes, the processor may control the display unit to display a screen by the first user only to the first user.

In addition, the processor controls the sensing unit to detect object selection by the second user on the first screen, and displays the object selected by the second user on the fifth screen or information linked to the object selected by the second user in 3D form The display unit may be controlled to display.

In addition, the processor controls the sensing unit to detect selection of the additional object by the first user on the first screen, and displays the additionally selected object or information linked to the additionally selected object on the fourth screen in 3D form The display unit may be controlled to display.

Also, when the electronic device is in a plurality of user modes, the processor may control the display unit to display the screen of the first user only to the first user.

In addition, when the mode of the electronic device is switched to the sharing mode, interaction between the screen corresponding to the first user and the screen corresponding to the second user may be possible.

Also, the object may be an electronic device that is one of an image, video, text, hyperlink, attachment, and multimedia.

According to various embodiments of the present invention as described above, it is possible to allow a user to interact with the web in 3D space through a consumer product, and to allow a plurality of users to interact simultaneously through a single consumer product.

For a more complete understanding of the present invention and its advantages, reference is made to the accompanying drawings and the accompanying drawings and detailed description for a brief description of the drawings, in which reference numerals indicate parts, as follows.
1 illustrates an interaction method according to an embodiment of the present invention.
2 illustrates a first screen for displaying a web page having at least one object.
3 illustrates a first screen for displaying an e-mail having at least one object.
4 illustrates a second screen for displaying the selected object in 3D form.
5 shows a process for converting an object from a 2D form to a 3D form.
6 illustrates a drawing buffer including a right eye image, a drawing buffer including a left eye image, and a second screen for displaying the selected object in 3D form.
7 shows the arrangement of the first screen and the second screen to form the depth.
8 shows a method for arranging a first screen and a second screen for forming depth.
9A illustrates a drawing buffer including a right eye image, FIG. 9B illustrates a drawing buffer including a left eye image, and FIG. 9C illustrates a second screen displaying a selected object in 3D form.
10A is a first screen including a plurality of objects, FIG. 10B is a display of a second screen including information linked to an object selected from the first screen, and FIG. 10C is a display of an object selected from the second screen. A display of a third screen including linked information, FIG. 10D illustrates a display of an additional screen including information linked to an object selected from the third screen.
11A is a display of the first screen for a first user, FIG. 11B is a display of a second screen including information linked to an object selected on the first screen of the first user, and FIG. 11C is a display of the first screen of the first user. A display of a fourth screen including information linked to an object selected from the first screen is shown.
12A is a first screen for a first user and a second user, and FIG. 12B is a second screen that can be viewed only by the first user and includes information linked to an object selected by the first user on the first screen. 2 shows the screen.
13A is a display of a first screen for a first user and a second user, and FIG. 13B includes information linked to an object that can be viewed only by the first user and is selected by the first user in the first screen 13C shows a fifth screen that can only be viewed by the second user and contains information linked to an object selected by the second user in the first screen.
14A is a display of a first screen for first and second users, and FIG. 14B is a second screen that can be viewed only by the first user and includes information linked to an object selected by the first user on the first screen. 14C is a display of a fifth screen that can only be viewed by a second user and includes information linked to an object selected by the second user in a first screen, and FIG. 14D is only a second screen display. It shows the display of the sixth screen that can be viewed by the user and that includes information linked to the object selected by the second user in the fifth screen.
15A is a display of a first screen for first and second users, and FIG. 15B is a diagram that can be viewed only by the first user and includes information linked to an object selected by the first user on the first screen. 15C is a display of a fifth screen that can be viewed only by the second user and includes information linked to an object selected by the second user in the first screen, and FIG. 15D is only the second screen. A display of a sixth screen that can be viewed by two users and that includes information linked to an object selected by the second user on the fifth screen, FIG. 15E is a diagram for sharing a fifth screen from a second user to a first user Upon detecting a share command regarding the creation and display of a new screen that can be viewed by the first user.
16A is a display of a first screen for first and second users, and FIG. 16B is a diagram that can be viewed only by the first user and includes information linked to an object selected by the first user in the first screen. Display of a second screen, FIG. 16C illustrates creation and display of a new screen that can be viewed by a second user when a sharing command for sharing a second screen with a second user is sensed from a first user.
17 illustrates a system for allowing an interaction according to an embodiment of the present invention.
18 illustrates an interaction method in which displaying the selected object or information linked to the selected object in a 3D form is optional according to an embodiment of the present invention.
19 shows a second screen that is arranged to form a depth with respect to the first screen and displays the selected multimedia object.
20 illustrates a second screen that is arranged to form a depth with respect to the first screen and displays the selected text object.
21 shows a system for allowing an interaction according to an embodiment of the present invention.
Fig. 22 shows a specific system diagram shown in either Fig. 17 or Fig. 21;
23 is a flowchart illustrating an interaction method according to an embodiment of the present invention.
24 is a block diagram schematically illustrating the configuration of an electronic device according to an embodiment of the present invention.

From the start, although implementations of one embodiment of the present invention are described below, it should be understood that the present system may be implemented using any number of techniques, presently known or unknown. The invention should be limited to the illustrative embodiments, drawings, and description set forth below, including the illustrative designs and embodiments depicted herein, but may be modified within the scope of the appended claims along with the full scope of equivalents.

Handheld electronic devices such as mobile phones, personal digital assistants, handheld computers, and similar devices will be referred to as handsets. Desktops, laptops, personal computers, and similar devices will be referred to as computing devices. Digital TVs (DTVs), projectors, and similar devices will be referred to as display devices. Handsets and display devices will be referred to as consumer products. Accordingly, the present invention may be practiced in some consumer products that incorporate 3D technology (with or without glass) to display content in 3D form.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the scope of rights. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment, a “module” or “unit” performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of “modules” or a plurality of “units” are integrated into at least one module and implemented with at least one processor (not shown) except for “modules” or “units” that need to be implemented with specific hardware. can be

Hereinafter, the embodiment will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted.

Referring to FIG. 1 , displaying web content including at least one object on a first screen (S102); detecting selection of at least one object by the first user (S104); and displaying the selected object or information linked to the selected object in 3D form on the second screen (S106);

In an embodiment of the present invention, the first screen and the second screen are arranged to form a depth.

In another embodiment of the present invention, the interaction method 100 further includes converting the selected object or information linked to the selected object from the 2D form to the 3D form ( S108 ).

In addition, the interaction method 100 further includes the step of retrieving information linked to the selected object from the information source (S110).

In addition, the interaction method 100 further includes the step of detecting the interaction of the first user with the second screen (S110) and the step of displaying the interaction result (S114).

In another embodiment of the present invention, the result of the interaction is displayed on the third screen.

In addition, the third screen is arranged to form a depth with respect to the second screen.

In another embodiment of the present invention, the interaction method 100 detects the selection of an object from the first screen by the second user ( S120 ) and displays the selected object or information linked to the selected object on the fifth screen in 3D It further includes the step of displaying in the form (S122).

In addition, the interaction method 100 further includes the step of retrieving information linked to the object selected by the second user from the information source (S124).

In addition, the interaction method 100 further includes a step (S126) of converting the object selected by the second user or information linked to the object selected by the second user from the 2D form to the 3D form.

Also, the first screen and the fifth screen are arranged to form a depth.

In another embodiment of the present invention, the interaction method 100 includes detecting an additional interaction with the fifth screen by the second user (S128) and displaying the result of the interaction (S130).

In addition, a result of the second user's interaction with the fifth screen is displayed on the sixth screen.

In addition, the sixth screen is arranged to form the depth of the fifth screen.

In addition, the interaction method 100 further includes a step (S132) of enabling interaction between the screen corresponding to the first user and the screen corresponding to the second user.

In addition, the object is selected from the group comprising images, videos, texts, hyperlinks, references, and multimedia.

As described above, the interaction method includes displaying the web content on the first screen (S102). Referring now to FIG. 2 , a first screen 200 for displaying a web page is shown. As shown in FIG. 2 , the web page includes at least one object, and the web page includes three objects 202 , 204 , and 206 . Here, the objects are an image 202 , a video 204 , and a hyperlink 206 .

Referring to FIG. 3 , a first screen 300 for displaying an e-mail according to an embodiment is shown. In this case, email attachments can be treated like objects. For example, the email being displayed on the first screen 300 is shown to include a first attachment entitled “text.doc” 302 and a multimedia 304, and “text.doc” 302 . and multimedia 304 can be treated like objects. It may be noted that the first screen 300 may be displayed in any other type of web content. Also, the web content may include other types of objects not shown in FIG. 2 or FIG. 3 .

The interaction method 100 includes a step S104 of detecting selection of at least one object by the first user. The user selection may be received from any of a mouse, a keypad, a remote control, a voice command, a touch, an air gesture, and the like. Assuming that the first screen displays a web page as shown in Fig. 2, and that the user's selection of the object 206 (ie, hyperlink) is detected, the result of the selection is displayed on the second screen in 3D form ( S106). The display of the second screen may be the selected object or information linked to the selected object. Assuming the object 206 is linked to the phrase “WELCOME TO STEREOSCOPIC WORLD”, FIG. 4 shows a second screen 402 displaying the preceding phrase in 3D form, where the word “stereoscopic” is another of the phrase. The words are arranged closer to the first user than the words. Any other 3D shape may be applied.

When the selected object or information linked to the selected object is in a 2D form, the interaction method 100 further includes converting the selected object or information linked to the selected object from a 2D form to a 3D form ( S108 ). Referring to FIG. 5 , a process 500 of an embodiment for converting from a 2D form to a 3D form includes a step S502 of calling a converter module by a web engine according to detection of object selection on web content.

As a result, the converter module enables 3D display of consumer products (S504). The converter module creates a double-sized drawing buffer (a frame buffer for the left eye and a frame buffer for the right eye) in order to support separate drawing for the frame for the left eye and the frame for the right eye (S506). A copy of the new web page/image/video (ie, object) is each rendered to half of the double-sized buffer (S508). Thereafter, the converter module combines the frame buffer for the left eye and the frame buffer for the right eye to create a 3D effect (S510). However, there may be other processes available for conversion other than the process for converting from 2D to 3D form shown in Fig. 5 of the present invention.

6, the positioning 602 of the right eye image of the drawing buffer drawn by the converter module and the positioning 604 of the left eye image of the drawing buffer drawn by the converter module are shown in 3D form of the above phrase shown at 604 (402) to arrive at a second screen displaying

In a preferred embodiment of the present invention, the first screen 200 and the second screen 402 are arranged to indicate depth. As shown in FIG. 7 , the second screen 402 appears closer to the user as compared to the first screen 200 . In addition to the content displayed on the second screen 402 in 3D form, the second screen 402 is also displayed in 3D form.

In order to arrange the first screen 200 and the second screen 402 to form depth, the interaction method 800 may be a method according to an embodiment as shown in FIG. 8 . The interaction method 800 includes a step S802 of calling an interactive 3D window by a web engine according to the step of detecting a selection of an object on the web content.

As a result, the interactive 3D module enables the 3D display mode of the consumer product (S803). The interactive 3D module creates a double-size drawing buffer to support separate drawing for frames for the left and right eyes (S806). A copy of the new web page/image/video (ie, object) is each rendered to half of the double-size buffer (S808). The frame buffer for the left eye and the frame buffer for the right eye are combined to create a 3D depth effect (S810). However, any other alternative process of arranging the first screen 200 and the second screen 402 to indicate depth may be applied, and the present invention is not limited to the process shown in FIG. 8 .

Referring to FIG. 9 , the positioning 902 of the right eye image of the drawing buffer drawn by the interactive 3D module and the positioning 904 of the left eye image of the drawing buffer drawn by the interactive 3D module are the last as shown in reference numeral 910 . It is shown at 906 and 908 to reach the display. A first screen 200 and a second screen 402 are disposed at reference numeral 910 to form a depth.

As described above, the interaction method 100 further includes the step of detecting the interaction between the first user and the second screen (S112) and the step of displaying the result of the interaction (S114). The result of the interaction may be displayed on a third screen, and the third screen may be arranged to indicate depth with respect to the second screen. The process may be repeated for the third screen.

As shown in FIG. 10 , according to an embodiment of the present invention, FIG. 10A shows a first screen including a plurality of objects 202 , 204 , and 206 . Assuming that the user selects the object 206 , information linked to the selected object 206 is displayed on the second screen 402 as shown in FIG. 10B . The second screen includes at least one object 1002, 1004, 1006. Assuming that the first user selects the object 1002, the information linked to the selected object 1002 as shown in FIG. 10C is It is displayed on the third screen 1008 . As shown in 10C, the third screen 1008 is arranged to indicate the depth to the second screen 402 . This process may be further repeated. In particular, if the third screen 1008 includes at least one object, the selection result of the object selection on the third screen 1008 may be displayed on an additional screen. 10C shows that the third screen 1008 includes three objects 1010 , 1012 , and 1014 . Assuming that the user selects the object 1012 , information linked to the selected object 1012 is displayed on an additional screen 1016 as shown in FIG. 10D .

In some examples, the user interaction with the second screen may be displayed on the second screen instead of the opening of the third screen. For example, if the second screen displays an image or text, and the interaction increases the size of the image or text being displayed, the result of the interaction may be displayed on the second screen instead of opening the third screen.

Also, if the second screen is displaying an image and the interaction is to rotate the displayed image, the result of the interaction may be displayed on the second screen instead of opening the third screen.

Also, if the second screen is displaying a video and the interaction is any one of play, pause, etc., the result of the interaction may be displayed on the second screen instead of opening the third screen. Therefore, displaying the result of the interaction in the additional window includes displaying the result in the same screen.

In addition, the interaction method 100 includes the steps of detecting selection of an additional object by the first user on the first screen 202 ( S116 ) and displaying the selected object or information linked to the selected object in a 3D form on the fourth screen. It further includes step S118. This is shown in FIG. 11 .

Referring to FIG. 11A , the first screen 200 is displayed to the first user. Assuming that selection of the object 202 is detected on the first screen 200, as shown in FIG. 11B , the second screen 402 including the selected object or information linked to the selected object is displayed to the first user .

At this point, the interaction method 100 may allow the first user to further select one of the first screen 200 or the second screen 402 . If the first user selects an additional object on the first screen 200 , the interaction method 100 detects the additional selection. Assuming that the selection of the object 206 is detected, the interaction method 100 performs to display the selected object or information linked to the selected object on the fourth screen 1102 as shown in FIG. 11C .

Further, in the case of a multi-user consumer product (ie, a consumer product capable of being controlled by a plurality of users in an independent manner and capable of displaying an image in 3D form), the interaction method shown in FIG. 12A is the first screen 200 . ) and displaying the web content for viewing by the first user and the second user.

If it is assumed that the first user selects an object on the first screen 200, the plurality of user modes are ON, and the selected object or information linked to the selected object is displayed on the second screen 402, FIG. 12B is shown. As described above, the second screen 402 can be shown only to the first user. The second user may continue to watch the first screen 200 without any interruption from the second screen 402 . The content of the second screen 402 (viewed by the first user) may be in 3D form. Moreover, to the first user, the second screen 402 may appear closer as compared to the first screen 200 .

In addition, the interaction method 100 includes the steps of detecting object selection by the second user from the first screen (S120) and displaying the selected object or information linked to the selected object in 3D form on the fifth screen (S122) further includes

As shown in FIG. 13A , the interaction method 100 includes, for example, displaying web content on the first screen 200 for viewing by a first user and a second user.

It is assumed that the first user selects an object from the first screen 200, the plurality of user modes are ON, and the selected object or information linked to the selected object is displayed on the second screen 402, as shown in FIG. 13B. As described above, the second screen 402 can be shown only to the first user. The second user may continue to watch the first screen 200 without any interruption from the second screen 402 . The content of the second screen 402 may be in a 3D form. Furthermore, the second screen 402 may appear closer to the first user as compared to the first screen 200 .

Assuming that the second user selects an object from the first screen 200 , as shown in FIG. 13C , a fifth screen 1302 that can be viewed only by a third user is opened. The first user can continue viewing the second screen 402 without any interruption by the fifth screen 1302 . For both the first user and the second user, the first screen 200 may appear as a background. The content of the second screen 402 (viewed by the first user) and the fifth screen 1302 (viewed by the second user) may be in 3D form. Moreover, the second screen 402 may appear closer to the first user than the first screen 200 . The fifth screen 1302 may appear closer to the second user than the first screen 200 .

In addition, the interaction method 100 may further include the steps of detecting an additional interaction with the fifth screen 1302 by the second user (S128) and displaying the result of the interaction (S130). This is shown in FIG. 14 .

Here, Figures 14A, 14B, 14C are similar to Figures 13A, 13B, 13C.

Look at Figure 14D. Assuming that the second user selects an object displayed on the fifth screen 1302 , the selection of the object by the second user is detected, and the selected object or information linked to the selected object is displayed on the sixth screen 1402 . . In a preferred embodiment of the present invention, the sixth screen 1402 is arranged to form a depth for the fifth screen 1302 .

In addition, the interaction method 100 further includes the step of allowing an interaction between the screen corresponding to the first user and the screen corresponding to the second user (S132). This is shown in FIG. 15 .

Here, Figs. 15A, 15B, 15C, 15D are similar to Figs. 14A, 14B, 14C, 14D.

15D and 15E , the interaction method 100 further includes detecting a second user sharing of the fifth screen 1402 with the first user. According to the detection, the interaction method 100 generates a new screen 1502 that can be shown to the first user, and displays the contents of the fifth screen 1402 on the created new screen 1502 .

Referring to FIG. 16 , a scheme similar to that shown in FIG. 15 , but reversed.

Referring to FIG. 16A , the display of the first screen 200 for a first user and a second user is shown. Assuming that the object selection is detected by the first user, the plurality of user modes are in the ON state, and the selected object or information linked to the selected object is displayed on the second screen 402, as shown in FIG. 16B, the second The screen 402 may be shown only to the first user.

Assuming that the second user continues to watch the first screen 200 and a share command of the second screen 402 together with the second user is sensed from the first user, the interaction method 100 is a new screen ( 1602) and display the contents of the second screen 402 on the screen 1602. It is also possible to create a single screen that can be viewed by the first user and the second user according to the sharing command detection.

Since each of the one or more screens may be interpreted as a graphical user interface, in an embodiment of the present invention, the first screen for displaying web content including at least one object and the selected object or selected object in 3D form It is noteworthy that a graphical user interface including a second screen for displaying information linked to an object is provided.

Turning now to an aspect of the system, the method described above may be performed with the system as shown in FIG. 17 . The system described herein may be implemented in an electronic device. 23 24 as will be seen below.

Referring now to FIG. 17 , a system 1700 for carrying out the invention described above includes a module for receiving and storing web content 1702 , which module may include a web browser or web based user interface (UI) module 1704 . is effectively linked to A web browser or UI based web module 1702 is further linked to a web engine 1706 and a display buffer 1708 .

According to a preferred embodiment of the present invention, the web engine 1706 may perform sensing by the user.

According to a preferred embodiment of the present invention, the system 1700 further comprises a converter module for converting the selected object or information linked to the selected object from 2D to 3D form by performing the process as described above. . The system 1700 further includes an interactive 3D window module 1712 for arranging the first screen 200 and the second screen 402 for indicating depth.

The method of interaction 100 of the present invention may be advantageously performed on a consumer product available to multiple users, a consumer product capable of being controlled by multiple users in an independent manner and capable of displaying images in 3D form. Therefore, the present invention also includes the steps of displaying web content including at least one object on the first screen; detecting selection of at least one object by the first user; detecting selection of at least one object by a second user; displaying the object selected by the first user or information linked to the object selected by the first user in a 3D form on a second screen; It provides an interaction method comprising; displaying the object selected by the second user or information linked to the object selected by the second user in a 3D form on a third screen.

In the above-described embodiment, the interaction method further includes converting the selected object or information linked to the selected object from a 2D form to a 3D form. In the aspect mentioned, each of the second screen and the third screen is arranged to form a depth with respect to the first screen. Additionally, the interaction method may include detecting an interaction with a second screen by a first user and displaying a result of the interaction on a fourth screen; and/or detecting an interaction with the third screen by the second user and displaying the interaction result on the fifth screen.

According to the above-described embodiment, the present invention provides a first screen for displaying web content, the web content including at least one object; a second screen for displaying an object selected by the first user or information linked to the object selected by the first user in a 3D form selected by the first user; It further provides a graphical user interface including a third screen for displaying an object selected by a second user in a 3D form or information linked to the object selected by the second user.

In order to perform the above method, the present invention further provides a user interface engine configured to display web content on a first screen, the web content including at least one object; a sensing module configured to detect selection of the at least one object by the first user and detect selection of the at least one object by the second user; and displaying the object selected by the first user or information linked to the object selected by the first user on a second screen, and the object selected by the second user or the second user on a third screen an interactive 3D module configured to display information linked to the object selected by ; It provides an interaction system that includes

In the embodiment described above, the selected object or information linked to the selected object is displayed in 3D form on an appropriate screen (eg, second, third, fourth, fifth or sixth screen). In an alternative embodiment, the second, third, fourth, fifth or sixth screen may be positioned to form a depth for the first screen, and the selected object or information linked to the selected object is in 3D form. It is noteworthy that it does not need to be displayed as . Then, it will be described in detail in the next paragraph. In this embodiment, the point of displaying the selected object or information linked to the selected object in 3D form is not necessarily a choice but a selectable choice.

According to an embodiment of the present invention shown in FIG. 18 , the interaction method 1800 includes displaying web content including at least one object on a first screen ( S1802 ); detecting selection of at least one object by the first user (S1804); and displaying the selected object or a second screen including information linked to the selected object ( S1806 ), and a second viewing window arranged to indicate the depth of the first screen.

According to another embodiment of the present invention, the interaction method 1800 further includes retrieving information linked to the object selected from the information source (S1808).

According to an embodiment of the present invention, the second screen displays the selected object or information linked to the selected object in 2D form, whereas according to another embodiment of the present invention, the second screen displays the selected object or information linked to the selected object. Linked information is displayed in 3D form.

When the selected object or information linked to the selected object is in a 2D form, if the user allows the first screen to display the selected object or information linked to the selected object in 3D form, the interaction method 1800 is the selected object or the selected object The method further includes converting the linked information from the 2D form to the 3D form (S1810).

According to another embodiment of the present invention, the interaction method 1800 further includes detecting an interaction between the user and the second screen (S1812) and displaying the result of the interaction (S1814). In a preferred embodiment, the result of the interaction is displayed on the third screen. In a further preferred embodiment, the third screen is arranged to form a depth with respect to the second screen.

In addition, the interaction method 1800 includes the steps of detecting selection of an additional object by the first user on the first screen (S1816) and displaying the selected object or information linked to the selected object on the fourth screen (S1818). Further comprising, the fourth screen is arranged to indicate depth with respect to the first screen. According to an embodiment of the present invention, the fourth screen is configured to display the selected object or information linked to the selected object in a 2D or 3D form.

In addition, the interaction method 1800 includes the steps of detecting object selection on the first screen by the second user ( S1820 ) and information linked to the object selected by the second user on the fifth screen or the object selected by the second user The method further includes displaying ( S1822 ), wherein the fifth screen is arranged to indicate the depth of the first screen.

Also, the interaction method 1800 further includes retrieving information linked to the object selected by the second user from the information source ( S1824 ).

According to an embodiment of the present invention, the fifth screen displays the object selected by the second user or information linked to the object selected by the second user in a 2D or 3D form. When the object selected by the second user or information linked to the object selected by the second user is in a 2D form, if the second user enters the object selected by the second user or the object selected by the second user on the fifth screen If it is desired to display the linked information in 3D form, the interaction method 1800 converts the information linked to the object selected by the second user or the object selected by the second user from the 2D form to the 3D form (S1826) include more

In addition, the interaction method 1800 further includes the step of detecting an additional interaction with the fifth screen by the second user (S1828) and the step of displaying the result of the interaction (S1830). In a preferred aspect, the result of the interaction is displayed on the sixth screen. In a more preferred aspect, the sixth screen is arranged to form a depth with respect to the fifth screen.

In addition, the interaction method 1800 further includes the step of allowing an interaction between the screen corresponding to the first user and the screen corresponding to the second user (S1832).

An aspect of the above-described interaction 1800 method is illustrated in FIG. 19 . As shown in FIG. 3 , a first screen 300 for displaying this mail is taken, and a second screen 1902 for displaying multimedia that is an attachment included in the first screen 300 is the first screen 300 . ) is displayed with a second screen 1902 positioned to form a depth for .

In this way, in FIG. 20 , as shown in FIG. 3 , a first screen 300 for displaying an e-mail is taken, and a second screen 2002 for displaying text as an attachment included in the first screen 300 . is displayed with a second screen 2002 arranged to form depth with respect to the first screen 300 .

When the above-described interaction method 1800 is implemented, a first screen for displaying the web content 300 including at least one object 302 and 304; and a second screen (1902,2002) for displaying the selected object or information linked to the selected object is obtained, and the second screen (1902,2002) has a depth for the first screen (300). placed to form.

In order to implement the above-described interaction method 1800, the present invention provides a user interface engine configured to display web content including at least one object on a first screen; a sensing module configured to detect selection of at least one object by the first user; and an interactive 3D module configured to display the selected object or information linked to the selected object on the second screen, wherein the second screen is disposed to indicate depth with respect to the first screen.

In a preferred aspect of one embodiment of the present invention, system 2100 is shown in FIG. 21 and includes a module 2102 for retrieving and storing web content. It is operatively connected to a web based web browser or UI module 2104 . A web based web browser or UI module 2104 is additionally connected to a web engine 2106 and a display buffer 2108 . In a preferred aspect of one embodiment of the present invention, the web engine 2106 may perform detection of selection by the user. In addition, the system 2100 further includes an interactive 3D window module 2110 for arranging the first screen and the second screen to form a depth.

The interactive 3D window module 2110 also places additional screens to form the desired depth for the desired window. For example, the interactive 3D window module 2110 arranges a third screen to form a sense of depth with respect to the second screen, arranges a fourth screen to form depth with respect to the fifth screen, and places a fourth screen on the first screen. The fifth screen is disposed to form a sense of depth with respect to the screen, and the sixth screen is disposed to form a sense of depth with respect to the fifth screen.

Method 2100 of the present invention may be advantageously performed on a consumer product available to multiple users, a consumer product capable of being controlled by multiple users in an independent manner and capable of displaying images in 3D form. Therefore, the present invention also includes the steps of displaying web content including at least one object on the first screen; detecting selection of at least one object by the first user; detecting selection of at least one object by a second user; displaying the object selected by the first user or information linked to the object selected by the first user in a 3D form on a second screen; Displaying the object selected by the second user or information linked to the object selected by the second user in a 3D form on a third screen, wherein the second screen and the third screen are at least partially is non-overlapping, and each of the second screen and the third screen is arranged to form a depth with respect to the first screen.

According to another embodiment of the present invention, each of the first screen and the third screen displays a selected object or information linked to the selected object in a 2D or 3D form.

In addition, the interaction method 2100 further includes converting the selected object or information linked to the selected object from a 2D form to a 3D form.

In addition, the interaction method 2100 includes the steps of detecting the interaction with the second screen by the first user and displaying the result of the interaction on the fourth screen; and/or detecting an interaction with the third screen by the second user and displaying a result of the interaction on the fifth screen.

According to an embodiment of the present invention, the interaction method 2100 further includes allowing an interaction between the screen corresponding to the first user and the screen corresponding to the second user. In a preferred example, the interaction method 2100 includes sharing a screen corresponding to the first user with a second user or allowing sharing of a screen corresponding to the second user with the first user.

In addition, once the above-described interaction method 2100 is executed, a graphical user interface is generated, and the graphical user interface includes: a first screen for displaying web content including at least one object; a second screen for displaying the object selected by the first user or information linked to the object selected by the first user; a third screen for displaying an object selected by a second user or information linked to an object selected by the second user, wherein the second screen and the third screen are at least partially non-overlapping, and the second screen and the third screen Each of the screens is arranged to form a depth with respect to the first screen.

According to an embodiment of the present invention, a system for performing the above-described interaction method 2100 includes a user interface engine configured to display web content including at least one object on a first screen; a sensing module configured to detect selection of the at least one object by the first user and detect selection of the at least one object selected by the second user; and displaying the object selected by the first user or information linked to the object selected by the first user on the second screen, and displaying the object selected by the second user or information linked to the object selected by the second user on a third screen an interactive 3D module configured to display on a screen, wherein the second screen and the third screen are at least partially non-overlapping and each of the second screen and the third viewing window are arranged to form a depth with respect to the first screen.

While FIGS. 11 and 21 depict block diagrams of the system, the hardware configuration of the system in the form of computer system 2200 is shown in FIG. 22 .

Computer system 2200 may include a set of instructions that may be executed to cause computer system 2200 to perform one or more of the methods of interaction 100 disclosed. The computer system 2200 may be connected to other computer systems or peripheral devices using, for example, a network, or may operate as a separate device.

In a network implementation, computer system 2200 operates in the capacity of a server or as a client user computer in a server-client network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. Computer system 2200 may also be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile device, palmtop, laptop computer, desktop computer, communication device, cordless phone , LAN phone, control system, camera, scanner, facsimile, printer, pager, personal trust device, web appliance, network router, switch or bridge, or any particular device taken by the machine. It may be executed by any other device capable of executing a specified set of instructions of an action. Moreover, although a single computer system 2200 is shown, the term “system” also refers to any of a system or subsystem that individually or jointly executes one or more sets of instructions to perform one or more computer functions. It can be taken including a vowel of

The computer system 2200 may include a processor 2202 (eg, a central processing unit (CPU), a graphics processing unit (GPU), or both). The processor 2202 may be a component of various systems. For example, the processor 2202 may be part of a standard personal computer or workstation. The processor 2202 is configured to analyze and process one or more general processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or data. It may be a device currently or later developed for The processor 2202 may implement manually generated code, a software program (ie, programming).

The computer system 2200 includes a memory 2204 that can communicate via a bus 2208 . Memory 2204 may be main memory, static memory, or dynamic memory. Memory 2204 includes, but is not limited to, various forms of computer-readable storage media such as non-volatile and volatile storage media, read-only memory, programmable read-only memory, electronically programmable read-only memory, including, but not limited to, electrically erasable read-only memory, flash memory, magnetic disks or tapes, optical disks, and the like. In one example, memory 2204 includes cache or access memory for processor 2202 . In an alternative embodiment, memory 2204 is separate from processor 2202 , such as cache memory, system memory, or other memory of processor 2202 . The memory 2204 may be an external storage device or a database for storing data. Examples are hard drives, compact discs (CDs), digital video discs (DVDs), memory cards, memory sticks, floppy discs, universal serial bus (USB), memory devices or data any other device operative to store

Memory 2204 is operable to store instructions executable by processor 2202 . The functions, acts, and tasks shown in the figure may be performed by the processor 2202 being programmed to execute instructions stored in the memory 2204 . The functions, actions, and tasks are independent of any particular form of instruction set, storage medium, processor, or processing strategy, and may be software, hardware, integrated circuits, firmware, micro-code, operating alone or in combination. and the like. As such, the processing strategy may include multiprocessing, multitasking, parallel processing, and the like.

As shown, computer system 2200 includes a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid state display, a cathode ray tube (CRT), a projector, a printer, Alternatively, the display unit 2208, such as a currently known or later developed display device for outputting other determined information, may or may not be further included.

The display unit 2208 may act as an interface so that a user can view the functions of the processor 2202 . Specifically, the display unit 2208 may act as an interface together with software stored in the memory 2204 or the driving unit 2212 .

The display unit 2208 should be able to display an image in 3D form. Additionally, the display unit 2208 may be viewed and controlled by a plurality of users. Examples of the display unit 2208 include http://www.theverge.com/2012/8/31/3282583/samsung-oled-tv-multi-view-hands-on-video and http://www.youtube Shown at .com/watch?v=88_LCrnLvVk .

Additionally, computer system 2200 may include an input device 2210 configured to allow a user to interact with any configuration of computer system 2200 . The input device 2210 may be a numeric keypad, keyboard, or cursor control device such as a mouse or joystick, a touch screen display, a remote control, an air gesture receiver, a voice command receiver, or any other device operative to interact with the computer system 2200 . can

Computer system 2200 may also include a disk or optical drive 2212 . The disk or optical drive 2212 may include a computer-readable medium (CMR) in which one or more instruction sets 2216 may be embedded (eg, software). Moreover, as described, instructions 2216 may implement one or more methods or logic. In a particular example, the instructions 2216 reside entirely or at least partially within the processor 2202 or memory 2204 during execution by the computer system 2200 . Memory 2204 and processor 2202 may also include computer-readable media, as described above.

The present invention provides that a device coupled to the network 2220 receives the command 2216 including or in response to a propagated signal for communicating with voice, video, audio, image, or any other data over the network, and Consider a computer readable medium 2214 executing. Moreover, the command 2216 may be sent or received via the network 2220 , via a communication port or interface 2218 , or via a bus bar 2206 .

The communication port or interface 2218 may be part of the processor 2202 or may be a separate component. Communication port 2218 may be created in software or may be a physical connection with hardware. Communication port 2218 may be configured to connect with network 2220 , external media, display 2208 , or any other configuration of system 2200 , or a combination thereof. The connection to the network 2220 may be established as a physical connection, such as a wired Ethernet connection, or wirelessly, as described below. As such, additional connections with other components of system 2200 may be physical connections or established wirelessly. Network 2220 may alternatively be coupled directly with bus 2206 .

Network 2220 may include a wired network, a wireless network, an Ethernet AVB network, or a combination thereof. The wireless network may be a cellular telephone network, 802.11, 802.16, 802.20, 802.1Q or WiMax network. Moreover, the network 2226 may be a public network such as the Internet, a private network such as an intranet, or a combination thereof, and may utilize a variety of networking protocols, including those available now or to be developed in the future, but may be TCP based networking protocols. /IP is not limited.

In alternative examples, dedicated hardware implementations such as application specific integrated circuits, programmable logic arrays, and other hardware devices may be configured to implement various portions of system 2200 .

Applications that may include systems may include a wide variety of electronic and computer systems. One or more depicted examples may implement functionality as part of an application-specific integrated circuit or employing two or more specific associated hardware modules or data signal related devices that may be communicated between modules and modules via control and modules. can Accordingly, the system of the present invention includes software, firmware, and hardware implementations.

23 is a flowchart of an interaction method according to an embodiment of the present invention. Specifically, the electronic device 1000 may display web content including at least one object on the first screen (S2310). In this case, the object may be one of an image, a video, text, a hyperlink, a reference, multimedia, and the like. When an object is selected by the user (S2320), the electronic device may display the selected object or information linked to the selected object in 3D form on a new screen (S2330). In this case, the new screen may be displayed to form a sense of depth with respect to the existing screen.

When the selected object or information linked to the selected object includes the object, the user may select an object displayed on a new screen. When a new object is selected, the above process may be repeated.

In addition, a plurality of users may simultaneously search for different content using one device. For example, it is assumed that there is a plurality of user mode, and the first user and the second user are watching a first screen displaying web content including at least one object. When the first user selects an object, the selected object or information linked to the selected object is displayed on the second screen, and the second screen is viewable only to the first user and does not prevent the second user from viewing the first screen does not

In addition, a plurality of users may share content with each other while simultaneously searching for different content using one device. For example, it is assumed that there is a plurality of user mode, and the first user and the second user are watching a first screen displaying web content including at least one object. When the first user selects an object, the object selected by the first user or information linked to the selected object is displayed on the second screen, the second screen is viewable only to the first user, and the second user views the first screen do not interfere with In this case, when the sharing mode is executed, not only the first user but also the second user may view the second screen together.

24 is a block diagram illustrating the configuration of the electronic device 1000 according to an embodiment of the present invention. 24 , the electronic device 1000 includes a sensing unit 2410 , a display unit 2420 , and a processor 2440 .

The electronic device 1000 may be a desktop PC, but this is only an exemplary embodiment and may be implemented as another display device capable of displaying web content, such as a tablet PC, a notebook PC, or a smart TV.

The sensing unit 2410 detects a user command for controlling the electronic device 1000 . In particular, when web content including at least one object is displayed, the sensing unit 2410 may detect the user's selection of the object. Also, the sensing unit 2410 detects a user command for setting various modes (a plurality of user modes and a sharing mode) of the electronic device. In this case, the sensing unit 2410 may be implemented as a touch sensing unit, but this is only an exemplary embodiment, and is implemented as various input devices (eg, remote control signal sensing unit, mouse, keyboard, etc.) for receiving user commands. can be

The display unit 2420 outputs image data under the control of the processor 2440 . In particular, the display unit 2420 may display the object selected by the user or information linked to the object selected by the user on one of the first to nth screens. In this case, the screen represents a window, a window window, and the like, and may refer to content being displayed in a box or frame. Also, the object can be one of an image, video, text, hyperlink, bibliography, multimedia, and the like.

The processor 2440 controls the overall operation of the electronic device 1000 according to a user command input through the sensing unit 2410 . In particular, when the user's object selection is sensed by the sensing unit 2410 , the processor 2440 may control the display 2420 to display the selected object or information linked to the selected object on a new screen in 3D form. . In addition, when the selected object or information linked to the selected object is in a 2D form, the processor 2440 controls the converter module to convert the selected object in the 2D form or information linked to the selected object to the 3D form, and the 3D object Alternatively, the display unit 2420 may be controlled to display information linked to the selected object in 3D form. Also, the processor 2440 may control the display unit 2420 to display a new screen to form a sense of depth with respect to the existing screen.

Also, the processor 2440 may retrieve information linked to the selected object from the information source. Specifically, the processor 2440 may search for information linked to a selected object from an information source (eg, the Internet, etc.) through various communication modules.

In an embodiment of the present invention, when web content including a first screen including at least one object is displayed and the sensing unit 2410 detects an object selection by the first user, the processor 2440 may The display unit 2420 may be controlled to display the object selected by the first user or information linked to the selected object on the second screen. In this case, the sensing unit 2410 may detect the first user's interaction with the second screen, and the processor 2440 may control the display 2420 to display the result of the interaction on the third screen. In this case, the third screen may be arranged to form a sense of depth with respect to the second screen.

Also, when the sensing unit 2410 detects that the first user selects an additional object on the first screen, the processor 2440 displays the additionally selected object by the first user or information linked to the additionally selected object on the fourth screen. The display unit 2420 may be controlled to display. The processor 2440 may control the display unit 2420 to display the fourth screen to form a sense of depth with respect to the first screen.

As another embodiment of the present invention, when the electronic device 1000 is in a plurality of user modes, there may be a first screen that displays web content including at least one object for the first user and the second user. When the sensing unit 2410 detects the selection of the object by the first user, the processor 2440 may control the display unit 2420 to display the object selected by the first user or information linked to the object on the second screen. there is. The processor 2440 controls the display unit 2420 to display the second screen only to the first user.

As another embodiment of the present invention, the electronic device 1000 may be in a plurality of user modes, and there may be a first screen that displays web content including at least one object for the first user and the second user. When the sensing unit 2410 detects the selection of the object by the first user, the processor 2440 may control the display unit 2420 to display the object selected by the first user or information linked to the object on the second screen. there is. When the sensing unit 2410 detects a share mode command by the first user or the second user, the processor 2440 controls the display unit 2420 to display the second screen to the first user and the second user.

In addition, the processor 2440 includes a converter module, an interactive 3D module, a web module, and an interactive 3D window module. The converter module displays web content in 3D form under the control of the processor 2440 . The converter module 2430 generates a double-sized drawing buffer (a frame buffer for the left eye and a frame buffer for the right eye) to support separate drawing for the left eye frame and the right eye frame. The converter module renders each copy of the object to be displayed in 3D form in half of the double-size buffer, and then combines the frame buffer for the left eye and the frame buffer for the right eye to create a 3D effect.

The specific implementations described in this embodiment are examples, and do not limit the technical scope in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of lines between the components shown in the drawings illustratively represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections.

In this specification (especially in the claims), the use of the term "above" and similar referential terms may correspond to both the singular and the plural. In addition, when a range is described, individual values within the range are included (unless there is a description to the contrary), and each individual value constituting the range is described in the detailed description. Finally, the steps constituting the method may be performed in an appropriate order unless the order is explicitly stated or there is no description to the contrary. It is not necessarily limited to the order of description of the above steps. The use of all examples or exemplary terms (eg, etc.) is merely for describing the technical idea in detail, and the scope is not limited by the examples or exemplary terms unless limited by the claims. In addition, those skilled in the art will recognize that various modifications, combinations, and changes can be made in accordance with design conditions and factors within the scope of the appended claims or their equivalents.

100: Interaction method
200: first screen
402: second screen
2410: detection unit
2420: display unit
2440: Processor

Claims (20)

In an electronic device interaction method,
Displaying web content including at least one object on a first screen, wherein the first screen has a first sense of depth and is a screen shown to a first user and a second user;
When the mode of the electronic device is a multi-user mode and the at least one object is selected by a first user, the selected object and information linked to the selected object are displayed on a second screen while the first screen is displayed. displaying in a 3D form, and the second screen is a screen that has a second sense of depth and is only visible to the first user; and
Based on a sharing mode command from the first user for sharing the selected object displayed on the second screen and information linked to the selected object, the mode of the electronic device is changed from the multi-user mode to the sharing mode When switched, while the first screen and the second screen are displayed, the selected object and information linked to the selected object are displayed on a third screen in 3D form, and the third screen forms a second sense of depth and a screen that is displayed only to a second user. According to claim 1,
The step of displaying in the 3D form,
An interaction method characterized in that the selected object or information linked to the selected object is displayed in a 2D form, and the selected object in the 2D form or information linked to the selected object in the 2D form is converted and displayed in a 3D form. delete According to claim 1,
and retrieving information linked to the selected object from an information source. According to claim 1,
detecting user interaction with the second screen; and
The interaction method further comprising; displaying the result of the interaction on a third screen. According to claim 1,
detecting selection of an additional object by the first user on the first screen; and
The method further comprising: displaying the additionally selected object or information linked to the additionally selected object in a 3D form on a fourth screen. According to claim 1,
When the electronic device is in a plurality of user modes, the screen by the first user is displayed only to the first user. According to claim 1,
detecting object selection by a second user on the first screen; and
The method further comprising: displaying the object selected by the second user or information linked to the object selected by the second user on a fifth screen in 3D form; 9. The method of claim 8,
When the mode of the electronic device is switched to the sharing mode, interaction between the screen corresponding to the first user and the screen corresponding to the second user is possible. The method of claim 1,
The object is an image, video, text (text), a hyperlink, an attachment, and an interaction method, characterized in that one of multimedia. In an electronic device,
display unit;
a sensing unit configured to sense selection of at least one object by the first user; and
including a processor;
the processor is
The display unit is controlled to display web content including at least one object on a first screen, and the first screen is a screen with a first sense of depth and shown to the first and second users,
When the mode of the electronic device is a multi-user mode and the at least one object is selected by the first user, the selected object and information linked to the selected object are displayed on a second screen while the first screen is displayed The display unit is controlled to display in 3D form, and the second screen is a screen that has a second sense of depth and is visible only to the first user,
Based on the sharing mode command from the first user for sharing the selected object on which the second screen is displayed and information linked to the selected object, the mode of the electronic device is changed from the multi-user mode to the sharing mode When switched, the display unit is controlled to display the selected object and information linked to the selected object in a 3D form on a third screen while the first screen and the second screen are displayed, and the third screen is 2 The electronic device is a screen that has a sense of depth and is visible only to the second user. 12. The method of claim 11,
The processor is
Displaying the selected object or information linked to the selected object in a 2D form, and controlling the display unit to display the selected object in the 2D form or information linked to the selected object in the 2D form to be converted into a 3D form electronic device. delete 12. The method of claim 11,
and retrieving information linked to the selected object from an information source. 12. The method of claim 11,
The processor is
Controls the sensing unit to detect user interaction with the second screen,
The electronic device of claim 1, wherein the display unit is controlled to display a result of the interaction on a third screen. 12. The method of claim 11,
The processor is
controlling the sensing unit to detect selection of an additional object by the first user on the first screen,
The electronic device of claim 1, wherein the display unit is controlled to display the additionally selected object or information linked to the additionally selected object on a fourth screen in a 3D form. 12. The method of claim 11,
The processor is
When the electronic device is in a plurality of user modes, the electronic device controls the display unit to display the screen of the first user only to the first user. 12. The method of claim 11,
The processor is
controlling the sensing unit to detect object selection by a second user on the first screen,
The electronic device of claim 1, wherein the display unit is controlled to display the object selected by the second user or information linked to the object selected by the second user on a fifth screen in a 3D form. 19. The method of claim 18,
When the mode of the electronic device is switched to the sharing mode, an interaction between the screen corresponding to the first user and the screen corresponding to the second user is possible. 12. The method of claim 11,
The electronic device, characterized in that the object is one of an image, video, text, hyperlink, attachment, and multimedia.

Images