EP2894557B1 - Display apparatus - Google Patents

Display apparatus Download PDF

Info

Publication number
EP2894557B1
EP2894557B1 EP15150849.6A EP15150849A EP2894557B1 EP 2894557 B1 EP2894557 B1 EP 2894557B1 EP 15150849 A EP15150849 A EP 15150849A EP 2894557 B1 EP2894557 B1 EP 2894557B1
Authority
EP
European Patent Office
Prior art keywords
content
display
user
processor
mobile terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP15150849.6A
Other languages
German (de)
French (fr)
Other versions
EP2894557A3 (en
EP2894557A2 (en
Inventor
Hongjin Lee
Chungha Lee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020140004174A external-priority patent/KR102219797B1/en
Priority claimed from KR1020140004173A external-priority patent/KR102219798B1/en
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Publication of EP2894557A2 publication Critical patent/EP2894557A2/en
Publication of EP2894557A3 publication Critical patent/EP2894557A3/en
Application granted granted Critical
Publication of EP2894557B1 publication Critical patent/EP2894557B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482Interaction with lists of selectable items, e.g. menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14Display of multiple viewports
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04803Split screen, i.e. subdividing the display area or the window area into separate subareas

Definitions

  • the present invention relates to a display apparatus and a method for operating the same, and more particularly, to a display apparatus and a method for operating the same, which can increase user convenience.
  • the display apparatuses include a Liquid Crystal Display (LCD) panel, a plasma display panel, an Organic Light Emitting Diode (OLED) display panel, etc.
  • LCD Liquid Crystal Display
  • OLED Organic Light Emitting Diode
  • Each of US2012/135234 and US2013/268875 discloses a display configured to enlarge a list of icons and to execute playback of applications corresponding to the icons which have been enlarged over a certain size.
  • US2013/194238 teaches to functionally split a display in areas dedicated for displaying contents corresponding to different users, whose positions are automatically detected. It also discloses a shared area reserved for a common use by all the users of the display.
  • the present invention has been made in view of the above problems, and it is an object of the present invention to provide a display apparatus and a method for operating the same, which can increase user convenience.
  • module and “unit” used to signify components are used herein to help the understanding of the components and thus they should not be considered as having specific meanings or roles. Accordingly, the terms “module” and “unit” may be used interchangeably.
  • a display apparatus refers to an apparatus equipped with a display that can display an image. With the display apparatus horizontally leveled or vertically upright, the display apparatus may display an image.
  • the display apparatus need not be positioned either vertically or horizontally and that the embodiments described herein can be applied to a display apparatus having essentially any orientation.
  • the following description is given with the appreciation that the display apparatus is a large display apparatus that can be used by a plurality of users, while it is placed horizontally like a table top.
  • the display apparatus may enable a user's touch input, gesture input, audio input, etc.
  • FIG. 1 illustrates a display apparatus according to an embodiment of the present invention.
  • a display apparatus 100 is a device that displays an image.
  • the display apparatus 100 may include a display 180 for displaying an image and a camera 195 for capturing an image of a user.
  • the camera 195 is disposed as a separately configured unit above the display 180 placed on a horizontal plane, the camera 195 may be configured in various manners.
  • a plurality of cameras may be attached in the vicinity of at least one of the corners of the square display 180 or a plurality of standing cameras may stand in the vicinity of at least one of the corners of the square display 180.
  • the display apparatus 100 displays content in correspondence with the position of a user recognized through the camera 195.
  • content corresponding to the users may be displayed in separate areas.
  • Various examples of displaying content in separate areas will be described later with reference to FIGS. 3 to 30B .
  • FIG. 2 is a block diagram of the display apparatus illustrated in FIG. 1 .
  • the display apparatus 100 may include an input unit 110, a communication unit 120, an interface unit 130, a memory 140, a sensor unit 150, a processor 170, a display 180, an audio output unit 185, a power supply 190, and the camera 195.
  • the display apparatus 100 may further include an audio input unit (not shown).
  • the input unit 110 may include a plurality of buttons or a touch screen installed to the display apparatus 100.
  • the display apparatus 100 may be powered on and operated through the plurality of buttons or the touch screen. Many other input operations may be performed using the plurality of buttons or the touch screen.
  • the communication unit 120 may exchange data with a mobile terminal 600 or a server 500 wirelessly. Particularly, the communication unit 120 may exchange data wirelessly with a mobile terminal of a user of the display apparatus 100.
  • Various data communication schemes such as Bluetooth, Wireless Fidelity (WiFi), WiFi Direct, Digital Living Network Alliance (DLNA), APiX, etc. are available for wireless data communication.
  • the communication unit 120 may receive preferred content for each user from the mobile terminal 600 or the server 500 and transmit the received preferred content to the processor 170 or the memory 140.
  • the communication unit 120 may receive edited content from the processor 170 and transmit the edited content to the mobile terminal 600 or the server 500.
  • the interface unit 130 may receive data from the outside in a communication scheme other than used in the communication unit 120 or may transmit a signal processed or generated in the processor 170 to the outside.
  • the interface unit 130 may exchange data with the outside by cable.
  • the interface unit 130 may be provided with a High Definition Multimedia Interface (HDMI) port, a Red, Green, Blue (RGB) port, a Universal Serial Bus (USB) port, etc.
  • HDMI High Definition Multimedia Interface
  • RGB Red, Green, Blue
  • USB Universal Serial Bus
  • the memory 140 may store various data required for overall operations of the display apparatus 100, such as programs for processing or controlling in the processor 170. Further, the memory 140 may temporarily store externally received content.
  • the sensor unit 150 may sense a user's touch input or a touch between the mobile terminal 600 and the display apparatus 100.
  • the sensor unit 150 may include a touch sensor, a position sensor, a motion sensor, a gyro sensor, etc.
  • the sensed user's touch input or the mobile terminal's touch input may be provided to the processor 170.
  • the audio output unit 185 converts an electrical signal received from the processor 170 to an audio signal and outputs the audio signal.
  • the audio output unit 185 may include a speaker, etc.
  • the audio output unit 185 may output a sound corresponding to a manipulation of the input unit 110, that is, a manipulation of a button.
  • the audio input unit may receive a user's voice.
  • the audio input unit may include a microphone.
  • the audio input unit may convert the received voice to an electrical signal and transmit the electrical signal to the processor 170.
  • the processor 170 provides overall control to each unit of the display apparatus 100.
  • the processor 170 may play back content for each user, stored in the memory 140 or received through the communication unit 120 and may control display of the content on the display 180.
  • the processor 170 may calculate the positions of first and second users based on an image captured by the camera 195.
  • the processor 170 may change at least one of the size or position of a display area of first content so that a display area of second content may not be overlapped with the display area of the first content.
  • the processor 170 may change a display position of the second content in correspondence with the movement of the second user and may change at least one of the size or position of the display area of the first content so that the display area of the second content may not be overlapped with the display area of the first content.
  • the processor 170 may move the first content or a part of the first content in correspondence with the position of the second user and may control display of the moved first content or the moved part of the first content on the display 180.
  • the processor 170 may control display of the mini map on the display 180.
  • a mini map as used herein can take the form of a diagrammatic representation of the first content, such as a file directory structure.
  • the processor 170 may move a part of the first content corresponding to the part of the mini map and may control display of the moved part of the first content on the display 180.
  • the processor 170 may zoom in or zoom out the mini map and may control display of the zoomed-in or zoomed-out mini map on the display 180.
  • the processor 170 may combine the at least two first content items and control display of the combined content items on the display 180.
  • the processor 170 may control display of a copy of the first content in correspondence with the position of the second user and, upon receipt of an edit input from the second user, may edit the copy of the first content.
  • the processor 170 may control display of the edited copy of the first content or a copy of the edited copy of the first content in correspondence with the position of the first user. During editing, the copy of the first content or the copy of the copied first content is not editable to the first user.
  • the processor 170 may sense it through the sensor unit 150 and perform pairing between the display apparatus 100 and the mobile terminal 600 through the communication unit 120.
  • the processor 170 may identify the user by device information or user information received from the mobile terminal 600.
  • the processor 170 may define a user area corresponding to the position of the mobile terminal 600.
  • the processor 170 may control display of an object related to data sharing between the mobile terminal 600 and the display apparatus 100.
  • the processor 170 may control display of content related to the user in the user area.
  • the processor 170 may control display of a whole screen displayed on the mobile terminal 600 in the user area defined on the display 180.
  • the processor 170 may control display of a content item selected from a screen displayed on the mobile terminal 600 in the user area of the display 180.
  • the processor 170 may control changing of the size of the displayed user area according to a user input.
  • the processor 170 may define user areas for a plurality of users on the display 180.
  • the user areas may not be overlapped with one another.
  • the processor 170 may control sharing of content between users corresponding to the user areas. That is, the processor 170 may control transmission of the content to a mobile terminal of another user.
  • the processor 170 may calculate the positions of the first and second users based on an image captured by the camera 195 and may display content for each user in correspondence with the position of the user.
  • the processor 170 may control enlargement of the content list on the display 180.
  • the processor 170 may control display of at least a part of a plurality content items listed in the content list. If any of the displayed content items is of a reproducible size, the processor 170 may control playback and display of the content item.
  • the processor 170 may control display of a plurality of content items. Upon receipt of an input to combine the plurality of content items, the processor 170 may combine the plurality of content items and control display of a content list including the combined content items.
  • the processor 170 may play back content corresponding to the content item and control display of the playback content.
  • the processor 170 may increase a display size of content being played back to a largest of display sizes of displayed content items.
  • the processor 170 may control display of a content list enlarged in a first direction.
  • the processor 170 may control enlargement of the playback content and display of the enlarged content and may control volume-up of the playback content.
  • the processor 170 may control output and display of images related to the content in the first direction.
  • the processor 170 may insert the additional image between the content-related images.
  • the processor 170 may control display of content including the additional image.
  • the processor 170 may generate a content file including the content items corresponding to the part of the displayed content items.
  • the display 180 may display images related to operations of the display apparatus 100.
  • the display 180 may be configured with a Plasma Display Panel (PDP) display, an LCD, an OLED display, etc. Further, the display 180 may be a flexible display or a Three-Dimensional (3D) display.
  • PDP Plasma Display Panel
  • LCD LCD
  • OLED Organic LED
  • 3D Three-Dimensional
  • the power supply 190 may supply operation power to each component under the control of the processor 170. Particularly, the power supply 190 may supply power to the processor 170 that may be implemented on a System On Chip (SOC), the display 180 that displays an image, and the audio output unit 185 for audio output.
  • SOC System On Chip
  • the power supply 190 may be provided with a converter (not shown) for converting Alternating Current (AC) to Direct Current (DC).
  • a converter for converting Alternating Current (AC) to Direct Current (DC).
  • the power supply 190 may further include an inverter (not shown) capable of Pulse Wave Modulation (PWM) to change an illuminance or drive dimming.
  • PWM Pulse Wave Modulation
  • One or more cameras 195 may be provided. An image captured by a camera 195 may be transmitted to the processor 170. The processor 170 may identify each user, determine the position of the user, and recognize a gesture by processing the captured image.
  • the display apparatus 100 may be any of a TeleVision (TV), a monitor, a projector, etc.
  • TV TeleVision
  • monitor a monitor
  • projector a projector
  • the mobile terminal 600 may be any of a smart phone, a music player, a tablet, an e-book reader, a tab-book, etc.
  • memory 140 is shown in FIG. 2 as separately configured from the processor 170, the present invention is not limited thereto. Rather, the memory 140 may be incorporated into the processor 170.
  • the block diagram of the display apparatus 100 illustrated in FIG. 2 is purely exemplary. Depending upon the specifications of the display apparatus 100 in actual implementation, the components of the display apparatus 100 may be combined or omitted or new components may be added. That is, two or more components are incorporated into one component or one component may be configured as separate components, as needed.
  • FIG. 3 is a flowchart illustrating a method for operating the display apparatus according to an embodiment of the present invention
  • FIGS. 4A to 20C are views referred to for describing the method for operating the display apparatus illustrated in FIG. 3 .
  • the processor 170 of the display apparatus 100 calculates the position of a first user (S310).
  • the processor 170 may receive an image captured by the camera 195 disposed above the display 180 that sits horizontally like a table top and may calculate the position of a user based on the captured image.
  • the processor 170 may calculate the position of the first user 10 based on an image captured by the camera 195.
  • the processor 170 may recognize the first user 10. For example, the processor 170 may recognize the first user 10 by matching the first user 10 to user images stored in a database using a facial recognition algorithm, an iris recognition algorithm, etc.
  • the processor 170 displays first content on the display 180 in correspondence with the position of the first user (S315).
  • the processor 170 may control display of the first content in correspondence with the position of the first user 10.
  • the first content may be displayed automatically after the first user 10 is recognized and located, or may be displayed in response to a content display input.
  • FIG. 4A illustrates an example of displaying image-based first content 410 in correspondence with the position of the first user 10. Particularly, the imaged-based first content 410 is rotated by 180 degrees according to the position of the first user 10 so that the first user 10 may view the content 410 conveniently.
  • the first user 10 may perform various tasks such as movement, zoom-in/zoom-out, combining, etc. on the first content 410 by applying a touch input or the like to the display 180.
  • the processor 170 then calculates the position of a second user (S320).
  • the processor 170 may calculate the position of the second user 20 based on an image captured by the camera 915.
  • the processor 170 may recognize the second user 20 as well as locate the second user 20.
  • the second user 20 is on the opposite side (hereinafter, referred to as a lower side) of the horizontally placed display 180 to the first user 10.
  • the processor 170 displays second content on the display 180 in correspondence with the position of the second user (S325).
  • the processor 170 changes a display area of the first content, taking into account the position of the second user (S330).
  • the processor 170 may control display of the second content on the display 180 in correspondence with the position of the second user 20.
  • the second content may be displayed automatically after the second user 20 is recognized and located, or may be displayed in response to a content display input.
  • the processor 170 may change the display area of the first content according to the position of the second user. For example, the processor 170 may change at least one of the size or position of the display area of the first content so that the display area of the second content may not be overlapped with the display area of the first content.
  • image-based second content 418 is additionally displayed in correspondence with the position of the second user 20.
  • the image-based second content 418 is displayed scaled-down in correspondence with the position of the second user 20 located across the first user 10. Therefore, the second user 20 may view the second content 418 conveniently.
  • the first content 410 illustrated in FIG. 4A is preferably scaled down.
  • the first content 410 is vertically scaled down to first content 415 and the vertically scaled-down first content 415 is displayed in correspondence with the position of the first user 10. Accordingly, the first user 10 may view the first content 415 without overlap with the second content 418.
  • FIG. 5A illustrates the first user 10 on a left side of the horizontally placed display 180. Therefore, the image-based first content 410 may be displayed rotated counterclockwise by 90 degrees.
  • the second user 20 is additionally located on a right side of the horizontally placed display 180.
  • the image-based second content is horizontally scaled down and rotated clockwise by 90 degrees.
  • the resulting second content 428 is displayed in front of the second user 20.
  • the first content 410 illustrated in FIG. 5A may be horizontally scaled down to first content 425 and the first content 425 may be displayed in front of the first user 10.
  • the processor 170 may change the display position of the second content along with the movement of the second user. Further, the processor 170 may change at least one of the size or position of the display area of the first content so that the changed display area of the second content may not be overlapped with the display area of the first content.
  • the processor 170 may trace the movement of the second user 20 based on an image captured by the camera 195 and may change the position of the display area of the second content in correspondence with the position of the second user.
  • the second content 428 displayed in a right part of the display 180 in FIG. 5B is rotated counterclockwise by 90 degrees and displayed in a lower right part of the display 180. Therefore, the second user 20 may readily view second content 515 in spite of the second user's movement.
  • First content 510 is displayed in correspondence with the position of the first user 10 on the left side of the horizontally placed display 180 in FIG. 5C .
  • the processor 170 may control display of second content 518 moved further to the left.
  • the first content 510 and the second content 518 may overlap with each other over an overlap area 519.
  • the processor 170 may change the display size or position of at least one of the first content 510 or the second content 518.
  • the processor 170 may assign a weight to the moved second content 518 to thereby change the display size or position of the first content 510.
  • the processor 170 may assign a weight to the displayed first content 510 to thereby change the display size or position of the second content 518.
  • the size of the first content 510 is changed.
  • the processor 170 may change the display size of the first content 510 in correspondence with a movement direction of the second user 20. Therefore, the resulting first content 511 does not overlap with the second content 518.
  • the first user 10 and the second user 20 are located on the upper and lower sides of the display 180, respectively and thus the first content 415 and the second content 418 are displayed separately in upper and lower parts of the display 180.
  • first user 10 and the second user 20 are located on the upper side of the display 180, the second user 20 to the left of the first user 10 and a third user 30 is located on the lower side of the display 180.
  • first content 611 and second content 613 are displayed separately in upper left and top right parts of the display 180, whereas third content 617 is displayed in the lower part of the display 180.
  • the first user 10 and the second user 20 are located on the left and right sides of the display 180, respectively, and the first content 425 and the second content 428 are displayed separately in the left and right parts of the display 180.
  • the first user 10 is located on the left side of the display 180, whereas the second user 20 and the third user 30 are located respectively on a right upper side and a right lower side of the display 180. Accordingly, the first content 425 is displayed in the left part of the display 180, whereas second content 621 and third content 623 are displayed respectively in a right upper part and a right lower part of the display 180.
  • the first, second, and third users 10, 20, and 30 are located from left to right in this order on the upper side of the display 180.
  • the third user 30 moves toward the second user 20, that is, to the left as viewed from across the third user 30, an overlap area 718 is created between second content 713 and third content 717 among first content 711, the second content 713, and the third content 7171 that are displayed side by side on the display 180.
  • the processor 170 may change the display size or position of the second content 713 or the display size of the third content 717 in order to eliminate the overlap area 718.
  • the processor 170 may control movement of the display position of the second content 713 illustrated in FIG. 7C further to the left or reduction of the display size of the second content 713. In another example, the processor 170 may control reduction of the display size of the third content 717.
  • the processor 170 may control movement of the first content or a part of the first content in correspondence with the position of the second user and may control display of the moved first content or the moved part of the first content on the display 180.
  • first content 810 may be displayed in an upper right part of the display 180.
  • a "right move” input may be defined as a touch-based drag input to the right on the display 180, a touch-based flick input to the right on the display 180, or a gesture-based input.
  • the processor 170 may control display of the mini map on the display 180.
  • the processor 170 may control movement of a part of the first content corresponding to the part of the mini map and may control display of the moved part of the first content.
  • the processor 170 may control zoom-in or zoom-out of the mini map and may control display of the zoomed-in or zoomed-out mini map on the display 180.
  • first content and second content are displayed separately in correspondence with the positions of the first and second users 10 and 20.
  • the processor 170 may control display of a mini map 912 for the first content on the display 180.
  • the mini map 912 may be displayed over the first content. Or the mini map 912 may be displayed apart from the first content.
  • the mini map 912 is smaller than the first content in display size. To facilitate a user's manipulation, the mini map 912 may be displayed in correspondence with the position of the user's hand.
  • an enlarged mini map 914 is displayed in response to an input to zoom in a mini map.
  • the enlarged mini map 914 includes a text area and an image area, by way of example.
  • the specific area is moved.
  • the processor 170 may control movement of the image area 931 and thus display of a moved image area 933 in correspondence with the position of the second user 20. Therefore, an image area, that is, a content part that the first user wants to share may be transferred to the second user, thereby increasing user convenience.
  • the processor 170 may set the image area 931 to move faster in a first area 911 for the first content corresponding to the position of the first user 10 than in a second area 913 for the second content corresponding to the position of the second user 20. According to this setting, if the second user 20 makes a gesture of grabbing the moving content, a user-friendly user interface may be created.
  • the processor 170 may control combining of the first content items and display of the combined first content items.
  • the first user 10 makes an input to combine a plurality of content items 1011 and 1012 displayed in correspondence with the position of the first user 10.
  • the processor 170 may combine the plurality of content items 1011 and 1012. That is, the content items 1011 and 1012 displayed apart from each other may be combined into one content item.
  • combined content items 1013 are displayed overlapped. A predetermined time later, the overlapped content items are combined into one file.
  • content 1014 combined into one file or one page is displayed.
  • a user may manage content items efficiently by combining the content items.
  • the content items may be combined into one file or one page as illustrated in FIGS. 10A , 10B , and 10C .
  • the content items may not be combined.
  • the first user 10 makes an input to combine the plurality of content items 1011 and 1012 displayed in correspondence with the position of the first user 10, as illustrated in FIG. 10A .
  • the processor 170 may combine the plurality of content items 1011 and 1012. That is, the plurality of content items 1011 and 1012 displayed apart from each other may be combined into one content item.
  • one content item 1011 remains, while the other content item 1012 disappears, rather than the content items 1011 and 1012 are combined into one file or one page.
  • the processor 170 may control display of a copy of the first content in correspondence with the position of the second user and, upon receipt of an edit input from the second user, may edit the copy of the first content.
  • the processor 170 may control display of an edited copy of the first content or a copy of the edited copy of the first content in correspondence with the position of the first user. During editing, the copy of the first content or the copy of the copy of the first content may not be editable to the first user.
  • first content 1211 and a copy 1213 of the first content are displayed respectively in correspondence with the positions of the first user 10 and the second user 20 respectively on the left and right sides of the display 180.
  • the copy 1213 of the first content displayed in front of the second user 20 may be moved and displayed, in response to a content movement input in a similar manner to in FIGS. 9A , 9B , and 9C , or in response to a content edit input from the second user 20.
  • the second user 20 may zoom in the copy of the first content.
  • the copy 1213 of the first content is displayed larger than the first content 1211.
  • Additional content may be added to the copy 1213 of the first content by an edit input of the second user 20. While text 1219 is added to the copy 1213 of the first content in FIG. 12A , various types of data including an image, an audio, a Web address, and the like may be added.
  • the processor 170 may control display of the copy of the first content or a copy of the copy of the first content, which is being edited or has been completely edited, in correspondence with the position of the first user, so that the first user may view the content edited by the second user.
  • a copy 1217 of the copy of the first content under editing and the first content 1211 are displayed side by side.
  • the copy 1217 of the copy of the first content may not be editable to the first user 10 and thus a lock object indicating uneditable may be displayed. After the second user 20 finishes editing the content, the copy 1217 of the copy of the first content may be editable to the first user 10.
  • the first user may readily view content which is being edited or has been edited completely.
  • the first content 1211 and the copy 1217 of the copy of the first content may be combined. That is, as illustrated in FIGS. 10A , 10B , and 10C , upon receipt of a combine input, the copy 1217 of the copy of the first content that contains a changed part may remain, while the first content 1211 may be removed. Accordingly, the updated first content may be stored or displayed.
  • the positions of the first and second users may be determined through mobile terminals of the first and second users, instead of the camera 195.
  • the processor 170 may sense a pressure applied by the mobile terminal and thus may locate a user of the mobile terminal.
  • the processor 170 may locate the user by sensing a variation in capacitance caused by the mobile terminal.
  • the processor 170 may perform pairing between the display apparatus 100 and a mobile terminal of the user, receive device information or user information about the mobile terminal through the communication unit 120, and recognize the user based on the received information.
  • the processor 170 may receive device information, user information, etc. from the mobile terminal through the communication unit 120 and may recognize and locate the user of the mobile terminal. When the mobile terminal is within the predetermined distance, the processor 170 may control data exchange with the mobile terminal by Near Field Communication (NFC).
  • NFC Near Field Communication
  • FIGS. 13A to 18B illustrate an exemplary operation for locating and recognizing a user through the sensor unit 150 and the communication unit 120.
  • the first user 10 places a first mobile terminal 600a on the upper part of the horizontally placed display 180.
  • the processor 170 may calculate the position of the first mobile terminal 600a through the sensor unit 150 and estimate (or calculate) the position of the first user 10 based on the position of the first mobile terminal 600a.
  • the processor 170 may perform pairing between the display apparatus 100 and the first mobile terminal 600a through the communication unit 120 and may identify the first mobile terminal 600a as used by the first user 10 based on device information or user information received from the first mobile terminal 600a.
  • the processor 170 may control display of the image-based first content 410 in correspondence with the position of the first user 10.
  • the first content 410 may be content stored in the first mobile terminal 600a.
  • the first content 410 may be latest reproduced content, favorite content, or user-set content.
  • a second mobile terminal 600b of the second user 20 is placed on the lower part of the display 180.
  • the processor 170 may calculate the position of the second mobile terminal 600b through the sensor unit 150 and estimate (or calculate) the position of the second user 20 based on the position of the second mobile terminal 600b.
  • the processor 170 may perform pairing between the display apparatus 100 and the second mobile terminal 600b through the communication unit 120 and may identify the second mobile terminal 600b as used by the second user 20 based on device information or user information received from the second mobile terminal 600b.
  • the processor 170 may control display of the image-based second content 418 in correspondence with the position of the second user 20.
  • the second content 418 may be content stored in the second mobile terminal 600b.
  • the second content 418 may be latest reproduced content, favorite content, or user-set content.
  • the first content 410 displayed in FIG. 13A is preferably scaled down.
  • the vertically scaled-down first content 415 is displayed in correspondence with the position of the first user 10.
  • the first user 10 may view the first content 415 without overlap with the second content 418.
  • FIGS. 14A to 14E are identical to the operations illustrated in FIGS. 5A to 5E except that a user is recognized and located. Thus, a description of the operations illustrated in FIGS. 14A to 14E will not be provided herein.
  • FIGS. 15A and 15B correspond to FIGS. 6A and 6B
  • FIGS. 16A , 16B and 16C correspond to FIGS. 7A, 7B , and 7C
  • FIGS. 17A and 17B correspond to FIGS. 8A and 8B
  • FIGS. 18A , 18B and 18C correspond to FIGS. 12A , 12B , and 12C .
  • FIGS. 9A to 11C While not shown, the operations illustrated in FIGS. 9A to 11C may be performed.
  • the processor 170 may change a display area of the content in proportion to the size of the mobile terminal.
  • the processor 170 may change the display area of the content in proportion to the amount of content data stored in the mobile terminal or the resolution of content data to be displayed. Therefore, the amount of the content data stored in the mobile terminal may be readily recognized.
  • the sensor unit 150 of the display apparatus 100 may sense a first surface and a second surface of a mobile terminal, separately.
  • the processor 170 may distinguish a touch made on the display apparatus 100 by a front surface of the mobile terminal having a touch screen formed thereon from a touch made on the display apparatus 100 by a rear surface of the mobile terminal based different capacitance variations.
  • the processor 170 may control display of different content corresponding to the front and rear surfaces of the mobile terminal.
  • a user area is defined for each user.
  • mobile terminals 600a, 600b, 600c, 600d, and 600e of users 10, 20, 30, 40, and 50 are placed on the display 180 and user areas Ar1, Ar2, Ar3, Ar4, and Ar5 are defined for the respective users 10, 20, 30, 40, and 50.
  • the processor 170 may differentiate at least one of the sizes, shapes, colors, etc. of the user areas Ar1, Ar2, Ar3, Ar4, and Ar5 depending on where the users 10, 20, 30, 40, and 50 put the mobile terminals 600a, 600b, 600c, 600d, and 600e as viewed from the users 10, 20, 30,40, and 50.
  • the first user area Ar1 is extended to the right of the mobile terminal 600a, including the mobile terminal 600a.
  • the second user 20 puts the mobile terminal 600b on the display 180 far from the second user 20
  • the second user area Ar2 is extended to the second user 20, including the mobile terminal 600b.
  • the third user 30 puts the mobile terminal 600c on the display 180, right in front of the third user 30, the third user area Ar3 is extended backward from the mobile terminal 600c, including the mobile terminal 600c.
  • the fourth user 40 puts the mobile terminal 600d on the display 180 at the fourth user's right side
  • the fourth user area Ar4 is extended to the left of the mobile terminal 600d, including the mobile terminal 600d.
  • the fifth user 50 puts the mobile terminal 600e on the display 180 at a specific position with respect to the fifth user 50
  • the fifth user area Ar5 is created around the mobile terminal 600e, including the mobile terminal 600e.
  • user areas are defined for a plurality of users depending on where the users put their mobile terminals on the display 180 in such a manner that each user area is apart from where a mobile terminal is put on the display 180, as illustrated in FIGS. 18A and 18B .
  • the processor 170 may arrange content items in a user-friendly manner in a user area for each user.
  • the processor 170 may sense the existence of the mobile terminals 600a, 600b, 600c, 600d, and 600e on the display 180 through the sensor unit 150 and may perform pairing between the display apparatus 100 and the mobile terminals 600a, 600b, 600c, 600d, and 600e through the communication unit 120. Further, the processor 170 may recognize the users 10, 20, 30, 40, and 50 based on device information or user information received from the mobile terminals 600a, 600b, 600c, 600d, and 600e.
  • the processor 170 may define the user areas Ar1, Ar2, Ar3, Ar4, and Ar5 in correspondence with the positions of the mobile terminals 600a, 600b, 600c, 600d, and 600e.
  • the processor 170 may control display of an object related to data sharing between the mobile terminal and the display apparatus 100.
  • This object may be displayed in a user area defined for the user. If the user selects the object, the processor 170 may control data sharing with the mobile terminal. For example, the processor 170 may control reception of data from the mobile terminal and display of the data in the user area.
  • the processor 170 may control setting of the size of a user area to a default value.
  • the processor 170 may control changing of the size of a user area according to a user input.
  • the size of a user area may be adjusted by an On Screen Display (OSD) menu or by a touch and drag of a borderline of the user area. If the borderline of the user area is dragged in or out diagonally, the whole user area may be contracted or enlarged. If the borderline of the user area is dragged in or out horizontally, the user area may be contracted or enlarged horizontally. If the borderline of the user area is dragged in or out in one direction, the layout of content items in the user area may be changed, as illustrated in FIGS. 25A and 25B .
  • OSD On Screen Display
  • the size of the first user area Ar1 is changed in response to an input to change the first user area Ar1 of the first user 10.
  • the first user area Ar1 may be enlarged or contracted.
  • the processor 170 may control display of user-related content in a user area defined for a user of the mobile terminal.
  • the processor 170 may control display of a whole screen displayed on the mobile terminal in the user area of the display 180.
  • the processor 170 may control display of a content item selected from a screen displayed on the mobile terminal 600 in the user area of the display 180.
  • content displayed on a screen of the mobile terminal 600a is displayed as it is in the first user area Ar1 of the first user 10.
  • the display apparatus 100 may receive data of a whole screen 1905 displayed on the mobile terminal 600a from the mobile terminal 600a and the processor 170 may control display of the received data in the first user area Ar1 of the first user 10.
  • the size of each of content items and the size of a screen 1910 including the content items are larger than the size of each of content items and the size of the screen 1905 including the content items in the mobile terminal 600a.
  • the first user 10 may readily view and share content items of the mobile terminal 600a.
  • a content item selected from a screen displayed on the mobile terminal 600a is displayed in the first user area Ar1 of the first user 10.
  • the display apparatus 100 may receive data related to the selected content item 1908 from the mobile terminal 600a and the processor 170 may control display of the received data in the first user area Ar1 of the first user 10.
  • the size of a content item 1918 displayed on the display 180 is larger than the content item 1908 displayed on the mobile terminal 600a.
  • the first user 10 may readily view and share content items of the mobile terminal 600a.
  • the content item 1908 may be highlighted or the color of its outline may be changed.
  • a touch input or a flicking input to the mobile terminal 600a may be set to be interpreted as a control input for the display apparatus 100, not a control input for the mobile terminal 600a. Therefore, a controller of the mobile terminal 600a may transmit the touch input or the flicking input to the display apparatus 100, ignoring the touch input or the flicking input in the data sharing mode.
  • the display apparatus 100 may share the whole folder.
  • the processor 170 may control display of an object indicating data sharing-on in a user area. Therefore, a user corresponding to the user area may readily recognize activation of data sharing.
  • the processor 170 may define user areas for a plurality of users and that in such a manner that the user areas are not overlapped.
  • the processor 170 may control sharing of the content between users corresponding to the user areas. That is, the processor 170 may control transmission of the content to a mobile terminal of another user.
  • first and second user areas Ara and Arb are defined on the display 180 respectively for the first and second users 10 and 20. Particularly, the first and second user areas Ara and Arb are partially overlapped with each other. The overlap area may be highlighted.
  • the first and second user areas Ara and Arb may be overlapped based on movement of a user, a change in the size of a user area, or the type or size of displayed content data.
  • first, second, and third content lists 2010a, 2010b, and 2010c for the first, second, and third users 10, 20, and 30 are displayed respectively in user areas Ar1, Arb, and Arc.
  • the second user 20 disposes a specific content item 2018 of the second content list 2010b in a common area Arz overlapped between the second user area Arb and the third user area Arc.
  • the processor 170 may control sharing of the content item 2018 between the third user 30 and the second user 20. That is, the content item 2018 may be transmitted to the second user area Arb or the second mobile terminal 600b of the second user 20.
  • first and second user areas Ara and Arm are defined on the display 180 respectively for the first and second users 10 and 20. Particularly, data of the first user area Ara is transmitted to the second user area Arm by data sharing.
  • the second user 20 may use and edit the shared content and share the edited content with the first user 10. Thus the content may be reconstructed.
  • FIG. 21 is a flowchart illustrating a method for operating the display apparatus according to another embodiment of the present invention
  • FIGS. 22 to 30B are views referred to for describing the method for operating the display apparatus illustrated in FIG. 2I .
  • the processor 170 of the display apparatus 100 controls display of a content list including a plurality of content items on the display (S2310).
  • a content list 2415 including a plurality of content items is displayed on the horizontally placed display 180.
  • the content list 2415 may be automatically displayed after user recognition or may be displayed in response to an additional input.
  • step S2310 a step of displaying a plurality of content items on the display 180 and a step of combining a plurality of content items, upon receipt of a content combine input, and displaying a content list including the combined content items may be performed.
  • the content list 2415 including a plurality of content items 2410, CT1 to CT24 may be displayed on the horizontally placed display 180.
  • the plurality of content items 2410, CT1 to CT24 are displayed in full screen on the display 180 in FIG. 22
  • the plurality of content items 2410, CT1 to CT24 may be displayed in a partial area of the display 180 in correspondence with the position of a user.
  • the processor 170 may calculate the position of a user based on an image captured by the camera 195.
  • the user makes a touch input of gathering the displayed plurality of content items 2410, CT1 to CT24 toward the user with the user's hands.
  • the plurality of content items 2410, CT1 to CT24 are combined into one file or the single content list 2415.
  • the processor 170 may generate the content list 2415 including the combined content items 2410 in response to an input to combine the plurality of content items 2410.
  • the processor 170 controls enlargement of the content list and display of at least a part of the plurality of content items included in the content list in response to an input to enlarge a content list (S2320).
  • the processor 170 sequentially controls enlargement and display of the content list in response to the input to enlarge a content list.
  • the processor 170 controls display of at least a part of the plurality of content items included in the content list.
  • the user may readily view content items included in the content list.
  • the user makes a touch input to enlarge the displayed content list 2415 by spreading the user's hands sideways.
  • the content list 2415 is enlarged.
  • FIG. 23(b) illustrates a content list 2416 enlarged after a first time
  • FIG. 23(c) illustrates a content list 2417 enlarged after a second time.
  • the content items of the content list 2416 are shown in FIG. 23(b) as content items of a first size, CTa, ..., CTk and are shown in FIG. 23(c) as content items of a second size larger than the first size, CT1, ... CT11.
  • FIG. 23(b) The content items of the content list 2416 are shown in FIG. 23(b) as content items of a first size, CTa, ..., CTk and are shown in FIG. 23(c) as content items of a second size larger than the first size, CT1, ... CT11.
  • the content items illustrated in FIGS. 23(b) and 23(c) may be displayed as thumbnail images, not file names.
  • the size or resolution of the thumbnail images may be increased.
  • FIG. 23(c) because the content items are larger in FIG. 23(c) than in FIG. 23(b) , fewer content items may be displayed in FIG. 23(c) .
  • the processor 170 controls playback and display of the content items (S2330).
  • the processor 170 Upon receipt of an enlarge input, the processor 170 continues to enlarge the content list. If any of the displayed content items is of a reproducible size, for example of a size that permits the content to be reproduced at a full resolution on the display, then the processor 170 may control playback and display of the content item.
  • the processor 170 may control playback and display of the content item 2610 as illustrated in FIG. 24 . Particularly, the processor 170 may set the size of the playback content item 2610 to be larger than the size of each of the other content items 2607, 2608, 2609, 2611, 2612, and 2613. Thus, the user may readily view the playback content.
  • the first user 10 makes a vertical enlarge touch input.
  • the processor 170 may control display of the content list 2715 enlarged vertically, as illustrated in FIG. 25A .
  • the content list 2715 is fully enlarged vertically on the display 180 in FIG. 25A .
  • the first user 10 makes a horizontal enlarge touch input.
  • the processor 170 may control display of the content list 2715 enlarged horizontally, as illustrated in FIG. 25B .
  • the content list 2715 is fully enlarged horizontally on the display 180 in FIG. 25B .
  • the processor 170 controls display of the playback content enlarged and controls volume-up of the content.
  • content 2810 of a first size is being played back in an area of the display 180 in response to a user's enlarge input.
  • Sound 2820a and 2820b is output at a first volume level through the audio output unit 185, in proportion to the display size of the content 2810.
  • content 2812 of a second size larger than the first size of the content 2810 illustrated in FIG. 26(a) is being played back in an area of the display 180 in response to a user's enlarge input.
  • Sound 2822a and 2822b is output at a second volume level higher than the first volume level of the content 2810 illustrated in FIG. 26(a) through the audio output unit 185, in proportion to the display size of the content 2812.
  • content 2816 of a third size larger than the second size of the content 2812 illustrated in FIG. 26(b) is being played back in an area of the display 180 in response to a user's enlarge input.
  • Sound 2826a and 2826b is output at a third volume level higher than the second volume level of the content 2812 illustrated in FIG. 26(b) through the audio output unit 185, in proportion to the display size of the content 2816.
  • the user may readily recognize the display size of the content.
  • the processor 170 may control display of images related to the content in a first direction.
  • the processor 170 may insert the additional image between the content-related images.
  • the processor 170 may control display of the content including the additional image.
  • FIGS. 27A to 27E illustrate an operation for generating new content by adding an additional image to content being reproduced.
  • the processor 170 of the display apparatus 100 may control display of a plurality of images included in the content 2910 unfolded to the right, as illustrated in FIG. 27B .
  • the content 2910 may be paused during playback.
  • the content 2910 includes thumbnail images 2912 to 2918, TH1 to TH7.
  • the user makes a touch input of spreading the fourth and fifth images TH4 and TH5 from each other and inserts a new image 2920, THa between the fourth image TH4 and the fifth image TH5. Then, the new image 2920 is displayed between the fourth image TH4 and the fifth image TH5.
  • the processor 170 may display of new content 2930 including the new inserted image, while removing the plurality of images, as illustrated in FIG. 27E . Accordingly, content including an intended image may be generated simply.
  • the new content 2930 may be played back automatically after it is created.
  • the processor 170 may generate a content file including the corresponding content items.
  • FIGS. 28A to 30B illustrate an operation for, upon receipt of an input to combine a part of a plurality of displayed content items, generating a content file including the corresponding content items.
  • the user 10 makes a touch input to combine a first photo item 3011, PHOTO1 with a first music item 3012, MUSIC1.
  • content items e.g., PHOTO and MUSIC.
  • first photo item 3011, PHOTO1 and the first music item 3012, MUSIC1 may be moved, and a moved first photo item 3011a and a moved first music item 3012a may be overlapped and displayed as content 3015.
  • the first photo item 3011a and the first music item 3012a may be displayed combined into one file 3055.
  • the user 10 makes a touch input to combine a second photo item 3022, PHOTO3 with a fourth photo item 3024, PHOTO4.
  • content items e.g., PHOTO and MUSIC
  • the second photo item 3022, PHOTO2 and the fourth photo item 3024, PHOTO4 may be moved, and a moved second photo item 3022a and a moved fourth photo item 3024a may be overlapped and displayed as content 3025.
  • the second photo item 3022a and the fourth photo item 3024a may be displayed combined into one file 3155.
  • a user-desired content file may be generated.
  • the user 10 makes a touch input to combine a third music item 3032, MUSIC3 with a second music item 3034, MUSIC2.
  • content items e.g., PHOTO and MUSIC
  • the third music item 3032, MUSIC3 and the second music item 3034, MUSIC2 may be moved, and a moved third music item 3032a and a moved second music item 3034a may be overlapped and displayed as content 3035.
  • the third music item 3032a and the second music item 3034a may be displayed combined into one file 3255.
  • a user-desired content file may be generated.
  • the method for operating a display apparatus may be implemented as code that can be written on a computer-readable recording medium and thus read by a processor.
  • the computer-readable recording medium may be any type of recording device in which data is stored in a computer-readable manner. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disc, an optical data storage, and a carrier wave (e.g., data transmission over the Internet).
  • the computer-readable recording medium can be distributed over a plurality of computer systems connected to a network so that computer-readable code is written thereto and executed therefrom in a decentralized manner. Programs, code, and code segments to realize the embodiments herein can be construed by one of ordinary skill in the art.

Description

    CROSS-REFERENCE TO RELATED APPLICATION
  • This application claims the benefit of Korean Patent Application Nos. 10-2014-0004173 and 10-2014-0004174, filed on January 13, 2014 in the Korean Intellectual Property Office.
  • BACKGROUND OF THE INVENTION 1. Field of the invention
  • The present invention relates to a display apparatus and a method for operating the same, and more particularly, to a display apparatus and a method for operating the same, which can increase user convenience.
  • 2. Description of the Related Art
  • Various types of display apparatuses are used to display an image. For example, the display apparatuses include a Liquid Crystal Display (LCD) panel, a plasma display panel, an Organic Light Emitting Diode (OLED) display panel, etc.
  • As a display apparatus finds uses in a variety of fields, various characteristics are required for the display apparatus in each use field. Particularly, research has been actively made on large display apparatuses. Each of US2012/135234 and US2013/268875 discloses a display configured to enlarge a list of icons and to execute playback of applications corresponding to the icons which have been enlarged over a certain size.
  • US2013/194238 teaches to functionally split a display in areas dedicated for displaying contents corresponding to different users, whose positions are automatically detected. It also discloses a shared area reserved for a common use by all the users of the display.
  • SUMMARY OF THE INVENTION
  • Therefore, the present invention has been made in view of the above problems, and it is an object of the present invention to provide a display apparatus and a method for operating the same, which can increase user convenience.
  • In accordance with an aspect of the present invention, the above and other objects can be accomplished by the provision of a display apparatus according to claims 1-12. Any embodiments that fall outside the scope of these claims are shown for illustrative purpose only and are thus not forming part of the invention.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The above and other objects, features and other advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
    • FIG. 1 illustrates a display apparatus according to an embodiment of the present invention;
    • FIG. 2 is a block diagram of the display apparatus illustrated in FIG. 1;
    • FIG. 3 is a flowchart illustrating a method for operating the display apparatus according to an embodiment of the present invention;
    • FIGS. 4A to 20C are views referred to for describing the method for operating the display apparatus illustrated in FIG. 3;
    • FIG. 21 is a flowchart illustrating a method for operating the display apparatus according to another embodiment of the present invention; and
    • FIGS. 22 to 30B are views referred to for describing the method for operating the display apparatus illustrated in FIG. 2I.
    DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • Embodiments of the present invention will be described below with reference to the attached drawings.
  • The terms "module" and "unit" used to signify components are used herein to help the understanding of the components and thus they should not be considered as having specific meanings or roles. Accordingly, the terms "module" and "unit" may be used interchangeably.
  • In the specification, a display apparatus refers to an apparatus equipped with a display that can display an image. With the display apparatus horizontally leveled or vertically upright, the display apparatus may display an image. The skilled person will recognize that the display apparatus need not be positioned either vertically or horizontally and that the embodiments described herein can be applied to a display apparatus having essentially any orientation. The following description is given with the appreciation that the display apparatus is a large display apparatus that can be used by a plurality of users, while it is placed horizontally like a table top.
  • Further, the display apparatus may enable a user's touch input, gesture input, audio input, etc.
  • FIG. 1 illustrates a display apparatus according to an embodiment of the present invention.
  • Referring to FIG. 1, a display apparatus 100 according to an embodiment of the present invention is a device that displays an image. The display apparatus 100 may include a display 180 for displaying an image and a camera 195 for capturing an image of a user.
  • While the camera 195 is disposed as a separately configured unit above the display 180 placed on a horizontal plane, the camera 195 may be configured in various manners.
  • For example, a plurality of cameras may be attached in the vicinity of at least one of the corners of the square display 180 or a plurality of standing cameras may stand in the vicinity of at least one of the corners of the square display 180.
  • In accordance with an embodiment of the present invention, the display apparatus 100 displays content in correspondence with the position of a user recognized through the camera 195.
  • Particularly in the presence of a plurality of users, content corresponding to the users may be displayed in separate areas. Various examples of displaying content in separate areas will be described later with reference to FIGS. 3 to 30B.
  • FIG. 2 is a block diagram of the display apparatus illustrated in FIG. 1.
  • Referring to FIG. 2, the display apparatus 100 may include an input unit 110, a communication unit 120, an interface unit 130, a memory 140, a sensor unit 150, a processor 170, a display 180, an audio output unit 185, a power supply 190, and the camera 195. The display apparatus 100 may further include an audio input unit (not shown).
  • The input unit 110 may include a plurality of buttons or a touch screen installed to the display apparatus 100. The display apparatus 100 may be powered on and operated through the plurality of buttons or the touch screen. Many other input operations may be performed using the plurality of buttons or the touch screen.
  • The communication unit 120 may exchange data with a mobile terminal 600 or a server 500 wirelessly. Particularly, the communication unit 120 may exchange data wirelessly with a mobile terminal of a user of the display apparatus 100. Various data communication schemes such as Bluetooth, Wireless Fidelity (WiFi), WiFi Direct, Digital Living Network Alliance (DLNA), APiX, etc. are available for wireless data communication.
  • The communication unit 120 may receive preferred content for each user from the mobile terminal 600 or the server 500 and transmit the received preferred content to the processor 170 or the memory 140.
  • Or the communication unit 120 may receive edited content from the processor 170 and transmit the edited content to the mobile terminal 600 or the server 500.
  • The interface unit 130 may receive data from the outside in a communication scheme other than used in the communication unit 120 or may transmit a signal processed or generated in the processor 170 to the outside. For example, the interface unit 130 may exchange data with the outside by cable. For this purpose, the interface unit 130 may be provided with a High Definition Multimedia Interface (HDMI) port, a Red, Green, Blue (RGB) port, a Universal Serial Bus (USB) port, etc.
  • The memory 140 may store various data required for overall operations of the display apparatus 100, such as programs for processing or controlling in the processor 170. Further, the memory 140 may temporarily store externally received content.
  • The sensor unit 150 may sense a user's touch input or a touch between the mobile terminal 600 and the display apparatus 100. For this purpose, the sensor unit 150 may include a touch sensor, a position sensor, a motion sensor, a gyro sensor, etc.
  • The sensed user's touch input or the mobile terminal's touch input may be provided to the processor 170.
  • The audio output unit 185 converts an electrical signal received from the processor 170 to an audio signal and outputs the audio signal. For this purpose, the audio output unit 185 may include a speaker, etc. The audio output unit 185 may output a sound corresponding to a manipulation of the input unit 110, that is, a manipulation of a button.
  • The audio input unit (not shown) may receive a user's voice. For this purpose, the audio input unit may include a microphone. The audio input unit may convert the received voice to an electrical signal and transmit the electrical signal to the processor 170.
  • The processor 170 provides overall control to each unit of the display apparatus 100.
  • Particularly, the processor 170 may play back content for each user, stored in the memory 140 or received through the communication unit 120 and may control display of the content on the display 180.
  • The processor 170 may calculate the positions of first and second users based on an image captured by the camera 195.
  • The processor 170 may change at least one of the size or position of a display area of first content so that a display area of second content may not be overlapped with the display area of the first content.
  • When the second user moves, the processor 170 may change a display position of the second content in correspondence with the movement of the second user and may change at least one of the size or position of the display area of the first content so that the display area of the second content may not be overlapped with the display area of the first content.
  • Upon receipt of an input to move the first content, the processor 170 may move the first content or a part of the first content in correspondence with the position of the second user and may control display of the moved first content or the moved part of the first content on the display 180.
  • Upon receipt of an input to display a mini map for the first content, the processor 170 may control display of the mini map on the display 180. A mini map, as used herein can take the form of a diagrammatic representation of the first content, such as a file directory structure. Upon receipt of an input to move a part of the mini map, the processor 170 may move a part of the first content corresponding to the part of the mini map and may control display of the moved part of the first content on the display 180. Upon receipt of an input to zoom the mini map, the processor 170 may zoom in or zoom out the mini map and may control display of the zoomed-in or zoomed-out mini map on the display 180.
  • Upon receipt of an input to combine at least two of a plurality of first content items displayed in correspondence with the position of the first user, the processor 170 may combine the at least two first content items and control display of the combined content items on the display 180.
  • The processor 170 may control display of a copy of the first content in correspondence with the position of the second user and, upon receipt of an edit input from the second user, may edit the copy of the first content.
  • The processor 170 may control display of the edited copy of the first content or a copy of the edited copy of the first content in correspondence with the position of the first user. During editing, the copy of the first content or the copy of the copied first content is not editable to the first user.
  • When the mobile terminal 600 of a user is placed on the display 180, the processor 170 may sense it through the sensor unit 150 and perform pairing between the display apparatus 100 and the mobile terminal 600 through the communication unit 120. The processor 170 may identify the user by device information or user information received from the mobile terminal 600.
  • After identifying the mobile terminal 600 of the user, the processor 170 may define a user area corresponding to the position of the mobile terminal 600.
  • After identifying the mobile terminal 600 of the user, the processor 170 may control display of an object related to data sharing between the mobile terminal 600 and the display apparatus 100.
  • If data is shared between the mobile terminal 600 and the display apparatus 100, the processor 170 may control display of content related to the user in the user area.
  • For example, the processor 170 may control display of a whole screen displayed on the mobile terminal 600 in the user area defined on the display 180.
  • In another example, the processor 170 may control display of a content item selected from a screen displayed on the mobile terminal 600 in the user area of the display 180.
  • The processor 170 may control changing of the size of the displayed user area according to a user input.
  • The processor 170 may define user areas for a plurality of users on the display 180. The user areas may not be overlapped with one another.
  • Particularly if specific content is displayed in a common area overlapped between the user areas, the processor 170 may control sharing of content between users corresponding to the user areas. That is, the processor 170 may control transmission of the content to a mobile terminal of another user.
  • The processor 170 may calculate the positions of the first and second users based on an image captured by the camera 195 and may display content for each user in correspondence with the position of the user.
  • Upon receipt of an input to enlarge a content list, the processor 170 may control enlargement of the content list on the display 180. The processor 170 may control display of at least a part of a plurality content items listed in the content list. If any of the displayed content items is of a reproducible size, the processor 170 may control playback and display of the content item.
  • Before displaying the content list, the processor 170 may control display of a plurality of content items. Upon receipt of an input to combine the plurality of content items, the processor 170 may combine the plurality of content items and control display of a content list including the combined content items.
  • During playback and display of content, if a content item is of a reproducible size, the processor 170 may play back content corresponding to the content item and control display of the playback content.
  • During playback and display of content, the processor 170 may increase a display size of content being played back to a largest of display sizes of displayed content items.
  • During enlarged display of content, if an enlargement input is for first-direction enlargement, the processor 170 may control display of a content list enlarged in a first direction.
  • Upon receipt of an input to enlarge content being played back, the processor 170 may control enlargement of the playback content and display of the enlarged content and may control volume-up of the playback content.
  • Upon receipt of a flicking input in a first direction, the processor 170 may control output and display of images related to the content in the first direction.
  • Upon receipt of an input to insert an additional image, the processor 170 may insert the additional image between the content-related images. Upon receipt of a flicking input for the content-related images in a second direction opposite to the first direction, the processor 170 may control display of content including the additional image.
  • Upon receipt of an input to combine a part of a plurality of displayed content items, the processor 170 may generate a content file including the content items corresponding to the part of the displayed content items.
  • The display 180 may display images related to operations of the display apparatus 100. To display images, the display 180 may be configured with a Plasma Display Panel (PDP) display, an LCD, an OLED display, etc. Further, the display 180 may be a flexible display or a Three-Dimensional (3D) display.
  • The power supply 190 may supply operation power to each component under the control of the processor 170. Particularly, the power supply 190 may supply power to the processor 170 that may be implemented on a System On Chip (SOC), the display 180 that displays an image, and the audio output unit 185 for audio output.
  • For this purpose, the power supply 190 may be provided with a converter (not shown) for converting Alternating Current (AC) to Direct Current (DC). For example, if the display 180 is configured with a liquid crystal panel having a plurality of backlight lamps, the power supply 190 may further include an inverter (not shown) capable of Pulse Wave Modulation (PWM) to change an illuminance or drive dimming.
  • One or more cameras 195 may be provided. An image captured by a camera 195 may be transmitted to the processor 170. The processor 170 may identify each user, determine the position of the user, and recognize a gesture by processing the captured image.
  • The display apparatus 100 may be any of a TeleVision (TV), a monitor, a projector, etc.
  • The mobile terminal 600 may be any of a smart phone, a music player, a tablet, an e-book reader, a tab-book, etc.
  • While the memory 140 is shown in FIG. 2 as separately configured from the processor 170, the present invention is not limited thereto. Rather, the memory 140 may be incorporated into the processor 170.
  • The block diagram of the display apparatus 100 illustrated in FIG. 2 is purely exemplary. Depending upon the specifications of the display apparatus 100 in actual implementation, the components of the display apparatus 100 may be combined or omitted or new components may be added. That is, two or more components are incorporated into one component or one component may be configured as separate components, as needed.
  • FIG. 3 is a flowchart illustrating a method for operating the display apparatus according to an embodiment of the present invention, and FIGS. 4A to 20C are views referred to for describing the method for operating the display apparatus illustrated in FIG. 3.
  • Referring to FIG. 3, the processor 170 of the display apparatus 100 calculates the position of a first user (S310).
  • As illustrated in FIG. 1, the processor 170 may receive an image captured by the camera 195 disposed above the display 180 that sits horizontally like a table top and may calculate the position of a user based on the captured image.
  • Referring to FIG. 4A, if a first user 10 is located on one side (hereinafter, referred to as an upper side) of the horizontally placed display 180, the processor 170 may calculate the position of the first user 10 based on an image captured by the camera 195.
  • In addition to the position of the first user 10, the processor 170 may recognize the first user 10. For example, the processor 170 may recognize the first user 10 by matching the first user 10 to user images stored in a database using a facial recognition algorithm, an iris recognition algorithm, etc.
  • Subsequently, the processor 170 displays first content on the display 180 in correspondence with the position of the first user (S315).
  • After recognizing the first user 10 and determining the position of the first user 10, the processor 170 may control display of the first content in correspondence with the position of the first user 10. The first content may be displayed automatically after the first user 10 is recognized and located, or may be displayed in response to a content display input.
  • FIG. 4A illustrates an example of displaying image-based first content 410 in correspondence with the position of the first user 10. Particularly, the imaged-based first content 410 is rotated by 180 degrees according to the position of the first user 10 so that the first user 10 may view the content 410 conveniently.
  • The first user 10 may perform various tasks such as movement, zoom-in/zoom-out, combining, etc. on the first content 410 by applying a touch input or the like to the display 180.
  • The processor 170 then calculates the position of a second user (S320).
  • If a second user 20 appears while the first user 10 is using the display apparatus 100 from behind the display apparatus 100, the processor 170 may calculate the position of the second user 20 based on an image captured by the camera 915.
  • The processor 170 may recognize the second user 20 as well as locate the second user 20.
  • Referring to FIG. 4B, the second user 20 is on the opposite side (hereinafter, referred to as a lower side) of the horizontally placed display 180 to the first user 10.
  • The processor 170 displays second content on the display 180 in correspondence with the position of the second user (S325). The processor 170 changes a display area of the first content, taking into account the position of the second user (S330).
  • After recognizing and locating the second user 20, the processor 170 may control display of the second content on the display 180 in correspondence with the position of the second user 20. The second content may be displayed automatically after the second user 20 is recognized and located, or may be displayed in response to a content display input.
  • The processor 170 may change the display area of the first content according to the position of the second user. For example, the processor 170 may change at least one of the size or position of the display area of the first content so that the display area of the second content may not be overlapped with the display area of the first content.
  • Referring to FIG. 4B, image-based second content 418 is additionally displayed in correspondence with the position of the second user 20. Particularly, the image-based second content 418 is displayed scaled-down in correspondence with the position of the second user 20 located across the first user 10. Therefore, the second user 20 may view the second content 418 conveniently.
  • Simultaneously with the display of the second content 418, the first content 410 illustrated in FIG. 4A is preferably scaled down. In FIG. 4B, the first content 410 is vertically scaled down to first content 415 and the vertically scaled-down first content 415 is displayed in correspondence with the position of the first user 10. Accordingly, the first user 10 may view the first content 415 without overlap with the second content 418.
  • Compared to FIG. 4A, FIG. 5A illustrates the first user 10 on a left side of the horizontally placed display 180. Therefore, the image-based first content 410 may be displayed rotated counterclockwise by 90 degrees.
  • In FIG. 5B, the second user 20 is additionally located on a right side of the horizontally placed display 180. Thus, the image-based second content is horizontally scaled down and rotated clockwise by 90 degrees. The resulting second content 428 is displayed in front of the second user 20. Additionally, the first content 410 illustrated in FIG. 5A may be horizontally scaled down to first content 425 and the first content 425 may be displayed in front of the first user 10.
  • If the second user moves, the processor 170 may change the display position of the second content along with the movement of the second user. Further, the processor 170 may change at least one of the size or position of the display area of the first content so that the changed display area of the second content may not be overlapped with the display area of the first content.
  • In FIG. 5C, with the first user 10 on the left side of the horizontally placed display 180, the second user 20 moves from the right side of the display 180 to the lower side of the display 180.
  • The processor 170 may trace the movement of the second user 20 based on an image captured by the camera 195 and may change the position of the display area of the second content in correspondence with the position of the second user.
  • Referring to FIG. 5C, the second content 428 displayed in a right part of the display 180 in FIG. 5B is rotated counterclockwise by 90 degrees and displayed in a lower right part of the display 180. Therefore, the second user 20 may readily view second content 515 in spite of the second user's movement.
  • First content 510 is displayed in correspondence with the position of the first user 10 on the left side of the horizontally placed display 180 in FIG. 5C.
  • Referring to FIG. 5D, the second user 20 moves further to the left from where the second user 20 stays in FIG. 5C. Accordingly, the processor 170 may control display of second content 518 moved further to the left.
  • As a result, the first content 510 and the second content 518 may overlap with each other over an overlap area 519.
  • To eliminate the overlap area 519, the processor 170 may change the display size or position of at least one of the first content 510 or the second content 518.
  • For example, the processor 170 may assign a weight to the moved second content 518 to thereby change the display size or position of the first content 510. Or the processor 170 may assign a weight to the displayed first content 510 to thereby change the display size or position of the second content 518.
  • Referring to FIG. 5E, the size of the first content 510 is changed. Specifically, the processor 170 may change the display size of the first content 510 in correspondence with a movement direction of the second user 20. Therefore, the resulting first content 511 does not overlap with the second content 518.
  • Referring to FIG. 6A, the first user 10 and the second user 20 are located on the upper and lower sides of the display 180, respectively and thus the first content 415 and the second content 418 are displayed separately in upper and lower parts of the display 180.
  • Referring to FIG. 6B, the first user 10 and the second user 20 are located on the upper side of the display 180, the second user 20 to the left of the first user 10 and a third user 30 is located on the lower side of the display 180. Thus first content 611 and second content 613 are displayed separately in upper left and top right parts of the display 180, whereas third content 617 is displayed in the lower part of the display 180.
  • Referring to FIG. 7A, the first user 10 and the second user 20 are located on the left and right sides of the display 180, respectively, and the first content 425 and the second content 428 are displayed separately in the left and right parts of the display 180.
  • Referring to FIG. 7b, the first user 10 is located on the left side of the display 180, whereas the second user 20 and the third user 30 are located respectively on a right upper side and a right lower side of the display 180. Accordingly, the first content 425 is displayed in the left part of the display 180, whereas second content 621 and third content 623 are displayed respectively in a right upper part and a right lower part of the display 180.
  • Referring to FIG. 7C, the first, second, and third users 10, 20, and 30 are located from left to right in this order on the upper side of the display 180. When the third user 30 moves toward the second user 20, that is, to the left as viewed from across the third user 30, an overlap area 718 is created between second content 713 and third content 717 among first content 711, the second content 713, and the third content 7171 that are displayed side by side on the display 180.
  • The processor 170 may change the display size or position of the second content 713 or the display size of the third content 717 in order to eliminate the overlap area 718.
  • For example, the processor 170 may control movement of the display position of the second content 713 illustrated in FIG. 7C further to the left or reduction of the display size of the second content 713. In another example, the processor 170 may control reduction of the display size of the third content 717.
  • Meanwhile, upon receipt of an input to move the first content, the processor 170 may control movement of the first content or a part of the first content in correspondence with the position of the second user and may control display of the moved first content or the moved part of the first content on the display 180.
  • Referring to FIG. 8A, with the first user 10 located on the left side of the display 180 and the second user 20 located on an upper right side of the display 180, if the first user 10 makes an input to move displayed first content 810 to the right with the right hand 11, the first content 810 is moved. Thus, moved first content 811 may be displayed in an upper right part of the display 180.
  • A "right move" input may be defined as a touch-based drag input to the right on the display 180, a touch-based flick input to the right on the display 180, or a gesture-based input.
  • Upon receipt of an input to display a mini map for the first content, the processor 170 may control display of the mini map on the display 180. Upon receipt of an input to move a part of a mini map, the processor 170 may control movement of a part of the first content corresponding to the part of the mini map and may control display of the moved part of the first content.
  • Upon receipt of an input to zoom a mini map, the processor 170 may control zoom-in or zoom-out of the mini map and may control display of the zoomed-in or zoomed-out mini map on the display 180.
  • Referring to FIG. 9A, as the first user 10 and the second user 20 are located respectively on the left and right sides of the display 180, first content and second content are displayed separately in correspondence with the positions of the first and second users 10 and 20.
  • Upon receipt of an input to display a mini map for the first content, the processor 170 may control display of a mini map 912 for the first content on the display 180.
  • If the first user 10 touches the display 180 with both hands, the mini map 912 may be displayed over the first content. Or the mini map 912 may be displayed apart from the first content.
  • The mini map 912 is smaller than the first content in display size. To facilitate a user's manipulation, the mini map 912 may be displayed in correspondence with the position of the user's hand.
  • Referring to FIG. 9B, an enlarged mini map 914 is displayed in response to an input to zoom in a mini map. The enlarged mini map 914 includes a text area and an image area, by way of example.
  • Referring to FIG. 9C, if the first user 10 selects a specific area of the enlarged mini map 914 and makes an input to move the specific area, the specific area is moved.
  • Specifically, if the first user 10 selects an image area 931 of the enlarged mini map 914 and makes a right move input for the image area 931, the processor 170 may control movement of the image area 931 and thus display of a moved image area 933 in correspondence with the position of the second user 20. Therefore, an image area, that is, a content part that the first user wants to share may be transferred to the second user, thereby increasing user convenience.
  • Content may move at different speeds in different areas. In the case of a right move input for the image area 931, the processor 170 may set the image area 931 to move faster in a first area 911 for the first content corresponding to the position of the first user 10 than in a second area 913 for the second content corresponding to the position of the second user 20. According to this setting, if the second user 20 makes a gesture of grabbing the moving content, a user-friendly user interface may be created.
  • Upon receipt of an input to combine at least two of a plurality of first content items displayed in correspondence with the position of the first user, the processor 170 may control combining of the first content items and display of the combined first content items.
  • Referring to FIG. 10A, the first user 10 makes an input to combine a plurality of content items 1011 and 1012 displayed in correspondence with the position of the first user 10. In this case, the processor 170 may combine the plurality of content items 1011 and 1012. That is, the content items 1011 and 1012 displayed apart from each other may be combined into one content item.
  • Referring to FIG. 10B, combined content items 1013 are displayed overlapped. A predetermined time later, the overlapped content items are combined into one file.
  • Referring to FIG. 10C, content 1014 combined into one file or one page is displayed. In this manner, a user may manage content items efficiently by combining the content items.
  • If content items are different, the content items may be combined into one file or one page as illustrated in FIGS. 10A, 10B, and 10C. On the other hand, if content items are identical, the content items may not be combined.
  • Referring to FIG. 11A, the first user 10 makes an input to combine the plurality of content items 1011 and 1012 displayed in correspondence with the position of the first user 10, as illustrated in FIG. 10A. In this case, the processor 170 may combine the plurality of content items 1011 and 1012. That is, the plurality of content items 1011 and 1012 displayed apart from each other may be combined into one content item.
  • Referring to FIG. 11B, combined content items 1016 are displayed overlapped.
  • Referring to FIG. 11C, in the case of the same content, one content item 1011 remains, while the other content item 1012 disappears, rather than the content items 1011 and 1012 are combined into one file or one page.
  • The processor 170 may control display of a copy of the first content in correspondence with the position of the second user and, upon receipt of an edit input from the second user, may edit the copy of the first content.
  • The processor 170 may control display of an edited copy of the first content or a copy of the edited copy of the first content in correspondence with the position of the first user. During editing, the copy of the first content or the copy of the copy of the first content may not be editable to the first user.
  • Referring to FIG. 12A, first content 1211 and a copy 1213 of the first content are displayed respectively in correspondence with the positions of the first user 10 and the second user 20 respectively on the left and right sides of the display 180.
  • Particularly, the copy 1213 of the first content displayed in front of the second user 20 may be moved and displayed, in response to a content movement input in a similar manner to in FIGS. 9A, 9B, and 9C, or in response to a content edit input from the second user 20.
  • To edit the first content, the second user 20 may zoom in the copy of the first content. Thus, in FIG. 12A, the copy 1213 of the first content is displayed larger than the first content 1211.
  • Additional content may be added to the copy 1213 of the first content by an edit input of the second user 20. While text 1219 is added to the copy 1213 of the first content in FIG. 12A, various types of data including an image, an audio, a Web address, and the like may be added.
  • The processor 170 may control display of the copy of the first content or a copy of the copy of the first content, which is being edited or has been completely edited, in correspondence with the position of the first user, so that the first user may view the content edited by the second user.
  • Referring to FIG. 12B, a copy 1217 of the copy of the first content under editing and the first content 1211 are displayed side by side.
  • The copy 1217 of the copy of the first content may not be editable to the first user 10 and thus a lock object indicating uneditable may be displayed. After the second user 20 finishes editing the content, the copy 1217 of the copy of the first content may be editable to the first user 10.
  • Therefore, the first user may readily view content which is being edited or has been edited completely.
  • After the editing, the first content 1211 and the copy 1217 of the copy of the first content may be combined. That is, as illustrated in FIGS. 10A, 10B, and 10C, upon receipt of a combine input, the copy 1217 of the copy of the first content that contains a changed part may remain, while the first content 1211 may be removed. Accordingly, the updated first content may be stored or displayed.
  • The positions of the first and second users may be determined through mobile terminals of the first and second users, instead of the camera 195.
  • For example, if the display apparatus 100 includes the sensor unit 150 that senses a touch with a mobile terminal, particularly, if the sensor unit 150 includes a pressure touch sensor, the processor 170 may sense a pressure applied by the mobile terminal and thus may locate a user of the mobile terminal.
  • In another example, if the sensor unit 150 includes a capacitive touch sensor, the processor 170 may locate the user by sensing a variation in capacitance caused by the mobile terminal.
  • After locating a user, the processor 170 may perform pairing between the display apparatus 100 and a mobile terminal of the user, receive device information or user information about the mobile terminal through the communication unit 120, and recognize the user based on the received information.
  • Or if the mobile terminal approaches within a predetermined distance, the processor 170 may receive device information, user information, etc. from the mobile terminal through the communication unit 120 and may recognize and locate the user of the mobile terminal. When the mobile terminal is within the predetermined distance, the processor 170 may control data exchange with the mobile terminal by Near Field Communication (NFC).
  • Now, embodiments of the present invention for sensing a touch made by a mobile terminal through the sensor unit 150, receiving device information or user information about the mobile terminal through the communication unit 120, and recognizing a user of the mobile terminal based on the received information will be described below.
  • FIGS. 13A to 18B illustrate an exemplary operation for locating and recognizing a user through the sensor unit 150 and the communication unit 120.
  • Referring to FIG. 13A, the first user 10 places a first mobile terminal 600a on the upper part of the horizontally placed display 180.
  • The processor 170 may calculate the position of the first mobile terminal 600a through the sensor unit 150 and estimate (or calculate) the position of the first user 10 based on the position of the first mobile terminal 600a.
  • Meanwhile, the processor 170 may perform pairing between the display apparatus 100 and the first mobile terminal 600a through the communication unit 120 and may identify the first mobile terminal 600a as used by the first user 10 based on device information or user information received from the first mobile terminal 600a.
  • Accordingly, the processor 170 may control display of the image-based first content 410 in correspondence with the position of the first user 10.
  • The first content 410 may be content stored in the first mobile terminal 600a. For example, the first content 410 may be latest reproduced content, favorite content, or user-set content.
  • Referring to FIG. 13B, with the mobile terminal 600a placed on the upper part of the display 180, a second mobile terminal 600b of the second user 20 is placed on the lower part of the display 180.
  • The processor 170 may calculate the position of the second mobile terminal 600b through the sensor unit 150 and estimate (or calculate) the position of the second user 20 based on the position of the second mobile terminal 600b.
  • Meanwhile, the processor 170 may perform pairing between the display apparatus 100 and the second mobile terminal 600b through the communication unit 120 and may identify the second mobile terminal 600b as used by the second user 20 based on device information or user information received from the second mobile terminal 600b.
  • Accordingly, the processor 170 may control display of the image-based second content 418 in correspondence with the position of the second user 20.
  • The second content 418 may be content stored in the second mobile terminal 600b. For example, the second content 418 may be latest reproduced content, favorite content, or user-set content.
  • Along with displaying the second content 418, the first content 410 displayed in FIG. 13A is preferably scaled down. In FIG. 13B, the vertically scaled-down first content 415 is displayed in correspondence with the position of the first user 10. Thus the first user 10 may view the first content 415 without overlap with the second content 418.
  • Operations illustrated in FIGS. 14A to 14E are identical to the operations illustrated in FIGS. 5A to 5E except that a user is recognized and located. Thus, a description of the operations illustrated in FIGS. 14A to 14E will not be provided herein.
  • Likewise, FIGS. 15A and 15B correspond to FIGS. 6A and 6B, FIGS. 16A, 16B and 16C correspond to FIGS. 7A, 7B, and 7C, FIGS. 17A and 17B correspond to FIGS. 8A and 8B, and FIGS. 18A, 18B and 18C correspond to FIGS. 12A, 12B, and 12C.
  • While not shown, the operations illustrated in FIGS. 9A to 11C may be performed.
  • In FIGS. 13A to 18B, when displaying content after recognizing a user using a mobile terminal of the user, the processor 170 may change a display area of the content in proportion to the size of the mobile terminal.
  • Or the processor 170 may change the display area of the content in proportion to the amount of content data stored in the mobile terminal or the resolution of content data to be displayed. Therefore, the amount of the content data stored in the mobile terminal may be readily recognized.
  • The sensor unit 150 of the display apparatus 100 may sense a first surface and a second surface of a mobile terminal, separately.
  • For example, the processor 170 may distinguish a touch made on the display apparatus 100 by a front surface of the mobile terminal having a touch screen formed thereon from a touch made on the display apparatus 100 by a rear surface of the mobile terminal based different capacitance variations. The processor 170 may control display of different content corresponding to the front and rear surfaces of the mobile terminal.
  • Referring to FIG. 19A, with a plurality of users located around the display 180, a user area is defined for each user.
  • Particularly, mobile terminals 600a, 600b, 600c, 600d, and 600e of users 10, 20, 30, 40, and 50 are placed on the display 180 and user areas Ar1, Ar2, Ar3, Ar4, and Ar5 are defined for the respective users 10, 20, 30, 40, and 50.
  • The processor 170 may differentiate at least one of the sizes, shapes, colors, etc. of the user areas Ar1, Ar2, Ar3, Ar4, and Ar5 depending on where the users 10, 20, 30, 40, and 50 put the mobile terminals 600a, 600b, 600c, 600d, and 600e as viewed from the users 10, 20, 30,40, and 50.
  • In FIG. 19A, when the first user 10 puts the mobile terminal 600a on the display 180 at the first user's left side, the first user area Ar1 is extended to the right of the mobile terminal 600a, including the mobile terminal 600a.
  • When the second user 20 puts the mobile terminal 600b on the display 180 far from the second user 20, the second user area Ar2 is extended to the second user 20, including the mobile terminal 600b. When the third user 30 puts the mobile terminal 600c on the display 180, right in front of the third user 30, the third user area Ar3 is extended backward from the mobile terminal 600c, including the mobile terminal 600c. When the fourth user 40 puts the mobile terminal 600d on the display 180 at the fourth user's right side, the fourth user area Ar4 is extended to the left of the mobile terminal 600d, including the mobile terminal 600d. When the fifth user 50 puts the mobile terminal 600e on the display 180 at a specific position with respect to the fifth user 50, the fifth user area Ar5 is created around the mobile terminal 600e, including the mobile terminal 600e.
  • It may be further contemplated as another embodiment of the present invention that user areas are defined for a plurality of users depending on where the users put their mobile terminals on the display 180 in such a manner that each user area is apart from where a mobile terminal is put on the display 180, as illustrated in FIGS. 18A and 18B.
  • The processor 170 may arrange content items in a user-friendly manner in a user area for each user.
  • When the mobile terminals 600a, 600b, 600c, 600d, and 600e are placed on the display 180, the processor 170 may sense the existence of the mobile terminals 600a, 600b, 600c, 600d, and 600e on the display 180 through the sensor unit 150 and may perform pairing between the display apparatus 100 and the mobile terminals 600a, 600b, 600c, 600d, and 600e through the communication unit 120. Further, the processor 170 may recognize the users 10, 20, 30, 40, and 50 based on device information or user information received from the mobile terminals 600a, 600b, 600c, 600d, and 600e.
  • After recognizing the mobile terminals 600a, 600b, 600c, 600d, and 600e, the processor 170 may define the user areas Ar1, Ar2, Ar3, Ar4, and Ar5 in correspondence with the positions of the mobile terminals 600a, 600b, 600c, 600d, and 600e.
  • After identifying a mobile terminal of a user, the processor 170 may control display of an object related to data sharing between the mobile terminal and the display apparatus 100.
  • This object may be displayed in a user area defined for the user. If the user selects the object, the processor 170 may control data sharing with the mobile terminal. For example, the processor 170 may control reception of data from the mobile terminal and display of the data in the user area.
  • Meanwhile, the processor 170 may control setting of the size of a user area to a default value.
  • Or the processor 170 may control changing of the size of a user area according to a user input.
  • For example, the size of a user area may be adjusted by an On Screen Display (OSD) menu or by a touch and drag of a borderline of the user area. If the borderline of the user area is dragged in or out diagonally, the whole user area may be contracted or enlarged. If the borderline of the user area is dragged in or out horizontally, the user area may be contracted or enlarged horizontally. If the borderline of the user area is dragged in or out in one direction, the layout of content items in the user area may be changed, as illustrated in FIGS. 25A and 25B.
  • Referring to FIG. 19B, the size of the first user area Ar1 is changed in response to an input to change the first user area Ar1 of the first user 10.
  • Particularly when the first user 10 spreads or gathers the user's hands along the diagonal line of the first user area Ar1, the first user area Ar1 may be enlarged or contracted.
  • If data is shared between a mobile terminal and the display apparatus 100, the processor 170 may control display of user-related content in a user area defined for a user of the mobile terminal.
  • For example, the processor 170 may control display of a whole screen displayed on the mobile terminal in the user area of the display 180.
  • In another example, the processor 170 may control display of a content item selected from a screen displayed on the mobile terminal 600 in the user area of the display 180.
  • Referring to FIG. 19C, content displayed on a screen of the mobile terminal 600a is displayed as it is in the first user area Ar1 of the first user 10.
  • After data sharing is set, if the first user 10 makes a touch input or a flicking input for the mobile terminal 600a, the display apparatus 100 may receive data of a whole screen 1905 displayed on the mobile terminal 600a from the mobile terminal 600a and the processor 170 may control display of the received data in the first user area Ar1 of the first user 10.
  • Preferably, the size of each of content items and the size of a screen 1910 including the content items
    are larger than the size of each of content items and the size of the screen 1905 including the content items in the mobile terminal 600a. Thus, the first user 10 may readily view and share content items of the mobile terminal 600a.
  • Referring to FIG. 19D, a content item selected from a screen displayed on the mobile terminal 600a is displayed in the first user area Ar1 of the first user 10.
  • After data sharing is set, when a touch input or a flicking input is made to a specific content item 1908 on the screen 1905 of the mobile terminal 600a, the display apparatus 100 may receive data related to the selected content item 1908 from the mobile terminal 600a and the processor 170 may control display of the received data in the first user area Ar1 of the first user 10.
  • Preferably, the size of a content item 1918 displayed on the display 180 is larger than the content item 1908 displayed on the mobile terminal 600a. Thus, the first user 10 may readily view and share content items of the mobile terminal 600a.
  • If a touch input or a flicking input is made to the specific content item 1908 on the screen 1905 of the mobile terminal 600a, the content item 1908 may be highlighted or the color of its outline may be changed.
  • In a data sharing mode between the mobile terminal 600a and the display apparatus 100, a touch input or a flicking input to the mobile terminal 600a may be set to be interpreted as a control input for the display apparatus 100, not a control input for the mobile terminal 600a. Therefore, a controller of the mobile terminal 600a may transmit the touch input or the flicking input to the display apparatus 100, ignoring the touch input or the flicking input in the data sharing mode.
  • In the data sharing mode between the mobile terminal 600a and the display apparatus 100, if a folder is displayed on the mobile terminal 600a, the display apparatus 100 may share the whole folder.
  • In the data sharing mode, the processor 170 may control display of an object indicating data sharing-on in a user area. Therefore, a user corresponding to the user area may readily recognize activation of data sharing.
  • The processor 170 may define user areas for a plurality of users and that in such a manner that the user areas are not overlapped.
  • Particularly, if specific content is displayed in a common area overlapped between user areas, the processor 170 may control sharing of the content between users corresponding to the user areas. That is, the processor 170 may control transmission of the content to a mobile terminal of another user.
  • Referring to FIG. 20A, first and second user areas Ara and Arb are defined on the display 180 respectively for the first and second users 10 and 20. Particularly, the first and second user areas Ara and Arb are partially overlapped with each other. The overlap area may be highlighted.
  • The first and second user areas Ara and Arb may be overlapped based on movement of a user, a change in the size of a user area, or the type or size of displayed content data.
  • Referring to FIG. 20B, first, second, and third content lists 2010a, 2010b, and 2010c for the first, second, and third users 10, 20, and 30 are displayed respectively in user areas Ar1, Arb, and Arc.
  • The second user 20 disposes a specific content item 2018 of the second content list 2010b in a common area Arz overlapped between the second user area Arb and the third user area Arc.
  • Thus, the processor 170 may control sharing of the content item 2018 between the third user 30 and the second user 20. That is, the content item 2018 may be transmitted to the second user area Arb or the second mobile terminal 600b of the second user 20.
  • Referring to FIG. 20C, first and second user areas Ara and Arm are defined on the display 180 respectively for the first and second users 10 and 20. Particularly, data of the first user area Ara is transmitted to the second user area Arm by data sharing.
  • The second user 20 may use and edit the shared content and share the edited content with the first user 10. Thus the content may be reconstructed.
  • FIG. 21 is a flowchart illustrating a method for operating the display apparatus according to another embodiment of the present invention, and FIGS. 22 to 30B are views referred to for describing the method for operating the display apparatus illustrated in FIG. 2I.
  • Referring to FIG. 21, the processor 170 of the display apparatus 100 controls display of a content list including a plurality of content items on the display (S2310).
  • In FIG. 23(a), a content list 2415 including a plurality of content items is displayed on the horizontally placed display 180.
  • The content list 2415 may be automatically displayed after user recognition or may be displayed in response to an additional input.
  • Before step S2310, a step of displaying a plurality of content items on the display 180 and a step of combining a plurality of content items, upon receipt of a content combine input, and displaying a content list including the combined content items may be performed.
  • Referring to FIG. 22, the content list 2415 including a plurality of content items 2410, CT1 to CT24 may be displayed on the horizontally placed display 180.
  • While the plurality of content items 2410, CT1 to CT24 are displayed in full screen on the display 180 in FIG. 22, the plurality of content items 2410, CT1 to CT24 may be displayed in a partial area of the display 180 in correspondence with the position of a user.
  • The processor 170 may calculate the position of a user based on an image captured by the camera 195.
  • In FIG. 22, the user makes a touch input of gathering the displayed plurality of content items 2410, CT1 to CT24 toward the user with the user's hands. Thus, the plurality of content items 2410, CT1 to CT24 are combined into one file or the single content list 2415.
  • That is, the processor 170 may generate the content list 2415 including the combined content items 2410 in response to an input to combine the plurality of content items 2410.
  • Subsequently, the processor 170 controls enlargement of the content list and display of at least a part of the plurality of content items included in the content list in response to an input to enlarge a content list (S2320).
  • The processor 170 sequentially controls enlargement and display of the content list in response to the input to enlarge a content list. Herein, the processor 170 controls display of at least a part of the plurality of content items included in the content list. Thus, the user may readily view content items included in the content list.
  • In FIG. 23(a), the user makes a touch input to enlarge the displayed content list 2415 by spreading the user's hands sideways. Thus, the content list 2415 is enlarged.
  • FIG. 23(b) illustrates a content list 2416 enlarged after a first time and FIG. 23(c) illustrates a content list 2417 enlarged after a second time.
  • The content items of the content list 2416 are shown in FIG. 23(b) as content items of a first size, CTa, ..., CTk and are shown in FIG. 23(c) as content items of a second size larger than the first size, CT1, ... CT11. However, many other variations can be made.
  • For example, the content items illustrated in FIGS. 23(b) and 23(c) may be displayed as thumbnail images, not file names. In response to an enlarge input, the size or resolution of the thumbnail images may be increased.
  • In another example, because the content items are larger in FIG. 23(c) than in FIG. 23(b), fewer content items may be displayed in FIG. 23(c).
  • If a part of the displayed content items are of a reproducible size, the processor 170 controls playback and display of the content items (S2330).
  • Upon receipt of an enlarge input, the processor 170 continues to enlarge the content list. If any of the displayed content items is of a reproducible size, for example of a size that permits the content to be reproduced at a full resolution on the display, then the processor 170 may control playback and display of the content item.
  • Referring to FIG. 24, if the user makes a sideways enlarge input, some content items 2607, 2608, 2609, 2610, 2611, 2612, and 2613 of the content list are arranged side by side, after the operation illustrated in FIG. 23(c).
  • If any 2610 of the content items in the content list is of a reproducible size, the processor 170 may control playback and display of the content item 2610 as illustrated in FIG. 24. Particularly, the processor 170 may set the size of the playback content item 2610 to be larger than the size of each of the other content items 2607, 2608, 2609, 2611, 2612, and 2613. Thus, the user may readily view the playback content.
  • Referring to FIG. 25A, with a content list 2710 including some content items 2711 displayed in an area of the display 180, the first user 10 makes a vertical enlarge touch input.
  • The processor 170 may control display of the content list 2715 enlarged vertically, as illustrated in FIG. 25A. The content list 2715 is fully enlarged vertically on the display 180 in FIG. 25A.
  • Referring to FIG. 25B, with the content list 2710 including the content items 2711 displayed in an area of the display 180, the first user 10 makes a horizontal enlarge touch input.
  • The processor 170 may control display of the content list 2715 enlarged horizontally, as illustrated in FIG. 25B. The content list 2715 is fully enlarged horizontally on the display 180 in FIG. 25B.
  • Referring to FIG. 26, as reproduced content is enlarged in response to a user's enlargement input, the audio volume of the content is increased.
  • In response to an input to enlarge playback content, the processor 170 controls display of the playback content enlarged and controls volume-up of the content.
  • Referring to FIG. 26(a), content 2810 of a first size is being played back in an area of the display 180 in response to a user's enlarge input. Sound 2820a and 2820b is output at a first volume level through the audio output unit 185, in proportion to the display size of the content 2810.
  • Referring to FIG. 26(b), content 2812 of a second size larger than the first size of the content 2810 illustrated in FIG. 26(a) is being played back in an area of the display 180 in response to a user's enlarge input. Sound 2822a and 2822b is output at a second volume level higher than the first volume level of the content 2810 illustrated in FIG. 26(a) through the audio output unit 185, in proportion to the display size of the content 2812.
  • Referring to FIG. 26(c), content 2816 of a third size larger than the second size of the content 2812 illustrated in FIG. 26(b) is being played back in an area of the display 180 in response to a user's enlarge input. Sound 2826a and 2826b is output at a third volume level higher than the second volume level of the content 2812 illustrated in FIG. 26(b) through the audio output unit 185, in proportion to the display size of the content 2816.
  • Since the sound volume of playback content is changed according to the display size of the content, the user may readily recognize the display size of the content.
  • If a user makes a first-direction flicking input for playback content, the processor 170 may control display of images related to the content in a first direction.
  • Upon receipt of an input to insert an additional image, the processor 170 may insert the additional image between the content-related images. Upon receipt of a flicking input in a second direction opposite to the first direction, the processor 170 may control display of the content including the additional image.
  • FIGS. 27A to 27E illustrate an operation for generating new content by adding an additional image to content being reproduced.
  • Referring to FIG. 27A, with the first user 10 located on a lower left side of the display apparatus 100, specific content 2910 is displayed on the display 180. The content 2910 may be being played back. The content 2910 may correspond to the content 2610 described with reference to FIG. 24.
  • If the first user 10 makes a touch-based rightward flicking input, the processor 170 of the display apparatus 100 may control display of a plurality of images included in the content 2910 unfolded to the right, as illustrated in FIG. 27B. The content 2910 may be paused during playback.
  • In FIG. 27B, the content 2910 includes thumbnail images 2912 to 2918, TH1 to TH7.
  • Referring to FIG. 27C, the user makes a touch input of spreading the fourth and fifth images TH4 and TH5 from each other and inserts a new image 2920, THa between the fourth image TH4 and the fifth image TH5. Then, the new image 2920 is displayed between the fourth image TH4 and the fifth image TH5.
  • If the user 10 makes a touch-based leftward flicking input for the plurality of images 2912 to 2917 and 2920, TH1 to TH7 and THa, the processor 170 may display of new content 2930 including the new inserted image, while removing the plurality of images, as illustrated in FIG. 27E. Accordingly, content including an intended image may be generated simply. The new content 2930 may be played back automatically after it is created.
  • Upon receipt of an input to combine a part of a plurality of displayed content items, the processor 170 may generate a content file including the corresponding content items.
  • FIGS. 28A to 30B illustrate an operation for, upon receipt of an input to combine a part of a plurality of displayed content items, generating a content file including the corresponding content items.
  • Referring to FIG. 28A, with different types of content items (e.g., PHOTO and MUSIC), the user 10 makes a touch input to combine a first photo item 3011, PHOTO1 with a first music item 3012, MUSIC1.
  • Then the first photo item 3011, PHOTO1 and the first music item 3012, MUSIC1 may be moved, and a moved first photo item 3011a and a moved first music item 3012a may be overlapped and displayed as content 3015.
  • As illustrated in FIG. 28B, the first photo item 3011a and the first music item 3012a may be displayed combined into one file 3055.
  • Referring to FIG. 29A, with different types of content items (e.g., PHOTO and MUSIC), the user 10 makes a touch input to combine a second photo item 3022, PHOTO3 with a fourth photo item 3024, PHOTO4.
  • Then the second photo item 3022, PHOTO2 and the fourth photo item 3024, PHOTO4 may be moved, and a moved second photo item 3022a and a moved fourth photo item 3024a may be overlapped and displayed as content 3025.
  • As illustrated in FIG. 29B, the second photo item 3022a and the fourth photo item 3024a may be displayed combined into one file 3155. Thus a user-desired content file may be generated.
  • Referring to FIG. 30A, with different types of content items (e.g., PHOTO and MUSIC), the user 10 makes a touch input to combine a third music item 3032, MUSIC3 with a second music item 3034, MUSIC2.
  • Then the third music item 3032, MUSIC3 and the second music item 3034, MUSIC2 may be moved, and a moved third music item 3032a and a moved second music item 3034a may be overlapped and displayed as content 3035.
  • As illustrated in FIG. 30B, the third music item 3032a and the second music item 3034a may be displayed combined into one file 3255. Thus a user-desired content file may be generated.
  • As is apparent from the above description of the display apparatus and the method for operating the same according to the present invention, content can be moved, shared, and edited by means of a large display apparatus that can be used by a plurality of users. Therefore, user convenience can be increased.
  • Particularly since content of a mobile terminal or content of the display apparatus set for each user can be displayed according to the position of the user, different content can be played back and displayed. Thus, user convenience can be increased.
  • The display apparatus and the method for operating the same according to the foregoing exemplary embodiments are not restricted to the exemplary embodiments set forth herein. Therefore, variations and combinations of the exemplary embodiments set forth herein may fall within the scope of the present invention.
  • The method for operating a display apparatus according to the foregoing exemplary embodiments may be implemented as code that can be written on a computer-readable recording medium and thus read by a processor. The computer-readable recording medium may be any type of recording device in which data is stored in a computer-readable manner. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disc, an optical data storage, and a carrier wave (e.g., data transmission over the Internet). The computer-readable recording medium can be distributed over a plurality of computer systems connected to a network so that computer-readable code is written thereto and executed therefrom in a decentralized manner. Programs, code, and code segments to realize the embodiments herein can be construed by one of ordinary skill in the art.
  • While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the scope defined by the following claims.

Claims (12)

  1. A display apparatus comprising:
    a memory (140) configured to store a plurality of content items;
    a display (180) configured to display a content list including the plurality of content items; and
    a processor (170) configured to, upon receipt of an input to enlarge the content list,
    control display in an enlarged manner of at least a part of the plurality of content items in the content list, the processor being further configured to, when a second part of the enlarged and displayed content items are of a reproducible size, control playback and display of content of content items corresponding to the second part of the displayed content items, characterized in that:
    the processor is configured to calculate a position of a first user (10) in respect to the display (180) based on positions of a first mobile terminal (660a) of the first user, control display of first content on the display in correspondence with the position of the first user,
    calculate a position of a second user (20) in respect to the display (180) based on positions of a second mobile terminal (600b) of the second user, control display of second content on the display in correspondence with the position of the second user, and
    change a display area of the first content, taking into account the position of the second user,
    wherein the processor is configured to control display of a copy (1213) of the first content in correspondence with the position of the second user and, upon receipt of an edit input from the second user, edit the copy of the first content,
    wherein the processor is configured to control display of a copy (1217) of the copy (1213) of the first content which is being editing or has been completely edited by the second user, in correspondence with the position of the first user,
    wherein the copy of the copy of the first content is not editable by the first user during said editing of the copy of the first content by the second user, and
    wherein, when a specific content item is positioned in an overlap area between the display area of the first content and a display area of the second content, the processor is configured to control sharing of the specific content item between the first mobile terminal and the second mobile terminal.
  2. The display apparatus according to claim 1, wherein, the processor (170) is further configured to
    control display of the plurality of content items on the display (180) and,
    upon receipt of an input to combine the plurality of content items, combine the plurality of content items and control display of a content list including the combined content items.
  3. The display apparatus according to claim 1, wherein the processor (170) is configured to control playback and display of one of the content items.
  4. The display apparatus according to claim 1, wherein upon receipt of an input to enlarge the content list in a first direction, the processor (170) is configured to control display of the content list to be enlarged in the first direction.
  5. The display apparatus according to claim 3, wherein the processor (170) is configured to control display of the playback content to be enlarged and a volume level for the playback of the content to be increased in response to an input to enlarge the playback content.
  6. The display apparatus according to claim 3, wherein upon receipt of a flicking input in a first direction in respect of the playback content, the processor (170) is configured to control display of images related to the playback content in the first direction.
  7. The display apparatus according to claim 6, wherein upon receipt of an image insert input, the processor (170) is configured to insert an additional image between the images of the playback content, and upon receipt of a flicking input in a second direction opposite to the first direction, the processor is configured to control display of the playback content including the additional image.
  8. The display apparatus according to claim 2, wherein upon receipt of a combine input for a part of the displayed plurality of content items, the processor (170) is configured to generate a content file including content items corresponding to the part of the displayed plurality of content items.
  9. The display apparatus according to claim 1, wherein upon receipt of an input to move the first content to the second user (20), the processor (170) is configured to move the first content, or a part of the first content, in correspondence with the position of the second user (20) and control display of the moved first content or the moved part of the first content on the display (180).
  10. The display apparatus according to claim 1, wherein upon receipt of an input to display a mini map for the first content, the processor (170) is configured to
    control display of the mini map on the display (180) and,
    upon receipt of an input to move a part of the mini map, the processor (170) is configured to move a part of the first content corresponding to the part of the mini map and
    control display of the moved part of the first content.
  11. The display apparatus according to claim 1, further comprising a sensor unit (150) configured to sense a touch made by the first mobile terminal (600a),
    wherein the processor (170) is configured to perform pairing between the display apparatus and the first mobile terminal (600a) and control display of an object related to data sharing with the first mobile terminal (600a) in the display area of the first content,
    wherein, upon receipt of a touch input or a flicking input for the first mobile terminal (600a), the processor (170) is configured to control display of a screen displayed on the first mobile terminal (600a) in the display area of the first content, and
    wherein upon receipt of a touch input or a flicking input in respect of a specific content item in the screen displayed on the first mobile terminal (600a), the processor (170) is configured to control display of the specific content item in the display area of the first content.
  12. The display apparatus according to claim 11, wherein the processor (170) is configured to distinguish a front surface and a second surface of the first mobile terminal (600a), and
    control display of different content corresponding to the front and rear surfaces of the first mobile terminal (600a).
EP15150849.6A 2014-01-13 2015-01-12 Display apparatus Not-in-force EP2894557B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020140004174A KR102219797B1 (en) 2014-01-13 2014-01-13 Display apparatus and method for operating the same
KR1020140004173A KR102219798B1 (en) 2014-01-13 2014-01-13 Display apparatus and method for operating the same

Publications (3)

Publication Number Publication Date
EP2894557A2 EP2894557A2 (en) 2015-07-15
EP2894557A3 EP2894557A3 (en) 2015-10-28
EP2894557B1 true EP2894557B1 (en) 2018-03-07

Family

ID=52465168

Family Applications (1)

Application Number Title Priority Date Filing Date
EP15150849.6A Not-in-force EP2894557B1 (en) 2014-01-13 2015-01-12 Display apparatus

Country Status (3)

Country Link
US (1) US10139990B2 (en)
EP (1) EP2894557B1 (en)
CN (1) CN104777997B (en)

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10086694B2 (en) 2011-09-16 2018-10-02 Gaseous Fuel Systems, Corp. Modification of an industrial vehicle to include a containment area and mounting assembly for an alternate fuel
US9696066B1 (en) 2013-01-21 2017-07-04 Jason E. Green Bi-fuel refrigeration system and method of retrofitting
USD781323S1 (en) * 2013-03-15 2017-03-14 Jason Green Display screen with engine control system graphical user interface
US9845744B2 (en) 2013-07-22 2017-12-19 Gaseous Fuel Systems, Corp. Fuel mixture system and assembly
CN104951057B (en) * 2014-03-28 2019-01-15 联想(北京)有限公司 A kind of information processing method and electronic equipment
US9931929B2 (en) 2014-10-22 2018-04-03 Jason Green Modification of an industrial vehicle to include a hybrid fuel assembly and system
US9885318B2 (en) 2015-01-07 2018-02-06 Jason E Green Mixing assembly
US10306193B2 (en) 2015-04-27 2019-05-28 Microsoft Technology Licensing, Llc Trigger zones for objects in projected surface model
US20160329006A1 (en) * 2015-05-04 2016-11-10 Microsoft Technology Licensing, Llc Interactive integrated display and processing device
CN106933465B (en) * 2015-12-31 2021-01-15 北京三星通信技术研究有限公司 Content display method based on intelligent desktop and intelligent desktop terminal
CN107636696B (en) * 2016-06-16 2021-04-06 深圳市柔宇科技股份有限公司 Multi-user interaction method and device and accompanying robot
US10205906B2 (en) 2016-07-26 2019-02-12 The Directv Group, Inc. Method and apparatus to present multiple audio content
JP6903935B2 (en) * 2017-02-17 2021-07-14 ソニーグループ株式会社 Information processing systems, information processing methods, and programs
JP6763333B2 (en) * 2017-03-30 2020-09-30 富士通株式会社 Information processing system, information processing device and information processing method
US10809962B2 (en) * 2017-11-28 2020-10-20 Dell Products L.P. Orienting content sent to display devices based on a position of a user relative to a computing device
US20190251884A1 (en) * 2018-02-14 2019-08-15 Microsoft Technology Licensing, Llc Shared content display with concurrent views
JP2019144321A (en) * 2018-02-16 2019-08-29 日本電気株式会社 Projection system, projection method, and program
CN108415574B (en) * 2018-03-29 2019-09-20 北京微播视界科技有限公司 Object data acquisition methods, device, readable storage medium storing program for executing and human-computer interaction device
US11462194B1 (en) * 2018-07-30 2022-10-04 Apple Inc. Position sensors for system with overlapped displays
CN111443840A (en) * 2020-03-25 2020-07-24 维沃移动通信有限公司 Display method and electronic equipment
CN112306231B (en) * 2020-09-17 2021-11-30 浙江大学 Method for selecting high-altitude free-hand three-dimensional target based on half-contour highlight

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5515491A (en) * 1992-12-31 1996-05-07 International Business Machines Corporation Method and system for managing communications within a collaborative data processing system
US6262724B1 (en) * 1999-04-15 2001-07-17 Apple Computer, Inc. User interface for presenting media information
US6664786B2 (en) * 2001-07-30 2003-12-16 Rockwell Automation Technologies, Inc. Magnetic field sensor using microelectromechanical system
US20080192059A1 (en) * 2007-02-09 2008-08-14 Microsoft Corporation Multi-user display
US7821510B2 (en) * 2007-04-13 2010-10-26 International Business Machines Corporation Dynamic conference table display system
US9134904B2 (en) * 2007-10-06 2015-09-15 International Business Machines Corporation Displaying documents to a plurality of users of a surface computer
US20090106667A1 (en) * 2007-10-19 2009-04-23 International Business Machines Corporation Dividing a surface of a surface-based computing device into private, user-specific areas
EP2104024B1 (en) * 2008-03-20 2018-05-02 LG Electronics Inc. Portable terminal capable of sensing proximity touch and method for controlling screen using the same
JP5248225B2 (en) * 2008-07-11 2013-07-31 富士フイルム株式会社 Content display device, content display method, and program
US8571192B2 (en) * 2009-06-30 2013-10-29 Alcatel Lucent Method and apparatus for improved matching of auditory space to visual space in video teleconferencing applications using window-based displays
US8386950B2 (en) * 2010-04-05 2013-02-26 Sony Ericsson Mobile Communications Ab Methods, systems and computer program products for arranging a plurality of icons on a touch sensitive display
US20110252349A1 (en) * 2010-04-07 2011-10-13 Imran Chaudhri Device, Method, and Graphical User Interface for Managing Folders
KR101699739B1 (en) * 2010-05-14 2017-01-25 엘지전자 주식회사 Mobile terminal and operating method thereof
JP2011248768A (en) 2010-05-28 2011-12-08 Sony Corp Information processor, information processing system and program
EP2431870B1 (en) * 2010-09-17 2019-11-27 LG Electronics Inc. Mobile terminal and control method thereof
CN102638611B (en) * 2011-02-15 2014-10-22 Lg电子株式会社 Method of transmitting and receiving data and display device using the same
JP6159078B2 (en) 2011-11-28 2017-07-05 京セラ株式会社 Apparatus, method, and program
JP5957892B2 (en) * 2012-01-13 2016-07-27 ソニー株式会社 Information processing apparatus, information processing method, and computer program
US20130219303A1 (en) * 2012-02-21 2013-08-22 Research In Motion Tat Ab Method, apparatus, and system for providing a shared user interface
US20130227476A1 (en) * 2012-02-24 2013-08-29 Nokia Corporation Method, apparatus and computer program product for management of information on a graphic user interface
WO2013151322A1 (en) 2012-04-06 2013-10-10 Samsung Electronics Co., Ltd. Method and device for executing object on display
US9146655B2 (en) * 2012-04-06 2015-09-29 Samsung Electronics Co., Ltd. Method and device for executing object on display
KR101978239B1 (en) 2012-06-22 2019-05-14 삼성전자주식회사 Method for editing contents and an electronic device thereof
US8732617B1 (en) * 2013-01-16 2014-05-20 Google Inc. Systems and methods for enhanced function access in mobile applications

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP2894557A3 (en) 2015-10-28
EP2894557A2 (en) 2015-07-15
US10139990B2 (en) 2018-11-27
US20150199089A1 (en) 2015-07-16
CN104777997B (en) 2019-05-07
CN104777997A (en) 2015-07-15

Similar Documents

Publication Publication Date Title
EP2894557B1 (en) Display apparatus
US10303324B2 (en) Electronic device configured to display three dimensional (3D) virtual space and method of controlling the electronic device
US9317198B2 (en) Multi display device and control method thereof
US10705702B2 (en) Information processing device, information processing method, and computer program
US11188192B2 (en) Information processing device, information processing method, and computer program for side menus
KR20150010432A (en) Display device and controlling method thereof
MX2014001469A (en) Electronic apparatus using motion recognition and method for controlling electronic apparatus thereof.
TWI555390B (en) Method for controlling electronic device and electronic apparatus using the same
TW201447739A (en) Electronic device and data processing method thereof
JP2016511471A (en) Method for controlling display of a plurality of objects by movement-related input to portable terminal and portable terminal
EP2824905B1 (en) Group recording method, machine-readable storage medium, and electronic device
KR20140126129A (en) Apparatus for controlling lock and unlock and method therefor
KR20130052750A (en) Method and apparatus for streaming control of media data
KR102521192B1 (en) Electronic apparatus and operating method thereof
TW201346646A (en) Computing system utilizing three-dimensional manipulation command gestures
KR101861377B1 (en) Method for controlling screen based on motion of mobile terminal and the mobile terminal therefor
EP3175332A1 (en) Interacting with user interface elements representing files
JP2013021480A (en) Information processing equipment and control method thereof, program, and program storage medium readable by computer
KR102255087B1 (en) Electronic device and method for displaying object
KR102219798B1 (en) Display apparatus and method for operating the same
KR102317619B1 (en) Electronic device and Method for controling the electronic device thereof
US20170123637A1 (en) Electronic device and method for managing image
KR20150122976A (en) Display device and method for controlling the same
US10185457B2 (en) Information processing apparatus and a method for controlling the information processing apparatus
EP3139260A1 (en) Electronic device and method for controlling the same

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150112

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 3/0488 20130101AFI20150923BHEP

Ipc: G06F 3/0481 20130101ALI20150923BHEP

Ipc: G09G 5/14 20060101ALI20150923BHEP

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20170223

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20170822

RIN1 Information on inventor provided before grant (corrected)

Inventor name: LEE, HONGJIN

Inventor name: LEE, CHUNGHA

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 977246

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180315

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602015008493

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20180307

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180607

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 977246

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180307

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180608

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180607

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602015008493

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180709

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

26N No opposition filed

Effective date: 20181210

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190112

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190131

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190131

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190112

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20191209

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20191209

Year of fee payment: 6

Ref country code: DE

Payment date: 20191205

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190112

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20150112

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602015008493

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210112

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210112

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210803

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180307