Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/10—Geometric effects
  • G06T15/20—Perspective computation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T19/00—Manipulating 3D models or images for computer graphics
  • G06T19/006—Mixed reality
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/20—Image signal generators
  • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
  • H04N13/279—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals the virtual viewpoint locations being selected by the viewers or determined by tracking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/366—Image reproducers using viewer tracking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic

Definitions

• the present invention relates generally to a content playing method and apparatus, and more particularly, to a method for playing content corresponding to a location of a user and an apparatus thereof.
• the demand for Three-Dimensional (3D) image technology has increased and with more common use of digital broadcasting, stereoscopic image use in 3D TV and 3D information terminals is being actively researched.
• the stereoscopic image implemented through 3D technology is formed by a stereo sight principle as experienced through two eyes. Because two eyes are spaced apart from each other by approximately 65 mm, a binocular parallax acts as a main factor of depth.
• the left and right eyes view different stereoscopic images
• the two different stereoscopic images are transferred to a brain through a retina, wherein the brain combines the two different stereoscopic image such that the user experiences the depth of the stereoscopic image.
• a 3D TV is capable of showing a 3D image having a fixed viewpoint regardless of a location of a user, the TV cannot provide a realistic image where the user is present in the TV.
• the present invention has been made to solve the above mentioned problems occurring in the prior art, and the present invention provides a method for playing realistic content stimulating at least one of the senses of a user, corresponding to a location of the user, and an apparatus thereof.
• a content playing method including determining a first location of a user; mapping a content space displayed on a display unit to correspond with an actual space in which the user is present based on the first determined location; determining a virtual viewpoint in the content space corresponding to a second location of the user; and playing content corresponding to the determined virtual viewpoint.
• a content playing apparatus including a content collecting unit for collecting content stimulating senses of a user; a content processor for performing a processing operation such that content input from the content collecting unit are played; a content playing unit for playing content input from the content collecting unit; a sensor for collecting information associated with a location of the user such that the content is played corresponding to the location of the user; and a controller for determining a virtual viewpoint in a virtual content space corresponding to the location of the user based on received information from the sensor and controlling such that content corresponding to the determined virtual viewpoint are played.
• the present invention provides a method for playing realistic content stimulating at least one of the senses of a user, corresponding to a location of the user, and an apparatus thereof.
• FIG. 1 is a block diagram illustrating a configuration of a content playing apparatus according to an embodiment of the present invention
• FIG. 2 is a diagram illustrating a three-dimensional coordinate system according to an embodiment of the present invention.
• FIGS. 3A, 3B, and 3C are diagrams illustrating a perspective view, a plan view, and a side view, respectively, of a actual space according to an embodiment of the present invention
• FIGS. 4A and 4B are diagrams illustrating a perspective view and a plan view, respectively, of a virtual content space according to an embodiment of the present invention
• FIG. 5 is a block diagram illustrating a realistic type playing unit according to an embodiment of the present invention.
• FIG. 6 is a diagram illustrating a space mapping method according to an embodiment of the present invention.
• FIG. 7 is a diagram illustrating a space mapping method according to an embodiment of the present invention.
• FIG. 8 is a diagram illustrating a virtual viewpoint determining method according to an embodiment of the present invention.
• FIG. 9 is a diagram illustrating an angle control method of a virtual camera according to an embodiment of the present invention.
• FIG. 10 is a diagram illustrating a stereoscopic image control method according to an embodiment of the present invention.
• FIG. 11 is a diagram illustrating a stereoscopic sound control method according to an embodiment of the present invention.
• FIG. 12 is a block diagram illustrating a network of a home network system to which a content playing apparatus is applied according to an embodiment of the present invention
• FIG. 13 is a flowchart illustrating a content playing method according to an embodiment of the present invention.
• FIG. 14 is a flowchart illustrating a content playing method according to an embodiment of the present invention.
• the term “content” refers to an operation for stimulating senses of a user, such as a sense of sight, a sense of hearing, and a sense of touch.
• the content may be an image, light, voice, and wind.
• realistic type playback may refer to playing content corresponding to a location of the user. That is, the user may experience the same content differently in different locations of the user. For example, when the content is a car displayed on a screen, a front surface or a side surface of the car is viewed by the user depending on the user location.
• the content playing method and apparatus are applicable to an electronic device having a function playing content stimulating senses of the user.
• the content playing method and apparatus are applicable to a notebook computer, a desktop PC, a tablet PC, a smart phone, a High Definition TeleVision (HDTV), a smart TV, a 3-Dimensional (3D) TV, an Internet Protocol Television (IPTV), a stereoscopic sound system, a theater system, a home theater, a home network system and the like.
• HDMI High Definition TeleVision
• IPTV Internet Protocol Television
• the content playing method and apparatus provide a function for tracking location variation of the user, and a function for realistically playing content corresponding to the tracked location of the user.
• the content playing method and apparatus may provide a function that receives content, such as images from a content provided through Local Area Network (LAN), wireless LAN, 3-Generation (3G) or 4-Generation (4G) wireless communication network, stores and plays a database including the received images in a real-time manner.
• the images may include stereoscopic image.
• the stereoscopic image may become 3D movie, 3D animation, or 3D computer graphics. Further, the stereoscopic image may be a multi-media combined with the stereoscopic sound.
• FIG. 1 is a block diagram illustrating a configuration of a content playing apparatus according to an embodiment of the present invention. It is assumed that the content playing apparatus of FIG. 1 is a 3D TV which enables content to appear to exist in a space between a screen and the user.
• a content playing apparatus 100 according to an embodiment of the present invention includes an input unit 110, a remote controller 120, a remote controller receiver 125, a sensor 130, a content collecting unit 140, a content processor 150, a sound output unit 161, an image display unit 162, a memory 170, an interface unit 180, and a controller 190.
• the input unit 110 may include a plurality of input keys and function keys for receiving input of numeral or character information, and for setting various functions.
• the functions may include arrow keys, side keys, and hot keys set to perform a predetermined function.
• the input unit 110 creates and transfers a key event associated with user setting and function control of the content playing apparatus 100.
• the key event may include a power on/off event, a volume control event, a screen on/off event, etc.
• the controller 190 controls the foregoing elements in response the key event.
• the remote controller 120 creates various key events for operating the content playing apparatus 100, and converts the created key event into a wireless signal, and transmits the wireless signal to the remote controller receiver 125.
• the remote controller 120 of the present invention may create a start event for requesting realistic type playback and a termination event for terminating the realistic type playback.
• the realistic type playback may be defined to play content corresponding to a location of the user.
• the remote controller receiver 125 converts a received wireless signal into an original key event, and transfers the original key event to the controller 190.
• the sensor 130 collects information associated with the location of the user such that the user may track the location of the user, and transfers the collected information to the controller 190.
• the sensor 130 may be implemented by an image sensor or an optical sensor for sensing light of a predetermined wavelength such as infrared ray.
• the sensor 130 converts a sensed physical amount into an electric signal, and an Analog to Digital Converter (ADC) converts the electric signal into data, and transfers the data to the controller 190.
• ADC Analog to Digital Converter
• the content collecting unit 140 performs a function for collecting content stimulating senses of the user. Specifically, the content collecting unit 140 performs a function for collecting images and sounds from a network or a peripheral device. That is, the content collecting unit 140 may include a broadcasting receiver 141 and an Internet communication unit 142. Specifically, the broadcasting receiver 141 selects one from a plurality of broadcasting channels, and demodulates a broadcasting signal of the selected broadcasting channel to original broadcasting content.
• the Internet communication unit 142 includes a wired modem or a wireless modem for receiving various information for home shopping, home banking, and on-line gaming, and MP3 use and additional information with respect thereto.
• the Internet communication unit 142 may include a mobile communication module (e.g., 3G mobile communication module, 3.5G mobile communication module, and 4G mobile communication module) and a near distance communication module (e.g., a Wi-Fi module).
• a mobile communication module e.g., 3G mobile communication module, 3.5G mobile communication module, and
• the content processor 150 performs a processing function to play content from a content collecting unit 140. Specifically, the content processor 150 classifies input content into a stereoscopic image and a stereoscopic sound.
• the content processor 150 may include a sound processor for decoding and outputting the classified stereoscopic sound to a sound output unit 161, and an image processor for decoding the classified stereoscopic sound into a left image and a right image and outputting the left image and the right image to the image processor 152. Further, the content processor 150 may compress and transfer input content to the controller 190 under the control of the controller 190. Accordingly, the controller 190 transfers compressed content to the memory 170.
• the sound processor 150 may control a direction or a distance of a stereoscopic sound according to the location of the user.
• the sound processor 150 may change a type of the sound output from the sound output unit 161 according to a location of the user or change the volume according to a type of the sound.
• the image processor 160 may control brightness, stereoscopic sensation, and depth according to the location of the user.
• the content playing unit 160 performs a function for playing content processed by the content processor 150.
• the content playing unit 160 may include a sound output unit 161 and an image display unit 162.
• the sound output unit 161 outputs a decoded stereoscopic sound, and includes a plurality of speakers, for example, 5.1 channel speakers.
• the image display unit 161 displays a stereoscopic image.
• the image display unit 162 displays a stereoscopic image with depth, as the stereoscopic image actually exists in a three-dimensional space between the image display unit 162 and the screen through a display unit for displaying a stereoscopic image and a 2D implementing unit for allowing a user to experience depth with respect to a displayed stereoscopic image.
• the display unit may be implemented as a Liquid Crystal Display (LCD), Organic Light Emitting Diodes (OLED), or Active Matrix Organic Light Emitting Diodes (AMOLED).
• the 3D implementing unit is a structural element formed in an accumulated with the display unit that makes different images to be recognized at binocular right and left eyes.
• the 3D implementing scheme is divided into a glass scheme and a glass-free scheme.
• the glass scheme includes a color filter scheme, a deflection filter scheme, and a shutter glass scheme.
• the glass-free scheme includes a lenticular lens scheme and a parallax barrier. Because the 3d implementing scheme is known in the art, a detailed description thereof is omitted.
• the memory 170 stores programs and data necessary for an operation of the content playing apparatus 100.
• the memory 170 may be configured by a volatile storage medium, a nonvolatile storage medium, or a combination thereof.
• the volatile storage medium includes a semiconductor memory such as RAM, DRAM, or SRAM.
• the non-volatile storage medium may include a hard disk.
• the memory 170 may be divided into a data area and a program area. Specifically, a data area of the memory 170 may store data created by the controller 160 according to use of the content playing apparatus 100. The data area may store content compressed in a predetermine format provided from the controller 160.
• the program area of the memory 140 may store an operating system (OS) for booting the content playing apparatus 100 and operating respective elements, and applications for supporting various user functions, for example, a web browser for accessing an Internet server, an MP3 user function for playing other sound sources, an image output function for playing photographs, and a moving playback function.
• OS operating system
• the program area of the present invention may store a realistic type playback program.
• the realistic type playing program may include a routine for determining an initial location of a user, a routine for mapping a actual space with a content space based on the initial location, a routine for tracking location variation, a routine for determining a virtual viewpoint in a content space corresponding to the location of the user, and a routine for playing content corresponding to a time point in the content space.
• the initial location is defined as a reference value for mapping a content space to a actual space.
• the actual space is a 3D space in which the user and the display unit are located.
• the content space is a virtual space in which content displayed through a display unit exists.
• the virtual viewpoint is defined as a viewpoint of the user in a content space mapped with a actual space.
• the interface unit 180 performs a function for connecting the content playing apparatus 100 with a peripheral device in a wired or wireless scheme.
• the interface unit 180 may include a Zigbee® module, a Wi-Fi module, or a Bluetooth® module.
• the interface unit 180 may receive and transfer a control signal for realistic type playback from the controller 190 to a peripheral device. That is, the controller 190 may control the peripheral device through the interface unit 180.
• the peripheral device may become a home network device, a stereoscopic sound device, a lamp, an air conditioner, and a heater.
• the controller 190 may control the peripheral device to play content for stimulating senses of the user, for example, a touch sense, a sight sense, and a smell sense.
• the controller 190 may control an overall operation of the content playing apparatus 100, and signal flow between internal structural elements of the content playing apparatus 100. Further, the controller 190 may control power supply to internal elements in a battery. Moreover, the controller 190 may execute various application stored in the program area. Specifically, in the present invention, if a start event for realistic type playback is sensed, the controller 190 may execute the foregoing realistic type playback program. That is, if the realistic type playback program is executed, the controller 190 determines an initial location of the user and tracks location variation of the user. Further, the controller 190 maps a content space with a actual space based on the initial location, determines a virtual viewpoint corresponding to the tracked location, and controls the content processor 150 to play content corresponding to the virtual viewpoint. Furthermore, the controller 190 may control a peripheral device through the interface unit 180 to play content corresponding to the virtual viewpoint. The realistic type playback function of the controller 190 will be described in detail.
• FIG. 2 is a diagram illustrating a three dimensional coordinate system according to an embodiment of the present invention.
• a method for expressing a 3D space according to the present may use a three-dimensional coordinate system.
• a solid line expresses a positive value
• a dotted line expresses a negative value.
• coordinates of the user in a actual space at time t are expressed as (x u ,t , y u,t , z u ,t )
• coordinates of a camera in a content space at time t are expressed as (x c ,t , y c ,t , z c ,t ).
• FIGS. 3A, 3B, and 3C are diagrams illustrating a perspective view, a plan view, and a side view of a actual space according to an embodiment of the present invention.
• a central point of a screen 301 of a display unit in a actual space is set to (0,0,0) of a coordinate system.
• a right direction and a left direction of the central point 302 become a positive direction and a negative direction of an X u axis with reference to a direction that the user views a screen, respectively.
• An upward direction and a downward direction in the central point 302 become a positive direction and a negative direction of an Y u axis.
• a direction to the user 303 on the screen 301 becomes a positive direction of the Z u axis, and an opposite direction thereof becomes a negative direction of the Z u axis.
• the location of the user may be expressed with (x u , y u , z u ).
• a horizontal length of the screen 301 may be expressed with Display Screen Width (DSW)
• a vertical length of the screen 302 be with Display Screen Height (DSH)
• a straight distance between the screen 301 and the user 303 be expressed with Watching Distance (WD).
• FIGS. 4A and 4B are diagrams illustrating a perspective view and a plan view illustrating a virtual content space.
• a virtual camera described herein is not a real camera, but refers to a user in a content space corresponding to that in a actual space.
• a focus 402 on a focus surface 401 in a content space is set to (0,0,0) of a coordinate system.
• a right direction and a left direction of the focus 402 become a positive direction and a negative direction of an X c axis based on a direction that the camera 430 directs the focus surface 401, respectively.
• An upward direction and a downward direction of the focus 402 become a positive direction and a negative direction of a Y c axis, respectively.
• a direction of the focus surface 401 to the virtual camera 403 becomes a positive direction of the Z c axis and an opposite direction thereof becomes a negative direction of the Z c axis.
• the location of the virtual camera 403 may be expressed with (x c , y c , z c ).
• a horizontal length of the focus surface 401 may be expressed with Focal Width (FW), a vertical length of the focus surface 401 be expressed with Focal Height (FH), and a straight distance between the focus surface 401 and the virtual camera 430 be expressed with Focal Length (FL).
• the FL may be set by the user.
• the size of the focus surface 401 may be set by adjusting an angle of the camera. That is, because the virtual camera 403 is virtual, the distance of the focus and the angle of the camera may be set as needed.
• FIG. 5 is a block diagram illustrating a realistic playing unit according to an embodiment of the present invention.
• the realistic type playing unit 500 may be configured inside the controller 190 or be separately configured. It is assumed that the realistic playing unit 500 may be configured inside the controller 190.
• the realistic playing unit 500 of the present invention such as a controller 190 may include a tracker 510, an initiator 520, a space mapper 530, a virtual viewpoint determinator 540, and a content processing controller 550.
• the tracker 510 tracks a location of a user 303. That is, the tracker 510 tracks coordinates (x u , y u , z u ) of the user using data received from the sensor 130.
• the tracker 510 detects characteristic information, for example, a face of the user 303 from received sensing information, and determines a central point of the detected face as coordinates (x u , y u , z u ) of the user 303.
• z u , WD may be computed using the size of the detected face.
• the initiator 520 determines an initial location of the user being a reference value for mapping a content space with a actual space. That is, if a start event for realistic type playback is sensed, the initiator 520 determines coordinates input from the tracker 510 an initial location of the user. Specifically, if the location of the user 303 is not changed within a preset error after watching the content starts, the initiator 520 may determine the location of the user 303 as an initial location. Further, if a predetermined key value is input from the remote controller 120, the initiator 520 may determine a location of the user in the input time of the predetermined key value as the initial location.
• the initiator 520 may determine a location of the user in the input time of the predetermined key value as the initial location. To do this, the tracker 510 may detect a predetermined gesture of the user, for example, action taking a hand down after lifting it using a template matching manner. If the predetermined gesture is detected, the tracker 510 informs the initiator 520 of this.
• a predetermined gesture of the user for example, action taking a hand down after lifting it using a template matching manner. If the predetermined gesture is detected, the tracker 510 informs the initiator 520 of this.
• FIG. 6 is a diagram illustrating a space mapping method according to an embodiment of the present invention.
• a following Equation (1) expresses a relative ratio of a content space coordinate system to a actual space coordinate system in a space mapping method according to an embodiment of the present invention.
• t 0 represents a time point determined as an initial location by the initiator 520.
• the space mapper 530 maps the content space 603 to the actual space 602 based on an initial location of the user 601. That is, the space mapper 530 determines FW, FH, and FL and then computes X_Ratio, Y_Ratio, and Z_Ratio(at t 0 ) as illustrated in the Equation (1).
• FIG. 7 is a diagram illustrating a space mapping method according to an embodiment of the present invention.
• a following Equation (2) expresses a relative ratio of a content space coordinate system to a actual space coordinate system in a space mapping method according to an embodiment of the present invention.
• DSW, DSH, and WD are values depending on the size of the display unit and a actual space.
• the space mapper 530 may add or subtract a predetermine adjustment to or from the foregoing values to extend or shorten a actual space mapped to the content space.
• the space mapper 530 may control the size of displayed content using the adjustment.
• the space mapper 530 may receive the adjustment from the remote controller 120 through the receiver 125 at any time, before start or during start of realistic type playback.
• FIG. 8 is a diagram illustrating a virtual viewpoint determining method according to an embodiment of the present invention.
• a following Equation (3) is a calculation equation of the virtual viewpoint determining method.
• a virtual viewpoint determinator 540 receives coordinates (x u ,t+1 , y u ,t+1 , z u ,t+1 ) of the user 801 from the tracker 510, and receives coordinate transformation values, namely, X_Ratio, Y_Ratio and Z_Ratio from the space mapper 530. As illustrated in the Equation (3), the virtual viewpoint determinator 540 computes coordinates (x c ,t+1 , y c ,t+1 , z c ,t+1 ) of the virtual camera 702 mapped to coordinates of the user 801 using the foregoing received information.
• a Minimum Transition Threshold (MTT) for moving a location of the camera may be previously set in the present invention.
• the MIT may be an option item that the user may directly set.
• the virtual viewpoint determinator 540 may compute coordinates of the virtual camera 802.
• the MTT may be differently set for X, Y, Z axes.
• the MTT for the Z axis may be set to have the greatest value.
• coordinates of the virtual camera 802 may be computed.
• FIG. 9 is a diagram illustrating an angle control method of a virtual camera according to an embodiment of the present invention.
• the virtual viewpoint determinator 540 calculates angle variation amounts ⁇ ( ⁇ X , ⁇ Y , ⁇ Z ) of the user 902 based on a central point 901 of a screen.
• FIG. 10 is a diagram illustrating a stereoscopic image control method according to an embodiment of the present invention.
• a content processing controller 550 receives a virtual viewpoint, namely, coordinates x c ,t+1 , y c ,t+1 , z c ,t+1 of a virtual camera from a virtual viewpoint determinator 540. Further, the content processing controller 550 may receive an angle adjustment of a virtual viewpoint, namely, an angle control value ⁇ of the virtual camera from the virtual viewpoint determinator 540. Moreover, the content processing controller 550 controls the content processor 150 based on received information to adjust brightness, stereoscopic sensation, and depth of a stereoscopic image. Referring to FIG. 10, if an initial location 1001 of a user is determined, the content processing controller 550 perform a control operation to display an object that the virtual camera orients in the initial location 902.
• a location of the user is moved to 1001 -> 1003 -> 1005, a location of a virtual camera in a content space mapped to a actual space to 1002 -> 1004 -> 1006. Accordingly, another part of the object is displayed according to location variation of a virtual camera. If the user moves from 1005 to 1007, the camera moves from 1007 to 1008. If the camera is located at 1008, because an object is separated from an angle of a virtual camera, it is not displayed longer. However, the content processing controller 550 rotates the camera in a direction of the object 1009 by an angle control value ⁇ to continuously display the object 1009.
• FIG. 11 is a diagram illustrating a stereoscopic sound control method according to an embodiment of the present invention.
• the content processing controller 550 controls the content processor 150 to adjust a direction and a distance of a stereoscopic sound.
• the content processing controller 550 may adjust a distance in a manner that a sound of a car is gradually increased such that the user experiences the car approaching.
• the content processing controller 550 may adjust a distance in a manner that a sound of a car is gradually reduced such that the user experiences the car leaving.
• the content processing controller 550 controls a direction of a stereoscopic sound such that a sound of the car is output from a front speaker and a centre speaker.
• the content processing controller 550 controls the direction of a stereoscopic sound such that a sound of the car is output from a rear speaker.
• the foregoing content playing apparatus 100 may further include constructions that are not mentioned such as a camera, a microphone, and a GPS receiving module. Since the structural elements can be variously changed according to convergence trend of a digital device, no elements can be listed. However, the content playing apparatus 100 may include structural elements equivalent to the foregoing structural elements. Further, the content playing apparatus 100 of the present invention may be substituted by specific constructions in the foregoing arrangements according to the provided form or another structure. This can be easily understood to those skilled in the present art.
• FIG. 12 is a block diagram illustrating a network of a home network system to which a content playing apparatus is applied according to an embodiment of the present invention.
• a home network system may include a content playing apparatus 1200, a home network server 1210, and a plurality of home network devices.
• the content playing apparatus may include the foregoing structural elements.
• the content playing apparatus 1200 may communicate with a home network server 1210 in a communication scheme such as a ZigBee scheme, a Bluetooth® scheme, or a Wireless LAN scheme. If a start event for realistic type playback is sensed, the content playing apparatus 1200 executes a realistic type playing program.
• the content playing apparatus 1200 may control the home network device to play the content corresponding to a location of the user.
• the home network server 1210 controls home network devices.
• the home network server 1210 may drive home network devices under the control of the content playing apparatus 1200.
• the home network devices have unique addresses, respectively, and are controlled through the addresses by the home network server 1210.
• the home network device plays content stimulating senses of the user under the control of the content playing apparatus 1200.
• the home network devices may include an air conditioner 1220, a humidifier 1230, a heater 1240, and a lamp 1250.
• the content playing apparatus 1200 may control the air conditioner 1220, the humidifier 1230, the heater 1240, and the lamp according to location variation of the user to adjust an intensity of wind, peripheral brightness, temperature, and humidity. For example, referring to FIG. 11, when the user is located at 1102, the content playing apparatus 1200 may increase the intensity of wind such as the user experiences the car being near the user.
• the user may be stimulated with a touch sense, and experience to be present in a real content space.
• FIG. 13 is a flowchart illustrating a content playing method according to an embodiment of the present invention.
• a controller 190 may firstly be in an idle state.
• the idle state may be defined as a state displaying an image before a realistic type playback step. If the user operates a remote controller 120 in the idle state, the controller 190 senses a start event for realistic playback. As illustrated previously, if a start event is sensed, an initiator 520 determines coordinates input from a tracker 510 as an initial location of a user in step 1301. The tracker 510 may track a location of the user before a realistic type playing step.
• a space mapper 530 maps a content space to a actual space based on the determined initial location in step 1302.
• a virtual viewpoint determinator 540 computes location variation amounts ( ⁇ x u , ⁇ y u , ⁇ z u ) of the user in step 1303.
• the virtual viewpoint determinator 540 compares the computed location variation amounts of the user with an MTT in step 1304. As a comparison result of step 1304, when the computed location variation amounts of the user is greater than the MTT, the process proceeds to step 1305.
• the virtual viewpoint determinator 540 determines and transfers locations x c ,t+1 , y c,t+1 , and z c ,t+1 of a virtual camera, namely, a virtual viewpoint to the content processing controller 550 using the Equation (3) in step 1305.
• the content processing controller 550 controls the content processor 150 based on the received virtual viewpoint in step 1306. That is, the content processing controller 550 controls the content processor 150 to play content corresponding to the received virtual viewpoint. Further, the content processing controller 550 controls the content processor 150 based on the received virtual viewpoint to adjust a direction or a distance of a stereoscopic sound in step 1306.
• the content processing controller 550 may control a peripheral device, such as home network devices based on the virtual viewpoint to adjust an intensity of wind, temperature, humidity, or brightness.
• the content processing controller 550 determines whether a termination event of realistic type playback is sensed in step 1307. If the termination event of realistic type playback is sensed at step 1307, a process for realistic playback is terminated. Conversely, if the termination event is not sensed, the process returns to step 1303.
• FIG. 14 is a flowchart illustrating a content playing method according to another embodiment of the present invention.
• a content playing method according to another embodiment of the present invention may include steps 1401 to 1407. Because steps 1401, 1402, 1404, 1405, and 1407 correlate with the forgoing steps 1301, 1302, 1304, 1305, and 1307, a description thereof will be omitted.
• Step 1403 is a step in which a virtual viewpoint determinator 540 calculates an angle variation amount ⁇ of FIG. 9 together with location variation amounts ⁇ x u , ⁇ y u , and ⁇ z u of the user.
• the content processing controller 550 plays content corresponding to a received virtual viewpoint, rotates a direction of the virtual viewpoint by the computed angel variation amount ⁇ , and plays content corresponding to the rotated virtual viewpoint.
• a method for providing a user interface in a portable terminal according to an embodiment of the present invention as described above may be implemented in an executable program command form by various computer means and be recorded in a computer readable recording medium.
• the computer readable recording medium may include a program command, a data file, and a data structure individually or a combination thereof.
• the program command recorded in a recording medium may be specially designed or configured for the present invention or be known to a person having ordinary skill in a computer software field to be used.
• the computer readable recording medium includes Magnetic Media such as hard disk, floppy disk, or magnetic tape, Optical Media such as Compact Disc Read Only Memory (CD-ROM) or Digital Versatile Disc (DVD), Magneto-Optical Media such as floptical disk, and a hardware device such as ROM.
• RAM random access memory
• flash memory storing and executing program commands.
• the program command includes a machine language code created by a complier and a high-level language code executable by a computer using an interpreter.
• the foregoing hardware device may be configured to be operated as at least one software module to perform an operation of the present invention, and a reverse operation thereof is the same.
• a content playing method and an apparatus thereof according to the present invention have an effect that they may realistically play content stimulating a sense of the user.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Signal Processing (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Computer Graphics (AREA)
• Geometry (AREA)
• Computer Hardware Design (AREA)
• Computing Systems (AREA)
• Software Systems (AREA)
• Human Computer Interaction (AREA)
• Acoustics & Sound (AREA)
• Processing Or Creating Images (AREA)
• User Interface Of Digital Computer (AREA)

Applications Claiming Priority (3)

Publications (2)

Family

ID=47839676

Family Applications (1)

Country Status (5)

Families Citing this family (19)

Citations (4)

Family Cites Families (13)

• 2011
  • 2011-11-07 KR KR1020110114883A patent/KR101926477B1/ko active IP Right Grant
• 2012
  • 2012-01-17 WO PCT/KR2012/000375 patent/WO2013012146A1/en active Application Filing
  • 2012-01-17 CN CN201280035942.2A patent/CN103703772A/zh active Pending
  • 2012-01-17 EP EP12814483.9A patent/EP2735164A4/de not_active Withdrawn
  • 2012-01-23 US US13/356,220 patent/US20130023342A1/en not_active Abandoned

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events