KR20130010424A - Contents play method and apparatus - Google Patents

Abstract

PURPOSE: A content playing method and a device are provided to realistically play content attracting a viewpoint of a user based on a user location. CONSTITUTION: A content playing device determines a first location of a user(1301). The content playing device maps a real space and a content space displayed through a display unit based on the determined first location(1302). The content playing device determines a virtual viewpoint on the content space corresponding to a second location of the user(1305). The content playing device plays content corresponding to the determined virtual viewpoint(1306). [Reference numerals] (1301) Determining a first location of a user; (1302) Mapping a real space and a content space; (1303) Calculating the location variation of the user; (1304) Is the calculated location variation more than a preset minimum location variation(MTT)?; (1305) Determining a viewpoint on the content space corresponding to the changed location of the user; (1306) Playing contents corresponding to the determined viewpoint; (1307) Ending tangible playing; (AA) Start; (BB.DD) No; (CC,EE) Yes; (FF) End

Description

Content playback method and device {CONTENTS PLAY METHOD AND APPARATUS}

The present invention relates to a method and apparatus for playing content, and more particularly, to a method and apparatus for playing content corresponding to a location of a user.

Recently, the demand for 3D stereoscopic images is rapidly increasing, and since digital broadcasting is commercialized, 3D stereoscopic images are actively researched in the fields of 3D (3-Dimensional) TV and 3D information terminal. In general, three-dimensional images are represented by the principle of stereo vision through two eyes. The binocular disparity, which appears as the eyes are about 65 mm apart, is an important factor in depth. The left and right eyes see different stereoscopic images, and when the two stereoscopic images are delivered to the brain through the retina, the brain fuses them to feel the depth of the stereoscopic images.

However, in the case of a 3D TV, only a 3D image having a fixed viewpoint is always displayed regardless of a user's position, and a 3D TV does not provide a realistic image as if it is in a TV.

It is an object of the present invention to provide a method and apparatus for realistically reproducing at least visual stimulating contents of a user's senses in response to a user's location.

In accordance with an aspect of the present invention, there is provided a method of reproducing a content, comprising: determining a first position of a user; Mapping a real space in which the user exists and a content space displayed through the display unit based on the determined first position; Determining a virtual viewpoint in the content space corresponding to the second location of the user; And playing the content corresponding to the determined virtual viewpoint.

In addition, the content reproducing apparatus according to the present invention includes a content collecting unit for collecting the content to stimulate the user's senses; A content processing unit which processes the content input from the content collecting unit to enable playback; A content reproducing unit reproducing the content input from the content processing unit; A sensor unit configured to collect information related to the location of the user so that the content is reproduced corresponding to the location of the user; And a controller configured to determine a virtual viewpoint in a virtual content space corresponding to the location of the user based on the information received from the sensor unit, and control to play the content corresponding to the determined virtual viewpoint. Characterized in that.

As described above, according to the method and apparatus for reproducing content according to the present invention, the present invention has an effect of realistically reproducing content stimulating a user's senses.

1 is a block diagram of a content reproducing apparatus according to an embodiment of the present invention.
2 shows a three-dimensional coordinate system for representing a 3D space according to the present invention.
3A, 3B and 3C are a perspective view, a top view and a side view for explaining the actual space according to the present invention, respectively.
4A and 4B are respectively a perspective view and a top view for explaining a virtual content space according to the present invention.
5 is a block diagram of a sensory playback unit according to an embodiment of the present invention.
6 is a view for explaining a spatial mapping method according to an embodiment of the present invention.
7 is a view for explaining a spatial mapping method according to another embodiment of the present invention.
8 is a diagram for describing a virtual view determination method according to an embodiment of the present invention.
9 is a view for explaining the angle adjustment method of the virtual camera according to an embodiment of the present invention.
10 is a view for explaining a stereoscopic image adjusting method according to an embodiment of the present invention.
11 is a view for explaining a stereo sound control method according to an embodiment of the present invention.
12 is a network diagram of a home network system to which a content player is applied according to an embodiment of the present invention.
13 is a flowchart illustrating a content playback method according to an embodiment of the present invention.
14 is a flowchart illustrating a content playback method according to another embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail a method and apparatus for playing content according to a preferred embodiment of the present invention. However, in describing the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Before describing the present invention, it is to be understood that the terminology used herein is for the purpose of description and should not be interpreted to limit the scope of the present invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense only and not for purposes of limitation, and that various equivalents and modifications may be substituted for them at the time of filing of the present application . Also, in the accompanying drawings, some of the elements are exaggerated, omitted or schematically shown, and the size of each element does not entirely reflect the actual size. Accordingly, the present invention is not limited by the relative size or spacing depicted in the accompanying drawings.

In the following description, content is defined as stimulating a user's senses, ie, sight, hearing, and touch. For example, the content may be an image, light, sound, wind, or the like. In addition, tangible playback may be defined as playing content in response to a user's location. That is, if the location of the user is different, the user's feeling may be different even with the content of the same property. For example, assuming that the content is a car displayed on the screen, it is as if the front of the car or the side of the car is visible depending on the position of the user.

The method and apparatus for reproducing content according to the present invention can be applied to an electronic device having a function of reproducing content stimulating a user's senses. Specifically, the method and device for playing content according to the present invention include a notebook PC, a desktop PC, a tablet PC, a smart phone, a high definition television (HDTV), a smart TV, a 3D (3-Dimensional) TV, an IPTV (Internet Protocol Television), and a stereoscopic device. It can be applied to sound systems, theater systems, home theaters, home network systems, and the like.

The method and apparatus for playing content according to the present invention provide a function of tracking a change in the position of a user and a function of realistically playing the content corresponding to the tracked position of the user. In addition, the content reproduction method and apparatus according to the present invention is a wired LAN (LAN), a wireless LAN, a mobile communication network such as 3G (3-Generation) or 4G (4-Generation), cable, broadcast satellite, broadcasting stations, etc. Through the content provider through the content, that is, the image may be received, and the function of storing the received image in a database provided with the real-time playback can be provided. Here, the image may include a stereoscopic image. Stereoscopic images can also be 3D movies, 3D animations, 3D computer graphics, and the like. Also, the stereoscopic image may be multimedia mixed with stereoscopic sound.

1 is a block diagram of a content reproducing apparatus according to an embodiment of the present invention. The description will be given on the assumption that the content reproducing apparatus according to Fig. 1 is a 3D TV capable of making the content appear to be in a three-dimensional space between the screen and the user. Referring to FIG. 1, the content playback apparatus 100 according to an embodiment of the present invention

Referring to FIG. 1, an apparatus for playing content according to an embodiment of the present invention may include an input unit 110, a remote controller 120, a remote controller receiver 125, a sensor unit 130, a content collector 140, and a content processor ( 150, a sound output unit 161, an image display unit 162, a storage unit 170, an interface unit 180, and a controller 190.

The input unit 110 may include a plurality of input keys and function keys for receiving numeric or character information and setting various functions. The function keys may include a direction key, a side key, and an accelerator key set for performing a specific function. In addition, the input unit 110 generates a key event related to user setting and function control of the content reproducing apparatus 100 and transmits the generated key event to the controller 190. The key event may include a power on / off event, a volume adjustment event, a screen on / off event, and the like. The controller 190 controls the above components in response to the key event.

The remote controller 120 generates various key events for the operation of the content reproducing apparatus 100, converts the generated key events into wireless signals, and transmits the generated key events to the remote controller receiver 125. In particular, in the present invention, the remote controller 120 may generate a start event for requesting realistic playback and an end event for ending the sensory playback. As described above, the tangible reproduction may be defined as playing the content corresponding to the position of the user. The remote control receiver 125 converts the received wireless signal into an original key event and transmits it to the controller 190.

The sensor unit 130 collects information related to the user's location so that the controller 190 can track the user's location, and transmits the collected information to the controller 190. Specifically, the sensor unit 130 may be implemented as an image sensor, an optical sensor that senses light of a specific wavelength such as infrared light. In addition, the sensor unit 130 converts the detected physical quantity into an electrical signal, converts the converted electrical signal back into data (AD), and transmits the converted AD to the controller 190.

The content collector 140 performs a function of collecting content that stimulates a user's senses. In particular, the content collection unit 140 collects images and sounds from a network or an external device. That is, the content collector 140 may include a broadcast receiver 141 and an internet communicator 142. In more detail, the broadcast receiving unit 141 selects one from a plurality of broadcast channels and demodulates the broadcast signal of the selected broadcast channel into original broadcast content. The internet communication unit 142 includes a wired or wireless modem for receiving various main information such as home shopping, home banking, online game, MP3, etc. and additional information thereof provided from the Internet. The internet communication unit 142 may include a mobile communication module (eg, a 3-generation mobile communication module, a 3.5-generation mobile communication module or a 4-generation mobile communication module, etc.) and a short-range communication module. (Eg, a Wi-Fi module).

The content processing unit 150 performs a function of processing the content input from the content collecting unit 140 to be playable under the control of the controller 190. In detail, the content processor 150 classifies the input content into a stereoscopic image and a stereo sound. The content processor 150 decodes the classified stereoscopic sound and outputs it to the audio output unit 161, and decodes the classified stereoscopic image into a left eye image and a right eye image, and outputs it to the image display unit 162. The image processing unit 152 may be included. Also, under the control of the controller 190, the content processor 150 may compress the input content and transmit the compressed content to the controller 190. Accordingly, the controller 190 transmits the compressed content to the storage unit 170. In particular, the sound processor 150 according to the present invention may adjust the sense of direction, distance, etc. of the three-dimensional sound according to the position of the user. In other words, the sound processor 150 may change the type of the sound output from the sound output unit 161 or the volume of the sound according to the type of the sound according to the position of the user. In addition, the image processing unit 160 according to the present invention may adjust the contrast, stereoscopic sense, depth of the stereoscopic image according to the position of the user.

The content reproducing unit 160 performs a function of reproducing the content processed by the content processing unit 150. The content player 160 may include a sound output unit 161 and an image display unit 162. The sound output unit 161 outputs decoded stereoscopic sound, and may include a plurality of speakers, for example, 5.1 channel speakers. The image display unit 162 displays the decoded stereoscopic image. In addition, the image display unit 162 may display the stereoscopic image with a sense of depth as if the stereoscopic image is actually in the three-dimensional space between the user and the screen of the image display unit 162. To this end, the image display unit 162 may include a display unit for displaying a stereoscopic image and a 3D implementation unit for allowing a user to feel the depth of the displayed stereoscopic image. The display unit may be formed of a liquid crystal display (LCD), organic light emitting diodes (OLEDs), active matrix organic light emitting diodes (AMOLEDs), and the like. The 3D realization unit is a component formed in a stacked form with the display unit, and allows different images to be recognized in both left and right sides of the user. As is well known, such a 3D implementation is divided into glasses and glasses-free. The glasses may include a color filter, a polarized filter, and a shutter glass, and the glasses may include a lenticular lens and a parallax barrier. Since the 3D implementation is well known techniques, a detailed description thereof will be omitted.

The storage unit 170 stores a program and data necessary for the operation of the content reproducing apparatus 100. The storage unit 170 may be configured as a volatile storage medium or a nonvolatile storage medium, or may be a combination of both storage media. Volatile storage media include semiconductor memories such as RAM, DRAM, and SRAM, and non-volatile storage media may include a hard disk. In addition, the storage unit 170 may be divided into a data area and a program area. In detail, the data area of the storage unit 170 may store data generated by the controller 160 according to the use of the content reproducing apparatus 100. In addition, the data area may store content compressed in a predetermined format received from the controller 160. The program area of the storage unit 140 may include an operating system (OS) for booting the content reproducing apparatus 100 and operating the above-described components, various user functions, for example, a web browser for accessing an Internet server, Applications for supporting MP3 user functions for reproducing other sound sources, image output functions for reproducing pictures, and video reproducing functions can be stored. In particular, in the present invention, the program area may store a sensory playback program. Such a realistic playback program includes a routine for determining an initial position of a user, a routine for mapping a real space and a content space based on the initial position, a routine for tracking a change in position, and a position of the user. And a routine for determining a virtual viewpoint in the corresponding content space, a routine for reproducing content corresponding to the viewpoint in the content space, and the like. Here, the initial position is defined as a reference value for mapping the real space and the content space. The actual space is a 3D space in which the user and the display unit exist, and the content space is a virtual space in which the content displayed through the display unit exists. In addition, the virtual viewpoint is defined as the user's viewpoint in the content space mapped to the real space.

The interface unit 180 connects the content reproducing apparatus 100 with an external device by wire or wirelessly. The interface unit 180 may include a Zigbee module, a Wi-Fi module or a Bluetooth module. In particular, in the present invention, the interface unit 180 may receive a control signal for immersive reproduction from the controller 190 and transmit it to an external device. That is, the controller 190 may control the external device through the interface unit 180. The external device may be a home network device, a stereo device, a lamp, an air conditioner, a heater, or the like. In other words, the controller 190 may control an external device to play content stimulating a user's senses such as touch, sight, and smell.

The controller 190 may control a general operation of the content reproducing apparatus 100, a signal flow between internal components of the content reproducing apparatus 100, and perform a function of processing data. The controller 190 may control power supply from the battery to the internal components. In addition, the controller 190 may execute various applications stored in the program area. In particular, in the present invention, when the start event for the realistic playback is detected, the controller 190 can execute the above-described realistic playback program. That is, when the sensory playback program is executed, the controller 190 determines the initial position of the user and tracks the change of the position of the user. The controller 190 maps a real space and a content space based on the initial position, determines a virtual viewpoint corresponding to the tracked position, and plays the content corresponding to the virtual viewpoint. To control. In addition, the controller 190 may control the external device through the interface unit 180 to play the content corresponding to the virtual view. The realistic playback function of the control unit 190 will be described in detail below.

2 shows a three-dimensional coordinate system for representing a 3D space according to the present invention. In the present invention, a method of representing 3D space may use a 3D coordinate system, as shown in FIG. 2. In FIG. 2, the solid line represents a positive value, and the dotted line represents a negative value. Also, the coordinates of the user at time t in the real space are represented by (x _{u, t} , y _{u, t} , z _{u, t} ), and the coordinates of the camera at time t in the content space are (x _{c, t} , y _{c, t} , z _{c, t} ).

3A, 3B and 3C are a perspective view, a top view and a side view for explaining the actual space according to the present invention, respectively. Referring to FIG. 3A, the center point 302 of the screen 301 of the display unit in the real space is set to (0,0,0) of the coordinate system. Based on the direction in which the user 303 faces the screen 301, the right direction becomes the positive direction of the X _u axis and the left direction becomes the negative direction of the X _u axis from the center point 302. At the center point 302, the upward direction becomes the positive direction of the Y _u axis and the downward direction becomes the negative direction of the Y _u axis. Further, the direction toward the user 303 in the display 301 is a positive direction of the Z axis _u, the opposite is the negative direction of the Z axis _u. The user's location can be represented by (x _u , y _u , z _u ). 3B and 3C, the horizontal length of the screen 301 is DSW (Display Screen Width), the vertical length of the screen 301 is DSH (Display Screen Height), and the straight line distance between the screen 301 and the user 303 is It can be represented as WD (Watching Distance).

4A and 4B are respectively a perspective view and a top view for explaining a content space according to the present invention. First, in the present invention, the virtual camera is not a real camera but is defined as a user in a content space corresponding to a user in a real space. Referring to FIG. 4A, the focal point 402 of the focal plane 401 is set to (0,0,0) of the coordinate system in the content space. Based on the direction in which the virtual camera 403 faces the focal plane 401, the right direction in the focus 402 becomes the positive direction of the X _c axis and the left direction becomes the negative direction of the X _c axis. At the focal point 402 the upward direction becomes the positive direction of the Y _c axis and the downward direction becomes the negative direction of the Y _c axis. In addition, the direction from the focal plane 401 toward the virtual camera 403 becomes a positive direction of the Z _c axis and vice versa to a negative direction of the Z _c axis. The position of the virtual camera 403 may be represented by (x _c , y _c , z _c ). The horizontal length of the focal plane 401 is the focal width (FW), the vertical length of the focal plane 401 is the focal height (FH), and the linear distance between the focal plane 401 and the virtual camera 403 is the focal length (FL). Can be represented. Here, the FL may be freely set by the user. The size of the focal plane 401 can also be freely set by adjusting the camera angle. That is, since the virtual camera 403 is virtual, the focal length and the camera angle can be freely set.

5 is a block diagram of a sensory playback unit according to an embodiment of the present invention. The sensory playback unit may be configured inside the controller 190 or may be configured separately. Hereinafter, it is assumed that the control unit 190 is configured inside. Referring to FIG. 5, the sensory playback unit 500 according to the present invention, that is, the control unit 190 may include a tracking unit 510, an initialization unit 520, a spatial mapping unit 530, and a virtual viewpoint determination unit 540. And the content processing control unit 550.

The tracking unit 510 tracks the location of the user 303. That is, the tracking unit 510 tracks the coordinates (x _u , y _u , z _u ) of the user 303 using the data received from the sensor unit 130. Specifically, the tracking unit 510 detects feature information of the user 303, for example, a face from the received detection information, and uses the center point of the detected face as coordinates (x _u , y _u , z) of the user 303. _u ) Here, z _u, that is, WD may be calculated using a value such as the size of the detected face.

The initialization unit 520 determines the initial position of the user 303 which is a reference value for mapping the real space and the content space. That is, the initialization unit 520 determines the coordinates input from the tracking unit 510 as the initial position of the user when a start event for immersive reproduction is detected. In detail, the initialization unit 520 may determine the location as the initial location if the location of the user 303 does not change within a preset error range for a predetermined time after viewing of the content starts. In addition, when a specific kit value is input from the remote controller 120, the initialization unit 520 may determine a position where the user is located at the input time as the initial position. In addition, when the start event is input from the tracking unit 510, the initialization unit 520 may determine a position where the user is located at the input time as the initial position. To this end, the tracking unit 510 may detect a specific gesture of the user, for example, raising and releasing a hand, by using a template matching technique. The tracking unit 510 notifies the initialization unit 520 when such a specific gesture is detected.

6 is a view for explaining a spatial mapping method according to an embodiment of the present invention. Equation 1 shows a relative ratio of an actual space coordinate system and a content space coordinate system in the spatial mapping method according to an embodiment of the present invention. In Equation 1 below, t ₀ means a time point determined as an initial position by the initialization unit 520.

As illustrated in FIG. 6, the space mapping unit 530 maps the real space 602 and the content space 603 based on the initial position of the user 601. That is, the spatial mapping unit 530 determines FW, FH, and FL, and then calculates X_Ratio, Y_Ratio, and Z_Ratio (at t ₀ ) as shown in Equation 1 above.

7 is a view for explaining a spatial mapping method according to another embodiment of the present invention. Equation 2 shows a relative ratio of an actual space coordinate system and a content space coordinate system in the spatial mapping method according to another embodiment of the present invention.

DSW, DSH and WD are values due to the size of the display and the actual space. As shown in Equation 2, the spatial mapping unit 530 may expand or reduce the actual space mapped to the content space by adding or subtracting a certain adjustment value to these values. In other words, the spatial mapping unit 530 may adjust the size of the displayed content by using the adjustment value. The spatial mapping unit 530 may receive such an adjustment value at any time, that is, from the remote controller 120 through the remote controller receiver 125 before or during the start of the realistic playback.

8 is a diagram for describing a virtual view determination method according to an embodiment of the present invention. Equation 3 is a calculation formula of this virtual viewpoint determination method.

Referring to FIG. 8, the virtual view determiner 540 receives the coordinates (x _{u , t + 1,} y _{u , t + 1,} z _{u , t + 1} ) of the user 801 from the tracker 510. The coordinate conversion values, i.e., X_Ratio, Y_Ratio, and Z_Ratio are received from the spatial mapping unit 530. As shown in Equation 3, the virtual viewpoint determiner 540 uses the received information, and coordinates (x _{c, t + 1,} y _c ) of the virtual camera 702 mapped to the coordinates of the user 801. _{, t + 1,} z _{c, t + 1} ). The position of the user may be subtly moved even when standing still, and a positional change may occur to some extent due to an error in the sensor and the recognition. Therefore, in consideration of all such position variations, if the displayed content is modified, the user may feel uncomfortable. In the present invention, a minimum position threshold (MTT) for moving the position of the camera may be preset. The minimum position change amount may be an option item that can be set directly by the user. If the virtual viewpoint determining unit 540 is position change amount of the user 801, i.e., △ x _u, △ y _u or △ z _u is greater than the minimum position change amount (MTT), to calculate the coordinate of the virtual camera 802 Can be. Here, the minimum position change amount may be set differently for each of the X, Y, and Z axes. For example, the minimum change amount of position with respect to the Z axis can be set to the largest. For example, the coordinates of the virtual camera 802 may be calculated only when the user wakes up while sitting and watching.

9 is a view for explaining the angle adjustment method of the virtual camera according to an embodiment of the present invention. Referring to FIG. 9, the virtual view determiner 540 calculates an angle change amount θ (θ _X , θ _Y , θ _Z ) of the user 902 based on the center point 901 of the screen. That is, the virtual viewpoint determiner 540 may determine the position change amount Δx _{u, 0} (= x _{u, t} -x _{u, 0} ), Δy _{u, 0} (= y _u, based on the initial position of the user 902 _{. t} -y _{u, 0} ) and Δz _{u, 0} (= z _{u, t} -z _{u, 0} ) are calculated, and the angle change is calculated by applying the calculated position change to the trigonometric function. That is, the calculated angle change amount θ may be used as an angle adjustment value of the virtual camera 903. The calculation of the angle change amount may be an option item that can be set directly by the user.

10 is a view for explaining a stereoscopic image adjusting method according to an embodiment of the present invention.

The content processing controller 550 receives a virtual viewpoint, ie, coordinates (x _{c, t + 1,} y _{c, t + 1,} z _{c, t + 1} ) of the virtual camera, from the virtual viewpoint determiner 540. In addition, the content processing controller 550 may receive an angle adjustment value of the virtual viewpoint, that is, an angle adjustment value θ of the virtual camera, from the virtual viewpoint determination unit 540. In addition, the content processing controller 550 controls the content processor 150 based on the received information to adjust the contrast, the stereoscopic sense, the depth of the stereoscopic image. Referring to FIG. 10, when the initial position 1001 of the user is determined, the content processing controller 550 controls the virtual camera to display an object viewed from the initial position 902. When the user moves from 1001->1003-> 1005, the virtual camera in the content space mapped to the real space is moved from 1002->1004-> 1006. Therefore, the object is displayed different parts according to the positional change of the virtual camera. When the user moves from 1005 to 1007, the camera moves from 1007 to 1008. When the camera is located at 1008, the object is no longer visible because it is outside the angle of the virtual camera. However, the content processing controller 550 may rotate the camera toward the object 1009 by the received angle adjustment value θ so that the object 1009 is continuously displayed.

11 is a view for explaining a stereo sound control method according to an embodiment of the present invention.

The content processing control unit 550 may control the content processing unit 150 to adjust the direction or distance of the stereo sound based on the received information. Referring to FIG. 11, when the user is located at 1101, the content processing control unit 550 may adjust the sense of distance by gradually increasing the sound of the car so that the car 1103 may gradually feel oncoming. In addition, when the user is located at 1102, the content processing control unit 550 may gradually reduce the car sound so that the car 1103 feels gradually away. In addition, the content processing control unit 550 allows the car sound to come from the front speaker when the user is located at 1101, while the car sound is the rear speaker when the user is located at 1102. ), You can control the direction of the stereo sound.

Meanwhile, the content reproducing apparatus 100 according to the present invention may further include components not mentioned above, such as a camera, a microphone, and a GPS receiving module, depending on the form of the provision. These components may not be enumerated because all of them vary according to the convergence trend of digital devices, but the components equivalent to those mentioned above are further included in the content reproducing apparatus 100. Can be configured. In addition, the content reproducing apparatus 100 of the present invention may be excluded from the above configuration or replaced by another configuration, depending on the form of the present invention. Which will be readily apparent to those skilled in the art.

12 is a network diagram of a home network system according to an embodiment of the present invention. Referring to FIG. 12, a home network system according to the present invention may include a content playback device 1200, a home network server 1210, and a plurality of home network devices. The content reproducing apparatus 1200 may include the above-mentioned components. In addition, the content reproducing apparatus 1200 may communicate with the home network server 1210 through a communication method such as ZigBee, Bluetooth, or Wireless LAN. When the content reproducing apparatus 1200 detects a start event for immersive reproduction, the content reproducing apparatus 1200 executes an immersive reproduction program. In other words, the content reproducing apparatus 1200 may control the home network apparatus to reproduce the content corresponding to the location of the user. The home network server 1210 controls home network devices. In addition, the home network server 1210 may drive home network devices under the control of the content playback device 1200. Each home network device has a unique address and is controlled by the home network server 1210 through this. Under the control of the content reproducing apparatus 1200, home network devices play content that stimulates a user's senses. For example, the home network devices may include an air conditioner 1220, a humidifier 1230, a heater 1240, a lamp 1250, and the like, as shown. The content reproducing apparatus 1200 controls the air conditioner 1220, the humidifier 1230, the heater 1240, the lamp 1250, and the like according to the user's position change to adjust the wind intensity, external brightness, temperature, humidity, and the like. Can be. For example, referring to FIG. 11, when the user is located at 1102, the content reproducing apparatus 1200 may control the air conditioner 1220 to increase the wind strength so that the user feels close to the running car 1103. In other words, the user is tactilely stimulated to feel as if he is in a real content space.

13 is a flowchart illustrating a content playback method according to an embodiment of the present invention. 1 to 13, first, the controller 190 may be in a standby state. In this case, the standby state may be defined as a state in which an image is displayed before the realistic playback step. When the user generates a start event by operating the remote controller 120 in this standby state, the controller 190 detects a start event for immersive playback. When the start event is detected as described above, the initialization unit 520 determines the coordinates input from the tracking unit 510 as the initial position of the user in step 1301. Here, the tracking unit 510 may track the location of the user even before the realistic playback step.

In operation 1302, the space mapping unit 530 maps an actual space and a content space based on the determined initial position. In operation 1303, the virtual viewpoint determiner 540 calculates a user's position change amount Δx _u , Δy _{u, and} Δz _u . Next, in operation 1304, the virtual view point determiner 540 compares the calculated position change amount of the user with the minimum position change amount MTT. In the case where the comparison of the user's position change amount with the calculated position change amount is greater than the minimum position change amount, the process proceeds to step 1305. In operation 1305, the virtual view determiner 540 uses the above Equation 3 to determine the position of the virtual camera (x _{c, t + 1,} y _{c, t + 1,} z _{c, t + 1} ), that is, the virtual view. The virtual viewpoint is determined and then transferred to the content processing control unit 550.

In operation 1306, the content processing controller 550 controls the content processor 150 based on the received virtual view. That is, the content processing control unit 550 controls the content processing unit 150 to play the content corresponding to the received virtual viewpoint. In addition, in operation 1306, the content processing controller 550 may control the content processor 150 based on the received virtual viewpoint to adjust a sense of direction or distance of stereoscopic sound. In addition, the content processing controller 550 may control an external device, that is, a home network device, on the basis of the received virtual viewpoint to adjust wind strength, temperature, humidity, or brightness.

Next, in step 1307, the content processing control unit 550 determines whether an end event of the realistic playback is detected. If the end event is detected as a result of the determination in step 1307, the process for immersive reproduction is terminated. On the other hand, if no end event is detected, the process returns to step 1303.

14 is a flowchart illustrating a content playback method according to another embodiment of the present invention. Referring to FIG. 14, a content reproduction method according to another embodiment of the present invention includes steps 1401 to 1407. Steps 1401, 1402, 1404, 1405, and 1407 correspond to steps 1301, 1302, 1304, 1305, and 1307 described above, respectively, and thus description thereof is omitted. In operation 1403, the virtual viewpoint determiner 540 calculates the angle change amount θ (see FIG. 9) together with the position change amount Δx _u , Δy _u, Δz _u of the user. In step 1406, the content processing control unit 550 plays the content corresponding to the received virtual view, rotates the direction of the virtual view by the calculated angle change amount θ, and plays the content corresponding to the rotated virtual view. It's a step.

The content reproduction method according to the present invention as described above is implemented in the form of program instructions that can be executed by various computer means can be recorded on a computer-readable recording medium. In this case, the computer-readable recording medium may include a program command, a data file, a data structure, etc. alone or in combination. On the other hand, the program instructions recorded on the recording medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software. The computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic disks such as floppy disks. -Magneto-Optical Media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. In addition, program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention.

The content reproducing method and apparatus of the present invention are not limited to the above-described embodiments, and various modifications can be made within the scope of the technical idea of the present invention.

100: content playback device
110: input unit 120: remote control
125: remote control receiver 130: sensor
140: content collector 141: broadcast receiver
142: Internet communication unit 150: Content processing unit
151: sound processor 152: image processor
160: sound output unit 170: image display unit
180: storage unit 190: interface unit
195: control unit
500: realistic playback unit
510: tracking unit 520: initialization unit
530: spatial mapping unit 540: virtual viewpoint determination unit
550: content processing control unit
1200: content playback device 1210: home network server
1220: air conditioner 1230: humidifier
1240: radiator 1250: light fixture

Claims (27)

Determining a first location of the user;
Mapping a real space in which the user exists and a content space displayed through the display unit based on the determined first position;
Determining a virtual viewpoint in the content space corresponding to the second location of the user; And
Playing the content corresponding to the determined virtual view. The method of claim 1,
Playing the content,
And displaying an image based on the determined virtual viewpoint. The method of claim 2,
The displaying step,
And displaying the image by adjusting at least one of contrast, depth, and depth of the image based on the determined virtual viewpoint. The method of claim 1,
The step of playing the content is
And outputting sound based on the determined virtual viewpoint. The method of claim 4, wherein
The outputting step,
And outputting the sound by adjusting at least one of a sense of direction and a distance of the sound based on the determined virtual viewpoint. The method of claim 1,
The reproduced content includes a video and a sound stimulating at least visual and hearing, respectively, of the user's senses. The method of claim 1, wherein the determining step,
Calculating a position change amount of the user;
And determining the virtual viewpoint corresponding to the changed position of the user when the calculated position change amount is larger than a set minimum position change amount. 2. The method of claim 1,
Calculating a coordinate conversion value between the coordinate system of the real space and the coordinate system of the content space. The method of claim 8, wherein the determining step,
And determining the virtual viewpoint corresponding to the second position of the user by using the coordinate transformation value. The method of claim 8, wherein the mapping comprises:
Setting a coordinate system of the real space by using a horizontal length DSW of the screen of the display unit, a vertical length DSH of the screen, and a linear distance WD between the screen and the user;
Setting a coordinate system of the content space using a horizontal length FW of the focal plane, a vertical length FH of the focal plane, and a linear distance FL of the focal plane and the virtual viewpoint;
And if the first position is determined, calculating the coordinate transformation value using the first position. The method of claim 10, wherein the setting of the coordinate system of the real space,
Adding or subtracting an adjustment value for reducing or expanding the real space to the virtual space to at least one of the horizontal length DSW, the vertical length DSH, and the linear distance WD. Content playback method characterized in that. The method of claim 1,
Calculating an amount of change in the position of the user based on the first position of the user;
Calculating an angle change amount by applying the calculated position change amount to a trigonometric function;
Rotating the direction of the virtual viewpoint by the calculated angle change amount;
And playing the content corresponding to the virtual viewpoint at which the direction is rotated. The method of claim 1,
Wherein the determining comprises:
Tracking your location,
And determining the fixed position as the first position for the mapping when the position of the user is fixed within a preset error range for a predetermined time as a result of the tracking. The method of claim 1,
Wherein the determining comprises:
And when the start event for immersive playback is detected while tracking the location of the user, determining the tracked location as the first location for the mapping. The method of claim 1,
Wherein the determining comprises:
And when the user's predetermined gesture is detected while tracking the user's location, determining the tracked location as the first location for the mapping. A content collecting unit collecting content stimulating a user's senses;
A content processing unit which processes the content input from the content collecting unit to enable playback;
A content reproducing unit reproducing the content input from the content processing unit;
A sensor unit configured to collect information related to the location of the user so that the content is reproduced corresponding to the location of the user; And
And a controller configured to determine a virtual viewpoint in a virtual content space corresponding to the location of the user based on the information received from the sensor unit, and to control a content corresponding to the determined virtual viewpoint to be played. Content playback device. 17. The method of claim 16,
The control unit,
An initialization unit for determining a first location of the user based on information received from the sensor unit;
A space mapping unit for mapping the real space in which the user exists and the content space based on the determined first position;
A virtual viewpoint determiner configured to determine a virtual viewpoint in the content space corresponding to the second location of the user;
And a content processing control unit which controls the content processing unit to play the content corresponding to the determined virtual viewpoint. The method of claim 17,
The content playback unit,
And an image display unit which displays an image based on the determined virtual viewpoint. The method of claim 18,
The image display unit includes:
And displaying the image by adjusting at least one of contrast, depth and depth of the image based on the determined virtual viewpoint. The method of claim 17,
The content playback unit,
And a sound output unit configured to output a sound based on the determined virtual viewpoint. 21. The method of claim 20,
The sound output unit includes:
And outputting the sound by adjusting at least one of a sense of direction and a distance of the sound based on the determined virtual viewpoint. 17. The method of claim 16,
The content playback unit,
And a video and a sound for stimulating at least visual and hearing, respectively, among the user's senses. The method of claim 17,
The virtual viewpoint determination unit,
Calculate the change in the position of the user based on the first position, calculate the change in the angle by applying the calculated change in the position to the trigonometric function, rotate the direction of the virtual view by the calculated change in the angle, the direction And control to play the content corresponding to the rotated virtual view. The method of claim 17,
The control unit further comprises a tracking unit for tracking the location of the user using the information received from the sensor unit,
And the initialization unit determines the fixed position as the first position for the mapping when the position of the user is fixed within a preset error range for a predetermined time during tracking of the position of the user. The method of claim 17,
The control unit further comprises a tracking unit for tracking the location of the user using the information received from the sensor unit,
And the initialization unit determines the tracked location as the first location for the mapping when a start event for immersive playback is detected during the location tracking of the user. The method of claim 17,
The control unit further comprises a tracking unit for tracking the location of the user using the information received from the sensor unit,
And the initialization unit determines the tracked location as the first location for the mapping when a predetermined gesture of the user is detected during the location tracking of the user. The method of claim 17,
Further comprising an interface unit for connecting to an external device having a content playback function,
And the content processing control unit controls the external device to play the content corresponding to the virtual viewpoint through the interface unit.

Images