KR101926477B1 - Contents play method and apparatus - Google Patents

Abstract

The present invention is intended to provide a content playback method and apparatus that realistically reproduce a content that stimulates the user's sense. The method and apparatus for playing back content according to the present invention can be applied to a notebook PC, a desktop PC, a tablet PC, a smart phone, a high definition television (HDTV), a smart TV, a 3D (3-Dimensional) TV, Television, a stereo system, a theater system, a home theater, a home network system, and the like. In order to realistically reproduce contents, a method for reproducing contents according to the present invention includes: determining a first position of a user; Mapping the actual space in which the user exists and a content space displayed through the display unit based on the determined first position; Determining a virtual viewpoint on the content space corresponding to a second location of the user; And reproducing the content corresponding to the determined virtual time point.

Description

[0001] CONTENTS PLAY METHOD AND APPARATUS [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a content playback method and apparatus, and more particularly, to a method and apparatus for playing content in response to a user's location.

Recently, demand for three-dimensional (3D) stereoscopic images has increased rapidly, and since digital broadcasting has been commercialized, stereoscopic images have been actively studied in the fields of 3D (3-Dimensional) TV and 3D information terminal. Generally, stereoscopic images expressed in three dimensions are made by the principle of stereo vision through two eyes. The binocular parallax caused by the distance of about 65mm between the two eyes is an important factor of depth. The left and right eyes see different stereoscopic images, and when two stereoscopic images are transmitted to the brain through the retina, the brain fuses them to feel the depth of the stereoscopic image.

However, in the case of the 3D TV, the 3D image with the fixed viewpoint is always displayed irrespective of the user's position, and the reality is not provided as if it is in the TV.

An object of the present invention is to provide a method and apparatus for realistically reproducing contents that stimulate at least the time of the user's sense corresponding to the position of the user.

According to another aspect of the present invention, there is provided a method of reproducing a content, including: determining a first position of a user; Mapping the actual space in which the user exists and a content space displayed through the display unit based on the determined first position; Determining a virtual viewpoint on the content space corresponding to a second location of the user; And reproducing the content corresponding to the determined virtual time point.

According to another aspect of the present invention, there is provided a content reproducing apparatus including a content collecting unit for collecting a content for stimulating a user's senses; A content processing unit that processes the content input from the content collecting unit so as to be reproducible; A content reproduction unit for reproducing the content input from the content processing unit; A sensor unit for collecting information related to the position of the user so that the content is reproduced corresponding to the position of the user; And a control unit for determining a virtual viewpoint on a virtual content space corresponding to the location of the user based on the information received from the sensor unit and controlling the content corresponding to the determined virtual time point to be reproduced .

As described above, according to the method and apparatus for reproducing contents according to the present invention, it is possible to realistically reproduce contents that stimulate the user's sense.

1 is a block diagram of a contents reproducing apparatus according to an embodiment of the present invention.
FIG. 2 shows a three-dimensional coordinate system for representing a 3D space according to the present invention.
3A, 3B and 3C are a perspective view, a top view, and a side view, respectively, for explaining an actual space according to the present invention.
4A and 4B are a perspective view and a top view, respectively, for explaining a virtual content space according to the present invention.
5 is a block diagram of a real-time playback unit according to an embodiment of the present invention.
6 is a diagram for explaining a spatial mapping method according to an embodiment of the present invention.
7 is a view for explaining a spatial mapping method according to another embodiment of the present invention.
8 is a view for explaining a method for determining a virtual viewpoint according to an embodiment of the present invention.
9 is a diagram for explaining an angle adjusting method of a virtual camera according to an embodiment of the present invention.
10 is a view for explaining a stereoscopic image adjusting method according to an embodiment of the present invention.
11 is a view for explaining a stereo sound adjusting method according to an embodiment of the present invention.
12 is a network configuration diagram of a home network system to which a content reproducing apparatus according to an embodiment of the present invention is applied.
13 is a flowchart illustrating a content playback method according to an embodiment of the present invention.
14 is a flowchart for explaining a content playback method according to another embodiment of the present invention.

Hereinafter, a content reproducing method and apparatus according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Before describing the present invention, it is to be understood that the terminology used herein is for the purpose of description and should not be interpreted to limit the scope of the present invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense only and not for purposes of limitation, and that various equivalents and modifications may be substituted for them at the time of filing of the present application . Also, in the accompanying drawings, some of the elements are exaggerated, omitted or schematically shown, and the size of each element does not entirely reflect the actual size. Accordingly, the present invention is not limited by the relative size or spacing depicted in the accompanying drawings.

In the following description, content is defined as stimulating a user's sense, namely, visual, auditory, tactile, and the like. For example, the content can be video, light, sound, wind, and the like. Further, realistic playback can be defined as playback of the content corresponding to the position of the user. That is, if the user's position is different, the user's feeling may be different even if the content of the same attribute. For example, assuming that the content is a car displayed on the screen, the front of the car can be seen or the side of the car can be seen depending on the location of the user.

The content reproducing method and apparatus according to the present invention can be applied to an electronic apparatus having a function of reproducing a content that stimulates the user's sense. More specifically, the method and apparatus for reproducing contents according to the present invention can be applied to various applications such as a notebook PC, a desktop PC, a tablet PC, a smart phone, an HDTV (High Definition TeleVision), a smart TV, a 3D (3-Dimensional) TV, A sound system, a theater system, a home theater, a home network system, and the like.

The content reproducing method and apparatus according to the present invention provides a function of tracking a change in a user's position and a function of realistically reproducing a content in response to the position of the tracked user. The contents reproducing method and apparatus according to the present invention can also be applied to a mobile communication network such as a wired LAN (Local Area Network), a wireless LAN, a 3-Generation (3G) A function of receiving a content, that is, a video from a content provider, storing the received video in a database and real-time playback of the received video. Here, the image may include a stereoscopic image. In addition, stereoscopic images can be 3D movies, 3D animations, 3D computer graphics, and the like. In addition, the stereoscopic image may be multimedia mixed with stereo sound.

1 is a block diagram of a contents reproducing apparatus according to an embodiment of the present invention. The content reproduction apparatus according to FIG. 1 is assumed to be a 3D TV that can make a content appear as if it is in a three-dimensional space between a screen and a user. Referring to FIG. 1, a content reproduction apparatus 100 according to an embodiment of the present invention includes:

1, a content reproducing apparatus according to an exemplary embodiment of the present invention includes an input unit 110, a remote controller 120, a remote controller receiver 125, a sensor unit 130, a content collecting unit 140, An audio output unit 161, an image display unit 162, a storage unit 170, an interface unit 180, and a control unit 190.

The input unit 110 may include a plurality of input keys and function keys for receiving numeric or character information and setting various functions. The function keys may include a direction key, a side key, and an accelerator key set for performing a specific function. Also, the input unit 110 generates a key event related to the user setting and the function control of the content reproduction apparatus 100, and transmits the key event to the control unit 190. The key event may include a power on / off event, a volume adjustment event, a screen on / off event, and the like. The control unit 190 controls the above-described configurations in response to these key events.

The remote controller 120 generates various key events for the operation of the content reproduction apparatus 100, converts the generated key events into radio signals, and transmits the radio signals to the remote controller receiver 125. Particularly, in the present invention, the remote controller 120 can generate a start event for requesting real-time playback and a termination event for ending real-time playback. Here, realistic playback can be defined as reproducing the content corresponding to the position of the user, as described above. The remote control receiver 125 converts the received radio signal into an original key event and transmits the original key event to the controller 190.

The sensor unit 130 collects information related to the user location so that the controller 190 can track the location of the user and transmits the collected information to the controller 190. [ Specifically, the sensor unit 130 may be implemented as an image sensor, an optical sensor that detects light of a specific wavelength such as infrared rays, or the like. Also, the sensor unit 130 converts the sensed physical quantity into an electrical signal, converts the converted electrical signal into data analog to digital (AD), and transmits the data to the control unit 190.

The content collecting unit 140 performs a function of collecting contents that stimulate the user's senses. In particular, the content collecting unit 140 performs a function of collecting images and sounds from a network or an external device. That is, the content collecting unit 140 may include a broadcast receiving unit 141 and an Internet communication unit 142. Specifically, the broadcast receiving unit 141 selects one of the plurality of broadcast channels and demodulates the broadcast signal of the selected broadcast channel into the original broadcast content. The Internet communication unit 142 includes a wired or wireless modem for receiving various kinds of main information such as home shopping, home banking, online games, MP3, and additional information provided from the Internet. The Internet communication unit 142 may be configured to communicate with a mobile communication module (e.g., a 3-Generation mobile communication module, a 3.5-Generation mobile communication module or a 4-Generation mobile communication module) (E.g., a Wi-Fi module).

The content processing unit 150 performs a function of processing the content input from the content collecting unit 140 so as to be reproducible under the control of the control unit 190. [ Specifically, the content processing unit 150 classifies the input content into a stereoscopic image and a stereoscopic image. The content processing unit 150 decodes the classified stereoscopic sound and outputs the decoded stereoscopic sound to the sound output unit 161. The stereoscopic image decoding unit 150 decodes the stereoscopic image into a left eye image and a right eye image and outputs the decoded stereoscopic image to the image display unit 162 And an image processing unit 152 for performing image processing. In addition, under the control of the control unit 190, the content processing unit 150 may compress the input content and transmit the compressed content to the control unit 190. Accordingly, the control unit 190 delivers the compressed content to the storage unit 170. In particular, the sound processor 150 according to the present invention can adjust a directional feeling, a distance, and the like of the stereophonic sound according to the position of the user. In other words, the sound processing unit 150 can change the type of sound output from the sound output unit 161 according to the user's position, or can change the sound volume according to the type of sound. In addition, the image processing unit 160 according to the present invention can adjust the contrast, depth, and depth of the stereoscopic image according to the user's position.

The content reproducing unit 160 performs a function of reproducing the content processed by the content processing unit 150. [ The content reproducing unit 160 may include an audio output unit 161 and a video display unit 162. The sound output unit 161 outputs decoded stereo sound, and may include a plurality of speakers, for example, a 5.1-channel speaker. The image display unit 162 displays the decoded stereoscopic image. Also, the image display unit 162 can display the stereoscopic image with a depth sense as if the stereoscopic image is actually present in a three-dimensional space between the user and the screen of the image display unit 162. [ For this purpose, the image display unit 162 may include a display unit for displaying a stereoscopic image and a 3D implementing unit for allowing the user to feel a depth feeling on the displayed stereoscopic image. The display unit may be formed of a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AMOLED), or the like. The 3D implementation unit is a component formed in a laminated form with the display unit, and allows different images to be recognized in the left and right sides of the user. As is well known, such a 3D implementation method is classified into a spectacle method and a non-spectacle method. The spectacle method includes a color filter method, a polarizing filter method, a shutter glass method, and the non-spectacle method includes a lenticular lens method and a parallax barrier method. These 3D implementations are well known in the art, so a detailed description thereof will be omitted.

The storage unit 170 stores programs and data necessary for the operation of the content reproduction apparatus 100. The storage unit 170 may be a volatile storage medium or a nonvolatile storage medium and may be a combination of both storage media. The volatile storage medium includes a semiconductor memory such as a RAM, a DRAM, and a SRAM, and a nonvolatile storage medium may include a hard disk. The storage unit 170 may be divided into a data area and a program area. Specifically, the data area of the storage unit 170 may store data generated by the control unit 160 in accordance with the use of the content reproduction apparatus 100. In addition, the data area can store content compressed in a predetermined format, which is received from the control unit 160. [ The program area of the storage unit 140 may include an operating system (OS) for operating the respective components of the content playback apparatus 100, a variety of user functions such as a web browser for accessing an Internet server, An MP3 user function for playing back other sound sources, an image output function for playing back pictures and the like, and a video playback function. Particularly, in the present invention, the program area can store the real-time playback program. The real-time playback program includes a routine for determining a user's initial position, a routine for mapping an actual space and a content space based on an initial position, a routine for tracking a position change, A routine for determining a virtual viewpoint in the corresponding content space, and a routine for reproducing the content corresponding to the viewpoint in the content space. The initial position is defined as a reference value for mapping the actual space and the content space. The actual space is a 3D space in which the user and the display unit exist, and the content space is a virtual space in which the content displayed through the display unit exists. Also, a virtual viewpoint is defined as a viewpoint of a user on a content space mapped to an actual space.

The interface unit 180 functions to connect the content reproduction apparatus 100 to an external device by wire or wirelessly. The interface unit 180 may include a Zigbee module, a Wi-Fi module, or a Bluetooth module. In particular, in the present invention, the interface unit 180 receives a control signal for real-time playback from the controller 190 and can transmit the control signal to an external device. That is, the control unit 190 can control an external device through the interface unit 180. Here, the external device may be a home network device, a stereophonic device, a lamp, an air conditioner, a radiator, or the like. In other words, the control unit 190 may control an external device to reproduce a content that stimulates a sense of the user, such as a tactile sense, a sight, a smell, and the like.

The control unit 190 can control the overall operation of the content reproduction apparatus 100 and the signal flow between the internal configurations of the content reproduction apparatus 100 and perform the function of processing the data. And the controller 190 may control the power supply from the battery to the internal configurations. In addition, the control unit 190 can execute various applications stored in the program area. Particularly, in the present invention, the control unit 190 can execute the real-time playback program described above when a start event for real-time playback is detected. That is, when the real-time playback program is executed, the controller 190 determines the initial position of the user and tracks the position change of the user. The control unit 190 maps the actual space and the content space based on the initial position, determines a virtual viewpoint corresponding to the tracked position, and controls the content processing unit 150 to reproduce the content corresponding to the virtual time point. . Also, the control unit 190 may control the external device through the interface unit 180 to reproduce the content corresponding to the virtual time point. The real-time playback function of the controller 190 will be described in detail below.

FIG. 2 shows a three-dimensional coordinate system for representing a 3D space according to the present invention. In the present invention, as shown in FIG. 2, a three-dimensional coordinate system can be used as a method of representing 3D space. In Fig. 2, the solid line represents a positive value, and the dotted line represents a negative value. In addition, in a real space the user coordinate of when the time t is expressed as (x _{u, t,} y _{u, t,} z _{u, t),} the coordinates of the camera at time t in the content space, (x _{c, t} , yc _{, t} , zc _{, t} ).

3A, 3B and 3C are a perspective view, a top view, and a side view, respectively, for explaining an actual space according to the present invention. 3A, the center point 302 of the screen 301 of the display unit in the actual space is set to (0, 0, 0) in the coordinate system. The right direction of the center point 302 becomes the positive direction of the X _u axis and the left direction becomes the negative direction of the X _u axis with reference to the direction in which the user 303 looks at the screen 301. The upward direction at the center point 302 is the positive direction of the Y _u axis and the downward direction is the negative direction of the Y _u axis. In addition, the direction from the screen 301 to the user 303 is the positive direction of the Z _u axis and vice versa is the negative direction of the Z _u axis. The user's position can be expressed as (x _u , y _u , z _u ). 3B and 3C, the horizontal length of the screen 301 is DSW (Display Screen Width), the vertical length of the screen 301 is DSH (Display Screen Height), the straight line distance between the screen 301 and the user 303 WD (Watching Distance).

4A and 4B are a perspective view and a top view, respectively, for explaining a content space according to the present invention. In the present invention, the virtual camera is not an actual camera but is defined as a user on a content space corresponding to a user on the actual space. Referring to FIG. 4A, the focus 402 of the focal plane 401 in the content space is set to (0, 0, 0) in the coordinate system. The right direction in the focus 402 becomes the positive direction of the X _c axis and the left direction becomes the negative direction of the X _c axis with respect to the direction in which the virtual camera 403 looks at the focal plane 401. The upward direction in the focus 402 is the positive direction of the Y _c axis and the downward direction is the negative direction of the Y _c axis. In addition, the direction from the focal plane 401 toward the virtual camera 403 is the positive direction of the Z _c axis and vice versa is the negative direction of the Z _c axis. The position of the virtual camera 403 can be expressed as (x _c , y _c , z _c ). The horizontal distance of the focal plane 401 is FW (Focal Width), the vertical length of the focal plane 401 is FH (Focal Height), and the linear distance between the focal plane 401 and the virtual camera 403 is FL . Here, FL can be freely set by the user. The size of the focal plane 401 can also be freely set by adjusting the camera angle. That is, since the virtual camera 403 is virtual, the focal distance and the camera angle can be freely set.

5 is a block diagram of a real-time playback unit according to an embodiment of the present invention. The sensory reproduction unit may be configured inside the control unit 190 or separately. Hereinafter, it is assumed that the control unit 190 is configured. 5, the sensory reproduction unit 500 according to the present invention includes a tracking unit 510, an initialization unit 520, a spatial mapping unit 530, a virtual time point determination unit 540, And a content processing control unit 550. [

The tracking unit 510 tracks the location of the user 303. That is, the tracking unit 510 tracks the coordinates (x _u , y _u , z _u ) of the user 303 using the data received from the sensor unit 130. Specifically, the tracking unit 510 detects the feature information, for example, a face of the user 303 from the received detection information, and obtains the center point of the detected face from the coordinate (x _u , y _u , z _u ). Here, z _u, that is, WD can be calculated using a value such as the size of the detected face.

The initialization unit 520 determines an initial position of the user 303 as a reference value for mapping the actual space and the content space. That is, the initialization unit 520 determines the coordinates input from the tracking unit 510 as a user's initial position when a start event for real-time playback is detected. Specifically, if the position of the user 303 does not change within a predetermined error range for a predetermined time after the start of viewing of the content, the initialization unit 520 can determine the position as the initial position. In addition, the initialization unit 520 can determine the position where the user is located at the input point as the initial position when the specific kit value is input from the remote controller 120. [ In addition, the initialization unit 520 can determine the place where the user is located as the initial position when the start event is input from the tracking unit 510. For this purpose, the tracking unit 510 can detect a specific gesture of the user, for example, a hand-lifting operation using a template matching technique or the like. The tracking unit 510 informs the initialization unit 520 when the specific gesture is detected.

6 is a diagram for explaining a spatial mapping method according to an embodiment of the present invention. Equation (1) represents a relative ratio between an actual spatial coordinate system and a contents spatial coordinate system in the spatial mapping method according to an embodiment of the present invention. In the following Equation (1), t ₀ denotes a time point at which the initial position is determined by the initialization unit 520.

The spatial mapping unit 530 maps the actual space 602 and the content space 603 based on the initial position of the user 601, as shown in FIG. That is, the space mapping unit 530 determines FW, FH, and FL, and then calculates X_Ratio, Y_Ratio, and Z_Ratio (at t ₀ ) as shown in Equation 1 above.

7 is a view for explaining a spatial mapping method according to another embodiment of the present invention. Equation (2) shows a relative ratio between the actual spatial coordinate system and the contents spatial coordinate system in the spatial mapping method according to another embodiment of the present invention.

DSW, DSH, and WD are values due to the size of the display unit and the actual space. The spatial mapping unit 530 may expand or contract the actual space mapped to the content space by adding or subtracting a constant adjustment value to these values, as shown in Equation (2). In other words, the spatial mapping unit 530 can adjust the size of the displayed content using this adjustment value. The spatial mapping unit 530 can receive the adjustment value from the remote controller 120 through the remote control receiving unit 125 at any time, that is, before or at the start of the sensory reproduction.

8 is a view for explaining a method for determining a virtual viewpoint according to an embodiment of the present invention. Equation (3) is a calculation formula of this virtual time point determination method.

8, the virtual time point determination unit 540 receives the coordinates (x _{u , t + 1,} y _{u , t + 1,} z _{u , t + 1} ) of the user 801 from the tracking unit 510 And receives coordinate transformation values (i.e., X_Ratio, Y_Ratio, and Z_Ratio) from the spatial mapping unit 530. The virtual time point determination unit 540 determines the coordinates of the virtual camera 702 mapped to the coordinates of the user 801 using the thus received information as shown in Equation (3) (x _{c, t + 1,} y _{c , t + 1,} zc _{, t + 1} ). The position of the user may be slightly moved even if the user is still standing, and the positional change may occur to some extent due to the error of the sensor and the recognition. Therefore, if the content to be displayed is modified in consideration of all such position variations, the user may feel inconvenience. In the present invention, a minimum transition threshold (MTT) for moving the position of the camera can be set in advance. Such a minimum position change amount may be an option item that can be directly set by the user. If the virtual viewpoint determining unit 540 is position change amount of the user 801, i.e., △ x _u, △ y _u or △ z _u is greater than the minimum position change amount (MTT), to calculate the coordinate of the virtual camera 802 . Here, the minimum position change amount may be set differently for the X, Y, and Z axes. For example, the minimum position change amount with respect to the Z-axis can be set to the greatest. For example, the coordinates of the virtual camera 802 can be calculated only when the user is sitting and watching.

9 is a diagram for explaining an angle adjusting method of a virtual camera according to an embodiment of the present invention. 9, the virtual viewpoint determining unit 540 calculates the angular change amount? (? _X ,? _Y ,? _Z ) of the user 902 on the basis of the center point 901 of the screen. That is, the virtual viewpoint determining unit 540 is the user 902, the initial position to a position change amount based on △ x _{u, 0} a _{(= x u, t - x} u, 0), △ y u, 0 (= y u, _t - y _{u, 0),} △ z _{u, 0 (u} = _{z, t} - calculating a z _{u, 0),} and applied to each of the trigonometric function thus calculated position variation amount calculates the angle change amount. That is, the calculated angle change amount [theta] can be used as the angle adjustment value of the virtual camera 903. [ The calculation of the angular change amount may be an option that can be directly set by the user.

10 is a view for explaining a stereoscopic image adjusting method according to an embodiment of the present invention.

The contents processing control unit 550 receives the virtual point of view, that is, the coordinates ( _{xc, t + 1,} yc _{, t + 1,} zc _{, t + 1} ) of the virtual camera from the virtual point of view decision unit 540. In addition, the content processing control unit 550 may receive the angle adjustment value of the virtual viewpoint, that is, the angle adjustment value [theta] of the virtual camera, from the virtual viewpoint determining unit 540. [ In addition, the content processing control unit 550 controls the content processing unit 150 based on the received information to control the contrast, depth, and depth of the stereoscopic image. Referring to FIG. 10, when the user's initial position 1001 is determined, the content processing control unit 550 controls the virtual camera to display an object viewed from the initial position 902. When the user moves from 1001 to 1003 to 1005, the virtual camera on the content space mapped to the actual space moves to 1002 -> 1004 -> 1006. Therefore, the object is displayed in a different position according to the positional variation of the virtual camera. When the user moves from 1005 to 1007, the camera moves from 1007 to 1008. When the camera is positioned at 1008, the object is no longer displayed because it is out of the angle of the virtual camera. However, the content processing control unit 550 may rotate the camera toward the object 1009 by the received angle adjustment value (?) So that the object 1009 is continuously displayed.

11 is a view for explaining a stereo sound adjusting method according to an embodiment of the present invention.

The content processing control unit 550 controls the content processing unit 150 based on the received information to control the directional feeling or the distance feeling of the stereophonic sound. Referring to FIG. 11, when the user is located at 1101, the contents processing control unit 550 can adjust the distance feeling by gradually increasing the sound of the automobile so that the automobile 1103 is gradually approaching. In addition, when the user is located at 1102, the contents processing control unit 550 can gradually reduce the automobile sound so that the automobile 1103 feels increasingly distant. When the user is located at 1101, the content processing control unit 550 causes the car sound to be output from the front speaker (Front Speak and Center Speaker). On the other hand, when the user is located at 1102, So that the directional feeling of the stereophonic sound can be controlled.

Meanwhile, the content reproducing apparatus 100 according to the present invention may further include configurations not mentioned above such as a camera, a microphone, and a GPS receiving module depending on the providing mode. These components can not be enumerated in all of the variations according to the convergence trend of the digital device, but the component having the same level of the above-mentioned components is further included in the content reproducing apparatus 100 . It is needless to say that the content reproducing apparatus 100 according to the present invention may be excluded from the specific configurations in the above configuration or may be replaced with other configurations in accordance with the provision mode thereof. Which will be readily apparent to those skilled in the art.

12 is a network configuration diagram of a home network system according to an embodiment of the present invention. Referring to FIG. 12, the home network system according to the present invention may include a content reproducing apparatus 1200, a home network server 1210, and a plurality of home network devices. The content reproduction apparatus 1200 may include the above-mentioned configurations. The content reproduction apparatus 1200 can communicate with the home network server 1210 through a communication method such as ZigBee, Bluetooth, or wireless LAN. When the content playback apparatus 1200 detects a start event for real-time playback, the real-time playback program is executed. That is, the content reproduction apparatus 1200 can control the home network apparatus to reproduce the content corresponding to the position of the user. The home network server 1210 controls the home network devices. Also, the home network server 1210 can drive the home network devices under the control of the content reproducing apparatus 1200. [ Each of the home network devices has its own address and is controlled by the home network server 1210 through it. The home network devices reproduce contents that stimulate the user's sense under the control of the content reproduction apparatus 1200. For example, the home network devices may include an air conditioner 1220, a humidifier 1230, a heater 1240 and a lamp 1250 as shown. The content reproducing apparatus 1200 controls the air conditioner 1220, the humidifier 1230, the heater 1240 and the lamp 1250 in accordance with the change of the user's position to adjust the wind intensity, the external brightness, the temperature, . For example, referring to FIG. 11, when the user is located at 1102, the content reproducing apparatus 1200 may control the air conditioner 1220 so that the user feels close to the car 1103 running. That is, the user is tactile and receives a feeling of being in the actual content space.

13 is a flowchart illustrating a content playback method according to an embodiment of the present invention. 1 to 13, the controller 190 may be in a standby state. Here, the standby state can be defined as a state in which an image is displayed before the realistic playback step. When the user operates the remote controller 120 to generate a start event in this standby state, the controller 190 senses a start event for real-time playback. If the start event is detected in step 1301, the initialization unit 520 determines the coordinates input from the tracking unit 510 as the initial position of the user. Here, the tracking unit 510 may track the position of the user even before the realistic playback step.

In step 1302, the spatial mapping unit 530 maps the actual space and the content space based on the determined initial position. In step 1303, the virtual time point determining unit 540 calculates the position change amount (? X _u ,? Y _u ,? Z _u ) of the user. Next, in step 1304, the virtual time point determining unit 540 compares the calculated position change amount of the user with the minimum position change amount MTT. If the calculated position variation of the user is greater than the minimum position variation as a result of the comparison at step 1304, the process proceeds to step 1305. In step 1305, the virtual-point-of-time determining unit 540 determines the virtual camera position ( _{xc, t + 1,} yc _{, t + 1,} zc _{, t + 1} ) (virtual viewpoint), and transmits the virtual viewpoint to the content processing control unit 550. [

In step 1306, the content processing control unit 550 controls the content processing unit 150 based on the received virtual viewpoint. That is, the content processing control unit 550 controls the content processing unit 150 to reproduce the content corresponding to the received virtual time point. Also, in step 1306, the content processing control unit 550 may control the content processing unit 150 based on the received virtual viewpoint to adjust the directional feeling, the distance feeling, and the like of the stereophonic sound. Also, the content processing control unit 550 can control the intensity, the temperature, the humidity, the brightness, or the like of the wind by controlling the external devices, that is, the home network devices, based on the received virtual viewpoint.

Next, in step 1307, the content processing control unit 550 determines whether or not the end event of the real-time playback is detected. If the end event is detected as a result of the determination in step 1307, the process for real-time playback ends. On the other hand, if a termination event is not detected, the process returns to step 1303.

14 is a flowchart for explaining a content playback method according to another embodiment of the present invention. Referring to FIG. 14, the content playback method according to another embodiment of the present invention includes steps 1401 to 1407. Steps 1401, 1402, 1404, 1405, and 1407 correspond to steps 1301, 1302, 1304, 1305, and 1307 described above, respectively. Step 1403 is a step in which the virtual time point determining unit 540 calculates the angle change amount? (See FIG. 9) together with the position change amounts? X _u ,? Y _{u, and} ? Z _u of the user. In step 1406, the content processing control unit 550 causes the content corresponding to the received virtual time point to be reproduced, rotates the direction of the virtual time point by the calculated angle variation amount?, And reproduces the content corresponding to the rotated virtual time point .

The above-described contents reproducing method according to the present invention can be implemented in the form of a program command which can be executed through various computer means and recorded in a computer-readable recording medium. At this time, the computer readable recording medium may include a program command, a data file, a data structure, or the like, alone or in combination. On the other hand, the program instructions recorded on the recording medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software. The computer-readable recording medium includes magnetic media such as a hard disk, a floppy disk and a magnetic tape, an optical medium such as a CD-ROM and a DVD, and a magnetic recording medium such as a floppy disk. Optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The program instructions also include machine language code, such as those generated by the compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention.

The contents reproducing method and apparatus of the present invention are not limited to the above-described embodiments but can be variously modified and carried out within the scope of the technical idea of the present invention.

100: Content playback device
110: input unit 120: remote control
125: Remote control receiving unit 130: Sensor unit
140: Content collecting unit 141: Broadcast receiving unit
142: Internet communication unit 150: Content processing unit
151: Acoustic processor 152:
160: sound output unit 170: image display unit
180: storage unit 190: interface unit
195:
500: real-
510: Trace unit 520: Initialization unit
530: Space mapping unit 540: Virtual point determining unit
550: Content processing control section
1200: Content playback device 1210: Home network server
1220: Air conditioner 1230: Humidifier
1240: Radiator 1250: Light

Claims (27)

A content reproducing method of an electronic device,
Determining a first position of the user;
Calculating a coordinate transformation value between a coordinate system of an actual space in which the user exists and a coordinate system of a contents space displayed through the display unit based on the determined first position;
Determining a virtual viewpoint on the content space corresponding to a second location of the user using the coordinate transformation value; And
And reproducing the content corresponding to the determined virtual time point. The method according to claim 1,
The method of claim 1,
And displaying an image based on the determined virtual viewpoint. 3. The method of claim 2,
Wherein the displaying comprises:
Wherein the image is displayed by adjusting at least one of brightness, depth, and depth of the image based on the determined virtual viewpoint. The method according to claim 1,
The step of reproducing the content includes:
And outputting a sound based on the determined virtual time point. 5. The method of claim 4,
Wherein the outputting step comprises:
Wherein the sound is output by adjusting at least one of a directional sense and a distance sensation of the sound based on the determined virtual viewpoint. The method according to claim 1,
Wherein the content to be reproduced includes an image and a sound that respectively stimulate at least the time and the auditory sense among the senses of the user. 2. The method of claim 1, wherein determining the virtual time point comprises:
Calculating a position change amount of the user;
And determining the virtual time point corresponding to the changed position of the user if the calculated position variation amount is larger than the set minimum position variation amount. delete delete 2. The method according to claim 1,
Setting a coordinate system of the actual space using a horizontal length (DSW) of the screen of the display unit, a vertical length (DSH) of the screen and a linear distance (WD) between the screen and the user,
Setting a coordinate system of the contents space using a horizontal length (FW) of the focal plane, a vertical length (FH) of the focal plane, and a straight line distance (FL) between the focal plane and the virtual viewpoint,
And when the first position is determined, calculating the coordinate conversion value using the first position. 11. The method of claim 10, wherein the step of setting the coordinate system of the real space comprises:
And adding or subtracting an adjustment value for mapping the actual space to the virtual space in at least one of the horizontal length (DSW), the vertical length (DSH), and the linear distance (WD) And reproducing the content. The method according to claim 1,
Calculating a position change amount of the user based on the first position of the user;
Calculating the amount of angular change by applying the calculated position variation amount to a trigonometric function,
Rotating the direction of the virtual viewpoint by the calculated angle change amount;
And reproducing the content corresponding to the virtual time point at which the direction is rotated. The method according to claim 1,
Wherein the determining comprises:
Tracking the location of the user,
And determining the fixed position as the first position for the calculation if the position of the user is fixed within a predetermined error range for a predetermined time after the tracking result. The method according to claim 1,
Wherein the determining comprises:
Wherein when the start event for realistic playback is detected while tracking the position of the user, the tracked position is determined as the first position for the calculation. The method according to claim 1,
Wherein the determining comprises:
When the predetermined gesture of the user is detected during the tracking of the position of the user, the tracked position is determined as the first position for the calculation. A contents collecting unit for collecting contents that stimulate the user's senses;
A content processing unit that processes the content input from the content collecting unit so as to be reproducible;
A content reproduction unit for reproducing the content input from the content processing unit;
A sensor unit for collecting information related to the position of the user so that the content is reproduced corresponding to the position of the user; And
Determining a virtual viewpoint in a virtual content space corresponding to the location of the user based on the information received from the sensor unit and controlling the content corresponding to the determined virtual time point to be reproduced Content playback apparatus. 17. The method of claim 16,
Wherein,
An initialization unit for determining a first position of the user based on information received from the sensor unit;
A space mapping unit for mapping the actual space in which the user is present and the content space based on the determined first position,
A virtual viewpoint determining unit for determining a virtual viewpoint on the content space corresponding to the second location of the user;
And a content processing control unit for controlling the content processing unit to reproduce the content corresponding to the determined virtual time point. 18. The method of claim 17,
Wherein,
And a video display unit for displaying an image on the basis of the virtual time point determined by the time determination unit. 19. The method of claim 18,
The image display unit includes:
And displays the image by adjusting at least one of brightness, depth, and depth of the image based on the determined virtual viewpoint. 18. The method of claim 17,
Wherein,
And an audio output unit for outputting sound based on the determined virtual time point. 21. The method of claim 20,
The sound output unit includes:
Wherein the control unit controls at least one of a directional feeling and a sense of distance of the sound based on the determined virtual time point to output the sound. 17. The method of claim 16,
Wherein,
And reproduces an image and a sound that respectively stimulate at least the visual sense and the auditory sense from among the senses of the user. 18. The method of claim 17,
The virtual-
Calculates a position change amount of the user based on the first position, calculates an angle change amount by applying the calculated position change amount to a trigonometric function, rotates the direction of the virtual viewpoint by the calculated angle change amount, And controls to reproduce the content corresponding to the rotated virtual time point. 18. The method of claim 17,
The control unit may further include a tracking unit for tracking the position of the user using the information received from the sensor unit,
Wherein the initialization unit determines the fixed position as the first position for mapping when the position of the user is fixed within a predetermined error range for a predetermined time during tracking of the position of the user. 18. The method of claim 17,
The control unit may further include a tracking unit for tracking the position of the user using the information received from the sensor unit,
Wherein the initialization unit determines the tracked position as a first position for the mapping when a start event for real-time playback is detected during the tracking of the position of the user. 18. The method of claim 17,
The control unit may further include a tracking unit for tracking the position of the user using the information received from the sensor unit,
Wherein the initialization unit determines the tracked position as the first position for the mapping when a predetermined gesture of the user is detected during the tracking of the position of the user. 18. The method of claim 17,
Further comprising an interface unit for connecting to an external device having a content playback function,
Wherein the content processing control unit controls the external device through the interface unit to reproduce the content corresponding to the virtual time point.

Images