KR20110115981A - Apparatus and method for playing three-dimension image data in portable terminal - Google Patents

Abstract

The present invention relates to an apparatus and method for stereoscopic image processing, and more particularly, to an apparatus and method for enabling a user to watch a seamless stereoscopic image at various angles by detecting a user's position in playing and processing a stereoscopic image in a portable terminal. An apparatus for reproducing stereoscopic image data in a portable terminal includes a control unit for processing a 3D image to be reproduced, a tracking unit to track a user's position when the 3D image is reproduced, and the user to move out of the viewing range by the tracking unit. In this case, it is to solve the problem that the stereoscopic effect is reversed even if the user is out of the viewing angle range, including an image processing unit for reconstructing the reproduced image.

Description

Apparatus and method for reproducing stereoscopic image data in a portable terminal {APPARATUS AND METHOD FOR PLAYING THREE-DIMENSION IMAGE DATA IN PORTABLE TERMINAL}

The present invention relates to an apparatus and method for stereoscopic image processing, and more particularly, to an apparatus and method for enabling a user to watch a seamless stereoscopic image even at various angles by detecting a user's position in playing and processing a stereoscopic image in a portable terminal. That is, the present invention is to solve the problem that the stereoscopic effect is reversed even if the user viewing the stereoscopic image of the portable terminal using the barrier LCD out of the viewing angle range.

Recently, imaging techniques are being actively researched for a method of implementing 3D images. This is to express more realistic and realistic image information. Using the human visual characteristics, the left view image and the right view image are respectively scanned at the corresponding positions on the existing display device, and the left view point and the right view point are separated from the user's left and right eyes to form a three-dimensional image. How to do this is recognized in many ways. For example, portable terminals (stereophones, stereoscopic cameras, stereoscopic camcorders, etc.) and 3D TVs equipped with Barrier LCDs can play stereoscopic content to provide users with more realistic images.

In general, stereoscopic images are taken using two camera sensors spaced apart by a specific distance, unlike conventional images, and then used by attaching two images. That is, a stereoscopic image is made by synthesizing the viewpoints seen from the right eye and the left eye of the user. These two images may be arranged in the longitudinal or transverse direction.

Current stereo image output methods are largely divided into glasses and glasses-free. The first method has a small viewing angle and a large stereo effect, and is mainly used for large output devices such as TVs. The second method uses the Barrier LCD, which is suitable for portable terminals without glasses, but has a large viewing angle.

In other words, the barrier LCD method used in the portable terminal does not use glasses, but has a disadvantage of narrow viewing angle, which is observed as a 3D image from the front of the LCD. As the image is reversed, the stereoscopic effect is reversed, which may cause dizziness in the user.

In order to solve the above problems, there is a need for an apparatus and method for providing a stereoscopic effect to a user outside the viewing angle in a portable terminal.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for reproducing stereoscopic image data in a portable terminal.

Another object of the present invention is to provide an apparatus and method for improving stereoscopic image output performance in a portable terminal.

Another object of the present invention is to provide an apparatus and method for confirming a viewer's viewpoint movement in a portable terminal and changing a pixel position of a stereoscopic image according to the observer's viewpoint movement.

Still another object of the present invention is to provide an apparatus and a method for stopping a stereoscopic effect when a viewer observes an observer outside a viewing angle range in a portable terminal.

According to a first aspect of the present invention for achieving the above object, an apparatus for reproducing a stereoscopic image data in a portable terminal comprises a control unit for processing to reproduce the 3D image, and tracking the position of the user during the 3D image playback And a tracking unit and an image processor for reconstructing the reproduced image when the tracking unit determines that the user is out of the viewing range.

According to a second aspect of the present invention for achieving the above object, a method for reproducing a stereoscopic image data in a portable terminal, the process of playing a 3D image, the process of tracking the location of the user during the 3D image playback; If the user is outside the viewing range, the method may include reconstructing the reproduced image.

According to a third aspect of the present invention for achieving the above objects, a portable terminal for reproducing a stereoscopic image data includes a display unit capable of outputting a 3D image or a 2D image, a memory unit storing a 3D image to be reproduced, and a viewer. A tracking unit for determining a viewing angle of the imager, an image processing unit for changing a pixel position of the 3D image data or providing a 2D effect when the viewer is out of the viewing angle, and reconstructing an image reproduced according to the viewing angle of the viewer It characterized in that it comprises a control unit for controlling, characterized in that it comprises a posture check unit for changing the output direction of the image reproduced according to the angle of the portable terminal.

The present invention can solve the problem that the stereoscopic effect is reversed when viewing outside the viewing angle range by changing the pixel position of the stereoscopic image according to the viewer's viewpoint shift in the portable terminal supporting the stereoscopic image output of the auto glasses type.

1 is a block diagram showing the configuration of a portable terminal providing a stereo effect according to the present invention;
2 is a flowchart illustrating a process of playing a 3D image in a portable terminal according to an embodiment of the present invention;
3 (a) is a diagram illustrating a problem that occurs when playing a 3D image in a typical portable terminal,
3 (b) is a diagram illustrating a process of playing a 3D image in a portable terminal according to an embodiment of the present invention;
3 (c) is a diagram illustrating a process of playing a 3D image in a portable terminal according to another embodiment of the present invention;
4 is a diagram illustrating a process of switching a screen when playing a 3D image in a portable terminal according to an exemplary embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

In the following description, a stereoscopic effect is reversed when viewing outside the viewing angle range by changing a pixel position of a stereoscopic image according to a viewer's viewpoint shift in a portable terminal supporting an autostereoscopic stereo image output according to the present invention. An apparatus and method for doing this will be described. In addition, the portable terminal is a portable terminal capable of outputting image data, and may be a portable multimedia player (PMP) using a barrier LCD system, a mobile communication terminal, a smartphone, a laptop, a 3D TV, and the like.

1 is a block diagram showing the configuration of a portable terminal providing a stereoscopic effect according to the present invention.

Referring to FIG. 1, the portable terminal includes a control unit 100, an attitude check unit 102, a tracking unit 104, an image processing unit 106, a memory unit 108, an input unit 110, and a display unit 112. And a communication unit 114.

The controller 100 of the portable terminal controls the overall operation of the portable terminal. For example, processing and control for voice call and data communication are performed, and according to the present invention, in addition to the normal function, the control unit 100 uses a user tracking function to determine a viewing angle corresponding to a 3D image viewing range. Process to determine if a deviation has occurred. In addition, the controller 100 processes to prevent the stereoscopic effect from being reversed when it is confirmed that the user leaves the viewing angle range.

In this case, when the controller 100 determines that the viewing angle range is deviated by tracking the user's viewpoint, the controller 100 reconstructs the input image so that the left and right sides of the image input through the barrier LCD are changed so as not to enter the eye. That is, the controller 100 may reconstruct the input image by changing positions of left and right pixels of the input image. In addition, the controller 100 processes the stereoscopic effect only when the user exists within the viewing angle range.

The posture confirming unit 102 includes a sensor for detecting the movement of the portable terminal, and detects the rotation of the portable terminal to change the direction of the output image. That is, the posture check unit 102 may rotate the output image horizontally or vertically when detecting the rotation of the portable terminal.

The tracking unit 104 is for determining the location of the user and tracks the user using a camera, an infrared sensor, or the like. The location of the user estimated by the tracking unit 104 is used as information for determining whether the image processing unit is operated, and the tracking unit 104 is used to prevent the image processing unit 106 from being frequently operated. The threshold value is set for the deviation of the viewing angle range. For this reason, the tracking unit 104 determines that the viewing angle range has been deviated when measuring the displacement of the threshold value or more.

In addition, after setting the timer, the tracking unit 104 determines the deviation of the viewing angle range at a predetermined time interval to prevent the determination of the deviation of the viewing angle range against the deviation caused by the user's irregular viewing posture.

The image processing unit 106 processes to reconstruct the input image of the image in order to solve the problem that the stereoscopic effect is reversed when the user views an image reproduced from a position outside the viewing angle range. That is, the image processor 106 may prevent the inversion of the stereoscopic effect by changing the pixel order of the image from left to right according to the position of the user determined by the tracking unit 104. In addition, when the user is located within the viewing angle range, the image processor 106 selects only one pixel among the left and right pixels to reconstruct the 2D image to stop the stereoscopic effect.

In addition, an image of the reproduced image may be stored in an output buffer (not shown) of the memory unit 108 or the display unit 112, and the image processing unit 106 may include the memory unit 108 or the The image of the image stored in the output buffer of the display unit 112 may be reconstructed and stored.

The memory unit 108 may include a read only memory (ROM), a random access memory (RAM), and a flash ROM. The ROM stores microcode and various reference data of a program for processing and controlling the control unit 100, the attitude check unit 102, the tracking unit 104, and the image processing unit 106.

The RAM is a working memory of the controller 100, and stores temporary data generated during execution of various programs. In addition, the flash ROM stores various updatable storage data such as a phone book, an outgoing message, and an incoming message.

The input unit 110 includes 0 to 9 numeric key buttons, a menu button (menu), a cancel button (clear), an OK button, a call button (TALK), an end button (END), an internet access button, a navigation key ( Or a plurality of function keys such as arrow keys) buttons and character input keys, and provides key input data corresponding to a key pressed by the user to the controller 100.

The display unit 112 displays status information, texts, a large amount of moving images, and still images generated during the operation of the portable terminal. The display unit 112 may use a color liquid crystal display (LCD), and the display unit 112 may include a touch input device to be used as an input device when applied to a portable terminal of a touch input method. According to the present invention it is possible to use a barrier LCD method.

The communication unit 114 performs a function of transmitting and receiving a radio signal of data input and output through an antenna (not shown). For example, in the case of transmission, after performing channel coding and spreading on the data to be transmitted, RF processing is performed to transmit the data. In the case of reception, the received RF signal is converted into a baseband signal. The baseband signal is de-spreaded and channel decoded to restore data.

The posture confirming unit 102, the tracking unit 104, and the image processing unit 106 may play the role of the control unit 100 of the portable terminal. It is an exemplary configuration for the purpose of never limiting the scope of the invention, and those skilled in the art will recognize that various modifications are possible within the scope of the invention. For example, the controller 100 may be configured to process all of them.

2 is a flowchart illustrating a process of playing a 3D image in a portable terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the portable terminal first checks whether a 3D image is played back in step 201. Herein, the portable terminal can play a 3D image by combining images taken from different positions of the same subject, and in addition, the portable terminal can use the Barrier LCD method to obtain a 3D effect without using glasses. have.

If the 3D image is not played back in step 201, the portable terminal proceeds to step 217 to perform a corresponding function (eg, standby mode).

In the case where the 3D image is played back in step 201, the portable terminal proceeds to step 203 to operate the tracking unit, and then proceeds to step 205 to determine the user's location. Here, the tracking unit tracks the viewpoint of an observer who views the image data reproduced in the portable terminal, and may include a sensor (eg, an acceleration sensor, an infrared sensor, etc.) and a camera module. First, the tracking unit may determine the observer's viewpoint by measuring the degree of inclination of the portable terminal through the sensor unit or by measuring infrared rays reflected from the user's face. In this case, the tracking unit may recognize a space using infrared rays by using the technology of the present application (“space input device for mobile phone” (application number: P2009-0038854)). In addition, the tracking unit detects the location of the user through the pupil tracking using the camera module to perform a viewpoint determination process.

In step 207, the portable terminal determines whether the user has left the viewing range. Here, the portable terminal provides the advantage that the user does not have to wear glasses by using the barrier LCD method, but the disadvantage that the viewing angle of the user is narrowed. Accordingly, the portable terminal determines whether to view the image data being played back in the viewing range outside the fixed viewing angle.

If it is determined in step 207 that the user does not deviate from the viewing range, the portable terminal proceeds to step 209 to identify a playback method operating outside the viewing range. Here, the playback method that operates outside the viewing range includes a method of continuously playing the video data in a 3D method outside the viewing range and the viewing range, and a method of playing 2D images outside the viewing range, which can be selected by the user as an option. have.

If it is set as the method to reproduce in the 3D method in step 209, the portable terminal proceeds to step 211 to process to change the left and right pixels of the image.

When the user is located within the viewing range, the left pixel of the image reproduced by the portable terminal is input to the user's left eye through the barrier LCD, and the right pixel is input to the user's right eye. However, when the user is located outside the viewing range, the left pixel of the image is input to the user's right eye through the barrier LCD, and the right pixel is input to the user's left eye to feel visual fatigue.

Therefore, in order to solve the above problems, when the user is out of the viewing range, the portable terminal according to the present invention processes to reconstruct the input image of the image so that the left and right of the image coming into the eye through the barrier LCD is changed. That is, the portable terminal processes to change the pixel order of the video image from left to right.

As a result, the left pixel is normally input to the left eye of the user outside the viewing range, and the right pixel is input to the right eye.

In step 213, the portable terminal processes the 3D image to reproduce the input image of the reconstructed image by inputting the reconstructed image accurately into the user's eyes.

On the other hand, if it is set as a method to play in the 2D method in step 209, the portable terminal proceeds to step 219 to process the 3D video to be reproduced as a 2D video. For example, the portable terminal prevents a pixel of an image composed of a left pixel and a right pixel from providing a 3D effect. That is, the portable terminal may process only the left pixel or the right pixel of the image to be input to the user's eyes.

As described above, in step 213 or 219, the portable terminal reproduces the image data, and proceeds to step 215 to confirm whether the reproduction of the 3D image is terminated.

If it is confirmed in step 215 that the 3D video playback is not terminated, the portable terminal performs the process of step 205 to determine the location of the user.

On the other hand, if it is confirmed in step 215 that the 3D video playback is finished, the portable terminal ends the present algorithm.

The portable terminal sets a timer to prevent frequent reconstruction of image data by a user located at each boundary of the field of view, and tracks the user's position at a predetermined time interval or checks a displacement above a threshold to determine whether the field of view is out of range. You can judge.

3 is a diagram illustrating a 3D video reproduction process of a portable terminal according to an exemplary embodiment of the present invention.

3A is a diagram illustrating a problem that occurs when playing a 3D image in a general portable terminal.

Referring to FIG. 3 (a), the portable terminal uses a Barrier LCD method as described above, and the user 300 can obtain a 3D effect without using glasses, but the user's viewing angle is narrowed. This happens.

That is, the portable terminal reproduces an image combining images captured at different viewpoints in order to provide a 3D effect. The image is composed of a left pixel input to the user's left eye and a right pixel input to the right eye, and the pixels pass through the transparent area of the barrier LCD 301 and input to both eyes of the user 300. Will be.

In summary, the left pixel is input through the transparent area of the barrier LCD 301 to the left eye and the right pixel is input to the right eye so that the user 300 can feel a stereo effect without using stereo glasses. .

However, the user 300 of the portable terminal may receive a left pixel as a left eye and a right pixel as a right eye within a predetermined viewing angle range. The stereoscopic effect is reversed.

3 (b) is a diagram illustrating a process of playing 3D video in a portable terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 3 (b), the portable terminal processes to reconstruct the input image of the image such that the left and right sides of the image coming into the eye through the barrier LCD are changed in order to solve the problem as shown in FIG. 3 (a).

That is, when the portable terminal checks the position of the user (face position, eye position, etc.) and checks the viewing at a position outside the viewing angle range (305), the portable terminal changes the position of the left pixel and the position of the right pixel barrier. The left and right sides of the image input through the LCD are changed. Here, changing the position of the left pixel and the position of the right pixel refers to moving the left pixel to the position of the right pixel and moving the position of the right pixel to the left (309).

While the portable terminal as described above prevents the stereoscopic effect from being controlled by adjusting the position of the left and right pixels of the image, the portable terminal according to the present invention may prevent the stereoscopic effect from being prevented by adjusting the position of the upper and lower pixels. .

3 (c) is a diagram illustrating a process of playing a 3D image in a portable terminal according to another exemplary embodiment of the present invention.

Referring to FIG. 3 (c), the portable terminal is configured to be input without distinguishing left and right eyes of an image coming into the eye through the barrier LCD in order to solve the problem as shown in FIG. 3 (a).

For example, the portable terminal passes only one pixel of the left pixel and the right pixel of the image to the barrier LCD. In the present invention, only the left pixel may be provided to the user (311).

Accordingly, the user of the portable terminal provides a 2D effect at a position outside the viewing angle to prevent the stereoscopic effect from being reversed at the position outside the viewing angle.

4 is a diagram illustrating a process of switching a screen when playing a 3D image in a portable terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the portable terminal combines 401 a pixel corresponding to a left view and a pixel corresponding to a right view as shown by reference numeral a to provide a 3D effect. If a user experiences a situation in which the stereoscopic effect is inverted out of the viewing angle, the portable terminal changes the position of the pixel corresponding to the left view and the pixel corresponding to the right view as shown by reference numeral b. . As described above, the pixel whose position is changed is changed left and right of an image input through the barrier LCD so that a user may feel a normal effect without inverting a stereoscopic effect.

The portable terminal may detect the position of the terminal through a sensor and output the output image in the horizontal direction or in the vertical direction.

That is, when the portable terminal rotates 405 to change the output direction as shown by reference numeral c, the portable terminal rotates the direction of the pixel whose position is changed as indicated by reference numeral d to change the output direction ( 407).

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of the following claims, but also by the equivalents of the claims.

100: control unit 102: posture check unit 104: tracking unit 106: image processing unit

Claims (19)

An apparatus for reproducing stereoscopic video data in a portable terminal,
A control unit which processes 3D video to be played;
Tracking unit for tracking the location of the user when playing the 3D image,
And reconstructing the reproduced image when the user determines that the user is out of the viewing range by the tracking unit.
The method of claim 1,
The apparatus for reproducing stereoscopic video data in the portable terminal,
A device comprising a display using a barrier LCD method.
The method of claim 1,
The image processing unit,
And changing positions of left and right pixels of the image.
The method of claim 1,
The image processing unit,
And reconstruct an image including only at least one pixel among left and right pixels of the image.
The method of claim 1,
The tracking unit,
A device comprising at least one of an infrared sensor, a camera, and a motion detection sensor, wherein the location of the user is determined using at least one of an infrared sensor, a camera, and a motion detection sensor.
The method of claim 1,
The tracking unit,
Device for detecting the movement of the user corresponding to the displacement above a predetermined threshold value.
The method of claim 1,
The tracking unit,
Device for detecting the user's movement at predetermined intervals.
The method of claim 1,
The image processing unit,
Reconstruct the image of the image stored in at least one of the output buffer of the memory unit, the display unit,
The image processing unit,
And store the reconstructed image in at least one of the memory unit and an output buffer of the display unit.
In the method for reproducing stereoscopic video data in a portable terminal,
The process of playing back 3D video,
Tracking the location of the user when playing the 3D image;
If the user is outside the viewing range, reconstructing the reproduced image.
The method of claim 9,
The portable terminal,
A method of playing back image data using a barrier LCD.
The method of claim 9,
Reconstructing the reproduced video,
And changing positions of left and right pixels of the image.
The method of claim 9,
Reconstructing the reproduced video,
And reconstructing an image including only at least one pixel among left and right pixels of the image.
The method of claim 9,
The process of tracking the location of the user,
And at least one of an infrared sensor, a camera, and a motion detection sensor.
The method of claim 9,
The process of tracking the location of the user,
And detecting a user's movement corresponding to a displacement equal to or greater than a predetermined threshold value.
The method of claim 9,
The process of tracking the location of the user,
And detecting a user's movement at predetermined intervals.
The method of claim 9,
Reconstructing the reproduced video,
Reconstructing an image of an image stored in at least one of an output buffer of a memory unit and a display unit;
And storing the reconstructed image in at least one of the memory unit and the output buffer of the display unit.
In a portable terminal for reproducing stereoscopic video data
A display unit capable of outputting 3D video or 2D video,
Memory unit for storing the 3D image to play,
Tracking unit for determining the viewing angle of the viewer,
An image processor for changing a pixel position of the 3D image data or providing a 2D effect when the viewer is outside the viewing angle;
It characterized in that it comprises a control unit for controlling to reconstruct the image played according to the viewing angle of the viewer,
And a posture check unit for changing an output direction of the image reproduced according to the angle of the portable terminal.
The method of claim 17,
The memory unit,
Portable terminal, which can be implemented as an output buffer of the display unit.
The method of claim 17,
The display unit,
A portable terminal comprising a barrier LCD supporting an auto glasses method.

Images