KR20120126897A - Electronic device and method for processing a 3-dimensional image - Google Patents

Abstract

PURPOSE: An electronic device and a three-dimensional image processing method are provided to sense a viewing-distance of a user by sensing distance by using a new mark. CONSTITUTION: A control unit detects a first mark and a second mark of glasses from the captured image frame(S110). In case the viewing direction is a screen direction, the control unit produces a distance value between the first mark and the second mark(S120,S125). The control unit calculates a depth setting value based on the calculated distance value(S130). The control unit controls the depth value of a three-dimensional image based on the calculated depth setting value(S135). [Reference numerals] (AA) Start; (BB) No; (CC) Yes; (DD) End; (S100) Receiving cubic image date; (S105) Filming user; (S110) Detecting a first mark and a second mark of glasses; (S115) Producing angle value; (S120) Whether viewing direction of a user is a screen direction?; (S125) Producing a distance value between the first mark and the second mark; (S130) Calculating a depth setting value based on the calculated distance value; (S135) Controlling the depth value of a three-dimensional image based on the calculated depth setting value; (S140) Displaying cubic image

Description

ELECTRICAL DEVICE AND METHOD FOR PROCESSING A 3-DIMENSIONAL IMAGE}

The present invention relates to an electronic device and a stereoscopic image processing device, and more particularly, to an electronic device and a stereoscopic image processing method for displaying a stereoscopic image.

Recently, display technology for representing 3D images has been researched and utilized in various fields. In particular, an electronic device displaying a 3D image by using a technology of displaying a 3D image is attracting attention.

The technique of displaying a 3D image uses the principle of binocular parallax, in which an observer feels a stereoscopic feeling due to binocular disparity, and is classified into a shutter glass method, a glasses-free method, a full three-dimensional method, and the like. In the glasses method, the user has to wear separate equipment such as separate polarized glasses, and in the glassesless method, the user can see a 3D image only at a specific location. Therefore, since there are various problems in the spectacle method and the glasses-free method, the research on the full three-dimensional method has been actively conducted recently.

The present invention is to provide an electronic device and a stereoscopic image processing method capable of displaying a stereoscopic image appropriate to a viewing distance.

In order to achieve the above technical problem, the stereoscopic image processing method according to the present invention comprises the steps of detecting a first mark and a second mark of the glasses in the image frame, the detected first mark and the detected second mark The method may include calculating a distance value of the liver, calculating a depth setting value based on the distance value, and adjusting a depth value of the stereoscopic image based on the calculated depth setting value. Here, the first mark and the second mark may be located on the left eye eye and right eye eye of the glasses, respectively. In addition, the first mark and the second mark may be formed of at least one of an LED light, a luminous material, and a fluorescent material. The distance value may be a pixel distance between the first mark and the second mark.

The stereoscopic image processing method may further include calculating an angle value based on a shape of at least one of the first mark and the second mark, and calculating the depth set value comprises: the distance value And calculating the depth setting value based on the angle value. The angle value may be calculated based on at least one of the ratio of the horizontal size, the horizontal size, and the vertical size of the shape.

The stereoscopic image processing method may further include determining whether a gaze direction of a user wearing the glasses is a screen direction based on the angle value, and adjusting a depth value of the stereoscopic image according to the checking result. Whether or not to perform the steps may be determined. The determining of the screen direction may further include checking whether the angle value is greater than or equal to a preset angle.

The calculating of the depth setting value may include calculating a viewing distance using the distance value and the angle value, and calculating the depth setting value based on the calculated viewing distance. The depth setting value may be calculated based on a ratio between the reference viewing distance and the calculated viewing distance.

According to another aspect of the present invention, an electronic device includes a receiver for receiving a stereoscopic image and a first mark and a second mark of glasses in a captured image frame, and the detected first mark and Calculating a distance value between the detected second marks, calculating a depth setting value based on the calculated distance value, and controlling adjustment of a depth value of the received stereoscopic image based on the calculated depth setting value It may include a control unit. Here, the first mark and the second mark may be located on the left eye eye and right eye eye of the glasses, respectively. In addition, the first mark and the second mark may be formed of at least one of an LED light, a luminous material, and a fluorescent material. The distance value may be a pixel distance between the first mark and the second mark.

The controller may calculate an angle value based on the shape of at least one of the first mark and the second mark, and calculate the depth setting value based on the distance value and the angle value. The angle value may be calculated based on at least one of a ratio of a horizontal size, a horizontal size, and a vertical size of the shape.

The controller may determine whether the eyeline direction of the user wearing the glasses is the screen direction based on the angle value, and control the adjustment of the depth value of the stereoscopic image according to the checking result. The controller may determine that the user's gaze direction is a screen direction when the angle value is smaller than a preset angle.

The control unit may calculate a viewing distance using the distance value and the angle value, and calculate the depth setting value based on the calculated viewing distance. The depth setting value may be calculated based on a ratio between the reference viewing distance and the calculated viewing distance.

According to the electronic device and the stereoscopic image processing method according to the present invention, since the visual distance is detected using the mark engraved on the glasses, the visual distance of the user can be accurately detected, and the depth of the stereoscopic image is appropriately adjusted according to the detected visual distance. By adjusting the value, the user can watch a stereoscopic image comfortably.

1 is a block diagram showing a configuration of a preferred embodiment of an electronic device according to the present invention;
2 is a block diagram showing a configuration of a preferred embodiment of the signal processing unit;
3 shows embodiments of the mark of the spectacles,
4 is a view showing an embodiment of a mark-shaped glasses,
FIG. 5 is a diagram schematically showing a distance between marks of glasses extracted from a captured image frame; FIG.
6 illustrates a preferred embodiment of a distance reference table;
7 is a view showing the shape of the mark of the glasses extracted from the captured image frame,
8 illustrates a preferred embodiment of an angular reference table;
9 illustrates another preferred embodiment of an angular reference table;
10 is a view showing a preferred embodiment of the depth value reference table;
11 is a diagram illustrating a binocular parallax scheme;
12 is a view showing a sense of distance of an object according to the size of binocular parallax;
13 is a view for explaining an embodiment of a method of adjusting a depth value of a stereoscopic image;
14 is a diagram illustrating coordinates of an object included in an image;
FIG. 15 shows coordinates of an object after displacement of the object shown in FIG. 14 is converted, and
16 is a view showing a preferred embodiment of the stereoscopic image processing method according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. At this time, the configuration and operation of the present invention shown in the drawings and described by it will be described as at least one embodiment, by which the technical spirit of the present invention and its core configuration and operation is not limited.

Although the terms used in the present invention have been selected in consideration of the functions of the present invention, it is possible to use general terms that are currently widely used, but this may vary depending on the intention or custom of a person skilled in the art or the emergence of new technology. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, it is to be understood that the term used in the present invention should be defined based on the meaning of the term rather than the name of the term, and on the contents of the present invention throughout.

1 is a block diagram illustrating a configuration of a preferred embodiment of an electronic device according to the present invention.

Referring to FIG. 1, the electronic device 100 according to the present invention includes a receiver 101, a signal processor 140, a display 150, an audio output unit 160, an input device 170, and a storage unit 180. And a controller 190. In some embodiments, the electronic device 100 may further include a photographing device 90. The electronic device 100 may be a personal computer system such as a desktop, a laptop, a tablet, or a handheld computer, and may be a mobile phone, a smart phone, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), It may be a mobile terminal such as a navigation device or the like, or may be a stationary home appliance such as a digital TV.

The receiver 101 receives image data including left eye view image data and right eye view image data. The receiver 101 may further receive broadcast information about the image data. The image data may be a stereo viewpoint image or a multiview image. That is, the image data may include at least one left eye view image data and at least one right eye view image data.

The receiver 101 may include a tuner 110, a demodulator 120, a mobile communication unit 115, a network interface unit 130, and an external signal receiver 130.

The tuner unit 110 selects an RF broadcast signal corresponding to a channel selected by a user from among radio frequency (RF) broadcast signals received through an antenna, and converts the selected RF broadcast signal into an intermediate frequency signal or a baseband video or audio signal. To convert.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation. Also, the demodulation unit 120 may perform channel decoding.

The demodulation unit 120 may perform demodulation and channel decoding, and then output a stream signal TS. In this case, the stream signal may be a signal multiplexed with a video signal, an audio signal, or a data signal. For example, the stream signal may be an MPEG-2 Transport Stream (TS) multiplexed with an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, and the like. Specifically, the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.

The stream signal output from the demodulator 120 may be input to the signal processor 140.

The mobile communication unit 115 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

 The external signal receiver 135 may provide an interface for connecting the external device to the electronic device 100. Here, the external device may refer to various types of video or audio output devices such as a DVD (Digital Versatile Disk), Blu-ray (Bluray), a game device, a camcorder, a computer (laptop), and a USB memory or a USB hard disk. It may be a device. The electronic device 100 may control the video signal and the audio signal received from the external signal receiver 135 to be displayed, and may store or use the data signal.

Also, the external device may be the photographing device 90. The photographing apparatus 90 may include a plurality of cameras. The photographing apparatus 90 may photograph a person and transmit an image frame photographed by the person to the electronic device 100.

The signal processor 140 demultiplexes the stream signal output by the demodulator 210 and performs signal processing on the demultiplexed signal, and then outputs an image to the display 150 to the audio output unit 160. The sound 161 is output. In addition, the signal processor 140 may receive image data, audio data, and broadcast data from the mobile communication unit 115, the network interface unit 130, and the external signal receiving unit 135.

The signal processor 140 may receive a signal indicating a depth setting value from the controller 190. The signal processor 140 may reconstruct the left eye view image data and the right eye view image data of the image data according to the received depth setting value.

In some embodiments, the signal processor 140 may calculate a displacement value of the object included in the image data, and change the calculated displacement value using the depth setting value. The signal processor 140 may reconstruct the left eye view image data and the right eye view image data so that the displacement value of the object is changed according to the changed displacement value.

The display 150 displays the image 152. In this case, the image 152 may be an image data reconstructed by the signal processor 140 is displayed. That is, the image data may be stereoscopic image data whose depth value is adjusted. In addition, the display 150 may operate in connection with the controller 190. The display 150 may display a graphical user interface (GUI) 153 that provides an easy-to-use interface between the user of the electronic device and an operating system or an application running on the operating system.

The voice output unit 160 may receive voice data from the signal processor 140 and the controller 190 and output a sound 161 in which the received voice data is reproduced.

The input device 170 may be a touch screen disposed on or in front of the display 150 and may be a touch pad. The touch screen may be integrated with the display 150 or may be a separate component. As the touch screen is disposed in front of the display 150, the user may directly manipulate the GUI 153. For example, the user can only place his finger on the object to be controlled.

The storage unit 180 generally provides a place for storing program codes and data used by the electronic device 100. For example, the storage unit 180 may be implemented as a read only memory (ROM), a random access memory (RAM), a hard disk drive, or the like. The program code and data may reside in a removable storage medium and may be loaded or installed onto the electronic device 100 as needed. Removable storage media herein include CD-ROMs, PC-CARDs, memory cards, floppy disks, magnetic tape, and network components.

The storage unit 180 may store information on the mark of the glasses, the distance reference table, the angle reference table, and the depth value setting information reference table. The information on the mark of the glasses may include image data of the mark, color information of the mark, and description information about the mark.

The controller 190 executes a command and performs an operation associated with the electronic device 100. For example, the controller 190 may control input and output between the components of the electronic device 100 and reception and processing of data by using a command retrieved from the storage unit 180. The controller 190 may be implemented on a single chip, multiple chips, or multiple electrical components. For example, various architectures may be used for the controller 190, including dedicated or embedded processors, single purpose processors, controllers, ASICs, and the like.

The controller 190 executes computer code together with an operating system to generate and use data. The operating system is generally known and will not be described in more detail. By way of example, the operating system may be a Window based OS, Unix, Linux, Palm OS, DOS, Android, Macintosh, and the like. The operating system, other computer code, and data may be present in the storage unit 180 that operates in conjunction with the control unit 190.

The controller 190 may recognize the user action and control the electronic device 100 based on the recognized user action. The user action may be performed by selecting a physical button of an electronic device or a remote control, performing a predetermined gesture on the touch screen display surface, selecting a soft button, or performing a predetermined gesture recognized from an image captured by the imaging device, and by voice recognition. It may include the implementation of any utterance recognized. The external signal receiver 135 may receive a signal for a user action of selecting a physical button of the remote controller through the remote controller. The gesture may include a touch gesture and a space gesture.

The input device 170 receives the gesture 171, and the controller 190 executes commands for performing operations associated with the gesture 171. In addition, the storage unit 180 may include a gesture operating program 181 which may be part of an operating system or a separate application. Gesture operator 181 generally recognizes the occurrence of gesture 171 and in turn responds to gesture 171 and / or gesture 171 to inform one or more software agents of what action (s) should be taken. Contains the command of.

When the user performs one or more gestures, the input device 170 transmits gesture information to the controller 190. Using commands from the storage unit 180, more specifically, the gesture operation program 181, the controller 190 interprets the gesture 171 and stores the storage unit 180, the display 150, and the voice output unit ( 160, different components of the electronic device 100, such as the signal processor 140, the network interface 130, and the input device 170, are controlled. The gesture 171 performs an operation in an application stored in the storage unit 180, modifies a GUI object displayed on the display 150, modifies data stored in the storage unit 180, and the network interface unit 130. ) May be identified as a command for performing an operation in the signal processor 140.

The controller 190 may determine the depth setting value based on the viewing distance. The controller 190 may detect the first mark and the second mark of the glasses in the image frame captured by the photographing apparatus 90, and calculate the viewing distance using the distance value between the detected first mark and the second mark. have. In addition, the controller 190 may calculate an angle value based on the shape of at least one of the detected first mark and the second mark, and correct the calculated viewing distance based on the calculated angle value. Here, the viewing distance before correction may be named as a user distance value, and the viewing distance after correction may be named as a viewing distance.

In some embodiments, the controller 190 may reconstruct the left eye view image data and the right eye view image data of the image data according to the determined depth setting value. In some embodiments, the controller 190 may control the signal processor 140 to reconstruct the left eye view image data and the right eye view image data of the image data according to the determined depth setting value.

2 is a block diagram showing the configuration of a preferred embodiment of the signal processor.

Referring to FIG. 2, the signal processor 140 may include a demultiplexer 210, an audio decoder 220, a video decoder 230, an image processor 240, a scaler 260, a mixer 270, and a formatter ( 280).

The demultiplexer 210 may receive a stream signal from the mobile communication unit 115, the network interface unit 130, and the external signal input unit 135. The demultiplexer 210 may convert the received stream signal into image data, The audio data and the data may be demultiplexed and output to the video decoder 230, the audio decoder 220, and the controller 190, respectively.

The audio decoder 220 may receive voice data from the demultiplexer 210, restore the received voice data, and output the restored data to the scaler 260 or the voice output unit 160.

The video decoder 230 receives image data from the demultiplexer 210, restores the received image data, and outputs the received image data to the image processor 240. The image signal may include a stereoscopic image signal. The video decoder 230 and the image processor 240 may be configured as one module. When the controller 190 performs a role performed by the image processor 240, the video decoder 230 may output the reconstructed image data to the controller 190.

The image processor 240 may reconstruct the left eye view image data and the right eye view image data included in the reconstructed image data according to the depth setting value. The image processor 240 may include a displacement calculator 245, a displacement changer 250, and an image reconstructor 255.

The displacement calculator 245 calculates a displacement value of the object included in the image data.

The displacement change unit 250 changes the displacement value calculated by the displacement calculator 245 using the depth set value. Here, the displacement change unit may calculate the changed displacement value based on the depth setting value and the calculated displacement value.

The image reconstructor 255 may reconstruct the left eye view image data and the right eye view image data of the image data such that the displacement value of the object is changed according to the displacement value changed by the displacement changer 250.

The scaler 260 may display the image data processed by the video decoder 230, the image processor 240, and the controller 190 and the audio data processed by the audio decoder 220 to display the display 150 or a speaker (not shown). Scale to the appropriate size signal to output through. In detail, the scaler 260 receives the stereoscopic image and scales it to match the resolution or the predetermined aspect ratio of the display 150. The display 150 may be manufactured to output an image screen having a predetermined resolution, for example, 720x480 format, 1024x768, or the like according to product specifications. Accordingly, the scaler 260 may convert the resolution of the stereoscopic image, which may be input with various values, to match the resolution of the corresponding display.

The mixer 270 mixes and outputs the outputs of the scaler 260 and the controller 190.

The formatter 280 outputs the converted image data to the display 150 to implement a stereoscopic image. When the stereoscopic image display method is a pattern retard method, the formatter 280 samples the received stereoscopic image frame into an image frame in which left eye view image data and elegant eye view image data are alternately arranged in a horizontal or vertical direction, Sampled image frames can be output.

When the 3D image display method is a shutter glass method, the formatter 280 may generate a sync signal to the output 3D image signal and transmit it to the glasses 201. The formatter 280 may include an infrared output unit (not shown) for transmission of a synchronization signal. Here, the synchronization signal is a signal for synchronizing the display time of the left eye view image or the right eye view image according to the stereoscopic image signal with the opening / closing time of the left eye lens or the right eye lens of the shutter glasses 201.

3 shows embodiments of the mark of the spectacles.

Referring to FIG. 3, the mark of the glasses may be one of the mark 310, the mark 320, and the mark 330. The mark of the spectacles is not limited to the embodiment shown in Fig. 3, and the mark identifiable on the subject of the included spectacles of the captured image frame can be used as the mark of the spectacles of the present invention. In addition, the mark of the glasses may be formed of at least one of LED light, luminous material and fluorescent material. Accordingly, the electronic device 100 according to the present invention can accurately track a user's position even in a dark environment and display a stereoscopic image having an optimal depth value according to the tracked user's position.

4 is a view illustrating an embodiment of glasses with marks.

Referring to FIG. 4, the mark of the glasses may be displayed on the glasses 400, and two marks of the same type or two marks of different types may be displayed on the glasses 400.

For example, the macro mark 410 and the mark 420 of the glasses may be displayed on the glasses 400. The mark 410 is located on the right eyeball of the glasses 400, and the mark 420 is located on the left eyeball of the glasses 400.

FIG. 5 is a diagram schematically illustrating a distance between marks of glasses extracted from a captured image frame.

Referring to FIG. 5, the controller 190 may detect the first mark 510 and the second mark 520 in the captured image frame 500. The controller 190 may recognize the first mark 510 and the second mark 520 using the color information of the mark in the image frame 500.

In addition, the controller 190 may calculate a distance value 530 between the first mark 510 and the second mark 520. The distance value 530 may be a pixel distance between the first mark 510 and the second mark 520, and may be a length according to a metric method.

In addition, the controller 190 may calculate a user distance value based on the distance value 530. In some embodiments, the controller 190 may calculate the user distance value y of the distance value 530 x by Equation 1 below.

 Here, a is a constant indicating a distance value when a reference distance and c is a reference distance, and b is a constant indicating a rate of change of the user distance value according to the change amount of the distance value 530. The constants a, b, c can be determined at the time of manufacture.

6 illustrates a preferred embodiment of a distance reference table.

Referring to FIG. 6, the controller 190 may calculate a user distance value with respect to the distance value 530 using the distance reference table 600. The controller 190 may detect a user distance value associated with the distance value 530 in the distance reference table 600. For example, when the distance value 530 is 100 pixels, the controller 190 may detect a user distance value 2.2m mapped to 100 pixels in the distance reference table 600. In addition, when the distance value 530 is 150 pixels, the controller 190 may calculate a user distance value 2.0m mapped to 150 pixels in the distance reference table 600.

If the same distance value as the distance value 530 does not exist in the distance reference table 600, the controller 190 detects and detects a user distance value associated with the distance value close to the distance value 530. The user distance value of the distance value 530 may be calculated using the distance value. When the distance value 530 does not exist in the distance reference table 600, the controller 190 may calculate the user distance value y of the distance value 530 x using Equation 2 below.

Here, x_low is a distance value existing in the distance reference table 600 among values smaller than x, x_high is a distance value existing in the distance reference table 600 among values larger than x, and y_high is a user distance value of x_high. , y_low is the user distance value of x_low. For example, when the distance value 530 is 125 pixels, the controller 190 may select 100 pixels as x_low and 150 pixels as x_high, detect 2.0 m with y_high and 2.2 m with y_low. . As a result, the controller 190 may calculate a user distance value of 2.1 m.

7 is a view showing the shape of the mark of the glasses extracted from the captured image frame.

Referring to FIG. 7, the controller 190 may detect one mark 710 in the captured image frame 700. Herein, the controller 190 may recognize the mark 710 using the color information of the mark 710. Color information of the mark 710 may be stored in the storage unit 180 in advance.

The controller 190 may calculate the horizontal gap 711 of the mark. The horizontal interval 711 may be a pixel distance or a length according to a metric system. The controller 190 may calculate an angle value based on the horizontal interval 711.

The controller 190 may calculate the vertical gap 712 of the mark. Herein, the vertical interval 712 may be a pixel distance and may be a length according to a metric system. The controller 190 may calculate an angle value based on the horizontal interval 711 and the vertical interval 712.

8 is a diagram illustrating a preferred embodiment of the angular reference table.

Referring to FIG. 8, the controller 190 may calculate an angle value for the horizontal interval 711 using the angle reference table 800. The controller 190 may detect an angle value associated with the horizontal interval 711 in the angle reference table 800. For example, when the horizontal interval 711 is 10 pixels, the controller 190 may detect an angle value 60 degrees mapped to 10 pixels in the angle reference table 800. Also, when the horizontal interval 711 is 15 pixels, the controller 190 may calculate an angle value 50 degrees mapped to 15 pixels in the angle reference table 800.

If the same horizontal interval as the horizontal interval 711 does not exist in the angle reference table 800, the controller 190 detects an angle value associated with the horizontal interval close to the horizontal interval 711, and detects the detected horizontal value. An angle value of the horizontal interval 711 may be calculated using the interval. In an embodiment, when the horizontal interval 711 does not exist in the distance reference table 800, the controller 190 may calculate an angle value y of the horizontal interval 711 x using Equation 3 below. Can be.

Here, x_low is a horizontal interval existing in the angle reference table 800 among values smaller than x, x_high is a horizontal interval existing in the angle reference table 800 among values larger than x, y_high is an angle value of x_high, y_low is the angle value of x_low. For example, when the horizontal interval 711 is 12.5 pixels, the controller 190 may select 10 pixels as x_low and 15 pixels as x_high, detect 50 degrees with y_high, and measure 60 degrees with y_low. . As a result, the controller 190 may calculate an angle value of 55 degrees.

In addition, the controller 190 may calculate a position angle coefficient associated with the angle value. According to an example calculation method, the controller 190 may calculate the position angle coefficient by using the angle reference table 800. For example, if the calculated angle value is 50 degrees, the controller 190 calculates 2.0 as the position angle coefficient.

In addition, the controller 190 may calculate a position angle coefficient associated with the horizontal interval 711 using the angle reference table 800. For example, if the horizontal interval 711 is 10 pixels, the controller 190 may calculate 2.2 as the position angle coefficient. If the position angle coefficient associated with the calculated horizontal interval 711 does not exist in the angle reference table 800, the controller 190 calculates the position angle coefficient of the horizontal interval 711 using Equation 4 below. Can be calculated.

Here, x_low is the horizontal spacing present in the angular reference table 800 among values smaller than x, x_high is the horizontal spacing present in the angular reference table 800 among values larger than x, and y_high is the position angle coefficient of x_high. , y_low is the position angle coefficient of x_low. For example, when the ratio is 12.5 pixels, the controller 190 may select 10 pixels as x_low and 15 pixels as x_high, detect 2.0 with y_high, and set 2.2 as y_low. As a result, the controller 190 may calculate 2.1 as the position angle coefficient.

9 shows another preferred embodiment of the angular reference table.

Referring to FIG. 9, the controller 190 may calculate an angle value with respect to the ratio using the angle reference table 900. The ratio here is the ratio of the horizontal spacing 711 and the vertical spacing 712. That is, the ratio may be calculated by the following Equation 5.

Where H is the horizontal spacing 711, V is the vertical spacing 712, and Y is the ratio.

The controller 190 may detect an angle value associated with the ratio calculated by the angle reference table 900. For example, when the ratio is 0.5, the controller 190 may detect an angle value 60 degrees mapped to 0.5 in the angle reference table 900. In addition, when the ratio is 0.64, the controller 190 may calculate an angle value 50 degrees mapped to 0.64 in the angle reference table 900.

In addition, the controller 190 may calculate the position angle coefficient using the angle reference table 900. For example, if the calculated angle value is 50 degrees, the controller 190 calculates 2.0 as the position angle coefficient. The controller 190 may calculate the position angle coefficient associated with the ratio using the angle reference table 900. For example, when the calculated ratio is 0.98, the controller 190 may calculate 1.2 as the position angle coefficient.

The controller 190 may calculate the viewing distance based on the user distance value and the position angle coefficient. In some embodiments, the controller 190 may calculate the viewing distance y based on Equation 6 below.

Where a is a position angle coefficient and b is a user distance value.

In addition, the controller 190 may calculate the viewing distance based on the user distance value and the angle value. In some embodiments, the controller 190 may calculate a viewing distance y based on Equation 7 below.

Here, a is a user distance value and θ is an angle value.

The controller 190 may calculate a depth setting value based on the calculated viewing distance.

FIG. 10 is a diagram illustrating a preferred embodiment of a depth value reference table. FIG.

Referring to FIG. 10, the controller 190 may calculate a depth setting value suitable for a viewing distance calculated using the depth value setting reference table 1000. For example, when the calculated viewing distance is a reference distance 1.4, the controller 190 may calculate 0 as the depth setting value. In addition, the controller 190 may calculate -2 as the depth setting value when the calculated viewing distance is 1 m. When the calculated viewing distance is 2m, the controller 190 may calculate 3 as the depth setting value.

When the calculated viewing distance is less than or equal to the specific distance, the controller 190 may calculate 0 as the depth setting value. That is, when the distance is less than a certain distance, the controller 190 may not adjust the depth value of the stereoscopic image.

In addition, the controller 190 may determine whether the user's gaze direction is the screen direction based on the angle value. That is, when the angle value is greater than or equal to the preset value, the controller 190 may determine that the user's gaze direction is not the screen direction. When the angle value is less than or equal to the preset value, the controller 190 may determine that the gaze direction of the user is the screen direction. You can judge.

The controller 190 may calculate the depth setting value as 0 according to the confirmation result in the gaze direction, and may not adjust the depth value of the stereoscopic image. That is, when the angle value is 70 degrees to 90 degrees, the controller 190 may calculate the depth set value as zero. In addition, when the angle value is 70 degrees to 90 degrees, the controller 190 may control not to adjust the depth value of the stereoscopic image.

The controller 190 may adjust the depth value of the stereoscopic image based on the calculated depth setting value.

In some embodiments, the controller 190 may calculate the depth setting value based on the user distance value instead of the viewing distance. That is, the controller 190 may calculate a depth setting value suitable for the user distance value calculated by using the depth value setting reference table 1000. For example, the controller 190 may calculate 0 as the depth setting value when the calculated user distance value is the reference distance 1.4, and -2 as the depth setting value when the calculated user distance value is 1m. Can be calculated.

11 is a diagram illustrating a binocular parallax method.

Referring to FIG. 11, in the binocular disparity method, at least one left eye view image 1101 and a right eye view image 1102 captured by a binocular camera or the like are shown to the viewer's eyes 1121 and 1122, respectively, to provide a sense of space or stereoscopic feeling. It is a 3D display method provided. The depth value varies according to the binocular disparity between the left eye view image 1101 and the right eye view image 1102, and the sense of space or three-dimensionality provided to the viewer varies according to the depth value. The depth value here is the distance from the three-dimensional display surface 1130 to the displayed three-dimensional image 1113.

The narrower the distance between the left eye view image 1101 and the right eye view image 1102 is, the smaller the depth value is, and it is recognized that the image is formed at a distance from the left eye 1121 and the right eye 1122, thereby providing a sense of space or stereoscopic feeling that is provided to the viewer. Can be made smaller. As the distance between the left eye view image 1101 and the right eye view image 1102 is wider, the depth value is increased, and it is recognized that the image is formed at a close distance from the left eye 1121 and the right eye 1122, thereby providing a sense of space or a three-dimensional feeling provided to the viewer. Can be large.

12 is a diagram illustrating a sense of distance of an object according to the size of binocular parallax.

Referring to FIG. 12, the distance between the image 1211 formed in the left eye and the image 1212 formed in the right eye becomes narrow when the distant object 1210 is viewed with both eyes, so that the binocular parallax when viewing the distant object 1210 is narrowed. Is small. Therefore, by adjusting the binocular disparity of the stereoscopic image to be smaller, the stereoscopic image 1113 can be seen far away and the depth value of the stereoscopic image 1113 can be adjusted to be smaller.

As the distance between the image 1221 formed in the left eye and the image 1222 formed in the right eye becomes wider when the nearby object 1220 is viewed with both eyes, the binocular disparity is large when the object 1220 is viewed near. That is, the binocular parallax of the object 1220 is greater than the binocular parallax of the object 1210. Therefore, the binocular disparity of the stereoscopic image can be largely adjusted to make the stereoscopic image 1113 appear closer and the depth value of the stereoscopic image 1113 can be increased.

FIG. 13 is a diagram for describing an embodiment of a method of adjusting a depth value of a stereoscopic image. FIG.

Referring to FIG. 13, the electronic device 100 may reduce the binocular parallax of the object 1310 in the stereoscopic image 1300 to reduce the depth value of the stereoscopic image 1300. That is, by adjusting the binocular parallax of the object 1310 to be smaller to make the object appear farther to the viewer than the original distance, the value of the stereoscopic image 1300 can be reduced.

Also, the electronic device 100 may increase the depth value of the stereoscopic image 1300 by increasing the binocular parallax of the object 1320 in the stereoscopic image 1300. That is, by adjusting the binocular parallax of the object 1320 to make the object appear closer to the viewer than the original distance, the value of the stereoscopic image 1300 may be increased.

14 is a diagram illustrating coordinates of an object included in an image.

Referring to FIG. 14, when a stereoscopic image enters a vector graphic in which the coordinates of the subject are known, the coordinates of the subject may be displayed according to Equations 8 and 9 below.

는 오브젝트의 좌안 오브젝트의 위치와 상기 오브젝트의 우안 오브젝트의 위치의 차이를 의미하며, 오브젝트 간의 거리를 다음의 수학식 10으로 나타내질 수 있다. 여기서 좌안 오브젝트는 좌안 시점 영상 데이터에 포함된 오브젝트 또는 상기 오브젝트의 이미지를 표시하는 좌표를 의미하고, 우안 오브젝트는 우안 시점 영상 데이터에 포함된 오브젝트 또는 상기 오브젝트의 이미지를 표시하는 좌표를 의미한다. 즉 오브젝트(1410)의 좌안 오브젝트는 오브젝트(1411)이고 오브젝트(1410)의 우안 오브젝트는 우안 오브젝트(1412)이다.Object distance

Denotes the difference between the position of the left eye object of the object and the position of the right eye object of the object, and the distance between the objects may be represented by Equation 10 below. Here, the left eye object refers to an object included in left eye view image data or coordinates for displaying an image of the object, and the right eye object refers to an object included in right eye view image data or a coordinate for displaying an image of the object. That is, the left eye object of the object 1410 is the object 1411 and the right eye object of the object 1410 is the right eye object 1412.

The object distance of the object 1410 is 1, the object distance of the object 1420 is 2, and the object distance of the object 1430 is 3.

는 다음의 수학식 11로 나타낼 수 있다.Also set of object distances

May be represented by Equation 11 below.

The object distance may be changed using Equation 12 below.

은 변경된 오브젝트 거리이다. a 는 입체감 정도를 나타내는 변수를 뜻하고, 변수 a의 값은 깊이 설정값 또는 깊이 설정값에 함수 관계를 갖는 값으로 설정될 수 있다.here,

Is the changed object distance. a denotes a variable representing a degree of three-dimensional effect, and a value of the variable a may be set to a value having a function relationship with the depth setting value or the depth setting value.

In binocular parallax, the object distance seems to be a perspective of the object, and it can be seen that it represents the binocular parallax of the object. Therefore, in Equation 12, the closer the variable a is to O, the smaller the depth value of the stereoscopic image, and in the case of 1, the depth value of the stereoscopic image is maintained, and the greater than 1, the greater the depth value of the stereoscopic image.

는 다음의 수학식 13으로 나타낼 수 있다.Also set of changed object distances

Can be represented by the following equation (13).

In some embodiments, the electronic device 100 may calculate a using Equation 14 below.

Where b is the depth setting value. For example, when b is 0, the depth value of the stereoscopic image is maintained without being adjusted.

FIG. 15 is a diagram illustrating coordinates of an object after the displacement of the object illustrated in FIG. 14 is converted.

Referring to FIG. 15, when a is 2 in Equation 12, the object distance of the object 1510 is changed to 2, the object distance of the object 1520 is changed from 2 to 4, and the object distance of the object 1530 is Changed from 3 to 6.

16 is a view showing a preferred embodiment of the stereoscopic image processing method according to the present invention.

Referring to FIG. 16, the receiver 101 receives image data including left eye view image data and right eye view image data (S100). The image data may be a stereo viewpoint image or a multiview image.

The photographing apparatus 90 picks up the user (S105).

The controller 190 detects the first mark and the second mark of the glasses in the image frame captured by the photographing apparatus 90 (S110). Here, the first mark and the second mark may be the marks 310, 320, and 330 shown in FIG. 3, and the glasses may be the glasses 400 shown in FIG. 4.

The controller 190 calculates an angle value based on the shape of at least one of the first mark and the second mark (S115). Herein, the controller 190 may calculate an angle value using the angle reference table 800 illustrated in FIG. 8 and the angle reference table 900 illustrated in FIG. 9.

The controller 190 checks whether the eyeline direction of the user wearing the glasses is the screen direction based on the angle value (S120). In this case, when the angle value is 70 degrees or more, the controller 190 may confirm that the gaze direction is not the screen direction. Otherwise, the controller 190 may identify that the gaze direction is the screen direction.

When the visual line direction is the screen direction, the controller 190 calculates a distance value between the detected first mark and the second mark (S125). The distance value may be a user distance value and may be a viewing distance. In addition, the controller 190 may calculate a user distance value using the distance reference table 600 shown in FIG. 6, and calculate a viewing distance using Equation 6.

The controller 190 calculates a depth setting value based on the calculated distance value (S130). Herein, the controller 190 may calculate the depth setting value using the depth value setting reference table 1000 illustrated in FIG. 10. In addition, the controller 190 may calculate the depth setting value based on the ratio of the calculated viewing distance and the reference viewing distance. When the calculated viewing distance is smaller than the reference viewing distance, the controller 190 may set the depth setting value to a negative value. When the calculated viewing distance is the reference viewing distance, the controller 190 may set the depth setting value to 0, and the calculated When the viewing distance is larger than the reference viewing distance, the depth setting value can be set positively. Here, the reference viewing distance may be a value set at the time of manufacture, or may be a value received by the receiver 101. Also, the reference viewing distance may be included in the stereoscopic image data.

The controller 190 controls the depth value of the stereoscopic image to be adjusted based on the calculated depth setting value (S135). Here, the displacement calculator 245 may calculate the object distance from the received stereoscopic image. In this case, the displacement calculator 245 may calculate the object distance using Equation 10. The displacement changing unit 250 may calculate a value based on the depth setting value using Equation 14. In addition, the displacement change unit 250 may calculate the changed object distance based on the calculated a using Equation 12. The image reconstructor 255 may reconstruct the left eye view image data and the right eye view image data of the received stereoscopic image such that the distance of the object is changed according to the calculated object distance.

The display 150 displays a stereoscopic image (S140). Here, the displayed stereoscopic image may be a stereoscopic image in which a depth value is adjusted, or a stereoscopic image displayed as it is.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer apparatus is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer-readable recording medium may also be distributed to networked computer devices so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation in the embodiment in which said invention is directed. It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the appended claims.

Claims (20)

Detecting a first mark and a second mark of the glasses in the captured image frame;
Calculating a distance value between the detected first mark and the detected second mark;
Calculating a depth setting value based on the distance value; And
And adjusting the depth value of the stereoscopic image based on the calculated depth setting value. The method of claim 1,
And the first mark and the second mark are located on the left eye eye and right eye eye of the glasses, respectively. The method of claim 1,
And the first mark and the second mark are formed of at least one of an LED light, a luminous material, and a fluorescent material. The method of claim 1,
Calculating an angle value based on a shape of at least one of the first mark and the second mark,
Calculating the depth setting value,
And calculating the depth setting value based on the distance value and the angle value. The method of claim 4, wherein
The angle value is calculated based on at least one of the ratio of the horizontal size, the horizontal size and the vertical size of the shape. The method of claim 4, wherein
Determining whether a gaze direction of a user wearing the glasses is a screen direction based on the angle value;
And determining whether to perform the step of adjusting the depth value of the stereoscopic image according to the checking result. The method according to claim 6,
Checking whether the screen orientation is,
And determining whether the angle value is greater than or equal to a preset angle. The method of claim 4, wherein
Calculating the depth setting value,
Calculating a viewing distance using the distance value and the angle value; And
And calculating the depth setting value based on the calculated viewing distance. The method of claim 8,
The depth setting value is calculated based on the ratio between the reference viewing distance and the calculated viewing distance. The method of claim 1,
And the distance value is a pixel distance between the first mark and the second mark. Receiving unit for receiving a stereoscopic image; And
The first mark and the second mark of the glasses are detected in the captured image frame, the distance value between the detected first mark and the detected second mark is calculated, and the depth setting value is determined based on the calculated distance value. And a controller configured to control the adjustment of the depth value of the received stereoscopic image based on the calculated depth setting value. 12. The method of claim 11,
And the first mark and the second mark are located at the left eye and right eye of the glasses, respectively. 12. The method of claim 11,
And the first mark and the second mark are formed of at least one of an LED light, a luminous material, and a fluorescent material. 12. The method of claim 11,
The control unit,
And calculating an angle value based on a shape of at least one of the first mark and the second mark, and calculating the depth setting value based on the distance value and the angle value. The method of claim 14,
And the angle value is calculated based on at least one of a ratio of a horizontal size, a horizontal size, and a vertical size of the shape. The method of claim 14,
The control unit,
The electronic device may be configured to determine whether a line of sight of the user wearing the glasses is the screen direction based on the angle value, and control the adjustment of the depth value of the stereoscopic image according to the checking result. 17. The method of claim 16,
The control unit,
And when the angle value is smaller than a preset angle, confirming that the user's gaze direction is a screen direction. The method of claim 14,
The control unit,
And calculates a viewing distance using the distance value and the angle value, and calculates the depth setting value based on the calculated viewing distance. 19. The method of claim 18,
And the depth setting value is calculated based on a ratio between the reference viewing distance and the calculated viewing distance. 12. The method of claim 11,
And the distance value is a pixel distance between the first mark and the second mark.

Images