KR102592092B1 - Method for Displaying Three Dimensional Picture by Display Apparatus Including a Monochrome Panel - Google Patents

Abstract

The present invention is a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the user's viewpoint, and provides a method of outputting a three-dimensional image based on the user's viewpoint information derived based on shooting information of a camera capturing the user. By varying the characteristics of the pattern displayed on the panel, the three-dimensional viewing angle and viewing distance at which the three-dimensional image displayed on the display panel is expressed in three dimensions can be adjusted flexibly according to the user's viewpoint, compared to when the pattern is fixed. This relates to a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the viewpoint.

Description

Method for displaying three dimensional pictures by display apparatus including a monochrome panel}

The present invention is a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the user's viewpoint, and provides a method of outputting a three-dimensional image based on the user's viewpoint information derived based on shooting information of a camera capturing the user. By varying the characteristics of the pattern displayed on the panel, the three-dimensional viewing angle and viewing distance at which the three-dimensional image displayed on the display panel is expressed in three dimensions can be adjusted flexibly according to the user's viewpoint, compared to when the pattern is fixed. This relates to a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the viewpoint.

Parallax Barrier is a technology that can display images in three dimensions on a flat display without glasses by using the parallax between the left and right eyes along with a lenticular lens. Specifically, the parallax barrier places a pattern that transmits or blocks light in front of the display panel, allowing the left and right eyes to see different images, allowing the user to feel the image on the display panel as three-dimensional.

Meanwhile, the problem with the parallax barrier is that the area where the displayed image is displayed three-dimensionally is limited. Specifically, in the conventional case, the three-dimensional viewing angle and viewing distance are fixed by implementing a parallax barrier using a fixed-print attached PB sheet, so the image can be viewed three-dimensionally only when viewed within an area that satisfies the three-dimensional viewing angle and viewing distance. If it is outside of this, there is a problem in that the image is not displayed three-dimensionally and a crosstalk phenomenon occurs in which the image appears broken. Specifically, when the appropriate stereoscopic viewing angle and viewing distance are exceeded, the user may see the image for the left eye and the image for the right eye overlap or appear blurred.

In addition, conventional stereoscopic image output devices are manufactured by manufacturing a PB sheet in consideration of the characteristics of the display panel (pixel shape, spacing), etc., and measuring the stereoscopic viewing angle and viewing distance with the PB sheet attached. In other words, conventionally, the PB sheet is not manufactured so that the stereoscopic image output device has the desired stereoscopic viewing angle and viewing distance, but on the contrary, the PB sheet is manufactured first, and when the PB sheet is attached, the stereoscopic viewing angle and viewing distance of the stereoscopic imaging device are adjusted. It is manufactured using a measuring method.

As a result, there is a need to develop a method and device that can adjust the stereoscopic viewing angle and viewing distance so that the image displayed on the display panel can be displayed in three dimensions by changing the pattern of the parallax barrier according to the user's viewpoint.

Domestic registered patent KR 10-2098151 B1 “Stereoscopic image display device”

The present invention is a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the user's viewpoint, and provides a method of outputting a three-dimensional image based on the user's viewpoint information derived based on shooting information of a camera capturing the user. By varying the characteristics of the pattern displayed on the panel, the three-dimensional viewing angle and viewing distance at which the three-dimensional image displayed on the display panel is expressed in three dimensions can be adjusted flexibly according to the user's viewpoint, compared to when the pattern is fixed. The purpose is to provide a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the viewpoint.

In order to solve the above problem, a method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern changes depending on the user's viewpoint, wherein the image output device changes the pattern according to the user's viewpoint Monochrome panel to display; A display panel disposed on the back of the monochrome panel and displaying a three-dimensional image by outputting a video image for either the left eye or the right eye for each pixel; A camera that takes images of the user; and a first operation unit that determines a pattern displayed on the monochrome panel based on the user's viewpoint information derived based on the shooting information of the camera; and a control unit including a second operation unit that converts the original image into a three-dimensional image displayed on the display panel based on the user's viewpoint information. The method of outputting the three-dimensional image includes, by the control unit, Based on the shooting information of the camera, one of the viewing distance between the user and the image output device, a first viewing angle when the user's location is projected on the first plane, and a second viewing angle when projected on the second plane. A viewpoint information deriving step of deriving viewpoint information including the above; A pattern information derivation step of deriving pattern information about the characteristics of a pattern output from a monochrome panel based on the viewpoint information by the first operation unit; An image conversion step of converting the original image into a three-dimensional image output from the display panel by the second operation unit based on the viewpoint information and pattern information, wherein the first plane is a plane implemented in a monochrome panel. and the second plane is orthogonal to the first plane.

In one embodiment of the present invention, the pattern includes first and second regions having different transmittances alternately arranged at a predetermined inclination, and the pattern information deriving step is performed based on the viewpoint information, The pattern output from the monochrome panel can be determined by deriving pattern information including one or more of the starting coordinates, the slope of the pattern, the width of the first area, and the width of the second area.

In one embodiment of the present invention, the control unit stores the separation distance between the monochrome panel and the display panel, and in the pattern information derivation step, the pattern information is generated by additionally considering the viewpoint information and the separation distance. It can be derived.

In one embodiment of the present invention, the image conversion step is to output the original image from the display panel by the second operation unit based on one or more of the viewpoint information and pattern information that varies depending on the user's viewpoint. It can be converted into a three-dimensional image.

In one embodiment of the present invention, the image output device allows the pattern of the monochrome panel to vary depending on the user's viewpoint, in contrast to the fact that the viewing angle limit at which a three-dimensional image is displayed in three dimensions is fixed when the pattern is fixed. A three-dimensional image can be displayed three-dimensionally from the viewpoint.

In one embodiment of the present invention, the original image is 3D data modeled in 3D, or the 2D original image is input to a deep learning-based machine learning model that can infer depth information for each pixel from 2D data. This is data in which color information and depth information are stored for each pixel, and the second operation unit displays the original image in three dimensions because the images shown to the left and right eyes are different when the original image is viewed through the pattern. It can be converted into a three-dimensional image.

According to one embodiment of the present invention, the image output device includes a camera that captures the user's image, and can vary the characteristics of the pattern implementing the parallax barrier according to the user's viewpoint information.

According to an embodiment of the present invention, when converting an original image into a three-dimensional image, a pattern that varies depending on the user's viewpoint information can be reflected.

According to an embodiment of the present invention, based on the shooting information of the camera, the viewing distance between the user and the image output device, the first viewing angle when the user's location is projected on the first plane, and the user's location on the second plane. Based on the viewpoint information including the second viewing angle when projected, the characteristics of the pattern such as the starting coordinates of the pattern, the slope of the pattern, the width of the first area, and the width of the second area can be determined.

According to one embodiment of the present invention, the characteristics of the pattern can be determined by further reflecting the separation distance between the monochrome panel and the display panel and the binocular distance between both eyes.

Figure 1 shows a parallax barrier implemented with a fixed-print adhesive PB sheet.
Figure 2 shows components of an image output device according to an embodiment of the present invention.
Figure 3 shows steps of a method for outputting a stereoscopic image according to an embodiment of the present invention.
Figure 4 shows the viewpoint information derivation step according to an embodiment of the present invention.
Figure 5 shows the pattern information derivation step according to an embodiment of the present invention.
Figure 6 shows an image conversion step according to an embodiment of the present invention.
Figure 7 shows the stereoscopic viewing angle and viewing distance of an image output device according to an embodiment of the present invention.
Figure 8 shows steps of a method for outputting a stereoscopic image according to another embodiment of the present invention.
Figure 9 schematically shows the internal configuration of a computing device according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that this aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain example aspects of one or more aspects. However, these aspects are illustrative and some of the various methods in the principles of the various aspects may be utilized, and the written description is intended to encompass all such aspects and their equivalents.

Additionally, various aspects and features may be presented by a system that may include multiple devices, components and/or modules, etc. It is also understood that various systems may include additional devices, components and/or modules, etc. and/or may not include all of the devices, components, modules, etc. discussed in connection with the drawings. It must be understood and recognized.

As used herein, “embodiments,” “examples,” “aspects,” “examples,” etc. may not be construed to mean that any aspect or design described is better or advantageous over other aspects or designs. . The terms '~part', 'component', 'module', 'system', 'interface', etc. used below generally refer to computer-related entities, such as hardware, hardware, etc. A combination of and software, it can mean software.

Additionally, the terms “comprise” and/or “comprising” mean that the feature and/or element is present, but exclude the presence or addition of one or more other features, elements and/or groups thereof. It should be understood as not doing so.

Additionally, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those skilled in the art to which the present invention pertains. It has the same meaning as Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the embodiments of the present invention, have an ideal or excessively formal meaning. It is not interpreted as

Figure 1 shows a parallax barrier implemented with a fixed-print adhesive PB sheet.

The conventional parallax barrier is a fixed printed PB sheet that is attached to the front of the display panel (2) and uses the parallax between the left and right eyes to see different images, thereby creating an image on the display panel (2). can be displayed three-dimensionally.

That is, each pixel of the display panel (2) can display either an image for the left eye or an image for the right eye, and the user can view the image of the display panel (2) through the PB sheet attached to the front of the display panel (2). When observing (or an image), the image for the left eye is displayed to the left eye and the image for the right eye is displayed to the right eye, so that the user can feel the image displayed on the display panel (2) in three dimensions.

Meanwhile, stereoscopic images implemented through a parallax barrier can be expressed stereoscopically only at a limited stereoscopic viewing angle and viewing distance. Specifically, the user can view the stereoscopic image in three dimensions when viewing the stereoscopic image on the display panel 2 within an area that satisfies a predetermined stereoscopic viewing angle and viewing distance, and in an area that does not satisfy the stereoscopic viewing angle and viewing distance. When watching a video, the stereoscopic image may not be displayed stereoscopically.

Additionally, the stereoscopic viewing angle and viewing distance can be determined by the parallax barrier, that is, the pattern of the PB sheet.

However, as mentioned above, as the conventional PB sheet has an irreversible pattern, the stereoscopic viewing angle and viewing distance also had to be fixed. Specifically, in the conventional PB sheet, the characteristics of the pattern (dimension, position, shape, etc.) are determined by considering the pixel arrangement and shape of the display panel (2), and the three-dimensional viewing angle and viewing distance are characteristics of the display panel (2) of the pattern. It can be fixed depending on (refractive index, etc.), the thickness of the PB sheet, and the characteristics of the pattern.

According to one embodiment of the present invention, a parallax barrier is implemented by displaying a pattern through a monochrome panel (1) with a variable pattern instead of a fixed print attachable PB sheet in front of the display panel (2), thereby improving the user's viewpoint. You can adjust the stereoscopic viewing angle and viewing distance accordingly.

Figure 2 shows components of an image output device according to an embodiment of the present invention.

As shown in Figure 2, the image output device includes a monochrome panel (1) that displays a changing pattern according to the user's viewpoint; A display panel (2) disposed on the rear of the monochrome panel (1) and displaying a three-dimensional image by outputting a video image for either the left eye or the right eye for each pixel; A camera that takes images of the user; and a first operation unit 41 that determines a pattern displayed on the monochrome panel 1 based on the user's viewpoint information derived based on the shooting information of the camera. and a second operation unit 42 that converts the original image into a three-dimensional image displayed on the display panel 2 based on the user's viewpoint information. The output method is based on the shooting information of the camera, the viewing distance between the user and the image output device, the first viewing angle when the user's position is projected onto the first plane, and the first viewing angle by the control unit 4. A viewpoint information deriving step of deriving viewpoint information including one or more of the second viewing angles when projected onto two planes; A pattern information derivation step of deriving pattern information about the characteristics of a pattern output from the monochrome panel (1) by the first operation unit (41) based on the viewpoint information; An image conversion step of converting the original image into a three-dimensional image output from the display panel 2 by the second operation unit 42 based on the viewpoint information and pattern information, wherein the first plane is It may be parallel to the plane implemented in the monochrome panel 1, and the second plane may be perpendicular to the first plane.

The monochrome panel 1 is a panel placed in front of the video output device, and may preferably be a black-and-white LCD panel or an LED panel. Specifically, the monochrome panel (1) can implement a parallax barrier by displaying a pattern in front of the display panel (2). Preferably, the pattern displayed on the monochrome panel 1 can be changed according to commands from the control unit 4 (the first operation unit 41 of the control unit 4), and more preferably, information captured by a camera that captures the user's image. It may vary depending on the time point information derived based on .

The display panel 2 is a panel disposed at the rear of the image output device to display a three-dimensional image. Preferably, it may be a panel that implements colors, such as an LCD or LED. Specifically, the second operation unit 42 of the control unit 4 can convert the input original image into a three-dimensional image and display it on the display panel. Preferably, the three-dimensional image is image data that is converted based on the original image and includes image images for each of the left and right eyes for each pixel, and is displayed on the display panel (2) by transmitting the pattern displayed on the monochrome panel (1). When watching a stereoscopic image, different video images are recognized in the left and right eyes, so the stereoscopic image can feel three-dimensional.

In one embodiment of the present invention, the display panel 2 may be a panel that can output a relatively high-quality image compared to the monochrome panel 1. Specifically, the display panel (2) uses a panel with excellent color reproduction and resolution, but the monochrome panel (1), which displays a pattern of alternating black and white areas, does not require high color reproduction and resolution and can be implemented with a low-resolution panel. Therefore, it is possible to reduce production costs.

In one embodiment of the present invention, the monochrome panel 1 may be a low-resolution black-and-white FHD panel, and the display panel 2 may be a high-resolution QHD panel.

The camera can capture the user watching the video output device. Specifically, the camera is built into the frame where the monochrome panel 1 is installed, captures a user located in front of the video output device, generates shooting information, and transmits the generated shooting information to the control unit 4. .

The control unit 4 is a computing device including one or more processors and one or more memories, and determines the characteristics of the patterns displayed on the monochrome panel 1 and the display panel 2 based on the shooting information received from the camera. , or you can convert the original image into a three-dimensional image.

Specifically, the control unit 4 can derive viewpoint information between the user and the image output device based on the shooting information received from the camera, and determine the characteristics of the pattern output from the monochrome panel 1 based on the viewpoint information. there is. Additionally, the control unit 4 may command the monochrome panel 1 to display a pattern having determined characteristics.

Meanwhile, the characteristics of the above-described pattern may include one or more of the inclination of the pattern, the width of each of the first area that blocks light and the second area that transmits light, and the start coordinates of the pattern.

Meanwhile, the control unit 4 can convert the input original image into a three-dimensional image. As described above, the stereoscopic image includes a video image for the left eye and a video image for the right eye for each pixel, and the corresponding stereoscopic image can be displayed on the display panel 2. That is, the control unit 4 can display either a video image for the left eye or a video image for the right eye for each pixel of the display panel 2.

When such a stereoscopic image is viewed through a pattern displayed through a monochrome pattern, the video image for the left eye is displayed to the left eye and the video image for the right eye is displayed to the right eye, so that the stereoscopic image as a whole can be viewed in three dimensions. On the other hand, when a stereoscopic video is viewed without a pattern, the stereoscopic video may appear like a regular 2D video, or the video image for the left eye and the video image for the right eye may appear overlapping.

As will be described later, the control unit 4 may further consider derived viewpoint information or pattern information about the characteristics of the pattern displayed on the monochrome panel 1 when converting the input original image into a three-dimensional image.

Meanwhile, the control unit 4 may further include a plurality of components for performing the above-described functions. Specifically, the control unit 4 includes a viewpoint information deriving unit 40 that derives the user's viewpoint information based on the shooting information received from the camera, and a viewpoint information deriving unit 40 that determines the pattern displayed on the monochrome panel 1 based on the derived viewpoint information. It may include a first operation unit 41 and a second operation unit 42 that converts the input original image into a three-dimensional image.

Additionally, the illustrated components of the control unit 4 illustrate only essential elements for implementing the invention, and the control unit 4 may further include separate components such as a database in addition to the above components.

Figure 3 shows steps of a method for outputting a stereoscopic image according to an embodiment of the present invention.

As shown in (a) of FIG. 3, the following steps are performed by each component of the image output device, so that a three-dimensional image can be output with the pattern changing depending on the user's viewpoint.

In step S1, the control unit 4 can input the original image and basic data for outputting the stereoscopic image. First, the original image may correspond to an image to be displayed on the display panel 2 that has not been converted to a three-dimensional image. Specifically, when the original image is simply displayed on the display panel (2), the original image itself is not expressed three-dimensionally, and the original image is converted to a three-dimensional image by the second operation unit 42 and the converted three-dimensional image is displayed. The corresponding stereoscopic image can be expressed three-dimensionally.

In one embodiment of the present invention, the original image may be 3D data itself modeled in 3D, or may be data generated based on 2D image data and including color information for each pixel and depth information for the corresponding pixel.

Meanwhile, the basic data for outputting the above-described stereoscopic image may be data for determining the characteristics of the pattern. Specifically, in steps described later, the control unit 4 may determine the characteristics of the pattern according to viewpoint information derived based on shooting information generated when a camera photographs the user in real time. In other words, the characteristics of the pattern can be derived differently depending on the viewpoint information.

Meanwhile, in one embodiment of the present invention, variables other than viewpoint information may be reflected in deriving the characteristics of the pattern, and the variables may correspond to basic data input in step S1.

For example, the data may be the separation distance between the monochrome panel (1) and the display panel (2). Specifically, when the first operation unit 41 determines pattern information based on the viewpoint information, the separation distance may be further taken into consideration.

Preferably, the separation distance between the monochrome panel 1 and the display panel 2 can be pre-stored in the control unit 4 as a fixed constant value designed when manufacturing the image output device, and the characteristics of the pattern according to the pre-stored separation distance. This can be decided differently.

Alternatively, the data may be the binocular distance for the distance between the left and right eyes. Specifically, when the first operation unit 41 determines pattern information based on the viewpoint information, the pupillary distance may be further taken into consideration.

Preferably, the binocular distance can be pre-stored in the control unit 4 as a fixed constant value as the distance between the left and right eyes of a typical person, and the characteristics of the pattern can be determined differently depending on the pre-stored binocular distance.

In this way, the basic data that can determine the characteristics of the pattern input in step S1 are stored as fixed constant values, and when determining the characteristics of the pattern based on the user's viewpoint information in the step described later, the corresponding data are taken into consideration. It could be.

In step S2, the control unit 4 may derive viewpoint information for the user based on the shooting information received from the camera. Specifically, the shooting information generated by the camera can be transmitted to the viewpoint information deriving unit 40 of the control unit 4, and the user's viewpoint information can be derived by the viewpoint information deriving unit 40.

Viewpoint information may refer to information about the viewpoint of a user watching a video output device. Specifically, the viewpoint information may include information about the position and angle of both eyes of the user based on the image output device (preferably reference coordinates corresponding to the center of the plane implemented by the monochrome panel 1).

Preferably, the viewpoint information includes information about the viewing distance between the user and the image output device, the first viewing angle when the user's location is projected on the first plane, and the second viewing angle when the user's position is projected on the second plane. can do.

As a result, the viewpoint information may include information about the distance and angle from both eyes of the user to the image output device when both eyes are looking at the image output device.

Meanwhile, in one embodiment of the present invention, the reference coordinates of the image output device may correspond to the center coordinates on the first plane implemented by the monochrome panel 1.

In step S3, the control unit 4 may derive pattern information about the characteristics of the pattern based on the derived viewpoint information. Specifically, the first operation unit 41 of the control unit 4 derives values for each of the start coordinates of the pattern, the slope of the pattern, the width of the first area, and the width of the second area based on the derived viewpoint information. Thus, a pattern corresponding to the derived value can be output from the monochrome panel (1).

That is, the first operation unit 41 changes the characteristics of the monochrome panel 1 according to the viewpoint of the user viewing the image output device, thereby adjusting the stereoscopic viewing angle and viewing distance at which the stereoscopic image is displayed in three dimensions corresponding to the user's viewpoint. It can be adjusted.

In step S4.1, the second operation unit 42 of the control unit 4 can convert the original image input in step S1 into a three-dimensional image. Specifically, the second operation unit 42 can convert the input original image into a three-dimensional image including a video image for the left eye and a video image for the right eye for each pixel.

In step S5, the control unit 4 can display a pattern according to the derived pattern information and a converted three-dimensional image on the monochrome panel 1 and the display panel 2, respectively. Specifically, a pattern corresponding to the pattern information derived by the first operation unit 41 is displayed on the monochrome panel 1, and a three-dimensional image converted by the second operation unit 42 is displayed on the display panel 2, When a user watches a 3D image through a pattern displayed on a Monochrome panel, the 3D image can be expressed in three dimensions.

Figure 3(b) shows components of an image output device that performs the above-described steps. As shown, the shooting information captured by the camera is transmitted to the viewpoint information deriving unit 40 of the control unit 4, so that the user's viewpoint information can be derived.

In addition, the first operation unit 41 derives pattern information about the characteristics of the pattern displayed on the monochrome panel 1 based on the derived viewpoint information, and a pattern corresponding to the pattern information is displayed on the monochrome panel 1. You can do it.

Meanwhile, the second operation unit 42 can convert the input original image into a three-dimensional image displayed on the display panel 2 and display the corresponding three-dimensional image on the display panel 2.

As such, in one embodiment of the present invention, by including a monochrome panel (1) with a variable pattern rather than a fixed print attached PB sheet, and a camera that photographs the user, a three-dimensional image is created in response to the user's viewpoint. You can adjust the stereoscopic viewing angle and viewing distance.

Figure 4 shows the viewpoint information derivation step according to an embodiment of the present invention.

As shown in FIG. 4, the control unit 4 stores the separation distance between the monochrome panel 1 and the display panel 2, and the pattern information deriving step includes the viewpoint information and the separation distance. By additionally considering, the pattern information can be derived.

In addition, the control unit 4 stores the pupillary distance, which is the distance between the user's left and right eyes, and the pattern determination step can derive the pattern information based on the viewpoint information and the pupillary distance.

As shown in Figure 4, the image output device of the present invention may include a monochrome panel 1 disposed on the front and a display panel 2 disposed on the rear. Specifically, the monochrome panel 1 and the display panel 2 are spaced apart at a predetermined interval, and the corresponding separation distance d ₂ can be stored in the control unit 4. Preferably, when the first operation unit 41 derives pattern information based on viewpoint information, information on the separation distance (d ₂ ) may be taken into consideration.

Meanwhile, the image output device includes a camera that photographs a user located in front of the image output device, and the viewpoint information deriving unit 40 derives viewpoint information for the user based on the photographed information received from the camera. You can.

Specifically, the viewpoint information may include information about the distance and angle of the user's location based on the reference coordinate, which is the center coordinate on the plane implemented by the monochrome panel 1. Preferably, the viewpoint information may include information about the distance and angle of the user's both eyes based on the reference coordinates.

In one embodiment of the present invention, the viewpoint information deriving unit 40 may derive viewpoint information using a known computer vision-based technology. Specifically, the viewpoint information deriving unit 40 can recognize the user or the user's face or both eyes of the user from the shooting information received from the camera, and derive coordinate information about the user's face or both eyes. The viewpoint information deriving unit 40 can derive coordinate information for both eyes of the user and information about the distance and angle from preset reference coordinates on the plane implemented by the monochrome panel 1.

In one embodiment of the present invention, the viewpoint information may include the viewing distance between the user and the image output device. Specifically, the viewing distance may mean the distance between the user, preferably both eyes of the user, and the reference coordinate corresponding to the center coordinate of the monochrome panel 1. That is, the viewing distance in FIG. 4 may correspond to the distance (d ₁ ) between the user's both eyes (O) from the reference coordinates on the plane implemented by the monochrome panel (1).

Intuitively, the viewing distance can refer to information about how far the user's both eyes (O) are from the monochrome panel (1) in a straight line.

In one embodiment of the present invention, the viewpoint information is the first viewing angle ( may include. Specifically, the first viewing angle is when the user's position (preferably coordinate information for both eyes of the user) is projected onto a virtual first plane parallel to the plane implemented by the monochrome panel 1, the reference coordinates and the user It may be the angle formed by the position of .

Preferably, the first plane may be a virtual plane identical to the plane implemented by the monochrome panel 1. That is, the first plane is a virtual plane that includes reference coordinates and is implemented by the monochrome panel 1. When the user's position is projected on the first plane, the angle between the reference coordinates and the projected user's coordinate information is the first viewing angle. ( ) can be derived as

In another embodiment of the present invention, the first viewing angle ( ) may be derived for each of the user's left and right eyes. That is, the viewpoint information deriving unit 40 derives coordinate information for each of the left and right eyes, projects the coordinate information for each of the left and right eyes onto the first plane, and derives the first viewing angle for each of the left and right eyes. You can.

In one embodiment of the present invention, the viewpoint information is the second viewing angle ( may include. Specifically, the second viewing angle is the angle formed between the reference coordinates and the user's position when the user's position (preferably coordinate information for both eyes of the user) is projected onto a virtual second plane orthogonal to the first plane. You can.

Preferably, the second plane may be a virtual plane orthogonal to the plane implemented by the monochrome panel 1. That is, the second plane is a virtual plane that includes reference coordinates and is orthogonal to the monochrome panel (1). When the user's position is projected on the second plane, the angle between the reference coordinates and the projected user's coordinate information is the second viewing angle. ( ) can be derived as

In another embodiment of the present invention, the second viewing angle ( ) may be derived for each of the user's left and right eyes. That is, the viewpoint information deriving unit 40 derives coordinate information for each of the left and right eyes, projects the coordinate information for each of the left and right eyes onto the second plane, and derives a second viewing angle for each of the left and right eyes. You can.

In this way, the viewpoint information deriving unit 40 uses computer vision-based technology to derive coordinate information for both eyes of the user, and when the separation distance, first viewing angle, and second viewing angle for the coordinate information are derived, , the angle and distance between the user's both eyes can be derived based on the reference coordinates.

As shown in (b) of FIG. 4, the viewpoint information deriving unit 40 calculates the viewing distance, first viewing angle, and second viewing angle between the user's location and the reference coordinates of the image output device based on the shooting information from the camera. Point-in-time information including information about can be derived.

Figure 5 shows the pattern information derivation step according to an embodiment of the present invention.

As shown in FIG. 5, the pattern is a pattern in which first and second areas having different transmittances are alternately arranged at a predetermined inclination, and the pattern information deriving step is based on the viewpoint information. The pattern output from the monochrome panel 1 can be determined by deriving pattern information including one or more of the starting coordinates, the slope of the pattern, the width of the first area, and the width of the second area.

In addition, in contrast to the fact that the viewing angle limit at which a stereoscopic image is displayed stereoscopically is fixed when the pattern is fixed, the image output device allows the pattern of the monochrome panel 1 to vary depending on the user's viewpoint to display the stereoscopic image from various viewpoints. can be displayed three-dimensionally.

Figure 5(a) shows a pattern displayed on the monochrome panel 1. As shown, the pattern has first and second areas with different transmittances having a predetermined slope ( ) can be placed alternately and repeatedly. Preferably, the pattern has a first area (A1) that has a first width and blocks light, and a second area (A2) that has a second width and transmits light with a predetermined slope ( ) and may be a shape that is repeatedly arranged.

In addition, the starting position of the pattern (l ₃ ) is the position of the first area (A1) that is placed first among the first area (A1) and the second area (A2) that are repeatedly placed based on the left side of the monochrome panel (1). It can mean.

That is, the starting position of the pattern (l ₃ ), the slope of the pattern ( ), when the width (l ₁ ) of the first area (A1) and the width (l ₂ ) of the second area (A2) are determined, the characteristics of the pattern displayed on the Monochrome panel can be determined.

Meanwhile, as described above, the pattern displayed on the monochrome panel 1 can vary depending on the user's viewpoint, and more specifically, each element that determines the characteristics of the pattern (the starting position of the pattern (l ₃ ) , the slope of the pattern ( ), the width (l ₁ ) of the first area (A1), and the width (l ₂ ) of the second area (A2) may be variable.

Specifically, as shown in (b) of FIG. 5, the first operation unit 41 determines the start position of the pattern (l) based on viewpoint information including information on the viewing distance, first viewing angle, and second viewing angle. ₃ ), the slope of the pattern ( ), the width (l ₁ ) of the first area (A1), and the width (l ₂ ) of the second area (A2) can be determined to derive pattern information about the characteristics of the pattern.

Specifically, the first operation unit 41 generates a pattern based on viewpoint information including information on the viewing distance, first viewing angle, and second viewing angle between the user and the image output device derived by the viewpoint information deriving unit 40. Starting position (l ₃ ), slope of the pattern ( ), the width (l ₁ ) of the first area (A1), and the width (l ₂ ) of the second area (A2) can be derived.

Meanwhile, in one embodiment of the present invention, each variable of pattern information can be derived based on viewpoint information through a known technique for implementing a parallax barrier depending on viewpoint.

For example, in [Figure 1] below, the distance b between the first areas can be determined according to [Equation 1] below, and the viewing distance z can be determined according to [Equation 2] below.

[Figure 1]

[Equation 1]

[Equation 2]

Here, b is the distance between the first areas, g is the distance between the monochrome panel (1) and the display panel (2), z is the viewing distance, i is the pixel size of the display panel (2), and e is the three-dimensional image. It means the distance from which it can be seen.

That is, b may correspond to the sum of the widths of each of the first and second areas, and when b is determined and the first and second areas each have the same width, each of the first and second areas The width for can be determined as b/2.

As described above, the conventional fixed print attachable PB sheet that implements a parallax barrier has the pattern itself printed, so it is not possible to adjust the stereoscopic viewing angle and viewing distance at which the stereoscopic image is displayed in three dimensions in response to the user's viewpoint.

On the other hand, the monochrome panel (1) implementing the parallax barrier in the present invention variably displays patterns by the first operation unit (41), thereby adjusting the three-dimensional viewing angle and viewing distance corresponding to the user's viewpoint. You can.

Figure 6 shows an image conversion step according to an embodiment of the present invention.

As shown in FIG. 6, in the image conversion step, the original image is converted to the display panel by the second operation unit 42 based on one or more of the viewpoint information and pattern information that varies depending on the user's viewpoint. It can be converted to a stereoscopic image output in (2).

In addition, the original image is 3D data modeled in 3D, or the color of each pixel is generated by inputting the 2D original image into a deep learning-based machine learning model that can infer depth information for each pixel from 2D data. It is data in which information and depth information are stored, and the second operation unit 42 converts the original image into a three-dimensional image in which the images shown to the left and right eyes are different when viewed through the pattern. It can be converted.

As shown in FIG. 6, the second operation unit 42 can convert the input original image into a three-dimensional image displayed on the display panel 2. As described above, the original image may be original data for a three-dimensional image that is to be expressed three-dimensionally in an image output device.

More specifically, when the pattern displayed on the monochrome panel (1) on the front of the video output device is transmitted and the stereoscopic image displayed on the display panel (2) on the back of the video output device is viewed, the stereoscopic image can be viewed in three dimensions. there is.

Meanwhile, the original video may be 3D data that has been modeled in 3D itself.

Meanwhile, in another embodiment of the present invention, the original image may be data in which 2D data and depth information data generated based on the 2D data are matched. Specifically, the original image may be data in which 2D data including color information for each pixel and depth information including depth information inferred for each pixel based on the 2D data are matched.

In one embodiment of the present invention, depth information data may be generated by a deep learning-based machine learning model that infers depth information for each pixel of 2D data. Specifically, the machine learning model is trained to receive 2D data and infer depth information for each pixel of the 2D data. By inputting 2D data into the machine learning model, depth information can be inferred for each pixel of the 2D data. You can.

In one embodiment of the present invention, depth information data may be expressed in the form of a disparity map.

Meanwhile, the second operation unit 42 may include a machine learning model on its own, receives a 2D original image, generates depth information data from the 2D original image, and combines the input 2D original image with the generated A three-dimensional image can be generated based on depth information data.

In one embodiment of the present invention, the second operation unit 42 can convert the original image into a three-dimensional image through a known algorithm or software. Specifically, the original video can be converted to a three-dimensional video through public services such as TriDef 3D and iZ3D Driver.

Figure 7 shows the stereoscopic viewing angle and viewing distance of an image output device according to an embodiment of the present invention.

As shown in FIG. 7, the control unit 4 includes a simulation unit that simulates the stereoscopic viewing angle and viewing distance at which the stereoscopic image is displayed based on the pattern and stereoscopic image currently displayed by the image output device, When it is determined whether the user's viewpoint information satisfies the stereoscopic viewing angle and viewing distance according to the pattern displayed by the current video output device and the stereoscopic image, and the user is determined to be located in an area corresponding to the stereoscopic viewing angle and viewing distance. In the pattern information deriving step based on the user's viewpoint information, the pattern according to the existing pattern information is maintained without performing the step, and if it is determined that the user is not located in an area that satisfies the stereoscopic viewing angle and viewing distance, the user By re-performing the pattern information derivation step based on the viewpoint information, the pattern displayed on the monochrome panel 1 can be changed.

Figure 7(a) shows the stereoscopic viewing angle and viewing distance at which the stereoscopic image is displayed when the image output device of the present invention displays a stereoscopic image. As mentioned above, stereoscopic images are not expressed stereoscopically at all viewing angles and viewing distances, and when viewed within an area that satisfies the stereoscopic viewing angles and viewing distances preset according to the pattern, the stereoscopic images are expressed stereoscopically. It can be.

That is, when the object O1 located in the area B1 in (a) of FIG. 7 views a 3D image, the 3D image may be displayed in three dimensions. Meanwhile, when an object O2 that is not located in the area B1 views a stereoscopic image, the stereoscopic image may not be displayed stereoscopically.

On the other hand, when the parallax barrier is implemented as a fixed printed attachable PB sheet as in the past, or when the pattern displayed on the monochrome panel (1) is fixed, the area that satisfies the three-dimensional viewing angle and viewing distance at which the three-dimensional image is expressed three-dimensionally (B1) can be fixed.

Figure 7(b) shows the stereoscopic viewing angle and viewing distance at which the stereoscopic image is expressed stereoscopically when the image output device of the present invention displays a stereoscopic image in a pattern different from that of Figure 7(a). As shown, when the pattern is variable, the stereoscopic viewing angle and viewing distance at which the stereoscopic image is displayed can also vary. Specifically, the three-dimensional viewing angle and viewing distance expressed in three dimensions can be determined by various variables, and more specifically, each element that determines the characteristics of the pattern (pattern start coordinates, pattern slope, width of the first area, second area) width), separation distance, etc.

For example, the stereoscopic viewing angle can become wider as the separation distance becomes thinner.

That is, in (b) of FIG. 7, the image output device changes the pattern according to the viewpoint of the photographed object O2, so that the area where the three-dimensional image is viewed three-dimensionally can change from B1 to B2. That is, the object O2, which is not located in the existing area B1, is located in the area B1 as the pattern changes, so that the three-dimensional image can be viewed in three dimensions.

Meanwhile, in one embodiment of the present invention, the pattern does not change unconditionally in response to all locations of the user. In other words, the image output device can change the pattern when the user's viewpoint information exists in a location where the three-dimensional image is not expressed three-dimensionally. For example, in (a) of FIG. 7, when the user is located at O1, the pattern does not change, and when the user is located at O2, the pattern may change.

Preferably, the image output device further includes a simulation unit that automatically simulates the three-dimensional viewing angle and viewing distance at which the three-dimensional image is viewed in three dimensions, based on the pattern and three-dimensional image displayed in the current image output device, and the derived viewpoint information. If the current user's viewpoint information does not fall within the area of the stereoscopic viewing angle and viewing distance derived by the simulation unit, the pattern information derivation step and the image conversion step can be performed to change the pattern and stereoscopic image.

As such, in one embodiment of the present invention, a viewpoint information deriving step, a pattern information deriving step; and whether the image conversion step is repeatedly performed and the user's viewpoint information derived in the current round is located in an area that satisfies the stereoscopic viewing angle and viewing distance according to the pattern displayed by the pattern information derivation step derived in the previous round. Determining and, if satisfied, pattern information derivation step; and maintaining and displaying a pattern according to the existing pattern information without re-performing the image conversion step, and if not satisfied, a pattern information deriving step; And by re-performing the image conversion step, new pattern information can be derived and the existing pattern can be displayed in a variable manner.

Figure 8 shows steps of a method for outputting a stereoscopic image according to another embodiment of the present invention.

As shown in Figure 8, in another embodiment of the present invention, the original image can be converted into a three-dimensional image by further reflecting one or more of the derived user's viewpoint information or pattern information.

Specifically, in step S4.2, the second operation unit 42 of the control unit 4 converts the original image input in step S1 into a three-dimensional image, using one of the user's viewpoint information and pattern information derived in steps S2 and S3. The above can be reflected.

Preferably, the second operation unit 42 can convert the original image into a three-dimensional image according to a pattern corresponding to the pattern information derived by the first operation unit 41.

In the conventional case, when a fixed print attached PB sheet was designed, an algorithm was performed to convert the original image into a three-dimensional image by reflecting the pattern of the PB sheet.

In another embodiment of the present invention, an algorithm can be performed to convert the original image into a three-dimensional image by changing the pattern as the pattern changes depending on the user's viewpoint.

As such, in another embodiment of the present invention, the three-dimensional image displayed on the display panel 2 can be varied by reflecting the user's viewpoint information or pattern information derived based on the user's viewpoint information.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent. Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (6)

A method of outputting a three-dimensional image using an image output device including a monochrome panel whose pattern varies depending on the user's viewpoint, comprising:
The video output device is,
Monochrome panel that displays a changing pattern depending on the user's viewpoint;
A display panel disposed on the back of the monochrome panel and displaying a three-dimensional image by outputting a video image for either the left eye or the right eye for each pixel;
A camera that takes images of the user; and
a first operation unit that determines a pattern displayed on the monochrome panel based on the user's viewpoint information derived based on the shooting information of the camera; and a second operation unit that converts the original image into a three-dimensional image displayed on the display panel based on the user's viewpoint information.
The method for outputting the three-dimensional image is,
By the control unit, based on the shooting information of the camera, the viewing distance between the user and the image output device, the first viewing angle when the user's position is projected on the first plane, and the viewing distance when the user's position is projected on the second plane. A viewpoint information deriving step of deriving viewpoint information including one or more of the second viewing angles;
A pattern information derivation step of deriving pattern information about the characteristics of a pattern output from a monochrome panel based on the viewpoint information by the first operation unit;
An image conversion step of converting the original image into a three-dimensional image output from the display panel by the second operation unit based on the viewpoint information and pattern information,
The first plane is parallel to the plane implemented by the monochrome panel, and the second plane is perpendicular to the first plane,
If the control unit determines that the user is not located in an area that satisfies the stereoscopic viewing angle and viewing distance at which the stereoscopic image is displayed in three dimensions, the control unit re-performs the pattern information derivation step,
The video output device is,
When the pattern is fixed, the viewing angle limit at which the stereoscopic image is displayed is fixed. In contrast, when the user's viewpoint information exists in a location where the stereoscopic image is not displayed stereoscopically, the pattern is varied and the stereoscopic image is displayed from various viewpoints. A method of outputting a three-dimensional image that can display three-dimensionally.
In claim 1,
The pattern is,
First and second regions having different transmittances are alternately arranged at a predetermined inclination,
The pattern information derivation step is,
Based on the viewpoint information, pattern information including one or more of the starting coordinates of the pattern, the slope of the pattern, the width of the first area, and the width of the second area are derived, and the pattern is output from the monochrome panel. Determining the output method of a three-dimensional image.
In claim 2,
In the control unit, the separation distance between the monochrome panel and the display panel is stored,
The pattern information derivation step is,
A method of outputting a three-dimensional image, in which the pattern information is derived by additionally considering the viewpoint information and the separation distance.
In claim 1,
The video conversion step is,
A three-dimensional image output method that converts the original image into a three-dimensional image output from the display panel based on one or more of the viewpoint information and pattern information that varies depending on the user's viewpoint, by the second operation unit.
delete In claim 1,
The original video is,
It is 3D data modeled in 3D, or
2D data containing color information for each pixel, and depth containing depth information for each pixel, created by inputting the 2D data into a deep learning-based learned machine learning model that can infer depth information for each pixel of the 2D data. It is data in which information data is matched,
The second operation unit is,
A three-dimensional image output method that converts the original image into a stereoscopic image in which the original image is displayed three-dimensionally because the images shown to the left and right eyes are different when viewed through the pattern.

Images