KR102021363B1 - Curved display apparatus and operation method thereof - Google Patents

Abstract

According to an embodiment of the present invention, a curved display device and a method of operating the same are provided. The apparatus may include a camera unit for photographing a user located in front of the curved display device to generate a user image; Recognizing a user from the user image, generating a first position coordinate of the user based on the position of the camera unit, converting the first position coordinate to a second position coordinate based on the center of curvature of the curved display unit, A controller configured to determine a display image for displaying to the user based on the second position coordinates and to determine a display area for displaying the display image among the curved display units; And a curved display configured to display the display image on the display area.

Description

CURVED DISPLAY APPARATUS AND OPERATION METHOD THEREOF}

The present invention relates to a curved display device and a method of operating the same, and more particularly, to a curved display device for implementing a pseudo hologram using a curved display and a method of operating the same.

The present invention is derived from the research carried out with the support of the future industry leading technology development project of the Ministry of Trade, Industry and Energy of Korea Industry and Technology Evaluation and Management Administration [project unique number: 10042418, research title: "60〃 or larger screen, transparent UD picture quality Development of UI and user interaction technology for visual space tracking for flexible display "].

1 illustrates a conventional hologram implementation. The hologram refers to a three-dimensional image of a three-dimensional image, and may be photographed by using a holography technology that records and reproduces stereoscopic information by using an interference phenomenon of light generated by two laser beams.

Holograms can be classified into three categories: "Analog Holograms, which are photographs that take a stereoscopic image of an object", "Digital Holograms that digitally reproduce light reflected on an object", and "Similar holograms such as semi-transmissive screen projection images." have.

Among the holograms, an “analog hologram” may produce a still stereoscopic image (still image) by using interference of laser light and light, but may not provide a moving stereoscopic image (ie, a moving image).

Complementing this, a "digital hologram" has been proposed (see (a) of FIG. 1). According to this, it is possible to express a moving image, but the stereoscopic image is recognized only at the point of focus using the constructive interference of light or the lens. There was a limit.

In addition, the semi-transmissive screen projection image is a similar hologram that uses a transflective screen mainly used in performances, and it uses an optical illusion that looks like a floating image in space, but it is possible to provide a stereoscopic image with different sides depending on the viewing position. No (see FIG. 1B).

Therefore, there is a need for a technique capable of solving such a problem and providing a stereoscopic moving picture simply.

An object of the present invention is to provide a curved display device for implementing a pseudo hologram using a curved display and a method of operating the same.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following descriptions.

According to one embodiment of the invention, a curved display device is provided. The apparatus may include a camera unit for photographing a user located in front of the curved display device to generate a user image; Recognizing a user from the user image, generating a first position coordinate of the user based on the position of the camera unit, converting the first position coordinate to a second position coordinate based on the center of curvature of the curved display unit, A controller configured to determine a display image for displaying to the user based on the second position coordinates and to determine a display area for displaying the display image among the curved display units; And a curved display configured to display the display image on the display area.

Preferably, the camera unit may be a depth camera.

Also, preferably, the curved display may have the same curvature.

Preferably, the controller may determine the display image among the multi-angle images based on the angle between the second position coordinates and the center of curvature.

Preferably, the control unit estimates the gaze direction of the user extending the second position coordinates and the center of curvature, determines a display center where the gaze direction and the curved display unit contact each other, and as the display center. As a reference, the display area may be determined such that a left / right boundary of the display area corresponds to angle of view information of the display image.

Preferably, the control unit corrects the display image by mapping the display image to correspond to the display area based on the curvature of the curved display unit, and the curved display unit corrects the corrected display in the display area. An image can be displayed.

Also, preferably, the camera unit may be located outside the curved display unit.

According to an embodiment of the present invention, a method of operating a curved display device is provided. The method may include generating a user image by photographing a user located in front of the curved display device using a camera unit; Recognizing a user from the user image and generating first position coordinates of the user based on the position of the camera unit; Converting the first position coordinates into second position coordinates based on a center of curvature of the curved display unit; Determining a display image for displaying to the user based on the second location coordinates; Determining a display area for displaying the display image among the curved display units; And displaying the display image on the display area of the curved display unit.

Preferably, the camera unit may be a depth camera.

Also, preferably, the curved display may have the same curvature.

Also, preferably, the display image may be determined from among multi-angle images based on an angle between the second position coordinate and the center of curvature.

Preferably, the determining of the display area may include estimating a gaze direction of the user extending the second position coordinate and the center of curvature; Determining a display center in which the line of sight and the curved display portion contact each other; And determining the display area based on the display center so that the left / right boundary of the display area corresponds to the angle of view information of the display image.

Preferably, the method may further include correcting the display image by mapping the display image to correspond to the display area based on the curvature of the curved display unit, and the displaying may include correcting the display image. Can be performed.

In addition, the method may further include detecting a change in the position of the user, and if the change in the position is detected, generating the first coordinate and displaying the display image may be performed again.

According to an embodiment of the present invention, a computer-readable recording medium having a program recorded thereon for performing a method of operating a curved display device is provided.

According to the present invention, the image display area on the curved display is determined according to the user's gaze direction, and the image having an angle corresponding to the user's gaze is selected and displayed from the multi-angle image, thereby displaying a three-dimensional image such as a hologram. Can be provided.

In addition, according to the present invention, by converting the position coordinates of the user based on the center of curvature of the curved display unit, it is possible to prevent the image distortion due to the position difference of the sensing unit for detecting the position / gaze of the user.

In addition, according to the present invention, by setting the boundary of the image display area in consideration of the angle of view of the display image, it is possible to prevent the distortion of the image generated in the process of displaying the image on the curved display.

Further, according to the present invention, the planar image may be mapped according to the curvature of the curved display unit to correct the display image of the curved surface suitable for the curved display unit.

BRIEF DESCRIPTION OF THE DRAWINGS In order to better understand the drawings cited in the detailed description of the invention, a brief description of each drawing is provided.
1 illustrates a conventional hologram implementation.
2 illustrates a curved display device according to an embodiment of the present invention.
3 illustrates a process of manufacturing a multi-angle image according to an embodiment of the present invention.
4 illustrates a method of operating a curved display device according to an exemplary embodiment of the present invention.
5 to 7 show an example of the operation of the curved display device according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the embodiments of the present invention, if it is determined that the detailed description of the related well-known configuration or function interferes with the understanding of the embodiments of the present invention, the detailed description thereof will be omitted. In addition, embodiments of the present invention will be described below, but the technical spirit of the present invention is not limited thereto and may be variously modified and modified by those skilled in the art.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "indirectly connected" with another element in between. . Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise. In addition, in describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms.

2 illustrates a curved display device according to an embodiment of the present invention.

2, the apparatus 200 may include a communication unit 210, a storage unit 220, a camera unit 230, a curved display unit 240, and a control unit 250.

The communication unit 210 is provided for a direct connection with an external device (various user terminals, servers, etc. communicating with the curved display device 200) or a network, and may be a wired and / or wireless communication unit 210. . In detail, the communication unit 210 transmits data from the storage unit 220, the camera unit 230, the curved display unit 240, the control unit 250, or the like, or wires the data from the outside. Wireless reception may be transmitted to the control unit 250 or stored in the storage unit 220. The data may include, for example, the position information of the camera unit 230, the position information and the center of curvature of the curved display unit 240, the first position coordinates, the second position coordinates, the gaze direction, the display image, the display area, and the multi. An angle image may be included.

The communication unit 210 may include a LAN, Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), Wireless Broadband Internet (WiBro), Radio Frequency (RF) communication, Wireless LAN, and Wi-Fi. Fidelity, NFC (Near Field Communication), Bluetooth, infrared communication can be communicated through. However, this is merely exemplary, and various wired and wireless communication technologies applicable in the art may be used according to the embodiment to which the present invention is applied.

The storage 220 may store an input / output signal or data corresponding to the operation of the curved display apparatus 200 under the control of the controller 250. In addition, the storage 220 stores a program and data necessary for the operation of the curved display apparatus 200. For example, the storage unit 220 may include location information of the camera unit 230, location information and curvature center of the curved display unit 240, first location coordinates, second location coordinates, gaze direction, display image, and display area. , Information about a multi-angle video and the like can be stored.

The storage unit 220 may include a hard disk drive (HDD), a read only memory (ROM), a random access memory (RAM), an electrically erasable and programmable read only memory (EEPROM), and a flash memory, as known to those skilled in the art. Implemented in various types of storage devices that can input and output information such as flash memory, Compact Flash (CF) card, Secure Digital (SD) card, Smart Media (SM) card, MMC (Multimedia) card, or Memory Stick It may be provided in the curved display device 200, or may be provided in a separate device.

The camera unit 230 may photograph a user adjacent to the curved display apparatus 200 to generate a user image. The camera unit 230 is preferably located outside the curved display unit 240 in order to facilitate the shooting of the user, but is not limited thereto. The camera unit 230 may be disposed in various places where the shooting of the user is possible. The camera unit 230 may be a depth camera, but as an example, various configurations may be applied according to an embodiment to which the present invention is applied.

The curved display unit 240 may display an image or data provided from the controller 250 on a screen, and provide a user interface for input of a user required for the operation of the curved display apparatus 200. In detail, the curved display unit 240 may display a display image that is a still image or a moving image received from the controller 250, and display the display image within a range of the display region of the display region. The display area may have a different position or size depending on the display image.

The curved display unit 240 may be configured in one or a plurality, and each curved display unit 240 may be a transparent display unit or an opaque display unit. The curved display unit 240 has a display bent at a predetermined curvature rather than a conventional flat screen. At this time, the curvature of the curved surface is preferably constant, but is not limited thereto, and may have a curved surface by various curvatures.

The display unit 240 may be formed of a liquid crystal display (LCD), organic light emitting diodes (OLEDs), active matrix organic light emitting diodes (AMOLEDs), or the like.

The controller 250 may control an overall operation of the curved display apparatus 200 and a signal flow between internal components of the curved display apparatus 200, and may perform a data processing function of processing data. In particular, in the present invention, the controller 250 recognizes the user, in particular, the user's face or head from the user image, and generates the first position coordinates of the user based on the position of the camera unit 230. In this case, the display apparatus converts the second position coordinates based on the center of curvature of the curved display unit 240 to be used to determine the display image from the multi-angle image. In addition, the controller 250 may determine a display area for the display image of the display of the curved display unit 240, and display the display image to the user through the curved display unit 240.

The operation of the controller 250 will be described in more detail with reference to FIGS. 3 to 7.

3 illustrates a process of manufacturing a multi-angle image according to an embodiment of the present invention.

As illustrated, the multi angle image 330 refers to a work in which the object 320 is photographed in a state in which the plurality of cameras 310 are disposed at different positions. Each camera 310 is a multi-angle positioning around the object 320, it is preferable to vary the viewpoint of the camera 310, but is not limited thereto.

The multi-angle image 330 may be composed of a plurality of images photographing the object 320 at various angles, and each image may be a still image or a moving image. In addition, each image may include location information or angle information of the camera 310 capturing the corresponding image.

4 illustrates a method of operating a curved display device according to an embodiment of the present invention, and FIGS. 5 to 7 illustrate examples of an operation of the curved display device according to an embodiment of the present invention.

In operation S410, the camera unit 230 may photograph the user 530 to generate a user image. The camera unit 230 may be located outside the curved display unit 240 to facilitate the capturing of the user 530 that is outside the curved display unit 240 (particularly, the front side). In order to facilitate generation of the coordinates 540, step S410 may be performed using a depth camera.

In operation S420, the controller 250 may generate the first position coordinates 540. Here, the first position coordinate 540 relates to the position information of the user 530 based on the position of the camera unit 230. In detail, the control unit 250 determines that the user 530 (particularly, the user) is from the user image. And the like, and generate information on the distance and direction between the recognized user 530 and the camera unit 230.

In operation S430, the controller 250 may convert the first position coordinates 540 into the second position coordinates 550. Here, the second position coordinate 550 relates to the position information of the user 530 based on the center of curvature 510 of the curved display unit 240, and specifically, the first position coordinate 540 of step S420. Based on the positional relationship 520 between the center of curvature 510 and the camera unit 230, the first position coordinates 540 may be converted into the second position coordinates 550.

In operation S440, the display image 610 may be determined. The display image 610 refers to an image to be displayed to the user 530 among the multi-angle images, and may be determined based on the second position coordinate 550 in step S430.

In detail, in operation S440, the angle of each image of the multi-angle image and the angle between the second position coordinates 550 and the center of curvature 510 may be compared to display an image having a corresponding angle among the multi-angle images. It can be performed by determining. For example, the display image 610 may be an image having the same angle as the angle between the second position coordinate 550 and the center of curvature 510 among the multi-angle images. On the contrary, if there is no image having the same angle among the multi-angle images, the image having the closest angle to the angle between the second position coordinate 550 and the center of curvature 510 among the multi-angle images is displayed. Can be determined.

In operation S450, the controller 250 may determine a display area. Herein, the display area may mean an area which is determined to display the display image 610 as at least a part of the display area of the curved display unit 240, and is based on the display center 560 and the display center 560. It may include information about the display range.

In detail, in operation S450, the control unit 250 estimates the gaze direction of the user 530 extending the second position coordinate 550 and the center of curvature 510, and the gaze direction and the curved display unit 240 The display center 560 is determined to be in contact with the display center 560, and the display range (that is, the left / right boundaries 630 and 640 of the display area) is determined based on the display center 560. It may be performed by determining the display area to correspond to.

As a result, when the user 530 looks at the curved display device 200, the display image 610 may be positioned at the center of the gaze of the user 530. In addition, the display area may be determined according to the angle of view information of the display image 610, thereby preventing distortion of the image while the display image 610, which is a planar image, is displayed on a curved surface.

In operation S460, the curved display unit 240 may display the display image 610 on the display area of the curved display unit 240. Accordingly, by displaying the image of the corresponding viewpoint in the corresponding area according to the position of the user 530 shouting outside the curved display unit 240 and the gaze direction based thereon, the user 530 may not be able to view the stereoscopic image of the hologram. You can experience the same effect.

In one embodiment, the method 400 further comprises the step of the controller 250 correcting the display image 610 by mapping the display image 610 to correspond to the display area based on the curvature of the curved display unit 240. It may include. For example, one-to-one (1: 1) mapping between the display image 610 of the plane and the display image 610 of the curved surface is performed on the image of the center region corresponding to the display center 560 among the display images 610 of the plane. The image of the peripheral area facing the left / right boundary in the plane display image 610 is one-to-many (1: multi) mapping between the plane display image 610 and the curved display image 610. You can correct it. Through this, the display image 610 of the plane constituting the multi-angle image may be corrected to the curved display image 610 suitable for the curved display unit 240, and step S460 may be performed on the corrected display image 610. Can be.

In one embodiment, the method 400 may further include the control unit 250 detecting a change in position of the user 530. The step may be performed by the controller 250 analyzing the user image to determine the position change or movement of the user 530. When the position change is detected by the above step, at least one of steps S420 to S460 may be re-executed based on the changed location of the user 530.

Meanwhile, various embodiments described herein may be implemented by hardware, middleware, microcode, software, and / or combinations thereof. For example, various embodiments may include one or more application specific semiconductors (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs). ), Processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions presented herein, or a combination thereof.

Also, for example, various embodiments may be embedded in or encoded on a computer-readable medium containing instructions. Instructions embedded in or encoded on a computer-readable medium may cause a programmable processor or other processor to perform a method, for example, when the instructions are executed. Computer-readable media includes computer storage media, and computer storage media may be any available media that can be accessed by a computer. For example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage media, magnetic disk storage media, or other magnetic storage devices.

Such hardware, software, firmware, etc. may be implemented within the same device or within separate devices to support the various operations and functions described herein. In addition, the components, units, modules, components, etc., described herein as "parts" may be implemented separately or separately as discrete but interoperable logic devices. The depiction of different features for modules, units, etc. is intended to highlight different functional embodiments and does not necessarily mean that they must be realized by individual hardware or software components. Rather, functionality associated with one or more modules or units may be performed by separate hardware or software components or integrated into common or separate hardware or software components.

Although the operations are shown in the drawings in a specific order, it should not be understood that these operations are performed in the specific order shown, or sequential order, to achieve the desired result, or that all illustrated actions need to be performed. . In some circumstances, multitasking and parallel processing may be advantageous. Moreover, the division of various components in the above-described embodiments should not be understood as requiring such division in all embodiments, and the described components are generally integrated together into a single software product or packaged into multiple software products. It should be understood that it can.

As described above, optimal embodiments have been disclosed in the drawings and the specification. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not intended to limit the scope of the invention as defined in the claims or the claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (15)

An operating method of a curved display device implementing a hologram through a curved display unit having a curved surface facing forward,
Generating a user image by photographing a user located in front of the curved display apparatus using a camera unit;
Recognizing the user from the user image and generating first location coordinates of the user based on the location of the camera unit;
Converting the first position coordinates into second position coordinates based on a center of curvature of the curved display unit;
Determining a display image for displaying to the user based on the second location coordinates;
Determining a display area for displaying the display image among the curved display units; And
Displaying the display image on the display area of the curved display unit;
In the determining of the display image, an image of an angle corresponding to or closest to an angle between the second position coordinate and the center of curvature among the multi-angle images is determined as the display image,
The determining of the display area may include:
Estimating a gaze direction of the user extending the second position coordinates and the center of curvature;
Determining a display center in which a normal line direction is in contact with the curved display unit; And
Determining the display area for displaying the display image based on the display center. The method of claim 1,
And the camera portion is a depth camera. The method of claim 1,
And the curved display has the same curvature. delete The method of claim 1,
In the determining of the display area for displaying the display image based on the display center,
And determining the display area based on the display center so that the left / right boundary of the display area corresponds to the angle of view information of the display image. The method of claim 5,
Correcting the display image by mapping the display image to correspond to the display area based on a curvature of the curved display unit;
The displaying is performed on the corrected display image. The method of claim 1,
Detecting a change in the location of the user;
If the position change is detected, generating the first coordinates and displaying the display image are performed again. A computer-readable recording medium having recorded thereon a program for performing the method according to any one of claims 1 to 3. A curved display device for implementing a hologram through a curved display unit having a curved surface facing forward,
A camera unit for photographing a user located in front of the curved display device to generate a user image;
Recognize a user from the user image, generate a first position coordinate of the user based on the position of the camera unit, convert the first position coordinate to a second position coordinate based on the center of curvature of the curved display unit And determining a display image corresponding to an angle between the second position coordinate and the center of curvature among the multi-angle images or displaying a closest angle image to the user, and extending the second position coordinate and the center of curvature. A controller configured to estimate a gaze direction of the user, determine a display center at which the gaze direction and the curved display unit contact each other, and determine a display area for displaying the display image among the curved display units based on the display center; And
And the curved display unit to display the display image on a display area. The method of claim 9,
And the camera portion is a depth camera. The method of claim 9,
And the curved display portion has the same curvature. delete The method of claim 9,
The control unit is a reference to the display center, and determines the display area such that the left / right boundary of the display area corresponds to the angle of view information of the display image. The method of claim 13,
The controller corrects the display image by mapping the display image to correspond to the display area based on the curvature of the curved display unit.
The curved display unit is configured to display the corrected display image on the display area. The method of claim 9,
The camera unit is located outside the curved display unit.

Images