KR20200060207A - Hologram content generating apparatus, hologram content integration control sysetm having the same and operating method thereof - Google Patents

Abstract

According to the present invention, a hologram content integration control system comprises: a sensor unit detecting at least one of a user′s face, motion, and voice; a hologram content generation apparatus generating hologram contents which can interact with a user using sensed information; a display device for displaying the generated hologram content; and a rotation module for rotating the display device.

Description

A hologram content generating device, a hologram content integrated control system including the same, and a method for operating the same

The present invention relates to a hologram content generating apparatus, a hologram content integrated control system including the same, and a method for operating the same.

In general, the hologram display technology is an object that represents an object in three dimensions in three-dimensional space, and is an ideal fully-realistic three-dimensional display technology. Specifically, the hologram display technology is a technology that gives the same effect as the actual existence of the object in the human eye as the wave-front generated by the given object is reproduced as it is.

Registered Patent: 10-1670328, Registration Date: October 24, 2016 Title: Real-time image display device through multiple real-time image acquisition cameras and image control recognition method using the same China Patent: CN 105929962, Registration Date: September 7, 2016, Title: A kind of 360 ° of holographic real-time interaction methods

An object of the present invention is to provide a hologram content generating apparatus displaying a full stereoscopic image capable of interacting between 3D data acquisition and user-restored images, a hologram content integrated control system including the same, and a method of operating the same.

According to an embodiment of the present invention, a hologram content integrated control system includes: a sensor unit that detects at least one of a user's face, motion, and voice; A hologram content generating device that generates hologram content that can interact with the user using the sensed information; A display device that displays the generated hologram content; And it may include a rotating module for rotating the display device.

In an embodiment, the sensor unit may include a face recognition sensor that recognizes the position of both eyes of the user.

In an embodiment, the face recognition sensor is characterized by distinguishing the movement direction, movement angle, and movement speed of the user.

In an embodiment, the rotation module is controlled based on data detected by the face recognition sensor.

In an embodiment, the rotation module is characterized in that it is controlled by a rotation angle unit step of 0.35 ° (360 ° / 1024) or less.

In an embodiment, the sensor unit may include a voice sensor that recognizes the user's voice.

In an embodiment, the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on data sensed by the voice sensor.

In an embodiment, the sensor unit may include a motion sensor that recognizes the user's motion or gesture.

In an embodiment, the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on data sensed by the motion sensor.

In an embodiment, the sensor unit is characterized in that it comprises a haptic sensor for sensing the tactile sense of the hologram content.

In an embodiment, the apparatus for generating hologram content is characterized by generating binocular hologram data providing a wide viewing angle.

In an embodiment, the hologram content generating apparatus is characterized in that it acquires a 3D image or object selected from the sensor for the left eye and the sensor for the right eye.

In an embodiment, the apparatus for generating hologram content is characterized by acquiring a 3D image or object selected from the time for the left eye and the time for the right eye.

An operation method of an apparatus for generating holographic content according to an embodiment of the present invention includes: obtaining binocular content for an object; Generating computer-generated hologram (CGH) data for the left eye and CGH data for the right eye corresponding to the binocular content; And correcting the CGH data for the left eye and the CGH data for the right eye according to the user's face, motion, or voice.

In an embodiment, the method may further include detecting a face, motion, or voice of the pre-user.

In an embodiment, the method may further include transmitting the CGH data for the left eye and the CGH data display terminal for the right eye.

In an embodiment, acquiring binocular content for the object may include: acquiring color information and depth information from a left-eye image for the object; And obtaining color information and depth information from the right-eye image of the object.

In an embodiment, acquiring binocular content for the object may include: acquiring color information and depth information at a time for the left eye for the object; And obtaining color information and depth information at the right eye time for the object.

The apparatus for generating holographic content according to an embodiment of the present invention includes at least one processor and a memory storing at least one instruction executed by the at least one processor, wherein the at least one instruction is binocular for an object. Acquire content; Generating CGH (Computer-Generated Hologram) data for the left eye and CGH data for the right eye corresponding to the binocular content; And it is characterized in that it is executed in the at least one processor to correct the CGH data for the left eye and the CGH data for the right eye according to the user's face, motion or voice.

In an embodiment, the at least one instruction reads the CGH data for the left eye and the CGH data for the right eye from the memory storing the initial value, and reads the CGH data for the left eye and the CGH data for the right eye of the user's face, motion, or It is characterized in that it is executed in the at least one processor to add an option value corresponding to data corresponding to voice.

A hologram content generating apparatus according to an embodiment of the present invention, a hologram content integrated control system including the same, and an operation method thereof, overcome the limitations of manual hologram content production center by conventional computer hologram generation, and at the same time, the conventional hologram It can be escaped from complicated devices for making and long-term content calculation process.

A hologram content generating apparatus according to an embodiment of the present invention, a hologram content integrated control system including the same, and an operation method thereof, have a wide viewing angle according to binocular stereoscopic holography to comfortably view the restored image even when using an existing flat panel hologram terminal. Characteristics can be provided.

The apparatus for generating holographic content according to an embodiment of the present invention, a hologram content integrated control system including the same, and a method for operating the hologram content may be used by users to command 360-degree omnidirectional hologram content of various types (real-life or CG) by voice commands or face positions. Alternatively, it is possible to actively interact with real-time 3D content only by a user's gesture command.

The apparatus for generating holographic content according to an embodiment of the present invention, a holographic content integrated control system including the same, and a method of operating the hologram content can provide a viewer with a fully realistic image in a wider viewing space and more comfortably.

The apparatus for generating hologram content according to an embodiment of the present invention, a hologram content integrated control system including the same, and an operation method thereof, are suitable for personal portable mobile applications and immersive to viewers because they can reduce the weight of the system and speed up the calculation processing. It can be expected to increase the effect of 3D realism.

An apparatus for generating holographic content according to an embodiment of the present invention, a hologram content integrated control system including the same, and an operation method thereof, may enable interaction between a user and a restored stereoscopic image in real time.

The accompanying drawings are provided to help understand the present embodiment, and provide embodiments with detailed description. However, the technical features of the present embodiment are not limited to specific drawings, and the features disclosed in each drawing may be combined with each other to form a new embodiment.
1 is a view showing an hologram content integrated control system 10 according to an embodiment of the present invention by way of example.
FIG. 2 is a diagram illustrating an integrated hologram content control system 10 for interaction between hologram content and a user.
3 is a flowchart exemplarily showing a method of operating the hologram content generating apparatus 100 according to an embodiment of the present invention.
4A is a diagram illustrating an embodiment of a process for generating and encoding holographic content that can be interacted with between user reconstructed images in the hologram content generating apparatus 100 according to an embodiment of the present invention.
4B is a view showing another embodiment of a process for generating and encoding holographic content that can be interacted with between user reconstructed images in the hologram content generating apparatus 100 according to an embodiment of the present invention.
5A is a view showing an embodiment of an integrated processing process (SW) of hologram content in the hologram content implementation integrated control system 10 according to an embodiment of the present invention.
5B is a view showing another embodiment of an integrated processing process (SW) of hologram content in the hologram content implementation integrated control system 10 according to an embodiment of the present invention.
6 exemplarily shows that the display device 150 and the hologram content are synchronized and updated according to the user's face recognition-based user face.
7 exemplarily shows that the display apparatus 150 and the hologram content are rotated in synchronization according to a user's voice recognition-based user voice command.
8 is a diagram illustrating an example of rotating the hologram content according to a gesture command based on a user's gesture recognition.
9 is a diagram illustrating an apparatus 1000 for generating holographic content according to an embodiment of the present invention.

Hereinafter, the contents of the present invention will be described clearly and in detail so that those skilled in the art of the present invention can easily implement the drawings using the drawings.

The present invention can be applied to various changes and may have various forms, and specific embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific disclosure form, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Terms such as first and second may be used to describe various components, but the components should not be limited by the terms.

The terms may be used for the purpose of distinguishing one component from other components. For example, the first component may be referred to as the second component without departing from the scope of the present invention, and similarly, the second component may also be referred to as the first component. When an element is said to be "connected" or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

Terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, the terms "include" or "have" are intended to indicate the presence of a feature, number, step, action, component, part or combination thereof carried out, one or more other features or numbers. It should be understood that it does not preclude the existence or addition possibilities of, steps, actions, components, parts or combinations thereof. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms, such as those defined in the commonly used dictionary, should be interpreted as meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. .

An apparatus for generating holographic content according to an embodiment of the present invention, a holographic content integrated control system including the same, and an operation method thereof, are full stereoscopic image processing technologies capable of producing 3D data of a photorealism-based object and producing content to display a complete image. have.

With the recent development of three-dimensional display technology, by expressing objects with perfect parallax and depth, there are no symptoms of eye fatigue and dizziness caused by an accommodation-convergence mismatch problem. Furthermore, the hologram display technology can watch a natural image such as seeing the real world as the viewpoint moves. However, there are several requirements to implement such a hologram display technology.

First, the technology for real-time image processing and optical restoration thereof is very difficult because the amount of data is very large for a 360-degree image content for holography based on various real objects or computer graphics (CG). Second, because it is developed in such a way as to restore an existing image due to a large burden on computational throughput, viewers are producing content that is observed with a passive attitude. Therefore, there is a need for the development of content that enables interactive experiences between users and content. Third, even if the thus prepared holographic content is prepared, existing hologram devices 150 or spatial light modulators (SLMs) capable of displaying the content have a complex optical system. Existing hologram devices 150 or spatial light modulators (SLMs) have a complex optical system in order to make light having uniform intensity and excellent coherence, so the system is bulky and generally constructed on an optical table. Is becoming. Accordingly, there is an increasing need to develop a mobile hologram device 150 that is light weight and wearable, which can be easily heard by a user or worn on a user's face.

Meanwhile, in order to realize a wide viewing angle hologram display, a terminal having a pixel pitch of less than 1 μm is required. By comparison, the pixel pitch of panels utilizing commercial liquid crystals (LC) is currently in the range of several μm to tens of μm. This size provides a viewing angle of less than 5 degrees. So, holograms are being reproduced through a system with separate expensive devices that are accompanied by large-capacity and high-precision data processing technology such as eye-tracking technology.

An apparatus for generating holographic content according to an embodiment of the present invention, an integrated hologram content control system and a method for operating the same, between a user and a device 150 for restoring a holographic image based on 360-degree 3D (Dimension) image information While providing a real-time interaction function, it is possible to easily produce wide viewing angle holographic content at the same time. Particularly suitable for VR (Virtual Reality) / AR (Augmented Reality) terminals that can be carried personally or for mobile / wearable applications, but also provide a sense of sympathy between the user-restored stereoscopic images (visual, Holographic content capable of interaction based on tactile, auditory, and gesture / movement can be implemented.

A hologram content generating apparatus according to an embodiment of the present invention, a hologram content integrated control system including the same, and an operation method thereof, display and interact with a wide viewing angle by interworking hologram data with voice recognition and gesture recognition information of a user Can be.

A hologram content generating apparatus according to an embodiment of the present invention, a hologram content integrated control system including the same, and an operation method thereof, use RGB-depth map as input information, and FFT (fast-Fourier transformation) as shown in Equation 1 below ) Using the computer-generated hologram (CGH) calculation formula, calculation for high-speed hologram generation can be performed.

When slicing a 3D space containing a 3D scene into multiple layers, the hologram plane (H), the viewer's observing view-window plane (VW), and each layered layer When the fields are cut parallel to each other, point clouds that are distributed almost continuously can be assigned to the nearest layer. Then, the Fourier transform can be performed as follows using the FFT algorithm to calculate complex fields in the hologram plane.

, 그리고 d_i 은 시청자의 관측면, 3D 씬의 i-번째 층, i-번째 층의 물체(object) 필드, 필드 렌즈의 초점 길이, 조명광의 파장, 그리고 홀로그램 면으로부터 시청 거리를 각각 나타낸다. 이렇게 계산된 깊이맵 기반 CGH는 원래의 3D 장면(scene)을 공간 상에 복원시킬 수 있다. view-window 위치에 눈을 두고서 관찰하는 시청자는 광학적으로 복원된 3D 장면을 볼 수 있게 된다.Where (u, v), (x _i , y _i ), U _i , f,

, And d _i denote the viewer's viewing plane, the i-th layer of the 3D scene, the object field of the i-th layer, the focal length of the field lens, the wavelength of the illumination light, and the viewing distance from the hologram plane, respectively. The calculated depth map-based CGH can restore the original 3D scene in space. Viewers watching with their eyes at the view-window position will be able to see the optically reconstructed 3D scene.

1 is a view showing an hologram content integrated control system 10 according to an embodiment of the present invention by way of example. Referring to FIG. 1, a hologram content implementation system may include a hologram content generation device 100, a display device 150, and a sensor unit 150.

The hologram content generation device 100 may include a 3D information acquisition unit 110, a hologram information generation unit 120, an encoding unit 130, and a content control unit 140.

The 3D information acquisition unit 110 may be implemented to acquire data or RGB color and depth map information that can provide 3D information of a selected scene or object. Here, the format of the data may include a format stored as point cloud information, or a format stored as RGB-Depth information.

The hologram generating unit 120 may be implemented to calculate (generate) a complex-valued hologram (CGH) by a computer and store the generated data in a specific memory folder.

In an embodiment, the hologram generating unit 120 may generate a computer-generating hologram (CGH) by using an RGB-Depth format as an input. In an embodiment, the hologram generator 120 may generate a computer-generating hologram (CGH) by using a point cloud format as an input.

The encoding unit 130 may be implemented to correct the CGH file calculated by the hologram generation unit 120 to suit the characteristics of the display terminal and collect the corrected file in an easy form.

The content control unit 140 may be implemented to transmit data consisting of one or at least two or more hologram pairs selected from hologram data generated / stored by the hologram information generation unit 120 and the encoding unit 130.

The content control unit 140 receives the selected hologram data transmitted through the transmission unit, and converts or additionally encodes the selected hologram data to be suitable for the display device 150 to be used. The 3D information acquisition unit 110 and hologram information The generation unit 120 and the encoding unit 130 may be controlled. In an embodiment, the selected two hologram pairs may be hologram data suitable for the viewer's left and right eyes, respectively, in consideration of the position on the reconstructed space and the arrangement of the optical device.

In an embodiment, the video content for the left / right eye is based on the depth value (z) and the binocular distance (xy) of the hologram image reconstructed in the 3D space in order to provide a parallax suitable for the left / right eye position. Can be created / encoded. That is, the video content for the left / right eyes selected in consideration of spatial matching may be generated or encoded.

The display apparatus 150 may be implemented to upload the hologram data selected from the hologram content generation apparatus 100 and to restore the hologram image on the three-dimensional space by illuminating with appropriately prepared light in the uploaded state.

In an embodiment, the display apparatus 150 may properly display and restore hologram content data generated by the encoding unit 130.

In an embodiment, the display device 150 may be combined with a rotation module including a rotation function and a rotation angle / speed / direction control function as appropriate in response to a command from the sensor unit 160.

In an embodiment, the display device 150 is combined with a rotation module and a content control unit 140 including a function in which content is updated in response to a change in degree of rotation angle / speed / direction according to a command of the sensor unit 160 Can be.

In an embodiment, the display device 150 may have an interface with a rotation module and an interface with a content control unit. In an embodiment, the display device 150 may include a function of observing the hologram reconstructed image in the user's eye location area in combination with the rotation module and the content control unit 140.

In an embodiment, the angular unit for a rotational step in the rotation module may be equal to or less than the minimum rotation angle used to obtain 360 ° digital 3D data in the 360 ° omnidirectional content creation step. have. For example, the unit step of the rotating module may be 15 ° or less, or may be 0.35 ° (= 360 ° / 1024) or less in detail.

FIG. 2 is a diagram illustrating an integrated hologram content control system 10 for interaction between hologram content and a user. Referring to FIG. 2, the hologram content integrated control system 10 may include a display device 150 installed on the rotation module 170. The display device 150 may include a face recognition sensor 161, a voice sensor 162, a motion sensor 163, or a haptic sensor 164.

In an embodiment, the display device 150 may include a spatial light modulator and a user's face tracking or eye tracking (161) for recognizing a user's face (face) for the user. The face recognition sensor 161 may include a function of recognizing a user's binocular position when the user's face (or eye) moves in a left or right direction while being spaced at a proper distance from the sensor. In an embodiment, the data recognized by the face recognition sensor 161 may distinguish not only the position of the face, but also the rotation (movement) direction, rotation (movement) angle, and rotation (movement) speed.

In an embodiment, the display device 150 may include a spatial light modulator and a user's voice sensor 162 for recognizing the user's voice. The voice sensor 162 may monitor a user's voice using the display device 150 and obtain / recognize information such as a rotation direction, a rotation angle, and a rotation speed according to the content indicated by the voice. In an embodiment, the data recognized by the voice sensor 162 may distinguish a rotation direction, rotation angle, rotation speed, and the like.

In an embodiment, the display device 150 may include a spatial light modulator and a user's motion or gesture sensor (163) that recognizes a user's motion / gesture for the user. The motion sensor 163 monitors the gesture of the user using the display device 150 and obtains information such as the location of the motion, the direction of the motion (up / down / left / right), and the speed of the motion. Can be. At this time, the obtained information may be input to the feedback program to selectively update the hologram data generated by the hologram content generating apparatus 100. In an embodiment, hologram data matched to information obtained by the feedback program may be selected and read.

In an embodiment, the display device 150 may include a spatial light modulator and a user's haptic gesture sensor 164 that provides a haptic function to the user. The display device 150 based on the haptic sensor 164 may provide various sensory information such as tactile and touch to a user using the haptic sensor 164. At this time, the obtained information may be input to the feedback program to selectively update the hologram data generated by the hologram content generating apparatus 100. In an embodiment, hologram data matched to information obtained by the feedback program may be selected and read.

In an embodiment, the hologram data may be waited in a ready-by state by completing pre-calculation of CGH for a given 3D content (so that CGH is updated in real time). In an embodiment, the read data may be uploaded to the hologram display device 150 via a transmission unit in the content control unit after being transferred to the content control unit. Thereafter, the hologram image may be restored on the three-dimensional space by illuminating the display device 150 with the appropriately prepared light in the uploaded state.

Meanwhile, the integrated control system 10 may include a rotational module 170 having a mount capable of connecting the display device 150 to rotate the display device 150 and a rotation control unit. Can be. In an embodiment, the rotation module control unit interlocks with the voice recognition (or face recognition) data by the user so that the content and display device 150 may match the direction, rotation speed, and rotation angle corresponding to the corresponding voice (or face location) information. ) Can be rotated in synchronization condition.

In an embodiment, when the hologram displayed on the display apparatus 150 is synchronized and displayed while the display apparatus 150 rotates, as a result, the reconstructed image of the hologram is an optimal hologram reconstruction image observed by the user's binocular region. Can be.

In an embodiment, the voice information to be input by the user to the voice sensor 162 uses appropriate voice commands (left, right, fast, slow, more, etc.) to correspond to the direction / speed / angle of the user's face center, respectively. It can be done.

The integrated control system 10 for generating hologram content according to an embodiment of the present invention overcomes the limitations of manual hologram content production center by conventional computer hologram generation, and at the same time, it is a complex apparatus for creating a conventional hologram and It can be escaped from the long-term content calculation process.

The integrated control system 10 for generating hologram content according to an embodiment of the present invention has a wide viewing angle (eg, more than ± 60 degrees experimentally in order to comfortably view a restored image even when using an existing flat type hologram terminal) Embodied angle) binocular stereoscopic holography to provide characteristics.

The integrated control system 10 for generating hologram content according to an embodiment of the present invention allows users to input 360-degree omnidirectional hologram content of various types (real or CG), such as voice commands, face positions, or user gesture commands. By itself, it is possible to actively interact with real-time 3D content.

The integrated control system 10 for generating hologram content according to an embodiment of the present invention can allow viewers to experience a fully sensational image in a wider viewing space and more comfortably.

The integrated control system 10 for generating holographic content according to an embodiment of the present invention is suitable for a personal portable mobile application because it can reduce the weight of the system and speed up arithmetic processing, thereby providing an effect of increasing immersion and 3D sensation to viewers. By doing so, interaction between the user-restored stereoscopic image is possible in real time.

3 is a flowchart exemplarily showing a method of operating the hologram content generating apparatus 100 according to an embodiment of the present invention. 1 to 3, the hologram content generating apparatus 100 may operate as follows.

Binocular content (left-eye content / right-eye content) for the object may be obtained (S110). Thereafter, a computer-generated hologram (CGH) corresponding to the obtained binocular content may be calculated (S120). CGH can be separated into CGH for the left eye and CGH for the right eye (S130). Thereafter, the separated CGH for the left eye is uploaded to the spatial light modulator (SLM) for the left eye of the display device 150, and the separated CGH for the right eye is the spatial light modulator for the right eye of the display device 150 ( Spatial Light Modulator (SLM).

4A is a diagram illustrating an embodiment of a process for generating and encoding holographic content that can be interacted with between user reconstructed images in the hologram content generating apparatus 100 according to an embodiment of the present invention.

The 3D information acquisition unit 110 may acquire 3D information capable of providing 3D information of a scene or object selected from the sensor for the left eye and the sensor for the right eye. For example, RGB color information for the left eye and depth map for the left eye may be obtained from the image for the left eye. Also, RGB color information for the right eye and depth map for the right eye may be obtained from the image for the right eye. Thereafter, the 3D information obtaining unit 110 may combine RGB color information for the left eye and depth information for the left eye, and combine RGB color information for the right eye and depth information for the right eye. Thereafter, the hologram information generating unit 120 may receive the combined RGB-Depth format and generate the corresponding left / right eye CGH.

Meanwhile, in FIG. 4A, the left-eye image and the right-eye image were obtained from separate image sensors. However, it should be understood that the 3D information acquisition unit of the present invention is not limited thereto. The present invention may acquire an image for the left eye at a time difference from one image sensor, and then obtain an image for the right eye.

4B is a diagram showing another embodiment of a process for generating and encoding holographic content that can be interacted between user reconstructed images in the hologram content generating apparatus 100 according to an embodiment of the present invention. Referring to FIG. 4B, the image for the left eye and CGH are generated during the time for the left eye (T_Left) with a time difference in one image sensor, and the image for the right eye and CGH are generated for the time for the right eye (T_Right). .

Meanwhile, a process of operating in an integrated solution system that can interact between a user's information and a reconstructed image according to an embodiment of the present invention is as follows.

5A is a view showing an embodiment of an integrated processing process (SW; software) of hologram content in the hologram content implementation integrated control system 10 according to an embodiment of the present invention. Referring to FIG. 5A, the integrated processing process may operate in conjunction with a user recognition sensor, a motion sensor, and a user voice recognition sensor. In an embodiment, the rotation module may be rotated in response to the user's face information recognized by the user recognition sensor. In an embodiment, the hologram content may be rotated in response to information recognized by the motion sensor. In an embodiment, the hologram content may be rotated in response to voice information recognized by the voice sensor.

On the other hand, the integrated processing process may add interactions interlocked by haptic sensors.

5B is a view showing another embodiment of an integrated processing process (SW) of hologram content in the hologram content implementation integrated control system 10 according to an embodiment of the present invention. Referring to FIG. 5B, the integrated processing process may add interactions interlocked by haptic sensors compared to that of FIG. 5A. For example, on the other hand, in addition to image rotation in 5b, content update functions such as image size change and transfiguration may be added.

6 exemplarily shows that the display device 150 and the hologram content are synchronized and updated according to the user's face recognition-based user face. Referring to FIG. 6, the display device 150 is also rotated by tracking the pupil of the user, and at the same time, the hologram content can also be displayed in response to the rotation.

7 exemplarily shows that the display apparatus 150 and the hologram content are rotated in synchronization according to a user's voice recognition-based user voice command. Referring to FIG. 7, hologram content may be rotated according to a user's voice command.

8 is a diagram illustrating an example of rotating the hologram content according to a gesture command based on a user's gesture recognition. Referring to FIG. 8, the hologram content may be moved according to a user's gesture command (for example, moving the right hand to the left or moving the left hand to the right).

9 is a diagram illustrating an apparatus 1000 for generating holographic content according to an embodiment of the present invention. Referring to FIG. 9, the hologram content generation apparatus 1000 may include at least one processor 1100, a network interface 1200, a memory 1300, a display 1400, and an input / output device 1500.

The processor 1100 may include at least one device through FIGS. 1 to 8 or may be implemented by at least one method described above through FIGS. 1 to 8. As described above, the processor 1100 acquires binocular content for an object, generates CGH (Computer-Generated Hologram) data for the left eye and CGH data for the right eye corresponding to the binocular content, and the user's face and motion Alternatively, instructions may be executed to correct CGH data for the left eye and CGH data for the right eye according to voice.

The processor 1100 may execute a program and control the hologram content generating device 1000. The hologram content generating apparatus 1000 may be connected to an external device (eg, a personal computer or a network) through the input / output device 1500 and exchange data.

The network interface 1200 may be implemented to communicate with an external network by various wired / wireless methods.

The memory 1300 may include instructions readable by a computer. The processor 1100 may perform the aforementioned operations as the instructions stored in the memory 1300 are executed in the processor 1100. The memory 1300 may be a volatile memory or a nonvolatile memory.

The memory 1300 may include a storage device to store user data. The storage device may be an embedded multimedia card (eMMC), a solid state drive (SSD), or universal flash storage (UFS). The storage device may include at least one nonvolatile memory device. Non-volatile memory devices include NAND Flash Memory, Vertical NAND Flash (VNAND), NOR Flash Memory, Resistive Random Access Memory (RRAM), and Phase Change Memory (Phase-Change Memory: PRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Spin Transfer Torque Random Access Memory (STT-RAM), etc. Can be

The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware components and software components. For example, the devices, methods, and components described in the embodiments include, for example, a processor, a controller, an Arithmetic Logic Unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). Arrays, Programmable Logic Units (PLUs), microprocessors, or any other device capable of executing and responding to instructions can be implemented using one or more general purpose computers or special purpose computers. The processing device may run an operating system (OS) and one or more software applications running on the operating system.

In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the corresponding technical field may use a processing device to process a plurality of processing elements and / or a plurality of types. It can be seen that it can contain elements. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The apparatus 1000 for generating hologram content according to an embodiment of the present invention includes a memory 1300 for storing at least one processor 1100 and at least one instruction executed by the at least one processor 1100, The at least one instruction acquires binocular content for an object; Generating CGH (Computer-Generated Hologram) data for the left eye and CGH data for the right eye corresponding to the binocular content; And the CGH data for the left eye and the CGH data for the right eye according to the user's face, motion, or voice.

In an embodiment, the at least one instruction reads the CGH data for the left eye and the CGH data for the right eye from the memory storing the initial value, and reads the CGH data for the left eye and the CGH data for the right eye of the user's face, motion, or It may be executed in the at least one processor to add an option value corresponding to data corresponding to voice.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and may be recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for an embodiment or may be known and available to those skilled in computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media, ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Meanwhile, the above-described contents of the present invention are only specific embodiments for carrying out the invention. The present invention will include technical ideas that are abstract and conceptual ideas that can be utilized as future technologies, as well as specific and practically available means themselves.

100, 1000: a device for generating holographic content
110: 3D information acquisition unit
120: hologram information generating unit
130: encoding unit
140: content control
150: display terminal
160: sensor unit
170: rotating module
10: Hologram content integrated control system

Claims (20)

In the hologram content integrated control system,
A sensor unit that detects at least one of a user's face, motion, and voice;
A hologram content generating device that generates hologram content that can interact with the user using the sensed information;
A display device that displays the generated hologram content; And
Hologram content integrated control system including a rotating module for rotating the display device. According to claim 1,
The sensor unit hologram content control system including a face recognition sensor for recognizing the user's binocular position. According to claim 2,
The face recognition sensor is a holographic content integrated control system, characterized in that for distinguishing the movement direction, movement angle, and movement speed of the user. According to claim 2,
Hologram content integrated control system, characterized in that the rotation module is controlled based on the data detected by the face recognition sensor. The method of claim 4,
The rotation module is a hologram content integrated control system, characterized in that controlled by a rotation angle unit step of 15 ° or less. According to claim 1,
The sensor unit hologram content control system including a voice sensor for recognizing the user's voice. The method of claim 6,
Hologram content integrated control system, characterized in that the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on the data detected by the voice sensor. According to claim 1,
The sensor unit hologram content control system including a motion sensor for recognizing the user's motion or gesture. The method of claim 8,
Hologram content integrated control system, characterized in that the rotation direction, rotation angle, or rotation speed of the hologram content is determined based on the data detected by the motion sensor. According to claim 1,
The sensor unit comprises a haptic sensor for sensing the tactile sense of the hologram content hologram integrated control system. According to claim 1,
The hologram content generating apparatus is a holographic content integrated control system, characterized in that for generating a binocular type hologram data providing a wide viewing angle. The method of claim 11,
The hologram content generation device is a hologram content integrated control system, characterized in that for obtaining a 3D for the selected image or object from the sensor for the left eye and the sensor for the right. The method of claim 11,
The hologram content generating apparatus obtains a 3D image or object selected from the left eye time and the right arm time. In the operation method of the hologram content generating device,
Obtaining binocular content for the object;
Generating computer-generated hologram (CGH) data for the left eye and CGH data for the right eye corresponding to the binocular content; And
And correcting the CGH data for the left eye and the CGH data for the right eye according to a user's face, motion, or voice. The method of claim 14,
And detecting the user's face, motion, or voice. The method of claim 14,
And transmitting the CGH data for the left eye and the CGH data display terminal for the right eye. The method of claim 14,
The step of obtaining binocular content for the object,
Obtaining color information and depth information from the image for the left eye of the object; And
And obtaining color information and depth information from the image for the right eye of the object. The method of claim 14,
The step of obtaining binocular content for the object,
Obtaining color information and depth information at the time for the left eye for the object; And
And obtaining color information and depth information at the right eye time for the object. In the apparatus for generating hologram content,
A memory for storing at least one processor and at least one instruction executed by the at least one processor,
The at least one instruction,
Obtain binocular content for the object;
Generating CGH (Computer-Generated Hologram) data for the left eye and CGH data for the right eye corresponding to the binocular content; And
The apparatus for generating hologram content, characterized in that it is executed in the at least one processor to correct the CGH data for the left eye and the CGH data for the right eye according to a user's face, motion, haptic or voice. The method of claim 19,
The at least one instruction reads CGH data for the left eye and CGH data for the right eye from the memory storing the initial value, and corresponds to the read face CGH data for the left eye and CGH data for the right eye, corresponding to the user's face, motion, haptic, or voice The apparatus for generating hologram content, characterized in that it is executed in the at least one processor to add an option value corresponding to the data.

Images