KR20150003576A - Apparatus and method for generating or reproducing three-dimensional image - Google Patents

Apparatus and method for generating or reproducing three-dimensional image Download PDF

Info

Publication number
KR20150003576A
KR20150003576A KR1020130076615A KR20130076615A KR20150003576A KR 20150003576 A KR20150003576 A KR 20150003576A KR 1020130076615 A KR1020130076615 A KR 1020130076615A KR 20130076615 A KR20130076615 A KR 20130076615A KR 20150003576 A KR20150003576 A KR 20150003576A
Authority
KR
South Korea
Prior art keywords
image
eye
left
right
spherical
Prior art date
Application number
KR1020130076615A
Other languages
Korean (ko)
Inventor
김승훈
Original Assignee
삼성전자주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 삼성전자주식회사 filed Critical 삼성전자주식회사
Priority to KR1020130076615A priority Critical patent/KR20150003576A/en
Publication of KR20150003576A publication Critical patent/KR20150003576A/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/207Image signal generators using stereoscopic image cameras using a single 2D image sensor
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/239Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers

Abstract

A method for generating a three-dimensional image is provided according to an embodiment of the present invention. The method for generating a three-dimensional image includes: a process of acquiring a plurality of captured images forming a left-eye image and a right-eye image and direction information about each captured images when taking pictures; a process of disposing the left-eye images of the captured images on a first spherical coordinate system, generating a left-eye spherical image, disposing the right-eye images of the captured images on a second spherical coordinate system, and generating a right-eye spherical image; and a process of storing the left-eye spherical image and the right-eye spherical image.

Description

[0001] Apparatus and method for generating or reproducing three-dimensional images [0002]

Embodiments of the present invention relate to a three-dimensional image generating apparatus, a three-dimensional image generating method, a three-dimensional image reproducing apparatus, and a three-dimensional image reproducing method.

As interest in three-dimensional contents increases, interest in techniques for generating and reproducing three-dimensional images is increasing. As a method of implementing a three-dimensional image, for example, when a user gives a same image when viewed in left and right directions and gives a binocular parallax, the image recognized in the left eye and the image recognized in the right eye are processed as a single image Thereby, there is a method of allowing a user to recognize a three-dimensional image. Polarizing glasses, color filter glasses, screens, and the like are used as methods for separating the images from the left and right eyes and recognizing them respectively.

Embodiments of the present invention provide an apparatus and method for three-dimensional image generation and playback that enable a three-dimensional image to be viewed in a 360-degree direction.

According to an aspect of the embodiment of the present invention, there is provided a method for capturing a captured image, the method comprising: acquiring a plurality of captured images including a left eye image and a right eye image;

Eye images of the plurality of captured images are arranged in a first rectangular coordinate system to generate a left-eyeball image using the direction information of the plurality of captured images, and the right-eye images of the plurality of captured images are arranged in a second rectangular coordinate system Creating a right eye spherical image; And

And storing the left-eye spherical image and the right-eye spherical image.

The 3D image generating method may include generating left eyeball images by stitching left eye images of the plurality of captured images arranged in the first rectangular coordinate system; And stitching right eye images of the plurality of captured images arranged in the second rectangular coordinate system to generate the right eyeball image.

Wherein the step of generating the left eyeball image comprises determining left eye images to be stitched according to the matching result of the left eye image and the right eye image of the plurality of captured images and generating the right eye spherical image, The right eye images to be stitched can be determined according to the matching result of the left eye image and the right eye image of the captured images.

The storing step may include storing coordinate values of a first rectangular coordinate system for each pixel of the left eyeball image and the left eyeball image together; And storing coordinate values of the right eyeball image and the coordinates of the second rectangular coordinate system for each pixel of the right eyeball image together.

According to an aspect of another embodiment of the present invention, there is provided a method comprising: loading a left eyeball image in which pixels are arranged in a first rectangular coordinate system; and a right eyeball image in which pixels are arranged in a second rectangular coordinate system;

Detecting a tilt of the control device;

Determining reproduction direction information according to the detected inclination;

Determining a reproduction area corresponding to the reproduction direction information;

Generating a planar three-dimensional image corresponding to the reproduction region from the left-eye spherical image and the right-eye spherical image; And

And reproducing the plane 3D image.

The 3D image reproducing method may further include determining a size of the 3D image in accordance with the movement of the controller.

Wherein the generating of the planar three-dimensional image includes generating a planar left eye image and a planar right eye image by arranging pixel values of the reproduction area of the left eyeball image and the right eyeball image on a plane, The step of reproducing an image may alternately reproduce the planar left eye image and the planar right eye image.

According to another aspect of an embodiment of the present invention, there is provided an optical system including a first optical system and a second optical system for condensing incident light;

At least one imaging element for photoelectrically converting incident light passing through the first optical system and the second optical system;

An image acquiring unit that generates a plurality of captured images including a left eye image and a right eye image using the first optical system, the second optical system, and the image pickup element;

A direction information obtaining unit for obtaining direction information at the time of shooting for each of the plurality of captured images;

A left eyeball image is generated by arranging the plurality of captured images in a first spherical coordinate system using orientation information of the plurality of captured images, and the right eye images of the plurality of captured images are arranged in a second spherical coordinate system, A spherical image generator for generating spherical images; And

And a data storage unit for storing the left-eye spherical image and the right-eye spherical image.

Wherein the spherical image generating unit comprises: a left eyeball image generating unit for generating the left eye spherical image by stitching left eye images of the plurality of captured images arranged in the first spherical coordinate system; And a right eyeball image generating unit for generating the right eyeball image by stitching right eye images of the plurality of captured images arranged in the second spherical coordinate system.

Wherein the left eyeball image generating unit determines left eye images to be stitched according to a matching result of the left eye image and the right eye image of the plurality of captured images and the right eyeball image generating unit generates the left eyeball image of the captured images, The right eye images to be stitched can be determined according to the matching result of the left eye image and the right eye image.

The data storage unit may store coordinate values of the first rectangular coordinate system for each pixel of the left and right sphere images and the second rectangular coordinate system for each pixel of the right sphere image, Coordinate values of the coordinate system can be stored together.

According to another aspect of the present invention, there is provided a stereoscopic image display apparatus, comprising: a data storage unit for storing a left-eye spherical image in which pixels are arranged in a first spherical coordinate system; and a right-eye spherical image in which pixels are arranged in a second spherical coordinate system;

A playback direction determination unit for detecting a tilt of the control device and determining playback direction information according to the detected tilt;

A reproduction area determination unit that determines a reproduction area corresponding to the reproduction direction information;

A planar three-dimensional image generation unit for generating a planar three-dimensional image corresponding to the reproduction area from the left-eyeball image and the right-eyeball image; And

And a reproducing unit for reproducing the plane 3D image.

The three-dimensional image reproducing apparatus may further include an image size determining unit that determines a size of the plane 3D image according to the movement of the controller.

Wherein the planar three-dimensional image generator generates a planar left eye image and a planar right eye image by arranging pixel values of the reproduction area of the left eyeball image and the right eyeball image on a plane, The flat right-eye image can be alternately reproduced.

According to another aspect of the present invention, there is provided a computer-readable recording medium storing computer program codes for performing a three-dimensional image generation method when read out and executed by a processor,

Acquiring a plurality of captured images including a left eye image and a right eye image, and direction information at the time of shooting for each of the plurality of captured images;

Eye images of the plurality of captured images are arranged in a first rectangular coordinate system to generate a left-eyeball image using the direction information of the plurality of captured images, and the right-eye images of the plurality of captured images are arranged in a second rectangular coordinate system Creating a right eye spherical image; And

And storing the left-eye spherical image and the right-eye spherical image.

According to another aspect of another embodiment of the present invention, there is provided a computer-readable recording medium storing computer program codes for performing a three-dimensional image reproducing method when read and executed by a processor,

Loading a left eyeball image in which pixels are arranged in a first rectangular coordinate system and a right eyeball image in which pixels are arranged in a second rectangular coordinate system;

Detecting a tilt of the control device;

Determining reproduction direction information according to the detected inclination;

Determining a reproduction area corresponding to the reproduction direction information;

Generating a plane 3D image using pixel values of the reproduction area of the left eyeball image and the right eyeball image; And

And reproducing the plane three-dimensional image.

According to embodiments of the present invention, there is provided an apparatus and method for three-dimensional image generation and playback that allow a three-dimensional image to be viewed in a 360-degree direction.

1 is a diagram illustrating a configuration of a three-dimensional image generating apparatus 100 according to an embodiment of the present invention.
FIG. 2 is a diagram showing an appearance of a three-dimensional image generating apparatus 100a according to an embodiment of the present invention in which a first optical system 110a and a second optical system 110b are disposed.
3A to 3C are diagrams showing an appearance of a three-dimensional image generating apparatus 100b according to another embodiment of the present invention.
4 is a diagram for explaining a process of generating a left-eyeball image or a right-eyeball image from a plurality of captured images.
FIG. 5 is a diagram illustrating a configuration of a rectangular image generation unit 150 according to another embodiment of the present invention.
6 is a view for explaining a process of composing a plurality of left eye images or a plurality of right eye images according to another embodiment of the present invention.
FIG. 7 is a diagram illustrating an example of a left-eye spherical image and a right-eye spherical image according to an embodiment of the present invention.
8 is a flowchart illustrating a method of generating a three-dimensional image according to an exemplary embodiment of the present invention.
9 is a diagram illustrating a configuration of a three-dimensional image reproducing apparatus 800a according to another embodiment of the present invention.
10 is a diagram illustrating a three-dimensional image reproducing apparatus 800b according to an embodiment of the present invention.
11 is a diagram illustrating a three-dimensional image reproducing apparatus 800c according to another embodiment of the present invention.
FIG. 12 is a diagram for explaining a process of determining a reproduction region and generating a plane three-dimensional image according to an embodiment of the present invention.
13 is a flowchart illustrating a three-dimensional image reproducing method according to another embodiment of the present invention.
FIG. 14 is a diagram showing the structure of a three-dimensional image reproducing apparatus 800d according to another embodiment of the present invention.
15 is a diagram for explaining a process of determining a reproduction area according to another embodiment of the present invention.

The following description and accompanying drawings are for understanding the operation according to the present invention, and parts that can be easily implemented by those skilled in the art can be omitted.

Furthermore, the specification and drawings are not intended to limit the present invention, and the scope of the present invention should be determined by the claims. The terms used in the present specification should be construed to mean the meanings and concepts consistent with the technical idea of the present invention in order to best express the present invention.

Embodiments of the present invention will now be described with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of a three-dimensional image generating apparatus 100 according to an embodiment of the present invention. The three-dimensional image generation apparatus 100 according to the present embodiment includes a first optical system 110a, a second optical system 110b, an imaging device 120, an image acquisition unit 130, a direction information acquisition unit 140, A generating unit 150, and a data storage unit 160. [

The three-dimensional image generating apparatus 100 according to the present embodiment includes a first optical system 110a and a second optical system 110b for simultaneously capturing a left eye image and a right eye image at the time of capturing an image.

FIG. 2 is a diagram showing an appearance of a three-dimensional image generating apparatus 100a according to an embodiment of the present invention in which a first optical system 110a and a second optical system 110b are disposed.

The first optical system 110a and the second optical system 110b are arranged to be spaced apart from each other and receive incident light from the object, as shown in Fig. Light is received by the first optical system 110a and the second optical system 110b in different paths to generate a left eye image from incident light passing through the first optical system 110a and incident light that has passed through the second optical system 110b The right-eye image can be generated.

According to one embodiment, the area corresponding to the first optical system 110a and the area corresponding to the second optical system 110b in the image pickup device 120 can be arranged so as not to overlap each other. As another example, the image pickup device 120 may include a first image pickup device corresponding to the first optical system 110a and a second image pickup device corresponding to the second optical system 110b. In this case, two physically separated imaging elements may be provided.

The first optical system 110a and the second optical system 110b may each include a lens, a diaphragm, and a shutter. The lens may comprise a plurality of groups and / or a plurality of lenses.

The optical signal transmitted through the first optical system 110a and the second optical system 110b reaches the light receiving surface of the image pickup element 120 to image an image of the object. The image pickup device 120 may be a CCD (Charge Coupled Device) image sensor or a CIS (Complementary Metal Oxide Semiconductor Image Sensor) for converting an optical signal into an electric signal.

3A to 3C are diagrams showing an appearance of a three-dimensional image generating apparatus 100b according to another embodiment of the present invention in which a first optical system 110a and a second optical system 110a are arranged.

According to the present embodiment, the three-dimensional image generating apparatus 100b includes an exchangeable lens having a first lens 310 corresponding to the first optical system 110a and a second lens 320 corresponding to the second optical system 110b, The first optical system 110a and the second optical system 110b can be implemented by being combined with the first optical system 300. [ In this case, the imaging device 120 can be used by being partitioned into a region corresponding to the first optical system 110a and a region corresponding to the second optical system 110b. The 3D image generating apparatus 100b according to the present embodiment generates a two-dimensional image when combined with the interchangeable lens for two-dimensional image taking shown in FIG. 3C, and combines with the interchangeable lens 300 for three- A three-dimensional image can be generated.

1) generates a captured image from an image pickup signal generated by photoelectric conversion in the image pickup device 120. The image pickup unit 130 The captured image generated by the image acquiring unit 130 may include a left eye image generated from the incident light passing through the first optical system 110a and a right eye image generated from the incident light passing through the second optical system 110b.

When the imaging device 120 is provided with only one imaging device 120 and the imaging device 120 is divided into two areas corresponding to the first optical system 110a and the second optical system 110b, An area corresponding to the first optical system 110a is extracted to generate a left eye image, and an area corresponding to the second optical system is extracted to generate a right eye image.

When the image pickup device 120 includes a first image pickup device and a second image pickup device which are physically separated, a left eye image is generated from an image pickup signal of the first image pickup device, and a right eye image is generated from an image pickup signal of the second image pickup device can do.

The image obtaining unit 130 according to the present embodiment includes a first optical system 110a and a second optical system 110b for continuously capturing a plurality of captured images to obtain a left eye image and a right eye image in a 360 degree direction expressed in a spherical coordinate system ), And the image pickup device 120 can be controlled. One embodiment of the present invention generates a three-dimensional image in a 360-degree direction expressed in a spherical coordinate system. An image in a 360-degree direction is generated by successively photographing a plurality of captured images and then connecting them. For this purpose, in order to obtain captured images required to generate an image in the 360-degree direction, the image obtaining unit 130 may perform a continuous shooting until a predetermined number of times or a required captured image is obtained, The first optical system 110a, the second optical system 110b, and the image pickup device 120. [ That is, when the shutter release signal is input, the image obtaining unit 130 may control the first optical system 110a, the second optical system 110b, and the image pickup device 120 to perform continuous photographing a predetermined number of times . The image acquisition unit 130 generates a captured image including a left eye image and a right eye image every time imaging is performed.

During continuous shooting, the user photographs a subject while changing the direction of the three-dimensional image generating apparatus 100 such that the first optical system 110a and the second optical system 110b sequentially face 360 degrees. According to one embodiment, a user interface may be provided that guides the user to appropriately change the orientation of the three-dimensional image generating apparatus 100. [ For example, the user interface can provide information on the moving direction of the three-dimensional image generating apparatus 100, information on the moving speed, information on the image photographed to date, information on the direction to be photographed, have. For this, the three-dimensional image generating apparatus 100 according to an embodiment may further include a user interface providing unit (not shown) for providing the user interface.

According to another embodiment of the present invention, the image acquisition unit 130 may control the first optical system 110a, the second optical system 110b, and the image pickup device 120 to capture a moving image. In this case, a plurality of frames are successively photographed after the shutter release signal is inputted.

The direction information obtaining unit 140 obtains direction information on each captured image while continuous shooting is performed. The direction information can be obtained from, for example, a tilt sensor, a gyro sensor, a geomagnetic sensor, an acceleration sensor, and the like provided in the three-dimensional image generation apparatus 100. The direction information obtaining unit 140 generates direction information every time image capturing is performed in synchronization with the image obtaining unit 130 or the image pickup device 120 and provides the direction information to the rectangular image generating unit 150.

When motion picture taking is performed, direction information is generated every time each frame is photographed, and provided to the spherical image generating unit 150.

The spherical image generation unit 150 generates a spherical image by arranging a plurality of captured images on a spherical coordinate system. At this time, a first spherical coordinate system for the left eye image and a second spherical coordinate system for the right eye image are defined. Of the plurality of captured images, the left eye images are arranged in the first rectangular coordinate system and converted into a left eyeball image. Of the plurality of captured images, the right eye images are arranged in the second spherical coordinate system and converted into a right eye spherical image.

4 is a diagram for explaining a process of generating a left-eyeball image or a right-eyeball image from a plurality of captured images.

The plurality of captured images acquired by the image acquisition unit 130 include a plurality of left eye images and a plurality of right eye images. 4, a process of generating a left-eyeball image from a plurality of left-eye images will be described. The process of generating a right eye spherical image from a plurality of right eye images is performed on the same principle.

The plurality of left eye images Im1, Im2, and Im3 have corresponding direction information. Using these direction information, each of the left eye images is arranged at a corresponding position in the first rectangular coordinate system. For example, when it is assumed that the photographer is located at the center point (0) in the sphere of the first rectangular coordinate system and the direction corresponding to the direction information corresponding to the corresponding left eye image at the center point O is viewed, Eye image is arranged in the region of the left eye image. In this way, a plurality of left eye images can be arranged in a first rectangular coordinate system, and they can be combined to generate a left eyeball image.

The direction information can be interpolated to the coordinates (r,?,?) Of the first rectangular coordinate system and converted into the coordinates of the first rectangular coordinate system. When the specification of the three-dimensional image generating apparatus is low during such processing, the cloud service can be utilized for the processing speed.

When the moving image is obtained by the image obtaining unit 130, the rectangular image generating unit 150 displays only the portion in which a plurality of frames are superimposed at the corresponding point in time, expressed as a moving image, and the remaining portion is expressed as a still image So that a left-eye spherical image and a right-eye spherical image can be generated. To this end, a portion other than the region photographed at the time point may be represented as a still image using frames photographed before or after the point of time.

FIG. 5 is a diagram illustrating a configuration of a rectangular image generation unit 150 according to another embodiment of the present invention. The spherical image generating unit 150 according to the present embodiment includes a left and right spherical image generating unit 410 and a right and left spherical image generating unit 420.

The left eyeball image generating unit 410 receives a plurality of left eye images from the image obtaining unit 130 and receives direction information corresponding to each of the plurality of left eye images in the direction information obtaining unit 140, And generates an image.

The right eye spherical image generation unit 420 receives a plurality of right eye images from the image acquisition unit 130, receives direction information corresponding to each of the plurality of right eye images from the direction information acquisition unit 140, And generates an image.

According to the present embodiment, when selecting a left eye image to be synthesized to generate a left eye spherical image from a plurality of left eye images, the left eyeball image generating unit 410 can select a left eye image to be synthesized by referring to a plurality of right eye images have. When the right eye image generating unit 420 selects a right eye image to be synthesized to generate a right eye spherical image from among a plurality of right eye images, the right eye image generating unit 420 can select a right eye image to be synthesized by referring to a plurality of left eye images. For example, when there are a plurality of left eye images corresponding to a specific direction, a left eye image having a high degree of similarity between the left eye image and the right eye image is preferentially selected to select a left eye image to be synthesized, You can choose. Here, the degree of similarity of the image can be determined by, for example, matching feature points (e.g., edge and color information) with each other.

6 is a view for explaining a process of composing a plurality of left eye images or a plurality of right eye images according to another embodiment of the present invention.

When a plurality of left eye images are arranged in the first rectangular coordinate system, the rectangular image generating unit 150 stitches the adjacent left eye images to generate a left eyeball image. For example, adjacent IMAGE 1, IMAGE 2, and IMAGE 3 are synthesized. At this time, the feature points of the adjacent images are extracted, and the positions of the IMAGE 1, IMAGE 2, and IMAGE 3 can be rearranged so that the feature points of the adjacent images are matched with each other. The feature points may be, for example, edge, gradient, color, and the like. When a plurality of left eye images are stitched together on the basis of the feature points, coordinate values in the first rectangular coordinate system of each pixel of the left eyeball image are defined and stored together with the stitched image. That is, for example, the first pixel may have a coordinate value of (r 1 , θ 1 , ρ 1 ), and the second pixel may have a coordinate value of (r 2 , θ 2 , ρ 2 ).

FIG. 7 is a diagram illustrating an example of a left-eye spherical image and a right-eye spherical image according to an embodiment of the present invention.

The spherical image generating unit 150 may generate the left and right spherical images 610 and 620 as shown in FIG. In the left eyeball image 610, the coordinate value of the first rectangular coordinate system is defined and stored for each pixel, and the coordinate value of the second rectangular coordinate system is defined and stored for each pixel in the right eyeball image 620. [

According to an embodiment of the present invention, the left eyeball image and the right eyeball image may be stored together as one image file. In this case, information indicating that the image stored in the image file is a spherical three-dimensional image, information on the sizes of the first coordinate system and the second coordinate system (e.g., radius), and the like may be stored in the header part of the image file.

According to another embodiment of the present invention, the left-eye spherical image and the right-eye spherical image may be respectively stored as separate image files. In this case, in the header part of the image file of the left eyeball image, information that the image stored in the image file is the left eyeball image, file name and position information of the image file of the right eye spherical image, information about the size of the first coordinate system For example, a radius) and the like can be stored. In the header part of the image file of the right eye spherical image, information that the image stored in the image file is the right eye spherical image, file name and position information of the image file of the corresponding left eyeball image, information on the size of the second coordinate system Radius) and the like can be stored.

The data storage unit 160 stores the generated image file. Although not shown in FIG. 1, the apparatus 100 for generating a three-dimensional image according to an embodiment of the present invention may further include an image file generating unit (not shown) for generating an image file.

8 is a flowchart illustrating a method of generating a three-dimensional image according to an exemplary embodiment of the present invention.

In the three-dimensional image generation method according to the present embodiment, direction information corresponding to each of a plurality of captured images is acquired while continuously capturing a plurality of captured images including a left eye image and a right eye image (S702).

Next, a left eyeball image is generated by arranging a plurality of left eye images in the first rectangular coordinate system, and a right eyeball image is generated by arranging a plurality of right eye images in the second rectangular coordinate system (S704). At this time, a plurality of left eye images are arranged in a first rectangular coordinate system using the direction information, and a plurality of right eye images are arranged in a second rectangular coordinate system.

Next, the left-eye spherical image and the right-sphere spherical image are stored (S706). At this time, coordinate values of the left eyeball image and the first rectangular coordinate system of each pixel are stored together, and the coordinate values of the right eyeball image and the second rectangular coordinate system of each pixel are stored together. In one embodiment, the left-eye spherical image and the right-eye spherical image may be stored together in one image file. In another embodiment, the left-eye spherical image and the right-eye spherical image may be stored in different image files, respectively.

9 is a diagram illustrating a configuration of a three-dimensional image reproducing apparatus 800a according to another embodiment of the present invention. The three-dimensional image reproducing apparatus 800a according to the present embodiment includes a data storing unit 810, a reproducing direction determining unit 820, a reproducing area determining unit 830a, a plane three-dimensional image generating unit 840, ).

The three-dimensional image reproducing apparatus 800a according to the present embodiment is a device for reproducing a three-dimensional image composed of a spherical left eye image and a spherical right eye image. The data storage unit 810 stores the left-eye spherical image and the right-eye spherical image. According to one embodiment, the 3D image can be stored together with the left-eye spherical image and the right-eye spherical image in one image file. In this case, information indicating that the image stored in the image file is a spherical three-dimensional image, information on the sizes of the first coordinate system and the second coordinate system (e.g., radius), and the like may be stored in the header part of the image file. In another embodiment, the left-eye spherical image and the right-eye spherical image may be stored as separate image files, respectively. In this case, in the header part of the image file of the left eyeball image, information that the image stored in the image file is the left eyeball image, file name and position information of the image file of the right eye spherical image, information about the size of the first coordinate system For example, a radius) and the like can be stored. In the header part of the image file of the right eye spherical image, information that the image stored in the image file is the right eye spherical image, file name and position information of the image file of the corresponding left eyeball image, information on the size of the second coordinate system Radius) and the like can be stored.

The three-dimensional image reproducing apparatus 800a according to the present embodiment receives the direction information of the tilt sensor included in the three-dimensional image reproducing apparatus 800a or the tilt information of the control apparatus communicating with the three-dimensional image reproducing apparatus 800a, A three-dimensional image can be reproduced.

10 is a diagram illustrating a three-dimensional image reproducing apparatus 800b according to an embodiment of the present invention. According to the present embodiment, the three-dimensional image reproducing apparatus 800b includes a tilt sensor to obtain tilt information and reproduce the three-dimensional image using the tilt information obtained. In this embodiment, a controller for detecting tilt information and a three-dimensional image reproducing apparatus 800b are integrally formed.

11 is a diagram illustrating a three-dimensional image reproducing apparatus 800c according to another embodiment of the present invention. According to the present embodiment, the three-dimensional image reproducing apparatus 800c can receive tilt information from the control apparatus 1000 communicating with the three-dimensional image reproducing apparatus 800c. The control apparatus 1000 can detect a tilt, including a tilt sensor, a gyro sensor, a geomagnetic sensor, an acceleration sensor, and the like. The user can reproduce an image in a desired direction of the spherical three-dimensional image by adjusting the inclination of the control device 1000. [

The reproduction direction determination unit 820 determines the reproduction reproduction direction information according to the inclination detected by the three-dimensional image reproduction apparatus 800a. As described above, the tilt can be detected from a tilt sensor provided in the three-dimensional image reproducing apparatus 800b, or can be received from the control apparatus 1000 capable of detecting the tilt by communicating with the three-dimensional image reproducing apparatus 800c.

The reproduction direction information can be generated by transforming the detected gradient into coordinates of the first rectangular coordinate system and the second rectangular coordinate system.

The reproduction area determination unit 830a determines a reproduction area corresponding to the reproduction direction information. The plane three-dimensional image generating unit 840 generates a plane three-dimensional image corresponding to the reproduction area from the left-eye spherical image and the right-eye spherical image.

FIG. 12 is a diagram for explaining a process of determining a reproduction region and generating a plane three-dimensional image according to an embodiment of the present invention. Hereinafter, a process of generating a left-eye 3D image from a left-eye spherical image will be described as an example.

When the playback direction information (r 1, θ 1, ρ 1) is determined, and playback sub-area determining unit (830a) detects a pixel of the left-eye rectangular image corresponding to the reproduction direction of the information (r 1, θ 1, ρ 1) , And determines the pixels within the predetermined region around the pixel as the reproduction region. The predetermined area may be determined according to various embodiments, for example, the area may be determined as a predetermined area or may be determined according to the set resolution.

The plane 3D image generating unit 840 extracts pixel values of pixels corresponding to the reproduction area from the left eyeball image, and generates a left eye plane 3D image 1110 by flattening the spherical image. When the spherical image is arranged on the plane, the edge portion is expressed by a curved line and the image is distorted in the flattening process. The plane three-dimensional image generating unit 840 arranges the spherical image on the plane, And the left-eye plane 3D image 1110 can be generated by correcting distortion of the image.

Similar to the process of generating the left-eye plane 3D image 1110, the reproduction area determination unit 830a can determine the reproduction area corresponding to the reproduction direction information in the right-eyeball image. In addition, the planar three-dimensional image generating unit 840 may generate a right-eye plane 3D image corresponding to the reproduction area from the right-eye spherical image.

The planar three-dimensional image generator 840 outputs the planar three-dimensional image including the left-eye three-dimensional plane image and the right-eye plane three-dimensional image to the playback unit 850.

The reproducing unit 850 reproduces a plane three-dimensional image including the left-eye plane 3D image and the right-eye plane 3D image. The reproducing unit 850 sequentially reproduces the left-eye plane 3D image and the right-eye plane 3D image corresponding to each other to reproduce the plane 3D image.

13 is a flowchart illustrating a three-dimensional image reproducing method according to another embodiment of the present invention.

In the three-dimensional image reproducing method according to the present embodiment, the left-eye spherical image and the right-eye spherical image corresponding to each other are loaded (S1202). Next, the tilt of the three-dimensional image reproducing apparatus 800a or the control apparatus 1000 is detected (S1204), and the reproducing direction information is determined from the detected tilt (S1206). The reproduction direction information is represented by the coordinate information of the first coordinate system and the coordinate information of the second coordinate system.

When the reproduction direction information is determined, a reproduction area corresponding to the reproduction direction information is determined (S1208). The reproduction region can be determined on the left eye spherical image and the right eye spherical image. The reproduction area may be determined to include pixels within a predetermined area around a pixel corresponding to the reproduction direction information.

Next, a pixel value of pixels corresponding to the reproduction area is extracted from the left-eye spherical image and the right-eye spherical image, and a plane three-dimensional image is generated using the extracted pixel values (S1210). The pixel values extracted from the left eyeball image and the right eyeball image are arranged in a plane, the distortion generated in the flattening process is corrected, the edge can be linearized, and converted into a left eye plane 3D image and a right eye 3D image at step S1210.

Next, the left three-dimensional plane image and the right-eye plane three-dimensional image are sequentially reproduced, and a three-dimensional plane image is reproduced (S1212).

FIG. 14 is a diagram showing the structure of a three-dimensional image reproducing apparatus 800d according to another embodiment of the present invention. The three-dimensional image reproducing apparatus 800d according to the present embodiment includes a data storing unit 810, a reproducing direction determining unit 820, an image size determining unit 1310, a reproducing area determining unit 830b, a three- 840, and a reproducing unit 850.

The data storage unit 810 stores a three-dimensional image including a left-eye spherical image and a right-eye spherical image.

The reproduction direction determination unit 820 determines the reproduction direction information according to the inclination detected by the three-dimensional image reproduction apparatus 800d.

The image size determination unit 1310 determines the size of the plane 3D image according to the movement of the 3D image reproducing apparatus 800d or the control apparatus 1000. [

According to an exemplary embodiment, when detecting the tilt in the three-dimensional image reproducing apparatus 800d as shown in FIG. 10, the image size determining unit 1310 may determine that the three-dimensional image reproducing apparatus 800d is moving, The size of the image can be enlarged or reduced. 11, when the tilt is detected by the control apparatus 1000, the image size determining unit 1310 determines the size of the plane 3D image when the movement of the controller 1000 is detected, Can be enlarged or reduced. For example, the user moves the three-dimensional image reproducing apparatus 800d or the control apparatus 1000 in the direction in which the back surface of the apparatus faces, in a state of viewing the screen of the three-dimensional image reproducing apparatus 800d or the control apparatus 1000, The size of the three-dimensional image can be enlarged, and the size of the three-dimensional plane image can be reduced by moving the image in the direction toward the front of the apparatus.

The reproduction region determining unit 830b determines whether or not the reproduction direction determination unit 830b reproduces the left eyeball image and the right eyeball image in accordance with the reproduction direction information determined by the reproduction direction determination unit 820 and the size information of the plane three- Area. As described above, the play area determination section (830b) is a playback direction information (r 1, θ 1, ρ 1) detects a pixel of the left-eye spherical image, mainly the pixel reproduction the pixels in the predetermined area corresponding to Area. The predetermined area is determined according to the size information of the plane 3D image determined by the image size determination unit 1310. [

15 is a diagram for explaining a process of determining a reproduction area according to another embodiment of the present invention.

As shown in Figure 15, the center point of the play area is determined in accordance with the reproduction direction information (r 1, θ 1, ρ 1). Next, the size of the left eye plane 3D image or the right eye plane 3D image is determined according to the size information of the 3D 3D image. For example, the size information of the planar three-dimensional image can be determined by the distance (d 1 or d 2 ) from the center point to the vertex. If the size information of the plane 3D image is d 1 , the reproduction area is determined by IMAGE 4. If the size information of the 3D image is d 2 , the reproduction area is determined by IMAGE 5.

The plane three-dimensional image generating unit 840 generates a plane three-dimensional image corresponding to the reproduction area from the left-eye spherical image and the right-eye spherical image.

The reproducing unit 850 reproduces a plane three-dimensional image including the left-eye plane 3D image and the right-eye plane 3D image. The reproducing unit 850 sequentially reproduces the left-eye plane 3D image and the right-eye plane 3D image corresponding to each other to reproduce the plane 3D image.

Meanwhile, the present invention can be realized by storing computer-readable codes in a computer-readable storage medium. The computer-readable storage medium includes all kinds of storage devices in which data that can be read by a computer system is stored.

The computer-readable code is configured to perform the steps of implementing the three-dimensional image generation method or the three-dimensional image reproduction method according to the present invention when read from and executed by a processor from the computer-readable storage medium. The computer readable code may be implemented in a variety of programming languages. And functional programs, codes, and code segments for implementing embodiments of the present invention may be readily programmed by those skilled in the art to which the present invention pertains.

Examples of computer-readable storage media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, as well as implementations in the form of carrier waves (e.g., transmission over the Internet). In addition, the computer-readable storage medium may be distributed over a networked computer system so that computer readable code is stored and executed in a distributed fashion.

The present invention has been described above with reference to preferred embodiments. It will be understood by those skilled in the art that the present invention may be embodied in various other forms without departing from the spirit or essential characteristics thereof. Therefore, the above-described embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and the inventions claimed by the claims and the inventions equivalent to the claimed invention are to be construed as being included in the present invention.

Claims (20)

  1. Acquiring a plurality of captured images including a left eye image and a right eye image, and direction information at the time of shooting for each of the plurality of captured images;
    Eye images of the plurality of captured images are arranged in a first rectangular coordinate system to generate a left-eyeball image using the direction information of the plurality of captured images, and the right-eye images of the plurality of captured images are arranged in a second rectangular coordinate system Creating a right eye spherical image; And
    And storing the left-eye spherical image and the right-eye spherical image.
  2. The method according to claim 1,
    Stitching left eye images of the plurality of captured images arranged in the first rectangular coordinate system to generate the left eyeball image; And
    Further comprising stitching right eye images of the plurality of captured images arranged in the second spherical coordinate system to generate the right eye spherical image.
  3. 3. The method of claim 2,
    Wherein the generating of the left eyeball image comprises: determining left eye images to be stitched according to a matching result of the left eye image and the right eye image of the plurality of captured images,
    Wherein the step of generating the right eyeball image determines right eye images to be stitched according to a matching result of the left eye image and the right eye image of the plurality of captured images.
  4. 2. The method of claim 1,
    Storing the coordinate values of the first rectangular coordinate system for each pixel of the left eyeball image and the left eyeball image together; And
    And storing coordinate values of the second spherical coordinate system for each pixel of the right-eye spherical image and the right-eye spherical image together.
  5. Loading a left eyeball image in which pixels are arranged in a first rectangular coordinate system and a right eyeball image in which pixels are arranged in a second rectangular coordinate system;
    Detecting a tilt of the control device;
    Determining reproduction direction information according to the detected inclination;
    Determining a reproduction area corresponding to the reproduction direction information;
    Generating a planar three-dimensional image corresponding to the reproduction region from the left-eye spherical image and the right-eye spherical image; And
    And reproducing the three-dimensional image.
  6. 6. The method of claim 5,
    Further comprising the step of determining the size of the planar three-dimensional image according to the movement of the controller.
  7. 6. The method of claim 5, wherein the generating the plane 3D image comprises:
    And generating a planar left eye image and a planar right eye image by arranging pixel values of the reproduction area of the left eyeball image and the right eye spherical image on a plane,
    Wherein the step of reproducing the planar three-dimensional image alternately reproduces the planar left-eye image and the planar right-eye image.
  8. A first optical system and a second optical system for condensing incident light;
    At least one imaging element for photoelectrically converting incident light passing through the first optical system and the second optical system;
    An image acquiring unit that generates a plurality of captured images including a left eye image and a right eye image using the first optical system, the second optical system, and the image pickup element;
    A direction information obtaining unit for obtaining direction information at the time of shooting for each of the plurality of captured images;
    A left eyeball image is generated by arranging the plurality of captured images in a first spherical coordinate system using orientation information of the plurality of captured images, and the right eye images of the plurality of captured images are arranged in a second spherical coordinate system, A spherical image generator for generating spherical images; And
    And a data storage unit for storing the left-eye spherical image and the right-eye spherical image.
  9. 9. The apparatus of claim 8, wherein the spherical image generator comprises:
    A left eyeball image generating unit for stitching left eye images of the plurality of captured images arranged in the first rectangular coordinate system to generate the left eyeball image; And
    And a right eyeball image generating unit for stitching right eye images of the plurality of captured images arranged in the second spherical coordinate system to generate the right eye spherical image.
  10. 10. The method of claim 9,
    Wherein the left eyeball image generating unit determines left eye images to be stitched according to a matching result of the left eye image and the right eye image of the plurality of captured images,
    Wherein the right eyeball image generating unit determines right eye images to be stitched according to a matching result of the left eye image and the right eye image of the plurality of captured images.
  11. 9. The method of claim 8,
    The data storage unit stores,
    Storing the coordinate values of the first rectangular coordinate system for each pixel of the left eyeball image and the left eyeball image together,
    And stores coordinate values of the right-eye spherical image and the second rectangular coordinate system for each pixel of the right-eye spherical image together.
  12. A data storage unit for storing a left eyeball image in which pixels are arranged in a first rectangular coordinate system and a right eyeball image in which pixels are arranged in a second rectangular coordinate system;
    A playback direction determination unit for detecting a tilt of the control device and determining playback direction information according to the detected tilt;
    A reproduction area determination unit that determines a reproduction area corresponding to the reproduction direction information;
    A planar three-dimensional image generation unit for generating a planar three-dimensional image corresponding to the reproduction area from the left-eyeball image and the right-eyeball image; And
    And a reproducing unit for reproducing the plane 3D image.
  13. 13. The three-dimensional image reproducing apparatus of claim 12, further comprising an image size determining unit that determines the size of the planar three-dimensional image according to the movement of the controller.
  14. 13. The method of claim 12,
    Wherein the planar three-dimensional image generation unit generates a planar left eye image and a planar right eye image by arranging pixel values of the reproduction area of the left eyeball image and the right eyeball image on a plane,
    Wherein the reproducing unit alternately reproduces the planar left eye image and the planar right eye image.
  15. A computer-readable recording medium storing computer program codes for performing a three-dimensional image generation method when executed by a processor, the method comprising:
    Acquiring a plurality of captured images including a left eye image and a right eye image, and direction information at the time of shooting for each of the plurality of captured images;
    Eye images of the plurality of captured images are arranged in a first rectangular coordinate system to generate a left-eyeball image using the direction information of the plurality of captured images, and the right-eye images of the plurality of captured images are arranged in a second rectangular coordinate system Creating a right eye spherical image; And
    And storing the left-eye spherical image and the right-eye spherical image.
  16. 16. The method of claim 15, wherein the three-
    Stitching left eye images of the plurality of captured images arranged in the first rectangular coordinate system to generate the left eyeball image; And
    Stitching right eye images of the plurality of captured images arranged in the second spherical coordinate system to generate the right eye spherical image.
  17. 16. The method of claim 15,
    Storing the coordinate values of the first rectangular coordinate system for each pixel of the left eyeball image and the left eyeball image together; And
    And storing the right eye spherical image and the coordinate values of the second spherical coordinate system for each pixel of the right eye spherical image together.
  18. A computer-readable recording medium storing computer program codes for performing a three-dimensional image reproducing method when read and executed by a processor, the method comprising:
    Loading a left eyeball image in which pixels are arranged in a first rectangular coordinate system and a right eyeball image in which pixels are arranged in a second rectangular coordinate system;
    Detecting a tilt of the control device;
    Determining reproduction direction information according to the detected inclination;
    Determining a reproduction area corresponding to the reproduction direction information;
    Generating a plane 3D image using pixel values of the reproduction area of the left eyeball image and the right eyeball image; And
    And reproducing the plane three-dimensional image.
  19. 19. The method of claim 18, wherein the three-
    And determining the size of the planar three-dimensional image according to the movement of the control device.
  20. 19. The method of claim 18, wherein generating the planar three-
    And generating a planar left eye image and a planar right eye image by arranging pixel values of the reproduction area of the left eyeball image and the right eye spherical image on a plane,
    Wherein the step of reproducing the planar three-dimensional image alternately reproduces the planar left-eye image and the planar right-eye image.

KR1020130076615A 2013-07-01 2013-07-01 Apparatus and method for generating or reproducing three-dimensional image KR20150003576A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020130076615A KR20150003576A (en) 2013-07-01 2013-07-01 Apparatus and method for generating or reproducing three-dimensional image

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
KR1020130076615A KR20150003576A (en) 2013-07-01 2013-07-01 Apparatus and method for generating or reproducing three-dimensional image
EP14150495.1A EP2821969A1 (en) 2013-07-01 2014-01-08 Apparatus and method for generating or displaying three-dimensional image
US14/155,783 US20150002641A1 (en) 2013-07-01 2014-01-15 Apparatus and method for generating or displaying three-dimensional image

Publications (1)

Publication Number Publication Date
KR20150003576A true KR20150003576A (en) 2015-01-09

Family

ID=50028753

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020130076615A KR20150003576A (en) 2013-07-01 2013-07-01 Apparatus and method for generating or reproducing three-dimensional image

Country Status (3)

Country Link
US (1) US20150002641A1 (en)
EP (1) EP2821969A1 (en)
KR (1) KR20150003576A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101639275B1 (en) * 2015-02-17 2016-07-14 서울과학기술대학교 산학협력단 The method of 360 degrees spherical rendering display and auto video analytics using real-time image acquisition cameras
WO2018074850A1 (en) * 2016-10-18 2018-04-26 삼성전자 주식회사 Image processing apparatus and image processing method therefor

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10313656B2 (en) * 2014-09-22 2019-06-04 Samsung Electronics Company Ltd. Image stitching for three-dimensional video
GB2555724A (en) * 2015-09-15 2018-05-09 Google Llc General spherical capture methods
US10217189B2 (en) * 2015-09-16 2019-02-26 Google Llc General spherical capture methods
US10339627B2 (en) * 2016-10-10 2019-07-02 Gopro, Inc. Apparatus and methods for the optimal stitch zone calculation of a generated projection of a spherical image
US20180276116A1 (en) * 2017-03-21 2018-09-27 Western Digital Technologies, Inc. Storage System and Method for Adaptive Scheduling of Background Operations

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6486908B1 (en) * 1998-05-27 2002-11-26 Industrial Technology Research Institute Image-based method and system for building spherical panoramas
IL139995A (en) * 2000-11-29 2007-07-24 Rvc Llc System and method for spherical stereoscopic photographing
US8369607B2 (en) * 2002-03-27 2013-02-05 Sanyo Electric Co., Ltd. Method and apparatus for processing three-dimensional images

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101639275B1 (en) * 2015-02-17 2016-07-14 서울과학기술대학교 산학협력단 The method of 360 degrees spherical rendering display and auto video analytics using real-time image acquisition cameras
WO2018074850A1 (en) * 2016-10-18 2018-04-26 삼성전자 주식회사 Image processing apparatus and image processing method therefor

Also Published As

Publication number Publication date
EP2821969A1 (en) 2015-01-07
US20150002641A1 (en) 2015-01-01

Similar Documents

Publication Publication Date Title
JP4737573B2 (en) 3D image output apparatus and method
JP5963422B2 (en) Imaging apparatus, display apparatus, computer program, and stereoscopic image display system
US8749620B1 (en) 3D light field cameras, images and files, and methods of using, operating, processing and viewing same
JP4823179B2 (en) Imaging apparatus and imaging control method
JP4938093B2 (en) System and method for region classification of 2D images for 2D-TO-3D conversion
CN103081455B (en) The multiple images being captured from handheld device carry out portrait images synthesis
JP4657313B2 (en) Stereoscopic image display apparatus and method, and program
US9412206B2 (en) Systems and methods for the manipulation of captured light field image data
EP2357836A2 (en) Method and apparatus for processing three-dimensional images
JP2004104330A (en) Image processor, image processing method, recording medium, and program
JP5679978B2 (en) Stereoscopic image alignment apparatus, stereoscopic image alignment method, and program thereof
JP5115139B2 (en) Composition determination apparatus, composition determination method, and program
JP2013192229A (en) Two dimensional/three dimensional digital information acquisition and display device
JP2014502818A (en) Primary image and secondary image image capturing apparatus and method for image processing
KR20150023370A (en) Method and apparatus for fusion of images
CN102783162B (en) Camera head
JPWO2011136190A1 (en) Stereoscopic image reproduction apparatus and method, stereoscopic imaging apparatus, and stereoscopic display apparatus
JP5968107B2 (en) Image processing method, image processing apparatus, and program
CN102282857A (en) Recording medium imaging apparatus and method, and a program for use
JP2004007395A (en) Stereoscopic image processing method and device
JP5620651B2 (en) Reproduction device, imaging device, and control method thereof
US8599245B2 (en) Image processing apparatus, camera, and image processing method
JP6029380B2 (en) Image processing apparatus, imaging apparatus including image processing apparatus, image processing method, and program
CN102595168A (en) Seamless left/right views for 360-degree stereoscopic video
JP2004007396A (en) Stereoscopic image processing method and device

Legal Events

Date Code Title Description
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right