WO2009127910A1

WO2009127910A1 - Realistic parallax visual system

Info

Publication number: WO2009127910A1
Application number: PCT/IB2008/053046
Authority: WO
Inventors: Wijemuni Gunawardana Asanka Jayamal De Silva Rajakaruna
Original assignee: Rajakaruna Wijemuni Gunawardan
Priority date: 2008-04-18
Filing date: 2008-04-18
Publication date: 2009-10-22

Abstract

Asystem comprising a 3D capturing device and a 3D display device, wherein: -said 3D capturing device comprises -and imager, -a first optical means arranged to project the incoming light onto said imager, and -a first scanning pinhole arranged to be moved throughout an entire first viewing area, therebysequentially transmitting the light from a scene through multiple viewpoint of said first viewing area to said first optical means, wherein said imager sequentally captures multiple element-views respectively associated to said sequentialy scanned multiple viewpoints of said first viewing area; -and said 3D display device comprises a display area arranged to sequentially displa said multiple element-views,. a second optical means arranged to project the light from said display area, and. a second scanning pinhole arranged to be moved throughout an entire second viewing area,- thereby sequentially transmitting said multiple element-views ftom siad second optical means through corresponding multiple viewpoints of said second viewing area to the viewer. Said 3D display device thus enables the viewer to see the 3D image with naked eyes.

Description

Field of the invention

The invention is based on an advanced and absolutely novel theory. Particularly, invention is related to 3-D televising field.

Background-art related to the invention

Presently available 3-D visual systems are divided in to two major categories as Holograms and Stereoscopes.

Hologram can capture only little objects, which are able to be placed before laser sources; it is unable to capture or display large objects/sceneries (a mountain for example) with the same. And, in most cases a monochromatic and still one uses, since colour and/or motion hologram images are difficult. For these reasons, Hologram is not widely used in video field.

The Stereoscope that televises any large, moving and colour visuals is the most widely using 3-D visual method at the present. But, unlike hologram, stereoscope shows only a particular aspect of the scene according to instantaneous viewpoint of pair of stereoscopic cameras; it does not allow viewer to see other aspects of the scene (despite all changes in viewpoint of viewer). The stereoscopic TV was not popular among ordinary users since it wants viewer to wear a special goggle.

Auto-stereoscopic system is an alternative solution that has overcome those problems for some degree; it allows viewer to view 3-D visuals with naked eye. Nevertheless, auto-stereoscope imposes considerable restrictions on position and orientation of viewer relative to usual stereoscope. There are some modern Autostereoscopic systems that allow to view from any view points (before of screen), it is not considered as a good solution as it still need viewer to wear a gadget (to inform system about his position). Besides, they allow only one viewer for a time. There are some auto-stereoscopic systems that display different aspects of visual. But it shows only a limited number of discrete aspects and viewpoint can only be changed horizontally.

According to all of these facts, it is true to say that the 3-D visual technology has been stuck in a great problem in making parallax of vision and searching for more practical and comfortable viewing methods; there is a growing need for systems that shows more aspects of visual and that does not need any wearing or limitation in position or orientation of viewer.

Of course, the present invention fulfils all these requirements extremely successful:

Each viewer before the invented display-unit is allowed to see 3-D visual from any position and from any vision-angle in order to see any aspect of the same (Figure-1). It does not need viewer to wear a goggle or any other gadget. Viewing such extremely realistic, life-sized view through display unit is akin to viewing a real scene through a window.

Advantageous effects of the Invention relative to the background-art

It is obvious that the present invention can extremely expand the entertaining and communicating ability of realistic 3-D vision. Besides, it would be popular among ordinary users since it does not need viewers to wear any goggle or any limitation in position or orientation.

At the present, stereoscopic cameras are employed in planetary observing unmanned robotic units. Those cameras often need to be changed in viewpoint to see different views. It makes problems as controlling signals, for each change, takes a considerable time to be exchanged between planet and earth. Plus, each change reduces life-time of entire mission as it consumes power from ^λλun-rechargeable" batteries. Substituting the present invention in stereoscope instead solves the problem as viewer on the earth can see any view of the planetary surface, transmitted by unmanned robotic unit, without any change in camera angle.

In medicine, invented system helps physician to observe MRI or such scanned images very effectively. Plus, above explained ability to show different aspects without camera movement, is very advantageous also in telemedicine. Not only in various technical fields, but the present invention applicable also in aesthetical fields such as dramas. Principle of the invention:

Parallax of a real scene can be seen through a window. The scene Is visible since light field emanated from the same is allowed through the window. If we cover the window with an opaque sheet, the scene will disappear since the light field has been interrupted.

But, if we make a pinhole on the opaque sheet, an inverted 2-D view of the scene is projected through the same as in a pinhole camera (Figure- 2.a: but, still it is unable to see the view without getting it projected on to a screen).

If we make a number of such pinholes on the opaque sheet, it will project a view per each pinhole, simultaneously; each view shows corresponding aspect of the scene according to the same hole-position (viewpoint).

What would result, if the number of holes over the whole area of the opaque sheet is increased in order that making it a "net"? Obviously the scene in front of the window will tend to be re-appeared. This phenomenon implies that: ^{• v}

A scene that seen through a window has been formed with a large array of 2-D element-views of the scene where there focal points exist compactly over the window- aria (Theory)

This is analogous with usual (2-D) TV theory where visual is formed with a large compact array of colour-elements (pixels) over screen (see the table in page-6)

Usual 2-D TV makes visuals by raster-scanning a light-dot on a screen rather than illuminating all pixels simultaneously. By following that example, it also must be able to make the above scene visible by raster- scanning a single pinhole rather than making a large number of pinholes. To do so, it must move the opaque sheet rapidly in order that raster- scanning pinhole over the window (actually it may not be practical to make a so rapid mechanical movement; this is only a concept to explain the principle in simple form). Requirements to display a Realistic Parallax visual:

As in above conceptual method, a Realistic Parallax visual" can be displayed if 3 requirements are fulfilled:

1) There must be a pinhole on an opaque sheet in front of viewer

2) The pinhole must be raster-scanned rapidly with projecting a corresponding 2-D view of the visual per each instantaneous position

3) Viewpoint of the each 2-D view is the instantaneous position of the pinhole

Televising system of the present invention is based on these requirements.

A conceptual method of televising a Realistic Parallax visual

Figure-2 illustrates a conceptual (impractical) process merely for explaining way of televising a Realistic Parallax visual, in simple form. Figure-2.b shows a couple of parallel windows exist by some distance. A large opaque box has been inserted as filling total gap between them. Each of two opposite box-walls, which are been contacted with windows, has a pinhole.

A view of scene before 1^st window is allowed through 1^st pinhole (akin to the circumstance shown in Figure-2.a). A high-speed imager has been fixed in the box in order to capture the image. A source-screen (a highspeed 2-D video screen) has been fixed in order to displaying the view captured-and-sent by the imager. The displayed view is projected out through the 2^nd pinhole on 2^nd window.

Thus, as seen to viewer in front of the 2nd window this circumstance is akin to the previous circumstance shown in Figure-2.a (viz a 2-D view is projected out towards him through a pinhole on a window). And, it is possible to raster-scan both pinholes over the windows, with synchronism to each other, by sliding the box between the windows. Then ...

1) There is a pinhole on opaque shutter in front of viewer

2) The pinhole is raster-scanned rapidly with projecting a corresponding 2-D view of the scene per each instantaneous position 3) Viewpoint of the each 2-D view is the instantaneous position of the pinhole

Thus, above explained three requirements to display a Realistic Parallax visual" have been fulfilled. Therefore, such a visual of the scene before of 1^st window must be appeared through the 2^nd window.

*Even if this method is not practical, it shows the way of televising a Realistic Parallax visual". The televising process is best understood by considering analogy between the same and usual 2-D TV (See the table below).

Tab/e:The analogy between Realistic Parallax TV" and a usual 2-D TV

Usual TV Invented TV

(Makes 2-D visuals) (Makes Realistic Parallax visuals")

There's a movable light-spot on There's a movable pinhole on the the normally dark TV-screen normally closed (opaque) TV- window

The light-dot is setup to raster- The hole is setup to raster-scan on the scan on the TV-screen, with TV-window, with synchronism to the synchronism to the camera- camera- window imager (CCD. / Tube)

Instantaneous colour of Instantaneous element-view scanning light-spot is changing projecting through the scanning hole according to the information of is changing according to the visual may be conveyed information of visual may be conveyed

A 2-D Video-frame is made with A Realistic Parallax Video-frame is a large number of color- made with a large number of samples appears on a compact element-views, projected through array of pixels -on the TV a compact array of focal points -on screen- as there's a sample per the TV window- as there's a view per each pixel. each focal point. Motion conveys as a succession Motion conveys as a succession of of 2-D still frames, in a sufficient Realistic Parallax still frames, in a rate (usually 25fps) sufficient rate (usually 25fps)

Realistic Parallax televising system (practical)

First step of convert above conceptual system (Figure-2.b) in to a practical system is substituting a couple of electronic shutters (preferably Liquid Crystal Displays) instead of above holed box-walls, as shown in Figure-2.c; two box-walls which were contacted with windows have been removed and a shutter has placed over each window. Function of each shutter is making a raster-scanning-pinhole (as previously done by moving box- walls).

Whole area of shutter has been divided in to a large number of tiny pixels. Each pixel is normalcy closed (opaque); no any position on shutter allows light to pass through, in normal state. It makes a pinhole that allows light to pass through, by switching a pixel to open. The pinhole is raster- scanned over the shutter (over the window) by shifting the same pixel-to- pixel; subsequent pixel is opened soon after closing the previous one and so on. Both scanning-pinholes of two shutters must be traveled with synchronism to each other.

Of course the shutters can attain a sufficient high frame-rate required to make a Realistic Parallax visual. But problem still remains: Still it requires sliding the box in order to tracking pinholes; at all times imager and source-screen must be at right front of 1^st and 2^nd scanning-pinholes respectively, for expected results. Solution for this is employing an "optical regulator" illustrated in next chapter. Function of optical regulator is refracting light-beam allowed through pinhole in order that focusing view, carried by the beam, on to a steady area despite all movement of pinhole (Figure-2.d).

Thus, when an optical regulator is coupled to 1^st shutter corresponding view allowed through the pinhole at each position (pixel) can be captured with a "fixed" imager. And according to reciprocal effect, also not necessary to move the source-screen if another optical regulator is coupled to 2^nd shutter (Figure-2.d); instantaneous view displayed on the "fixed" source-screen is projected through scanning-hole of 2^nd shutter at its instantaneous position (pixel); no any information (but focal point) of view has been changed. Thus, when an optical regulator is placed on each window...

1) Two pinholes on windows are rapidly raster-scanned with synchronism to each other.

2) For any instant, view entered through 1^st pinhole is projected out through the 2^nd pinhole.

According to these facts, functionality of this arrangement (Figure-2.d) is akin to the functionality of above (conceptual) arrangement shown in Figure-2.b. Therefore, a Realistic Parallax visual" of the scene before of 1st window must be appeared through the 2nd window as in the conceptual method. First part of the arrangement, comprising 1st window, 1st shutter and 1st optical regulator is a model for invented camera. Similarly second part is a model for invented TV. A preferred embodiment for a practical camera is shown in Figure-3; outstanding difference of the same in outlook relative to a usual 2-D camera is having a very wide window (face). Figure-4 shows a preferred embodiment for a practical TV.

Characteristics of optical regulator (further explanation)

As explained in above chapter, function of optical regulator is refracting light beam allowed through scanning-pinhole in order that projecting element-view, carried by the light beam, on to the imager at a fixed position. Shutter, optical regulator and imager must be centred to each other for optimal results. Then, as illustrated in Figure-5, for any (instantaneous) position of scanning-pinhole, light beam allowed through the same has following characteristics:

^♦(ASSUMPTION: width of pinhole is extremely small so that allowing only one ray per each direction)

1) Light ray perpendicular to the optical regulator is projected onto the centre CO") of the imager.

2) Light ray entered through the pinhole which makes an "α" angle with perpendicular (to optical regulator ) is projected onto an "A" point and ray which makes a ^Λλ-α" angle is projected onto a "B" point where AO=OB and AoB= 180°

Thus, for any instantaneous position of scanning-pinhole, the view brought by the light beam has been projected and centered on the imager in original form. Therefore if an optical regulator is employed:

Each element-view allowed through scanning-pinhole can be captured by a "fixed" imager in invented camera. And according to reciprocal effect of this...

Each element-view produced by a fixed source-screen in invented TV can be projected out through scanning-pinhole at corresponding position.

Theory of optical regulator

Theory of optical regulator is simply understood by the straightforward practical steps, described below

Stage A

Step-l (Figure-6:A.l):

A light dot (such as a tiny elec. bulb) is placed on the focal plane of a convergent lens: light, emitted from the light-dot progresses beyond the lens -after refraction- as a parallel beam so that making an image of the light-dot in infinity; direction of the beam depends on the position of the light-dot.

Step-2 (Figure-6:A.2):

The lens is covered with an opaque sheet that having a pinhole: then parallel light beam is almost covered, but allows only a ray (a narrow beam) through the pinhole.

Step-3 (Figure-6:A:3):

Pinhole is moved over the lens-area by dragging the opaque sheet. Then light-ray is (virtually) accompanied by the same. But neither direction nor colour of the same changes despite all movements of the pinhole (they depend only on position and colour of the light-dot respectively)

Stage B

In stage "B'', the light dots is replaced with a flat video-screen and above three practical steps are re-done.

*Note: In this case, each pixel of the screen works as a separate light-dot so it is able to understand the result of each step in stage "B" by comparing the result of corresponding step in stage ^λλA" (compare ^Xλstep-1" of "stage-A" with "step-l" of ^Λλstage-B" and so on) Step-1 (Fiαure-6.B.l):

A video-screen is laid on the focal plane of the lens. Light, emitted from each pixel progresses beyond the lens -after refraction- as a parallel beam so that making an image of the pixel in infinity. Thus it makes an image of the whole picture what is displayed on the screen, in infinity.

Step-2 (Fiαure-6.B.2):

The lens is covered with an opaque sheet that having a pinhole: then each parallel light beam is almost covered, but allows only a ray (a narrow beam) through the pinhole. Thus, light beam, made with those light rays, still carries the whole image of the video even if light is almost covered (But, brightness of the image has been decreased according to the iris- effect).

Step-3 (FiQure-6.B.3):

Pinhole is moved within the lens-area by dragging the opaque sheet. Then light beam is (virtually) accompanied by the same. But no any information (but focal point) of the image, carried by the light beam, is changed (information of image is tally with instantaneous video frame and independent of hole position).

If the video screen is displaying a still picture in above circumstance (Figure-6A3), the scanning pinhole projects the same image all the time. But if video screen displays a motion picture (sequence of video frames); "Tor any instant, the instantaneous video frame on the fixed video screen is projected through the scanning-pinhole at the instantaneous position" [A]

But if there is a scene before the pinhole, a view of the same is carried by light rays allowed through the pinhole (as in Figure- 2.a); at all times viewpoint of the view is instantaneous hole-position itself. According to reciprocal effect of [A], each view allowed through the pinhole must be focused onto the video screen. Therefore if the video screen is replaced with an imager:

"For any instant; the instantaneous view allowed through the scanning-pinhole at its instantaneous position is projected on to the fixed imager" [B]

According to above [A] and [B] characteristics a convergent lens can be employed as optical regulator; characteristic [A] is applied in the invented TV and [B] is applied in the invented camera. Note: Described, camera system is appropriate to capturing images with electro magnetic medium within and/or outside visible range. Imager (in sensitivity), optical regulator (in refraction) and housing (in opaqueness) must be fit to the applying waveband.

Conveying visuals

Outputs of the invented camera and usual (2-D) camera are analogous to each other as both of them are sequences of 2-D frames. But a sequence of 2-D frames (element-frames), outputted by the invented camera, has an extra high frame rate and —therefore— conveying through a usual video link is impractical. In wide applications such as broadcasting and recording it must convert the sequence of element-frames in to electronic information and need some special digital data compression means.

As a straightforward solution, an optical fiber link can be employed (as in endoscope) in order to bring element-views from camera to TV without converting to electronic signals.

Capturing visuals with a 2-D camera:

According to the theory, a Realistic Parallax visual of a scene is an array of 2-D images taken from a large number of different viewpoints over an area. So, it must be able to make such a visual as an array of 2-D pictures taken by using just a 2-D camera instead of the invented camera. In this case, the 2D camera must be traveled over an aria in front of the scene, akin to the scanning-pinhole of invented camera. (This method is capable for capturing landscapes as aerial visuals)

Making Computer Generated Images: The way of capturing a Realistic Parallax visual with an array of 2-D pictures has been described in above chapter. So, it must be able to do this also in computer generated virtual 3-D environment besides in a real environment: a Virtual Realistic Parallax frame" is formed with a gamut of 2-D images, captured by a virtual 2-D camera, in a computer generated virtual environment.

Making Still photographs: A video frame taken from a Realistic Parallax video comprises information for a Realistic Parallax still photograph. As well as, a hologram is akin to a Realistic Parallax still photograph. So it must be able to make such a photograph by converting the video frame into a hologram-image (advantage of this is it can make images of even large sceneries/objects that cannot be captured with hologram-technique).

This is done in three steps:

1) Converting the video frame into a computer based virtual 3-D image (as explained below).

2) Synthesizing relevant light interference pattern by computing the virtual image (of L).

3) Getting the light interference pattern (of 2.) printed onto a hologram slide.

Note: Similarly, a "Computer generated Realistic Parallax image" can be converted to a hologram slide; above step 1 is not required in this case.

Converting an original image into a computer-based virtual image:

In stereoscopic method, spatial position of a point in televised scene can be determined by considering geometry of its image-positions in each (left and right) view.

In the same way, in the invented system, spatial position of any point in televised scene must be able to determine by selecting a couple of element-frames akin to two views of a stereoscopic system. Then, spatial configuration of each object in the scene is recreated as a "wire diagram" in computer based virtual 3-D space by joining a number of so-determined points on each object. Then, the virtual 3-D image can be produced by applying colour information, received from the original image, on the wire diagram.

Brief description about drawings

Figure-1

Viewing different profiles of visual from different viewpoints, through the invented TV

Figure-2

Illustrates a conceptual (non practical) process merely for explaining essence of Realistic Parallax televising, in simple form.

Figure-2. a : Light allowed through opaque sheet brings a 2-D view of the scene

Figure-2. b : Arrangement between two windows brings above 2-D view from

1st window to 2nd window; the Realistic Parallax view is brought from 1st window to 2nd window if pinholes are raster scanned rapidly by moving the box.

Figure-2. c : Box- wall has been removed and an electronic shutter has been fixed, on each window.

Figure-2.d : An optical regulator (lens) has placed on each shutter; no any mechanical movement needed anymore

Figure-3

A preferred embodiment for practical Realistic Parallax camera

Figure-4

A preferred embodiment for practical Realistic Parallax TV

Figure-5 Figure-5.a: Function of Optical Regulator in camera

Figure-5. b: Function of Optical Regulator in TV

Figure-6

Steps of a straightforward practical for understanding "how optical regulator works?"

Claims

CLAIMSWhat is claimed is:

1. A process of capturing and reproducing a realistic parallax (3D) visual of a real scene or a computer-generated virtual scene; the visual represents all aspects of the scene that can be seen through a considered two-dimensional viewing-aria (may be a window) in front of the scene. Viewer is able to see any of the aspects in reproduced image -by naked eyes- with accordance to his random position-and-orientation relative to display-unit. Visual is captured as a gamut of element-views, taken from a large number of viewpoints over the total viewing-aria in front of the scene:

In a real environment, element-views are captured by an inbuilt imager of camera-device. In a computer based virtual environment, the element- views are captured by computing; as frames of a "computer-generated movie file".

The visual is reproduced by projecting the captured array of element- views from display-aria; element-view taken from each view point on viewing-aria is projected to the outer space through the corresponding point on display-aria as a divergent light beam.

2. A system for displaying a realistic parallax visual as in Claim-1, comprising:

a) Rendering relevant succession of element-views on the source- screen at a high refreshing rate b) Regulating light field emanated from the source screen by an optical means c) Blocking the regulated light by a shutter in order that allowing light to outer space only through a pinhole moving over the shutter-aria; at all times, the allowed light beam conveys instantaneous element-view on said source-screen through the moving-pinhole at instantaneous position d) Displaying the realistic parallax visual by mapping each element- view onto corresponding position of scanning-pinhole

3. A system for capturing a realistic parallax visual of a real scene as in Claim-1, comprising: a) Blocking the light field emanated from said scene, by a shutter in order that allowing light to inside only through a pinhole moving over the shutter- aria b) Regulating the allowed light beam by an optical means in order that projecting visual information, brought through the moving- pinhole, onto an imager c) Capturing the visual information by imager

4. Systems as in claim 1, 2 and 3, where in;

a) Total aria of the shutter has been divided into a plurality of little opaque segments. b) A hole is formed by electronically switching a segment on (into transparent state) c) The hole is shifted to subsequent position by switching the segment-at-the-subsequent position on soon after closing the previous segment. d) The hole is scanned (traveled) by shifting the hole (as in V) segment-to- segment.

5. System as in claim 1, 2, 3 and 4 wherein a Liquid Crystal Display is employed as shutter

6. A system as in Claim-1, for capturing a realistic parallax visual of a scene to be displayed by a realistic parallax display devise, as a gamut of 2D views (may be still photos or video frames) taken from a large number of viewpoints by a 2D camera-device.

7. A system as in Claim-1 and 6, for capturing a realistic parallax visual of a scene in a computer based virtual environment to be displayed by a realistic parallax display devise, as a gamut of

2D views (video frames) taken from a large number of viewpoints by a virtual 2D camera (as a computer generated movie file).

8. A method of converting a realistic parallax image into a computer- based virtual 3-D image, comprising the steps of:

a) Identifying a large number of spatial positions on image by a means of computing b) Building a 3-D wire-diagrams in computer-based virtual 3-D space by joining the identified positions c) Mapping colour information on to the built wire diagram (optional)

9. A method of displaying a realistic parallax still image on a hologram-slide, comprising the steps of:

a) Converting a "realistic parallax video frame" into a computer-based virtual 3-D image, as in claim 8. b) Resulting relevant light interference pattern from the resulted virtual image by means of software. c) Getting the same pattern printed on to a hologram slide to be displayed as a hologram image.

10. Applying optical fibers to convey realistic parallax visuals

11. System as in claim 1, 3, 4, 5, 6 and 10 capturing images with electro magnetic medium within visible range

12. System as in claim 1, 3, 4, 5, 6 and 10 capturing images with electro magnetic medium outside visible range