Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/08—Volume rendering
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/06—Ray-tracing
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/50—Lighting effects
  • G06T15/506—Illumination models
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T19/00—Manipulating 3D models or images for computer graphics
  • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2200/00—Indexing scheme for image data processing or generation, in general
  • G06T2200/21—Indexing scheme for image data processing or generation, in general involving computational photography
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10052—Images from lightfield camera

Definitions

• the invention lies in the field of computational. More precisely, the invention relates to a representation format that can be used for transmission, rendering, proceedings of light- field data.
• a light field represents the amount of light passing through each point of a three- dimensional (3D) space along each possible direction. It is modelled by a function of seven variables representing radiance as a function of time, wavelength, position and direction. In Computer Graphics, the support of light field is reduced to four-dimensional (4D) oriented line space.
• 4D light-field data Compared to classical two-dimensional or 2D images obtained from a conventional camera, 4D light-field data enable a user to have access to more post-processing features that enhance the rendering of images and the interactivity with the user. For example, with 4D light-field data, it is possible to perform refocusing of images with freely selected distances of focalization meaning that the position of a focal plane can be specified/ selected a posteriori, as well as changing slightly the point of view in the scene of an image. In order to acquire 4D light-field data, several techniques can be used. For example, a plenoptic camera is able to acquire 4D light- field data. Details of the architecture of a plenoptic camera are provided in Figure 1A.
• Figure 1A is a diagram schematically representing a plenoptic camera 100.
• the plenoptic camera 100 comprises a main lens 101, a microlens array 102 comprising a plurality of micro-lenses 103 arranged in a two-dimensional array and an image sensor 104.
• Figure IB represents a multi-array camera 110.
• the multi-array camera 110 comprises a lens array 112 and an image sensor 114.
• the main lens 101 receives light from an object (not shown on the figure) in an object field of the main lens 101 and passes the light through an image field of the main lens 101.
• Another way of acquiring a 4D light field is to use a conventional camera that is configured to capture a sequence of 2D images of a same scene at different focal planes.
• a conventional camera that is configured to capture a sequence of 2D images of a same scene at different focal planes.
• the technique described in the document "Light ray field capture using focal plane sweeping and its optical reconstruction using 3D displays" by J.-H. Park et al., published in OPTICS EXPRESS, Vol. 22, No. 21 , in October 2014 may be used to achieve the acquisition of 4D light field data by means of a conventional camera.
• 4D light-field data can be represented, when recorded by a plenoptic camera by a collection of micro-lens images. 4D light-field data in this representation are named raw images or raw 4D light-field data. Secondly, 4D light-field data can be represented, either when recorded by a plenoptic camera or by a camera array, by a set of sub-aperture images.
• a sub-aperture image corresponds to a captured image of a scene from a point of view, the point of view being slightly different between two sub-aperture images. These sub-aperture images give information about the parallax and depth of the imaged scene.
• 4D light-field data can be represented by a set of epipolar images see for example the article entitled : "Generating EPI Representation of a 4D Eight Fields with a Single Lens Focused Plenoptic Camera", by S. Wanner and al., published in the conference proceedings of ISVC 2011.
• light- field data can take up large amounts of storage space, up to several Tera Bytes (TB), which can make storage cumbersome and processing less efficient.
• TB Tera Bytes
• light- field acquisition devices are extremely heterogeneous.
• Light-field cameras are of different types, for example plenoptic or camera arrays. Within each type there are many differences such as different optical arrangements, or micro-lenses of different focal length sand, above all, each camera has its own proprietary file format. At present there is no standard supporting the acquisition and transmission of multi-dimensional information for an exhaustive over- view of the different parameters upon which a light- field depends. As such, acquired light-field data for different cameras have a diversity of formats. The present invention has been devised with the foregoing in mind.
• first generating ray a first rotation angle of a first straight line describing a surface of a hyperboloid of one sheet representing the pixel beam, called first generating ray, around a second straight line being a revolution axis of the hyperboloid, said rotation transforming the first generating ray into a second generating ray crossing a reference straight line,
• a pixel beam is defined as a pencil of rays of light that reaches a given pixel of a sensor of an optical acquisition system when propagating through the optical system of the optical acquisition system.
• a pixel beam is represented by a hyperboloid of one sheet.
• a hyperboloid of one sheet is a ruled surface that supports the notion of pencil of rays of light and is compatible with the notion of the "etendue" of physical light beams.
• Etendue is a property of light in an optical system which characterizes how “spread out” the light is in area and angle.
• the "etendue” is the product of the area of the source and ae solid angle that an optical system's entrance pupil subtends as seen from the source. Equivalently, from the optical system point of view, the “etendue” equals the area of the entrance pupil times the solid angle the source subtends as seen from the pupil.
• One distinctive feature of the “etendue” is that it never decreases in any optical system.
• the “etendue” is related to the Lagrange invariant and the optical invariant, which share the property of being constant in an ideal optical system.
• the radiance of an optical system is equal to the derivative of the radiant flux with respect to the "etendue". It is an advantage to represent 4D light- field data using pixel beams because the pixel beams convey per se information related to the "etendue”.
• 4D light-field data may be represented by a collection of pixel beams which can take up large amounts of storage space since pixel beams may be represented by six to ten parameters.
• a compact representation format for 4D light-field data developed by the inventors of the present invention relies on a ray-based representation of the plenoptic function. This compact representation format requires the rays to be sorted in a way that is not random. Indeed, since rays are mapped along lines, it is efficient, in terms of compactness, to store in sequence the parameters of a given line, i.e. the slope related and the intercept, along with the collection of rays belonging to said given line and then a next line and so on.
• Pixel beams may be represented by two rays, called generating rays, describing a surface of the hyperboloid representing the pixel beam.
• Generating rays may suffer strong deviations while passing through the microlenses of the optical acquisition system and thus hit the main lens of the optical acquisition system at larger aperture angles. Aberrations like distortion may thus disturb the collection of generating rays making the use of the compact representation format less efficient.
• the method according to an embodiment of the invention consists in sorting the pairs of generating rays defining the pixel beams of a collection of pixel beams in order to reduce the randomness in position and orientation in order to make the use of the compact representation format more efficient. Furthermore, the method according to an embodiment of the invention constrains the pairs of generating rays defining the pixels beams of a collection of pixel beams in such a way that one of the parameters representing a pixel beam becomes implicit enabling to reduce the dimensions of the data set representing said pixel beam by one dimension thus contributing to the compactness of the compact representation format.
• the reference straight line is parallel to an optical axis of a main lens of the optical acquisition system.
• the reference straight line is parallel to a central axis of a lens-array of the optical acquisition system.
• computing the first, respectively the second, rotation angle consists in:
• the second and the fourth generating rays are sorted, prior to the generation of the set of data representative of the pixel beam, so that the first and the second rotation angles are arranged in increasing order.
• Arranging the second and the fourth generating rays so that the rotation angles in increasing order makes easier the reconstruction of a pixel beam by a rendering device, such as a television set, a smartphone, a tablet, a personal computer, a laptop, etc. by constraining the direction of cross products vectors involved in the computation of the set of data representative of the pixel beam to be reconstructed.
• a rendering device such as a television set, a smartphone, a tablet, a personal computer, a laptop, etc. by constraining the direction of cross products vectors involved in the computation of the set of data representative of the pixel beam to be reconstructed.
• the computed distance between one of the second or the fourth generating ray and the revolution axis of the hyperboloid is the shortest distance between one of the second or the fourth generating ray and the revolution axis of the hyperboloid.
• the shortest distance between one the second or the fourth generating ray and the revolution axis of the hyperboloid corresponds to the waist of the pixel beam, i.e. the smallest section of the hyperboloid representing the pixel beam.
• first generating ray a first rotation angle of a first straight line describing a surface of a hyperboloid of one sheet representing the pixel beam, called first generating ray, around a second straight line being a revolution axis of the hyperboloid, said rotation transforming the first generating ray into a second generating ray crossing a reference straight line,
• Such an apparatus may be embedded in an optical acquisition system such as a plenoptic camera or any other device such as smartphones, laptop, personal computers, tablets, etc.
• an optical acquisition system such as a plenoptic camera or any other device such as smartphones, laptop, personal computers, tablets, etc.
• the processor is configured to compute the first, respectively the second, rotation angle by:
• first straight line being a revolution axis of a hyperboloid of one sheet representing a volume in an object space of an optical acquisition system occupied by a set of rays of light that at least one pixel of a sensor of said optical acquisition system can sense through a pupil of said optical acquisition system, said volume being called a pixel beam, using a distance between said revolution axis of the hyperboloid and a second straight line describing a surface of the hyperboloid, called first generating ray, crossing a reference straight line,
• adevice for rendering a light-field content comprising a processor configured to:
• first straight line being a revolution axis of a hyperboloid of one sheet representing a volume in an object space of an optical acquisition system occupied by a set of rays of light that at least one pixel of a sensor of said optical acquisition system can sense through a pupil of said optical acquisition system, said volume being called a pixel beam, using a distance between said revolution axis of the hyperboloid and a second straight line describing a surface of the hyperboloid, called first generating ray, crossing a reference straight line,
• Such a device capable of rendering a light field content may be a television set, a smartphone, a laptop, a personal computer, a tablet, etc.
• a photosensor configured to capture light projected on the photosensor from the array of micro lenses, the photosensor comprising sets of pixels, each set of pixels being optically associated with a respective micro lens of the array of micro lenses;
• Such a light field imaging device is for example a plenoptic camera.
• ignal representative of a light- field content comprising a set of data representative of a volume in an object space of an optical acquisition system occupied by a set of rays of light that at least one pixel of a sensor of said optical acquisition system can sense through a pupil of said optical acquisition system, said volume being called a pixel beam, the set of data representative of the pixel beam comprising:
• first generating ray a first straight line describing a surface of a hyperboloid of one sheet representing the pixel beam
• Some processes implemented by elements of the invention may be computer implemented. Accordingly, such elements may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module” or "system”. Furthermore, such elements may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.
• a tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like.
• a transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.
• FIG. 1A is a diagram schematically representing a plenoptic camera
• FIG. IB represents a multi-array camera
• FIG. 2A is a functional diagram of a light-field camera according to an embodiment of the invention.
• Figure 2B is a functional diagram of a light-field data formator and light-field data processor according to an embodiment of the invention.
• FIG. 3 is an example of a raw light- field image formed on a photosensor array
• FIG. 4 represents a volume occupied by a set of rays of light in an object space of an optical system of a camera or optical acquisition system
• FIG. 5 represents a hyperboloid of one sheet
• FIG. 6A is a functional block diagram illustrating modules of a device for sorting generating rays of a pixel beam in accordance with one or more embodiments of the invention
• FIG. 6B is a flow chart illustrating steps of a method for sorting the generating rays of a pixel beam in accordance with one or more embodiments of the invention
• FIG. 7A and 7B graphically illustrate the use of reference planes for parameterisation of light-field data in accordance with one or more embodiments of the invention
• FIG. 8 schematically illustrates representation of light-field rays with respect to reference planes in accordance with embodiments of the invention
• FIG. 9A is a flow chart illustrating steps of a method in accordance with one or more embodiments of the invention.
• FIG. 9B is a functional block diagram illustrating modules of a device for providing a light data format in accordance with one or more embodiments of the invention.
• FIG. 10 schematically illustrates parameters for representation of light- field rays in accordance with embodiments of the invention.
• FIG. 11 is a 2D ray diagram graphically illustrating intersection data in accordance with embodiments of the invention.
• FIG. 12 graphically illustrates a digital line generated in accordance with embodiments of the invention.
• FIG. 13 graphically illustrates digitals line generated in accordance with embodiments of the invention
• FIG. 14A-14C graphically illustrate Radon transforms applied to a digital line in accordance with embodiments of the invention.
• FIG 15 is a 2D ray diagram graphically illustrating intersection data for a plurality of cameras in accordance with embodiments of the invention.
• aspects of the present principles can be embodied as a system, method or computer readable medium. Accordingly, aspects of the present principles can take the form of an entirely hardware embodiment, an entirely software embodiment, (including firmware, resident software, micro-code, and so forth) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "circuit", "module”, or “system”. Furthermore, aspects of the present principles can take the form of a computer readable storage medium. Any combination of one or more computer readable storage medium (a) may be utilized.
• Embodiments of the invention provide formatting of light-field data for further processing applications such as format conversion, refocusing, viewpoint change and 3D image generation.
• Figure 2A is a block diagram of a light-field camera device in accordance with an embodiment of the invention.
• the light-field camera comprises an aperture/ shutter 202, a main (objective) lens 201, a micro lens array 210 and a photosensor array 220 in accordance with the light-field camera of Figure 1A.
• the light-field camera includes a shutter release that is activated to capture a light-field image of a subject or scene. It will be appreciated that the functional features may also be applied to the light-field camera of Figure IB.
• the photosensor array 220 provides light-field image data which is acquired by LF Data acquisition module 240 for generation of a light-field data format by light-field data formatting module 250 and/ or for processing by light- field data processor 255.
• light- field data may be stored, after acquisition and after processing, in memory 290 in a raw data format, as sub aperture images or focal stacks, or in a light- field data format in accordance with embodiments of the invention.
• the light- field data formatting module 150 and the light- field data processor 255 are disposed in or integrated into the light-field camera 200.
• the light- field data formatting module 250 and/or the light- field data processor 255 may be provided in a separate component external to the light- field capture camera. The separate component may be local or remote with respect to the light- field image capture device.
• any suitable wired or wireless protocol may be used for transmitting light-field image data to the formatting module 250 or light- field data processor 255; for example the light-field data processor may transfer captured light- field image data and/ or other data via the Internet, a cellular data network, a WiFi network, a Bluetooth® communication protocol, and/ or any other suitable means.
• the light-field data formatting module 250 is configured to generate data representative of the acquired light- field, in accordance with embodiments of the invention.
• the light- field data formatting module 250 may be implemented in software, hardware or a combination thereof.
• the light-field camera 200 may also include a user interface 260 for enabling a user to provide user input to control operation of camera 100 by controller 270.
• Control of the camera may include one or more of control of optical parameters of the camera such as shutter speed, or in the case of an adjustable light-field camera, control of the relative distance between the microlens array and the photosensor, or the relative distance between the objective lens and the microlens array. In some embodiments the relative distances between optical elements of the light-field camera may be manually adjusted. Control of the camera may also include control of other light-field data acquisition parameters, light-field data formatting parameters or light-field processing parameters of the camera.
• a pixel beam 40 is defined as a pencil of rays of light that reaches a given pixel 42 when propagating through the optical system 41 via an entrance pupil 44. As light travels on straight lines in free space, the shape of such a pixel beam 40 can be defined by two sections, one being the conjugate 45 of the pixel 42, and the other being the entrance pupil 44.
• the pixel 42 is defined by its non-null surface and its sensitivity map.
• a hyperboloid of one sheet is a ruled surface
• at least one family of straight lines, called generating rays, rotating around a revolution axis, called chief ray, of the hyperboloid describe such a surface.
• the knowledge of parameters defining the chief ray and any generating ray belonging to a family of generating lines of the hyperboloid are sufficient to define a pixel beam 40, 50.
• a solution for reducing the amount of storage space required to store a representation a pixel beam is described hereinafter in reference to figure 9B.
• chief rays will behave smoothly passing through the microlenses centres of the microlens array of the camera, generating rays suffer from stronger deviations on the borders of the microlenses.
• This disturbance of the generating rays makes it difficult to run the method described in reference to figure 9B since said method works with ordered collections of rays.
• the inventors of the present invention propose a method for sorting the generating rays of a collection of pixel beams of a camera in order to feed the method according to figure 9B with such a sorted collection of generating rays.
• the processor 601 controls operations of the apparatus 600.
• the storage unit 602 stores at least one program capable of sorting the generating rays of a collection of pixel beams of a camera to be executed by the processor 601, and various data, including parameters related to the optical system 210 of the optical acquisition system, parameters used by computations performed by the processor 601, intermediate data of computations performed by the processor 601, and so on.
• the processor 601 may be formed by any known and suitable hardware, or software, or a combination of hardware and software.
• the processor 601 may be formed by dedicated hardware such as a processing circuit, or by a programmable processing unit such as a CPU (Central Processing Unit) that executes a program stored in a memory thereof.
• CPU Central Processing Unit
• the storage unit 602 may be formed by any suitable storage or means capable of storing the program, data, or the like in a computer-readable manner. Examples of the storage unit 602 include non-transitory computer-readable storage media such as semiconductor memory devices, and magnetic, optical, or magneto-optical recording media loaded into a read and write unit.
• the program causes the processor 601 to perform a process for computing parameters representing a volume occupied by a set of rays of light in an object space of an optical system and encoding these parameters with an image captured by the optical acquisition system according to an embodiment of the present disclosure as described hereinafter with reference to figure 9B.
• Figure 6B is a flow chart illustrating the steps of a method for sorting the generating rays of a collection of pixel beams of a camera according to one or more embodiments of the invention.
• the triplet (x 1( y , / ) can define any generating ray p crossing ⁇ .
• the dimensionality is thus reduced from four to three.
• the triplet (x ⁇ y ⁇ , Z / ⁇ i) can define the generating ray ( 1( l ⁇ ), where, is the intersection point between the reference straight line ⁇ and the generating ray p, and Zj ⁇ the coordinate of along Oz.
• the computation of the values of the parameters (X Q , y ⁇ , z 0 ), , b, c and ⁇ ⁇ , 6 y is realized, for example, by running a program capable of modelling a propagation of rays of light through the optical system of the camera.
• a program is for example an optical design program such as Zemax ⁇ , ASAP ⁇ or Code V ⁇ .
• An optical design program is used to design and analyze optical systems.
• An optical design program models the propagation of rays of light through the optical system; and can model the effect of optical elements such as simple lenses, aspheric lenses, gradient index lenses, mirrors, and diffractive optical elements, etc.
• the optical design program may be executed by the processor 601 of the apparatus 600.
• x, y, z are the coordinates of a point belonging to the surface of the hyperboloid
• (x 0 , y 0 , Z Q ) are the coordinates of the centre of the waist of the considered pixel beam.
• a step S603 the processor 601 computes the centering of the hyperboloid on the point of coordinates(x 0 , y 0 , Z 0 ) and then compute the normalization of the hyperboloid which gives :
• T transforming (x, y, z) coordinates into ⁇ X, Y, Z)
• the central axis of the hyperboloid is the Oz axis
• two points belonging to this axis have the following set of coordinates (0,0,0) and (0,0,1) in the ( KZ) coordinate system.
• This central axis of the hyperboloid, transformed back in the original coordinate system (x, y, z), is the chief ray p c of the pixel beam.
• Any of these generating rays, transformed back in the original coordinate system, can be selected as PQ a generating ray of a pixel beam.
• two points GQ which coordinates are (1, 0, 0) and IQ which coordinates are (1, 1,1) in the ( K ) coordinate system as defining the initial generating ray PQ in the ( KZ) coordinate system.
• the processor 601 applies the function T as defined above to a reference straight line ⁇ in the object space of the camera.
• the reference straight line ⁇ is an optical axis of a main lens of the camera.
• the reference straight line ⁇ is a central axis of a lens- array of the camera, in a third embodiment the reference straight line ⁇ is a line with a direction forming an angle inferior or equal to— with the optical axis of the main lens of the
• C3 D — Z A0 . cos ⁇ . dX A — (sin ⁇ — Y A0 ). dX A — Z A0 . sin ⁇ . dY A + (cos ⁇ — X A0 ). dY A — (sin ⁇ — Y A0 ). sin ⁇ . dZ A — (cos ⁇ — X A0 ). cos ⁇ . dZ A
• the two rays ⁇ ⁇ an d PG ⁇ P 6 are selected among four different rays generating the pixel beam.
• a step S607 the transformed generated rays ⁇ ⁇ an d p ⁇ are reordered if needed in the collection of generating rays so that: 0 ⁇ ⁇ ⁇ — ⁇ ⁇ ⁇ ⁇ .
• the conditions 1., 2. and 3. define the chief ray p c from the triplet (p G( p a , PG ⁇ P 6 ⁇ enabling the reconstruction of the pixel beam.
• Figure 7B illustrates a light- field ray, such as generating rays defining a pixel beam, passing through two reference planes PI and P2 used for parameterization positioned parallel to one another and located at known depths and ⁇ 2 respectively.
• the light-field ray intersects the first reference plane P? at depth at intersection point (xi,ji) and intersects the second reference plane P ⁇ at depth ⁇ 2 at intersection point (x2,j2).
• the light- field ray may be identified by four coordinates (>3 ⁇ 4 j?, >3 ⁇ 4 J2) .
• the light- field can thus be parameterized by a pair of reference planes for parameterization Pi, P2 also referred herein as parametrization planes, with each light-field ray being represented as a point
• an origin of the reference co-ordinate system may be placed at the center of a plane P/generated by the basis vectors of the coordinate axis system
• Both sets of equation should deliver the same point x3 as the rendered light- field ray at the new location.
• intersection data (xi,ji, X2, 2) geometrically defining intersection of the generating ray crossing the reference straight line with reference planes Pi, P2 is obtained.
• step S804 2D ray a diagram graphically representing the intersection data (xi, i, X2, 2) is obtained by ray diagram generator module 803.
• the data lines of the ray diagram used to parameterise are sampled by 256 cells providing an image of 256x256 pixels.
• a representative digital line is generated by digital line generation module 805 in step S806.
• digital lines are generated by approximating an analytical line to its nearest grid point, for example by applying Bresenham's algorithm.
• Bresenham's algorithm provides a way to provide a digital line with minimal operation.
• Other methods may apply a fast discrete Radon transform calculation.
• An example of Bresenham application is one adapted from the following reference: http:/ / www.cs.helsinki.fi/group / goa/mallinnus/lines/bresenh.html.
• the digital format defines the data line by two points of a grid (0,d) and (IV- 1, s) d being the intersection corresponding to the value of X2 when and s being the slope parameter corresponding to the value of X2 when From the digital format generated the slope a of each individual line may be expressed as a function of d, N and s, as: s d where: s G ⁇ 0, 1, ... . N - 1 ⁇ and d E ⁇ 0, 1, ... . N - 1 ⁇
• Figure 12 illustrates an example of a digital line generated by application of Bresenham's algorithm.
• Figure 13 illustrates a group of digital lines having the same slope a (or s— d) but different intercepts d, the group of data lines being contiguous.
• the group of data lines is referred to herein as a bundle of lines and corresponds to a beam resulting from the camera not being ideally a pinhole camera.
• Each line addresses different pixels. In other words, one pixel belongs only to a unique line of a bundle with the same slope but different intercepts.
• the upper and lower boundaries of the axis intersections d are given as d max and d respectively.
• the header of the beam can simply contain the slope a and the thickness of the beam defined by the upper and lower boundaries of the axis intersections d max - d
• the ray values will be stored as RGB colors along digital lines whose header can be d and s. Void cells of the ray diagram in the sampled space do not need to be stored. Coordinates xl; x2 of the rays can be deduced from the parameters d, s and from the position of the cell along the digital line.
• Parameters to be estimated from the light-field or from camera's geometry are the slope a the lower and upper bounds of the digital line intercepts (d m i n , dmax) > an d th e digital line parameters (d;, s ) .
• the discrete Radon transform has already been discussed as a tool to measure the support location of the light- field in the ray diagram.
• Figure 14B shows the discrete Radon transform in the digital line parameter space (d, s)of the datalines of Figure 14A.
• Figure 14C is a zoom of the region of interest comprised in Figure 14B.
• the beam of digital lines is located by the search for the maximum value parameters. There could be some offset between the geometrical center of symmetry of the DRT and the actual position of the maximum due to image content so that later on, an algorithm is used to pin-point the center of symmetry instead of the maximum.
• the values of m and d maXx , d mirix , d maXy , d milly may be evaluated in the discrete domain.
• DRT discrete Radon transform
• the sets of the equations may be solved for /c, x 3 and 3 .
• (x , y , z ) correspond to the coordinates of the camera, or in other words the voxel where the corresponding bundle of light is focused into a circle of the radius A.
• the digital lines may be scanned as before on n(x 1( x 2 ) using the Bresenham digital lines; For each individual (Xi, x 2 ), value, the corresponding (yi, y 2 ) values captured in the light-field are stored. To find such values, expression C is exploited. All the following are either known or estimated from expressions F and G x3; y3; z3; zl; z2 Moving on each line in ⁇ ( ⁇ 1 , ⁇ 2 ), for each (x ⁇ , X 2 ), the following relationship in
• y 2 my 1 + mxj + fc(x 3 + y 3 * )
• a more complex algorithm may be used.
• the parameters (m, k) are found for all the peaks in the radon transform of ⁇ ( ⁇ 1( x 2 ), and put in one set. The same is done for the peaks in ( i, 2) and the parameters are put in another set.
• the maximum peak intensity is found in the 2D radon transform of (xi, X2) and the corresponding peak in ⁇ ji, J2) is found by matching the previously found parameters ( , k).

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Graphics (AREA)
• Architecture (AREA)
• Computer Hardware Design (AREA)
• General Engineering & Computer Science (AREA)
• Software Systems (AREA)
• Studio Devices (AREA)
• Image Processing (AREA)
• Automatic Focus Adjustment (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=55221248

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (5)

Citations (1)

Family Cites Families (9)

• 2015
  • 2015-12-30 EP EP15307174.1A patent/EP3188123A1/en not_active Withdrawn
• 2016
  • 2016-12-29 US US16/067,260 patent/US10762612B2/en active Active
  • 2016-12-29 WO PCT/EP2016/082853 patent/WO2017114904A1/en not_active Ceased
  • 2016-12-29 KR KR1020187018547A patent/KR20180098564A/ko not_active Withdrawn
  • 2016-12-29 CN CN201680081243.XA patent/CN108604372B/zh active Active
  • 2016-12-29 JP JP2018534046A patent/JP2019511851A/ja not_active Ceased
  • 2016-12-29 EP EP16826362.2A patent/EP3398161B1/en active Active

Patent Citations (1)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events