WO2022261105A1 - 2d digital image capture system and simulating 3d digital image and sequence - Google Patents

2d digital image capture system and simulating 3d digital image and sequence Download PDF

Info

Publication number
WO2022261105A1
WO2022261105A1 PCT/US2022/032515 US2022032515W WO2022261105A1 WO 2022261105 A1 WO2022261105 A1 WO 2022261105A1 US 2022032515 W US2022032515 W US 2022032515W WO 2022261105 A1 WO2022261105 A1 WO 2022261105A1
Authority
WO
WIPO (PCT)
Prior art keywords
scene
display
digital image
images
image
Prior art date
Application number
PCT/US2022/032515
Other languages
French (fr)
Inventor
Jerry Nims
William M. Karszes
Samuel Pol
Original Assignee
Jerry Nims
Karszes William M
Samuel Pol
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/355,906 external-priority patent/US20210321077A1/en
Priority claimed from US17/459,067 external-priority patent/US20210392314A1/en
Priority claimed from US17/511,490 external-priority patent/US20220051427A1/en
Priority claimed from US17/525,246 external-priority patent/US20220078392A1/en
Application filed by Jerry Nims, Karszes William M, Samuel Pol filed Critical Jerry Nims
Priority to EP22820900.3A priority Critical patent/EP4352953A1/en
Priority claimed from US17/834,023 external-priority patent/US20220385807A1/en
Publication of WO2022261105A1 publication Critical patent/WO2022261105A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/243Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/128Adjusting depth or disparity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Definitions

  • the present disclosure is directed to 2D image capture, image processing, and simulating display of a 3D or multi-dimensional image or image sequence.
  • HVS human visual system
  • Binocular disparity and motion parallax provide two independent quantitative cues for depth perception. Binocular disparity refers to the difference in position between the two retinal image projections of a point in 3D space.
  • the present disclosure may overcome the above- mentioned disadvantages and may meet the recognized need for 2D image capture system and display of 3D digital image and 3D sequence a smart device having a memory device for storing an instruction, a processor in communication with the memory and configured to execute the instruction, a plurality of digital image capture devices in communication with the processor and each image capture device configured to capture a digital image of the scene, the plurality of digital image capture devices positioned linearly in series within approximately an interpupillary distance, wherein a first digital image capture devices is centered proximate a first end of the interpupillary distance, a second digital image capture devices is centered on a second end of the interpupillary distance, and any remaining the plurality of digital image capture devices are evenly spaced therebetween, processing steps to configure datasets, and a display configured to display a simulated multidimensional digital image sequence and/or a multidimensional digital image, storage of multidimensional digital image sequence and/or a multidimensional digital image via block chain, storage of audio files for playback
  • a feature of the system and methods of use is its ability to capture a plurality of images of a scene with 2D capture devices positioned approximately an intraocular or interpupillary distance width IPD apart (distance between pupils of human visual system).
  • a feature of the system and methods of use is its ability to convert input 2D source images into multi-dimensional/multi-spectral image sequence.
  • the output image follows the rule of a “key subject point” maintained within an optimum parallax to maintain a clear and sharp image.
  • a feature of the system and methods of use is its ability to utilize existing viewing devices to display simulated multidimensional digital image sequence.
  • a feature of the system and methods of use is its ability of taking, viewing, and sending over the internet multidimensional digital image sequence.
  • This self-contained system can be integrated into a smart phone, tablet or used with external devices.
  • a series of 4 camera lens allow us to produce a special motion parallax image, DIGY, that can be viewed without a special screen.
  • the system can be used in a fully automated mode or in manual for operator inter action with the scene.
  • a feature of the system and methods of use is the ability to integrate viewing devices or other viewing functionality into the display, such as barrier screen (black line), lenticular, arced, curved, trapezoid, parabolic, overlays, waveguides, black line and the like with an integrated LCD layer in an LED or OLED, LCD, OLED, and combinations thereof or other viewing devices.
  • viewing devices or other viewing functionality such as barrier screen (black line), lenticular, arced, curved, trapezoid, parabolic, overlays, waveguides, black line and the like with an integrated LCD layer in an LED or OLED, LCD, OLED, and combinations thereof or other viewing devices.
  • Another feature of the digital multi-dimensional image platform based system and methods of use is the ability to produce digital multi-dimensional images that can be viewed on viewing screens, such as mobile and stationary phones, smart phones (including iPhone), tablets, computers, laptops, monitors and other displays and/or special output devices, directly without 3D glasses or a headset.
  • viewing screens such as mobile and stationary phones, smart phones (including iPhone), tablets, computers, laptops, monitors and other displays and/or special output devices, directly without 3D glasses or a headset.
  • a system to simulate a 3D image sequence from a series of 2D images of a scene includes a smart device having a memory device for storing an instruction, a processor in communication with said memory device configured to execute said instruction, a plurality of digital image capture devices in communication with said processor and each of said plurality image capture devices configured to capture a digital image of the scene, said plurality of digital image capture devices approximately positioned linearly in series within approximately an interpupillary distance width, wherein a first digital image capture devices is positioned proximate a first end of said interpupillary distance width, a second digital image capture devices is positioned proximate a second end of said interpupillary distance width, and any remaining said plurality of digital image capture devices are evenly spaced therebetween to capture a series of 2D images of the scene, a display in communication with said processor, said display configured to display said a multidimensional digital image sequence, and overlay an audio file on said multidimensional digital image sequence via an input on said display.
  • a feature of the present disclosure may include a system having a series of capture devices, such as two, three, four or more, such plurality of capture devices (digital image cameras) positioned in series linearly within an intraocular or interpupilary distance width, the distance between an average human’s pupils, the system captures and stores two, three, four or more, a plurality of 2D source images of a scene, the system labels and identifies the images based on the source capture device that captured the image.
  • a series of capture devices such as two, three, four or more, such plurality of capture devices (digital image cameras) positioned in series linearly within an intraocular or interpupilary distance width, the distance between an average human’s pupils
  • the system captures and stores two, three, four or more, a plurality of 2D source images of a scene
  • the system labels and identifies the images based on the source capture device that captured the image.
  • a feature of the present disclosure may include a system having a display device configured from a stack of components, such as top glass cover, capacitive touch screen glass, polarizer, diffusers, and backlight.
  • an image source such as LCD, such LED, ELED, PDP, QLED, and other types of display technologies.
  • display device may include a lens array preferably positioned between capacitive touch screen glass and LCD panel stack of components, and configured to bend or refract light in a manner capable of displaying both a high quality 2D image and an interlaced stereo pair of left and right images as 3D or multidimensional digital image of scene.
  • a feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine the convergence point or key subject point, since the viewing of an image that has not been aligned to a key subject point causes confusion to the human visual system and results in blur and double images.
  • a feature of the present disclosure is the ability to select the convergence point or key subject point anywhere between near or close plane and far or back plane, manual mode user selection.
  • a feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine Circle of Comfort (CoC), since the viewing of an image that has not been aligned to the Circle of Comfort (CoC) causes confusion to the human visual system and results in blur and double images.
  • CoC Circle of Comfort
  • a feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine Circle of Comfort (CoC) fused with Horopter arc or points and Panum area, since the viewing of an image that has not been aligned to the Circle of Comfort (CoC) fused with Horopter arc or points and Panum area causes confusion to the human visual system and results in blur and double images.
  • CoC Circle of Comfort
  • a feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine gray scale depth map, the system interpolates intermediate points based on the assigned points (closest point, key subject point, and furthest point) in a scene, the system assigns values to those intermediate points and renders the sum to a gray scale depth map.
  • the gray scale map to generate volumetric parallax using values assigned to the different points (closest point, key subject point, and furthest point) in a scene. This modality also allows volumetric parallax or rounding to be assigned to singular objects within a scene.
  • a feature of the present disclosure is its ability to measure depth or z-axis of objects or elements of objects and/or make comparisons based on known sizes of objects in a scene.
  • a feature of the present disclosure is its ability to utilize a key subject algorithm to manually or automatically select the key subject in a plurality of images of a scene displayed on a display and produce multidimensional digital image sequence for viewing on a display.
  • a feature of the present disclosure is its ability to utilize an image alignment, horizontal image translation, or edit algorithm to manually or automatically align the plurality of images of a scene about a key subject for display.
  • a feature of the feature of the present disclosure is its ability to utilize an image translation algorithm to align the key subject point of two images of a scene for display.
  • a feature of the feature of the present disclosure is its ability to generate DIFYS (Differential Image Format) is a specific technique for obtaining multi-view of a scene and creating a series of image that creates depth without glasses or any other viewing aides.
  • the system utilizes horizontal image translation along with a form of motion parallax to create 3D viewing.
  • DIFYS are created by having different view of a single scene flipped by the observer’s eyes. The views are captured by motion of the image capture system or by multiple cameras taking a scene with each of the cameras within the array viewing at a different position.
  • a first aspect of the present disclosure of simulating a 3D image sequence from a sequence of 2D image frames may be utilized to capture(ing) a plurality of 2D image frames (images) of a scene from a plurality of different observation points, wherein a first proximal plane and a second distal plane is identified within each image frame in the sequence, and wherein each observation point maintains substantially the same first proximal image plane for each image frame; determining a depth estimate for the first proximal and second distal plane within each image frame in the sequence, aligning the first proximal plane of each image frame in the sequence and shifting the second distal plane of each subsequent image frame in the sequence based on the depth estimate of the second distal plane for each image frame, to produce a modified image frame corresponding to each 2D image frame and displaying the modified image frames sequentially.
  • the present disclosure varies the focus of objects at different planes in a displayed scene to match vergence and stereoscopic retinal disparity demands to better simulate natural viewing conditions.
  • the cues to ocular accommodation and vergence are brought into agreement.
  • the viewer brings different obj ects into focus by shifting accommodation.
  • the mismatch between accommodation and vergence is decreased, natural viewing conditions are better simulated, and eye fatigue is decreased.
  • the present disclosure may be utilized to determine three or more planes for each image frame in the sequence.
  • the planes have different depth estimates.
  • each respective plane is shifted based on the difference between the depth estimate of the respective plane and the first proximal plane.
  • the first, proximal plane of each modified image frame is aligned such that the first proximal plane is positioned at the same pixel space.
  • the first plane comprises a key subject point.
  • the planes comprise at least one foreground plane.
  • the planes comprise at least one background plane.
  • the sequential observation points lie on a straight line.
  • FIG. 1 A illustrates a 2D rendering of an image based upon a change in orientation of an observer relative to a display
  • FIG. IB illustrates a 2D rendering of an image with binocular disparity as a result of the horizontal separation parallax of the left and right eyes;
  • FIG. 2A is an illustration of a cross-section view of the structure of the human eyeball
  • FIG. 2B is a graph relating density of rods and cones to the position of the fovea
  • FIG. 3 is a top view illustration of an observer’s field of view
  • FIG. 4A is a side view illustration identifying planes of a scene captured using a camera or other capture device
  • FIG. 4B A is a front view illustration of an exemplary embodiment of two images of a scene in FIG. 4A captured utilizing capture devices shown in FIGs. 8G;
  • FIG. 5 is a top view illustration identifying planes of a scene and a circle of comfort in scale with FIG. 4;
  • FIG. 6 is a block diagram of a computer system of the present disclosure
  • FIG. 7 is a block diagram of a communications system implemented by the computer system in FIG. 1;
  • FIG. 8A is a diagram of an exemplary embodiment of a computing device with four image capture devices positioned vertically in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
  • FIG. 8B is a diagram of an exemplary embodiment of a computing device with four image capture devices positioned horizontally in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
  • FIG. 8C is an exploded diagram of an exemplary embodiment of the four image capture devices in series linearly of FIGs. 8 A and 8B;
  • FIG. 8D is a cross-sectional diagram of an exemplary embodiment of the four image capture devices in series linearly of FIGs. 8 A and 8B;
  • FIG. 8E is an exploded diagram of an exemplary embodiment of the three image capture devices in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
  • FIG. 8F is a cross-sectional diagram of an exemplary embodiment of the three image capture devices in series linearly of FIG. 8E;
  • FIG. 8G is an exploded diagram of an exemplary embodiment of the two image capture devices in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
  • FIG. 8H is a cross-sectional diagram of an exemplary embodiment of the two image capture devices in series linearly of FIG. 8G;
  • FIG. 9 is a diagram of an exemplary embodiment of human eye spacing the intraocular or interpupillary distance width, the distance between an average human’s pupils;
  • FIG. 10 is a top view illustration identifying planes of a scene and a circle of comfort in scale with right triangles defining positioning of capture devices on lens plane;
  • FIG. 10A is a top view illustration of an exemplary embodiment identifying right triangles to calculate the radius of the Circle of Comfort of FIG. 10;
  • FIG. 10B is a top view illustration of an exemplary embodiment identifying right triangles to calculate linear positioning of capture devices on lens plane of FIG. 10;
  • FIG. IOC is a top view illustration of an exemplary embodiment identifying right triangles to calculate the optimum distance of backplane of FIG. 10;
  • FIG. 11 is a diagram illustration of an exemplary embodiment of a geometrical shift of a point between two images (frames), such as in FIG. 11 A according to select embodiments of the instant disclosure;
  • FIG. 11 A is a front top view illustration of an exemplary embodiment of four images of a scene captured utilizing capture devices shown in FIGs. 8A-8F and aligned about a key subject point;
  • FIG. 1 IB is a front view illustration of an exemplary embodiment of four images of a scene captured utilizing capture devices shown in FIGs. 8A-8F and aligned about a key subject point;
  • FIG. 12 is an exemplary embodiment of a flow diagram of a method of generating a multidimensional image(s)/sequence captured utilizing capture devices shown in FIGs. 8A-8H;
  • FIG. 13 is a top view illustration of an exemplary embodiment of a display with user interactive content to select photography options of computer system
  • FIG. 14A is a top view illustration identifying two frames captured utilizing capture devices shown in FIGs. 8A-8F showing key subject aligned as shown in FIG.1 IB and near plane object offset between two frames;
  • FIG. 14B is a top view illustration of an exemplary embodiment of left and right eye virtual depth via object offset between two frames of FIG. 14 A;
  • Fig. 15A is a cross-section diagram of an exemplary embodiment of a display stack according to select embodiments of the instant disclosure
  • Fig. 15B is a cross-section diagram of an exemplary embodiment of an arced or curved shaped lens according to select embodiments of the instant disclosure, tracing RGB light there through;
  • Fig. 15C is a cross-section diagram of a prototype embodiment of a trapezoid shaped lens according to select embodiments of the instant disclosure, tracing RGB light there through;
  • FIG. 15D is a cross-section diagram of an exemplary embodiment of a dome shaped lens according to select embodiments of the instant disclosure, tracing RGB light there through;
  • FIG. 16A is a diagram illustration of an exemplary embodiment of a pixel interphase processing of images (frames), such as in FIG. 8A according to select embodiments of the instant disclosure;
  • FIG. 16B is a top view illustration of an exemplary embodiment of a display of computer system running an application
  • FIG. 17 is a top view illustration of an exemplary embodiment of viewing a multidimensional digital image on display with the image within the Circle of Comfort, proximate Horopter arc or points, within Panum area, and viewed from viewing distance;
  • FIG. 18 is an exemplary embodiment of a flow diagram of a method of generating a non-fungible token (NFT) utilizing a computing device with image capture devices capture devices shown in FIGs. 8A-8H;
  • NFT non-fungible token
  • FIG. 19 is an exemplary embodiment of a flow diagram of a method of selecting a sequence of DIGYs and adding an audio file thereto utilizing a computing device with image capture devices capture devices shown in FIGs. 8A-8H;
  • FIG. 20 is an exemplary embodiment of a flow diagram of a method of generating a DIGYs and adding an audio file thereto utilizing a computing device with image capture devices capture devices shown in FIGs. 8A-8H.
  • Binocular disparity and motion parallax provide two independent quantitative cues for depth perception. Binocular disparity refers to the difference in position between the two retinal image projections of a point in 3D space. As illustrated in FIGs. 1A and IB, the robust precepts of depth that are obtained when viewing an object 102 in an image scene 110 demonstrates that the brain can compute depth from binocular disparity cues alone.
  • the Horopter 112 is the locus of points in space that have the same disparity as the fixation point 114. Objects lying on a horizontal line passing through the fixation point 114 results in a single image, while objects a reasonable distance from this line result in two images 116, 118.
  • Classical motion parallax is dependent upon two eye functions. One is the tracking of the eye to the motion (eyeball moves to fix motion on a single spot) and the second is smooth motion difference leading to parallax or binocular disparity.
  • Classical motion parallax is when the observer is stationary and the scene around the observer is translating or the opposite where the scene is stationary, and the observer translates across the scene.
  • Motion parallax 120 refers to the relative image motion (between objects at different depths) that results from translation of the observer 104. Isolated from binocular and pictorial depth cues, motion parallax 120 can also provide precise depth perception, provided that it is accompanied by ancillary signals that specify the change in eye orientation relative to the visual scene 110. As illustrated, as eye orientation 104 changes, the apparent relative motion of the object 102 against a background gives hints about its relative distance. If the object 102 is far away, the object 102 appears stationary. If the object 102 is close or near, the object 102 appears to move more quickly.
  • FIGs. 2A and 2B show the anatomy of the eye 200 and a graphical representation of the distribution of rods and cones, respectively.
  • the fovea 202 is responsible for sharp central vision (also referred to as foveal vision), which is necessary where visual detail is of primary importance.
  • the fovea 202 is the depression in the inner retinal surface 205, about 1.5 mm wide and is made up entirely of cones 204 specialized for maximum visual acuity.
  • Rods 206 are low intensity receptors that receive information in grey scale and are important to peripheral vision, while cones 204 are high intensity receptors that receive information in color vision.
  • FIG. 2B shows the distribution of cones 204 and rods 206 in the eye 200.
  • a large proportion of cones 204 providing the highest visual acuity, lie within a 1.5° angle around the center of the fovea 202.
  • FIG. 3 illustrates a typical field of view 300 of the human visual system (HVS).
  • HVS human visual system
  • the fovea 202 sees only the central IS (degrees) of the visual field 302, with the preferred field of view 304 lying within ⁇ 15 (degrees) of the center of the fovea 202.
  • Focusing an object on the fovea therefore, depends on the linear size of the object 102, the viewing angle and the viewing distance.
  • a large object 102 viewed in close proximity will have a large viewing angle falling outside the foveal vision, while a small object 102 viewed at a distance will have a small viewing angle falling within the foveal vision.
  • An object 102 that falls within the foveal vision will be produced in the mind’s eye with high visual acuity.
  • FIG. 4A illustrates a typical view of a scene S to be captured by a camera or digital image capture device, such as image capture module 830.
  • Scene S may include four planes defined as: (1) Lens frame is defined as the plane passing through the lens or sensor (image capture module 830) in the recording device or camera, (2) Key Subject plane KSP may be the plane passing through the focal point of the sensor in the scene (here couple in the scene, the Key Subject KS of the scene S), (3) Near Plane NP may be the plane passing through the closest point in focus to the lens plane (the bush B in the foreground), and (4) Far Plane FP which is the plane passing through the furthest point in focus (tree T in the background).
  • the relative distances from image capture module 830 are denoted by N, Ks, B.
  • Depth of field of the scene S is defined by the distance between Near Plane NP and Far Plane FP.
  • the sense of depth of a stereoscopic image varies depending on the distance between the camera and the key subject, known as the image capturing distance or KS.
  • the sense of depth is also controlled by the vergence angle and the intraocular distance between the capture of each successive image by the camera which effects binocular disparity.
  • Circle of Confusion defines the area of a scene S that is captured in focus.
  • the near plane NP, key subject plane KSP and the far plane FP are in focus. Areas outside this circle are blurred.
  • FIG. 4B illustrates a typical view of a scene S to be captured by a camera or digital image capture device, such as image capture module 830, more specifically image capture module 830 shown in FIG. 8G.
  • Two image capture devices 831 and 832, or any other selected pair 831, 832, 833, 834 may be utilized to capture plurality of digital images of scene S as left image 810L and right image 81 OR of scene S, shown in FIG. 8 A (plurality of digital images).
  • computer system 10 via image manipulation application and display 208 may be configured to enable user U to select or identify two image capture devices of image capture devices 831 (1), 832 (2), 833 (3), or 834 (4) to capture two digital images of scene S as left image 810L and right image 810R of scene S.
  • User U may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 810L and right image 81 OR of scene S, as shown in FIG. 4B
  • FIG. 5 illustrates a Circle of Comfort (CoC) in scale with FIGs. 4.1 and 3.1. Defining the Circle of Comfort (CoC) as the circle formed by passing the diameter of the circle along the perpendicular to Key Subject plane KSP (in scale with FIG. 4) with a width determined by the 30 degree radials of FIG. 3) from the center point on the lens plane, image capture module 830. (R is the radius of Circle of Comfort (CoC).)
  • the object field is the entire image being composed.
  • the “key subject point” is defined as the point where the scene converges, i.e., the point in the depth of field that always remains in focus and has no parallax differential in the key subject point.
  • the foreground and background points are the closest point and furthest point from the viewer, respectively.
  • the depth of field is the depth or distance created within the object field (depicted distance from foreground to background).
  • the principal axis is the line perpendicular to the scene passing through the key subject point.
  • the parallax or binocular disparity is the difference in the position of any point in the first and last image after the key subject alignment.
  • the key subject point displacement from the principal axis between frames is always maintained as a whole integer number of pixels from the principal axis.
  • the total parallax is the summation of the absolute value of the displacement of the key subject point from the principal axis in the closest frame and the absolute value of the displacement of the key subject point from the principal axis in the furthest frame.
  • the technique introduces the Circle of Comfort (CoC) that prescribe the location of the image capture system relative to the scene S.
  • the Circle of Comfort (CoC) relative to the Key Subject KS point of convergence, focal point sets the optimum near plane NP and far plane FP, i.e., controls the parallax of the scene S.
  • the system was developed so any capture device such as iPhone, camera or video camera can be used to capture the scene. Similarly, the captured images can be combined and viewed on any digital output device such as smart phone, tablet, monitor, TV, laptop, or computer screen.
  • any capture device such as iPhone, camera or video camera
  • the captured images can be combined and viewed on any digital output device such as smart phone, tablet, monitor, TV, laptop, or computer screen.
  • the present disclosure may be embodied as a method, data processing system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product on a computer-readable storage medium having computer-readable program code means embodied in the medium. Any suitable computer readable medium may be utilized, including hard disks, ROM, RAM, CD-ROMs, electrical, optical, magnetic storage devices and the like.
  • These computer program instructions or operations may also be stored in a computer- usable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions or operations stored in the computer- usable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks/step or steps.
  • the computer program instructions or operations may also be loaded onto a computer or other programmable data processing apparatus (processor) to cause a series of operational steps to be performed on the computer or other programmable apparatus (processor) to produce a computer implemented process such that the instructions or operations which execute on the computer or other programmable apparatus (processor) provide steps for implementing the functions specified in the flowchart block or blocks/step or steps.
  • blocks or steps of the flowchart illustrations support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It should also be understood that each block or step of the flowchart illustrations, and combinations of blocks or steps in the flowchart illustrations, can be implemented by special purpose hardware-based computer systems, which perform the specified functions or steps, or combinations of special purpose hardware and computer instructions or operations.
  • Computer programming for implementing the present disclosure may be written in various programming languages, database languages, and the like. However, it is understood that other source or object oriented programming languages, and other conventional programming language may be utilized without departing from the spirit and intent of the present disclosure.
  • FIG. 6 there is illustrated a block diagram of a computer system 10 that provides a suitable environment for implementing embodiments of the present disclosure.
  • the computer architecture shown in FIG. 6 is divided into two parts - motherboard 600 and the input/output (I/O) devices 620
  • Motherboard 600 preferably includes subsystems or processor to execute instructions such as central processing unit (CPU) 602, a memory device, such as random access memory (RAM) 604, input/output (I/O) controller 608, and a memory device such as read only memory (ROM) 606, also known as firmware, which are interconnected by bus 10.
  • CPU central processing unit
  • RAM random access memory
  • I/O controller 608 input/output controller
  • ROM read only memory
  • a basic input output system (BIOS) containing the basic routines that help to transfer information between elements within the subsystems of the computer is preferably stored in ROM 606, or operably disposed in RAM 604.
  • Computer system 10 further preferably includes I/O devices 620, such as main storage device 634 for storing operating system 626 and executes an instruction via application program(s) 624, and display 628 for visual output, and other I/O devices 632 as appropriate.
  • Main storage device 634 preferably is connected to CPU 602 through a main storage controller (represented as 608) connected to bus 610.
  • Network adapter 630 allows the computer system to send and receive data through communication devices or any other network adapter capable of transmitting and receiving data over a communications link that is either a wired, optical, or wireless data pathway. It is recognized herein that central processing unit (CPU) 602 performs instructions, operations or commands stored in ROM 606 or RAM 604.
  • computer system 10 may include smart devices, such as smart phone, iPhone, android phone (Google, Samsung, or other manufactures), tablets, desktops, laptops, digital image capture devices, and other computing devices with two or more digital image capture devices and/or 3D display 608 (smart device).
  • smart devices such as smart phone, iPhone, android phone (Google, Samsung, or other manufactures), tablets, desktops, laptops, digital image capture devices, and other computing devices with two or more digital image capture devices and/or 3D display 608 (smart device).
  • display 608 may be configured as a foldable display or multi-foldable display capable of unfolding into a larger display surface area.
  • I/O devices 632 may be connected in a similar manner, including but not limited to, devices such as microphone, speakers, flash drive, CD-ROM player, DVD player, printer, main storage device 634, such as hard drive, and/or modem each connected via an I/O adapter. Also, although preferred, it is not necessary for all of the devices shown in FIG. 6 to be present to practice the present disclosure, as discussed below. Furthermore, the devices and subsystems may be interconnected in different configurations from that shown in FIG. 6, or may be based on optical or gate arrays, or some combination of these elements that is capable of responding to and executing instructions or operations. The operation of a computer system such as that shown in FIG.
  • FIG. 7 there is illustrated a diagram depicting an exemplary communication system 700 in which concepts consistent with the present disclosure may be implemented. Examples of each element within the communication system 700 of FIG. 7 are broadly described above with respect to FIG. 6.
  • the server system 760 and user system 720 have attributes similar to computer system 10 of FIG. 6 and illustrate one possible implementation of computer system 10.
  • Communication system 700 preferably includes one or more user systems 720, 722, 724 (It is contemplated herein that computer system 10 may include smart devices, such as smart phone, iPhone, android phone (Google, Samsung, or other manufactures), tablets, desktops, laptops, cameras, and other computing devices with display 628 (smart device)), one or more server system 760, and network 750, which could be, for example, the Internet, public network, private network or cloud.
  • User systems 720-724 each preferably includes a computer-readable medium, such as random access memory 604, 606, coupled to a processor.
  • the processor, CPU 702 executes program instructions or operations (application software 624) stored in memory 604, 606.
  • Communication system 700 typically includes one or more user system 720.
  • user system 720 may include one or more general-purpose computers (e.g., personal computers), one or more special purpose computers (e.g., devices specifically programmed to communicate with each other and/or the server system 760), a workstation, a server, a device, a digital assistant or a "smart" cellular telephone or pager, a digital camera, a component, other equipment, or some combination of these elements that is capable of responding to and executing instructions or operations.
  • general-purpose computers e.g., personal computers
  • special purpose computers e.g., devices specifically programmed to communicate with each other and/or the server system 760
  • a workstation e.g., a server, a device, a digital assistant or a "smart" cellular telephone or pager, a digital camera, a component, other equipment, or some combination of these elements that is capable of responding to and executing instructions or operations.
  • server system 760 preferably includes a computer-readable medium, such as random access memory 604, 606, coupled to a processor.
  • the processor executes program instructions stored in memory 604, 606.
  • Server system 760 may also include a number of additional external or internal devices, such as, without limitation, a mouse, a CD-ROM, a keyboard, a display, a storage device and other attributes similar to computer system 10 of FIG. 6.
  • Server system 760 may additionally include a secondary storage element, such as database 770 for storage of data and information.
  • Server system 760 although depicted as a single computer system, may be implemented as a network of computer processors.
  • Memory 604, 606 in server system 760 contains one or more executable steps, program(s), algorithm(s), or application(s) 624 (shown in FIG.6).
  • the server system 760 may include a web server, information server, application server, one or more general-purpose computers (e.g., personal computers), one or more special purpose computers (e.g., devices specifically programmed to communicate with each other), a workstation or other equipment, or some combination of these elements that is capable of responding to and executing instructions or operations.
  • Communications system 700 is capable of delivering and exchanging data (including three-dimensional 3D image files) between user systems 720 and a server system 760 through communications link 740 and/or network 750.
  • data including three-dimensional 3D image files
  • users can preferably communicate data over network 750 with each other user system 720, 722, 724, and with other systems and devices, such as server system 760, to electronically transmit, store, print and/or view multidimensional digital master image(s).
  • Communications link 740 typically includes network 750 making a direct or indirect communication between the user system 720 and the server system 760, irrespective of physical separation.
  • Examples of a network 750 include the Internet, cloud, analog or digital wired and wireless networks, radio, television, cable, satellite, and/or any other delivery mechanism for carrying and/or transmitting data or other information, such as to electronically transmit, store, print and/or view multidimensional digital master image(s).
  • the communications link 740 may include, for example, a wired, wireless, cable, optical or satellite communication system or other pathway.
  • the intraocular distance between the capture of successive images or frames of the scene S is fixed to match the average separation of the human left and right eyes in order to maintain constant binocular disparity.
  • the distance to key subject KS is chosen such that the captured image of the key subject is sized to fall within the foveal vision of the observer in order to produce high visual acuity of the key subject and to maintain a vergence angle equal to or less than the preferred viewing angle of fifteen degrees (15).
  • Figures 8A-8F disclose an image or frame capture system for capturing a stereoscopic image (e.g., a 2D frame of a 3D sequence) of scene S, such as FIG. 4.
  • a stereoscopic image e.g., a 2D frame of a 3D sequence
  • the image capture distance, the distance from the image capture system and points or planes in the scene S, such as key subject KS and focal length of camera (i.e., zooming in and out) may be ideally held constant while capturing a stereoscopic image (e.g., a 2D frame of a 3D sequence) of scene S; however, the vergence angle will vary accordingly if the spacing between the capture devices of each successive stereoscopic image is kept constant.
  • Back side 810 may include I/O devices 632, such as an exemplary embodiment of image capture module 830 and may include one or more sensors 840 to measure distance between computer system 10 and selected depths in an image or scene S (depth).
  • Image capture module 830 may include a plurality or four digital image capture devices 831, 832, 833, 834 with four digital image capture devices (positioned vertically, in series linearly within an intraocular or interpupillary distance width IPD (distance between pupils of human visual system within a Circle of Comfort relationship to optimize digital multi-dimensional images for the human visual system) as to back side 810 or proximate and parallel thereto long edge 812.
  • IPD intraocular or interpupillary distance width
  • Interpupillary distance width IPD is preferably the distance between an average human’s pupils may have a distance between approximately two and a half inches, 2.5 inches (6.35 cm), more preferably between approximately 40-80 mm, the vast majority of adults have IPDs in the range 50-75 mm, the wider range of 45-80 mm is likely to include (almost) all adults, and the minimum IPD for children (down to five years old) is around 40 mm).
  • plurality of image capture modules 830 may include one or more sensors 840 may be configured as combinations of image capture device 830 and sensor 840 configured as an integrated unit or module where sensor 840 controls or sets the depth of image capture device 830, whether different depths in scene S, such as foreground, and person P or object, background, such as closest point CP, key subject point KS, and a furthest point FP, shown in FIG. 4.
  • plurality of image capture devices may include first digital image capture device 831 centered proximate first end IPD IPD.l of interpupillary distance width IPD, fourth digital image capture device 834 centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining digital image capture devices second digital image capture device 832 and third digital image capture device 833 evenly spaced therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPD, respectively.
  • smart device or portable smart device with a display may be configured as rectangular or square or other like configurations providing a surface area having first edge 811 and second edge 812.
  • digital image capture devices 831-834 or image capture module 830 may be surrounded by recessed, stepped, or beveled edge 814, each image capture devices 831-34 may be encircled by recessed, stepped, or beveled ring 816, and digital image capture devices 831-34 or image capture module 830 may be covered by lens cover 820 with a lens thereunder lens 818.
  • digital image capture devices 831-34 may be individual capture devices and not part of image capture module.
  • digital image capture devices 831-34 may be positioned anywhere on back side 810 and generally parallel thereto long edge 812.
  • image capture devices may include additional capture devices positioned within an intraocular or interpupillary distance width IPD.
  • digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
  • Back side 810 may include I/O devices 632, such as an exemplary embodiment of image capture module 830 and may include one or more sensors 840 to measure distance between computer system 10 and selected depths in an image or scene S (depth).
  • Image capture module 830 may include a plurality or four digital image capture devices 831, 832, 833, 834 with four digital image capture devices (positioned vertically, in series linearly within an intraocular or interpupillary distance width IPD (distance between pupils of human visual system within a Circle of Comfort relationship to optimize digital multi-dimensional images for the human visual system) as to back side 810 or proximate and parallel thereto short edge 812.
  • IPD intraocular or interpupillary distance width
  • plurality of image capture modules 830 may include one or more sensors 840 may be configured as combinations of image capture device 830 and sensor 840 configured as an integrated unit or module where sensor 840 controls or sets the depth of image capture device 830, such as different depths in scene S, such as foreground, background, and person P or object, such as closest point CP, key subject point KS, and furthest point FP, shown in FIG. 4.
  • plurality of image capture devices may include first digital image capture device 831 centered proximate first end IPD IPD.l of interpupillary distance width IPD, fourth digital image capture device 834 centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining image capture devices second digital image capture device 832 and third digital image capture device 833 evenly spaced therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPD.
  • digital image capture devices 831-34 or image capture module 830 may be surrounded by recessed, stepped, or beveled edge 814, each image capture devices 831-34 may be encircled by recessed, stepped, or beveled ring 816, and image capture devices 831-34 or image capture module 830 may be covered by lens cover 820 with a lens thereunder lens 818.
  • digital image capture devices 831-34 may be individual capture devices and not part of image capture module.
  • digital image capture devices 831-34 may be positioned anywhere on back side 810 and generally parallel thereto long edge 812.
  • digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
  • interpupillary distance width IPD may have a measurement of width to position digital image capture devices 831-334 center-to-center within between approximately maximum width of 115 millimeter to a minimum width of 50 millimeter; more preferably approximately maximum width of 72.5 millimeter to a minimum width of 53.5 millimeter; and most preferably between approximately maximum mean width of 64 millimeter to a minimum mean width of 61.7 millimeter, and an average width of 63 millimeter (2.48 inches) center-to- center width of the human visual system shown in FIG. 9.
  • binocular disparity is a stereognostic perception factor that occurs as a result of the average separation of the left and right eyes by approximately 64 mm.
  • binocular disparity is comparatively large, the observer has the sense that the distance to the key subject is relatively close.
  • the binocular disparity is comparatively small, the observer has the sense that the distance to the key subject KS is relatively far or large.
  • the vergence angle V refers to the angle between the left and right eyes having the key subject as a vertex when the eyes are focused on the key subject KS.
  • the vergence angle increases (as both eyes rotate inward), the distance of the key subject KS is perceived by the observer as being relatively small. As the vergence angle decreases (as both eyes rotate outward), the distance of the key subject KS is perceived by the observer as being relatively large.
  • Image capture module 830 may include digital image capture devices 831-834 with four image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • Digital image capture devices 831-834 may include first digital image capture device 831, second digital image capture device 832, third digital image capture device 833, fourth digital image capture device 834.
  • First digital image capture device 831 may be centered proximate first end IPD IPD.1 of interpupillary distance width IPD
  • fourth digital image capture device 834 may be centered proximate second end IPD.2 of interpupillary distance width IPD
  • remaining digital image capture devices such as second digital image capture device 832 and third digital image capture device 833 may be positioned or evenly spaced therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPD.
  • each digital image capture devices 831-834 or lens 818 may surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
  • digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
  • Image capture module 830 may include digital image capture devices 831-834 with four image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • Digital image capture devices 831-834 may include first digital image capture device 831, second digital image capture device 832, third digital image capture device 833, fourth digital image capture device 834.
  • Each digital image capture devices 831-834 or lens 818 may be surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818. It is contemplated herein that digital image capture devices 831-834 may include optical module, such as lens 818 configured to focus light from scene S on sensor module, such as image capture sensor 822 configured to generate image signals of captured image of scene S, and data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
  • optical module such as lens 818 configured to focus light from scene S on sensor module, such as image capture sensor 822 configured to generate image signals of captured image of scene S
  • data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
  • sensor 840 when sensor 840 is not utilized to calculate different depths in scene S (distance from digital image capture devices 831-834 to foreground, background, and person P or object, such as closest point CP, key subject point KS, and furthest point FP, shown in FIG. 4) then a user may be prompted to capture the scene S images a set distance from digital image capture devices 831-834 to key subject point KS in a scene S, including but not limited to six feet (6 ft.) distance from closest point CP or key subject KS point of a scene S.
  • digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
  • Image capture module 830 may include digital image capture devices 831-833 with a plurality or three image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • Digital image capture devices 831-833 may include first digital image capture device 831, second digital image capture device 832, and third digital image capture device 833.
  • First digital image capture device 831 may be centered proximate first end IPD IPD.l of interpupillary distance width IPD, third digital image capture device 833 may be centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining image capture devices, such as second digital image capture device 832 may be centered on center line CL therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPDE.
  • each digital image capture devices 831-834 or lens 818 may surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
  • digital image capture devices 831-833 may be utilized to capture a series of 2D images of the scene S.
  • Image capture module 830 may include digital image capture devices 831-833 with three image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • Digital image capture devices 831-833 may include first digital image capture device 831, second digital image capture device 832, and third digital image capture device 833.
  • Each digital image capture devices 831-833 or lens 818 may be surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
  • digital image capture devices 831-833 may include optical module, such as lens 818 configured to focus an image of scene S on sensor module, such as image capture sensor 822 configured to generate image signals for the captured image of scene S, and data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
  • optical module such as lens 818 configured to focus an image of scene S on sensor module, such as image capture sensor 822 configured to generate image signals for the captured image of scene S
  • data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
  • Image capture module 830 may include a plurality or two digital image capture devices 831-832 with two image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • Image capture devices 831-832 may include first image capture device 831 and second image capture device 832.
  • First image capture device 831 may be centered proximate first end IPD IPD.l of interpupillary distance width IPD and second image capture device 832 may be centered proximate second end IPD.2 of interpupillary distance width IPD.
  • each image capture devices 831-832 or lens 818 may surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
  • Image capture module 830 may include digital or image capture devices 831-832 with two image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • Image capture devices 831-832 may include first image capture device 831 and second image capture device 832.
  • Each image capture devices 831-832 or lens 818 may be surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
  • image capture devices 831-832 may include optical module, such as lens 818 configured to focus an image of scene S on sensor module, such as image capture sensor 822 configured to generate image signals for the captured image of scene S, and data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
  • optical module such as lens 818 configured to focus an image of scene S on sensor module, such as image capture sensor 822 configured to generate image signals for the captured image of scene S
  • data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
  • image capture module 830 and/or digital image capture devices 831-834 are used to obtain offset 2D digital image views of scene S.
  • image capture module 830 may include a plurality of image capture devices other than the number set forth herein, provided plurality of image capture devices is positioned approximately within intraocular or interpupillary distance width IPD, the distance between an average human’s pupil.
  • image capture module 830 may include a plurality of image capture devices positioned within a linear distance approximately equal to interpupillary distance width IPD.
  • image capture module 830 may include a plurality of image capture devices positioned vertically (computer system 10 or other smart device or portable smart device having short edge 811), horizontally (computer system 10 or other smart device or portable smart device having long edge 812) or otherwise positioned spaced apart in series linearly and within approximately within a linear distance approximately equal to interpupillary distance width IPD.
  • image capture module 830 and digital image capture devices 831-34 positioned linearly within the intraocular or interpupillary distance width IPD enables accurate scene S reproduction therein display 628 to produce a multidimensional digital image on display 628.
  • FIG. 9 by way of example, and not limitation, there is illustrated a front facial view of a human with left eye LE and right eye RE and each having a midpoint of a pupil PI, P2 to illustrate the human eye spacing or the intraocular or interpupillary distance IPD width, the distance between an average human’s visual system pupils.
  • Interpupillary distance (IPD) is the distance measured in millimeters/inches between the centers of the pupils of the eyes.
  • Interpupillary distance width IPD is preferably the distance between an average human’s pupils may have a distance between approximately two and a half inches, 2.5 inches (6.35 cm), more preferably between approximately 40-80 mm, the vast majority of adults have IPDs in the range 50-75 mm, the wider range of 45-80 mm is likely to include (almost) all adults, and the minimum IPD for children (down to five years old) is around 40 mm).
  • left and right images may be produce as set forth in FIGs. 6.1-6.3 from US patent 9,992,473, US patent 10,033,990, and US patent 10,178,247 and electrically communicated to left pixel 550L and right pixel 550R.
  • 2D image may be electrically communicated to center pixel 550C.
  • FIG. 10 there is illustrated by way of example, and not limitation a representative illustration of Circle of Comfort (CoC) in scale with FIG. 4 and 3.
  • the image captured on the lens plane will be comfortable and compatible with human visual system of user U viewing the final image displayed on display 628 if a substantial portion of the image(s) are captured within the Circle of Comfort (CoC).
  • Any object, such as near plane N, key subject plane KSP, and far plane FP captured by two image capture devices, such as image capture devices 831-833 or image capture devices 831-834 (interpupillary distance IPD) within the Circle of Comfort (CoC) will be in focus to the viewer when reproduced as digital multi-dimensional image sequence viewable on display 628.
  • the back-object plane or far plane FP may be defined as the distance to the intersection of the 15 degree radial line to the perpendicular in the field of view to the 30 degree line or R the radius of the Circle of Comfort (CoC).
  • the Circle of Comfort (CoC) as the circle formed by passing the diameter of the circle along the perpendicular to Key Subject KS plane (KSP) with a width determined by the 30 degree radials from the center point on the lens plane, image capture module 830.
  • Linear positioning or spacing of image capture devices such as digital image capture devices 831-833, or digital image capture devices 831-834 (interpupillary distance IPD) on lens plane within the 30 degree line just tangent to the Circle of Comfort (CoC) may be utilized to create motion parallax between the plurality of images when viewing digital multi-dimensional image sequence viewable on display 628, will be comfortable and compatible with human visual system of user U.
  • IPD interpupillary distance
  • CoC Circle of Comfort
  • FIG. 10A, 10B, IOC, and 11 there is illustrated by way of example, and not limitation right triangles derived from FIG. 10. All the definitions are based on holding right triangles within the relationship of the scene to image capture. Thus, knowing the key subject KS distance (convergence point) we can calculate the following parameters.
  • FIG. 6A to calculate the radius R of Comfort (CoC).
  • R KS*tan 30 degree
  • FIG. 6B to calculate the optimum distance between image capture devices, such as image capture devices 831-833, or image capture devices 831-834 (interpupillary distance IPD).
  • TR/KS tan 15 degree
  • TR KS* tan 15 degree; and IPD is 2*TR
  • FIG. 6C calculate the optimum far plane FP
  • Ratio of near plane NP to far plane FP ((KS/ (KS 8 tan 30 degree))*tan 15 degree
  • a user of image capture devices such as digital image capture devices 831-833, or digital image capture devices 831-834 composes the scene S and moves the digital image capture devices 830 in our case so the circle of confusion conveys the scene S.
  • digital image capture devices 830 are using multi cameras linearly spaced there is a binocular disparity between the plurality of images or frames captured by linear offset of digital image capture devices 830, such as digital image capture devices 831-833, or digital image capture devices 831-834.
  • This disparity can be change by changing digital image capture devices 830 settings or moving the key subject KS back or away from digital image capture devices to lessen the disparity or moving the key subject KS closer to digital image capture devices to increase the disparity.
  • Our system is a fixed digital image capture devices system and as a guideline, experimentally developed, the near plane NP should be no closer than approximately six feet from digital image capture devices 830.
  • FIG. 12 there is illustrated process steps as a flow diagram 1200 of a method of capturing plurality of 2D image(s) of scene S, generating frames 1101-1104, manipulating, reconfiguring, processing, displaying, storing a digital multi-dimensional image sequence as performed by a computer system 10, and viewable on display 628.
  • steps designate a manual mode of operation may be performed by a user U, whereby the user is making selections and providing input to computer system 10 in the step whereas otherwise operation of computer system 10 is based on the steps performed by application program(s) 624 in an automatic mode.
  • step 1210 providing computer system 10 having digital image capture devices 830, display 628, and applications 624 as described above in FIGS. 1-11, to enable capture of a plurality of 2-dimensional (2D) images with a disparity due to spacing of digital image capture devices 831-834, digital image capture devices 831-834, or the like within approximately an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil, and displaying 3-dimensional (3D) image sequence on display 628.
  • 2D 2-dimensional
  • computer system 10 via image capture application 624 (method of capture) is configured to capture a plurality digital images of scene S via image capture module 830 having a plurality of image capture devices, such as digital image capture devices 831-834, 830 (n devices)., or the like positioned in series linearly within an intraocular or interpupillary distance width IPD (distance between pupils of human visual system within a Circle of Comfort relationship to optimize digital multi-dimensional images for the human visual system) capture a plurality of 2D digital source images.
  • IPD intraocular or interpupillary distance width
  • I/O devices 632 may include one or more sensors 840 in communication with computer system 10 to measure distance between computer system 10 (image capture devices, such as digital image capture devices 831-834, 830 (n devices)) and selected depths in scene S (depth) such as Key Subject KS and set the focal point of one or more digital image capture devices 831-834, 830 (n devices).
  • image capture devices such as digital image capture devices 831-834, 830 (n devices)
  • depth depths in scene S (depth) such as Key Subject KS and set the focal point of one or more digital image capture devices 831-834, 830 (n devices).
  • user U may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 1102 and right image 1103 of scene S, as shown in FIG. 16.
  • selection box 812 may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 1102 and right image 1103 of scene S, as shown in FIG. 16.
  • block or step 1215 utilizing computer system 10, display 628, and application program(s) 206 (via image capture application) settings to align(ing) or position(ing) an icon, such as cross hair 814, of FIG. 16B, on key subject KS of a scene S displayed thereon display 628, for example by touching or dragging image of scene S or pointing computer system 10 in a different direction to align cross hair 814, of FIG. 16, on key subject KS of a scene S.
  • block or step 1215 using, obtaining or capturing images(n) of scene S) focused on selected depths in an image or scene (depth) of scene S.
  • computer system 10 via image manipulation application 624 and display 628 may be configured to operate in auto mode wherein one or more sensors 840 may measure the distance between computer system 10 (image capture devices, such as, digital image capture devices 831-834, 830 (n devices)) and selected depths in scene S (depth) such as Key Subject KS.
  • image capture devices such as, digital image capture devices 831-834, 830 (n devices)
  • depth such as Key Subject KS.
  • a user may determine the correct distance between computer system 10 and selected depths in scene S (depth) such as Key Subject KS.
  • user U may be instructed on best practices for capturing images(n) of scene S via computer system 10 via image capture application 624 and display 628, such as frame the scene S to include the key subject KS in scene S, selection of the prominent foreground feature of scene S, and furthest point FP in scene S, may include identifying key subject(s) KS in scene S, selection of closest point CP in scene S, the prominent background feature of scene S and the like.
  • position key subject(s) KS in scene S a specified distance from digital image capture devices 831-834 (n devices), 830 (n devices).
  • position closest point CP in scene S a specified distance from digital image capture devices 831-834 (n devices).
  • a first exemplary option may be DIFY capture wherein user U may specify or select digital image(s) speed setting 1302 where user U may increase or decrease play back speed or frames (images) per second of the sequential display of digital image(s) on display 628 captured by capture devices 831-834 (n devices).
  • user U may specify or select digital image(s) number of loops or repeats 1304 to set the number of loops of images(n) of the plurality of 2D image(s) 1000 of scene S captured by capture devices 831-834 (n devices), 830 (n devices) where images(n) of the plurality of 2D image(s) 1000 of scene S captured by capture devices 831-834 (n devices) are displayed in a sequential order on display 628, similar to FIG. 11.
  • user U may specify or select order of playback of digital image(s) sequences for playback or palindrome sequence 1306 to set the order of display of images(n) of the plurality of 2D image(s) 1000 of scene S captured by capture devices 831-834 (n devices), 830 (n devices).
  • the timed sequence showing of the images produces the appropriate binocular disparity through the motion pursuit ratio effect. It is contemplated herein that computer system 10 and application program(s) 624 may utilize default or automatic setting herein.
  • touch screen display 628 enabling user U to select photography options of computer system 10 (3D Stereo).
  • block or step 1215 utilizing computer system 10, display 628, and application program(s) 624 (via image capture application) settings to align(ing) or position(ing) an icon, such as cross hair 1310 on key subject KS of a scene S displayed thereon display 628, for example by touching or dragging image of scene S or pointing computer system 10 in a different direction to align cross hair 1310, on key subject KS of a scene S.
  • User U may tap or other identification interaction with selection box 1312 to select or identify key subject KS in the source images, left image 1102L and right image 1103R of scene S, selected from images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S from image capture devices 831-834 (n devices), or any combination of two images from images(n) 1101, 1102, 1103, 1104 (set of frames 1100) .
  • computer system 10 via image manipulation application and display 624 may be configured to enable user U to select or identify images of scene S as left image 1102 and right image 1103 of scene S.
  • User U may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 1102 and right image 1103 of scene S, as shown in FIG. 16B.
  • user U may utilize computer system 10, display 628, and application program(s) 624 to input plurality of images, files, and dataset (Dataset) of scene S, such as via AirDrop, DROP BOX, or other application.
  • application program(s) 624 to input plurality of images, files, and dataset (Dataset) of scene S, such as via AirDrop, DROP BOX, or other application.
  • step 1215, computer system 10 via image capture application 624, image manipulation application 624, image display application 624 may be performed utilizing distinct and separately located computer systemslO, such as one or more user systems 720 first smart device, 722 second smart device, 724 smart device (smart devices) and application program(s) 624.
  • step 1215 may be performed proximate scene S via computer system 10 (first processor) and application program(s) 624communicating between user systems 720, 722, 724 and application program(s) 624.
  • camera system may be positioned or stationed to capture segments of different viewpoints of an event or entertainment, such as scene S.
  • 724 may capture and transmit a plurality of digital images of scene S as digital multi-dimensional image sequence (DIFY) of scene S sets of images(n) of scene S from capture devices 831-834 (n devices) relative to key subject KS point.
  • DIFY digital multi-dimensional image sequence
  • Images captured at or near interpupillary distance IPD matches the human visual system, which simplifies the math, minimizes cross talk between the two images, reduces fuzziness and image movement to produce digital multi-dimensional image sequence (DIFY) viewable on display 628.
  • DIY digital multi-dimensional image sequence
  • block or step 1215 utilizing computer system 10, display 628, and application program(s) 624 (via dataset capture application) settings to align(ing) or position(ing) an icon, such as cross hair 1310, of FIGs. 13 or 16B, on key subject KS of a scene S displayed thereon display 628, for example by touching or dragging dataset of scene S, or touching and dragging key subject KS, or pointing computer system 10 in a different direction to align cross hair 1310, of FIGs. 13, or 16B, on key subject KS of a scene S.
  • I/O devices 632 may include one or more sensors 852 in communication with computer system 10 to measure distance between computer system 10/ capture device(s) 830 (n devices) and selected depths in scene S (depth) such as Key Subject KS and set the focal point of an arc or trajectory of vehicle 400 and capture device(s) 830. It is contemplated herein that computer system 10, display 628, and application program(s) 624, may operate in auto mode wherein one or more sensors 840 may measure the distance between capture device(s) 830 and selected depths in scene S (depth) such as Key Subject KS.
  • a user may determine the correct distance between user U and selected depths in scene S (depth) such as Key Subject KS.
  • display 628 may utilize one or more sensors 852 to measure distance between capture device(s) 830 (n devices) and selected depths in scene S (depth) such as Key Subject KS and provide on screen instructions or message (distance preference) to instruct user U to move capture device(s) 830 (n devices) closer or father away from Key Subject KS or near plane NP to optimize capture device(s) 830 (n devices) and images, files, and dataset (Dataset) of scene S.
  • computer system 10 via image manipulation application 624 is configured to receive a plurality of images of scene S captured by digital image capture devices 831-834 (n devices), 830 (n devices) through an image acquisition application.
  • the image acquisition application converts each image to a digital source image, such as a JPEG, GIF, TIF format.
  • each digital source image includes a number of visible objects, subjects or points therein, such as foreground or closest point associated with near plane NP, far plane FP or furthest point associated with a far plane FP, and key subject KS.
  • the near plane NP, far plane FP point are the closest point and furthest point from the viewer (plurality of capture devices 831 and 832, 833, or 834, 830 (n devices)), respectively.
  • the depth of field is the depth or distance created within the object field (depicted distance between foreground to background).
  • the principal axis is the line perpendicular to the scene passing through the key subject KS point, while the parallax is the displacement of the key subject KS point from the principal axis, see FIG. 11. In digital composition the displacement is always maintained as a whole integer number of pixels from the principal axis.
  • step 1220, computer system 10 via image capture application 624, image manipulation application 624, image display application 624 may be performed utilizing distinct and separately located computer systemslO, such as one or more user systems 720, 722, 724 and application program(s) 624.
  • step 1220 may be performed remote from scene S via computer system 10 (third processor) and application program(s) 624 communicating between user systems 720, 222, 224 and application program(s) 624.
  • 724 may receive sets of plurality of images(n) of scene S from capture devices 831-834 (n devices) relative to key subject KS point and transmit a manipulated plurality of digital multi-dimensional image sequence (DIFY or 3D stereo) of scene to computer system 10 (first processor) and application program(s) 624, step 1220 A.
  • DIFY digital multi-dimensional image sequence
  • computer system 10 via automatic key subject selection algorithm or key subject application program(s) 624 is configured to identify a key subject KS in each source image, plurality of images of scene S captured by digital image capture devices 831- 834 (n devices). Moreover, computer system 10 via key subject, application program(s) 624 is configured to identify (ing) at least in part a pixel, set of pixels (finger point selection on display 628) in one or more plurality of images(n) of scene S from digital image capture devices 831-834 (n devices) as key subject KS, respectively.
  • computer system 10 via key subject application program(s) 624 is configured to align source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) horizontally about key subject KS; (horizontal image translation (HIT) as shown in 11A and 11B with a distance key Subject KS within a Circle of Comfort relationship to optimize digital multi-dimensional image sequence 1010 for the human visual system.
  • digital image capture devices 831-834 n devices
  • HIT horizontal image translation
  • a key subject point is identified in the series of 2D images of the scene S, and each of the series of 2D images of the scene is aligned to key subject KS point, and all other points in the series of 2D images of the scene shift based on a spacing of the plurality of digital image capture devices to generate a modified sequence of 2D images.
  • Key subject KS may be identified in each plurality of images of scene S captured by digital image capture devices 831-834 (n devices) corresponds to the same key subject KS of scene S as shown in FIG. 11 A, 1 IB, and 4.
  • a computer system 10, display 628, and application program(s) 624 may perform an algorithm or set of steps to automatically identify subject KS therein the plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
  • a computer system 10, (in manual mode- manual key subject selection algorithm), display 628, and application program(s) 624 settings to at least in part enable a user U to align(ing) or edit alignment of a pixel, set of pixels (finger point selection), key subject KS point of at least two images(n) of plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
  • Source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) of scene S are all obtained with digital image capture devices 831-834 (n devices) with the same image capture distance and same focal length.
  • Computer system 10 via key subject application 624 creates a point of certainty, key subject KS point by performing a horizontal image shift of source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices), whereby source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) overlap at this one point, as shown in FIG. 13.
  • This image shift does two things, first it sets the depth of the image. All points in front of key subject KS point are closer to the observer and all points behind key subject KS point are further from the observer.
  • an auto mode computer system 10 via image manipulation application may identify the key subject KS based on a depth map of the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
  • Computer system 10 via image manipulation application may identify a foreground, closest point and background, furthest point using a depth map of the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
  • computer system 10 via image manipulation application and display 628 may be configured to enable user U to select or identify key subject KS in the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) of scene S.
  • User U may tap, move a cursor or box or other identification to select or identify key subject KS in the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) of scene S, as shown in FIG. 13.
  • Horizontal image translation sets the key subject plane KSP as the plane of the screen from which the scene emanates (first or proximal plane). This step also sets the motion of objects, such as bush B in near plane NP (third or near plane) and tree T in far plane FP (second or distal plane) relative to one another. Objects in front of key subject KS or key subject plane KSP move in one direction (left to right or right to left) while objects behind key subject KS or key subject plane KSP move in the opposite direction from objects in the front. Objects behind the key subject plane KSP will have less parallax for a given motion.
  • each layer 1100 includes the primary image element of input file images of scene S, such as image or frame 1101, 1102, 1103 and 1104 from digital image capture devices 831-834 (n devices), respectively.
  • Image acquisition application 624 performs a process to translate image or frame 1101, 1102, 1103 and 1104 image or frame 1101, 1102, 1103 and 1104 is overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT).
  • Parallax line 1107 represents the linear displacement of key subject KS points 1109.1-1109.4 from the principal axis 1112.
  • delta 1120 between the parallax line 1107 represents a linear amount of the parallax 1120, such as front parallax 1120.2 and back parallax 1120.1.
  • computer system 10 via depth map application program(s) 624 is configured to create(ing) depth map of source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) and makes a grey scale image through an algorithm.
  • a depth map is an image or image channel that contains information relating to the distance of objects, surfaces, or points in scene S from a viewpoint, such as digital image capture devices 831-834 (n devices). For example, this provides more information as volume, texture and lighting are more fully defined.
  • the parallax can be tightly controlled. For this computer system 10 may limit the number of output frames to four without going to a depth map.
  • the outer digital image capture devices 831 and 834 are locked into the interpupillary distance (IPD) of the observer or user U viewing display 628.
  • application program(s) 624 may identify key subject KS based on the depth map of the source images.
  • computer system 10 via depth map application program(s) 624 may identify Near Plane NP may be the plane passing through the closest point in focus to the lens plane (the bush B in the foreground), Far Plane FP which is the plane passing through the furthest point in focus (tree T in the background) a foreground, closest point and background, furthest point using a depth map of the source image.
  • Computer system 10 via depth map application program(s) 624 may define two or more planes for each of series of 2D images of the scene and one or more planes may have different depth estimate.
  • Computer system 10 via depth map application program(s) 624 may identify a first proximal plane, such as key subject plane KSP and a second distal plane within the series of 2D images of the scene, such as Near Plane NP or Far Plane FP.
  • computer system 10 via interlay(ing) application program(s) 624 is configured overlay 2D RGB high resolution digital camera thereon model or mesh of images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S with RGB high resolution color (DIFY or 3D stereo).
  • computer system 10 via dataset manipulation application 624 may be utilized to generate a model or mesh of scene S from images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S.
  • computer system 10 via frame establishment program(s) 624 is configured to create or generate frames, recording of images of images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S from a virtual camera shifting, rotation, or arcing position, such as such as 0.5 to 1 degree of separation or movement between frames, such as -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5; for DIFY represented 1101, 1102, 1103, 1104 (set of frames 1100) of 3D color mesh Dataset of scene S to generate parallax; for 3D Stereo as left and right; for example 1102, 1103 images of 3D color mesh Dataset and 1101, 1102, 1103, 1104 for (DIFY).
  • Computer system 10 via key subject, application program(s) 624 may establish increments, of shift for example one (1) degree of total shift between the views (typically 10-70 pixel shift on display 628).
  • computer system 10 via frame establishment program(s) 624 is configured to input or upload source images captured external from computer system 10.
  • utilizing computer system 10 via horizontal and vertical frame DIF translation application 624 may be configured to horizontally and vertically align or transform each source image, plurality of images of scene S captured by digital image capture devices 831- 834 (n devices) requires a dimensional image format (DIF) transform.
  • the DIF transform is a geometric shift that does not change the information acquired at each point in the source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) but can be viewed as a shift of all other points in the source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices), in Cartesian space (illustrated in FIG. 11).
  • the DIF transform is represented by the equation:
  • the geometric shift corresponds to a geometric shift of pixels which contain the plenoptic information
  • the DIF transform then becomes:
  • computer system 10 via horizontal and vertical frame DIF translation application 624 may also apply a geometric shift to the background and or foreground using the DIF transform.
  • the background and foreground may be geometrically shifted according to the depth of each relative to the depth of the key subject KS identified by the depth map 1220B of the source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
  • Controlling the geometrical shift of the background and foreground relative to the key subject KS controls the motion parallax of the key subject KS.
  • the apparent relative motion of the key subject KS against the background or foreground provides the observer with hints about its relative distance.
  • motion parallax is controlled to focus objects at different depths in a displayed scene to match vergence and stereoscopic retinal disparity demands to better simulate natural viewing conditions.
  • stereoscopic retinal disparity an intraocular or interpupillary distance width IPD (distance between pupils of human visual system)
  • multidimensional image sequence 1010 on display 628 requires two different eye actions of user U.
  • the first is the eyes will track the closest item, point, or object (near plane NP) in multidimensional image sequence 1010 on display 628, which will have linear translation back and forth to the stationary key subject plane KSP due to image or frame 1101, 1102, 1103 and 1104 is overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT)).
  • HIT horizontal image translation
  • the eyes will perceive depth due to the smooth motion change of any point or object relative to the key subject plane KSP and more specifically to the key subject KS point.
  • DIFYs are composed of one mechanical step and two eye functions.
  • Linear translation back and forth to the stationary key subject plane KSP due to image or frame 1101, 1102, 1103 and 1104 may be overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT).
  • HIT horizontal image translation
  • Difference in frame position along the key subject plane KSP Smooth Eye Motion
  • Comparison of any two points other than key subject KS also produces depth (binocular disparity).
  • Points behind key subject plane KSP move in opposite direction than those points in front of key subject KS. Comparison of two points in front or back or across key subject KS plane shows depth.
  • computer system 10 via palindrome application 626 is configured to create, generate, or produce multidimensional digital image sequence 1010 aligning sequentially each image of images(n) of scene S from digital image capture devices 831-834 (n devices) in a seamless palindrome loop (align sequentially), such as display in sequence a loop of first digital image, image or frame 1101 from first digital image capture device 831 (1), second digital image, image or frame 1102 from second digital image capture device 832 (2), capture device 832, third digital image, image or frame 1103 from third digital image capture device 833 (3), fourth digital image, image or frame 1104 from fourth digital image capture device 834 (4).
  • an alternate sequence a loop of first digital image, image or frame 1101 from first digital image capture device 831 (1), second digital image, image or frame 1102 from second digital image capture device 832 (2), capture device 832, third digital image, image or frame 1103 from third digital image capture device 833 (3), fourth digital image, image or frame 1104 from fourth digital image capture device 834 (4), fourth digital image, image or frame 1104 from fourth digital image capture device 834 (4), third digital image, image or frame 1103 from third digital image capture device 833 (3), second digital image, image or frame 1102 from second digital image capture device 832 (2), of first digital image, image or frame 1101 from first digital image capture device 831 (1) - 1,2, 3, 4, 4, 3, 2,1 (align sequentially).
  • Preferred sequence is to follow the same sequence or order in which images were captured source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) and an inverted or reverse sequence is added to create a seamless palindrome loop.
  • first proximal plane such as key subject plane KSP of each image of images(n) of scene S from digital image capture devices 831-834 (n devices) and shifting second distal plane, such as such as foreground plane, Near Plane NP, or background plane, Far Plane FP of each subsequent image frame in the sequence based on the depth estimate of the second distal plane for series of 2D images of the scene to produce second modified sequence of 2D images.
  • computer system 10 via interphasing application 626 may be configured to interphase columns of pixels of each set of frames 1100, specifically as left image 1102 and right image 1103 to generate a multidimensional digital image aligned to the key subject KS point and within a calculated parallax range. As shown in FIG.
  • interphasing application 626 may be configured to takes sections, strips, rows, or columns of pixels from left image 1102 and right image 1103, such as column 1602A of the source images, left image 1102, and right image 1103 of terrain T of scene S and layer them alternating between column 1602 A of left image 1102 - LE, and column 1602 A of right image 1103 - RE and reconfigures or lays them out in series side-by-side interlaced, such as in repeating series 160A two columns wide, and repeats this configuration for all layers of the source images, left image 1102 and right image 1103 of terrain T of scene S to generate multidimensional image 1010 with column 1602 A dimensioned to be one pixel 1550 wide.
  • source images, plurality of images of scene S captured by capture device(s) 830 match size and configuration of display 628 aligned to the key subject KS point and within a calculated parallax range.
  • source images, plurality of images of scene S captured by digital image capture devices 831-834 match size and configuration of display 628 aligned to the key subject KS point and within a calculated parallax range.
  • computer system 10 via image editing application 624 is configured to crop, zoom, align, enhance, or perform edits thereto each image(n) of scene S from capture devices 831-834 (n devices) or edit multidimensional digital image sequence 1010.
  • computer system 10 and editing application program(s) 624 may enable user U to perform frame enhancement, layer enrichment, animation, feathering (smooth), (Photoshop or Acorn photo or image tools), to smooth or fill in the images (n) together, or other software techniques for producing 3D effects on display 628. It is contemplated herein that a computer system 10 (auto mode), display 628, and application program(s) 624 may perform an algorithm or set of steps to automatically or enable automatic performance of align(ing) or edit(ing) alignment of a pixel, set of pixels of key subject KS point, crop, zoom, align, enhance, or perform edits of the plurality of images of scene S captured by digital image capture devices 831-834 (n devices) or edit multidimensional digital image sequence 1010.
  • utilizing computer system 10 in manual mode), display 628, and application program(s) 624 settings to at least in part enable a user U to align(ing) or edit(ing) alignment of a pixel, set of pixels of key subject KS point, crop, zoom, align, enhance, or perform edits of the plurality of images of scene S captured by digital image capture devices 831-834 (n devices) or edit multidimensional digital image sequence 1010.
  • user U via display 628 and editing application program(s) 624 may set or chose the speed (time of view) for each frame and the number of view cycles or cycle forever as shown in FIG. 13.
  • Time interval may be assigned to each frame in multidimensional digital image sequence 1010. Additionally, the time interval between frames may be adjusted at step 1240 to provide smooth motion and optimal 3D viewing of multidimensional digital image sequence 1010.
  • a computer system 10, display 628, and application program(s) 624 may perform an algorithm or set of steps to automatically or manually edit or apply effects to set of frames 1100.
  • computer system 10 and editing application program(s) 206 may include edits, such as frame enhancement, layer enrichment, feathering, (Photoshop or Acorn photo or image tools), to smooth or fill in the images (n) together, and other software techniques for producing 3D effects to display 3-D multidimensional image of terrain T of scene S thereon display 628.
  • computer system 10 via image display application 624 is configured to enable images(n) of scene S to display, via sequential palindrome loop, multidimensional digital image sequence 1010 of scene S on display 628 for different dimensions of displays 628.
  • multidimensional digital image sequence 1010 of scene S, resultant 3D image sequence may be output as a DIF sequence to display 628.
  • computer system 10, display 628, and application program(s) 624 may be responsive in that computer system 10 may execute an instruction to size each image (n) of scene S to fit the dimensions of a given display 628.
  • user U may elect to return to block or step 1220 to choose a new key subject KS in each source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) and progress through steps 1220-1250 to view on display 628, via creation of a new or second sequential loop, multidimensional digital image sequence 1010 of scene S for new key subject KS.
  • multidimensional digital image sequence 1010 Now given the multidimensional image sequence 1010, we move to observe the viewing side of the device.
  • computer system 10 via output application 730 (206) may be configured to display multidimensional image(s) 1010 on display 628 for one more user systems 220, 222, 224 via communications link 240 and/or network 250, or 5G computer systems 10 and application program(s) 206.
  • Display 628 may include an array of or plurality of pixels emitting light, such as LCD panel stack of components 1520 having electrodes, such as front electrodes and back electrodes, polarizers, such as horizontal polarizer and vertical polarizer, diffusers, such as gray diffuser, white diffuser, and backlight to emit red R, green G, and blue B light.
  • display 628 may include other standard LCD user U interaction components, such as top glass cover 1510 with capacitive touch screen glass 1512 positioned between top glass cover 1510 andLCD panel stack components 1520.
  • display 628 may include a lens array, such as lenticular lens 1514 preferably positioned between capacitive touch screen glass 1512 and LCD panel stack of components 1520, and configured to bend or refract light in a manner capable of displaying an interlaced stereo pair of left and right images as a 3D or multidimensional digital image(s) 1010 on display 628 and, thereby displaying a multidimensional digital image of scene S on display 628.
  • Transparent adhesives 1530 may be utilized to bond elements in the stack, whether used as a horizontal adhesive or a vertical adhesive to hold multiple elements in the stack.
  • a 1920x1200 pixel image via a plurality of pixels needs to be divided in half, 960x1200, and either half of the plurality of pixels may be utilized for a left image and right image.
  • lens array may include other techniques to bend or refract light, such as barrier screen (black line), lenticular, parabolic, overlays, waveguides, black line and the like capable of separate into a left and right image.
  • barrier screen black line
  • lenticular lenticular
  • parabolic overlays
  • waveguides black line
  • lenticular lens 514 may be orientated in vertical columns when display 628 is held in a landscape view to produce a multidimensional digital image on display 628. However, when display 628 is held in a portrait view the 3D effect is unnoticeable enabling 2D and 3D viewing with the same display 628.
  • smoothing, or other image noise reduction techniques, and foreground subject focus may be used to soften and enhance the 3D view or multidimensional digital image on display 628.
  • FIG. 15B there is illustrated by way of example, and not limitation a representative segment or section of one embodiment of exemplary refractive element, such as lenticular lens 1514 of display 628.
  • exemplary refractive element such as lenticular lens 1514 of display 628.
  • Each sub-element of lenticular lens 1514 being arced or curved or arched segment or section 1540 (shaped as an arc ) of lenticular lens 1514 may be configured having a repeating series of trapezoidal lens segments or plurality of sub-elements or refractive elements.
  • each arced or curved or arched segment 1540 may be configured having lens peak 1541 of lenticular lens 1540 and dimensioned to be one pixel 1550 (emitting red R, green G, and blue B light) wide such as having assigned center pixel 1550C thereto lens peak 1541. It is contemplated herein that center pixel 1550C light passes through lenticular lens 1540 as center light 1560C to provide 2D viewing of image on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 or trapezoidal segment or section 1540 of lenticular lens 1514.
  • each arced or curved segment 1540 may be configured having angled sections, such as lens angle A1 of lens refractive element, such as lens sub-element 1542 (plurality of sub elements) of lenticular lens 1540 and dimensioned to be one pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens, left lens sub-element 1542L having angle Al, and right lens sub-element 1542R having angle Al, for example an incline angle and a decline angle respectively to refract light across center line CL.
  • lens angle A1 of lens refractive element such as lens sub-element 1542 (plurality of sub elements) of lenticular lens 1540 and dimensioned to be one pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens, left lens sub-element 1542L having angle Al, and right lens sub-element 1542R having angle Al, for example an incline angle and a decline angle respectively to re
  • pixel 1550L/R light passes through lenticular lens 1540 and bends or refracts to provide left and right images to enable 3D viewing of image on display 628; via left pixel 1550L light passes through left lens angle 1542L and bends or refracts, such as light entering left lens angle 1542L bends or refracts to cross center line CL to the right R side, left image light 1560L toward left eye LE and right pixel 1550R light passes through right lens angle 1542R and bends or refracts, such as light entering right lens angle 1542R bends or refracts to cross center line CL to the left side L, right image light 1560R toward right eye RE, to produce a multidimensional digital image on display 628.
  • left and right images may be produce as set forth in FIGs. 6.1-6.3 from US patent 9,992,473, US patent 10,033,990, and US patent 10,178,247 and electrically communicated to left pixel 550L and right pixel 550R.
  • 2D image may be electrically communicated to center pixel 550C.
  • each lens peak 1541 has a corresponding left and right angled lens 1542, such as left angled lens 1542L and right angled lens 1542R on either side of lens peak 1541 and each assigned one pixel, center pixel 1550C, left pixel 1550L and right pixel 1550R, assigned respectively thereto.
  • each pixel may be configured from a set of sub-pixels.
  • each pixel may be configured as one or two 3x3 sub-pixels of LCD panel stack components 1520 emitting one or two red R light, one or two green G light, and one or two blue B light therethrough segments or sections of lenticular lens 1540 to produce a multidimensional digital image on display 628.
  • Red R light, green G light, and blue B may be configured as vertical stacks of three horizontal sub-pixels.
  • trapezoid shaped lens 1540 bends or refracts light uniformly through its center C, left L side, and right R side, such as left angled lens 1542L and right angled lens 1542R, and lens peak 1541.
  • each segment or plurality of sub-elements or refractive elements being trapezoidal shaped segment or section 1540 of lenticular lens 1514 may be configured having a repeating series of trapezoidal lens segments.
  • each trapezoidal segment 1540 may be configured having lens peak 1541 of lenticular lens 1540 and dimensioned to be one or two pixel 1550 wide and flat or straight lens, such as lens valley 1543 and dimensioned to be one or two pixel 1550 wide (emitting red R, green G, and blue B light).
  • lens valley 1543 may be assigned center pixel 1550C. It is contemplated herein that center pixel 1550C light passes through lenticular lens 1540 as center light 1560C to provide 2D viewing of image on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 or trapezoidal segment or section 1540 of lenticular lens 1514.
  • each trapezoidal segment 1540 may be configured having angled sections, such as lens angle 1542 of lenticular lens 1540 and dimensioned to be one or two pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens angle 1542L and right lens angle 1542R, respectively.
  • pixel 1550L/R light passes through lenticular lens 1540 and bends to provide left and right images to enable 3D viewing of image on display 628; via left pixel 1550L light passes through left lens angle 1542L and bends or refracts, such as light entering left lens angle 1542L bends or refracts to cross center line CL to the right R side, left image light 1560L toward left eye LE; and right pixel 1550R light passes through right lens angle 1542R and bends or refracts, such as light entering right lens angle 1542R bends or refracts to cross center line CL to the left side L, right image light 1560R toward right eye RE to produce a multidimensional digital image on display 628.
  • angle A1 of lens angle 1542 is a function of the pixel 1550 size, stack up of components of display 628, refractive properties of lenticular lens 514, and distance left eye LE and right eye RE are from pixel 1550, viewing distance VD.
  • FIG. 15D there is illustrated by way of example, and not limitation a representative segment or section of one embodiment of exemplary lenticular lens 1514 of display 628.
  • Each segment or plurality of sub-elements or refractive elements being parabolic or dome shaped segment or section 1540A (parabolic lens or dome lens, shaped a dome) of lenticular lens 1514 may be configured having a repeating series of dome shaped, curved, semi-circular lens segments.
  • each dome segment 1540 A may be configured having lens peak 1541 of lenticular lens 1540 and dimensioned to be one or two pixel 1550 wide (emitting red R, green G, and blue B light) such as having assigned center pixel 1550C thereto lens peak 1541. It is contemplated herein that center pixel 1550C light passes through lenticular lens 540 as center light 560C to provide 2D viewing of image on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 or trapezoidal segment or section 1540 of lenticular lens 1514.
  • each trapezoidal segment 1540 may be configured having angled sections, such as lens angle 1542 of lenticular lens 1540 and dimensioned to be one pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens angle 1542L and right lens angle 1542R, respectively.
  • pixel 1550L/R light passes through lenticular lens 1540 and bends to provide left and right images to enable 3D viewing of image on display 628; via left pixel 1550L light passes through left lens angle 1542L and bends or refracts, such as light entering left lens angle 1542L bends or refracts to cross center line CL to the right R side, left image light 1560L toward left eye LE and right pixel 1550R light passes through right lens angle 1542R and bends or refracts, such as light entering right lens angle 1542R bends or refracts to cross center line CL to the left side L, right image light 1560R toward right eye RE to produce a multidimensional digital image on display 628.
  • dome shaped lens 1540B bends or refracts light almost uniformly through its center C, left L side, and right R side.
  • exemplary lenticular lens 1514 may be configured in a variety of other shapes and dimensions.
  • a digital form of alternating black line or parallax barrier may be utilized during multidimensional digital image viewing on display 628 without the addition of lenticular lens 1514 to the stack of display 628 and then digital form of digital form of alternating black line or parallax barrier (alternating) may be disabled during two dimensional (2D) image viewing on display 628.
  • a parallax barrier is a device placed in front of an image source, such as a liquid crystal display, to allow it to show a stereoscopic or multiscopic image without the need for the viewer to wear 3D glasses. Placed in front of the normal LCD, it consists of an opaque layer with a series of precisely spaced slits, allowing each eye to see a different set of pixels, so creating a sense of depth through parallax.
  • a digital parallax barrier is a series of alternating black lines in front of an image source, such as a liquid crystal display (pixels), to allow it to show a stereoscopic or multiscopic image.
  • face-tracking software functionality may be utilized to adjust the relative positions of the pixels and barrier slits according to the location of the user's eyes, allowing the user to experience the 3D from a wide range of positions.
  • parallax and key subject KS reference point calculations may be formulated for distance between virtual camera positions, interphasing spacing, display 628 distance from user U, lenticular lens 1514 configuration (lens angle Al, 1542, lens per millimeter and millimeter depth of the array), lens angle 1542 as a function of the stack up of components of display 628, refractive properties of lenticular lens 1514, and distance left eye LE and right eye RE are from pixel 1550, viewing distance VD, distance between virtual camera positions (interpupillary distance IPD), and the like to produce digital multi-dimensional images as related to the viewing devices or other viewing functionality, such as barrier screen (black line), lenticular, parabolic, overlays, waveguides, black line and the like with an integrated LCD layer in an LED or OLED, LCD, OLED, and combinations thereof or other viewing devices.
  • barrier screen black line
  • lenticular, parabolic overlays, waveguides, black line and the like with an integrated LCD layer in an LED or OLED, LCD, O
  • number of lenses per mm or inch of lenticular lens 514 is determined by the pixels per inch of display 628.
  • angles Al are contemplated herein, distance of pixels 1550C, 1550L, 1550R from of lens 1540 (approximately 0.5 mm), and user U viewing distance from smart device display 628 from user’s eyes (approximately fifteen (15) inches), and average human interpupilary spacing between eyes (approximately 2.5 inches) may be factored or calculated to produce digital multi-dimensional images. Governing rules of angles and spacing assure the viewed images thereon display 628 is within the comfort zone of the viewing device to produce digital multi-dimensional images, see FIGs. 5, 6, 11 below.
  • angle Al of lens 1541 may be calculated and set based on viewing distance VD between user U eyes, left eye LE and right eye RE, and pixels550, such as pixels 1550C, 1550L, 1550R, a comfortable distance to hold display 628 from user’s U eyes, such as ten (10) inches to arm/wrist length, or more preferably between approximately fifteen (15) inches to twenty -four (24) inches, and most preferably at approximately fifteen (15) inches.
  • the user U moves the display 628 toward and away from user’s eyes until the digital multi-dimensional images appear to user, this movement factor in user’s U actual interpupilary distance IPD spacing and to match user’s visual system (near sited and far sited discrepancies) as a function of width position of interlaced left and right images from distance between virtual camera positions (interpupilary distance IPD), key subject KS depth therein each of digital images(n) of scene S (key subject KS algorithm), horizontal image translation algorithm of two images (left and right image) about key subject KS, interphasing algorithm of two images (left and right image) about key subject KS, angles Al, distance of pixels 1550 from of lens 1540 (pixel-lens distance (PLD) approximately 0.5 mm)) and refractive properties of lens array, such as trapezoid shaped lens 1540 all factored in to produce digital multi-dimensional images for user U viewing display 628.
  • IPD interpupilary distance
  • PLD pixel-lens distance
  • First known elements are number of pixels 1550 and number of images, two image, distance between virtual camera positions, or (interpupilary distance IPD). Images captured at or near interpupilary distance IPD matches the human visual system, simplifies the math, minimizes cross talk between the two images, fuzziness, image movement to produce digital multi-dimensional image viewable on display 628.
  • trapezoid shaped lens 1540 may be formed from polystyrene, polycarbonate or other transparent materials or similar materials, as these material offers a variety of forms and shapes, may be manufactured into different shapes and sizes, and provide strength with reduced weight; however, other suitable materials or the like, can be utilized, provided such material has transparency and is machineable or formable as would meet the purpose described herein to produce a left and right stereo image and specified index of refraction. It is further contemplated herein that trapezoid shaped lens 1541 may be configured with 4.5 lenticular lens per millimeter and approximately 0.33 mm depth.
  • DIFY in block or step 1250, computer system 10 via image display application 624 is configured to set of frames 1100 of terrain T of scene S to display, via sequential palindrome loop, multidimensional digital image sequence 1010 on display 628 for different dimensions of displays 628.
  • multidimensional digital image sequence 1010 of scene S, resultant 3D image sequence may be output as a DIF sequence or .MPO file to display 628.
  • computer system 10, display 628, and application program(s) 624 may be responsive in that computer system 10 may execute an instruction to size each image (n) of scene S to fit the dimensions of a given display 628.
  • multidimensional image sequence 1010 on display 628 utilizes a difference in position of objects in each of images(n) of scene S from set of frames 1100 relative to key subject plane KSP, which introduces a parallax disparity between images in the sequence to display multidimensional image sequence 1010 on display 628 to enable user U, in block or step 1250 to view multidimensional image sequence 1010 on display 628.
  • computer system 10 via output application 624 may be configured to display multidimensional image sequence 1010 on display 628 for one more user system 720, 722, 724 via communications link 740 and/or network 750, or 5G computer systems 10 and application program(s) 624.
  • 3D Stereo in block or step 1250, computer system 10 via output application 624 may be configured to display multidimensional image 1010 on display 628.
  • Multidimensional image 1010 may be displayed via left and right pixel 1102L/1103R light passes through lenticular lens 1540 and bends or refracts to provide 3D viewing of multidimensional image 1010 on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550.
  • user U may elect to return to block or step 1220 to choose a new key subject KS in each source image, set of frames 1100 of terrain T of scene S and progress through steps 1220-1250 to view on display 628, via creation of a new or second sequential loop, multidimensional digital image sequence 1010 of scene S for new key subject KS.
  • Display 628 may include display device (e.g., viewing screen whether implemented on a smart phone, PDA, monitor, TV, tablet or other viewing device, capable of projecting information in a pixel format) or printer (e.g., consumer printer, store kiosk, special printer or other hard copy device) to print multidimensional digital master image on, for example, lenticular or other physical viewing material.
  • display device e.g., viewing screen whether implemented on a smart phone, PDA, monitor, TV, tablet or other viewing device, capable of projecting information in a pixel format
  • printer e.g., consumer printer, store kiosk, special printer or other hard copy device
  • steps 1220-1240 may be performed by computer system 10 via image manipulation application 626 utilizing distinct and separately located computer systems 10, such as one or more user systems 720, 722, 724 and application program(s) 626 performing steps herein.
  • steps 1220-1240 may be performed remote from scene S via computer system 10 or server 760 and application program(s) 624 and communicating between user systems 720, 722, 724 and application program(s) 626 via communications link 740 and/or network 750, or via wireless network, such as 5G, computer systems 10 and application program(s) 626 via more user systems 720, 722, 724.
  • computer system 10 via image manipulation application 624 may manipulate 24 settings to configure each images(n) (L&R segments) of scene S from of scene S from virtual camera to generate multidimensional digital image sequence 1010 aligned to the key subject KS point and transmit for display multidimensional digital image/sequence 1010 to one or more user systems 720, 722, 724 via communications link 740 and/or network 750, or via wireless network, such as 5G computer systems 10 or server 760 and application program(s) 624.
  • steps 1220-1240 may be performed by computer system 10 via image manipulation application 624 utilizing distinct and separately located computer systemslO positioned on the vehicle.
  • steps 1220-1240 via computer system 10 and application program(s) 624 computer systemslO may manipulate 24 settings to configure each images(n) (L&R segments) of scene S from of scene S from capture device(s) 830 to generate a multidimensional digital image/sequence 1010 aligned to the key subject KS point.
  • computer system 10 via image manipulation application 626 may utilize multidimensional image/sequence 1010 to navigate the vehicle V through terrain T of scene S.
  • computer system 10 via image manipulation application 626 may enable user U remote from vehicle V to utilize multidimensional image/sequence 1010 to navigate the vehicle V through terrain T of scene S.
  • computer system 10 via output application 624 may be configured to enable display of multidimensional image sequence 1010 on display 628 to enable a plurality of user U, in block or step 1250 to view multidimensional image sequence 1010 on display 628 live or as a replay/rebroadcast.
  • step 1250 may be performed by computer system 10 via output application 624 utilizing distinct and separately located computer systems 10, such as one or more user systems 720, 722, 724 and application program(s) 624 performing steps herein.
  • computer system 10 output application 624 may receive manipulated plurality of two digital images of scene S and display multidimensional image/sequence 1010 to one more user systems 720, 722, 724 via communications link 740 and/or network 750, or via wireless network, such as 5G computer systems 10 and application program(s) 624.
  • wireless such as 5G second computer system 10 and application program(s) 624 may transmit sets of images(n) of scene S configured relative to key subject plane KSP as multidimensional image sequence 1010 on display 628 to enable a plurality of user U, in block or step 1250 to view multidimensional image/sequence 1010 on display 628 live or as a replay/rebroadcast.
  • a first exemplary option may be DIFY capture wherein user U may specify or select digital image(s) speed setting 1302 where user U may increase or decrease play back speed or frames (images) per second of the sequential display of digital image(s) on display 628 multidimensional image/sequence 1010.
  • user U may specify or select digital image(s) number of loops or repeats 1304 to set the number of loops of images(n) of the plurality of 2D image(s) 1000 of scene S where images(n) of the plurality of 2D image(s) 1000 of scene S are displayed in a sequential order on display 628, similar to FIG. 11. Still furthermore, user U may specify or select order of playback of digital image(s) sequences for playback or palindrome sequence 1306 to set the order of display of images(n) of the multidimensional image/sequence 1010 of scene S. The timed sequence showing of the images produces the appropriate binocular disparity through the motion pursuit ratio effect.
  • DIFY referring to FIGs. 14A and 14B, there is illustrated by way of example, and not limitation, frames captured in a set sequence which are played back to the eye in a set sequence and a representation of what the human eyes perceives viewing the DIFY on display 628. Explanation of DIFY and its geometry to produce motion parallax. Motion parallax is the change in angle of a point relative to a stationary point. (Motion Pursuit). Note because we have set the key subject KS point all points in foreground will move to the right, while all points in the background will move to the left. The motion is reversed in a paledrone where the images reverse direction. The angular change of any point in different views relative to the key subject creates motion parallax.
  • a DIFY is a series of frames captured in a set sequence which are played back to the eye in the set sequence as a loop.
  • the play back of two frames is depicted in FIG. 14A.
  • FIG. 14A represents the position of an object, such as a near plane NP object in FIG. 4 on the near plane NP and its relation to key subject KS point in frame 1101 and 1104 wherein key subject KS point is constant due to the image translation imposed on the frames, frame 1101, 1102, 1103 and 1104.
  • FIG. 14B there is illustrated by way of example, and not limitation what the human eye perceives from the viewing of the two frames (assume first and last frame, such as frame 1101 and 1104 having frame in near plane NP as point 1401 and frame 2 in near plane NP as point 1402) depicted in FIG. 14A on display 628 where image plane or screen plane is the same as key subject KS point and key subject plane KSP and user U viewing display 628 views virtual depth near plane NP 1410 in front of display 628 or between display 628 and user U eyes, left eye LE and right eye RE.
  • HIT horizontal image translation
  • Virtual depth near plane NP 1410 is near plane NP as it represents frame 1 in near plane NP as object in near plane point 1401 and frame 2 in near plane NP as object in near plane pointl402, the closest points user U eyes, left eye LE and right eye RE see when viewing multidimensional image sequence 1010 on display 628.
  • Virtual depth near plane NP 1410 simulates a visual depth between key subject KS and object in near plane point 1401 and object in near plane point 1402 as virtual depth 1420, depth between the near plane NP and key subject plane KSP. This depth is due to binocular disparity between the two views for the same point, object in near plane point 1401 and object in near plane point 1402. Object in near plane point 1401 and object in near plane point 1402 are preferably same point in scene S, at different views sequenced in time due to binocular disparity. Moreover, outer rays 1430 and more specifically user U eyes, left eye LE and right eye RE viewing angle 1440 is preferably approximately twenty-seven (27) degrees from the retinal or eye axis.
  • Near plane point 1401 and near plane point 1402 preferably lie within the depth of field, outer rays 1430, and near plane NP has to be outside the inner cross over position 1450 of outer rays 1430.
  • the motion from XI to X2 is the motion user U eyes, left eye LE and right eye RE will track.
  • Xn is distance from eye lens, left eye LE or right eye RE to image point 1411, 1412 on virtual near image plane 1410.
  • X’n is distance of leg formed from right triangle of Xn to from eye lens, left eye LE or right eye RE to image point 1411, 1412 on virtual near image plane 1410 to the image plane, 628, KS, KSP.
  • the smooth motion is the binocular disparity caused by the offset relative to key subject KS at each of the points user U eyes, left eye LE and right eye RE observe.
  • a coordinate system may be developed relative to the center of the eye CL and to the center of the intraocular spacing, half of interpupillary distance width IPD, 1440.
  • Two angles b and a are the angles utilized to explain the DIFY motion pursuit b is the angle formed when a line is passed from the eye lens, left eye LE and right eye RE, through the virtual near plane 1410 to the image on the image plane, 628, KS, KSP.
  • Q is b2- b ⁇ . While a is the angle from the fixed key subject KS of the two frames 1101, 1104 on the image plane 628, KS, KSP to the point 1411, 1412 on virtual near image plane 1410. The change in a represents the eye pursuit.
  • FIG. 17 there is illustrated by way of example, and not limitation a representative illustration of Circle of Comfort CoC fused with Horopter arc or points and Panum area.
  • Horopter is the locus of points in space that have the same disparity as fixation, Horopter arc or points. Objects in the scene that fall proximate Horopter arc or points are sharp images and those outside (in front of or behind) Horopter arc or points are fuzzy or blurry.
  • Panum is an area of space, Panum area 1720, surrounding the Horopter for a given degree of ocular convergence with inner limit 1721 and an outer limit 1722, within which different points projected on to the left and right eyes LE/RE result in binocular fusion, producing a sensation of visual depth, and points lying outside the area result in diplopia - double images. Moreover, fuse the images from the left and right eyes for objects that fall inside Panum’ s area, including proximate the Horopter, and user U will we see single clear images. Outside Panum’ s area, either in front or behind, user U will see double images.
  • computer system 10 via image capture application 624, image manipulation application 624, image display application624 may be performed utilizing distinct and separately located computer systems 10, such as one or more user systems 220, 222, 224 and application program(s) 206.
  • wireless such as 5G second computer system 10 and application program(s) 206 may transmit sets of images(n) of scene S relative to key subject plane introduces a (left and right) binocular disparity to display a multidimensional digital image on display 628 to enable a plurality of user U, in block or step 1250 to view multidimensional digital image on display 628 live or as a replay/rebroadcast.
  • FIG. 17 illustrates display and viewing of multidimensional image 1010 on display 628 via left and right pixel 1550L/R light of multidimensional image 1010 passes through lenticular lens 1540 and bends or refracts to provide 3D viewing of multidimensional image 1010 on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 with near object, key subject KS, and far object within the Circle of Comfort CoC and Circle of Comfort CoC is proximate Horopter arc or points and within Panum area 1720 to enable sharp single image 3D viewing of multidimensional image 1010 on display 628comfortable and compatible with human visual system of user U.
  • Blockchain is a shared, ledger that facilitates the process of recording transactions and tracking assets in a network.
  • An asset can be tangible (title to a house, car, cash, land, artwork or DIGY, 3D Stereo, Datasets thereof) or intangible (intellectual property, patents, copyrights, branding).
  • a blockchain is a decentralized, distributed, and oftentimes public, digital ledger consisting of records called blocks that are used to record transactions across many computers using cryptography so that any involved block cannot be altered retroactively, without the alteration of all subsequent blocks. This allows the participants to verify and audit transactions independently. Each block contains data or Dataset, a cryptographic hash of the previous block (chain), a timestamp, and cryptographic hash (identifies block and all its content).
  • Ethereum is a popular block chain standard as well as other private and open source blockchain algorithms.
  • DIGY, 3D Stereo, Datasets, and other datasets and any authentication documents verify creator or author, authenticity statements, history, date of origin, chain of title, chain of owners, or like (authentication) may be included in the dataset (Dataset).
  • Dataset may be stored as a block on the blockchain network and each subsequent transaction thereafter related to Dataset may be set forth in subsequent blocks each contains a cryptographic hash of the previous block, a timestamp of the Dataset transaction data decentralized source of trust.
  • Non-fungible tokens For select DIGY, DIGY sequences, or Stereo 3D image files, Datasets or other datasets- non-fungible tokens (NFT)s are unique collectible crypto assets stored on blockchain with unique identification codes and metadata that distinguish DIGY, DIGY sequences, or Stereo 3D image file, Dataset, or dataset from any other digital content making it unique one of a kind or limited edition digital content. Enabling the creation of unique digital content and/or set numbers of copies- a certificate of authenticity for a one-of-a-kind or limited number of digital memorabilia that can't be duplicated. The memorabilia is stored on a blockchain network.
  • An NFT is simply a record of who owns a unique piece of digital content. That content can be anything from art, music, photograph, 3D graphics, tweets, memes, games, videos, GIFs - you name it. If it's digital and it was created, it can be an NFT.
  • NFT digital content can be captured by a third party device and mass distributed via Internet by piracy and devalue the NFT digital content of standard NFT content.
  • Digital content herein are DIGYs and 3D Stereo images generated by artist users and creating their own NFT artwork based on DIGYs and 3D Stereo images using our DIGY and 3D Stereo platforms above.
  • FIG. 18 there is illustrated a flow diagram 1800 of a method of creating an NFT for DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset).
  • NFT for selected image file - DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset).
  • User of Smart Device 10 via a system or App will utilize one of the blockchain providers to issue an NFTs for their DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset).
  • Ethereum is currently the leading blockchain service for NFT issuance. However, there is a range of other blockchains that are becoming increasingly popular. One may be offered by Smart Device manufacture or other service provider.
  • NFT token standard, compatible wallet services and marketplaces to offer for sale or exchange NFT of DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset).
  • Dataset data file(s)
  • Once created user can offer their NFT, such as NFT- DIGY, DIGY sequences, or Stereo 3D image file for sale or verification on NFT compatible wallet services and marketplace, such as OpenSea which is an Ethereum-basedNFT marketplace. Similar marketplace may be offered by Smart Device manufacture or other service provider.
  • DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) are unique in that they can’t be captured by a third party device such as another smart device viewing the DIGY, DIGY sequences, or Stereo 3D since such devices do not have access to the original digital files and may not have a license to the DIGY and 3D Stereo platforms.
  • Current NFT digital content can be captured by a third party device and mass distributed via Internet by piracy and devalue the NFT digital content of standard NFT content.
  • DIGY herein may include 2D video, 2D image collage, 3D DIGY.
  • a sequence is a plurality of DIGYS put together in series or a loop in sequence to create a story.
  • FIG. 19 there is illustrated a flow diagram 1900 of a method of linking or looping multiple DIGY - MP4 files in sequence or as a loop with or without audio files such as AAC (m4a) format, AIFF, Apple Lossless, MP3, and WAV or other like audio formats stored there with.
  • AAC m4a
  • AIFF Apple Lossless
  • MP3 Apple Lossless
  • WAV Wireless Fidelity
  • reordering DIGY file(s) by tapping/touching/selecting/dragging by dragging DIGY file(s) to a different order or position is sequence.
  • sharing DIGY sequence - MP4 file may be attached to an email or text, air dropped, or uploaded to social media to share.
  • a user of may use transient detection to find the downbeat at the beginning of the measure and then synchronized this up with the start of the first DIGY.
  • Each DIGY is trimmed to 2 seconds then sequenced together to match the music transitions of the sequenced DIGYs.
  • FIG. 20 there is illustrated a flow diagram 2000 of a method of manipulating a DIGY or DIGY sequence image file to synchronize with an audio file.
  • Block or step 2020 tapping/touching/selecting DIGY, DIGY sequence, Stereo 3D image file (Dataset) via input to display 628 of Smart Device 10 to select a Dataset file from memory device 604, 606 via an input from said display 628.
  • block or step 2025 executing an instruction 206 via processor 102 to prepare, convert DIGY.gif file to DIGY.mps file via an input from said display 628; to create a DIGY frame time line via an input from display 628; to import image frames (3D DIGY, 2D Images, 2D video) into said frame time line via an input from said display206; to adjust sequence/frame dwell time, via an input from said display 206.
  • executing an instruction 206 via processor 102 to record an audio files via microphone in Smart Device 10 select an audio file from memory device 604, 606 or online service, such as ITUNES via an input from said display 628 and import or download audio file to time line via an input from said display 628. It is contemplated herein that DIGY sequence and audio file may be converted to .mp4 file for sharing with other Smart Devices 222 via network 250.
  • step 2045 saving completed image and audio file - MP4 by tapping/touching/selecting Save to save completed image and audio MP4 file in main storage device 214 via an input from display 628.
  • step 2050 play, viewing and listening to completed image and audio file on Smart Device (computer system 10) via an input from display 628.
  • sharing DIGY and audio may be attached to an email or text, air dropped, or uploaded to social media to share.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

A system to capture a plurality of two dimensional digital source images of scene by user, a smart device having a memory device for storing an instruction, a processor in communication with the memory and configured to execute the instruction, a plurality of digital image capture devices in communication with the processor and each image capture device configured to capture a digital image of the scene, the plurality of digital image capture devices positioned linearly in series within approximately an interpupillary distance, wherein a first digital image capture devices is centered proximate a first end of the interpupillary distance, a second digital image capture devices is centered on a second end of the interpupillary distance, and any remaining the plurality of digital image capture devices are evenly spaced therebetween, a display in communication with the processor, display configured to display multidimensional digital image sequence and add audio file thereto.

Description

2D DIGITAL IMAGE CAPTURE SYSTEM AND SIMULATING 3D DIGITAL IMAGE
AND SEQUENCE
FIELD OF THE DISCLOSURE
[0001] The present disclosure is directed to 2D image capture, image processing, and simulating display of a 3D or multi-dimensional image or image sequence.
BACKGROUND
[0002] The human visual system (HVS) relies on two dimensional images to interpret three dimensional fields of view. By utilizing the mechanisms with the HVS we create images/scenes that are compatible with the HVS.
[0003] Mismatches between the point at which the eyes must converge and the distance to which they must focus when viewing a 3D image have negative consequences. While 3D imagery has proven popular and useful for movies, digital advertising, many other applications may be utilized if viewers are enabled to view 3D images without wearing specialized glasses or a headset, which is a well-known problem. Misalignment in these systems results in jumping images, out of focus, or fuzzy features when viewing the digital multidimensional images. The viewing of these images can lead to headaches and nausea.
[0004] In natural viewing, images arrive at the eyes with varying binocular disparity, so that as viewers look from one point in the visual scene to another, they must adjust their eyes' vergence. The distance at which the lines of sight intersect is the vergence distance. Failure to converge at that distance results in double images. The viewer also adjusts the focal power of the lens in each eye (i.e., accommodates) appropriately for the fixated part of the scene. The distance to which the eye must be focused is the accommodative distance. Failure to accommodate to that distance results in blurred images. Vergence and accommodation responses are coupled in the brain, specifically, changes in vergence drive changes in accommodation and changes in accommodation drive changes in vergence. Such coupling is advantageous in natural viewing because vergence and accommodative distances are nearly always identical. [0005] In 3D images, images have varying binocular disparity thereby stimulating changes in vergence as happens in natural viewing. But the accommodative distance remains fixed at the display distance from the viewer, so the natural correlation between vergence and accommodative distance is disrupted, leading to the so-called vergence-accommodation conflict. The conflict causes several problems. Firstly, differing disparity and focus information cause perceptual depth distortions. Secondly, viewers experience difficulties in simultaneously fusing and focusing on key subject within the image. Finally, attempting to adjust vergence and accommodation separately causes visual discomfort and fatigue in viewers.
[0006] Perception of depth is based on a variety of cues, with binocular disparity and motion parallax generally providing more precise depth information than pictorial cues. Binocular disparity and motion parallax provide two independent quantitative cues for depth perception. Binocular disparity refers to the difference in position between the two retinal image projections of a point in 3D space.
[0007] Conventional stereoscopic displays forces viewers to try to decouple these processes, because while they must dynamically vary vergence angle to view objects at different stereoscopic distances, they must keep accommodation at a fixed distance or else the entire display will slip out of focus. This decoupling generates eye fatigue and compromises image quality when viewing such displays.
[0008] Recently, a subset of photographers is utilizing 1980s cameras such as NIMSLO and NASHIKA 3D 35mm analog film cameras or digital camera moved between a plurality of points to take multiple frames of a scene, develop the film of the multiple frames from the analog camera, upload images into image software, such as PHOTOSHOP, and arrange images to create a wiggle gram, moving GIF effect.
[0009] Therefore, it is readily apparent that there is a recognizable unmet need for a smart device having an integrated 2D digital image capture system, image manipulation application, & display of 3D digital images or image sequence that may be configured to address at least some aspects of the problems discussed above. SUMMARY
Briefly described, in an example embodiment, the present disclosure may overcome the above- mentioned disadvantages and may meet the recognized need for 2D image capture system and display of 3D digital image and 3D sequence a smart device having a memory device for storing an instruction, a processor in communication with the memory and configured to execute the instruction, a plurality of digital image capture devices in communication with the processor and each image capture device configured to capture a digital image of the scene, the plurality of digital image capture devices positioned linearly in series within approximately an interpupillary distance, wherein a first digital image capture devices is centered proximate a first end of the interpupillary distance, a second digital image capture devices is centered on a second end of the interpupillary distance, and any remaining the plurality of digital image capture devices are evenly spaced therebetween, processing steps to configure datasets, and a display configured to display a simulated multidimensional digital image sequence and/or a multidimensional digital image, storage of multidimensional digital image sequence and/or a multidimensional digital image via block chain, storage of audio files for playback while viewing the image file, storage of related verification documents authenticating image or audio file, transmission of stored files, block chain storage of such files, creation of non-fungible assets of such stored files, non-fungible token (NFT).
[0010] Accordingly, a feature of the system and methods of use is its ability to capture a plurality of images of a scene with 2D capture devices positioned approximately an intraocular or interpupillary distance width IPD apart (distance between pupils of human visual system).
[0011] Accordingly, a feature of the system and methods of use is its ability to convert input 2D source images into multi-dimensional/multi-spectral image sequence. The output image follows the rule of a “key subject point” maintained within an optimum parallax to maintain a clear and sharp image.
[0012] Accordingly, a feature of the system and methods of use is its ability to utilize existing viewing devices to display simulated multidimensional digital image sequence.
[0013] Accordingly, a feature of the system and methods of use is its ability of taking, viewing, and sending over the internet multidimensional digital image sequence. This self-contained system can be integrated into a smart phone, tablet or used with external devices. A series of 4 camera lens allow us to produce a special motion parallax image, DIGY, that can be viewed without a special screen. The system can be used in a fully automated mode or in manual for operator inter action with the scene.
[0014] Accordingly, a feature of the system and methods of use is the ability to integrate viewing devices or other viewing functionality into the display, such as barrier screen (black line), lenticular, arced, curved, trapezoid, parabolic, overlays, waveguides, black line and the like with an integrated LCD layer in an LED or OLED, LCD, OLED, and combinations thereof or other viewing devices.
[0015] Another feature of the digital multi-dimensional image platform based system and methods of use is the ability to produce digital multi-dimensional images that can be viewed on viewing screens, such as mobile and stationary phones, smart phones (including iPhone), tablets, computers, laptops, monitors and other displays and/or special output devices, directly without 3D glasses or a headset.
[0016] In an exemplary embodiment a system to simulate a 3D image sequence from a series of 2D images of a scene, the system includes a smart device having a memory device for storing an instruction, a processor in communication with said memory device configured to execute said instruction, a plurality of digital image capture devices in communication with said processor and each of said plurality image capture devices configured to capture a digital image of the scene, said plurality of digital image capture devices approximately positioned linearly in series within approximately an interpupillary distance width, wherein a first digital image capture devices is positioned proximate a first end of said interpupillary distance width, a second digital image capture devices is positioned proximate a second end of said interpupillary distance width, and any remaining said plurality of digital image capture devices are evenly spaced therebetween to capture a series of 2D images of the scene, a display in communication with said processor, said display configured to display said a multidimensional digital image sequence, and overlay an audio file on said multidimensional digital image sequence via an input on said display.
[0017] A feature of the present disclosure may include a system having a series of capture devices, such as two, three, four or more, such plurality of capture devices (digital image cameras) positioned in series linearly within an intraocular or interpupilary distance width, the distance between an average human’s pupils, the system captures and stores two, three, four or more, a plurality of 2D source images of a scene, the system labels and identifies the images based on the source capture device that captured the image.
[0018] A feature of the present disclosure may include a system having a display device configured from a stack of components, such as top glass cover, capacitive touch screen glass, polarizer, diffusers, and backlight. Moreover, an image source, such as LCD, such LED, ELED, PDP, QLED, and other types of display technologies. Furthermore, display device may include a lens array preferably positioned between capacitive touch screen glass and LCD panel stack of components, and configured to bend or refract light in a manner capable of displaying both a high quality 2D image and an interlaced stereo pair of left and right images as 3D or multidimensional digital image of scene.
[0019] A feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine the convergence point or key subject point, since the viewing of an image that has not been aligned to a key subject point causes confusion to the human visual system and results in blur and double images.
[0020] A feature of the present disclosure is the ability to select the convergence point or key subject point anywhere between near or close plane and far or back plane, manual mode user selection.
[0021] A feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine Circle of Comfort (CoC), since the viewing of an image that has not been aligned to the Circle of Comfort (CoC) causes confusion to the human visual system and results in blur and double images.
[0022] A feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine Circle of Comfort (CoC) fused with Horopter arc or points and Panum area, since the viewing of an image that has not been aligned to the Circle of Comfort (CoC) fused with Horopter arc or points and Panum area causes confusion to the human visual system and results in blur and double images.
[0023] A feature of the present disclosure is the ability to overcome the above defects via another important parameter to determine gray scale depth map, the system interpolates intermediate points based on the assigned points (closest point, key subject point, and furthest point) in a scene, the system assigns values to those intermediate points and renders the sum to a gray scale depth map. The gray scale map to generate volumetric parallax using values assigned to the different points (closest point, key subject point, and furthest point) in a scene. This modality also allows volumetric parallax or rounding to be assigned to singular objects within a scene.
[0024] A feature of the present disclosure is its ability to measure depth or z-axis of objects or elements of objects and/or make comparisons based on known sizes of objects in a scene.
[0025] A feature of the present disclosure is its ability to utilize a key subject algorithm to manually or automatically select the key subject in a plurality of images of a scene displayed on a display and produce multidimensional digital image sequence for viewing on a display.
[0026] A feature of the present disclosure is its ability to utilize an image alignment, horizontal image translation, or edit algorithm to manually or automatically align the plurality of images of a scene about a key subject for display.
[0027] A feature of the feature of the present disclosure is its ability to utilize an image translation algorithm to align the key subject point of two images of a scene for display.
[0028] A feature of the feature of the present disclosure is its ability to generate DIFYS (Differential Image Format) is a specific technique for obtaining multi-view of a scene and creating a series of image that creates depth without glasses or any other viewing aides. The system utilizes horizontal image translation along with a form of motion parallax to create 3D viewing. DIFYS are created by having different view of a single scene flipped by the observer’s eyes. The views are captured by motion of the image capture system or by multiple cameras taking a scene with each of the cameras within the array viewing at a different position.
[0029] In accordance with a first aspect of the present disclosure of simulating a 3D image sequence from a sequence of 2D image frames, may be utilized to capture(ing) a plurality of 2D image frames (images) of a scene from a plurality of different observation points, wherein a first proximal plane and a second distal plane is identified within each image frame in the sequence, and wherein each observation point maintains substantially the same first proximal image plane for each image frame; determining a depth estimate for the first proximal and second distal plane within each image frame in the sequence, aligning the first proximal plane of each image frame in the sequence and shifting the second distal plane of each subsequent image frame in the sequence based on the depth estimate of the second distal plane for each image frame, to produce a modified image frame corresponding to each 2D image frame and displaying the modified image frames sequentially.
[0030] The present disclosure varies the focus of objects at different planes in a displayed scene to match vergence and stereoscopic retinal disparity demands to better simulate natural viewing conditions. By adjusting the focus of key objects in a scene to match their stereoscopic retinal disparity, the cues to ocular accommodation and vergence are brought into agreement. As in natural vision, the viewer brings different obj ects into focus by shifting accommodation. As the mismatch between accommodation and vergence is decreased, natural viewing conditions are better simulated, and eye fatigue is decreased.
[0031] The present disclosure may be utilized to determine three or more planes for each image frame in the sequence.
[0032] Furthermore, it is preferred that the planes have different depth estimates.
[0033] In addition, it is preferred that each respective plane is shifted based on the difference between the depth estimate of the respective plane and the first proximal plane.
[0034] Preferably, the first, proximal plane of each modified image frame is aligned such that the first proximal plane is positioned at the same pixel space.
[0035] It is also preferred that the first plane comprises a key subject point.
[0036] Preferably, the planes comprise at least one foreground plane.
[0037] In addition, it is preferred that the planes comprise at least one background plane.
[0038] Preferably, the sequential observation points lie on a straight line.
[0039] In accordance with a second aspect of the present invention there is a non-transitory computer readable storage medium storing instructions, the instructions when executed by a processor causing the processor to perform the method according to the second aspect of the present invention.
[0040] These and other features of the smart device having 2D digital image capture system, image manipulation application, & display of 3D digital image or image sequence will become more apparent to one skilled in the art from the prior Summary and following Brief Description of the Drawings, Detailed Description of exemplary embodiments thereof, and claims when read in light of the accompanying Drawings or Figures.
BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The present disclosure will be better understood by reading the Detailed Description of the Preferred and Selected Alternate Embodiments with reference to the accompanying drawing Figures, in which like reference numerals denote similar structure and refer to like elements throughout, and in which:
[0042] FIG. 1 A illustrates a 2D rendering of an image based upon a change in orientation of an observer relative to a display;
[0043] FIG. IB illustrates a 2D rendering of an image with binocular disparity as a result of the horizontal separation parallax of the left and right eyes;
[0044] FIG. 2A is an illustration of a cross-section view of the structure of the human eyeball;
[0045] FIG. 2B is a graph relating density of rods and cones to the position of the fovea;
[0046] FIG. 3 is a top view illustration of an observer’s field of view;
[0047] FIG. 4A is a side view illustration identifying planes of a scene captured using a camera or other capture device;
[0048] FIG. 4B A is a front view illustration of an exemplary embodiment of two images of a scene in FIG. 4A captured utilizing capture devices shown in FIGs. 8G;
[0049] FIG. 5 is a top view illustration identifying planes of a scene and a circle of comfort in scale with FIG. 4;
[0050] FIG. 6 is a block diagram of a computer system of the present disclosure;
[0051] FIG. 7 is a block diagram of a communications system implemented by the computer system in FIG. 1;
[0052] FIG. 8A is a diagram of an exemplary embodiment of a computing device with four image capture devices positioned vertically in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils; [0053] FIG. 8B is a diagram of an exemplary embodiment of a computing device with four image capture devices positioned horizontally in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
[0054] FIG. 8C is an exploded diagram of an exemplary embodiment of the four image capture devices in series linearly of FIGs. 8 A and 8B;
[0055] FIG. 8D is a cross-sectional diagram of an exemplary embodiment of the four image capture devices in series linearly of FIGs. 8 A and 8B;
[0056] FIG. 8E is an exploded diagram of an exemplary embodiment of the three image capture devices in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
[0057] FIG. 8F is a cross-sectional diagram of an exemplary embodiment of the three image capture devices in series linearly of FIG. 8E;
[0058] FIG. 8G is an exploded diagram of an exemplary embodiment of the two image capture devices in series linearly within an intraocular or interpupillary distance width, the distance between an average human’s pupils;
[0059] FIG. 8H is a cross-sectional diagram of an exemplary embodiment of the two image capture devices in series linearly of FIG. 8G;
[0060] FIG. 9 is a diagram of an exemplary embodiment of human eye spacing the intraocular or interpupillary distance width, the distance between an average human’s pupils;
[0061] FIG. 10 is a top view illustration identifying planes of a scene and a circle of comfort in scale with right triangles defining positioning of capture devices on lens plane;
[0062] FIG. 10A is a top view illustration of an exemplary embodiment identifying right triangles to calculate the radius of the Circle of Comfort of FIG. 10;
[0063] FIG. 10B is a top view illustration of an exemplary embodiment identifying right triangles to calculate linear positioning of capture devices on lens plane of FIG. 10;
[0064] FIG. IOC is a top view illustration of an exemplary embodiment identifying right triangles to calculate the optimum distance of backplane of FIG. 10; [0065] FIG. 11 is a diagram illustration of an exemplary embodiment of a geometrical shift of a point between two images (frames), such as in FIG. 11 A according to select embodiments of the instant disclosure;
[0066] FIG. 11 A is a front top view illustration of an exemplary embodiment of four images of a scene captured utilizing capture devices shown in FIGs. 8A-8F and aligned about a key subject point;
[0067] FIG. 1 IB is a front view illustration of an exemplary embodiment of four images of a scene captured utilizing capture devices shown in FIGs. 8A-8F and aligned about a key subject point;
[0068] FIG. 12 is an exemplary embodiment of a flow diagram of a method of generating a multidimensional image(s)/sequence captured utilizing capture devices shown in FIGs. 8A-8H;
[0069] FIG. 13 is a top view illustration of an exemplary embodiment of a display with user interactive content to select photography options of computer system;
[0070] FIG. 14A is a top view illustration identifying two frames captured utilizing capture devices shown in FIGs. 8A-8F showing key subject aligned as shown in FIG.1 IB and near plane object offset between two frames;
[0071] FIG. 14B is a top view illustration of an exemplary embodiment of left and right eye virtual depth via object offset between two frames of FIG. 14 A;
[0072] Fig. 15A is a cross-section diagram of an exemplary embodiment of a display stack according to select embodiments of the instant disclosure;
[0073] Fig. 15B is a cross-section diagram of an exemplary embodiment of an arced or curved shaped lens according to select embodiments of the instant disclosure, tracing RGB light there through;
[0074] Fig. 15C is a cross-section diagram of a prototype embodiment of a trapezoid shaped lens according to select embodiments of the instant disclosure, tracing RGB light there through;
[0075] Fig. 15D is a cross-section diagram of an exemplary embodiment of a dome shaped lens according to select embodiments of the instant disclosure, tracing RGB light there through; [0076] FIG. 16A is a diagram illustration of an exemplary embodiment of a pixel interphase processing of images (frames), such as in FIG. 8A according to select embodiments of the instant disclosure;
[0077] FIG. 16B is a top view illustration of an exemplary embodiment of a display of computer system running an application;
[0078] FIG. 17 is a top view illustration of an exemplary embodiment of viewing a multidimensional digital image on display with the image within the Circle of Comfort, proximate Horopter arc or points, within Panum area, and viewed from viewing distance;
[0079] FIG. 18 is an exemplary embodiment of a flow diagram of a method of generating a non-fungible token (NFT) utilizing a computing device with image capture devices capture devices shown in FIGs. 8A-8H;
[0080] FIG. 19 is an exemplary embodiment of a flow diagram of a method of selecting a sequence of DIGYs and adding an audio file thereto utilizing a computing device with image capture devices capture devices shown in FIGs. 8A-8H; and
[0081] FIG. 20 is an exemplary embodiment of a flow diagram of a method of generating a DIGYs and adding an audio file thereto utilizing a computing device with image capture devices capture devices shown in FIGs. 8A-8H.
[0082] It is to be noted that the drawings presented are intended solely for the purpose of illustration and that they are, therefore, neither desired nor intended to limit the disclosure to any or all of the exact details of construction shown, except insofar as they may be deemed essential to the claimed disclosure.
DETAILED DESCRIPTION
[0083] In describing the exemplary embodiments of the present disclosure, as illustrated in figures specific terminology is employed for the sake of clarity. The present disclosure, however, is not intended to be limited to the specific terminology so selected, and it is to be understood that each specific element includes all technical equivalents that operate in a similar manner to accomplish similar functions. The claimed invention may, however, be embodied in many different forms and should not be construed to be limited to the embodiments set forth herein. The examples set forth herein are non-limiting examples and are merely examples among other possible examples.
[0084] Perception of depth is based on a variety of cues, with binocular disparity and motion parallax generally providing more precise depth information than pictorial cues. Binocular disparity and motion parallax provide two independent quantitative cues for depth perception. Binocular disparity refers to the difference in position between the two retinal image projections of a point in 3D space. As illustrated in FIGs. 1A and IB, the robust precepts of depth that are obtained when viewing an object 102 in an image scene 110 demonstrates that the brain can compute depth from binocular disparity cues alone. In binocular vision, the Horopter 112 is the locus of points in space that have the same disparity as the fixation point 114. Objects lying on a horizontal line passing through the fixation point 114 results in a single image, while objects a reasonable distance from this line result in two images 116, 118.
[0085] Classical motion parallax is dependent upon two eye functions. One is the tracking of the eye to the motion (eyeball moves to fix motion on a single spot) and the second is smooth motion difference leading to parallax or binocular disparity. Classical motion parallax is when the observer is stationary and the scene around the observer is translating or the opposite where the scene is stationary, and the observer translates across the scene.
[0086] By using two images 116, 118 of the same object 102 obtained from slightly different angles, it is possible to triangulate the distance to the object 102 with a high degree of accuracy. Each eye views a slightly different angle of the object 102 seen by the left eye 104 and right eye 106. This happens because of the horizontal separation parallax of the eyes. If an object is far away, the disparity 108 of that image 110 falling on both retinas will be small. If the object is close or near, the disparity 108 of that image 110 falling on both retinas will be large.
[0087] Motion parallax 120 refers to the relative image motion (between objects at different depths) that results from translation of the observer 104. Isolated from binocular and pictorial depth cues, motion parallax 120 can also provide precise depth perception, provided that it is accompanied by ancillary signals that specify the change in eye orientation relative to the visual scene 110. As illustrated, as eye orientation 104 changes, the apparent relative motion of the object 102 against a background gives hints about its relative distance. If the object 102 is far away, the object 102 appears stationary. If the object 102 is close or near, the object 102 appears to move more quickly.
[0088] In order to see the object 102 in close proximity and fuse the image on both retinas into one object, the optical axes of both eyes 104, 106 converge on the object 102. The muscular action changing the focal length of the eye lens so as to place a focused image on the fovea of the retina is called accommodation. Both the muscular action and the lack of focus of adjacent depths provide additional information to the brain that can be used to sense depth. Image sharpness is an ambiguous depth cue. However, by changing the focused plane (looking closer and / or further than the object 102), the ambiguities are resolved.
[0089] FIGs. 2A and 2B show the anatomy of the eye 200 and a graphical representation of the distribution of rods and cones, respectively. The fovea 202 is responsible for sharp central vision (also referred to as foveal vision), which is necessary where visual detail is of primary importance. The fovea 202 is the depression in the inner retinal surface 205, about 1.5 mm wide and is made up entirely of cones 204 specialized for maximum visual acuity. Rods 206 are low intensity receptors that receive information in grey scale and are important to peripheral vision, while cones 204 are high intensity receptors that receive information in color vision. The importance of the fovea 202 will be understood more clearly with reference to Figure 2B, which shows the distribution of cones 204 and rods 206 in the eye 200. As shown, a large proportion of cones 204, providing the highest visual acuity, lie within a 1 S angle around the center of the fovea 202
[0090] The importance of the fovea 202 will be understood more clearly with reference to FIG. 2B, which shows the distribution of cones 204 and rods 206 in the eye 200. As shown, a large proportion of cones 204, providing the highest visual acuity, lie within a 1.5° angle around the center of the fovea 202.
[0091] FIG. 3 illustrates a typical field of view 300 of the human visual system (HVS). As shown, the fovea 202 sees only the central IS (degrees) of the visual field 302, with the preferred field of view 304 lying within ± 15 (degrees) of the center of the fovea 202. Focusing an object on the fovea, therefore, depends on the linear size of the object 102, the viewing angle and the viewing distance. A large object 102 viewed in close proximity will have a large viewing angle falling outside the foveal vision, while a small object 102 viewed at a distance will have a small viewing angle falling within the foveal vision. An object 102 that falls within the foveal vision will be produced in the mind’s eye with high visual acuity. However, under natural viewing conditions, viewers do not just passively perceive. Instead, they dynamically scan the visual scene 110 by shifting their eye fixation and focus between objects at different viewing distances. In doing so, the oculomotor processes of accommodation and vergence (the angle between lines of sight of the left eye 104 and right eye 106) must be shifted synchronously to place new objects in sharp focus in the center of each retina. Accordingly, nature has reflexively linked accommodation and vergence, such that a change in one process automatically drives a matching change in the other.
[0092] FIG. 4A illustrates a typical view of a scene S to be captured by a camera or digital image capture device, such as image capture module 830. Scene S may include four planes defined as: (1) Lens frame is defined as the plane passing through the lens or sensor (image capture module 830) in the recording device or camera, (2) Key Subject plane KSP may be the plane passing through the focal point of the sensor in the scene (here couple in the scene, the Key Subject KS of the scene S), (3) Near Plane NP may be the plane passing through the closest point in focus to the lens plane (the bush B in the foreground), and (4) Far Plane FP which is the plane passing through the furthest point in focus (tree T in the background). The relative distances from image capture module 830 are denoted by N, Ks, B. Depth of field of the scene S is defined by the distance between Near Plane NP and Far Plane FP.
[0093] As described above, the sense of depth of a stereoscopic image varies depending on the distance between the camera and the key subject, known as the image capturing distance or KS. The sense of depth is also controlled by the vergence angle and the intraocular distance between the capture of each successive image by the camera which effects binocular disparity.
[0094] In photography the Circle of Confusion defines the area of a scene S that is captured in focus. Thus, the near plane NP, key subject plane KSP and the far plane FP are in focus. Areas outside this circle are blurred.
[0095] FIG. 4B illustrates a typical view of a scene S to be captured by a camera or digital image capture device, such as image capture module 830, more specifically image capture module 830 shown in FIG. 8G. Two image capture devices 831 and 832, or any other selected pair 831, 832, 833, 834 may be utilized to capture plurality of digital images of scene S as left image 810L and right image 81 OR of scene S, shown in FIG. 8 A (plurality of digital images). Alternatively, computer system 10 via image manipulation application and display 208 may be configured to enable user U to select or identify two image capture devices of image capture devices 831 (1), 832 (2), 833 (3), or 834 (4) to capture two digital images of scene S as left image 810L and right image 810R of scene S. User U may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 810L and right image 81 OR of scene S, as shown in FIG. 4B
[0096] FIG. 5 illustrates a Circle of Comfort (CoC) in scale with FIGs. 4.1 and 3.1. Defining the Circle of Comfort (CoC) as the circle formed by passing the diameter of the circle along the perpendicular to Key Subject plane KSP (in scale with FIG. 4) with a width determined by the 30 degree radials of FIG. 3) from the center point on the lens plane, image capture module 830. (R is the radius of Circle of Comfort (CoC).)
[0097] Conventional stereoscopic displays forces viewers to try to decouple these processes, because while they must dynamically vary vergence angle to view objects at different stereoscopic distances, they must keep accommodation at a fixed distance or else the entire display will slip out of focus. This decoupling generates eye fatigue and compromises image quality when viewing such displays.
[0098] In order to understand the present disclosure certain variables, need to be defined. The object field is the entire image being composed. The “key subject point” is defined as the point where the scene converges, i.e., the point in the depth of field that always remains in focus and has no parallax differential in the key subject point. The foreground and background points are the closest point and furthest point from the viewer, respectively. The depth of field is the depth or distance created within the object field (depicted distance from foreground to background). The principal axis is the line perpendicular to the scene passing through the key subject point. The parallax or binocular disparity is the difference in the position of any point in the first and last image after the key subject alignment. In digital composition, the key subject point displacement from the principal axis between frames is always maintained as a whole integer number of pixels from the principal axis. The total parallax is the summation of the absolute value of the displacement of the key subject point from the principal axis in the closest frame and the absolute value of the displacement of the key subject point from the principal axis in the furthest frame. [0099] When capturing images herein, applicant refers refer to depth of field or circle of confusion and circle of comfort is referred to when viewing image on the viewing device.
[00100] US patent 9,992,473, US patent 10,033,990, and US patent 10,178,247 are incorporated herein by reference in their entirety.
[00101] Creating depth perception using motion parallax is known. However, in order to maximize depth while maintaining a pleasing viewing experience, a systematic approach is introduced. The system combines factors of the human visual system with image capture procedures to produce a realistic depth experience on any 2D viewing device.
[00102] The technique introduces the Circle of Comfort (CoC) that prescribe the location of the image capture system relative to the scene S. The Circle of Comfort (CoC) relative to the Key Subject KS (point of convergence, focal point) sets the optimum near plane NP and far plane FP, i.e., controls the parallax of the scene S.
[00103] The system was developed so any capture device such as iPhone, camera or video camera can be used to capture the scene. Similarly, the captured images can be combined and viewed on any digital output device such as smart phone, tablet, monitor, TV, laptop, or computer screen.
[00104] As will be appreciated by one of skill in the art, the present disclosure may be embodied as a method, data processing system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product on a computer-readable storage medium having computer-readable program code means embodied in the medium. Any suitable computer readable medium may be utilized, including hard disks, ROM, RAM, CD-ROMs, electrical, optical, magnetic storage devices and the like.
[00105] The present disclosure is described below with reference to flowchart illustrations of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block or step of the flowchart illustrations, and combinations of blocks or steps in the flowchart illustrations, can be implemented by computer program instructions or operations. These computer program instructions or operations may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions or operations, which execute on the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks/step or steps.
[00106] These computer program instructions or operations may also be stored in a computer- usable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions or operations stored in the computer- usable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks/step or steps. The computer program instructions or operations may also be loaded onto a computer or other programmable data processing apparatus (processor) to cause a series of operational steps to be performed on the computer or other programmable apparatus (processor) to produce a computer implemented process such that the instructions or operations which execute on the computer or other programmable apparatus (processor) provide steps for implementing the functions specified in the flowchart block or blocks/step or steps.
[00107] Accordingly, blocks or steps of the flowchart illustrations support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It should also be understood that each block or step of the flowchart illustrations, and combinations of blocks or steps in the flowchart illustrations, can be implemented by special purpose hardware-based computer systems, which perform the specified functions or steps, or combinations of special purpose hardware and computer instructions or operations.
[00108] Computer programming for implementing the present disclosure may be written in various programming languages, database languages, and the like. However, it is understood that other source or object oriented programming languages, and other conventional programming language may be utilized without departing from the spirit and intent of the present disclosure.
[00109] Referring now to FIG. 6, there is illustrated a block diagram of a computer system 10 that provides a suitable environment for implementing embodiments of the present disclosure. The computer architecture shown in FIG. 6 is divided into two parts - motherboard 600 and the input/output (I/O) devices 620 Motherboard 600 preferably includes subsystems or processor to execute instructions such as central processing unit (CPU) 602, a memory device, such as random access memory (RAM) 604, input/output (I/O) controller 608, and a memory device such as read only memory (ROM) 606, also known as firmware, which are interconnected by bus 10. A basic input output system (BIOS) containing the basic routines that help to transfer information between elements within the subsystems of the computer is preferably stored in ROM 606, or operably disposed in RAM 604. Computer system 10 further preferably includes I/O devices 620, such as main storage device 634 for storing operating system 626 and executes an instruction via application program(s) 624, and display 628 for visual output, and other I/O devices 632 as appropriate. Main storage device 634 preferably is connected to CPU 602 through a main storage controller (represented as 608) connected to bus 610. Network adapter 630 allows the computer system to send and receive data through communication devices or any other network adapter capable of transmitting and receiving data over a communications link that is either a wired, optical, or wireless data pathway. It is recognized herein that central processing unit (CPU) 602 performs instructions, operations or commands stored in ROM 606 or RAM 604.
[00110] It is contemplated herein that computer system 10 may include smart devices, such as smart phone, iPhone, android phone (Google, Samsung, or other manufactures), tablets, desktops, laptops, digital image capture devices, and other computing devices with two or more digital image capture devices and/or 3D display 608 (smart device).
[00111] It is further contemplated herein that display 608 may be configured as a foldable display or multi-foldable display capable of unfolding into a larger display surface area.
[00112] Many other devices or subsystems or other I/O devices 632 may be connected in a similar manner, including but not limited to, devices such as microphone, speakers, flash drive, CD-ROM player, DVD player, printer, main storage device 634, such as hard drive, and/or modem each connected via an I/O adapter. Also, although preferred, it is not necessary for all of the devices shown in FIG. 6 to be present to practice the present disclosure, as discussed below. Furthermore, the devices and subsystems may be interconnected in different configurations from that shown in FIG. 6, or may be based on optical or gate arrays, or some combination of these elements that is capable of responding to and executing instructions or operations. The operation of a computer system such as that shown in FIG. 6 is readily known in the art and is not discussed in further detail in this application, so as not to overcomplicate the present discussion. [00113] Referring now to FIG. 7, there is illustrated a diagram depicting an exemplary communication system 700 in which concepts consistent with the present disclosure may be implemented. Examples of each element within the communication system 700 of FIG. 7 are broadly described above with respect to FIG. 6. In particular, the server system 760 and user system 720 have attributes similar to computer system 10 of FIG. 6 and illustrate one possible implementation of computer system 10. Communication system 700 preferably includes one or more user systems 720, 722, 724 (It is contemplated herein that computer system 10 may include smart devices, such as smart phone, iPhone, android phone (Google, Samsung, or other manufactures), tablets, desktops, laptops, cameras, and other computing devices with display 628 (smart device)), one or more server system 760, and network 750, which could be, for example, the Internet, public network, private network or cloud. User systems 720-724 each preferably includes a computer-readable medium, such as random access memory 604, 606, coupled to a processor. The processor, CPU 702, executes program instructions or operations (application software 624) stored in memory 604, 606. Communication system 700 typically includes one or more user system 720. For example, user system 720 may include one or more general-purpose computers (e.g., personal computers), one or more special purpose computers (e.g., devices specifically programmed to communicate with each other and/or the server system 760), a workstation, a server, a device, a digital assistant or a "smart" cellular telephone or pager, a digital camera, a component, other equipment, or some combination of these elements that is capable of responding to and executing instructions or operations.
[00114] Similar to user system 720, server system 760 preferably includes a computer-readable medium, such as random access memory 604, 606, coupled to a processor. The processor executes program instructions stored in memory 604, 606. Server system 760 may also include a number of additional external or internal devices, such as, without limitation, a mouse, a CD-ROM, a keyboard, a display, a storage device and other attributes similar to computer system 10 of FIG. 6. Server system 760 may additionally include a secondary storage element, such as database 770 for storage of data and information. Server system 760, although depicted as a single computer system, may be implemented as a network of computer processors. Memory 604, 606 in server system 760 contains one or more executable steps, program(s), algorithm(s), or application(s) 624 (shown in FIG.6). For example, the server system 760 may include a web server, information server, application server, one or more general-purpose computers (e.g., personal computers), one or more special purpose computers (e.g., devices specifically programmed to communicate with each other), a workstation or other equipment, or some combination of these elements that is capable of responding to and executing instructions or operations.
[00115] Communications system 700 is capable of delivering and exchanging data (including three-dimensional 3D image files) between user systems 720 and a server system 760 through communications link 740 and/or network 750. Through user system 720, users can preferably communicate data over network 750 with each other user system 720, 722, 724, and with other systems and devices, such as server system 760, to electronically transmit, store, print and/or view multidimensional digital master image(s). Communications link 740 typically includes network 750 making a direct or indirect communication between the user system 720 and the server system 760, irrespective of physical separation. Examples of a network 750 include the Internet, cloud, analog or digital wired and wireless networks, radio, television, cable, satellite, and/or any other delivery mechanism for carrying and/or transmitting data or other information, such as to electronically transmit, store, print and/or view multidimensional digital master image(s). The communications link 740 may include, for example, a wired, wireless, cable, optical or satellite communication system or other pathway.
[00116] Referring again to FIG. 2A, 5, 8A-8F, and 14B for best results and simplified math, the intraocular distance between the capture of successive images or frames of the scene S is fixed to match the average separation of the human left and right eyes in order to maintain constant binocular disparity. In addition, the distance to key subject KS is chosen such that the captured image of the key subject is sized to fall within the foveal vision of the observer in order to produce high visual acuity of the key subject and to maintain a vergence angle equal to or less than the preferred viewing angle of fifteen degrees (15).
[00117] Figures 8A-8F disclose an image or frame capture system for capturing a stereoscopic image (e.g., a 2D frame of a 3D sequence) of scene S, such as FIG. 4. Here the image capture distance, the distance from the image capture system and points or planes in the scene S, such as key subject KS and focal length of camera (i.e., zooming in and out) may be ideally held constant while capturing a stereoscopic image (e.g., a 2D frame of a 3D sequence) of scene S; however, the vergence angle will vary accordingly if the spacing between the capture devices of each successive stereoscopic image is kept constant. [00118] Referring now to FIG. 8 A, by way of example, and not limitation, there is illustrated a computer system 10, such as smart device or portable smart device having back side 810, a first edge, such as short edge 811 and a second edge, such as long edge 812. Back side 810 may include I/O devices 632, such as an exemplary embodiment of image capture module 830 and may include one or more sensors 840 to measure distance between computer system 10 and selected depths in an image or scene S (depth). Image capture module 830 may include a plurality or four digital image capture devices 831, 832, 833, 834 with four digital image capture devices (positioned vertically, in series linearly within an intraocular or interpupillary distance width IPD (distance between pupils of human visual system within a Circle of Comfort relationship to optimize digital multi-dimensional images for the human visual system) as to back side 810 or proximate and parallel thereto long edge 812. Interpupillary distance width IPD is preferably the distance between an average human’s pupils may have a distance between approximately two and a half inches, 2.5 inches (6.35 cm), more preferably between approximately 40-80 mm, the vast majority of adults have IPDs in the range 50-75 mm, the wider range of 45-80 mm is likely to include (almost) all adults, and the minimum IPD for children (down to five years old) is around 40 mm). It is contemplated herein that plurality of image capture modules 830 and may include one or more sensors 840 may be configured as combinations of image capture device 830 and sensor 840 configured as an integrated unit or module where sensor 840 controls or sets the depth of image capture device 830, whether different depths in scene S, such as foreground, and person P or object, background, such as closest point CP, key subject point KS, and a furthest point FP, shown in FIG. 4. For reference herein plurality of image capture devices, may include first digital image capture device 831 centered proximate first end IPD IPD.l of interpupillary distance width IPD, fourth digital image capture device 834 centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining digital image capture devices second digital image capture device 832 and third digital image capture device 833 evenly spaced therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPD, respectively.
[00119] It is contemplated herein that smart device or portable smart device with a display may be configured as rectangular or square or other like configurations providing a surface area having first edge 811 and second edge 812.
[00120] It is contemplated herein that digital image capture devices 831-834 or image capture module 830 may be surrounded by recessed, stepped, or beveled edge 814, each image capture devices 831-34 may be encircled by recessed, stepped, or beveled ring 816, and digital image capture devices 831-34 or image capture module 830 may be covered by lens cover 820 with a lens thereunder lens 818.
[00121] It is contemplated herein that digital image capture devices 831-34 may be individual capture devices and not part of image capture module.
[00122] It is further contemplated herein that digital image capture devices 831-34 may be positioned anywhere on back side 810 and generally parallel thereto long edge 812.
[00123] It is contemplated herein that image capture devices may include additional capture devices positioned within an intraocular or interpupillary distance width IPD.
[00124] It is further contemplated herein that digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
[00125] Referring now to FIG. 8B, by way of example, and not limitation, there is illustrated a computer system 10 or other smart device or portable smart device having back side 810, short edge 811 and a long edge 812. Back side 810 may include I/O devices 632, such as an exemplary embodiment of image capture module 830 and may include one or more sensors 840 to measure distance between computer system 10 and selected depths in an image or scene S (depth). Image capture module 830 may include a plurality or four digital image capture devices 831, 832, 833, 834 with four digital image capture devices (positioned vertically, in series linearly within an intraocular or interpupillary distance width IPD (distance between pupils of human visual system within a Circle of Comfort relationship to optimize digital multi-dimensional images for the human visual system) as to back side 810 or proximate and parallel thereto short edge 812. It is contemplated herein that plurality of image capture modules 830 and may include one or more sensors 840 may be configured as combinations of image capture device 830 and sensor 840 configured as an integrated unit or module where sensor 840 controls or sets the depth of image capture device 830, such as different depths in scene S, such as foreground, background, and person P or object, such as closest point CP, key subject point KS, and furthest point FP, shown in FIG. 4. For reference herein plurality of image capture devices, may include first digital image capture device 831 centered proximate first end IPD IPD.l of interpupillary distance width IPD, fourth digital image capture device 834 centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining image capture devices second digital image capture device 832 and third digital image capture device 833 evenly spaced therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPD.
[00126] It is contemplated herein that digital image capture devices 831-34 or image capture module 830 may be surrounded by recessed, stepped, or beveled edge 814, each image capture devices 831-34 may be encircled by recessed, stepped, or beveled ring 816, and image capture devices 831-34 or image capture module 830 may be covered by lens cover 820 with a lens thereunder lens 818.
[00127] It is contemplated herein that digital image capture devices 831-34 may be individual capture devices and not part of image capture module.
[00128] It is further contemplated herein that digital image capture devices 831-34 may be positioned anywhere on back side 810 and generally parallel thereto long edge 812.
[00129] It is further contemplated herein that digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
[00130] With respect to computer system 10 and image capture devices 830, it is to be realized that the optimum dimensional relationships, to include variations in size, materials, shape, form, position, connection, function and manner of operation, assembly and use, are intended to be encompassed by the present disclosure.
[00131] In this disclosure interpupillary distance width IPD may have a measurement of width to position digital image capture devices 831-334 center-to-center within between approximately maximum width of 115 millimeter to a minimum width of 50 millimeter; more preferably approximately maximum width of 72.5 millimeter to a minimum width of 53.5 millimeter; and most preferably between approximately maximum mean width of 64 millimeter to a minimum mean width of 61.7 millimeter, and an average width of 63 millimeter (2.48 inches) center-to- center width of the human visual system shown in FIG. 9.
[00132] Referring again to FIGs. 1A, IB, 2A, 5, 9, 14B binocular disparity is a stereognostic perception factor that occurs as a result of the average separation of the left and right eyes by approximately 64 mm. When binocular disparity is comparatively large, the observer has the sense that the distance to the key subject is relatively close. When the binocular disparity is comparatively small, the observer has the sense that the distance to the key subject KS is relatively far or large. The vergence angle V refers to the angle between the left and right eyes having the key subject as a vertex when the eyes are focused on the key subject KS. As the vergence angle increases (as both eyes rotate inward), the distance of the key subject KS is perceived by the observer as being relatively small. As the vergence angle decreases (as both eyes rotate outward), the distance of the key subject KS is perceived by the observer as being relatively large.
[00133] Referring now to FIG. 8C, by way of example, and not limitation, there is illustrated an exploded diagram of an exemplary embodiment of image capture module 830. Image capture module 830 may include digital image capture devices 831-834 with four image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Digital image capture devices 831-834 may include first digital image capture device 831, second digital image capture device 832, third digital image capture device 833, fourth digital image capture device 834. First digital image capture device 831 may be centered proximate first end IPD IPD.1 of interpupillary distance width IPD, fourth digital image capture device 834 may be centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining digital image capture devices, such as second digital image capture device 832 and third digital image capture device 833 may be positioned or evenly spaced therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPD. In one embodiment each digital image capture devices 831-834 or lens 818 may surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
[00134] It is further contemplated herein that digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
[00135] Referring now to FIG. 8D, by way of example, and not limitation, there is illustrated a cross-sectional diagram of an exemplary embodiment of image capture module 830, of FIG 8C. Image capture module 830 may include digital image capture devices 831-834 with four image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Digital image capture devices 831-834 may include first digital image capture device 831, second digital image capture device 832, third digital image capture device 833, fourth digital image capture device 834. Each digital image capture devices 831-834 or lens 818 may be surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818. It is contemplated herein that digital image capture devices 831-834 may include optical module, such as lens 818 configured to focus light from scene S on sensor module, such as image capture sensor 822 configured to generate image signals of captured image of scene S, and data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
[00136] It is contemplated herein that other sensor components 822 to generate image signals for the captured image of scene S and other data processing module 824 to process or manipulate the image data may be utilized herein.
[00137] It is contemplated herein that when sensor 840 is not utilized to calculate different depths in scene S (distance from digital image capture devices 831-834 to foreground, background, and person P or object, such as closest point CP, key subject point KS, and furthest point FP, shown in FIG. 4) then a user may be prompted to capture the scene S images a set distance from digital image capture devices 831-834 to key subject point KS in a scene S, including but not limited to six feet (6 ft.) distance from closest point CP or key subject KS point of a scene S.
[00138] It is further contemplated herein that digital image capture devices 831-34 may be utilized to capture a series of 2D images of the scene S.
[00139] Referring now to FIG. 8E, by way of example, and not limitation, there is illustrated an exploded diagram of an exemplary embodiment of image capture module 830. Image capture module 830 may include digital image capture devices 831-833 with a plurality or three image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Digital image capture devices 831-833 may include first digital image capture device 831, second digital image capture device 832, and third digital image capture device 833. First digital image capture device 831 may be centered proximate first end IPD IPD.l of interpupillary distance width IPD, third digital image capture device 833 may be centered proximate second end IPD.2 of interpupillary distance width IPD, and remaining image capture devices, such as second digital image capture device 832 may be centered on center line CL therebetween first end IPD IPD.l and second end IPD.2 of interpupillary distance width IPDE. In one embodiment each digital image capture devices 831-834 or lens 818 may surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818. [00140] It is further contemplated herein that digital image capture devices 831-833 may be utilized to capture a series of 2D images of the scene S.
[00141] Referring now to FIG. 8F, by way of example, and not limitation, there is illustrated a cross-sectional diagram of an exemplary embodiment of image capture module 830, of FIG 8E. Image capture module 830 may include digital image capture devices 831-833 with three image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Digital image capture devices 831-833 may include first digital image capture device 831, second digital image capture device 832, and third digital image capture device 833. Each digital image capture devices 831-833 or lens 818 may be surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818. It is contemplated herein that digital image capture devices 831-833 may include optical module, such as lens 818 configured to focus an image of scene S on sensor module, such as image capture sensor 822 configured to generate image signals for the captured image of scene S, and data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
[00142] Referring now to FIG. 8G, by way of example, and not limitation, there is illustrated an exploded diagram of an exemplary embodiment of image capture module 830. Image capture module 830 may include a plurality or two digital image capture devices 831-832 with two image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Image capture devices 831-832 may include first image capture device 831 and second image capture device 832. First image capture device 831 may be centered proximate first end IPD IPD.l of interpupillary distance width IPD and second image capture device 832 may be centered proximate second end IPD.2 of interpupillary distance width IPD. In one embodiment each image capture devices 831-832 or lens 818 may surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818.
[00143] [0094] Referring now to FIG. 8H, by way of example, and not limitation, there is illustrated an cross-sectional diagram of an exemplary embodiment of image capture module 830, of FIG 8G. Image capture module 830 may include digital or image capture devices 831-832 with two image capture devices in series linearly within an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Image capture devices 831-832 may include first image capture device 831 and second image capture device 832. Each image capture devices 831-832 or lens 818 may be surrounded by beveled edge 814, encircled by ring 816, and/or covered by lens cover 820 with a lens thereunder lens 818. It is contemplated herein that image capture devices 831-832 may include optical module, such as lens 818 configured to focus an image of scene S on sensor module, such as image capture sensor 822 configured to generate image signals for the captured image of scene S, and data processing module 824 configured to generate image data for the captured image on the basis of the generated image signals from image capture sensor 822.
[00144] It is contemplated herein that other sensor components to generate image signals for the captured image of scene S and other data processing module 824 to process or manipulate the image data may be utilized herein.
[00145] It is contemplated herein that image capture module 830 and/or digital image capture devices 831-834 are used to obtain offset 2D digital image views of scene S. Moreover, it is further contemplated herein that image capture module 830 may include a plurality of image capture devices other than the number set forth herein, provided plurality of image capture devices is positioned approximately within intraocular or interpupillary distance width IPD, the distance between an average human’s pupil. Furthermore, it is further contemplated herein that image capture module 830 may include a plurality of image capture devices positioned within a linear distance approximately equal to interpupillary distance width IPD. Still furthermore, it is further contemplated herein that image capture module 830 may include a plurality of image capture devices positioned vertically (computer system 10 or other smart device or portable smart device having short edge 811), horizontally (computer system 10 or other smart device or portable smart device having long edge 812) or otherwise positioned spaced apart in series linearly and within approximately within a linear distance approximately equal to interpupillary distance width IPD.
[00146] It is further contemplated herein that image capture module 830 and digital image capture devices 831-34 positioned linearly within the intraocular or interpupillary distance width IPD enables accurate scene S reproduction therein display 628 to produce a multidimensional digital image on display 628. [00147] Referring now to FIG. 9, by way of example, and not limitation, there is illustrated a front facial view of a human with left eye LE and right eye RE and each having a midpoint of a pupil PI, P2 to illustrate the human eye spacing or the intraocular or interpupillary distance IPD width, the distance between an average human’s visual system pupils. Interpupillary distance (IPD) is the distance measured in millimeters/inches between the centers of the pupils of the eyes. This measurement is different from person to person and also depends on whether they are looking at near objects or far away. Plmay be represented by first end IPD.l of interpupillary distance width IPD and PS may be represented by second end IPD.2 of interpupillary distance width IPD. Interpupillary distance width IPD is preferably the distance between an average human’s pupils may have a distance between approximately two and a half inches, 2.5 inches (6.35 cm), more preferably between approximately 40-80 mm, the vast majority of adults have IPDs in the range 50-75 mm, the wider range of 45-80 mm is likely to include (almost) all adults, and the minimum IPD for children (down to five years old) is around 40 mm).
[00148] It is contemplated herein that left and right images may be produce as set forth in FIGs. 6.1-6.3 from US patent 9,992,473, US patent 10,033,990, and US patent 10,178,247 and electrically communicated to left pixel 550L and right pixel 550R. Moreover, 2D image may be electrically communicated to center pixel 550C.
[00149] Referring now to FIG. 10, there is illustrated by way of example, and not limitation a representative illustration of Circle of Comfort (CoC) in scale with FIG. 4 and 3. For the defined plane, the image captured on the lens plane will be comfortable and compatible with human visual system of user U viewing the final image displayed on display 628 if a substantial portion of the image(s) are captured within the Circle of Comfort (CoC). Any object, such as near plane N, key subject plane KSP, and far plane FP captured by two image capture devices, such as image capture devices 831-833 or image capture devices 831-834 (interpupillary distance IPD) within the Circle of Comfort (CoC) will be in focus to the viewer when reproduced as digital multi-dimensional image sequence viewable on display 628. The back-object plane or far plane FP may be defined as the distance to the intersection of the 15 degree radial line to the perpendicular in the field of view to the 30 degree line or R the radius of the Circle of Comfort (CoC). Moreover, defining the Circle of Comfort (CoC) as the circle formed by passing the diameter of the circle along the perpendicular to Key Subject KS plane (KSP) with a width determined by the 30 degree radials from the center point on the lens plane, image capture module 830. [00150] Linear positioning or spacing of image capture devices, such as digital image capture devices 831-833, or digital image capture devices 831-834 (interpupillary distance IPD) on lens plane within the 30 degree line just tangent to the Circle of Comfort (CoC) may be utilized to create motion parallax between the plurality of images when viewing digital multi-dimensional image sequence viewable on display 628, will be comfortable and compatible with human visual system of user U.
[00151] Referring now to FIG. 10A, 10B, IOC, and 11, there is illustrated by way of example, and not limitation right triangles derived from FIG. 10. All the definitions are based on holding right triangles within the relationship of the scene to image capture. Thus, knowing the key subject KS distance (convergence point) we can calculate the following parameters.
[00152] FIG. 6A to calculate the radius R of Comfort (CoC).
[00153] R/KS = tan 30 degree
[00154] R = KS*tan 30 degree
[00155] FIG. 6B to calculate the optimum distance between image capture devices, such as image capture devices 831-833, or image capture devices 831-834 (interpupillary distance IPD).
[00156] TR/KS = tan 15 degree
[00157] TR = KS* tan 15 degree; and IPD is 2*TR
[00158] FIG. 6C calculate the optimum far plane FP
[00159] Tan 15 degree = R/B
[00160] B = (KS * tan 30 degree)/tan 15 degree
[00161] Ratio of near plane NP to far plane FP = ((KS/ (KS 8 tan 30 degree))*tan 15 degree
[00162] In order to understand the meaning of TR, point on the linear image capture line of the lens plane that the 15 degree line hits/touches the Comfort (CoC). The images are arranged so the key subject KS point is the same in all images captured via plurality of images from image capture devices, such as digital image capture devices 831-833, or digital image capture devices 831-834.
[00163] A user of image capture devices, such as digital image capture devices 831-833, or digital image capture devices 831-834 composes the scene S and moves the digital image capture devices 830 in our case so the circle of confusion conveys the scene S. Since digital image capture devices 830 are using multi cameras linearly spaced there is a binocular disparity between the plurality of images or frames captured by linear offset of digital image capture devices 830, such as digital image capture devices 831-833, or digital image capture devices 831-834. This disparity can be change by changing digital image capture devices 830 settings or moving the key subject KS back or away from digital image capture devices to lessen the disparity or moving the key subject KS closer to digital image capture devices to increase the disparity. Our system is a fixed digital image capture devices system and as a guideline, experimentally developed, the near plane NP should be no closer than approximately six feet from digital image capture devices 830.
[00164] Referring now to FIG. 12, there is illustrated process steps as a flow diagram 1200 of a method of capturing plurality of 2D image(s) of scene S, generating frames 1101-1104, manipulating, reconfiguring, processing, displaying, storing a digital multi-dimensional image sequence as performed by a computer system 10, and viewable on display 628. Note in FIG. 12 some steps designate a manual mode of operation may be performed by a user U, whereby the user is making selections and providing input to computer system 10 in the step whereas otherwise operation of computer system 10 is based on the steps performed by application program(s) 624 in an automatic mode.
[00165] In block or step 1210, providing computer system 10 having digital image capture devices 830, display 628, and applications 624 as described above in FIGS. 1-11, to enable capture of a plurality of 2-dimensional (2D) images with a disparity due to spacing of digital image capture devices 831-834, digital image capture devices 831-834, or the like within approximately an intraocular or interpupillary distance width IPD, the distance between an average human’s pupil, and displaying 3-dimensional (3D) image sequence on display 628. Moreover, the sequential display of digital image(s) on display 628 (DIFY or stereo 3D) where images(n) of the plurality of 2D image(s) of scene S captured by capture devices 831-834 (n devices) are displayed in a sequential order on display 628 as a digital multi-dimensional image sequence (DIFY or stereo 3D).
[00166] In block or step 1215, computer system 10 via image capture application 624 (method of capture) is configured to capture a plurality digital images of scene S via image capture module 830 having a plurality of image capture devices, such as digital image capture devices 831-834, 830 (n devices)., or the like positioned in series linearly within an intraocular or interpupillary distance width IPD (distance between pupils of human visual system within a Circle of Comfort relationship to optimize digital multi-dimensional images for the human visual system) capture a plurality of 2D digital source images. Computer system 10 integrating I/O devices 632 with computer system 10, I/O devices 632 may include one or more sensors 840 in communication with computer system 10 to measure distance between computer system 10 (image capture devices, such as digital image capture devices 831-834, 830 (n devices)) and selected depths in scene S (depth) such as Key Subject KS and set the focal point of one or more digital image capture devices 831-834, 830 (n devices).
[00167] 3D Stereo, user U may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 1102 and right image 1103 of scene S, as shown in FIG. 16. Additionally, in block or step 1215, utilizing computer system 10, display 628, and application program(s) 206 (via image capture application) settings to align(ing) or position(ing) an icon, such as cross hair 814, of FIG. 16B, on key subject KS of a scene S displayed thereon display 628, for example by touching or dragging image of scene S or pointing computer system 10 in a different direction to align cross hair 814, of FIG. 16, on key subject KS of a scene S. In block or step 1215, using, obtaining or capturing images(n) of scene S) focused on selected depths in an image or scene (depth) of scene S.
[00168] Alternatively, computer system 10 via image manipulation application 624 and display 628 may be configured to operate in auto mode wherein one or more sensors 840 may measure the distance between computer system 10 (image capture devices, such as, digital image capture devices 831-834, 830 (n devices)) and selected depths in scene S (depth) such as Key Subject KS. Alternatively, in manual mode, a user may determine the correct distance between computer system 10 and selected depths in scene S (depth) such as Key Subject KS.
[00169] It is recognized herein that user U may be instructed on best practices for capturing images(n) of scene S via computer system 10 via image capture application 624 and display 628, such as frame the scene S to include the key subject KS in scene S, selection of the prominent foreground feature of scene S, and furthest point FP in scene S, may include identifying key subject(s) KS in scene S, selection of closest point CP in scene S, the prominent background feature of scene S and the like. Moreover, position key subject(s) KS in scene S a specified distance from digital image capture devices 831-834 (n devices), 830 (n devices). Furthermore, position closest point CP in scene S a specified distance from digital image capture devices 831-834 (n devices).
[00170] Referring now to FIG. 13, there is illustrated by way of example, and not limitation, touch screen display 628 enabling user U to select photography options of computer system 10. A first exemplary option may be DIFY capture wherein user U may specify or select digital image(s) speed setting 1302 where user U may increase or decrease play back speed or frames (images) per second of the sequential display of digital image(s) on display 628 captured by capture devices 831-834 (n devices). Furthermore, user U may specify or select digital image(s) number of loops or repeats 1304 to set the number of loops of images(n) of the plurality of 2D image(s) 1000 of scene S captured by capture devices 831-834 (n devices), 830 (n devices) where images(n) of the plurality of 2D image(s) 1000 of scene S captured by capture devices 831-834 (n devices) are displayed in a sequential order on display 628, similar to FIG. 11. Still furthermore, user U may specify or select order of playback of digital image(s) sequences for playback or palindrome sequence 1306 to set the order of display of images(n) of the plurality of 2D image(s) 1000 of scene S captured by capture devices 831-834 (n devices), 830 (n devices). The timed sequence showing of the images produces the appropriate binocular disparity through the motion pursuit ratio effect. It is contemplated herein that computer system 10 and application program(s) 624 may utilize default or automatic setting herein.
[00171] Referring now to FIG. 16B, there is illustrated by way of example, and not limitation, touch screen display 628 enabling user U to select photography options of computer system 10 (3D Stereo). In block or step 1215, utilizing computer system 10, display 628, and application program(s) 624 (via image capture application) settings to align(ing) or position(ing) an icon, such as cross hair 1310 on key subject KS of a scene S displayed thereon display 628, for example by touching or dragging image of scene S or pointing computer system 10 in a different direction to align cross hair 1310, on key subject KS of a scene S. In block or step 1215, obtaining or capturing images(n) of scene S from image capture devices 831-834 (n devices) focused on selected depths in an image or scene (depth) of scene S. User U may tap or other identification interaction with selection box 1312 to select or identify key subject KS in the source images, left image 1102L and right image 1103R of scene S, selected from images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S from image capture devices 831-834 (n devices), or any combination of two images from images(n) 1101, 1102, 1103, 1104 (set of frames 1100) . Moreover, computer system 10 via image manipulation application and display 624 may be configured to enable user U to select or identify images of scene S as left image 1102 and right image 1103 of scene S. User U may tap or other identification interaction with selection box 812 to select or identify key subject KS in the source images, left image 1102 and right image 1103 of scene S, as shown in FIG. 16B.
[00172] Alternatively, in block or step 1215, user U may utilize computer system 10, display 628, and application program(s) 624 to input plurality of images, files, and dataset (Dataset) of scene S, such as via AirDrop, DROP BOX, or other application.
[00173] It is recognized herein that step 1215, computer system 10 via image capture application 624, image manipulation application 624, image display application 624 may be performed utilizing distinct and separately located computer systemslO, such as one or more user systems 720 first smart device, 722 second smart device, 724 smart device (smart devices) and application program(s) 624. For example, using a camera system remote from image manipulation system, and remote from image viewing system, step 1215 may be performed proximate scene S via computer system 10 (first processor) and application program(s) 624communicating between user systems 720, 722, 724 and application program(s) 624. Here, camera system may be positioned or stationed to capture segments of different viewpoints of an event or entertainment, such as scene S. Next, via communications link 740 and/or network 750, or 5G computer systems 10 and application program(s) 624 via more user systems 720, 722, 724 may capture and transmit a plurality of digital images of scene S as digital multi-dimensional image sequence (DIFY) of scene S sets of images(n) of scene S from capture devices 831-834 (n devices) relative to key subject KS point.
[00174] Images captured at or near interpupillary distance IPD matches the human visual system, which simplifies the math, minimizes cross talk between the two images, reduces fuzziness and image movement to produce digital multi-dimensional image sequence (DIFY) viewable on display 628.
[00175] Additionally, in block or step 1215, utilizing computer system 10, display 628, and application program(s) 624 (via dataset capture application) settings to align(ing) or position(ing) an icon, such as cross hair 1310, of FIGs. 13 or 16B, on key subject KS of a scene S displayed thereon display 628, for example by touching or dragging dataset of scene S, or touching and dragging key subject KS, or pointing computer system 10 in a different direction to align cross hair 1310, of FIGs. 13, or 16B, on key subject KS of a scene S. In block or step 1215, obtaining or capturing plurality images, files, and dataset (Dataset) of scene S from plurality of capture device(s) 830 (n devices) focused on selected depths in an image or scene (depth) of scene S.
[00176] Moreover, in block or step 1215, integrating I/O devices 632 with computer system 10, I/O devices 632 may include one or more sensors 852 in communication with computer system 10 to measure distance between computer system 10/ capture device(s) 830 (n devices) and selected depths in scene S (depth) such as Key Subject KS and set the focal point of an arc or trajectory of vehicle 400 and capture device(s) 830. It is contemplated herein that computer system 10, display 628, and application program(s) 624, may operate in auto mode wherein one or more sensors 840 may measure the distance between capture device(s) 830 and selected depths in scene S (depth) such as Key Subject KS. Alternatively, in manual mode, a user may determine the correct distance between user U and selected depths in scene S (depth) such as Key Subject KS. Or computer system 10, display 628 may utilize one or more sensors 852 to measure distance between capture device(s) 830 (n devices) and selected depths in scene S (depth) such as Key Subject KS and provide on screen instructions or message (distance preference) to instruct user U to move capture device(s) 830 (n devices) closer or father away from Key Subject KS or near plane NP to optimize capture device(s) 830 (n devices) and images, files, and dataset (Dataset) of scene S.
[00177] In block or step 1220, computer system 10 via image manipulation application 624 is configured to receive a plurality of images of scene S captured by digital image capture devices 831-834 (n devices), 830 (n devices) through an image acquisition application. The image acquisition application converts each image to a digital source image, such as a JPEG, GIF, TIF format. Ideally, each digital source image includes a number of visible objects, subjects or points therein, such as foreground or closest point associated with near plane NP, far plane FP or furthest point associated with a far plane FP, and key subject KS. The near plane NP, far plane FP point are the closest point and furthest point from the viewer (plurality of capture devices 831 and 832, 833, or 834, 830 (n devices)), respectively. The depth of field is the depth or distance created within the object field (depicted distance between foreground to background). The principal axis is the line perpendicular to the scene passing through the key subject KS point, while the parallax is the displacement of the key subject KS point from the principal axis, see FIG. 11. In digital composition the displacement is always maintained as a whole integer number of pixels from the principal axis. [00178] It is recognized herein that step 1220, computer system 10 via image capture application 624, image manipulation application 624, image display application 624 may be performed utilizing distinct and separately located computer systemslO, such as one or more user systems 720, 722, 724 and application program(s) 624. For example, using an image manipulation system remote from image capture system, and remote from image viewing system, step 1220 may be performed remote from scene S via computer system 10 (third processor) and application program(s) 624 communicating between user systems 720, 222, 224 and application program(s) 624. Next, via communications link 740 and/or network 750, or 5G computer systems 10 (third processor) and application program(s) 624 via more user systems 720, 722, 724 may receive sets of plurality of images(n) of scene S from capture devices 831-834 (n devices) relative to key subject KS point and transmit a manipulated plurality of digital multi-dimensional image sequence (DIFY or 3D stereo) of scene to computer system 10 (first processor) and application program(s) 624, step 1220 A.
[00179] In block or step 1220A, computer system 10 via automatic key subject selection algorithm or key subject application program(s) 624 is configured to identify a key subject KS in each source image, plurality of images of scene S captured by digital image capture devices 831- 834 (n devices). Moreover, computer system 10 via key subject, application program(s) 624 is configured to identify (ing) at least in part a pixel, set of pixels (finger point selection on display 628) in one or more plurality of images(n) of scene S from digital image capture devices 831-834 (n devices) as key subject KS, respectively. Moreover, computer system 10 via key subject application program(s) 624 is configured to align source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) horizontally about key subject KS; (horizontal image translation (HIT) as shown in 11A and 11B with a distance key Subject KS within a Circle of Comfort relationship to optimize digital multi-dimensional image sequence 1010 for the human visual system.
[00180] Moreover, a key subject point is identified in the series of 2D images of the scene S, and each of the series of 2D images of the scene is aligned to key subject KS point, and all other points in the series of 2D images of the scene shift based on a spacing of the plurality of digital image capture devices to generate a modified sequence of 2D images. [00181] Key subject KS may be identified in each plurality of images of scene S captured by digital image capture devices 831-834 (n devices) corresponds to the same key subject KS of scene S as shown in FIG. 11 A, 1 IB, and 4. It is contemplated herein that a computer system 10, display 628, and application program(s) 624 may perform an algorithm or set of steps to automatically identify subject KS therein the plurality of images of scene S captured by digital image capture devices 831-834 (n devices). Alternatively, in block or step 1220A, utilizing computer system 10, (in manual mode- manual key subject selection algorithm), display 628, and application program(s) 624 settings to at least in part enable a user U to align(ing) or edit alignment of a pixel, set of pixels (finger point selection), key subject KS point of at least two images(n) of plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
[00182] Source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) of scene S are all obtained with digital image capture devices 831-834 (n devices) with the same image capture distance and same focal length. Computer system 10 via key subject application 624 creates a point of certainty, key subject KS point by performing a horizontal image shift of source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices), whereby source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) overlap at this one point, as shown in FIG. 13. This image shift does two things, first it sets the depth of the image. All points in front of key subject KS point are closer to the observer and all points behind key subject KS point are further from the observer.
[00183] Moreover, in an auto mode computer system 10 via image manipulation application may identify the key subject KS based on a depth map of the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices).
[00184] Computer system 10 via image manipulation application may identify a foreground, closest point and background, furthest point using a depth map of the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices). Alternatively in manual mode, computer system 10 via image manipulation application and display 628 may be configured to enable user U to select or identify key subject KS in the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) of scene S. User U may tap, move a cursor or box or other identification to select or identify key subject KS in the source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) of scene S, as shown in FIG. 13.
[00185] Horizontal image translation (HIT) sets the key subject plane KSP as the plane of the screen from which the scene emanates (first or proximal plane). This step also sets the motion of objects, such as bush B in near plane NP (third or near plane) and tree T in far plane FP (second or distal plane) relative to one another. Objects in front of key subject KS or key subject plane KSP move in one direction (left to right or right to left) while objects behind key subject KS or key subject plane KSP move in the opposite direction from objects in the front. Objects behind the key subject plane KSP will have less parallax for a given motion.
[00186] In the example of FIGs. 11, 11 A and 1 IB, each layer 1100 includes the primary image element of input file images of scene S, such as image or frame 1101, 1102, 1103 and 1104 from digital image capture devices 831-834 (n devices), respectively. Image acquisition application 624, performs a process to translate image or frame 1101, 1102, 1103 and 1104 image or frame 1101, 1102, 1103 and 1104 is overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT). Parallax line 1107 represents the linear displacement of key subject KS points 1109.1-1109.4 from the principal axis 1112. Preferably delta 1120 between the parallax line 1107 represents a linear amount of the parallax 1120, such as front parallax 1120.2 and back parallax 1120.1.
[00187] Calculate parallax, minimum parallax and maximum parallax as a function of number of pixel, pixel density and number of frames, and closest and furthest points, and other parameters as set US patent 9,992,473, US patent 10,033,990, and US patent 10,178,247, incorporated herein by reference in their entirety.
[00188] In block or step 1220B, computer system 10 via depth map application program(s) 624 is configured to create(ing) depth map of source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) and makes a grey scale image through an algorithm. A depth map is an image or image channel that contains information relating to the distance of objects, surfaces, or points in scene S from a viewpoint, such as digital image capture devices 831-834 (n devices). For example, this provides more information as volume, texture and lighting are more fully defined. Once a depth map 1220B is generated then the parallax can be tightly controlled. For this computer system 10 may limit the number of output frames to four without going to a depth map. If we use four from a depth map or two from a depth map, we are not limited by the intermediate camera positions. Note the outer digital image capture devices 831 and 834 are locked into the interpupillary distance (IPD) of the observer or user U viewing display 628.
[00189] Moreover, computer system 10 via key subject, application program(s) 624 may identify key subject KS based on the depth map of the source images. Similarly, computer system 10 via depth map application program(s) 624 may identify Near Plane NP may be the plane passing through the closest point in focus to the lens plane (the bush B in the foreground), Far Plane FP which is the plane passing through the furthest point in focus (tree T in the background) a foreground, closest point and background, furthest point using a depth map of the source image.
[00190] Computer system 10 via depth map application program(s) 624 may define two or more planes for each of series of 2D images of the scene and one or more planes may have different depth estimate. Computer system 10 via depth map application program(s) 624 may identify a first proximal plane, such as key subject plane KSP and a second distal plane within the series of 2D images of the scene, such as Near Plane NP or Far Plane FP.
[00191] In block or step 1220C, computer system 10 via interlay(ing) application program(s) 624 is configured overlay 2D RGB high resolution digital camera thereon model or mesh of images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S with RGB high resolution color (DIFY or 3D stereo).
[00192] In block or step 1220D, computer system 10 via dataset manipulation application 624 may be utilized to generate a model or mesh of scene S from images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S.
[00193] In block or step 1225, computer system 10 via frame establishment program(s) 624 is configured to create or generate frames, recording of images of images(n) 1101, 1102, 1103, 1104 (set of frames 1100) of scene S from a virtual camera shifting, rotation, or arcing position, such as such as 0.5 to 1 degree of separation or movement between frames, such as -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5; for DIFY represented 1101, 1102, 1103, 1104 (set of frames 1100) of 3D color mesh Dataset of scene S to generate parallax; for 3D Stereo as left and right; for example 1102, 1103 images of 3D color mesh Dataset and 1101, 1102, 1103, 1104 for (DIFY). Computer system 10 via key subject, application program(s) 624 may establish increments, of shift for example one (1) degree of total shift between the views (typically 10-70 pixel shift on display 628). This simply means a complete sensor (capture device) rotation of 360 degrees around key subject KS would have 360 views so we are only using/need view 1 and view 2, for 3D Stereo as left and right; 1102, 1103 images Dataset. This gives us 1 degree of separation/disparity for each view assuming rotational parallax orbiting around a key subject (zero parallax point). This will likely establish a minimum disparity/parallax that can be adjusted up as the sensor (image capture module 830) moves farther away from key subject KS.
[00194] In block or step 1225A, computer system 10 via frame establishment program(s) 624 is configured to input or upload source images captured external from computer system 10.
[00195] In block or step 1230, utilizing computer system 10 via horizontal and vertical frame DIF translation application 624 may be configured to horizontally and vertically align or transform each source image, plurality of images of scene S captured by digital image capture devices 831- 834 (n devices) requires a dimensional image format (DIF) transform. The DIF transform is a geometric shift that does not change the information acquired at each point in the source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) but can be viewed as a shift of all other points in the source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices), in Cartesian space (illustrated in FIG. 11). As a plenoptic function, the DIF transform is represented by the equation:
Figure imgf000041_0001
[00197] Where D u,v= D q,f
[00198] In the case of a digital image source, the geometric shift corresponds to a geometric shift of pixels which contain the plenoptic information, the DIF transform then becomes:
Figure imgf000041_0002
[00200] Moreover, computer system 10 via horizontal and vertical frame DIF translation application 624 may also apply a geometric shift to the background and or foreground using the DIF transform. The background and foreground may be geometrically shifted according to the depth of each relative to the depth of the key subject KS identified by the depth map 1220B of the source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices). Controlling the geometrical shift of the background and foreground relative to the key subject KS controls the motion parallax of the key subject KS. As described, the apparent relative motion of the key subject KS against the background or foreground provides the observer with hints about its relative distance. In this way, motion parallax is controlled to focus objects at different depths in a displayed scene to match vergence and stereoscopic retinal disparity demands to better simulate natural viewing conditions. By adjusting the focus of key subjects KS in a scene to match their stereoscopic retinal disparity (an intraocular or interpupillary distance width IPD (distance between pupils of human visual system), the cues to ocular accommodation and vergence are brought into agreement.
[00201] Referring again to FIG. 4, viewing a DIFY, multidimensional image sequence 1010 on display 628 requires two different eye actions of user U. The first is the eyes will track the closest item, point, or object (near plane NP) in multidimensional image sequence 1010 on display 628, which will have linear translation back and forth to the stationary key subject plane KSP due to image or frame 1101, 1102, 1103 and 1104 is overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT)). This tracking occurs through the eyeball moving to follow the motion. Second, the eyes will perceive depth due to the smooth motion change of any point or object relative to the key subject plane KSP and more specifically to the key subject KS point. Thus, DIFYs are composed of one mechanical step and two eye functions.
[00202] A mechanical step of translating of the frames so the Key Subject KS point overlaps on all frames. Linear translation back and forth to the stationary key subject plane KSP due to image or frame 1101, 1102, 1103 and 1104 may be overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT). Eye following motion of near plane NP object which exhibits greatest movement relative to the key subject KS (Eye Rotation). Difference in frame position along the key subject plane KSP (Smooth Eye Motion) which introduces binocular disparity. Comparison of any two points other than key subject KS also produces depth (binocular disparity). Points behind key subject plane KSP move in opposite direction than those points in front of key subject KS. Comparison of two points in front or back or across key subject KS plane shows depth.
[00203] In block or step 1235A, computer system 10 via palindrome application 626 is configured to create, generate, or produce multidimensional digital image sequence 1010 aligning sequentially each image of images(n) of scene S from digital image capture devices 831-834 (n devices) in a seamless palindrome loop (align sequentially), such as display in sequence a loop of first digital image, image or frame 1101 from first digital image capture device 831 (1), second digital image, image or frame 1102 from second digital image capture device 832 (2), capture device 832, third digital image, image or frame 1103 from third digital image capture device 833 (3), fourth digital image, image or frame 1104 from fourth digital image capture device 834 (4). Moreover, an alternate sequence a loop of first digital image, image or frame 1101 from first digital image capture device 831 (1), second digital image, image or frame 1102 from second digital image capture device 832 (2), capture device 832, third digital image, image or frame 1103 from third digital image capture device 833 (3), fourth digital image, image or frame 1104 from fourth digital image capture device 834 (4), fourth digital image, image or frame 1104 from fourth digital image capture device 834 (4), third digital image, image or frame 1103 from third digital image capture device 833 (3), second digital image, image or frame 1102 from second digital image capture device 832 (2), of first digital image, image or frame 1101 from first digital image capture device 831 (1) - 1,2, 3, 4, 4, 3, 2,1 (align sequentially). Preferred sequence is to follow the same sequence or order in which images were captured source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) and an inverted or reverse sequence is added to create a seamless palindrome loop.
[00204] It is contemplated herein that other sequences may be configured herein, including but not limited to 1,2, 3, 4, 3, 2,1 (align sequentially) and the like.
[00205] It is contemplated herein that horizontally and vertically align(ing) of first proximal plane, such as key subject plane KSP of each image of images(n) of scene S from digital image capture devices 831-834 (n devices) and shifting second distal plane, such as such as foreground plane, Near Plane NP, or background plane, Far Plane FP of each subsequent image frame in the sequence based on the depth estimate of the second distal plane for series of 2D images of the scene to produce second modified sequence of 2D images.
[00206] In block or step 1235B, computer system 10 via interphasing application 626 may be configured to interphase columns of pixels of each set of frames 1100, specifically as left image 1102 and right image 1103 to generate a multidimensional digital image aligned to the key subject KS point and within a calculated parallax range. As shown in FIG. 16 A, interphasing application 626 may be configured to takes sections, strips, rows, or columns of pixels from left image 1102 and right image 1103, such as column 1602A of the source images, left image 1102, and right image 1103 of terrain T of scene S and layer them alternating between column 1602 A of left image 1102 - LE, and column 1602 A of right image 1103 - RE and reconfigures or lays them out in series side-by-side interlaced, such as in repeating series 160A two columns wide, and repeats this configuration for all layers of the source images, left image 1102 and right image 1103 of terrain T of scene S to generate multidimensional image 1010 with column 1602 A dimensioned to be one pixel 1550 wide.
[00207] It is contemplated herein that source images, plurality of images of scene S captured by capture device(s) 830 match size and configuration of display 628 aligned to the key subject KS point and within a calculated parallax range.
[00208] Now given the multidimensional image sequence 1010, we move to observe the viewing side of the device.
[00209] It is contemplated herein that source images, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) match size and configuration of display 628 aligned to the key subject KS point and within a calculated parallax range.
[00210] In block or step 1240, computer system 10 via image editing application 624 is configured to crop, zoom, align, enhance, or perform edits thereto each image(n) of scene S from capture devices 831-834 (n devices) or edit multidimensional digital image sequence 1010.
[00211] Moreover, computer system 10 and editing application program(s) 624 may enable user U to perform frame enhancement, layer enrichment, animation, feathering (smooth), (Photoshop or Acorn photo or image tools), to smooth or fill in the images (n) together, or other software techniques for producing 3D effects on display 628. It is contemplated herein that a computer system 10 (auto mode), display 628, and application program(s) 624 may perform an algorithm or set of steps to automatically or enable automatic performance of align(ing) or edit(ing) alignment of a pixel, set of pixels of key subject KS point, crop, zoom, align, enhance, or perform edits of the plurality of images of scene S captured by digital image capture devices 831-834 (n devices) or edit multidimensional digital image sequence 1010. [00212] Alternatively, in block or step 1240, utilizing computer system 10 , (in manual mode), display 628, and application program(s) 624 settings to at least in part enable a user U to align(ing) or edit(ing) alignment of a pixel, set of pixels of key subject KS point, crop, zoom, align, enhance, or perform edits of the plurality of images of scene S captured by digital image capture devices 831-834 (n devices) or edit multidimensional digital image sequence 1010.
[00213] Furthermore, user U via display 628 and editing application program(s) 624 may set or chose the speed (time of view) for each frame and the number of view cycles or cycle forever as shown in FIG. 13. Time interval may be assigned to each frame in multidimensional digital image sequence 1010. Additionally, the time interval between frames may be adjusted at step 1240 to provide smooth motion and optimal 3D viewing of multidimensional digital image sequence 1010.
[00214] It is contemplated herein that a computer system 10, display 628, and application program(s) 624 may perform an algorithm or set of steps to automatically or manually edit or apply effects to set of frames 1100. Moreover, computer system 10 and editing application program(s) 206 may include edits, such as frame enhancement, layer enrichment, feathering, (Photoshop or Acorn photo or image tools), to smooth or fill in the images (n) together, and other software techniques for producing 3D effects to display 3-D multidimensional image of terrain T of scene S thereon display 628.
[00215] In block or step 1250, computer system 10 via image display application 624 is configured to enable images(n) of scene S to display, via sequential palindrome loop, multidimensional digital image sequence 1010 of scene S on display 628 for different dimensions of displays 628. Again, multidimensional digital image sequence 1010 of scene S, resultant 3D image sequence, may be output as a DIF sequence to display 628. It is contemplated herein that computer system 10, display 628, and application program(s) 624 may be responsive in that computer system 10 may execute an instruction to size each image (n) of scene S to fit the dimensions of a given display 628.
[00216] Moreover, user U may elect to return to block or step 1220 to choose a new key subject KS in each source image, plurality of images of scene S captured by digital image capture devices 831-834 (n devices) and progress through steps 1220-1250 to view on display 628, via creation of a new or second sequential loop, multidimensional digital image sequence 1010 of scene S for new key subject KS. [00217] Now given the multidimensional image sequence 1010, we move to observe the viewing side of the device. Moreover, in block or step 735, computer system 10 via output application 730 (206) may be configured to display multidimensional image(s) 1010 on display 628 for one more user systems 220, 222, 224 via communications link 240 and/or network 250, or 5G computer systems 10 and application program(s) 206.
[00218] For 3D Stereo, referring now to FIG. 15 A, there is illustrated by way of example, and not limitation a cross-sectional view of an exemplary stack up of components of display 628. Display 628 may include an array of or plurality of pixels emitting light, such as LCD panel stack of components 1520 having electrodes, such as front electrodes and back electrodes, polarizers, such as horizontal polarizer and vertical polarizer, diffusers, such as gray diffuser, white diffuser, and backlight to emit red R, green G, and blue B light. Moreover, display 628 may include other standard LCD user U interaction components, such as top glass cover 1510 with capacitive touch screen glass 1512 positioned between top glass cover 1510 andLCD panel stack components 1520. It is contemplated herein that other forms of display 628 may be included herein other than LCD, such LED, ELED, PDP, QLED, and other types of display technologies. Furthermore, display 628 may include a lens array, such as lenticular lens 1514 preferably positioned between capacitive touch screen glass 1512 and LCD panel stack of components 1520, and configured to bend or refract light in a manner capable of displaying an interlaced stereo pair of left and right images as a 3D or multidimensional digital image(s) 1010 on display 628 and, thereby displaying a multidimensional digital image of scene S on display 628. Transparent adhesives 1530 may be utilized to bond elements in the stack, whether used as a horizontal adhesive or a vertical adhesive to hold multiple elements in the stack. For example, to produce a 3D view or produce a multidimensional digital image on display 628, a 1920x1200 pixel image via a plurality of pixels needs to be divided in half, 960x1200, and either half of the plurality of pixels may be utilized for a left image and right image.
[00219] It is contemplated herein that lens array may include other techniques to bend or refract light, such as barrier screen (black line), lenticular, parabolic, overlays, waveguides, black line and the like capable of separate into a left and right image.
[00220] It is further contemplated herein that lenticular lens 514 may be orientated in vertical columns when display 628 is held in a landscape view to produce a multidimensional digital image on display 628. However, when display 628 is held in a portrait view the 3D effect is unnoticeable enabling 2D and 3D viewing with the same display 628.
[00221] It is still further contemplated herein that smoothing, or other image noise reduction techniques, and foreground subject focus may be used to soften and enhance the 3D view or multidimensional digital image on display 628.
[00222] Referring now to FIG. 15B, there is illustrated by way of example, and not limitation a representative segment or section of one embodiment of exemplary refractive element, such as lenticular lens 1514 of display 628. Each sub-element of lenticular lens 1514 being arced or curved or arched segment or section 1540 (shaped as an arc ) of lenticular lens 1514 may be configured having a repeating series of trapezoidal lens segments or plurality of sub-elements or refractive elements. For example, each arced or curved or arched segment 1540 may be configured having lens peak 1541 of lenticular lens 1540 and dimensioned to be one pixel 1550 (emitting red R, green G, and blue B light) wide such as having assigned center pixel 1550C thereto lens peak 1541. It is contemplated herein that center pixel 1550C light passes through lenticular lens 1540 as center light 1560C to provide 2D viewing of image on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 or trapezoidal segment or section 1540 of lenticular lens 1514. Moreover, each arced or curved segment 1540 may be configured having angled sections, such as lens angle A1 of lens refractive element, such as lens sub-element 1542 (plurality of sub elements) of lenticular lens 1540 and dimensioned to be one pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens, left lens sub-element 1542L having angle Al, and right lens sub-element 1542R having angle Al, for example an incline angle and a decline angle respectively to refract light across center line CL. It is contemplated herein that pixel 1550L/R light passes through lenticular lens 1540 and bends or refracts to provide left and right images to enable 3D viewing of image on display 628; via left pixel 1550L light passes through left lens angle 1542L and bends or refracts, such as light entering left lens angle 1542L bends or refracts to cross center line CL to the right R side, left image light 1560L toward left eye LE and right pixel 1550R light passes through right lens angle 1542R and bends or refracts, such as light entering right lens angle 1542R bends or refracts to cross center line CL to the left side L, right image light 1560R toward right eye RE, to produce a multidimensional digital image on display 628. [00223] It is contemplated herein that left and right images may be produce as set forth in FIGs. 6.1-6.3 from US patent 9,992,473, US patent 10,033,990, and US patent 10,178,247 and electrically communicated to left pixel 550L and right pixel 550R. Moreover, 2D image may be electrically communicated to center pixel 550C.
[00224] In this figure each lens peak 1541 has a corresponding left and right angled lens 1542, such as left angled lens 1542L and right angled lens 1542R on either side of lens peak 1541 and each assigned one pixel, center pixel 1550C, left pixel 1550L and right pixel 1550R, assigned respectively thereto.
[00225] In this figure the viewing angle A1 is a function of viewing distance VD, size S of display 628, wherein Al= 2 arctan (S/2VD)
[00226] In one embodiment, each pixel may be configured from a set of sub-pixels. For example, to produce a multidimensional digital image on display 628 each pixel may be configured as one or two 3x3 sub-pixels of LCD panel stack components 1520 emitting one or two red R light, one or two green G light, and one or two blue B light therethrough segments or sections of lenticular lens 1540 to produce a multidimensional digital image on display 628. Red R light, green G light, and blue B may be configured as vertical stacks of three horizontal sub-pixels.
[00227] It is recognized herein that trapezoid shaped lens 1540 bends or refracts light uniformly through its center C, left L side, and right R side, such as left angled lens 1542L and right angled lens 1542R, and lens peak 1541.
[00228] Referring now to FIG. 15C, there is illustrated by way of example, and not limitation a prototype segment or section of one embodiment of exemplary lenticular lens 1514 of display 628. Each segment or plurality of sub-elements or refractive elements being trapezoidal shaped segment or section 1540 of lenticular lens 1514 may be configured having a repeating series of trapezoidal lens segments. For example, each trapezoidal segment 1540 may be configured having lens peak 1541 of lenticular lens 1540 and dimensioned to be one or two pixel 1550 wide and flat or straight lens, such as lens valley 1543 and dimensioned to be one or two pixel 1550 wide (emitting red R, green G, and blue B light). For example, lens valley 1543 may be assigned center pixel 1550C. It is contemplated herein that center pixel 1550C light passes through lenticular lens 1540 as center light 1560C to provide 2D viewing of image on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 or trapezoidal segment or section 1540 of lenticular lens 1514. Moreover, each trapezoidal segment 1540 may be configured having angled sections, such as lens angle 1542 of lenticular lens 1540 and dimensioned to be one or two pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens angle 1542L and right lens angle 1542R, respectively. It is contemplated herein that pixel 1550L/R light passes through lenticular lens 1540 and bends to provide left and right images to enable 3D viewing of image on display 628; via left pixel 1550L light passes through left lens angle 1542L and bends or refracts, such as light entering left lens angle 1542L bends or refracts to cross center line CL to the right R side, left image light 1560L toward left eye LE; and right pixel 1550R light passes through right lens angle 1542R and bends or refracts, such as light entering right lens angle 1542R bends or refracts to cross center line CL to the left side L, right image light 1560R toward right eye RE to produce a multidimensional digital image on display 628.
[00229] It is contemplated herein that angle A1 of lens angle 1542 is a function of the pixel 1550 size, stack up of components of display 628, refractive properties of lenticular lens 514, and distance left eye LE and right eye RE are from pixel 1550, viewing distance VD.
[00230] In this FIG.15C, the viewing angle A1 is a function of viewing distance VD, size S of display 628, wherein Al= 2 arctan (S/2VD).
[00231] Referring now to FIG. 15D, there is illustrated by way of example, and not limitation a representative segment or section of one embodiment of exemplary lenticular lens 1514 of display 628. Each segment or plurality of sub-elements or refractive elements being parabolic or dome shaped segment or section 1540A (parabolic lens or dome lens, shaped a dome) of lenticular lens 1514 may be configured having a repeating series of dome shaped, curved, semi-circular lens segments. For example, each dome segment 1540 A may be configured having lens peak 1541 of lenticular lens 1540 and dimensioned to be one or two pixel 1550 wide (emitting red R, green G, and blue B light) such as having assigned center pixel 1550C thereto lens peak 1541. It is contemplated herein that center pixel 1550C light passes through lenticular lens 540 as center light 560C to provide 2D viewing of image on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 or trapezoidal segment or section 1540 of lenticular lens 1514. Moreover, each trapezoidal segment 1540 may be configured having angled sections, such as lens angle 1542 of lenticular lens 1540 and dimensioned to be one pixel wide, such as having left pixel 1550L and right pixel 1550R assigned thereto left lens angle 1542L and right lens angle 1542R, respectively. It is contemplated herein that pixel 1550L/R light passes through lenticular lens 1540 and bends to provide left and right images to enable 3D viewing of image on display 628; via left pixel 1550L light passes through left lens angle 1542L and bends or refracts, such as light entering left lens angle 1542L bends or refracts to cross center line CL to the right R side, left image light 1560L toward left eye LE and right pixel 1550R light passes through right lens angle 1542R and bends or refracts, such as light entering right lens angle 1542R bends or refracts to cross center line CL to the left side L, right image light 1560R toward right eye RE to produce a multidimensional digital image on display 628.
[00232] It is recognized herein that dome shaped lens 1540B bends or refracts light almost uniformly through its center C, left L side, and right R side.
[00233] It is recognized herein that representative segment or section of one embodiment of exemplary lenticular lens 1514 may be configured in a variety of other shapes and dimensions.
[00234] Moreover, to achieve highest quality two dimensional (2D) image viewing and multidimensional digital image viewing on the same display 628 simultaneously, a digital form of alternating black line or parallax barrier (alternating) may be utilized during multidimensional digital image viewing on display 628 without the addition of lenticular lens 1514 to the stack of display 628 and then digital form of digital form of alternating black line or parallax barrier (alternating) may be disabled during two dimensional (2D) image viewing on display 628.
[00235] A parallax barrier is a device placed in front of an image source, such as a liquid crystal display, to allow it to show a stereoscopic or multiscopic image without the need for the viewer to wear 3D glasses. Placed in front of the normal LCD, it consists of an opaque layer with a series of precisely spaced slits, allowing each eye to see a different set of pixels, so creating a sense of depth through parallax. A digital parallax barrier is a series of alternating black lines in front of an image source, such as a liquid crystal display (pixels), to allow it to show a stereoscopic or multiscopic image. In addition, face-tracking software functionality may be utilized to adjust the relative positions of the pixels and barrier slits according to the location of the user's eyes, allowing the user to experience the 3D from a wide range of positions. The book Design and Implementation of Autostereoscopic Displays by Keehoon Hong, Soon-gi Park, Jisoo Hong, Byoungho Lee incorporated herein by reference. [00236] It is contemplated herein that parallax and key subject KS reference point calculations may be formulated for distance between virtual camera positions, interphasing spacing, display 628 distance from user U, lenticular lens 1514 configuration (lens angle Al, 1542, lens per millimeter and millimeter depth of the array), lens angle 1542 as a function of the stack up of components of display 628, refractive properties of lenticular lens 1514, and distance left eye LE and right eye RE are from pixel 1550, viewing distance VD, distance between virtual camera positions (interpupillary distance IPD), and the like to produce digital multi-dimensional images as related to the viewing devices or other viewing functionality, such as barrier screen (black line), lenticular, parabolic, overlays, waveguides, black line and the like with an integrated LCD layer in an LED or OLED, LCD, OLED, and combinations thereof or other viewing devices.
[00237] Incorporated herein by reference is paper entitled Three-Dimensional Display Technology, pages 1-80, by Jason Geng of other display techniques or the like that may be utilized to produce display 628, incorporated herein by reference.
[00238] It is contemplated herein that number of lenses per mm or inch of lenticular lens 514 is determined by the pixels per inch of display 628.
[00239] It is contemplated herein that other angles Al are contemplated herein, distance of pixels 1550C, 1550L, 1550R from of lens 1540 (approximately 0.5 mm), and user U viewing distance from smart device display 628 from user’s eyes (approximately fifteen (15) inches), and average human interpupilary spacing between eyes (approximately 2.5 inches) may be factored or calculated to produce digital multi-dimensional images. Governing rules of angles and spacing assure the viewed images thereon display 628 is within the comfort zone of the viewing device to produce digital multi-dimensional images, see FIGs. 5, 6, 11 below.
[00240] It is recognized herein that angle Al of lens 1541 may be calculated and set based on viewing distance VD between user U eyes, left eye LE and right eye RE, and pixels550, such as pixels 1550C, 1550L, 1550R, a comfortable distance to hold display 628 from user’s U eyes, such as ten (10) inches to arm/wrist length, or more preferably between approximately fifteen (15) inches to twenty -four (24) inches, and most preferably at approximately fifteen (15) inches.
[00241] In use, the user U moves the display 628 toward and away from user’s eyes until the digital multi-dimensional images appear to user, this movement factor in user’s U actual interpupilary distance IPD spacing and to match user’s visual system (near sited and far sited discrepancies) as a function of width position of interlaced left and right images from distance between virtual camera positions (interpupilary distance IPD), key subject KS depth therein each of digital images(n) of scene S (key subject KS algorithm), horizontal image translation algorithm of two images (left and right image) about key subject KS, interphasing algorithm of two images (left and right image) about key subject KS, angles Al, distance of pixels 1550 from of lens 1540 (pixel-lens distance (PLD) approximately 0.5 mm)) and refractive properties of lens array, such as trapezoid shaped lens 1540 all factored in to produce digital multi-dimensional images for user U viewing display 628. First known elements are number of pixels 1550 and number of images, two image, distance between virtual camera positions, or (interpupilary distance IPD). Images captured at or near interpupilary distance IPD matches the human visual system, simplifies the math, minimizes cross talk between the two images, fuzziness, image movement to produce digital multi-dimensional image viewable on display 628.
[00242] It is further contemplated herein that trapezoid shaped lens 1540 may be formed from polystyrene, polycarbonate or other transparent materials or similar materials, as these material offers a variety of forms and shapes, may be manufactured into different shapes and sizes, and provide strength with reduced weight; however, other suitable materials or the like, can be utilized, provided such material has transparency and is machineable or formable as would meet the purpose described herein to produce a left and right stereo image and specified index of refraction. It is further contemplated herein that trapezoid shaped lens 1541 may be configured with 4.5 lenticular lens per millimeter and approximately 0.33 mm depth.
[00243] DIFY, in block or step 1250, computer system 10 via image display application 624 is configured to set of frames 1100 of terrain T of scene S to display, via sequential palindrome loop, multidimensional digital image sequence 1010 on display 628 for different dimensions of displays 628. Again, multidimensional digital image sequence 1010 of scene S, resultant 3D image sequence, may be output as a DIF sequence or .MPO file to display 628. It is contemplated herein that computer system 10, display 628, and application program(s) 624 may be responsive in that computer system 10 may execute an instruction to size each image (n) of scene S to fit the dimensions of a given display 628.
[00244] In block or step 1250, multidimensional image sequence 1010 on display 628, utilizes a difference in position of objects in each of images(n) of scene S from set of frames 1100 relative to key subject plane KSP, which introduces a parallax disparity between images in the sequence to display multidimensional image sequence 1010 on display 628 to enable user U, in block or step 1250 to view multidimensional image sequence 1010 on display 628.
[00245] Moreover, in block or step 1250, computer system 10 via output application 624 may be configured to display multidimensional image sequence 1010 on display 628 for one more user system 720, 722, 724 via communications link 740 and/or network 750, or 5G computer systems 10 and application program(s) 624.
[00246] 3D Stereo, in block or step 1250, computer system 10 via output application 624 may be configured to display multidimensional image 1010 on display 628. Multidimensional image 1010 may be displayed via left and right pixel 1102L/1103R light passes through lenticular lens 1540 and bends or refracts to provide 3D viewing of multidimensional image 1010 on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550.
[00247] In block or step 1250, utilizing computer system 10, display 628, and application program(s) 624 settings to configure each images(n) (L&R segments) of scene S from set of frames 1100 of terrain T of scene S simultaneously with Key Subject aligned between images for binocular disparity for display/view/save multi-dimensional digital image(s) 1010 on display 628, wherein a difference in position of each images(n) of scene S from virtual cameras relative to key subject KS plane introduces a (left and right) binocular disparity to display a multidimensional digital image 1010 on display 628 to enable user U , in block or step 1250 to view multidimensional digital image on display 628.
[00248] Moreover, user U may elect to return to block or step 1220 to choose a new key subject KS in each source image, set of frames 1100 of terrain T of scene S and progress through steps 1220-1250 to view on display 628, via creation of a new or second sequential loop, multidimensional digital image sequence 1010 of scene S for new key subject KS.
[00249] Display 628 may include display device (e.g., viewing screen whether implemented on a smart phone, PDA, monitor, TV, tablet or other viewing device, capable of projecting information in a pixel format) or printer (e.g., consumer printer, store kiosk, special printer or other hard copy device) to print multidimensional digital master image on, for example, lenticular or other physical viewing material. [00250] It is recognized herein that steps 1220-1240, may be performed by computer system 10 via image manipulation application 626 utilizing distinct and separately located computer systems 10, such as one or more user systems 720, 722, 724 and application program(s) 626 performing steps herein. For example, using an image processing system remote from image capture system, and from image viewing system, steps 1220-1240 may be performed remote from scene S via computer system 10 or server 760 and application program(s) 624 and communicating between user systems 720, 722, 724 and application program(s) 626 via communications link 740 and/or network 750, or via wireless network, such as 5G, computer systems 10 and application program(s) 626 via more user systems 720, 722, 724. Here, computer system 10 via image manipulation application 624 may manipulate 24 settings to configure each images(n) (L&R segments) of scene S from of scene S from virtual camera to generate multidimensional digital image sequence 1010 aligned to the key subject KS point and transmit for display multidimensional digital image/sequence 1010 to one or more user systems 720, 722, 724 via communications link 740 and/or network 750, or via wireless network, such as 5G computer systems 10 or server 760 and application program(s) 624.
[00251] Moreover, it is recognized herein that steps 1220-1240, may be performed by computer system 10 via image manipulation application 624 utilizing distinct and separately located computer systemslO positioned on the vehicle. For example, using an image processing system remote from image capture system, steps 1220-1240 via computer system 10 and application program(s) 624 computer systemslO may manipulate 24 settings to configure each images(n) (L&R segments) of scene S from of scene S from capture device(s) 830 to generate a multidimensional digital image/sequence 1010 aligned to the key subject KS point. Here, computer system 10 via image manipulation application 626 may utilize multidimensional image/sequence 1010 to navigate the vehicle V through terrain T of scene S. Alternatively, computer system 10 via image manipulation application 626 may enable user U remote from vehicle V to utilize multidimensional image/sequence 1010 to navigate the vehicle V through terrain T of scene S.
[00252] It is contemplated herein that computer system 10 via output application 624 may be configured to enable display of multidimensional image sequence 1010 on display 628 to enable a plurality of user U, in block or step 1250 to view multidimensional image sequence 1010 on display 628 live or as a replay/rebroadcast. [00253] It is recognized herein that step 1250, may be performed by computer system 10 via output application 624 utilizing distinct and separately located computer systems 10, such as one or more user systems 720, 722, 724 and application program(s) 624 performing steps herein. For example, using an output or image viewing system, remote from scene S via computer system 10 and application program(s) 624 and communicating between user systems 720, 722, 724 and application program(s) 626 via communications link 740 and/or network 750, or via wireless network, such as 5G, computer systems 10 and application program(s) 624 via more user systems 720, 722, 724. Here, computer system 10 output application 624 may receive manipulated plurality of two digital images of scene S and display multidimensional image/sequence 1010 to one more user systems 720, 722, 724 via communications link 740 and/or network 750, or via wireless network, such as 5G computer systems 10 and application program(s) 624.
[00254] Moreover, via communications link 740 and/or network 750, wireless, such as 5G second computer system 10 and application program(s) 624 may transmit sets of images(n) of scene S configured relative to key subject plane KSP as multidimensional image sequence 1010 on display 628 to enable a plurality of user U, in block or step 1250 to view multidimensional image/sequence 1010 on display 628 live or as a replay/rebroadcast.
[00255] Referring now to FIG. 13, there is illustrated by way of example, and not limitation, touch screen display 628 enabling user U to select photography options of computer system 10. A first exemplary option may be DIFY capture wherein user U may specify or select digital image(s) speed setting 1302 where user U may increase or decrease play back speed or frames (images) per second of the sequential display of digital image(s) on display 628 multidimensional image/sequence 1010. Furthermore, user U may specify or select digital image(s) number of loops or repeats 1304 to set the number of loops of images(n) of the plurality of 2D image(s) 1000 of scene S where images(n) of the plurality of 2D image(s) 1000 of scene S are displayed in a sequential order on display 628, similar to FIG. 11. Still furthermore, user U may specify or select order of playback of digital image(s) sequences for playback or palindrome sequence 1306 to set the order of display of images(n) of the multidimensional image/sequence 1010 of scene S. The timed sequence showing of the images produces the appropriate binocular disparity through the motion pursuit ratio effect. It is contemplated herein that computer system 10 and application program(s) 624 may utilize default or automatic setting herein. [00256] DIFY, referring to FIGs. 14A and 14B, there is illustrated by way of example, and not limitation, frames captured in a set sequence which are played back to the eye in a set sequence and a representation of what the human eyes perceives viewing the DIFY on display 628. Explanation of DIFY and its geometry to produce motion parallax. Motion parallax is the change in angle of a point relative to a stationary point. (Motion Pursuit). Note because we have set the key subject KS point all points in foreground will move to the right, while all points in the background will move to the left. The motion is reversed in a paledrone where the images reverse direction. The angular change of any point in different views relative to the key subject creates motion parallax.
[00257] A DIFY is a series of frames captured in a set sequence which are played back to the eye in the set sequence as a loop. For example, the play back of two frames (assume first and last frame, such as frame 1101 and 1104) is depicted in FIG. 14A. FIG. 14A represents the position of an object, such as a near plane NP object in FIG. 4 on the near plane NP and its relation to key subject KS point in frame 1101 and 1104 wherein key subject KS point is constant due to the image translation imposed on the frames, frame 1101, 1102, 1103 and 1104. Frames, frame 1101, 1102, 1103 and 1104 in FIG. 11 A and 1 IB may be overlapping and offset from the principal axis 1112 by a calculated parallax value, (horizontal image translation (HIT) and preset by the spacing of virtual camera. FIG. 14B there is illustrated by way of example, and not limitation what the human eye perceives from the viewing of the two frames (assume first and last frame, such as frame 1101 and 1104 having frame in near plane NP as point 1401 and frame 2 in near plane NP as point 1402) depicted in FIG. 14A on display 628 where image plane or screen plane is the same as key subject KS point and key subject plane KSP and user U viewing display 628 views virtual depth near plane NP 1410 in front of display 628 or between display 628 and user U eyes, left eye LE and right eye RE. Virtual depth near plane NP 1410 is near plane NP as it represents frame 1 in near plane NP as object in near plane point 1401 and frame 2 in near plane NP as object in near plane pointl402, the closest points user U eyes, left eye LE and right eye RE see when viewing multidimensional image sequence 1010 on display 628.
[00258] Virtual depth near plane NP 1410 simulates a visual depth between key subject KS and object in near plane point 1401 and object in near plane point 1402 as virtual depth 1420, depth between the near plane NP and key subject plane KSP. This depth is due to binocular disparity between the two views for the same point, object in near plane point 1401 and object in near plane point 1402. Object in near plane point 1401 and object in near plane point 1402 are preferably same point in scene S, at different views sequenced in time due to binocular disparity. Moreover, outer rays 1430 and more specifically user U eyes, left eye LE and right eye RE viewing angle 1440 is preferably approximately twenty-seven (27) degrees from the retinal or eye axis. (Similar to the depth of field for a cell phone or tablet utilizing display 628.) This depiction helps define the limits of the composition of scene S. Near plane point 1401 and near plane point 1402 preferably lie within the depth of field, outer rays 1430, and near plane NP has to be outside the inner cross over position 1450 of outer rays 1430.
[00259] The motion from XI to X2 is the motion user U eyes, left eye LE and right eye RE will track. Xn is distance from eye lens, left eye LE or right eye RE to image point 1411, 1412 on virtual near image plane 1410. X’n is distance of leg formed from right triangle of Xn to from eye lens, left eye LE or right eye RE to image point 1411, 1412 on virtual near image plane 1410 to the image plane, 628, KS, KSP. The smooth motion is the binocular disparity caused by the offset relative to key subject KS at each of the points user U eyes, left eye LE and right eye RE observe.
[00260] For each eye, left eye LE or right eye RE, a coordinate system may be developed relative to the center of the eye CL and to the center of the intraocular spacing, half of interpupillary distance width IPD, 1440. Two angles b and a are the angles utilized to explain the DIFY motion pursuit b is the angle formed when a line is passed from the eye lens, left eye LE and right eye RE, through the virtual near plane 1410 to the image on the image plane, 628, KS, KSP. Q is b2- bΐ. While a is the angle from the fixed key subject KS of the two frames 1101, 1104 on the image plane 628, KS, KSP to the point 1411, 1412 on virtual near image plane 1410. The change in a represents the eye pursuit. Motion of the eyeball rotating, following the change in position of a point on the virtual near plane. While b is the angle responsible for smooth motion or binocular disparity when compared in the left and right eye. The outer ray 1430 emanating from the eye lens, left eye LE and right eye RE connecting to point 1440 represents the depth of field or edge of the image, half of the image. This line will change as the depth of field of the virtual camera changes.
[00261] di/f=Xi [00262] If we define the pursuit motion as the difference in position of a point along the virtual near plane, then by utilizing the tangents we derive:
[00263]
[00264] X2-Xl=di/(tan [ocl-tanoc2)|
[00265] These equations show us that the pursuit motion, X2 -XI is not a direct function of the viewing distance. As the viewing distance increases the perceived depth di will be smaller but because of the small angular difference the motion will remain approximately the same relative to the full width of the image.
[00266] Mathematically that the ratio of retinal motion over the rate of smooth eye pursuit determines depth relative to the fixation point in central human vision. The creation of the KSP provides the fixation point necessary to create the depth. Mathematically, then all points will move differently from any other point as the reference point is the same in all cases.
[00267] Referring now to FIG. 17, there is illustrated by way of example, and not limitation a representative illustration of Circle of Comfort CoC fused with Horopter arc or points and Panum area. Horopter is the locus of points in space that have the same disparity as fixation, Horopter arc or points. Objects in the scene that fall proximate Horopter arc or points are sharp images and those outside (in front of or behind) Horopter arc or points are fuzzy or blurry. Panum is an area of space, Panum area 1720, surrounding the Horopter for a given degree of ocular convergence with inner limit 1721 and an outer limit 1722, within which different points projected on to the left and right eyes LE/RE result in binocular fusion, producing a sensation of visual depth, and points lying outside the area result in diplopia - double images. Moreover, fuse the images from the left and right eyes for objects that fall inside Panum’ s area, including proximate the Horopter, and user U will we see single clear images. Outside Panum’ s area, either in front or behind, user U will see double images.
[00268] It is recognized herein that computer system 10 via image capture application 624, image manipulation application 624, image display application624 may be performed utilizing distinct and separately located computer systems 10, such as one or more user systems 220, 222, 224 and application program(s) 206. Next, via communications link 240 and/or network 250, wireless, such as 5G second computer system 10 and application program(s) 206 may transmit sets of images(n) of scene S relative to key subject plane introduces a (left and right) binocular disparity to display a multidimensional digital image on display 628 to enable a plurality of user U, in block or step 1250 to view multidimensional digital image on display 628 live or as a replay/rebroadcast.
[00269] Moreover, FIG. 17 illustrates display and viewing of multidimensional image 1010 on display 628 via left and right pixel 1550L/R light of multidimensional image 1010 passes through lenticular lens 1540 and bends or refracts to provide 3D viewing of multidimensional image 1010 on display 628 to left eye LE and right eye RE a viewing distance VD from pixel 1550 with near object, key subject KS, and far object within the Circle of Comfort CoC and Circle of Comfort CoC is proximate Horopter arc or points and within Panum area 1720 to enable sharp single image 3D viewing of multidimensional image 1010 on display 628comfortable and compatible with human visual system of user U.
[00270] Blockchain is a shared, ledger that facilitates the process of recording transactions and tracking assets in a network. An asset can be tangible (title to a house, car, cash, land, artwork or DIGY, 3D Stereo, Datasets thereof) or intangible (intellectual property, patents, copyrights, branding). A blockchain is a decentralized, distributed, and oftentimes public, digital ledger consisting of records called blocks that are used to record transactions across many computers using cryptography so that any involved block cannot be altered retroactively, without the alteration of all subsequent blocks. This allows the participants to verify and audit transactions independently. Each block contains data or Dataset, a cryptographic hash of the previous block (chain), a timestamp, and cryptographic hash (identifies block and all its content). The timestamp proves that the transaction data existed when the block was published to get into its hash. As blocks each contain information about the block previous to it, they form a chain, with each additional block reinforcing the ones before it. Therefore, blockchains are resistant to modification of their data because once recorded, the data in any given block cannot be altered retroactively without altering all subsequent blocks. If change a block then all following blocks invalid and have a different hash. Secure through Proof of Work mechanism slows down transaction rate to create a new block plus distributed. Blockchains are typically managed by a peer-to-peer network, an open and decentralized database, for use as a publicly distributed ledger, where nodes collectively adhere to a protocol to communicate and validate new blocks. Each new block is sent to everyone on the network to verify the new block creating a consensus. Although blockchain records are not unalterable as forks are possible, blockchains may be considered secure by design via a distributed computing system.
[00271] Ethereum is a popular block chain standard as well as other private and open source blockchain algorithms.
[00272] DIGY, 3D Stereo, Datasets, and other datasets and any authentication documents verify creator or author, authenticity statements, history, date of origin, chain of title, chain of owners, or like (authentication) may be included in the dataset (Dataset). Dataset may be stored as a block on the blockchain network and each subsequent transaction thereafter related to Dataset may be set forth in subsequent blocks each contains a cryptographic hash of the previous block, a timestamp of the Dataset transaction data decentralized source of trust.
[00273] Utilize Non-fungible tokens - For select DIGY, DIGY sequences, or Stereo 3D image files, Datasets or other datasets- non-fungible tokens (NFT)s are unique collectible crypto assets stored on blockchain with unique identification codes and metadata that distinguish DIGY, DIGY sequences, or Stereo 3D image file, Dataset, or dataset from any other digital content making it unique one of a kind or limited edition digital content. Enabling the creation of unique digital content and/or set numbers of copies- a certificate of authenticity for a one-of-a-kind or limited number of digital memorabilia that can't be duplicated. The memorabilia is stored on a blockchain network. It's 'non-fungible” because it can't be readily exchanged as similar reproducible content freely moving through the internet. Moreover, authenticating precious art work utilizing DIGY, DIGY sequences, or Stereo 3D image file, Dataset of artwork along with appraisal, and any authentication documents verify creator or author, authenticity statements, history, date of origin, chain of title, chain of owners, or like (authentication) may be included in the dataset (Dataset) and making an smart contract or NFT of Dataset.
[00274] An NFT is simply a record of who owns a unique piece of digital content. That content can be anything from art, music, photograph, 3D graphics, tweets, memes, games, videos, GIFs - you name it. If it's digital and it was created, it can be an NFT.
[00275] Current NFT digital content can be captured by a third party device and mass distributed via Internet by piracy and devalue the NFT digital content of standard NFT content. [00276] Digital content herein are DIGYs and 3D Stereo images generated by artist users and creating their own NFT artwork based on DIGYs and 3D Stereo images using our DIGY and 3D Stereo platforms above.
[00277] Referring now to FIG. 18, there is illustrated a flow diagram 1800 of a method of creating an NFT for DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset).
[00278] In block or step 1810, opening, tapping, or launching app NFT (application program(s) 206) on Smart Device (computer system 10) represented by FIG. 13 and 16B.
[00279] In block or step 1815, tapping/touching/selecting “Create NFT” on Smart Device.
[00280] In block or step 1820, tapping/touching/selecting/accessing “DIGY/ DIGY sequence/Stereo 3D image file Library” icon on display 628 to access image files or the like on Smart Device.
[00281] In block or step 1825, tapping/touching/selecting a DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset) file from Library main storage device 214 of Smart Device 10
[00282] In block or step 1830, tapping/touching/selecting “Create NFT” on display 628 of Smart Device 10.
[00283] In block or step 1835, create NFT for selected image file - DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset). User of Smart Device 10 via a system or App will utilize one of the blockchain providers to issue an NFTs for their DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset). Ethereum is currently the leading blockchain service for NFT issuance. However, there is a range of other blockchains that are becoming increasingly popular. One may be offered by Smart Device manufacture or other service provider.
[00284] In block or step 1840, utilize NFT token standard, compatible wallet services and marketplaces to offer for sale or exchange NFT of DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset). Once created user can offer their NFT, such as NFT- DIGY, DIGY sequences, or Stereo 3D image file for sale or verification on NFT compatible wallet services and marketplace, such as OpenSea which is an Ethereum-basedNFT marketplace. Similar marketplace may be offered by Smart Device manufacture or other service provider. [00285] DIGY, DIGY sequence, Stereo 3D image file, or other data file(s) (Dataset) are unique in that they can’t be captured by a third party device such as another smart device viewing the DIGY, DIGY sequences, or Stereo 3D since such devices do not have access to the original digital files and may not have a license to the DIGY and 3D Stereo platforms. Current NFT digital content can be captured by a third party device and mass distributed via Internet by piracy and devalue the NFT digital content of standard NFT content.
[00286] DIGY herein may include 2D video, 2D image collage, 3D DIGY.
[00287] A sequence is a plurality of DIGYS put together in series or a loop in sequence to create a story.
[00288] Referring now to FIG. 19, there is illustrated a flow diagram 1900 of a method of linking or looping multiple DIGY - MP4 files in sequence or as a loop with or without audio files such as AAC (m4a) format, AIFF, Apple Lossless, MP3, and WAV or other like audio formats stored there with.
[00289] In block or step 1910, opening, tapping, or launching app Photon3D (application program(s) 206) on display 628 of Smart Device (computer system 10) represented by FIG. 13 and 16B.
[00290] In block or step 1915, tapping/touching/selecting “Create DIGY sequence” on display of Smart Device 10.
[00291] In block or step 1920, tapping/touching/selecting “DIGY Library” icon on display 628 to access, for example, DIGY MP4 files or the like on Smart Device 10.
[00292] In block or step 1925, tapping/touching/dragging/selecting first DIGY file from DIGY Library main storage device 214 of Smart Device 10 to time line.
[00293] In block or step 1930, tapping/touching/dragging/selecting second or plurality of DIGY file(s) from DIGY Library on display of Smart Device 10 and placing them in sequence or desired order.
[00294] In block or step 1935, trimming/editing/cropping DIGY file(s) or set duration (start/stop/speed). Arrange DIGY(s) by setting transitions/cuts. Add optional audio files/sound FX as a second audio track and mix sound amplitude. [00295] In block or step 1940, reordering DIGY file(s) by tapping/touching/selecting/dragging by dragging DIGY file(s) to a different order or position is sequence.
[00296] In block or step 1945, saving completed DIGY sequence - MP4 by tapping/touching/selecting Save to save completed image and audio MP4 file in main storage device 214 of Smart Device 10.
[00297] In block or step 1950, viewing and listening to completed DIGY sequence - MP4 file on Smart Device (computer system 10).
[00298] In block or step 1955, sharing DIGY sequence - MP4 file may be attached to an email or text, air dropped, or uploaded to social media to share.
[00299] It is contemplated herein that a user of may match the transitions between two different DIGYs whether manually or programmatically implemented using audio peak detection or even a time signature/tempo matching algorithm BPM: 120 beats per minute; Time signature: 4/4. 4 bars = 4 sections (DIGYs) with 3 transitions. [ DIGY1 - DIGY 2 - DIGY3 - DIGY4] 4 bars x 4 beats per bar = 16 beats; 16 beats @ 120bpm = 8 seconds; 4 sections (DIGY) @ 2 seconds per section = 8 seconds. 1 DIGY per section = 4 DIGYs.
[00300] It is contemplated herein that a user of may use transient detection to find the downbeat at the beginning of the measure and then synchronized this up with the start of the first DIGY. Each DIGY is trimmed to 2 seconds then sequenced together to match the music transitions of the sequenced DIGYs.
[00301] Referring now to FIG. 20, there is illustrated a flow diagram 2000 of a method of manipulating a DIGY or DIGY sequence image file to synchronize with an audio file.
[00302] In block or step 2010, opening, tapping, or launching the app Photon3D (application program(s) 206) on display 628 of Smart Device (computer system 10) represented by FIG. 13 and 16B.
[00303] In block or step 2015, tapping/touching/selecting “Music App” via input to display 628 of Smart Device 10.
[00304] In block or step 2020, tapping/touching/selecting DIGY, DIGY sequence, Stereo 3D image file (Dataset) via input to display 628 of Smart Device 10 to select a Dataset file from memory device 604, 606 via an input from said display 628. [00305] In block or step 2025, executing an instruction 206 via processor 102 to prepare, convert DIGY.gif file to DIGY.mps file via an input from said display 628; to create a DIGY frame time line via an input from display 628; to import image frames (3D DIGY, 2D Images, 2D video) into said frame time line via an input from said display206; to adjust sequence/frame dwell time, via an input from said display 206.
[00306] In block or step 2030, executing an instruction 206 via processor 102 to record an audio files via microphone in Smart Device 10, select an audio file from memory device 604, 606 or online service, such as ITUNES via an input from said display 628 and import or download audio file to time line via an input from said display 628. It is contemplated herein that DIGY sequence and audio file may be converted to .mp4 file for sharing with other Smart Devices 222 via network 250.
[00307] In block or step 2035, executing an instruction 206 via processor 102 to drag, overlay, or place audio file from memory device 604, 606 to DIGY frame time line via an input from display 628.
[00308] In block or step 2040, executing an instruction 206 via processor 102 to adjust, crop, link, or arrange audio file relative to DIGY image file via an input from display 628.
[00309] In block or step 2045, saving completed image and audio file - MP4 by tapping/touching/selecting Save to save completed image and audio MP4 file in main storage device 214 via an input from display 628.
[00310] In block or step 2050, play, viewing and listening to completed image and audio file on Smart Device (computer system 10) via an input from display 628.
[00311] In block or step 2055, sharing DIGY and audio (file) may be attached to an email or text, air dropped, or uploaded to social media to share.
[00312] With respect to the above description then, it is to be realized that the optimum dimensional relationships, to include variations in size, materials, shape, form, position, movement mechanisms, function and manner of operation, assembly and use, are intended to be encompassed by the present disclosure.
[00313] The foregoing description and drawings comprise illustrative embodiments. Having thus described exemplary embodiments, it should be noted by those skilled in the art that the within disclosures are exemplary only, and that various other alternatives, adaptations, and modifications may be made within the scope of the present disclosure. Merely listing or numbering the steps of a method in a certain order does not constitute any limitation on the order of the steps of that method. Many modifications and other embodiments will come to mind to one skilled in the art to which this disclosure pertains having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation. Moreover, the present disclosure has been described in detail, it should be understood that various changes, substitutions and alterations can be made thereto without departing from the spirit and scope of the disclosure as defined by the appended claims. Accordingly, the present disclosure is not limited to the specific embodiments illustrated herein but is limited only by the following claims.

Claims

Claims:
1. A system to simulate a 3D image sequence from a series of 2D images of a scene, the system comprising: a smart device having: a memory device for storing an instruction; a processor in communication with said memory device configured to execute said instruction; a plurality of digital image capture devices in communication with said processor and each of said plurality image capture devices configured to capture a digital image of the scene, said plurality of digital image capture devices approximately positioned linearly in series within approximately an interpupillary distance width, wherein a first digital image capture devices is positioned proximate a first end of said interpupillary distance width, a second digital image capture devices is positioned proximate a second end of said interpupillary distance width, and any remaining said plurality of digital image capture devices are evenly spaced therebetween to capture a series of 2D images of the scene; and a display in communication with said processor, said display configured to display said a multidimensional digital image sequence.
2. The system of Claim 1, wherein said processor executes an instruction to save said series of 2D images of the scene in said memory device.
3. The system of Claim 2, wherein said processor executes an instruction to select said series of 2D images of said scene to make a DIGY.
4. The system of Claim 3, wherein said processor executes an instruction to select an automatic key subject selection algorithm via an input from said display, wherein said processor identifies a key subject point in said series of 2D images of the scene, and each of said series of 2D images of the scene is aligned by said processor to said key subject point, and all other points in said series of 2D images of the scene shift based on a spacing of said plurality of digital image capture devices to generate a modified sequence of 2D images.
5. The system of Claim 3, wherein said processor executes an instruction to select a manual key subject selection algorithm via an input from said display, wherein said processor enables a user to position an icon within said scene via an input from said display to identify a key subject point in said series of 2D images of the scene, and each of said series of 2D images of the scene is aligned by said processor to said key subject point, and all other points in said series of 2D images of the scene shift based on a spacing of said plurality of digital image capture devices to generate a modified sequence of 2D images.
6. The system of claim 3, wherein said processor executes an instruction to select a key subject point in said series of 2D images of the scene via an input from said display.
7. The system of claim 1, wherein said processor executes an instruction to define two or more planes for each of said series of 2D images of the scene, wherein said two or more planes have different depth estimate.
8. The system of claim 7, wherein said processor executes an instruction to identify a first proximal plane and a second distal plane within said series of 2D images of the scene.
9. The system of claim 8, wherein said processor executes an instruction to determine a depth estimate for said first proximal plane and said second distal plane within said series of 2D images of the scene.
10. The system of claim 9, wherein said processor executes an instruction to horizontally and vertically align said first proximal plane of each image frame in said series of 2D images and shifting the second distal plane of each subsequent image frame in the sequence based on the depth estimate of the second distal plane for said series of 2D images of the scene to produce a second modified sequence of 2D images.
11. The system of claim 8, wherein said first proximal plane and said second distal plane further comprising at least a foreground plane and a background plane.
12. The system of claim 10, wherein said processor executes an instruction to align said second modified series of 2D images sequentially in a palindrome loop as a multidimensional digital image sequence.
13. The system of claim 12, wherein said processor executes an instruction to save said multidimensional digital image sequence to said memory.
14. The system of Claim 13, wherein said processor executes an instruction to record an audio file via a microphone in communication with said processor.
15. The system of Claim 14, wherein said processor executes an instruction to save said audio file to said memory.
16. The system of Claim 15, wherein said processor executes an instruction to select a multidimensional digital image sequence from said memory via an input from said display and to display said multidimensional digital image sequence on said display.
17. The system of Claim 16, wherein said processor executes an instruction to select a audio file from said memory via an input from said display and to overlay said audio file on said multidimensional digital image sequence on said display.
18. The system of Claim 17, wherein said processor executes an instruction to crop said audio file to align with said multidimensional digital image sequence via an input from said display.
19. The system of Claim 18, wherein said processor executes an instruction to save said audio file and said multidimensional digital image sequence via an input from said display.
20. The system of Claim 19, wherein said processor executes an instruction to play said audio file and display said multidimensional digital image sequence via an input from said display.
21. The system of Claim 20, wherein said processor executes an instruction to share said audio file and said multidimensional digital image sequence via an input from said display with a second smart device via an input from said display.
22. The system of Claim 20, wherein said processor executes an instruction to generate a non- fungible token of said audio file and said multidimensional digital image sequence with a second smart device via an input from said display
23. The system of Claim 22, wherein said processor executes an instruction to utilize a wallet service and a marketplace to exchange said non-fungible token of said audio file and said multidimensional digital image sequence via an input from said display.
PCT/US2022/032515 2021-06-07 2022-06-07 2d digital image capture system and simulating 3d digital image and sequence WO2022261105A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22820900.3A EP4352953A1 (en) 2021-06-07 2022-06-07 2d digital image capture system and simulating 3d digital image and sequence

Applications Claiming Priority (18)

Application Number Priority Date Filing Date Title
US202163197941P 2021-06-07 2021-06-07
US63/197,941 2021-06-07
US202163212025P 2021-06-17 2021-06-17
US63/212,025 2021-06-17
US17/355,906 2021-06-23
US17/355,906 US20210321077A1 (en) 2020-01-09 2021-06-23 2d digital image capture system and simulating 3d digital image sequence
US17/459,067 2021-08-27
US17/459,067 US20210392314A1 (en) 2020-01-09 2021-08-27 Vehicle terrain capture system and display of 3d digital image and 3d sequence
US17/511,490 2021-10-26
US17/511,490 US20220051427A1 (en) 2020-01-09 2021-10-26 Subsurface imaging and display of 3d digital image and 3d image sequence
US17/525,246 US20220078392A1 (en) 2020-01-09 2021-11-12 2d digital image capture system, frame speed, and simulating 3d digital image sequence
US17/525,246 2021-11-12
US29816462 2021-11-22
US29816461 2021-11-22
US29/816,462 2021-11-22
US29/816,461 2021-11-22
US17/834,023 US20220385807A1 (en) 2021-05-28 2022-06-07 2d digital image capture system and simulating 3d digital image and sequence
US17/834,023 2022-06-07

Publications (1)

Publication Number Publication Date
WO2022261105A1 true WO2022261105A1 (en) 2022-12-15

Family

ID=84425456

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2022/032524 WO2022261111A1 (en) 2021-06-07 2022-06-07 2d digital image capture system and simulating 3d digital image and sequence
PCT/US2022/032515 WO2022261105A1 (en) 2021-06-07 2022-06-07 2d digital image capture system and simulating 3d digital image and sequence

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/US2022/032524 WO2022261111A1 (en) 2021-06-07 2022-06-07 2d digital image capture system and simulating 3d digital image and sequence

Country Status (2)

Country Link
CN (1) CN117897951A (en)
WO (2) WO2022261111A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170364752A1 (en) * 2016-06-17 2017-12-21 Dolby Laboratories Licensing Corporation Sound and video object tracking
US20190394441A1 (en) * 2012-02-24 2019-12-26 Matterport, Inc. Capturing and aligning panoramic image and depth data
US20200195905A1 (en) * 2018-12-14 2020-06-18 Cloudminds (Shenzhen) Robotics Systems Co., Ltd. Method and apparatus for obtaining image, storage medium and electronic device
EP3694208A1 (en) * 2019-02-05 2020-08-12 Jerry Nims A method and system for simulating a 3-dimensional image sequence
US20220078392A1 (en) * 2020-01-09 2022-03-10 JUC Holdings Limited 2d digital image capture system, frame speed, and simulating 3d digital image sequence

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5786909A (en) * 1992-04-17 1998-07-28 Noritsu Koki Co., Ltd. Photographic printing apparatus with exposure position rectifying and method thereof
US20060023197A1 (en) * 2004-07-27 2006-02-02 Joel Andrew H Method and system for automated production of autostereoscopic and animated prints and transparencies from digital and non-digital media
US10178247B2 (en) * 2015-01-30 2019-01-08 Jerry Nims Digital multi-dimensional image photon platform system and methods of use
US9992473B2 (en) * 2015-01-30 2018-06-05 Jerry Nims Digital multi-dimensional image photon platform system and methods of use
JP6829015B2 (en) * 2016-06-23 2021-02-10 マクセル株式会社 Mobile terminal
US10917633B2 (en) * 2017-09-15 2021-02-09 Intel Corporation Generating stereoscopic light field panoramas using concentric viewing circles

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190394441A1 (en) * 2012-02-24 2019-12-26 Matterport, Inc. Capturing and aligning panoramic image and depth data
US20170364752A1 (en) * 2016-06-17 2017-12-21 Dolby Laboratories Licensing Corporation Sound and video object tracking
US20200195905A1 (en) * 2018-12-14 2020-06-18 Cloudminds (Shenzhen) Robotics Systems Co., Ltd. Method and apparatus for obtaining image, storage medium and electronic device
EP3694208A1 (en) * 2019-02-05 2020-08-12 Jerry Nims A method and system for simulating a 3-dimensional image sequence
US20220078392A1 (en) * 2020-01-09 2022-03-10 JUC Holdings Limited 2d digital image capture system, frame speed, and simulating 3d digital image sequence

Also Published As

Publication number Publication date
CN117897951A (en) 2024-04-16
WO2022261111A1 (en) 2022-12-15

Similar Documents

Publication Publication Date Title
US20220385880A1 (en) 2d digital image capture system and simulating 3d digital image and sequence
Kwon et al. A study of layout, rendering, and interaction methods for immersive graph visualization
WO2018076661A1 (en) Three-dimensional display apparatus
US20220078392A1 (en) 2d digital image capture system, frame speed, and simulating 3d digital image sequence
US20060250391A1 (en) Three dimensional horizontal perspective workstation
CN107810633A (en) Three-dimensional rendering system
US11917119B2 (en) 2D image capture system and display of 3D digital image
JP2015069210A (en) Display apparatus and method providing multi-view image
US20210392314A1 (en) Vehicle terrain capture system and display of 3d digital image and 3d sequence
US20210297647A1 (en) 2d image capture system, transmission & display of 3d digital image
US20210321077A1 (en) 2d digital image capture system and simulating 3d digital image sequence
WO2022261105A1 (en) 2d digital image capture system and simulating 3d digital image and sequence
EP4352953A1 (en) 2d digital image capture system and simulating 3d digital image and sequence
Dąbała et al. Manipulating refractive and reflective binocular disparity
EP4233008A1 (en) Vehicle terrain capture system and display of 3d digital image and 3d sequence
Kim et al. Glasses-free randot stereotest
US20240155096A1 (en) 2d image capture system & display of 3d digital image
EP4162321A1 (en) 2d image capture system & display of 3d digital image
WO2021247416A1 (en) 2d image capture system, transmission & display of 3d digital image
WO2021262847A1 (en) 2d digital image capture system and simulating 3d digital image sequence
WO2022104077A1 (en) 2d digital image capture system, frame speed, and simulating 3d digital image sequence
Regalbuto Remote Visual Observation of Real Places through Virtual Reality Headsets
CN214335377U (en) Full-color high-definition (5 k-8 k) high-brightness double-vertical-screen stereo image viewing device
US20220051427A1 (en) Subsurface imaging and display of 3d digital image and 3d image sequence
CN109151273A (en) A kind of light fan stereo camera and stereoscopic measurement method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22820900

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022820900

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022820900

Country of ref document: EP

Effective date: 20240108