US20110242342A1 - Combining data from multiple image sensors - Google Patents

Combining data from multiple image sensors Download PDF

Info

Publication number
US20110242342A1
US20110242342A1 US13/079,616 US201113079616A US2011242342A1 US 20110242342 A1 US20110242342 A1 US 20110242342A1 US 201113079616 A US201113079616 A US 201113079616A US 2011242342 A1 US2011242342 A1 US 2011242342A1
Authority
US
United States
Prior art keywords
data
image
line
data line
sensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/079,616
Other languages
English (en)
Inventor
Sergiu R. Goma
Joseph Cheung
Hau Hwang
Milivoje Aleksic
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Priority to US13/079,616 priority Critical patent/US20110242342A1/en
Priority to JP2013503850A priority patent/JP2013528021A/ja
Priority to PCT/US2011/031282 priority patent/WO2011127076A1/en
Priority to CN201180021758.8A priority patent/CN102884783B/zh
Priority to BR112012025182A priority patent/BR112012025182A2/pt
Priority to ARP110101136A priority patent/AR081160A1/es
Priority to EP11719089A priority patent/EP2556658A1/en
Priority to KR1020127029023A priority patent/KR101399718B1/ko
Priority to TW100111894A priority patent/TWI508520B/zh
Assigned to QUALCOMM INCORPORATED reassignment QUALCOMM INCORPORATED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ALEKSIC, MILIVOJE, CHEUNG, JOSEPH, GOMA, SERGIU R., HWANG, HAU
Publication of US20110242342A1 publication Critical patent/US20110242342A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/665—Control of cameras or camera modules involving internal camera communication with the image sensor, e.g. synchronising or multiplexing SSIS control signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements

Definitions

  • the present disclosure is generally related to combining data from multiple image sensors.
  • wireless computing devices such as portable wireless telephones, personal digital assistants (PDAs), and paging devices that are small, lightweight, and easily carried by users.
  • portable wireless telephones such as cellular telephones and internet protocol (IP) telephones
  • IP internet protocol
  • wireless telephones can communicate voice and data packets over wireless networks.
  • many such wireless telephones include other types of devices that are incorporated therein.
  • a wireless telephone can also include a digital still camera and a digital video camera.
  • such wireless telephones can process executable instructions, including software applications, such as a web browser application, that can be used to access the Internet.
  • wireless devices may execute three dimensional (3D) applications.
  • 3D applications typically at least two image sensors are used to capture depth information from a scene.
  • Frame data from two image sensors is combined and processed to infer distance information and used to construct a 3D representation.
  • Combining image data from each of the sensors typically involves performing frame synchronization and line synchronization, which may result in synchronization and alignment challenges.
  • filtering of image data from multiple sensors and interleaving such image data may be further complicated when source sensors provide data at different frequencies or phases. It would be advantageous to effectively synchronize data from multiple sensors and efficiently process the data to reduce overall image processing system cost and complexity.
  • image data from each of the multiple sensors is to be synchronized at a line level and processed.
  • An image processing system to combine data from multiple sensors is disclosed where image data from a first image sensor and a second image sensor is synchronized and processed. Synchronized data lines are generated by synchronizing and combining first data from a first data stream generated by the first image sensor with second data from a second data stream generated by the second image sensor.
  • the image signal processor is configured to process the synchronized data lines received from a combiner and to output the processed frame to a display.
  • a method in a particular embodiment, includes providing a common control signal to multiple image sensors to be synchronized. The method further includes receiving a first data line from a first image sensor of the multiple image sensors, receiving a second data line from a second image sensor of the multiple image sensors, and combining the first data line and the second data line to generate a synchronized data line.
  • an apparatus in another particular embodiment, includes a first input configured to receive a first data line from a first image sensor of multiple image sensors to be synchronized via a common control signal.
  • the apparatus further includes a second input configured to receive a second data line from a second image sensor of the multiple image sensors, and a combiner coupled to the first input and to the second input, wherein the combiner is configured to combine the first data line and the second data line to generate a synchronized data line.
  • a method in another particular embodiment, includes providing a common control signal to multiple image sensors. Each of the multiple image sensors is responsive to the common control signal to generate image data. The method further includes receiving synchronized data output from each of the multiple image sensors, combining the synchronized data output from each of the multiple image sensors to generate a synchronized data line, and providing the synchronized data line to an image processor via a single camera input of the image processor.
  • an apparatus in another particular embodiment, includes a sensor synchronizer configured to provide a common control signal to multiple image sensors. Each of the multiple image sensors is responsive to the common control signal to generate image data.
  • the apparatus further includes a combiner configured to combine synchronized data output received from each of the multiple image sensors to generate a synchronized data line to be provided to an image processor via a single camera input of the image processor.
  • a method in another particular embodiment, includes providing a common control signal to multiple image sensors. Each of the multiple image sensors is responsive to the common control signal to generate image data. The method further includes receiving synchronized data output from each of the multiple image sensors.
  • a method in another particular embodiment, includes receiving a common control signal at multiple image sensors. Each of the multiple image sensors is responsive to the common control signal to generate image data. The method further includes generating synchronized data output from each of the multiple image sensors.
  • an apparatus in another particular embodiment, includes a sensor synchronizer configured to provide a common control signal to multiple image sensors to cause the multiple image sensors to generate image data.
  • the apparatus further includes a sensor data interface configured to receive synchronized data output from each of the multiple image sensors.
  • a method in another particular embodiment, includes receiving lines of image data at an image processor having an input for a single camera. Each line of the image data includes first line data from a first image captured by a first camera and second line data from a second image captured by a second camera. The method further includes generating an output frame having a first section corresponding to line data of the first image and having a second section corresponding to line data of the second image. The first section and the second section are configured to be used to generate a three-dimensional (3D) image format or a 3D video format.
  • 3D three-dimensional
  • an apparatus in another particular embodiment, includes an image processor having an input for a single camera.
  • the image processor is configured to receive lines of image data via the input.
  • Each line of the image data includes first line data from a first image captured by a first camera and second line data from a second image captured by a second camera.
  • the image processor is configured to generate an output frame having a first section corresponding to line data of the first image and having a second section corresponding to line data of the second image.
  • the first section and the second section are configured to be used to generate a three-dimensional (3D) image format or a 3D video format
  • a method of combining data from multiple sensors into a frame includes receiving a first data stream from a first image sensor, receiving a second data stream a second image sensor, and combining data from the first data stream and from the second data stream to generate a frame.
  • the method further includes processing the frame at an image signal processor to generate a processed frame and outputting the processed frame for display.
  • Each of the first image sensor and the second image sensor is directly responsive to the image signal processor.
  • an apparatus in another particular embodiment, includes a first image sensor configured to generate a first data stream, a second image sensor configured to generate a second data stream, and a combiner configured to combine first data from the first data stream and second data from the second data stream to generate a frame.
  • the apparatus further includes an image signal processor configured to process the frame and to output a processed frame to a display.
  • Each of the first image sensor and the second image sensor is directly responsive to the image signal processor.
  • a method in another particular embodiment, includes receiving first image data of an image from a first image sensor, receiving second image data of an image from a second image sensor, and synchronizing line by line exposure of the first image sensor and the second image sensor during image data acquisition.
  • the first image sensor and the second image sensor are independent of each other.
  • the synchronizing may be line by line and may be frame by frame.
  • an apparatus in another particular embodiment, includes a memory buffer.
  • the memory buffer includes a section to align incoming streams in a deterministic order through the streaming of each frame and a programmable gap section between streams.
  • a method in another particular embodiment, includes receiving rows of image data at an image processor having an input for a single camera. Each row of the image data includes data from a row of a first image captured by a first camera and data from a row of a second image captured by a second camera. The method also includes generating an output having a three dimensional (3D) image format or a 3D video format. The output corresponds to the first image and the second image.
  • 3D three dimensional
  • an apparatus in another particular embodiment, includes an image processor having an input for a single camera.
  • the apparatus also includes a combiner configured to send rows of image data to the image processor.
  • Each row of the image data includes first data from a row of a first image captured by a first camera and second data from a row of a second image captured by a second camera.
  • the image processor is configured to generate an output having either a three dimensional (3D) image format or a 3D video format. The output corresponds to the first image and the second image.
  • One particular advantage provided by at least one of the disclosed embodiments is that a single image signal processor may be used to synchronize and control image data from multiple image sensors. Another particular advantage is that having gaps between streams offers the flexibility of processing the combined stream in an image signal processor as a single frame, and avoids contamination of streams by subsequent block-based processing (i.e., if the gap is equal with the biggest block-based processing contamination of streams is avoided).
  • FIG. 1 is a block diagram of a particular illustrative embodiment of an image processing system to combine data from multiple image sensors;
  • FIG. 2 is a block diagram of a second illustrative embodiment of an image processing system to combine data from multiple image sensors;
  • FIG. 3 is a block diagram of a third illustrative embodiment of an image processing system to combine data from multiple image sensors;
  • FIG. 4 is a block diagram of a particular illustrative embodiment of a selected portion of the image processing system of FIG. 2 , where a first image sensor and a second image sensor receive common control data;
  • FIG. 5 is a block diagram of a fourth illustrative embodiment of an image processing system to combine data from multiple image sensors
  • FIG. 6 is a diagrammatic representation of a first embodiment of a first data stream at an output of a first image sensor and a second data stream at an output of a second image sensor being combined to form a synchronized data line;
  • FIG. 7 is a diagrammatic representation of a second embodiment of a first data stream at an output of a first image sensor and a second data stream at an output of a second image sensor being combined to form a synchronized data line;
  • FIG. 8 is a diagrammatic representation of a first embodiment of phase diagram illustrating a two line phase difference between a first data stream from a first sensor and a second data stream from a second sensor;
  • FIG. 9 is a diagrammatic representation of a second embodiment of phase diagram illustrating a one line phase difference between a first data stream from a first sensor and a second data stream from a second sensor;
  • FIG. 10 is a diagrammatic representation illustrating pixel data of each of a plurality of sensors and illustrating synchronized data lines
  • FIG. 11 is a timing diagram illustrating frame valid and line valid signal timing for multiple sensors
  • FIG. 12 is a diagrammatic representation of a third embodiment of phase diagram illustrating a three line phase difference between a first data stream from a first sensor and a second data stream from a second sensor;
  • FIG. 13 is a block diagram of a particular illustrative embodiment of an image processing system to combine data from multiple image sensors to produce a three dimensional image format;
  • FIG. 14 is a diagrammatic representation illustrating various embodiments of mobile devices having image processing systems to combine data from multiple image sensors
  • FIG. 15 is a diagrammatic representation illustrating an example of images that are captured by an array of cameras including overlap between images
  • FIG. 16 is a diagrammatic representation illustrating an example of images that are captured by an array of cameras including overlap between images where each image may have its own shifting component and rotation component;
  • FIG. 17 is a block diagram illustrating a particular embodiment of an array of cameras and electrical connections associated with the array of cameras;
  • FIG. 18 is a block diagram of a first particular illustrative embodiment of a camera array processing system
  • FIG. 19 is a block diagram of a first particular illustrative embodiment of a camera array processing system
  • FIG. 20 is a diagrammatic representation illustrating a camera system that includes a main lens and multiple cameras arranged in an array
  • FIG. 21 is a diagrammatic representation illustrating a multiple camera module in an automobile
  • FIG. 22 is a flow diagram of a particular illustrative embodiment of a method of combining data from multiple sensors into a synchronized data line;
  • FIG. 23 is a flow diagram of a particular illustrative embodiment of a method of providing a common control signal to multiple image sensors and providing a synchronized data line to an image processor via a single camera input of the image processor;
  • FIG. 24 is a flow diagram of a particular illustrative embodiment of a method of providing a common control signal to multiple image sensors and receiving synchronized data output from each of the multiple image sensors;
  • FIG. 25 is a flow diagram of a particular illustrative embodiment of a method of receiving a common control signal at multiple image sensors and generating synchronized data output from each of the multiple image sensors;
  • FIG. 26 is a flow diagram of a particular illustrative embodiment of a method of combining data from multiple sensors at an image signal processor having an input for a single camera;
  • FIG. 27 is a flow diagram of a particular illustrative embodiment of a method of combining data from multiple sensors into a frame
  • FIG. 28 is a flow diagram of a particular illustrative embodiment of a method of synchronizing line by line exposure of a first image sensor and a second image sensor;
  • FIG. 29 is a flow diagram of a first illustrative embodiment of a method of combining data from multiple sensors to generate three dimensional image data
  • FIG. 30 is a flow diagram of a second illustrative embodiment of a method of combining data from multiple sensors to generate three dimensional image data
  • FIG. 31 is a flow diagram of a particular illustrative embodiment of a method of synchronizing line by line exposure of a first image sensor and a second image sensor;
  • FIG. 32 is a flow diagram of a particular illustrative embodiment of a method of combining data from multiple sensors to generate three dimensional image data from a synchronized data line;
  • FIG. 33 is a block diagram of a particular illustrative embodiment of an image processing system to combine data from multiple image sensors;
  • FIG. 34 is a block diagram of a first illustrative embodiment of a wireless device including an image processing system to combine data from multiple image sensors;
  • FIG. 35 is a block diagram of a second illustrative embodiment of a wireless device including an image processing system to combine data from multiple image sensors.
  • the image processing system 100 includes a multiple camera module 102 , a sensor module 104 , and a single camera chip module 106 .
  • the sensor module 104 may include a plurality of sensors, such as sensors 202 and 204 of FIG. 2 and FIG. 3 , where each of the plurality of sensors is configured to generate a data stream that includes data lines of an image.
  • the single camera module 106 may include an image processor having a single camera input, such as image processor 208 of FIG. 2 and FIG. 3 .
  • the image processing system 200 includes a first sensor 202 and a second sensor 204 .
  • the image processing system 200 further includes a combiner 206 , an image signal processor or video front end 208 , and a sensor synchronizer 230 .
  • the image signal processor 208 may be coupled to a display device (not shown).
  • the combiner 206 includes one or more line buffers 216 .
  • the image processing system 200 may be integrated in at least one semiconductor die.
  • the first sensor 202 is configured to generate a first data stream, illustrated as a first image data stream 212 .
  • the first image data stream 212 includes a first data line 222 .
  • the second sensor 204 is configured to generate a second data stream, illustrated as a second image data stream 214 .
  • the second image data stream 214 includes a second data line 224 .
  • the first and second sensors 202 , 204 may be substantially similar image sensors that are independent of each other and that receive a common control signal 234 from the sensor synchronizer 230 .
  • the sensor synchronizer 230 is configured to receive a control/data signal 232 and to output the common control signal 234 to the first and second sensors 202 , 204 , enabling the first and second sensors 202 , 204 to generate closely aligned data streams 212 , 214 .
  • the data streams 212 , 214 may have substantially the same timing characteristics, such as frequency and phase.
  • the control/data signal 232 may be received from the image signal processor 208 .
  • the combiner 206 is responsive to the first image data stream 212 and the second image data stream 214 .
  • the combiner 206 is configured to combine data from the first image data stream 212 and data from the second image data stream 214 within the line buffer 216 .
  • the line buffer 216 is configured to align first data, such as the first data line 222 from the first sensor 202 , and second data, such as the second data line 224 from the second sensor 204 .
  • the combiner 206 is responsive to data stored within the line buffer 216 and provides line data 218 to the image signal processor 208 .
  • the line data 218 may include a plurality of rows, where each row is a combination of corresponding rows from each sensor 202 , 204 , such as described with respect to FIG. 6 .
  • the image signal processor 208 is configured to process the line data 218 and to generate processed line data 240 .
  • the processed line data 240 may be provided as processed frame data. While two sensors have been shown, it should be understood that other embodiments may include more than two sensors.
  • FIG. 3 depicts an embodiment 300 that includes more than two sensors.
  • An Nth sensor 305 is configured to generate an Nth data stream, illustrated as an Nth image data stream 315 .
  • the Nth image data stream 315 includes an Nth data line 325 .
  • the Nth sensor 305 may be substantially similar to the first and second image sensors 202 , 204 and may receive the common control signal 234 from the sensor synchronizer 230 , enabling the first, second, and Nth sensors 202 , 204 , 305 to generate closely aligned data streams 212 , 214 , 315 .
  • the data streams 212 , 214 , 315 may have substantially the same timing characteristics, such as frequency and phase.
  • the combiner 206 is responsive to the first image data stream 212 , the second image data stream 214 , and the Nth image data stream 315 .
  • the combiner 206 is configured to combine data from the first image data stream 212 , the second image data stream 214 , and the Nth image data stream 315 within the line buffer 216 .
  • the line buffer 216 may be dimensioned for a worst case of misalignment (i.e., if the synchronization misalignment is three lines then the line buffer 212 should be sized to buffer at least six lines).
  • the combined data may be efficiently processed using a single image signal processor.
  • overall image system cost and complexity may be reduced compared to multiple processor systems (e.g., a processor assigned to each sensor).
  • the portion 400 of the image processing system includes the first sensor 202 , the second sensor 204 , and the sensor synchronizer 230 .
  • the first sensor 202 and the second sensor 204 are identical sensors or nearly identical sensors that receive the same start-up or reset signals and the same clock input from the sensor synchronizer 230 .
  • the first sensor 202 and the second sensor 204 may each receive common control data/signals from the sensor synchronizer 230 .
  • control data/signals may include a control clock signal 404 , a control data signal 406 , a camera clock signal 408 , and a camera reset signal 410 .
  • the control data/signals 404 - 410 may be formed and transmitted via an interface compliant with a two wire inter-chip communication protocol, such as an Inter-Integrated Circuit (I2C) multi-master serial computer bus.
  • I2C Inter-Integrated Circuit
  • control data/signals 404 - 410 may be formed and transmitted according to an interface compliant with a specification of a serial interface between a digital camera module and mobile phone engine, such as a Camera Serial Interface (CSI), an interface between a peripheral device (camera) and a host processor (baseband, application engine) such as a Camera Serial Interface 2 (CSI-2), a parallel interface between a digital camera module and a mobile phone engine such as a Camera Parallel Interface (CPI), or other control interfaces.
  • CSI Camera Serial Interface
  • CSI-2 Camera Serial Interface 2
  • CPI Camera Parallel Interface
  • the first sensor 202 may be configured to send first timing data 420 and first sensor image data 422 to the combiner 206 as illustrated in the system of FIG. 2 or FIG. 5 .
  • the second sensor 204 may be configured to send second timing data 430 and second sensor image data 432 to the combiner 206 of FIG. 2 or FIG. 5 .
  • the first sensor 202 and the second sensor 204 each operate in identical or nearly identical conditions from a timing standpoint.
  • the first and second sensors 202 , 204 each receive the same control clock signal 404 , the same control data signal 406 , the same camera clock signal 408 , and the same camera reset signal 410 .
  • the first and second sensors 202 , 204 are identical or nearly identical, they operate substantially similarly under the same timing conditions.
  • data output from the first sensor 202 has substantially the same frequency and phase as data output from the second sensor 204 .
  • a phase difference between data output from the first sensor 202 and the second sensor 204 may be less than a single horizontal line of phase difference, enabling a single image signal processor to be used to synchronize and control image data from the two image sensors 202 , 204 .
  • the system 500 includes the first image sensor 202 , the second image sensor 204 , the combiner 206 , the sensor synchronizer 230 , and the image signal processor 208 .
  • the system 500 further includes a register interface 510 and a clock management device 512 .
  • the register interface 510 may be within the sensor synchronizer 230 .
  • the register interface 510 may be a standalone module.
  • the system 500 may further include a thin output formatter 506 (shown in shadow) and a transport packer and formatter 508 (shown in shadow).
  • the combiner 206 is configured to receive the first timing data 420 and the first sensor image data 422 from the first sensor 202 .
  • the combiner 206 is also configured to receive the second timing data 430 and the second sensor image data 432 from the second sensor 204 .
  • the combiner 206 is further configured to receive a clock signal 526 from the clock management device 512 .
  • the combiner 206 uses the first timing data 420 , the first sensor image data 422 , the second timing data 430 , and the second sensor image data 432 to generate a synchronized data line which is provided to the image signal processor 208 .
  • the image signal processor 208 processes the synchronized data line to create processed data line data.
  • the processed data line data may be provided to another component, such as to a display device.
  • image data from multiple sensors may be combined, processed and rendered for display at a display device.
  • the first timing data 420 may be associated with a first pixel clock
  • the first sensor image data 422 may be associated with a first pixel size
  • the second timing data 430 may be associated with a second pixel clock
  • the second sensor image data 432 may be associated with a second pixel size.
  • the size of the single image line may be substantially double that of the first line of the first image data or the corresponding line of the second image data (e.g., double that of the first pixel size or the second pixel size), and the rate of pixel clock of the combined single image line may be substantially double the rate of the first pixel clock or the second pixel clock (e.g., may have a clock frequency that is double the first pixel clock frequency or the second pixel clock frequency).
  • the generated synchronized data line is sent to the image signal processor 208 via a combiner timing data signal 528 and a combiner image data signal 530 .
  • the synchronized data line that is generated by the combiner 206 may be provided to the thin output formatter 506 to create formatted data which is provided to the transport packer and formatter 508 prior to being provided to the image signal processor 208 .
  • the thin output formatter 506 receives the combiner timing data signal 528 and the combiner image data signal 530 to create formatted data.
  • the formatted data may include output formatter timing data signal 536 , output formatter image data signal 538 , output formatter stats data signal 540 , output formatter start data signal 542 , and output formatter valid data signal 544 .
  • the transport packer and formatter 508 receives the formatted data 536 - 544 from the thin output formatter 506 and generates a transport data stream including a transport timing data signal 546 and a transport image data signal 548 .
  • the register interface 510 may be coupled to the image signal processor 208 and coupled to the clock management device 512 .
  • the register interface 510 may receive a clock signal 527 from the clock management device 512 and may be coupled to a register bus 572 .
  • the clock management device 512 is configured to receive the second timing data signal 430 and to output the clock signal 526 .
  • the clock signal 526 is substantially double the frequency of the second timing data signal 430 to enable the combiner 206 to maintain a frame processing rate while combining concurrent data from multiple sensors.
  • FIG. 6 a diagrammatic representation of a particular embodiment of a first data stream at an output of a first image sensor and a second data stream at an output of a second image sensor being combined to form a synchronized data line is depicted and generally designated 600 .
  • a first sensor such as the first sensor 202 of FIG. 2 , generates a first data stream 602 that corresponds to first image data of an image.
  • a second sensor such as the second sensor 204 of FIG. 2 , generates a second data stream 604 that corresponds to second image data of the image. Data from the first data stream 602 and data from the second data stream 604 are combined to form a data out data stream 606 .
  • the first data stream 602 includes data associated with a first line of the first image data of the image and the second data stream 604 includes data associated with a corresponding line of the second image data of the image.
  • the first data stream 602 includes line data 610 having a first line index value, line data 612 having a second line index value, line data 614 having a third line index value, and line data 616 having a fourth line index value.
  • the second data stream 604 includes corresponding line data to that of the first data stream, including corresponding line data 620 having the first line index value, corresponding line data 622 having the second line index value, corresponding line data 624 having the third line index value, and corresponding line data 626 having the fourth line index value.
  • the data out data stream 606 includes a combination of the first line of the first image data of the image and the corresponding line of the second image data of the image. As illustrated, the first data stream 602 and the second data stream 604 are interleaved to form the data out data stream 606 .
  • the data out data stream 606 includes combined line data 630 having the first line index value, combined line data 632 having the second line index value, and combined line data 634 having the third line index value.
  • the combined line data 630 includes the line data 610 and the corresponding line data 620 .
  • the combined line data 632 includes the line data 612 and the corresponding line data 622 .
  • the combined line data 634 includes the line data 614 and the corresponding line data 624 .
  • Each combined line 630 - 634 may be generated by combining corresponding lines within a line buffer, such as the line buffer 216 of FIG. 2 .
  • the data from the first data stream 602 is combined with the data from the second data stream 604 to generate a plurality of synchronized data lines that form a frame 660 .
  • the frame 660 may include a plurality of rows 642 , where each row corresponds to a line index value and stores a synchronized data line that includes a line of the first image data having the line index value and a corresponding line of the second image data having the line index value.
  • a first row of the frame 660 may include the combined line data 630
  • a second row of the frame 660 may include the combined line data 632
  • a third row of the frame 660 may include the combined line data 634 , etc.
  • Each synchronized image data line forms part of the frame 660 such that the data in the frame 660 is aligned.
  • the frame 660 is depicted with an order of the rows 642 matching a read order of the image data from the image sensors (i.e. combined data from the top line of the image sensors (line index 1 ) is in a top line of the frame 660 and combined data from a next line of the image sensors (line index 2 ) is in a next line of the frame 660 .
  • the rows of the frame 660 may not match a read order of the image data and may instead correspond to any other order of the image data.
  • a top row of the frame 660 may to line index 2 while a next row of the frame 660 may correspond to line index 1 .
  • the frame 660 may be programmable such that each of the rows 642 can be programmed to correspond to any of the line index values of the image data.
  • the first line 610 of the first image data is associated with a first pixel size (e.g., a number of pixels per line) and a first pixel clock
  • the corresponding line 620 of the second image data is associated with a second pixel size and a second pixel clock.
  • the first sensor and the second sensor generating the data streams 602 , 604 are configured to receive a common clock signal and a common reset signal.
  • the size of the single image line is substantially double that of the first line 610 of the first image data or the corresponding line 620 of the second image data
  • the pixel clock signal of the combined single image line e.g., a third pixel clock signal
  • the combined line data 630 may have an image size that is substantially double that of the line data 610 or double that of the corresponding line data 620 .
  • the pixel clock frequency of the combined line data 630 may have a frequency that is substantially double that of the first pixel clock signal associated with the line data 610 or double that of the second pixel clock signal associated with the corresponding line data 620 , such that the pixel clock frequency of the combined line data 630 may be associated with the third pixel clock signal having substantially double the frequency of that of the first pixel clock or the second pixel clock.
  • a synchronized line size may be substantially three times the sensor line size and a pixel clock rate may be substantially three times a pixel clock rate of the individual sensors.
  • a synchronized line size can be set as greater than or equal to a sum of the line sizes that are combined, and a pixel clock rate can be set so that the output line bandwidth is equal to or greater than the sum of the input bandwidth.
  • the frame 660 may be processed at an image signal processor, such as the image signal processor 208 of FIG. 2 , to generate a processed frame 650 .
  • the processed frame 650 includes a first section 652 including first image data from a first image sensor, such as the sensor 202 of FIG. 2 , a second section 654 including second image data from a second image sensor, such as the sensor 204 of FIG. 2 , and a gap section 656 .
  • the gap section 656 may include non-image data disposed between the first section 652 and the second section 654 .
  • the first section 652 includes a line of the first image data and the second section 654 includes a corresponding line of the second image data.
  • the gap section 656 may be used for edge filtering and may include a black gap that is approximately five pixels in width.
  • the gap section 656 may be added between lines and have a size equal to the size of an interpolation kernel or a size of a largest two-dimensional filter applied to the frame 650 by the image signal processor.
  • statistics for automatic exposure, automatic focus, and automatic white balance may be collected from either the first section 652 or the second section 654 , either of which may be a full image from one of the respective sensors. Therefore, the statistics for automatic exposure, automatic focus, and automatic white balance may be collected from half of the final image (e.g., the first section 652 ) and may be applied to both sensors since both sensors are receiving substantially identical timing information. As such, data output from multiple sensors has substantially the same frequency and phase such that synchronization may occur within one image line of image data of the image.
  • the frame 650 may be stored in a memory that is integrated in at least one semiconductor die.
  • the frame 650 may be stored in memory that is incorporated into a consumer electronic device, such as a set top box, a music player, a video player, an entertainment unit, a navigation device, a communications device, a personal digital assistant (PDA), a fixed location data unit, and a computer.
  • the electronic devices may utilize image processing methods, including 3D applications that process image data from multiple sensors.
  • FIG. 7 a diagrammatic representation of a second embodiment of a first data stream at an output of a first image sensor and a second data stream at an output of a second image sensor being combined to form a synchronized data line is depicted and generally designated as 700 .
  • a first sensor such as the first image sensor 202 of FIG. 2 , generates a first data stream 702 that corresponds to first image data of an image.
  • a second sensor such as the second image sensor 204 of FIG. 2 , generates a second data stream 704 that corresponds to second image data of the image. Data from the first data stream 702 and data from the second data stream 704 are combined to form a data out data stream 706 .
  • the first data stream 702 includes data associated with a first line of the first image data of the image and the second data stream 704 includes data associated with a corresponding line of the second image data of the image.
  • the first data stream 702 includes line data 710 having a first line index value, line data 712 having a second line index value, line data 714 having a third line index value, and line data 716 having a fourth line index value.
  • the second data stream 704 includes corresponding line data to that of the first data stream, including corresponding line data 720 having the first line index value, corresponding line data 722 having the second line index value, corresponding line data 724 having the third line index value, and corresponding line data 726 having the fourth line index value.
  • the data out data stream 706 includes a combination of the first line of the first image data of the image and the corresponding line of the second image data of the image. As illustrated, the first data stream 702 and the second data stream 704 are interleaved with a gap section 708 to form the data out data stream 706 .
  • the illustrated portion of the data out data stream 706 includes combined line data 730 having the first line index value, combined line data 732 having the second line index value, and combined line data 734 having the third line index value.
  • the combined line data 730 includes the line data 710 separated from the corresponding line data 720 by the gap section 708 .
  • the combined line data 732 includes the line data 712 separated from the corresponding line data 722 by the gap section 708 .
  • the combined line data 734 includes the line data 714 separated from the corresponding line data 724 by the gap section 708 .
  • Each combined line 730 - 734 may be generated by combining corresponding lines with the gap section 708 between the corresponding lines within a line buffer, such as the line buffer 216 of FIG. 2 .
  • the data from the first data stream 702 is combined with the data from the second data stream 704 to generate a plurality of synchronized data lines that form a frame 740 .
  • the frame 740 may include a plurality of rows 742 , where each row corresponds to a line index value and stores a line of the first image data having the line index value and stores a corresponding line of the second image data having the line index value.
  • a first row of the frame 740 may include the combined line data 730
  • a second row of the frame 740 may include the combined line data 732
  • a third row of the frame 740 may include the combined line data 734 , etc. such that the data in the frame 740 is aligned.
  • the first line 710 of the first image data is associated with a first pixel size (e.g., a number of pixels per line) and a first pixel clock
  • the corresponding line 720 of the second image data is associated with a second pixel size and a second pixel clock.
  • the first sensor and the second sensor generating the data streams 702 , 704 are configured to receive a common clock signal and a common reset signal.
  • the pixel clock signal of the combined single image line (e.g., a third pixel clock signal) has a clock rate that is approximately double that of the first pixel clock signal or the second pixel clock signal.
  • the combined line data 730 may have an image size that is approximately double that of the line data 710 or double that of the corresponding line data 720 .
  • the pixel clock frequency of the combined line data 730 may have a frequency that is approximately double that of the first pixel clock signal associated with the line data 710 or double that of the second pixel clock signal associated with the corresponding line data 720 , such that the pixel clock frequency of the combined line data 730 may be associated with the third pixel clock signal having approximately double the frequency of that of the first pixel clock or the second pixel clock.
  • Each gap section 708 may include non-image data.
  • the non-image data area in the frame 740 formed by the gap sections 708 may be used for edge filtering.
  • the gap section 708 may include a black gap that is approximately five pixels in width.
  • each gap section 708 has a size equal to the size of an interpolation kernel or a size of a largest two-dimensional filter applied to the frame 740 by an image processor, such as the image processor 208 of FIG. 2 .
  • the frame 740 may be processed by the image processor to generate a 3D image.
  • FIG. 8 a diagrammatic representation of a first embodiment of a phase diagram illustrating a two line phase difference between a first data stream from a first sensor and a second data stream from a second sensor is depicted and generally designated 800 .
  • a first sensor such as the first sensor 202 of FIG. 2 , generates a first data stream that includes first sensor first line data 802 , first sensor second line data 804 , and first sensor third line data 806 .
  • a second sensor such as the second sensor 204 of FIG. 2 , generates a second data stream that includes second sensor first line data 812 , second sensor second line data 814 , and second sensor third line data 816 .
  • first, second and third line data is illustrated.
  • any number of line data may be generated (e.g., 720 lines as illustrated in FIGS. 6 and 7 ).
  • the first sensor first line data 802 may be received by a combiner such as combiner 216 of FIG. 2 during a first line phase
  • the first sensor second line data 804 may be received during a second line phase
  • the first sensor third line data 806 and the second sensor first line data 812 may be received during a third line phase.
  • the combined line 820 includes a combination of the first line of the first image data of the image and the corresponding line of the second image data of the image. As illustrated, the first data stream and the second data stream are interleaved to form the combined line 820 .
  • the combined line 820 includes combined line data 822 having the first sensor first line data 802 and the second sensor first line data 812 , combined line data 824 having the first sensor second line data 804 and the second sensor second line data 814 , and combined line data 826 having the first sensor third line data 806 and the second sensor third line data 816 .
  • Each combined line 822 - 826 may be generated by combining corresponding lines within a line buffer, such as the line buffer 216 of FIG. 2 .
  • the line buffer may be configured to buffer at least a portion of a next data line (e.g. the first sensor second line data 804 ) that is received from the first image sensor before a synchronized data line (e.g. the combined line data 822 ) is generated.
  • a next data line e.g. the first sensor second line data 804
  • a synchronized data line e.g. the combined line data 822
  • data output from multiple sensors having a two line phase difference may be combined such that synchronization may occur within one image line of image data of the image.
  • FIG. 9 a diagrammatic representation of a first embodiment of a phase diagram illustrating a one line phase difference between a first data stream from a first sensor and a second data stream from a second sensor is depicted and generally designated 900 .
  • a first sensor such as the first sensor 202 of FIG. 2 , generates a first data stream that includes first sensor first line data 902 , first sensor second line data 904 , first sensor third line data 906 , and first sensor fourth line data 908 .
  • a second sensor such as the second sensor 204 of FIG. 2 , generates a second data stream that includes second sensor first line data 912 , second sensor second line data 914 , second sensor third line data 916 , and second sensor fourth line data 918 .
  • first, second, third, and fourth line data is illustrated.
  • any number of line data may be generated (e.g., 720 lines as illustrated in FIGS. 6 and 7 ).
  • the first sensor first line data 902 may be received by a combiner such as combiner 216 of FIG. 2 during a first line phase
  • the first sensor second line data 904 and the second sensor first line data 912 may be received during a second line phase.
  • the combined line 920 includes a combination of the first line of the first image data of the image and the corresponding line of the second image data of the image. As illustrated, the first data stream and the second data stream are interleaved to form the combined line 920 .
  • the combined line 920 includes combined line data 922 having the first sensor first line data 902 and the second sensor first line data 912 , combined line data 924 having the first sensor second line data 904 and the second sensor second line data 914 , combined line data 926 having the first sensor third line data 906 and the second sensor third line data 916 , and combined line data 928 having the first sensor fourth line data 908 and the second sensor fourth line data 918 .
  • Each combined line 922 - 926 may be generated by combining corresponding lines within a line buffer, such as the line buffer 216 of FIG. 2 .
  • a line buffer such as the line buffer 216 of FIG. 2 .
  • data output from multiple sensors having a one line phase difference may be combined such that synchronization may occur within one image line of image data of the image.
  • a diagrammatic representation illustrating red-green-blue (RGB) data for each of a plurality of sensors and illustrating synchronized data lines is depicted and generally designated 1000 .
  • a first sensor such as the first sensor 202 of FIG. 3 , generates a first data stream that includes first sensor first line data 1002 , first sensor second line data 1004 , and first sensor third line data 1006 .
  • a second sensor such as the second sensor 204 of FIG. 3 , generates a second data stream that includes second sensor first line data 1012 , second sensor second line data 1014 , and second sensor third line data 1016 .
  • a third sensor such as the Nth sensor 305 of FIG.
  • a third data stream that includes third sensor first line data 1022 , third sensor second line data 1024 , and third sensor third line data 1026 .
  • Data from the first data stream, data from the second data stream, and data from the third data stream are combined to form a combined line 1020 .
  • each of the first line data 1002 , 1012 , 1022 includes alternating red and green pixel values
  • each of the second line data 1004 , 1014 , 1024 includes alternating green and blue pixel values
  • each of the third line data 1006 , 1016 , 1026 includes alternating red and green pixel values according to a Bayer filter pattern.
  • the first data stream, the second data stream, and the third data stream are interleaved to form a combined line data stream 1020 .
  • the combined line data stream 1020 includes combined line data 1040 having the first sensor first line data 1002 , the second sensor first line data 1012 , and the third sensor first line data 1002 , combined line data 1050 having the first sensor second line data 1004 , the second sensor second line data 1014 , and the third sensor second line data 1024 , and combined line data 1060 having the first sensor third line data 1006 , the second sensor third line data 1016 , and the third sensor third line data 1026 .
  • Each combined line 1040 - 1060 may be generated by combining corresponding lines within a line buffer, such as the line buffer 216 of FIG. 3 .
  • Each combined line 1040 - 1060 may include raw pixel (e.g., R, G, and B) values read from its respective sensor line data to be demosaiced at an image signal processor, such as the image signal processor 208 of FIG. 3 .
  • an image signal processor such as the image signal processor 208 of FIG. 3
  • data output from multiple sensors may be combined such that synchronization may occur within one image line of image data of the image.
  • FIG. 10 illustrates raw pixel values as including RGB data according to a Bayer filter pattern
  • the line data from the sensors may not include Bayer RGB pixel values.
  • the sensors may instead provide: luma, blue-difference chroma, red-difference chroma (YCbCr) values; cyan, yellow, green, and magenta (CYGM) values; red, green, blue, and emerald (RGBE) values; red, green, blue, and white (RGBW) values; or any other type of values, as illustrative, non-limiting examples.
  • one or more of the sensors may implement panchromatic cells, microlenses over groups of photoreceptors, vertical color filters, or any other sensor technology capable of line-by-line readout of raw image data.
  • the signals include a frame valid (FV) signal 1102 and a line valid (LV) signal 1104 of a first sensor, an FV 1106 and an LV 1108 of a second sensor, and an FV 1110 and an LV 1112 of a third sensor.
  • the first sensor, the second sensor, and a third sensor may be the first, second, and third sensors 202 , 204 , 305 of FIG. 3 .
  • a combined frame valid signal 1114 is also illustrated in conjunction with a combined line valid/data signal 1116 and a line load (LL) signal 1118 .
  • the signals 1114 - 1118 correspond to signaling related to one or more synchronized data lines of a frame, such as the frame 660 of FIG. 6 generated by a combiner, and the signals 1102 - 1112 correspond to signaling received at the combiner.
  • a first line data 1120 , a second line data 1122 , and a third line data 1124 are received from the first sensor
  • a first line data 1130 , a second line data 1132 , and a third line data 1134 are received from the second sensor
  • a first line data 1140 , a second line data 1142 , and a third line data 1144 are received from the third sensor.
  • the first line data 1130 is received from the second sensor prior to the first line data 1120 and the first line data 1140 .
  • a phase difference between receipt of the first line data 1130 and the second line data 1120 is illustrated as a first phase difference 1180 .
  • the first line data 1120 of the first sensor is received prior to the first line data 1140 of the third sensor, illustrated as a second phase difference 1182 .
  • the line data from each of the sensors may follow a rising edge of a corresponding frame valid signal, indicating that data received via an image data line is valid line data from each particular sensor.
  • the combined frame valid line 1114 remains low, indicating non-valid data, until after each of the first line data 1120 , 1130 , and 1140 have been received, such as at the combiner 206 of FIG.
  • the combined frame valid signal 1114 rises to indicate valid data on the line valid signal 1116 .
  • a first synchronized data line 1150 is generated in conjunction with a valid signal being asserted on the LL line 1118 .
  • the combined frame valid signal 1114 remains in a valid state while the LL signal 1118 returns to a non-valid state, after which the LL signal 1118 returns to a valid state upon generation of a second synchronized data line 1160 , which is followed by generation of a third synchronized data line 1170 .
  • FIG. 12 a diagrammatic representation of a first embodiment of a phase diagram illustrating a three line phase difference between a first data stream from a first sensor, a second data stream from a second sensor, and a third data stream from a third sensor is depicted and generally designated 1200 .
  • a first sensor such as the first sensor 202 of FIG. 3 , generates a first data stream that includes first sensor first line data 1202 , first sensor second line data 1204 , first sensor third line data 1206 , and first sensor fourth line data 1208 .
  • a second sensor such as the second sensor 204 of FIG.
  • a third sensor such as the Nth sensor 305 of FIG. 3 , generates a third data stream that includes third sensor first line data 1222 , third sensor second line data 1224 , third sensor third line data 1226 , and third sensor fourth line data 1228 .
  • Data from the first data stream, data from the second data stream, and data from the third data stream are combined to form a combined line 1220 .
  • first, second, third, and fourth line data is illustrated.
  • any number of line data may be generated (e.g., 720 lines as illustrated in FIGS. 6 and 7 ).
  • the third sensor first line data 1222 may be received by a combiner such as combiner 216 of FIG. 2 during a first line phase
  • the first sensor first line data 1202 and the third sensor second line data 1224 may be received during a second line phase
  • the first sensor second line data 1204 and the third sensor third line data 1226 may be received during a third line phase
  • the first sensor third line data 1206 , the second sensor first line data 1212 , and the third sensor fourth line data 1228 may be received during a fourth line phase.
  • the combined line 1220 includes a combination of the first line of the first image data of the image and the corresponding line of the second image data and the third image data of the image. As illustrated, the first data stream, the second data stream, and the third data stream are interleaved to form the combined line 1220 .
  • the combined line 1220 includes combined line data 1232 having the first sensor first line data 1202 , the second sensor first line data 1212 , and the third sensor first line data 1222 , combined line data 1234 having the first sensor second line data 1204 , the second sensor second line data 1214 , and the third sensor second line data 1224 , and combined line data 1236 having the first sensor third line data 1206 , the second sensor third line data 1216 , and the third sensor third line data 1226 .
  • Each combined line 1232 - 1236 may be generated by combining corresponding lines within a line buffer, such as the line buffer 216 of FIG. 3 . As such, data output from multiple sensors having a three line phase difference may be combined such that synchronization may occur within one image line of image data of the image.
  • the image processing system 1300 includes a first image sensor 1302 and a second image sensor 1304 .
  • the image processing system 1300 further includes a combiner 1306 and an image processor 1308 .
  • the image processor 1308 is coupled to a display device 1310 .
  • the combiner 1306 includes at least one line buffer 1312 .
  • the image processing system 1300 may be integrated in at least one semiconductor die.
  • the first image sensor 1302 is configured to generate a first data stream, illustrated as a first image data stream 1314
  • the second image sensor 1304 is configured to generate a second data stream, illustrated as a second image data stream 1316 .
  • the first image data stream 1314 may be asynchronous to the second image data stream 1316 .
  • the first and second image sensors 1302 , 1304 may be substantially similar image sensors that are independent of each other and that may receive a common control signal from a processor (e.g., the combiner 1306 or the image processor 1308 ) to generate closely aligned image data streams 1314 , 1316 .
  • the image data streams 1314 , 1316 may have substantially the same timing characteristics, such as frequency and phase.
  • each of the image sensors 1302 , 1304 may be directly responsive to, and controlled by, the single processor. While two image sensors 1302 , 1304 have been shown, it should be understood that more than two image sensors may be used with the image processing system 1300 .
  • the combiner 1306 is responsive to the first image data stream 1314 and the second image data stream 1316 .
  • the combiner 1306 is configured to combine data from the first image data stream 1314 and data from the second image data stream 1316 within the line buffer 1312 .
  • the line buffer 1312 is configured to align first data from the first image sensor 1302 and second data from the second image sensor 1304 .
  • the combiner 1306 is responsive to data stored within the line buffer 1312 and provides frame data 1318 to the image processor 1308 .
  • the frame data 1318 may include a plurality of rows of image data, where each row is a combination of corresponding rows from each sensor 1302 , 1304 , such as described with respect to FIG. 3 .
  • the image processor 1308 is configured to process the frame data 1318 and to output processed frame data 1320 to the display device 1310 .
  • the processed frame data 1320 may have a 3D image format or a 3D video format.
  • the display device 1310 renders and displays image data in response to receiving the processed frame data 1320 .
  • image data from multiple image sensors may be combined, processed and then rendered for display at the display device 1310 .
  • the display device 1310 may be decoupled from the image processor 1308 to not be directly responsive to the image processor 1308 .
  • the display device 1310 may be a separate device from the image processor 1308 .
  • the line buffer 1312 may be dimensioned for a worst case of misalignment (e.g., if the synchronization misalignment is three lines, then the line buffer 1312 should be sized to store at least six lines).
  • the combined data may be efficiently processed using a single image processor 1308 .
  • overall image system cost and complexity may be reduced compared to multiple processor systems (e.g., a processor assigned to each sensor).
  • Embodiments may be configured to provide 3D/stereoscopic images and/or video data.
  • the first image sensor 1302 and the second image sensor 1304 may be positioned side by side so as to provide left/right (stereoscopic) images.
  • the signal provided by the combiner 1306 is received and may be processed by the image processor 1308 to produce 3D images.
  • a user command may allow the image processor 1308 to receive and process data from only a single sensor (i.e., the first image sensor 1302 or the second image sensor 1304 ) to produce two dimensional (2D) images in lieu of producing 3D images.
  • An image processor having an input for a single camera is able to process data that can be used for 3D processing by using combined data from two cameras provided by the combiner 1306 .
  • the image processor 1308 may receive image data from the combiner 1306 or from a memory that stores image data from the combiner 1306 .
  • the image processor 1308 processes received image data as 2D image/video data so that subsequent processing by the image processor 1308 provides a 3D stereoscopic image/video stream based on the processed data from the image processor 1308 .
  • the image processor 1308 may be configured to directly provide a 3D stereoscopic image/video stream based on received image data.
  • a 3D capture system comprises the combiner 1306 implemented as a first integrated circuit and the image processor 1308 implemented as a second integrated circuit.
  • the first and second integrated circuits may be connected, for example, by one or more of a serial, parallel, or I2C bus.
  • the system 1300 may be implemented at a reduced cost as compared to a system that use a separate processor for each camera or that uses a processor having multiple camera inputs.
  • the mobile device includes an image processing system having an array of two adjacent cameras.
  • the mobile device includes an image processing system having an array of three cameras arranged in an in-line configuration. Alternatively, any number of cameras may be arranged in an in-line configuration.
  • the mobile device includes an image processing system having a three-by-three array of cameras.
  • the mobile device includes an image processing system having a five-by-five array of cameras. Alternatively, any number of cameras may be arranged in a two-dimensional array.
  • an example of an array of images that are captured by multiple cameras or image sensors is illustrated and generally designated 1500 .
  • an image captured by one camera may overlap with images captured by other neighboring cameras. Image overlap may be useful in combining the images captured by each of the cameras into a single image.
  • the array 1500 corresponds to a four-by-four array of cameras or image sensors.
  • any number of cameras may be arranged in a two-dimensional array (e.g., a five-by-five array as illustrated in FIGS. 18 and 19 ).
  • each camera captures a single camera image of the array 1500 .
  • the array 1500 is a four-by-four array.
  • the array 1500 includes a first row 1502 , a second row 1504 , a third row 1506 , and a fourth row 1508 .
  • the array 1500 includes a first column 1510 , a second column 1512 , a third column 1514 , and a fourth column 1516 .
  • a single camera image 1518 may be captured by a camera corresponding to the first row 1502 and the fourth column 1516 .
  • the single camera image 1518 may overlap with camera images captured by other cameras of neighboring cells.
  • a camera of a neighboring cell may include a camera corresponding to the first row 1502 and the third column 1514 , a camera corresponding to the second row 1504 and the third column 1514 , or a camera corresponding to the second row 1504 and the fourth column 1516 .
  • a single camera image overlap 1520 may be associated with the single camera image 1508 captured by a camera corresponding to the first row 1502 and the fourth column 1516 .
  • FIG. 15 illustrates a theoretical case of absolute alignment of each camera of each of the rows 1502 - 1508 and each camera of each of the columns 1510 - 1516 .
  • the image overlap with an image captured by a camera corresponding to a neighboring row may be the same as the image overlap with an image captured by a camera corresponding to a neighboring column (e.g., the third column 1514 ).
  • An individual image may be captured with a particular horizontal resolution (“H-res”).
  • a horizontal resolution 1522 may be associated with the single camera image 1518 captured by the camera corresponding to the first row 1502 and the fourth column 1516 .
  • FIG. 15 illustrates a theoretical case of image overlap where each camera has the same horizontal resolution 1522 .
  • an overall horizontal resolution 1524 i.e., a number of pixels in each of the rows 1502 - 1508
  • n number of cells in a row
  • the overall horizontal resolution 1524 may account for image overlap.
  • the overall horizontal resolution 1524 may be calculated as H_res*n ⁇ overlap*(n ⁇ 2), where overlap indicates a number of overlapping pixels of adjacent images. A similar calculation may be performed to determine an overall vertical resolution.
  • FIG. 15 illustrates a theoretical case of absolute alignment of each camera with the same image overlap.
  • the image overlap between images captured by the individual cameras may allow each of the individual images to be “stitched together” into a single image.
  • an example of an array of images that are captured by multiple cameras or image sensors is illustrated and generally designated 1600 .
  • an image captured by one camera may overlap with images captured by other neighboring cameras. Image overlap may be useful in combining the images captured by each of the cameras into a single image.
  • the array 1600 corresponds to a four-by-four array of cameras or image sensors. Alternatively, any number of cameras may be arranged in a two-dimensional array.
  • FIG. 16 illustrates that, due to mechanical constraints, it may not be feasible to achieve absolute alignment of cameras in a row or column (as illustrated in FIG. 15 ).
  • Each image of the array 1600 may have its own rotation 1602 , shift 1604 , and tilt (not shown).
  • One or more positioning tolerances 1606 may be associated with each image.
  • the positioning tolerances 1606 may include a rotation tolerance, a shift tolerance, a tilt tolerance, or a combination thereof.
  • Image overlap may be useful in combining the images captured by each of the cameras into a single image.
  • FIG. 16 illustrates that the image overlap that is used in combining the images may account for the rotation 1602 , shift 1604 , and tilt of each image that results from mechanical constraints associating with building a device. Once the device is built, the image overlap may be known and stable. As such, the image overlap may be quantified and may be corrected in a later stage.
  • FIG. 17 a particular embodiment of an array of cameras and electrical connections associated with the array is illustrated and generally designated 1700 .
  • FIG. 17 illustrates that each camera of the array 1700 has a first type of interface (i.e., a data interface 1702 ) and a second type of interface (i.e., a control interface 1704 ).
  • the array 1700 includes a three-by-three array of cameras.
  • the array 1700 may include cameras arranged in a four-by-four array (e.g., the arrays 1500 , 1600 of FIGS. 15 and 16 ) or any other number of cameras arranged in a two-dimensional array.
  • the data interface 1702 may include a serial data bus (e.g., a Mobile Industry Processor Interface or a Standard Mobile Imaging Architecture interface).
  • the data interface 1702 in FIG. 17 is associated with a first row 1706 of the array 1700 , a second row 1708 of the array 1700 , and a third row 1710 of the array 1700 .
  • Lines associated with the data interface 1702 may be used to collect data from cameras in each of the rows 1706 - 1710 to be processed in parallel.
  • four wires may be needed (e.g., differential data and clock).
  • each of the rows 1706 - 1710 includes a camera in a first column 1712 of the array 1700 , a camera in a second column 1714 of the array 1700 , and a camera in a third column 1716 of the array 1700 .
  • the data interface 1702 may be used to collect data from nine cameras to be processed in parallel.
  • the control interface 1704 may include lines that are used to synchronize all cameras in the array 1700 .
  • control interface lines may be associated with clock, reset, and I2C communication.
  • the control interface 1704 may be used to synchronize the nine cameras of the array 1700 .
  • a particular embodiment of a camera array processing system is illustrated and generally designated 1800 .
  • All cameras in an array 1802 may be responsive to common control signals, aligned, and processed prior to the resulting image data being combined into a final image.
  • the array 1802 includes a five-by-five array of cameras.
  • the array 1802 may include any other number of cameras arranged in a two-dimensional array (e.g., the three-by-three array 1700 of FIG. 17 ).
  • All cameras in the array 1802 may be synchronized using a method of concurrent image sensor support using a single ISP pipeline. Further, each row of cameras may be aligned using an alignment method. That is, one row of images may be collected, aligned in the same order and sent for processing as a single line with a size n*line, where n is the number of cameras in a row and line is the horizontal size (i.e., “H_res” as described in FIG. 15 ) of one camera.
  • the five-by-five array 1802 illustrated in FIG. 18 includes a first row 1804 , a second row 1806 , a third row 1808 , a fourth row 1810 , and a fifth row 1812 .
  • the five-by-five array 1802 includes a first column 1814 , a second column 1816 , a third column 1818 , a fourth column 1820 , and a fifth column 1822 .
  • Each of the rows 1804 - 1812 may be processed in parallel for color at a VFE component, and each of the rows 1804 - 1812 may be processed at a graphics processor or graphics processing unit (GPU) component to align and rectify each individual image in a row. After rectification and alignment, GPU processing may be performed to combine all of the rows 1804 - 1812 together, resulting in a final image.
  • GPU graphics processing unit
  • the first row 1804 may be associated with a first alignment block 1824
  • the second row 1806 may be associated with a second alignment block 1826
  • the third row 1808 may be associated with a third alignment block 1828
  • the fourth row 1810 may be associated with a fourth alignment block 1830
  • the fifth row 1812 may be associated with a fifth alignment block 1832 .
  • the first alignment block 1824 may be adapted to collect image data lines from each camera in the first row 1804 (i.e., five cameras in the five columns 1814 - 1822 of the array 1802 ).
  • the first alignment block 1824 may be adapted to align the image data lines in the same order and send the image data lines for processing as a single line.
  • the first alignment block 1824 may be adapted to send the image data lines for processing as a single line to a first VFE component 1834 to be processed for color such as described with respect to the combiner of 206 of FIGS. 2 , 3 , and 5 .
  • a size of the single line may be determined based on the number of cameras (i.e., five cameras) and the horizontal size (i.e., “H_res” as described in FIG. 15 ) of each camera in the first row 1804 .
  • the second alignment block 1826 may be adapted to collect images from each camera in the second row 1806 , to align the images in a same order, and to send the images for processing as a single line to a second VFE component 1836 .
  • the third alignment block 1828 may be adapted to collect an image data line from each camera in the third row 1808 , to align the image data lines in a same order, and to send the image data lines for processing as a single line to a third VFE component 1838 .
  • the fourth alignment block 1830 may be adapted to collect image data lines from each camera in the fourth row 1810 , to align the image data lines in a same order, and to send the image data lines for processing as a single line to a fourth VFE component 1840 .
  • the fifth alignment block 1832 may be adapted to collect image data lines from each camera in the fifth row 1812 , to align the image data lines in a same order, and to send the image data lines for processing as a single line to a fifth VFE component 1842 .
  • a control synchronization block 1844 may be used to synchronize each of the cameras of the array 1802 (i.e., twenty five cameras in the case of the illustrative five-by-five array 1802 of FIG. 18 ) in a manner similar to the sensor synchronizer 230 .
  • the control synchronization block 1834 may implement the control interface 1704 of FIG. 17 .
  • the control synchronization block 1844 may be communicatively coupled to each of the cameras of the array 1802 and to each of the VFE components 1834 - 1842 . Synchronization of all cameras in the array 1802 may allow for usage of a rolling shutter on a high resolution.
  • the rolling shutter effect may be diminished (with the size of the array). For example, in the five-by-five array 1802 of FIG. 18 , synchronization of the twenty five cameras may diminish the rolling shutter effect associated with a Complementary Metal Oxide Semiconductor (CMOS) camera.
  • CMOS Complementary Metal Oxide Semiconductor
  • the first VFE component 1834 may be communicatively coupled to a first GPU component 1846 to align and rectify each individual image in the first row 1804 (i.e., five images captured by the cameras in the five columns 1814 - 1822 ).
  • the second VFE component 1836 may be communicatively coupled to a second GPU component 1848 to align and rectify each individual image in the second row 1806 .
  • the third VFE component 1838 may be communicatively coupled to a third GPU component 1850 to align and rectify each individual image in the third row 1808 .
  • the fourth VFE component 1840 may be communicatively coupled to a fourth GPU component 1852 to align and rectify each individual image in the fourth row 1810 .
  • the fifth VFE component 1842 may be communicatively coupled to a fifth GPU component 1854 to align and rectify each individual image in the fifth row 1812 .
  • Each of the GPU components 1846 - 1854 may be communicatively coupled to a GPU processing component 1856 that is adapted to combine all of the rows 1804 - 1812 together, resulting in a final image.
  • each of the alignment blocks 1824 - 1832 is associated with its own VFE component and its own GPU rectification and alignment component.
  • FIG. 18 illustrates that each of the rows 1804 - 1812 may be processed in parallel for color using separate VFE components, and each of the rows 1804 - 1812 may be processed using separate GPU components to align and rectify each individual image in a particular row.
  • each of the alignment blocks 1824 - 1832 may be associated with a single VFE component and a single GPU rectification and alignment component (see FIG. 19 ).
  • FIG. 19 a particular embodiment of a camera array processing system is illustrated and generally designated 1900 .
  • All cameras in an array 1902 may be synchronized, aligned, and processed prior to being combined into a final image.
  • the array 1902 includes a five-by-five array of cameras.
  • the array 1902 may include any other number of cameras arranged in a two-dimensional array.
  • FIG. 19 illustrates that a single VFE component and a single GPU rectification and alignment component may be used to process all rows of the array 1902 , rather than the multiple VFE and GPU rectification and alignment components illustrated in FIG. 18 .
  • the five-by-five array 1902 illustrated in FIG. 19 includes a first row 1904 , a second row 1906 , a third row 1908 , a fourth row 1910 , and a fifth row 1912 . Further, the five-by-five array 1902 includes a first column 1914 , a second column 1916 , a third column 1918 , a fourth column 1920 , and a fifth column 1922 .
  • the first row 1904 may be associated with a first alignment block 1924
  • the second row 1906 may be associated with a second alignment block 1926
  • the third row 1908 may be associated with a third alignment block 1928
  • the fourth row 1910 may be associated with a fourth alignment block 1930
  • the fifth row 1912 may be associated with a fifth alignment block 1932 .
  • the first alignment block 1924 may be adapted to collect image data lines from each camera in the first row 1904 (i.e., five cameras in the five columns 1914 - 1922 of the array 1902 ).
  • the first alignment block 1924 may be adapted to align the image data lines in a same order and send the image data lines for processing as a single line.
  • the second alignment block 1926 may be adapted to collect image data lines from each camera in the second row 1906 , to align the image data lines in a same order, and to send the image data lines for processing as a single line.
  • the third alignment block 1928 may be adapted to collect image data lines from each camera in the third row 1908 , to align the image data lines in a same order, and to send the image data lines for processing as a single line.
  • the fourth alignment block 1930 may be adapted to collect image data lines from each camera in the fourth row 1910 , to align the image data lines in a same order, and to send the image data lines for processing as a single line.
  • the fifth alignment block 1932 may be adapted to collect image data lines from each camera in the fifth row 1912 , to align the image data lines in a same order, and to send the image data lines for processing as a single line.
  • each of the alignment blocks 1924 - 1932 may be adapted to send its images for processing to a single VFE component 1934 to be processed for color.
  • the single VFE component 1934 may process each of the five lines that are sent from the five alignment blocks 1924 - 1932 .
  • the size of a single line from a particular alignment block may be determined based on a number of cameras in a particular row (i.e., five cameras) and a horizontal size (i.e., “H_res” as described in FIG. 15 ) of each camera in the particular row.
  • the size of the multiple lines processed by the single VFE component 1934 of FIG. 19 may be five times the size of a single line processed by one of the VFE components 1834 - 1842 of FIG. 18 .
  • a control synchronization block 1936 may be used to synchronize each of the cameras of the array 1902 such as providing common control signals to cameras in one or more rows 1904 - 1912 .
  • the control synchronization block 1936 may be communicatively coupled to each of the cameras of the array 1902 and to the single VFE components 1934 . Synchronization of all cameras in the array 1902 may allow for usage of a rolling shutter on a high resolution. As all cameras may be read out at the same time, a rolling shutter effect may be diminished (with the size of the array). For example, in the five-by-five array 1902 of FIG. 19 , synchronization of the twenty five cameras may diminish a rolling shutter effect associated with a CMOS camera.
  • the single VFE component 1934 may be communicatively coupled to a single GPU component 1938 to align and rectify each individual image in each of the rows 1904 - 1912 .
  • the single GPU component 1938 of FIG. 19 may align and rectify twenty five images compared to the five images processed by each of the GPU alignment and rectification components 1846 - 1854 of FIG. 18 .
  • the single GPU component 1938 may be communicatively coupled to a GPU processing component 1940 that is adapted to combine all of the rows 1904 - 1912 together, resulting in a final image.
  • FIG. 20 illustrates a high-resolution digital camera system 2000 that includes a main lens 2002 configured to focus incoming light 2004 and multiple cameras arranged in an array 2006 .
  • a high-resolution image can be generated as a composite (or “mosaic”) image by combining images captured at each of the cameras in the array 2006 .
  • each of the cameras of the array 2006 may be a CMOS-type camera or a Charge Coupled Device (CCD) type camera.
  • the main lens 2002 may focus a captured scene to a plane 2008 (referred to as a “focus plane” of the main lens 2002 or an “object plane” of cameras in the array 2006 ), and each camera in the array 2006 may capture a portion of the total image.
  • Each camera of the array 2006 has a field of view that partially overlaps its neighbors' fields of view at the plane 2008 . This overlap may enable images taken from neighboring cameras in the array 2006 to be aligned on a row-by-row basis and “stitched” together during subsequent processing and may provide tolerance for non-ideal position and alignment of cameras within the array (such as described with respect to FIG. 16 ).
  • a composite image can be generated by aligning image data from the cameras of the array 2006 on a row-by-row basis.
  • the array 2006 of FIG. 20 includes a three-by-three array with three rows.
  • Each camera within a particular row of the array 2006 may include an image sensor that has light detectors arranged in columns and rows (“sensor columns” and “sensor rows”).
  • the cameras within an array row may be positioned so that sensor rows are substantially aligned.
  • a first sensor row of each camera in an array row is substantially aligned with the first sensor row of every other camera in the same array row.
  • the first sensor row of image data is read from each camera in an array row and provided to image processing circuitry (such as described with respect to FIGS. 17-19 ).
  • the image data from the first sensor row is merged according to the position of each camera in the array row.
  • the merged image data is processed as if it were a single row of image data from a larger camera.
  • the second, third, and subsequent image sensor rows are read, merged, and provided to the image processing circuitry to be processed as rows of the composite image.
  • Each array row may be processed in parallel with the other array rows.
  • FIG. 20 may provide an inexpensive alternative to high-resolution cameras.
  • a 100 megapixel (mpix) camera can be built using an array of twenty 5 mpix CMOS cameras behind a main lens. Because image capture can be performed using multiple CMOS cameras with each camera capturing a portion of the image, “rolling shutter” artifacts may be reduced as compared to a single 100 mpix CMOS camera capturing the entire image.
  • the system 2100 illustrates a multiple camera module, such as a multiple camera module as illustrated in FIGS. 1-5 , mounted on an automobile.
  • the multiple camera module may be configured to generate synchronized line data frames for formatting as three dimensional image or video data, such as described with respect to FIGS. 6-7 .
  • a three dimensional view may be obtained to provide an operator of the automobile with depth perception on an internal display (not shown) while backing the automobile.
  • the multiple camera module may be mounted to any type of vehicle, without limitation to automobiles.
  • FIG. 22 a flow diagram of a particular illustrative embodiment of a method of combining data from multiple sensors into a synchronized data line depicted and generally designated 2200 .
  • the method 2200 may be performed by the system of FIG. 2 , the system of FIG. 3 , the system of FIG. 5 , or any combination thereof
  • a common control signal may be provided to multiple image sensors to be synchronized, at 2202 .
  • the common control signal may include a common clock signal and a common reset signal, such as the common control signals 404 - 410 depicted in FIG. 4 .
  • a first data line from a first image sensor of the multiple image sensors may be received, at 2204 .
  • a second data line from a second image sensor of the multiple image sensors may be received, at 2206 .
  • the first sensor and the second sensor may be the sensors 202 , 204 of FIG. 2 .
  • the first data line and the second data line may be combined line to generate a synchronized data line, at 2208 .
  • the method 2200 may include interleaving a first data stream received from the first image sensor and a second data stream received from the second image sensor on a line by line basis.
  • the synchronized data line may be formed as described with respect to the combiner 406 of combining the first sensor image data 422 and the second sensor image data 432 illustrated in FIG. 5 .
  • the synchronized data line may form part of a frame, such as the frame 660 of FIG. 6 .
  • the frame can include a first section (e.g. the first section 652 ) including first image data from the first image sensor, a second section (e.g. the second section 654 ) including second image data from the second image sensor, and a gap section (e.g. the gap section 656 ) including non-image data disposed between the first section and the second section.
  • the frame may not include a gap section between the first section and the second section.
  • Receiving the first data line may be completed before receiving the second data line is completed, and the synchronized data line may be generated after receiving the second data line is completed.
  • the combined data line 822 of FIG. 8 is generated after the second sensor first line data 812 has been received.
  • a third data line may be received from a third image sensor of the multiple image sensors, such as illustrated in FIG. 11 .
  • the third data line can be combined with the first data line and the second data line to generate the synchronized data line, such as the first synchronized data line 1150 of FIG. 11 .
  • FIG. 23 a flow diagram of a particular illustrative embodiment of a method of providing a common control signal to multiple image sensors and providing a synchronized data line to an image processor via a single camera input of the image processor is depicted and generally designated 2300 .
  • the method 2300 may be performed at one or more of the systems of FIGS. 2 , 3 , and 5 , as illustrative, non-limiting examples.
  • a common control signal may be provided to multiple image sensors, at 2302 .
  • Each of the multiple image sensors may be responsive to the common control signal to generate image data.
  • the common control signal may be provided by a sensor synchronizer that is coupled to each of the multiple image sensors, such as the sensor synchronizer 230 of FIG. 2 .
  • the sensor synchronizer may be coupled to each of the multiple image sensors via an inter-integrated circuit (I2C) control interface, via an interface compliant with a camera serial interface (CSI) specification, or via an interface compliant with a camera parallel interface (CPI) specification.
  • I2C inter-integrated circuit
  • CSI camera serial interface
  • CPI camera parallel interface
  • Synchronized data output from each of the multiple image sensors may be received, at 2304 .
  • a first data line may be received from a first image sensor of the multiple image sensors and a second data line may be received from a second image sensor of the multiple image sensors.
  • Receiving the first data line may be completed before receiving the second data line is completed, and a synchronized data line may be generated after receiving the second data line is completed, such as the combined data line 822 that is generated after the second sensor first line data 812 has been received in FIG. 8 .
  • the synchronized data output from each of the multiple image sensors may be combined to generate a synchronized data line, at 2306 .
  • the combiner 206 of FIG. 2 may interleaving a first data stream received from the first image sensor 202 and a second data stream received from the second image sensor 204 on a line by line basis.
  • the synchronized data line may be provided to an image processor via a single camera input of the image processor, at 2308 .
  • the synchronized data line may form part of a frame that has a multiple rows, such as the frame 660 of FIG. 6 .
  • the frame may include a first section including first image data from the first image sensor, a second section including second image data from the second image sensor, and a gap section between the first and second sections.
  • FIG. 24 a flow diagram of a particular illustrative embodiment of a method of providing a common control signal to multiple image sensors and receiving synchronized data output from each of the multiple image sensors is depicted and generally designated 2400 .
  • a common control signal to multiple image sensors may be provided, at 2402 .
  • Each of the multiple image sensors is responsive to the common control signal to generate image data.
  • the common control signal may provided by a sensor synchronizer that is coupled to each of the multiple image sensors, such as the sensor synchronizer 230 of any of FIGS. 2-5 , the control synchronization block 1844 of FIG. 18 , the control synchronization block 1936 of FIG. 19 , or any combination thereof.
  • Synchronized data output from each of the multiple image sensors may be received, at 2404 .
  • the synchronized data output may include first data lines received from a first image sensor and second data lines received from a second image sensor.
  • a phase offset between each received data line from the first image sensor and each corresponding data line from the second image sensor may be substantially constant, such as the one-line phase difference of FIG. 9 , the two-line phase difference of FIG. 8 , or the 3-line phase difference of FIG. 12 , as illustrative, non-limiting examples.
  • FIG. 25 a flow diagram of a particular illustrative embodiment of a method of receiving a common control signal at multiple image sensors and generating synchronized data output from each of the multiple image sensors is depicted and generally designated 2500 .
  • a common control signal may be received at multiple image sensors, at 2502 .
  • Each of the multiple image sensors is responsive to the common control signal to generate image data.
  • the common control signal may received from a sensor synchronizer that is coupled to each of the multiple image sensors, such as the sensor synchronizer 230 of any of FIGS. 2-5 , the control synchronization block 1844 of FIG. 18 , the control synchronization block 1936 of FIG. 19 , or any combination thereof.
  • Synchronized data output from each of the multiple image sensors may be generated, at 2504 .
  • the synchronized data output may include first data lines received from a first image sensor and second data lines received from a second image sensor.
  • a phase offset between each received data line from the first image sensor and each corresponding data line from the second image sensor may be substantially constant, such as the one-line phase difference of FIG. 9 , the two-line phase difference of FIG. 8 , or the 3-line phase difference of FIG. 12 , as illustrative, non-limiting examples.
  • FIG. 26 a flow diagram of a particular illustrative embodiment of a method of combining data from multiple sensors at an image signal processor having an input for a single camera is depicted and generally designated 2600 .
  • Lines of image data may be received at an image processor having an input for a single camera, at 2602 .
  • Each line of the image data may include first line data from a first image captured by a first camera and second line data from a second image captured by a second camera.
  • the image processor may include the image signal processor 208 of FIGS. 2-3 or FIG. 5 , the image processor 1308 of FIG. 13 , the VFEs 1834 - 1842 of FIG. 18 , the VFEs 1934 - 1942 of FIG. 19 , or any combination thereof.
  • the lines of image data may be received at the image processor from a combiner that is coupled to the first camera and to the second camera.
  • Line by line readout of first image data from the first camera and second image data from the second camera may be synchronized, using the combiner, to generate each line of the image data.
  • the combiner may be the combiner 206 of FIGS. 2-3 or FIG. 5 , the combiner 1306 of FIG. 13 , the data alignment blocks 1824 - 1832 of FIG. 18 , the data alignment blocks 1924 - 1932 of FIG. 19 , or any combination thereof.
  • An output frame having a first section corresponding to line data of the first image and having a second section corresponding to line data of the second image may be generated, at 2604 .
  • the first section and the second section may be configured to be used to generate a three-dimensional (3D) image format or a 3D video format.
  • the output frame is processed to generate 3D image data, and the 3D image data is sent to a display device.
  • the output frame is processed to generate 3D video data, and the 3D video data is sent to a display device, such as the display device 1310 of FIG. 13 .
  • the display device may be a component of at least one of a communication device, a camera, a personal digital assistant, and a computer.
  • FIG. 27 a flow diagram of an illustrative embodiment of a method of combining data from multiple sensors into a frame is depicted and generally designated 2700 .
  • the method 2700 may be performed by the system of FIG. 2 , the system of FIG. 5 , or any combination thereof.
  • a first data stream may be received from a first image sensor, such as the first sensor 202 of FIG. 2 , at 2702 .
  • the first data stream such as the first image data stream 212 of FIG. 2 , the first data stream 602 of FIG. 6 , or the timing data signal 420 and the image data signal 422 of FIG. 5 , may correspond to first image data of an image.
  • a second data stream may be received from a second image sensor, such as the second sensor 204 of FIG. 2 , at 2704 .
  • the second data stream such as the second image data stream 214 of FIG. 2 , the second data stream 604 of FIG. 6 , or the timing signal data 430 and the image data signal 432 of FIG. 5 , may correspond to second image data of the image.
  • Data from the first data stream and data from the second data stream may be combined, at 2706 .
  • a combiner such as the combiner 206 of FIG. 2 or the combiner 206 of FIG. 5 , may combine the first image data from the first data stream and the second image data from the second data stream and generate a resulting frame.
  • the first data stream may include data associated with a first line of the first image data including line data having a first line index value, line data having a second line index value, etc.
  • the second data stream may include line data that corresponds to that of the first data stream, including corresponding line data having a first line index value, and corresponding line data having a second line index value, etc.
  • the line data from the first data stream having the first line index value and the line data from the second data stream having the corresponding first line index value may be appended to each other, or combined, to form a single image line.
  • the process may be repeated for each line index value to generate the resulting frame, such as the frame 660 of FIG. 6 .
  • the frame may include a plurality of rows, where each row corresponds to a line index value and stores a line of the first image data having the line index value and stores a corresponding line of the second image data having the line index value.
  • the size of the single image line is substantially double that of the first line of the first image data or the corresponding line of the second image data.
  • the frame may be processed at an image signal processor to generate a processed frame, at 2708 .
  • the image signal processor may be the image signal processor 208 of FIG. 2 or the image signal processor 208 of FIG. 5
  • the processed frame may be the processed frame 240 of FIG. 2 or the processed frame 650 of FIG. 6
  • the processed frame may include a first section including first image data from the first image sensor, such as the first section 652 , a second section including second image data from the second image sensor, such as the second section 654 , and a gap section, such as the gap section 656 of FIG. 6 .
  • the gap section may include non-image data disposed between the first section and the second section.
  • the first section may include a line of the first image data and the second section may include a corresponding line of the second image data.
  • the processed frame may be output to be displayed at a display device, at 2710 .
  • the first image sensor and the second image sensor are each directly responsive to the image signal processor, and the display device may be decoupled from the image signal processor.
  • a flow diagram of an illustrative embodiment of a method of synchronizing a first image sensor and a second image sensor is depicted and generally designated 2800 .
  • the method 2800 may be performed at the system 200 of FIG. 2 , the system 600 of FIG. 5 , or any combination thereof.
  • First image data of an image may be received from a first image sensor, at 2802 .
  • the first image sensor may be the first sensor 202 of FIG. 2 .
  • a first data stream associated with the first image data may be received from the first image sensor, at 2804 .
  • the first data stream may be generated by the image sensor and may be the first image data stream 212 of FIG. 2 , the first data stream 602 of FIG. 6 , or the timing data signal 420 and the image data signal 422 of FIG. 5 .
  • Second image data of the image may be received from a second image sensor, at 2806 .
  • the second image sensor may be the second sensor 204 of FIG. 2 .
  • a second data stream associated with the second image data may be received from the second image sensor, at 2808 .
  • the second data stream may be generated by the image sensor and may be the second image data stream 214 of FIG. 2 , the second data stream 604 of FIG. 6 , or the timing data signal 430 and the image data signal 432 of FIG. 5 .
  • Line by line exposure of the first image sensor and the second image sensor during image data acquisition may be synchronized, at 2810 .
  • the synchronization may occur during image data acquisition of an image at a host including a combiner, such as the combiner 206 of FIG. 2 or the combiner 206 of FIG. 5 .
  • the first image sensor and the second image sensor are independent of each other.
  • the first and second sensors 202 , 204 of FIG. 2 are directly responsive to the image signal processor 208 via the control signal 232 to have similar timing characteristics while remaining independent of each other.
  • the first data stream and the second data stream may be interleaved, at 2812 .
  • the first data stream and the second data stream may be interleaved on a line by line basis.
  • line data from the first data stream having a first line index value and line data from the second data stream having a corresponding first line index value may be appended to each other to form an interleaved single image line.
  • the combined data may be efficiently processed using a single image signal processor.
  • overall image system cost and complexity may be reduced compared to multiple processor systems in which a processor is assigned to each sensor.
  • a flow diagram of a first illustrative embodiment of a method of combining data from multiple image sensors to generate 3D image data is depicted and generally designated 2900 .
  • the method 2900 may be performed by the system of FIG. 13 .
  • the method includes synchronizing line by line readout of first image data from a first camera and a second camera to generate rows of image data, at 2902 .
  • the first image data from the first camera may be the image data stream 1314 from the first image sensor 1302 of FIG. 1 and the second image data may be the image data stream 1316 from the second image sensor 1304 of FIG. 13 .
  • the method includes receiving rows of the image data at an image processor having an input for a single camera, at 2904 .
  • Each row of the image data includes data from a row of a first image captured by the first camera and data from a row of a second image captured by the second camera.
  • the rows of image data may be the data out stream 706 depicted in FIG. 7 .
  • the method includes generating, with the image processor, an output having one of a 3D image format and a 3D video format, at 2906 .
  • the output corresponds to the first image and the second image.
  • the output is sent to a display device (e.g., the display device 1310 of FIG. 13 ), at 2908 .
  • FIG. 30 a flow diagram of an illustrative embodiment of a method of combining data from multiple sensors into a frame is depicted and generally designated 3000 .
  • the method 3000 may be performed by the system of FIG. 13 .
  • a first data stream is received from a first image sensor, such as the first image sensor 1302 of FIG. 13 , at 3002 .
  • the first data stream such as the first image data stream 1314 of FIG. 13 or the first data stream 702 of FIG. 7 may correspond to first image data of a first image.
  • a second data stream may be received from a second image sensor, such as the second image sensor 1304 of FIG. 13 , at 3004 .
  • the second data stream, such as the second image data stream 1316 of FIG. 13 , the second data stream 704 of FIG. 7 may correspond to second image data of a second image.
  • the first image and the second image may be images of a scene.
  • the first image and the second image of the scene may be taken at substantially the same time, or may be taken at different times.
  • the first image may be taken from a different vantage point than the second image so that depth information can be determined from the first image and the second image of the scene.
  • Data from the first data stream and data from the second data stream is combined, at 3006 .
  • a combiner such as the combiner 1306 of FIG. 13 may combine the first image data from the first data stream and the second image data from the second data stream and generate a resulting frame.
  • the first data stream may include data associated with a first line of the first image data including line data having a first line index value, line data having a second line index value, etc.
  • the second data stream may include line data that corresponds to that of the first data stream, including corresponding line data having a first line index value, and corresponding line data having a second line index value, etc.
  • the line data from the first data stream having the first line index value and the line data from the second data stream having the corresponding first line index value may be appended to each other, or combined, to form a single image line.
  • the process may be repeated for each line index value to generate the resulting frame, such as the frame 740 of FIG. 7 .
  • the frame may include a plurality of rows, where each row corresponds to a line index value and stores a line of the first image data having the line index value and stores a corresponding line of the second image data having the line index value.
  • the size of the single image line is substantially double that of the first line of the first image data or the corresponding line of the second image data.
  • the frame is received as rows of image data via an input for a single camera, at 3008 .
  • the input for the single camera may be the input of an image processor, such as the image processor 1308 of FIG. 13 .
  • the frame may be frame 740 of FIG. 7 .
  • An output is generated from the frame, at 3010 .
  • the output has one of a 3D image format and a 3D video format.
  • the output corresponds to the first image and the second image.
  • the output may be the processed frame data 1320 of FIG. 13 .
  • the output is sent to a display device, at 3012 .
  • the display device may be the display device 1310 of FIG. 13 .
  • a flow diagram of an illustrative embodiment of a method of synchronizing a first image sensor and a second image sensor is depicted and generally designated 3100 .
  • the method 3100 may be performed by the system 1300 of FIG. 13 .
  • First image data of an image may be received from a first image sensor, at 3102 .
  • the first image sensor may be the first image sensor 1302 of FIG. 13 .
  • a first data stream associated with the first image data may be received from the first image sensor, at 3104 .
  • the first data stream may be generated by the image sensor and may be the first image data stream 1314 of FIG. 13 or the first data stream 702 of FIG. 7 .
  • Second image data of the image may be received from a second image sensor, at 3106 .
  • the second image sensor may be the second image sensor 1304 of FIG. 13 .
  • a second data stream associated with the second image data may be received from the second image sensor, at 3108 .
  • the second data stream may be generated by the image sensor and may be the second image data stream 1316 of FIG. 13 or the second data stream 704 of FIG. 7 .
  • Line by line exposure of the first image sensor and the second image sensor during image data acquisition may be synchronized, at 3110 .
  • the synchronization may occur during image data acquisition of an image at a host including a combiner, such as the combiner 1306 of FIG. 13 .
  • the first image sensor and the second image sensor are independent of each other.
  • the first and second image sensors 1302 , 1304 of FIG. 13 may be directly responsive to the image processor 1308 via a control signal to have similar timing characteristics while remaining independent of each other.
  • the first and second image sensors 1302 , 1304 are directly responsive to the combiner 1306 via a control signal to have similar timing characteristics while remaining independent of each other.
  • the first data stream and the second data stream may be interleaved, at 3112 .
  • the first data stream and the second data stream may be interleaved on a line by line basis. For example, line data from the first data stream having a first line index value and line data from the second data stream having a corresponding first line index value may be appended to each other to form an interleaved single image line.
  • the combined data may be efficiently processed using a single image processor.
  • overall image system cost and complexity may be reduced compared to multiple processor systems in which a processor is assigned to each sensor.
  • FIG. 32 a flow diagram of an illustrative embodiment of a method of generating a 3D image with an image processor is depicted and generally designated as 3200 .
  • the image processor may be the image processor 1308 of FIG. 13 .
  • the method 3200 may be used when the image processor treats a frame received from a combiner (e.g., the combiner 1306 of FIG. 1 ), a frame from a memory, or when a user chooses to alter a displayed 3D image using a zoom feature or a pan feature of a device displaying the 3D image.
  • a combiner e.g., the combiner 1306 of FIG. 1
  • a frame from a memory or when a user chooses to alter a displayed 3D image using a zoom feature or a pan feature of a device displaying the 3D image.
  • the image processor rectifies a first image and a second image based on parameters of a calibration matrix, at 3202 .
  • the calibration matrix may provide adjustments for relative positions of a first image sensor and a second image sensor that capture the first image and the second image. The relative positions of the two cameras may be selected to ensure minimal scene distortion and eye strain.
  • the calibration matrix may be determined during a manufacturing process for a device that takes the 3D image where the positions of the first image sensor and the second image sensor are fixed relative to each other.
  • the calibration may be stored in a memory of the device.
  • a processor of the device may be used to run a calibration routine to determine the calibration matrix and store the calibration matrix in the memory.
  • the calibration routine may require the first image sensor and the second image sensor to be focused on a particular calibration scene positioned a set distance from the image sensors.
  • the calibration routine may be performed after position adjustment of the image sensors relative to each other.
  • the image processor detects keypoints in the first image, at 3204 .
  • the image processor may detect distinctive (high frequency) points in the first image.
  • the image processor block matches between local image patches in the first image and the second image to compute disparities for each detected keypoint in the first image, at 3206 .
  • a reliability estimator may be produced for every keypoint to insure that erroneous matches are discarded.
  • the image processor determines a convergence adjustment based on a disparity range determined from the computed disparities, at 3208 .
  • the convergence adjustment takes scene depth and display geometry into consideration.
  • the image processor selectively shifts at least one of the first image and the second image based on the convergence adjustment when the convergence adjustment is within capabilities of a display device that will display the 3D image to generate output, at 3210 .
  • the image processor uses the first image with disparity adjusted to match a majority of the scene when the convergence adjustment is not within the capabilities of the display device to generate the output, at 3212 .
  • the image processor crops the output based on one or more display characteristics of the display device, at 3214 .
  • FIG. 33 is a block diagram of a particular embodiment of an image processing system 3300 to combine data from multiple image sensors.
  • the image processing system 3300 may include an image sensor device 3302 that is coupled to a first lens 3304 , coupled to a second lens 3306 , and coupled to an application processor chipset of a portable multimedia device 3308 .
  • the image sensor device 3302 may include a combiner 3310 and an image processor 3312 that receives input for a single camera.
  • the image processor 3312 may receive the single camera input from the combiner 3310 or from a memory device 3314 of the application processor chipset of the portable multimedia device 3308 .
  • the combiner 3310 may combine data from a first data stream and from a second data stream to generate a frame, such as by implementing the system 1300 of FIG. 13 , by operating in accordance with any of the embodiments of FIGS. 29-31 , or any combination thereof.
  • the combiner 3310 is coupled to receive image data from a first sensor 3316 via a first analog-to-digital convertor 3318 .
  • the combiner 3310 is coupled to receive image data from a second sensor 3320 via a second analog-to-digital convertor 3322 .
  • the combiner 3310 or the image processor 3312 may control the first sensor 3316 and the second sensor 3320 , which may be otherwise independent of each other.
  • the image processor 3312 may control the first sensor 3316 and the second sensor 3320 via a sensor synchronizer 3330 (shown in shadow).
  • an integrated circuit that includes image processing circuitry, such as the combiner 3310 , is configured to generate a frame.
  • the image processing circuitry is configured to receive a first data stream from a first image sensor, such as the first sensor 3316 , to receive a second data stream from a second image sensor, such as the second sensor 3320 , and to combine data from the first data stream and from the second data stream to generate the frame.
  • the first data stream 702 and the second data stream 704 of FIG. 7 may be combined by the combiner 3310 to form the frame 740 of FIG. 7 .
  • Output from the combiner 3310 may be sent to a memory device 3314 of the application processor chipset of the portable multimedia device 3308 , to an image processor 3312 , or both.
  • the image processor 3312 may be configured to perform additional image processing operations, such as one or more operations performed by an image processing system.
  • the image processor 3312 may receive a frame from the combiner 3310 or from the memory device 3314 .
  • the image processor 3312 may produce processed image data such as a processed frame having a 3D image format or a 3D video format. In an embodiment, an average time for producing processed image data is about 20 milliseconds.
  • the image processor 3312 may provide the processed image data to the application processor chipset of the portable multimedia device 3308 for further processing, transmission, storage, display to a display device 3324 , or any combination thereof.
  • FIG. 34 a block diagram of a particular illustrative embodiment of an electronic device, such as a wireless phone, including a frame generator module, as described herein, is depicted and generally designated 3400 .
  • the device 3400 includes a processor 3410 coupled to a memory 3432 .
  • the processor includes or is coupled to a controller 3464 .
  • the electronic device may be a set top box, a music player, a video player, an entertainment unit, a navigation device, a communications device, a personal digital assistant (PDA), a camera, a fixed location data unit, or a computer.
  • PDA personal digital assistant
  • FIG. 34 also shows a display controller 3426 that is coupled to the processor 3410 and to a display 3428 .
  • a coder/decoder (CODEC) 3434 can also be coupled to the processor 3410 .
  • a speaker 3439 and a microphone 3438 can be coupled to the CODEC 3434 .
  • a camera controller 3470 can also be coupled to the processor 3410 .
  • a first camera 3472 and a second camera 3473 can be coupled to the camera controller 3470 .
  • FIG. 34 also indicates that a wireless interface 3440 can be coupled to the processor 3410 and to a wireless antenna 3442 .
  • the processor 3410 , the display controller 3426 , the memory 3432 , the CODEC 3434 , the wireless interface 3440 , and the controller 3464 are included in a system-in-package or system-on-chip 3422 .
  • an input device 3430 and a power supply 3444 are coupled to the on-chip system 3422 .
  • FIG. 34 also indicates that a wireless interface 3440 can be coupled to the processor 3410 and to a wireless antenna 3442 .
  • the processor 3410 , the display controller 3426 , the memory 3432 , the CODEC 3434 , the wireless interface 3440 , and the controller 3464 are included in a system-in-package or system-on-chip 3422 .
  • an input device 3430 and a power supply 3444 are coupled to the on-chip system 3422 .
  • FIG. 34 illustrates a wireless interface 3440
  • the display 3428 , the input device 3430 , the speaker 3439 , the microphone 3438 , the wireless antenna 3442 , and the power supply 3444 are external to the on-chip system 3422 .
  • each can be coupled to a component of the on-chip system 3422 , such as an interface or a controller.
  • the processor 3410 executes processor-readable program instructions from a processor-readable medium, such as program instructions 3482 stored at the memory 3432 .
  • the memory 3432 may be readable by the processor 3410 and the instructions 3482 may be operational instructions that are executable by the processor 3410 to perform the method 2200 of FIG. 22 .
  • the instructions 3482 may include instructions that are executable by the processor 3410 to receive a first data stream from a first image sensor, such as the camera 3473 to receive a second data stream from a second image sensor, such as the camera 3472 , and to combine data from the first data stream and from the second data stream to generate a frame.
  • the first image sensor may be the first sensor 202 of FIG.
  • the instructions 3482 may further include instructions that are executable by the processor 3410 to process the frame at the processor 3410 or at an image signal processor (not shown) to generate a processed frame.
  • the instructions 3482 may further include instructions that are executable by the processor 3410 to output the processed frame to be displayed at the display device 3428 or stored at the memory 3432 as image data 3480 .
  • the device 3500 includes a processor 3502 coupled to a memory 3504 .
  • the processor 3502 includes or is coupled to an image processor 3506 .
  • the image processor 3506 may receive a single camera input and may output 3D data 3590 .
  • the 3D data 3590 may be in 3D image format or 3D video format.
  • the electronic device 3500 may be a set top box, a music player, a video player, an entertainment unit, a navigation device, a communications device, a personal digital assistant (PDA), a camera, a fixed location data unit, a computer, or combinations thereof.
  • PDA personal digital assistant
  • FIG. 35 also shows a display controller 3508 that is coupled to the processor 3502 and to a display 3510 .
  • a coder/decoder (CODEC) 3512 can also be coupled to the processor 3502 .
  • a speaker 3514 and a microphone 3516 can be coupled to the CODEC 3512 .
  • a camera controller 3518 can also be coupled to the processor 3502 .
  • the camera controller 3518 may include a combiner 3520 .
  • the combiner 3520 may provide image data to the image processor 3506 .
  • the combiner 3520 may be the combiner 1306 of FIG. 13 , or other hardware circuitry or processor configured to combine data from multiple cameras as illustrated with respect to FIG. 7 .
  • a first camera 3522 and a second camera 3524 can be coupled to the camera controller 3518 .
  • FIG. 35 also indicates that a wireless interface 3526 can be coupled to the processor 3502 and to a wireless antenna 3528 .
  • the processor 3502 , the display controller 3508 , the memory 3504 , the CODEC 3512 , the camera controller 3518 , and the wireless interface 3526 are included in a system-in-package or system-on-chip 3530 .
  • an input device 3532 and a power supply 3534 are coupled to the on-chip system 3530 .
  • FIG. 35 illustrates that a wireless interface 3526 can be coupled to the processor 3502 and to a wireless antenna 3528 .
  • the processor 3502 , the display controller 3508 , the memory 3504 , the CODEC 3512 , the camera controller 3518 , and the wireless interface 3526 are included in a system-in-package or system-on-chip 3530 .
  • an input device 3532 and a power supply 3534 are coupled to the on-chip system 3530 .
  • the display 3510 , the input device 3532 , the speaker 3514 , the microphone 3516 , the wireless antenna 3528 , and the power supply 3534 are external to the on-chip system 3530 .
  • each can be coupled to a component of the on-chip system 3530 , such as an interface or a controller.
  • the processor 3502 executes processor-readable program instructions from a processor-readable medium, such as program instructions 3536 stored at the memory 3504 .
  • the memory 3504 may be readable by the processor 3502 and the instructions 3536 may be operational instructions that are executable by the processor 3502 to perform the method 2500 of FIG. 25 .
  • the instructions 3536 may include instructions that are executable by the processor 3502 to receive a first data stream from a first image sensor, such as the camera 3522 , to receive a second data stream from a second image sensor, such as the camera 3524 , and to combine data from the first data stream and from the second data stream using the combiner 3520 of the camera controller 3518 to generate a frame.
  • the first image sensor may be the first image sensor 1302 of FIG. 13
  • the second image sensor may be the second image sensor 1304 of FIG. 13
  • the instructions 3536 may further include instructions that are executable by the processor 3502 to process the frame at the image processor 3506 to generate a processed frame.
  • the instructions 3536 may further include instructions that are executable by the processor 3502 to output the processed frame as 3D data to the display controller 3508 for display at the display device 3510 or to store the processed frame at the memory 3504 as image data 3538 .
  • FIGS. 22-32 may be performed by executing program code that may be stored in memory in the form of computer readable instructions.
  • a processor such as a digital signal processor (DSP) an image signal processor (ISP), or other processor, may execute instructions stored in memory in order to carry out one or more of the image processing methods.
  • the methods may be executed by a DSP or ISP that invokes various hardware components to accelerate the image processing.
  • the units described herein may be implemented as, or methods may be performed by, a microprocessor, one or more application specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), or any combination thereof.
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays
  • a software module may reside in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of non-transitory computer readable storage medium known in the art.
  • An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium.
  • the storage medium may be integral to the processor.
  • the processor and the storage medium may reside in an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • the ASIC may reside in a computing device or a user terminal.
  • the processor and the storage medium may reside as discrete components in a computing device or user terminal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Studio Devices (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Image Processing (AREA)
  • Facsimiles In General (AREA)
  • Stereoscopic And Panoramic Photography (AREA)
  • Cameras In General (AREA)
  • Facsimile Scanning Arrangements (AREA)
  • Image Input (AREA)
  • Details Of Cameras Including Film Mechanisms (AREA)
  • Facsimile Heads (AREA)
  • Image Analysis (AREA)
US13/079,616 2010-04-05 2011-04-04 Combining data from multiple image sensors Abandoned US20110242342A1 (en)

Priority Applications (9)

Application Number Priority Date Filing Date Title
US13/079,616 US20110242342A1 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors
ARP110101136A AR081160A1 (es) 2010-04-05 2011-04-05 Combinacion de datos de multiples sensores de imagen
PCT/US2011/031282 WO2011127076A1 (en) 2010-04-05 2011-04-05 Combining data from multiple image sensors
CN201180021758.8A CN102884783B (zh) 2010-04-05 2011-04-05 组合来自多个图像传感器的数据
BR112012025182A BR112012025182A2 (pt) 2010-04-05 2011-04-05 combinar dados a partir de múltiplos sensores de imagem
JP2013503850A JP2013528021A (ja) 2010-04-05 2011-04-05 複数の画像センサからのデータの合成
EP11719089A EP2556658A1 (en) 2010-04-05 2011-04-05 Combining data from multiple image sensors
KR1020127029023A KR101399718B1 (ko) 2010-04-05 2011-04-05 다중 이미지 센서들로부터 데이터의 결합
TW100111894A TWI508520B (zh) 2010-04-05 2011-04-06 結合來自多個影像感測器之資料

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US32094010P 2010-04-05 2010-04-05
US32425910P 2010-04-14 2010-04-14
US35931210P 2010-06-28 2010-06-28
US41275510P 2010-11-11 2010-11-11
US13/079,616 US20110242342A1 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors

Publications (1)

Publication Number Publication Date
US20110242342A1 true US20110242342A1 (en) 2011-10-06

Family

ID=44709236

Family Applications (3)

Application Number Title Priority Date Filing Date
US13/079,616 Abandoned US20110242342A1 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors
US13/079,619 Abandoned US20110242355A1 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors
US13/079,629 Active 2032-02-24 US9001227B2 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors

Family Applications After (2)

Application Number Title Priority Date Filing Date
US13/079,619 Abandoned US20110242355A1 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors
US13/079,629 Active 2032-02-24 US9001227B2 (en) 2010-04-05 2011-04-04 Combining data from multiple image sensors

Country Status (15)

Country Link
US (3) US20110242342A1 (es)
EP (3) EP2556657A1 (es)
JP (3) JP5559416B2 (es)
KR (3) KR101512222B1 (es)
CN (3) CN102870401B (es)
AR (3) AR081161A1 (es)
AU (1) AU2011237801A1 (es)
BR (2) BR112012025366A2 (es)
CA (1) CA2794682A1 (es)
IL (1) IL222289A0 (es)
RU (1) RU2012146968A (es)
SG (1) SG184225A1 (es)
TW (3) TW201205308A (es)
WO (3) WO2011127078A1 (es)
ZA (1) ZA201208298B (es)

Cited By (122)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120113230A1 (en) * 2010-11-04 2012-05-10 Samsung Electronics Co., Ltd. Digital photographing apparatus and method of controlling the same
US20120314101A1 (en) * 2011-06-07 2012-12-13 Ooba Yuuji Imaging device and imaging method
US20130057710A1 (en) * 2011-05-11 2013-03-07 Pelican Imaging Corporation Systems and methods for transmitting and receiving array camera image data
DE102012003127A1 (de) * 2012-02-16 2013-09-05 Leica Camera Ag Verfahren für eine Autofokuseinrichtung
US20130327831A1 (en) * 2012-06-11 2013-12-12 Datalogic ADC, Inc. Dynamic imager switching
CN103685978A (zh) * 2012-09-26 2014-03-26 全视技术有限公司 用于同步多个视频传感器的装置及方法
US20140092439A1 (en) * 2012-09-28 2014-04-03 Scott A. Krig Encoding images using a 3d mesh of polygons and corresponding textures
CN103916498A (zh) * 2014-03-28 2014-07-09 宁波萨瑞通讯有限公司 串口摄像头和并口摄像头兼容方法
US8885059B1 (en) 2008-05-20 2014-11-11 Pelican Imaging Corporation Systems and methods for measuring depth using images captured by camera arrays
US8896668B2 (en) 2010-04-05 2014-11-25 Qualcomm Incorporated Combining data from multiple image sensors
US8908041B2 (en) 2013-01-15 2014-12-09 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US8928793B2 (en) 2010-05-12 2015-01-06 Pelican Imaging Corporation Imager array interfaces
US20150009351A1 (en) * 2013-07-04 2015-01-08 Olympus Corporation Image acquisition apparatus
US9001227B2 (en) 2010-04-05 2015-04-07 Qualcomm Incorporated Combining data from multiple image sensors
US20150109468A1 (en) * 2013-10-18 2015-04-23 The Lightco Inc. Image capture control methods and apparatus
US9025895B2 (en) 2011-09-28 2015-05-05 Pelican Imaging Corporation Systems and methods for decoding refocusable light field image files
US9041824B2 (en) 2010-12-14 2015-05-26 Pelican Imaging Corporation Systems and methods for dynamic refocusing of high resolution images generated using images captured by a plurality of imagers
US9049381B2 (en) 2008-05-20 2015-06-02 Pelican Imaging Corporation Systems and methods for normalizing image data captured by camera arrays
US9100586B2 (en) 2013-03-14 2015-08-04 Pelican Imaging Corporation Systems and methods for photometric normalization in array cameras
US9124864B2 (en) 2013-03-10 2015-09-01 Pelican Imaging Corporation System and methods for calibration of an array camera
US9123118B2 (en) 2012-08-21 2015-09-01 Pelican Imaging Corporation System and methods for measuring depth using an array camera employing a bayer filter
US9143711B2 (en) 2012-11-13 2015-09-22 Pelican Imaging Corporation Systems and methods for array camera focal plane control
US20150296154A1 (en) * 2013-10-18 2015-10-15 The Lightco Inc. Image capture related methods and apparatus
US9185276B2 (en) 2013-11-07 2015-11-10 Pelican Imaging Corporation Methods of manufacturing array camera modules incorporating independently aligned lens stacks
US9195254B2 (en) * 2012-12-21 2015-11-24 Qualcomm, Incorporated Method and apparatus for multi-level de-emphasis
US9210392B2 (en) 2012-05-01 2015-12-08 Pelican Imaging Coporation Camera modules patterned with pi filter groups
JP2015535673A (ja) * 2012-12-27 2015-12-14 インテル・コーポレーション ゲットアンドセットアーキテクチャに基づくトランスポートメカニズム内のコマンド実行
US9214013B2 (en) 2012-09-14 2015-12-15 Pelican Imaging Corporation Systems and methods for correcting user identified artifacts in light field images
US9247117B2 (en) 2014-04-07 2016-01-26 Pelican Imaging Corporation Systems and methods for correcting for warpage of a sensor array in an array camera module by introducing warpage into a focal plane of a lens stack array
US9253380B2 (en) 2013-02-24 2016-02-02 Pelican Imaging Corporation Thin form factor computational array cameras and modular array cameras
US9253450B2 (en) 2012-04-18 2016-02-02 Utc Fire & Security Americas Corporation, Inc. Total bus surveillance system
US9264610B2 (en) 2009-11-20 2016-02-16 Pelican Imaging Corporation Capturing and processing of images including occlusions captured by heterogeneous camera arrays
CN105335779A (zh) * 2014-08-14 2016-02-17 夏志刚 数卡器及使用数卡器进行数卡的方法
US20160103209A1 (en) * 2013-07-16 2016-04-14 Fujifilm Corporation Imaging device and three-dimensional-measurement device
US20160119575A1 (en) * 2014-10-24 2016-04-28 Texas Instruments Incorporated Image data processing for digital overlap wide dynamic range sensors
US9374514B2 (en) 2013-10-18 2016-06-21 The Lightco Inc. Methods and apparatus relating to a camera including multiple optical chains
US20160182890A1 (en) * 2012-10-05 2016-06-23 Qualcomm Incorporated Method and apparatus for bus sharing by multiple imaging sensors
US9412206B2 (en) 2012-02-21 2016-08-09 Pelican Imaging Corporation Systems and methods for the manipulation of captured light field image data
US9426361B2 (en) 2013-11-26 2016-08-23 Pelican Imaging Corporation Array camera configurations incorporating multiple constituent array cameras
US9426365B2 (en) 2013-11-01 2016-08-23 The Lightco Inc. Image stabilization related methods and apparatus
US9438888B2 (en) 2013-03-15 2016-09-06 Pelican Imaging Corporation Systems and methods for stereo imaging with camera arrays
US9454245B2 (en) 2011-11-01 2016-09-27 Qualcomm Incorporated System and method for improving orientation data
US9462170B2 (en) 2014-02-21 2016-10-04 The Lightco Inc. Lighting methods and apparatus
US9467627B2 (en) 2013-10-26 2016-10-11 The Lightco Inc. Methods and apparatus for use with multiple optical chains
WO2016168781A1 (en) * 2015-04-17 2016-10-20 The Lightco Inc. Methods and apparatus for syncronizing readout of multiple image sensors
US9497429B2 (en) 2013-03-15 2016-11-15 Pelican Imaging Corporation Extended color processing on pelican array cameras
US9497370B2 (en) 2013-03-15 2016-11-15 Pelican Imaging Corporation Array camera architecture implementing quantum dot color filters
US9516222B2 (en) 2011-06-28 2016-12-06 Kip Peli P1 Lp Array cameras incorporating monolithic array camera modules with high MTF lens stacks for capture of images used in super-resolution processing
US9521319B2 (en) 2014-06-18 2016-12-13 Pelican Imaging Corporation Array cameras and array camera modules including spectral filters disposed outside of a constituent image sensor
US9544503B2 (en) 2014-12-30 2017-01-10 Light Labs Inc. Exposure control methods and apparatus
US9547160B2 (en) 2013-01-05 2017-01-17 Light Labs Inc. Methods and apparatus for capturing and/or processing images
US9554031B2 (en) 2013-12-31 2017-01-24 Light Labs Inc. Camera focusing related methods and apparatus
US9578259B2 (en) 2013-03-14 2017-02-21 Fotonation Cayman Limited Systems and methods for reducing motion blur in images or video in ultra low light with array cameras
US9633442B2 (en) 2013-03-15 2017-04-25 Fotonation Cayman Limited Array cameras including an array camera module augmented with a separate camera
EP3185538A1 (en) * 2015-12-24 2017-06-28 Samsung Electronics Co., Ltd Electronic device and method of controlling the same
US9733486B2 (en) 2013-03-13 2017-08-15 Fotonation Cayman Limited Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing
US9736365B2 (en) 2013-10-26 2017-08-15 Light Labs Inc. Zoom related methods and apparatus
US9741118B2 (en) 2013-03-13 2017-08-22 Fotonation Cayman Limited System and methods for calibration of an array camera
US9749549B2 (en) 2015-10-06 2017-08-29 Light Labs Inc. Methods and apparatus for facilitating selective blurring of one or more image portions
US9766380B2 (en) 2012-06-30 2017-09-19 Fotonation Cayman Limited Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors
US9774789B2 (en) 2013-03-08 2017-09-26 Fotonation Cayman Limited Systems and methods for high dynamic range imaging using array cameras
US9794476B2 (en) 2011-09-19 2017-10-17 Fotonation Cayman Limited Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures
US9800856B2 (en) 2013-03-13 2017-10-24 Fotonation Cayman Limited Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies
US9807382B2 (en) 2012-06-28 2017-10-31 Fotonation Cayman Limited Systems and methods for detecting defective camera arrays and optic arrays
US9813616B2 (en) 2012-08-23 2017-11-07 Fotonation Cayman Limited Feature based high resolution motion estimation from low resolution images captured using an array source
US9824427B2 (en) 2015-04-15 2017-11-21 Light Labs Inc. Methods and apparatus for generating a sharp image
WO2017203901A1 (en) * 2016-05-26 2017-11-30 Sony Semiconductor Solutions Corporation Processing apparatus, image sensor, and system
US9857584B2 (en) 2015-04-17 2018-01-02 Light Labs Inc. Camera device methods, apparatus and components
CN107615748A (zh) * 2015-06-10 2018-01-19 索尼公司 图像处理装置和方法
US9888194B2 (en) 2013-03-13 2018-02-06 Fotonation Cayman Limited Array camera architecture implementing quantum film image sensors
US9898856B2 (en) 2013-09-27 2018-02-20 Fotonation Cayman Limited Systems and methods for depth-assisted perspective distortion correction
US9912864B2 (en) 2014-10-17 2018-03-06 Light Labs Inc. Methods and apparatus for using a camera device to support multiple modes of operation
US9930233B2 (en) 2015-04-22 2018-03-27 Light Labs Inc. Filter mounting methods and apparatus and related camera apparatus
WO2018062599A1 (ko) * 2016-09-27 2018-04-05 주식회사 켐트로닉스 Svm 시스템 및 그의 영상입력 및 처리방법
US9942474B2 (en) 2015-04-17 2018-04-10 Fotonation Cayman Limited Systems and methods for performing high speed video capture and depth estimation using array cameras
US9948832B2 (en) 2016-06-22 2018-04-17 Light Labs Inc. Methods and apparatus for synchronized image capture in a device including optical chains with different orientations
US9955070B2 (en) 2013-03-15 2018-04-24 Fotonation Cayman Limited Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information
EP3189659A4 (en) * 2014-09-03 2018-05-02 Intel Corporation Imaging architecture for depth camera mode with mode switching
US9967535B2 (en) 2015-04-17 2018-05-08 Light Labs Inc. Methods and apparatus for reducing noise in images
US9979878B2 (en) 2014-02-21 2018-05-22 Light Labs Inc. Intuitive camera user interface methods and apparatus
US9998638B2 (en) 2014-12-17 2018-06-12 Light Labs Inc. Methods and apparatus for implementing and using camera devices
US10003738B2 (en) 2015-12-18 2018-06-19 Light Labs Inc. Methods and apparatus for detecting and/or indicating a blocked sensor or camera module
US10009538B2 (en) 2013-02-21 2018-06-26 Fotonation Cayman Limited Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information
US10075651B2 (en) 2015-04-17 2018-09-11 Light Labs Inc. Methods and apparatus for capturing images using multiple camera modules in an efficient manner
US10089740B2 (en) 2014-03-07 2018-10-02 Fotonation Limited System and methods for depth regularization and semiautomatic interactive matting using RGB-D images
US10110794B2 (en) 2014-07-09 2018-10-23 Light Labs Inc. Camera device including multiple optical chains and related methods
US10122993B2 (en) 2013-03-15 2018-11-06 Fotonation Limited Autofocus system for a conventional camera that uses depth information from an array camera
US10119808B2 (en) 2013-11-18 2018-11-06 Fotonation Limited Systems and methods for estimating depth from projected texture using camera arrays
US10129483B2 (en) 2015-06-23 2018-11-13 Light Labs Inc. Methods and apparatus for implementing zoom using one or more moveable camera modules
EP3386185A4 (en) * 2015-12-24 2018-11-14 Samsung Electronics Co., Ltd. Electronic device and control method for electronic device
US10191356B2 (en) 2014-07-04 2019-01-29 Light Labs Inc. Methods and apparatus relating to detection and/or indicating a dirty lens condition
CN109309784A (zh) * 2017-07-28 2019-02-05 展讯通信（上海）有限公司 移动终端
US10225445B2 (en) 2015-12-18 2019-03-05 Light Labs Inc. Methods and apparatus for providing a camera lens or viewing point indicator
US10250871B2 (en) 2014-09-29 2019-04-02 Fotonation Limited Systems and methods for dynamic calibration of array cameras
US10306218B2 (en) 2016-03-22 2019-05-28 Light Labs Inc. Camera calibration apparatus and methods
US10365480B2 (en) 2015-08-27 2019-07-30 Light Labs Inc. Methods and apparatus for implementing and/or using camera devices with one or more light redirection devices
US10390005B2 (en) 2012-09-28 2019-08-20 Fotonation Limited Generating images from light fields utilizing virtual viewpoints
US10482618B2 (en) 2017-08-21 2019-11-19 Fotonation Limited Systems and methods for hybrid depth regularization
US10491806B2 (en) 2015-08-03 2019-11-26 Light Labs Inc. Camera device control related methods and apparatus
US10554958B2 (en) 2017-03-13 2020-02-04 Microsoft Technology Licensing, Llc Systems and methods for interleaving multiple active camera frames
EP3531296A4 (en) * 2016-10-20 2020-05-27 Hitachi Automotive Systems, Ltd. CAMERA DEVICE
US10692262B2 (en) 2017-01-12 2020-06-23 Electronics And Telecommunications Research Institute Apparatus and method for processing information of multiple cameras
US20200202196A1 (en) * 2018-12-21 2020-06-25 Waymo Llc Searching an autonomous vehicle sensor data repository
US11270110B2 (en) 2019-09-17 2022-03-08 Boston Polarimetrics, Inc. Systems and methods for surface modeling using polarization cues
US20220092726A1 (en) * 2020-09-18 2022-03-24 Kabushiki Kaisha Toshiba Image processing apparatus
US11290658B1 (en) 2021-04-15 2022-03-29 Boston Polarimetrics, Inc. Systems and methods for camera exposure control
US11302012B2 (en) 2019-11-30 2022-04-12 Boston Polarimetrics, Inc. Systems and methods for transparent object segmentation using polarization cues
US20220164350A1 (en) * 2020-11-25 2022-05-26 Waymo Llc Searching an autonomous vehicle sensor data repository based on context embedding
US11525906B2 (en) 2019-10-07 2022-12-13 Intrinsic Innovation Llc Systems and methods for augmentation of sensor systems and imaging systems with polarization
US11580667B2 (en) 2020-01-29 2023-02-14 Intrinsic Innovation Llc Systems and methods for characterizing object pose detection and measurement systems
US11627257B2 (en) 2020-11-26 2023-04-11 Samsung Electronics Co., Ltd. Electronic device including image sensor having multi-crop function
US11689813B2 (en) 2021-07-01 2023-06-27 Intrinsic Innovation Llc Systems and methods for high dynamic range imaging using crossed polarizers
US20230275716A1 (en) * 2022-02-28 2023-08-31 e-con Systems India Private Limited System and method for assisting data transmission over virtual channels
US11792538B2 (en) 2008-05-20 2023-10-17 Adeia Imaging Llc Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US11797863B2 (en) 2020-01-30 2023-10-24 Intrinsic Innovation Llc Systems and methods for synthesizing data for training statistical models on different imaging modalities including polarized images
EP4184913A4 (en) * 2020-07-29 2023-12-20 Huawei Technologies Co., Ltd. FUSION APPARATUS FOR MULTIPLE DATA TRANSMISSION CHANNELS AND ELECTRONIC DEVICE
US11953700B2 (en) 2020-05-27 2024-04-09 Intrinsic Innovation Llc Multi-aperture polarization optical systems using beam splitters
US11954886B2 (en) 2021-04-15 2024-04-09 Intrinsic Innovation Llc Systems and methods for six-degree of freedom pose estimation of deformable objects
EP4369717A1 (en) * 2022-11-14 2024-05-15 Samsung Electronics Co., Ltd. Image processing apparatus including line interleaving controller for a plurality of image sensors and operating method thereof
US12020455B2 (en) 2021-03-10 2024-06-25 Intrinsic Innovation Llc Systems and methods for high dynamic range image reconstruction
US12069227B2 (en) 2021-03-10 2024-08-20 Intrinsic Innovation Llc Multi-modal and multi-spectral stereo camera arrays
US12067746B2 (en) 2021-05-07 2024-08-20 Intrinsic Innovation Llc Systems and methods for using computer vision to pick up small objects

Families Citing this family (120)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8874090B2 (en) * 2010-04-07 2014-10-28 Apple Inc. Remote control operations in a video conference
US20140192238A1 (en) * 2010-10-24 2014-07-10 Linx Computational Imaging Ltd. System and Method for Imaging and Image Processing
US8555067B2 (en) 2010-10-28 2013-10-08 Apple Inc. Methods and apparatus for delivering electronic identification components over a wireless network
JP2012198075A (ja) * 2011-03-18 2012-10-18 Ricoh Co Ltd ステレオカメラ装置、画像補整方法
US9313390B2 (en) * 2011-04-08 2016-04-12 Qualcomm Incorporated Systems and methods to calibrate a multi camera device
JP6021489B2 (ja) * 2011-10-03 2016-11-09 キヤノン株式会社 撮像装置、画像処理装置およびその方法
JP5918982B2 (ja) * 2011-11-22 2016-05-18 キヤノン株式会社 撮像装置、再生装置、その制御方法、撮像システム、及びプログラム
US11094137B2 (en) 2012-02-24 2021-08-17 Matterport, Inc. Employing three-dimensional (3D) data predicted from two-dimensional (2D) images using neural networks for 3D modeling applications and other applications
US10848731B2 (en) 2012-02-24 2020-11-24 Matterport, Inc. Capturing and aligning panoramic image and depth data
US9324190B2 (en) 2012-02-24 2016-04-26 Matterport, Inc. Capturing and aligning three-dimensional scenes
JP6007509B2 (ja) * 2012-02-27 2016-10-12 株式会社リコー シリアルｉ／ｆバス制御装置及び撮像装置
US9229526B1 (en) * 2012-09-10 2016-01-05 Amazon Technologies, Inc. Dedicated image processor
CN105556944B (zh) 2012-11-28 2019-03-08 核心光电有限公司 多孔径成像系统和方法
US9686460B2 (en) * 2012-12-27 2017-06-20 Intel Corporation Enabling a metadata storage subsystem
KR20140094395A (ko) * 2013-01-22 2014-07-30 삼성전자주식회사 복수 개의 마이크로렌즈를 사용하여 촬영하는 촬영 장치 및 그 촬영 방법
JP6139713B2 (ja) 2013-06-13 2017-05-31 コアフォトニクス リミテッド デュアルアパーチャズームデジタルカメラ
CN108519655A (zh) 2013-07-04 2018-09-11 核心光电有限公司 小型长焦透镜套件
US10033989B2 (en) * 2013-07-05 2018-07-24 Mediatek Inc. Synchronization controller for multi-sensor camera device and related synchronization method
CN108989649B (zh) 2013-08-01 2021-03-19 核心光电有限公司 具有自动聚焦的纤薄多孔径成像系统及其使用方法
CN104994369B (zh) * 2013-12-04 2018-08-21 南京中兴软件有限责任公司 一种图像处理方法、用户终端、图像处理终端及系统
CN103702073A (zh) * 2013-12-11 2014-04-02 天津大学 一拖多个成像传感器同步成像的一体化相机电路
KR101530163B1 (ko) * 2013-12-12 2015-06-17 (주)씨프로 Cctv용 파노라마 카메라 장치
KR101528556B1 (ko) * 2013-12-12 2015-06-17 (주)씨프로 Cctv용 파노라마 카메라 장치
KR102128468B1 (ko) 2014-02-19 2020-06-30 삼성전자주식회사 복수의 이미지 신호 프로세서들을 포함하는 이미지 처리 장치 및 이미지 처리 방법
KR102157675B1 (ko) * 2014-07-25 2020-09-18 삼성전자주식회사 촬영 장치 및 그 촬영 방법
US9392188B2 (en) 2014-08-10 2016-07-12 Corephotonics Ltd. Zoom dual-aperture camera with folded lens
US9313391B1 (en) * 2014-08-14 2016-04-12 Amazon Technologies, Inc. Camera interfaces for electronic devices
US9842071B2 (en) * 2014-11-11 2017-12-12 Microchip Technology Incorporated Multi-channel I2S transmit control system and method
WO2016108093A1 (en) 2015-01-03 2016-07-07 Corephotonics Ltd. Miniature telephoto lens module and a camera utilizing such a lens module
US9491495B2 (en) 2015-01-16 2016-11-08 Analog Devices Global Method and apparatus for providing input to a camera serial interface transmitter
JP2016178371A (ja) * 2015-03-18 2016-10-06 キヤノン株式会社 撮像装置
EP3492958B1 (en) 2015-04-02 2022-03-30 Corephotonics Ltd. Dual voice coil motor structure in a dual-optical module camera
US9927600B2 (en) 2015-04-16 2018-03-27 Corephotonics Ltd Method and system for providing auto focus and optical image stabilization in a compact folded camera
US10043307B2 (en) 2015-04-17 2018-08-07 General Electric Company Monitoring parking rule violations
US9940524B2 (en) 2015-04-17 2018-04-10 General Electric Company Identifying and tracking vehicles in motion
EP3091510B1 (en) * 2015-05-06 2021-07-07 Reactive Reality AG Method and system for producing output images
WO2016189878A1 (ja) * 2015-05-27 2016-12-01 京セラ株式会社 演算装置、カメラ装置、車両及びキャリブレーション方法
US10036895B2 (en) 2015-05-28 2018-07-31 Corephotonics Ltd. Bi-directional stiffness for optical image stabilization in a dual-aperture digital camera
EP3787281B1 (en) 2015-08-13 2024-08-21 Corephotonics Ltd. Dual aperture zoom camera with video support and switching / non-switching dynamic control
KR101993077B1 (ko) 2015-09-06 2019-06-25 코어포토닉스 리미티드 소형의 접이식 카메라의 롤 보정에 의한 자동 초점 및 광학식 손떨림 방지
US9992477B2 (en) 2015-09-24 2018-06-05 Ouster, Inc. Optical system for collecting distance information within a field
KR102480600B1 (ko) * 2015-10-21 2022-12-23 삼성전자주식회사 이미지 처리 장치의 저조도 화질 개선 방법 및 상기 방법을 수행하는 이미지 처리 시스템의 동작 방법
KR101751140B1 (ko) 2015-12-24 2017-06-26 삼성전기주식회사 이미지 센서 및 카메라 모듈
CN109889708B (zh) 2015-12-29 2021-07-06 核心光电有限公司 具有自动可调节长焦视场的双孔径变焦数字摄影机
US10108462B2 (en) 2016-02-12 2018-10-23 Microsoft Technology Licensing, Llc Virtualizing sensors
JP6743427B2 (ja) * 2016-03-08 2020-08-19 株式会社リコー 情報処理装置、撮像システムおよびデータ転送方法
CN105635601B (zh) * 2016-03-28 2019-04-23 广州市盛光微电子有限公司 一种多传感器多通道视频数据输入方法和装置
US10719107B2 (en) 2016-03-29 2020-07-21 Intel Corporation Method and apparatus to maintain node power budget for systems that share a power supply
US9813783B2 (en) 2016-04-01 2017-11-07 Intel Corporation Multi-camera dataset assembly and management with high precision timestamp requirements
DE102016208210A1 (de) * 2016-05-12 2017-11-16 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. 3d-multiaperturabbildungsvorrichtungen, multiaperturabbildungsvorrichtung, verfahren zum bereitstellen eines ausgangssignals einer 3d-multiaperturabbildungsvorrichtung und verfahren zum erfassen eines gesamtgesichtsfeldes
US10488631B2 (en) 2016-05-30 2019-11-26 Corephotonics Ltd. Rotational ball-guided voice coil motor
WO2017221106A1 (en) 2016-06-19 2017-12-28 Corephotonics Ltd. Frame synchronization in a dual-aperture camera system
US10706518B2 (en) 2016-07-07 2020-07-07 Corephotonics Ltd. Dual camera system with improved video smooth transition by image blending
KR102390572B1 (ko) 2016-07-07 2022-04-25 코어포토닉스 리미티드 폴디드 옵틱용 선형 볼 가이드 음성 코일 모터
US10284838B2 (en) 2016-08-19 2019-05-07 Titan Medical Inc. Method and apparatus for transmitting images captured by first and second image sensors
US20180063428A1 (en) * 2016-09-01 2018-03-01 ORBI, Inc. System and method for virtual reality image and video capture and stitching
JP6330873B2 (ja) * 2016-09-14 2018-05-30 株式会社リコー 撮像装置
EP3550821A4 (en) * 2016-11-29 2020-04-01 Sony Corporation IMAGING DEVICE, IMAGING CONTROL METHOD, AND PROGRAM
KR102269547B1 (ko) 2016-12-28 2021-06-25 코어포토닉스 리미티드 확장된 광-폴딩-요소 스캐닝 범위를 갖는 폴디드 카메라 구조
CN113791484A (zh) 2017-01-12 2021-12-14 核心光电有限公司 紧凑型折叠式摄影机及其组装方法
KR102557662B1 (ko) * 2017-02-09 2023-07-19 삼성전자주식회사 이미지 처리 장치 및 이를 포함하는 전자 장치
JP2018128401A (ja) * 2017-02-10 2018-08-16 東芝Ｉｔコントロールシステム株式会社 Ｘ線透視検査装置
IL302577A (en) 2017-02-23 2023-07-01 Corephotonics Ltd Lens designs for a folded camera
US10841478B2 (en) * 2017-03-02 2020-11-17 Sony Semiconductor Solutions Corporation Image sensor and control system
CN110582724B (zh) 2017-03-15 2022-01-04 核心光电有限公司 具有全景扫描范围的照相装置
CN108881946A (zh) * 2017-05-10 2018-11-23 北京猎户星空科技有限公司 传感器数据的生成、传输、处理方法、装置及其系统
CN108881112B (zh) * 2017-05-10 2021-07-30 北京猎户星空科技有限公司 基于传感器的数据的生成、传输、处理方法、装置及其系统
TWI617195B (zh) * 2017-06-22 2018-03-01 宏碁股份有限公司 影像擷取裝置及其影像拼接方法
CN109214983B (zh) * 2017-06-30 2022-12-13 宏碁股份有限公司 图像获取装置及其图像拼接方法
US10904512B2 (en) 2017-09-06 2021-01-26 Corephotonics Ltd. Combined stereoscopic and phase detection depth mapping in a dual aperture camera
KR102385333B1 (ko) * 2017-09-15 2022-04-12 삼성전자주식회사 복수의 이미지 센서들을 제어하기 위한 전자 장치 및 방법
US10951834B2 (en) 2017-10-03 2021-03-16 Corephotonics Ltd. Synthetically enlarged camera aperture
KR102424791B1 (ko) 2017-11-23 2022-07-22 코어포토닉스 리미티드 컴팩트 폴디드 카메라 구조
KR20190065736A (ko) * 2017-12-04 2019-06-12 삼성전자주식회사 3d 이미지를 생성하기 위한 전자 장치 및 방법
CN114609746A (zh) 2018-02-05 2022-06-10 核心光电有限公司 折叠摄像装置
EP4191315B1 (en) 2018-02-12 2024-09-25 Corephotonics Ltd. Folded camera with optical image stabilization
US10764512B2 (en) 2018-03-26 2020-09-01 Mediatek Inc. Method of image fusion on camera device equipped with multiple cameras
CN108540689B (zh) * 2018-04-09 2020-10-02 珠海全志科技股份有限公司 图像信号处理器、应用处理器及移动装置
US10694168B2 (en) 2018-04-22 2020-06-23 Corephotonics Ltd. System and method for mitigating or preventing eye damage from structured light IR/NIR projector systems
CN111936908B (zh) 2018-04-23 2021-12-21 核心光电有限公司 具有扩展的两个自由度旋转范围的光路折叠元件
EP3573342B1 (en) * 2018-05-25 2021-03-31 Harman Becker Automotive Systems GmbH Multi-rate digital sensor synchronization
KR102107299B1 (ko) * 2018-06-21 2020-05-06 한국산업기술대학교산학협력단 영상 데이터 전송 방법 및 이를 수행하는 장치들
WO2019245118A1 (ko) * 2018-06-22 2019-12-26 재단법인 실감교류인체감응솔루션연구단 스테레오 카메라의 동기화 장치, 스테레오 카메라 및 스테레오 카메라의 동기화 방법
WO2020031005A1 (en) 2018-08-04 2020-02-13 Corephotonics Ltd. Switchable continuous display information system above camera
WO2020039302A1 (en) 2018-08-22 2020-02-27 Corephotonics Ltd. Two-state zoom folded camera
US11287081B2 (en) 2019-01-07 2022-03-29 Corephotonics Ltd. Rotation mechanism with sliding joint
US11092456B2 (en) * 2019-03-08 2021-08-17 Aptiv Technologies Limited Object location indicator system and method
EP4224841A1 (en) 2019-03-09 2023-08-09 Corephotonics Ltd. System and method for dynamic stereoscopic calibration
CN110392149A (zh) * 2019-07-23 2019-10-29 华为技术有限公司 图像摄取显示终端
KR102365748B1 (ko) 2019-07-31 2022-02-23 코어포토닉스 리미티드 카메라 패닝 또는 모션에서 배경 블러링을 생성하는 시스템 및 방법
CN110381272B (zh) 2019-08-22 2022-05-13 睿镞科技(北京)有限责任公司 图像传感器组合系统及产生单一和复合视野图像的装置
GB2587668A (en) * 2019-10-02 2021-04-07 Advanced Mobility Res And Development Ltd Systems and methods for aircraft
US11659135B2 (en) 2019-10-30 2023-05-23 Corephotonics Ltd. Slow or fast motion video using depth information
US11949976B2 (en) 2019-12-09 2024-04-02 Corephotonics Ltd. Systems and methods for obtaining a smart panoramic image
CN114641983A (zh) 2019-12-09 2022-06-17 核心光电有限公司 用于获得智能全景图像的系统及方法
CN111031209B (zh) * 2019-12-17 2021-11-05 威海华菱光电股份有限公司 图像处理装置
WO2021162173A1 (ko) * 2020-02-14 2021-08-19 엘지전자 주식회사 멀티 카메라, 이미지를 촬영하는 장치 및 그 방법
CN114641805A (zh) 2020-02-22 2022-06-17 核心光电有限公司 用于微距摄影的分屏特征
WO2021220080A1 (en) 2020-04-26 2021-11-04 Corephotonics Ltd. Temperature control for hall bar sensor correction
KR102495627B1 (ko) 2020-05-17 2023-02-06 코어포토닉스 리미티드 전체 시야 레퍼런스 이미지 존재 하의 이미지 스티칭
EP3966631B1 (en) 2020-05-30 2023-01-25 Corephotonics Ltd. Systems and methods for obtaining a super macro image
KR20210150704A (ko) 2020-06-04 2021-12-13 삼성전자주식회사 라인 인터리빙 컨트롤러 및 이를 포함하는 이미지 신호 프로세서
KR102455520B1 (ko) 2020-06-05 2022-10-17 한국과학기술원 마이크로렌즈 어레이를 이용한 초박형 카메라 장치 그리고 이의 다기능 이미징 방법
US11637977B2 (en) 2020-07-15 2023-04-25 Corephotonics Ltd. Image sensors and sensing methods to obtain time-of-flight and phase detection information
KR20230004887A (ko) 2020-07-15 2023-01-06 코어포토닉스 리미티드 스캐닝 폴디드 카메라에서의 시점 수차-보정
EP4065934A4 (en) 2020-07-31 2023-07-26 Corephotonics Ltd. LARGE STROKE LINEAR POSITION DETECTION HALL EFFECT SENSOR MAGNET GEOMETRY
KR102480820B1 (ko) 2020-08-12 2022-12-22 코어포토닉스 리미티드 스캐닝 폴디드 카메라의 광학 이미지 안정화
CN116420105A (zh) * 2020-09-30 2023-07-11 斯纳普公司 用于增强现实护目镜中的计算机视觉模式的低功耗相机管线
US12015842B2 (en) 2020-09-30 2024-06-18 Snap Inc. Multi-purpose cameras for simultaneous capture and CV on wearable AR devices
CN112689072A (zh) * 2020-12-23 2021-04-20 合肥安迅精密技术有限公司 一种相机系统
KR102629883B1 (ko) 2020-12-26 2024-01-25 코어포토닉스 리미티드 스캐닝 줌 카메라를 갖는 멀티-애퍼처 모바일 카메라에서의 비디오 지원
CN115868168A (zh) 2021-03-11 2023-03-28 核心光电有限公司 用于弹出式相机的系统
CN115190254A (zh) * 2021-04-07 2022-10-14 影石创新科技股份有限公司 多个视觉传感器的同步曝光电路、曝光方法及曝光装置
WO2022259154A2 (en) 2021-06-08 2022-12-15 Corephotonics Ltd. Systems and cameras for tilting a focal plane of a super-macro image
WO2023064505A1 (en) * 2021-10-14 2023-04-20 Redzone Robotics, Inc. Data translation and interoperability
US11995012B2 (en) * 2022-03-15 2024-05-28 Samsung Electronics Co., Ltd. High speed interface for multi image sensor device
US11818329B1 (en) * 2022-09-21 2023-11-14 Ghost Autonomy Inc. Synchronizing stereoscopic cameras using padding data setting modification
WO2024107200A1 (en) * 2022-11-18 2024-05-23 Zeku, Inc. Multi-sensor image processing on mobile devices and method of operating the same
US20240196084A1 (en) * 2022-12-12 2024-06-13 Sony Group Corporation User interface for rolling shutter camera arrays and accurate volumetric capture workflows
CN118200687A (zh) * 2024-02-18 2024-06-14 厦门瑞为信息技术有限公司 不同Sensor的画面亮度归一到同一个Sensor维度的方法

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07255068A (ja) * 1994-03-14 1995-10-03 Sony Corp テレビジョン方式及び表示装置
US5995140A (en) * 1995-08-28 1999-11-30 Ultrak, Inc. System and method for synchronization of multiple video cameras
US20040075741A1 (en) * 2002-10-17 2004-04-22 Berkey Thomas F. Multiple camera image multiplexer
US6750904B1 (en) * 1998-10-31 2004-06-15 International Business Machines Corporation Camera system for three dimensional images and video
US20040196378A1 (en) * 2003-02-17 2004-10-07 Axis Ab., A Swedish Corporation Method and apparatus for panning and tilting a camera
US20050280702A1 (en) * 2004-06-17 2005-12-22 Hitachi, Ltd. Imaging apparatus
US20080024596A1 (en) * 2006-07-25 2008-01-31 Hsiang-Tsun Li Stereo image and video capturing device with dual digital sensors and methods of using the same
US20080309774A1 (en) * 2007-06-15 2008-12-18 Microsoft Corporation Multiple sensor input data synthesis
US20090026267A1 (en) * 2007-06-04 2009-01-29 Hand Held Products, Inc. Indicia reading terminal processing plurality of frames of image data responsively to trigger signal activation

Family Cites Families (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8030A (en) * 1851-04-08 Celia b
US4323925A (en) 1980-07-07 1982-04-06 Avco Everett Research Laboratory, Inc. Method and apparatus for arraying image sensor modules
US4523226A (en) * 1982-01-27 1985-06-11 Stereographics Corporation Stereoscopic television system
JPH0530423A (ja) 1991-07-25 1993-02-05 Nippon Television Network Corp ワイド画面ｅｄｔｖ信号用映像切替装置
WO1994018789A1 (en) 1993-02-11 1994-08-18 Polycom, Inc. Resolution enhancement system
JPH06350937A (ja) 1993-06-14 1994-12-22 Pioneer Electron Corp 画像合成再生装置
EP0631250B1 (en) 1993-06-21 2002-03-20 Nippon Telegraph And Telephone Corporation Method and apparatus for reconstructing three-dimensional objects
WO1995006283A1 (en) 1993-08-24 1995-03-02 Downs Roger C Topography processor system
US5870137A (en) 1993-12-29 1999-02-09 Leica Mikroskopie Systeme Ag Method and device for displaying stereoscopic video images
US5786850A (en) 1994-02-17 1998-07-28 Ultrak, Inc. Multiple room portable camera system
US6064355A (en) 1994-05-24 2000-05-16 Texas Instruments Incorporated Method and apparatus for playback with a virtual reality system
DE19542308A1 (de) 1994-11-25 1996-05-30 Zeiss Carl Fa Verfahren und Vorrichtung zur Darstellung dreidimensionaler Videobilder
JP3771954B2 (ja) 1995-08-04 2006-05-10 ソニー株式会社 画像表示制御装置および方法
US6108005A (en) 1996-08-30 2000-08-22 Space Corporation Method for producing a synthesized stereoscopic image
UA22127C2 (uk) 1996-09-10 1998-04-30 Сергій Іванович Мірошніченко Телевізійhа система високої розрізhяльhої здатhості
US6278480B1 (en) 1997-02-07 2001-08-21 Canon Kabushiki Kaisha Compound eye camera system
US6137580A (en) * 1998-09-22 2000-10-24 Creo Srl Autofocus system
AU2487000A (en) 1998-12-30 2000-07-31 Chequemate International Inc. System and method for recording and broadcasting three-dimensional video
US6611289B1 (en) 1999-01-15 2003-08-26 Yanbin Yu Digital cameras using multiple sensors with multiple lenses
US7050085B1 (en) 2000-10-26 2006-05-23 Imove, Inc. System and method for camera calibration
JP3619063B2 (ja) 1999-07-08 2005-02-09 キヤノン株式会社 立体画像処理装置、その方法、立体視パラメータ設定装置、その方法そしてコンピュータプログラム記憶媒体
GB2355612A (en) 1999-10-19 2001-04-25 Tricorder Technology Plc Image processing arrangement producing a combined output signal from input video signals.
US6950121B2 (en) 1999-12-28 2005-09-27 Vrex, Inc. 3D camera
US6765568B2 (en) 2000-06-12 2004-07-20 Vrex, Inc. Electronic stereoscopic media delivery system
JP2001069530A (ja) 2000-07-28 2001-03-16 Fujitsu Ltd 立体映像高能率符号化装置
EP1384046B1 (en) 2001-05-04 2018-10-03 Vexcel Imaging GmbH Digital camera for and method of obtaining overlapping images
US7002618B2 (en) 2001-06-01 2006-02-21 Stereographics Corporation Plano-stereoscopic DVD movie
JP3789794B2 (ja) 2001-09-26 2006-06-28 三洋電機株式会社 立体画像処理方法、装置、およびシステム
US7277121B2 (en) 2001-08-29 2007-10-02 Sanyo Electric Co., Ltd. Stereoscopic image processing and display system
US7319720B2 (en) 2002-01-28 2008-01-15 Microsoft Corporation Stereoscopic video
CA2380105A1 (en) 2002-04-09 2003-10-09 Nicholas Routhier Process and system for encoding and playback of stereoscopic video sequences
WO2004001667A2 (en) * 2002-06-21 2003-12-31 The Trustees Of Columbia University In The City Of New York Systems and methods for de-blurring motion blurred images
US7804995B2 (en) 2002-07-02 2010-09-28 Reald Inc. Stereoscopic format converter
JP4252105B2 (ja) 2002-12-13 2009-04-08 シャープ株式会社 画像データ作成装置及び画像データ再生装置
KR100477801B1 (ko) 2002-12-26 2005-03-22 한국전자통신연구원 3차원 영상정보 기술장치와 그 방법 및 이를 이용한 3차원영상정보 검색장치 및 그 방법
SE0300428D0 (sv) 2003-02-17 2003-02-17 Axis Ab Digital camera having panning and/or tilting functionality
KR100771299B1 (ko) 2003-06-26 2007-10-29 삼성전기주식회사 색 보간용 라인 버퍼 및 라인 데이터 제공 방법
SE0302065D0 (sv) * 2003-07-14 2003-07-14 Stefan Carlsson Video - method and apparatus
RU2250504C1 (ru) 2003-07-18 2005-04-20 Мироничев Сергей Юрьевич Способ многоканального видеоаудионаблюдения и реализующая его интегрированная высокочастотная система
US7112774B2 (en) 2003-10-09 2006-09-26 Avago Technologies Sensor Ip (Singapore) Pte. Ltd CMOS stereo imaging system and method
JP4483261B2 (ja) 2003-10-24 2010-06-16 ソニー株式会社 立体視画像処理装置
JP4293053B2 (ja) * 2004-05-19 2009-07-08 ソニー株式会社 撮像装置及び方法
JP4424088B2 (ja) * 2004-06-25 2010-03-03 株式会社日立製作所 撮像装置
EP1812968B1 (en) 2004-08-25 2019-01-16 Callahan Cellular L.L.C. Apparatus for multiple camera devices and method of operating same
KR100782811B1 (ko) 2005-02-04 2007-12-06 삼성전자주식회사 영상의 주파수 특성에 따라 포맷을 달리하는 스테레오 영상 합성 방법 및 장치와, 그 영상의 송신 및 수신 방법과, 그 영상의 재생 방법 및 장치
US7561191B2 (en) 2005-02-18 2009-07-14 Eastman Kodak Company Camera phone using multiple lenses and image sensors to provide an extended zoom range
US7206136B2 (en) 2005-02-18 2007-04-17 Eastman Kodak Company Digital camera using multiple lenses and image sensors to provide an extended zoom range
US7577881B1 (en) 2005-05-10 2009-08-18 Ikanos Communications Inc. Method and apparatus for an interleaver
JP4665166B2 (ja) 2005-06-29 2011-04-06 ソニー株式会社 ステレオ画像処理装置、ステレオ画像処理方法およびステレオ画像処理用プログラム
US20070146478A1 (en) 2005-07-14 2007-06-28 Butler-Smith Bernard J Stereoscopic 3D rig calibration and viewing device
WO2007014293A1 (en) 2005-07-25 2007-02-01 The Regents Of The University Of California Digital imaging system and method to produce mosaic images
JP4630149B2 (ja) 2005-07-26 2011-02-09 シャープ株式会社 画像処理装置
JP4185926B2 (ja) 2005-08-26 2008-11-26 ファナック株式会社 ロボット協調制御方法及びシステム
KR101185870B1 (ko) 2005-10-12 2012-09-25 삼성전자주식회사 3ｄ 입체 영상 처리 장치 및 방법
CN101025475A (zh) 2005-10-17 2007-08-29 威盛电子股份有限公司 立体三维影像显示系统与方法
JP4199238B2 (ja) * 2006-01-11 2008-12-17 パナソニック株式会社 撮影システム
US20070248260A1 (en) 2006-04-20 2007-10-25 Nokia Corporation Supporting a 3D presentation
US7701439B2 (en) 2006-07-13 2010-04-20 Northrop Grumman Corporation Gesture recognition simulation system and method
KR101311896B1 (ko) 2006-11-14 2013-10-14 삼성전자주식회사 입체 영상의 변위 조정방법 및 이를 적용한 입체 영상장치
US8330801B2 (en) 2006-12-22 2012-12-11 Qualcomm Incorporated Complexity-adaptive 2D-to-3D video sequence conversion
US8520079B2 (en) 2007-02-15 2013-08-27 Pictometry International Corp. Event multiplexer for managing the capture of images
EP1988713A1 (en) * 2007-04-30 2008-11-05 STMicroelectronics (Research & Development) Limited Image processing apparatus and method using padding data
JP4720785B2 (ja) 2007-05-21 2011-07-13 富士フイルム株式会社 撮像装置、画像再生装置、撮像方法及びプログラム
KR101313740B1 (ko) 2007-10-08 2013-10-15 주식회사 스테레오피아 원소스 멀티유즈 스테레오 카메라 및 스테레오 영상 컨텐츠제작방법
CN101286000A (zh) * 2008-05-30 2008-10-15 钟磊 N镜头立体数码相机
WO2010029040A2 (en) * 2008-09-11 2010-03-18 University Of Malta Method and apparatus for generating and transmitting synchronized video data
JP4793451B2 (ja) 2009-01-21 2011-10-12 ソニー株式会社 信号処理装置、画像表示装置、信号処理方法およびコンピュータプログラム
GB0912970D0 (en) 2009-07-27 2009-09-02 St Microelectronics Res & Dev Improvements in or relating to a sensor and sensor system for a camera
US9544661B2 (en) * 2009-09-03 2017-01-10 Lg Electronics Inc. Cable broadcast receiver and 3D video data processing method thereof
US8896668B2 (en) 2010-04-05 2014-11-25 Qualcomm Incorporated Combining data from multiple image sensors
US20110242342A1 (en) 2010-04-05 2011-10-06 Qualcomm Incorporated Combining data from multiple image sensors

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07255068A (ja) * 1994-03-14 1995-10-03 Sony Corp テレビジョン方式及び表示装置
US5995140A (en) * 1995-08-28 1999-11-30 Ultrak, Inc. System and method for synchronization of multiple video cameras
US6750904B1 (en) * 1998-10-31 2004-06-15 International Business Machines Corporation Camera system for three dimensional images and video
US20040075741A1 (en) * 2002-10-17 2004-04-22 Berkey Thomas F. Multiple camera image multiplexer
US20040196378A1 (en) * 2003-02-17 2004-10-07 Axis Ab., A Swedish Corporation Method and apparatus for panning and tilting a camera
US20050280702A1 (en) * 2004-06-17 2005-12-22 Hitachi, Ltd. Imaging apparatus
US20080024596A1 (en) * 2006-07-25 2008-01-31 Hsiang-Tsun Li Stereo image and video capturing device with dual digital sensors and methods of using the same
US20090026267A1 (en) * 2007-06-04 2009-01-29 Hand Held Products, Inc. Indicia reading terminal processing plurality of frames of image data responsively to trigger signal activation
US20080309774A1 (en) * 2007-06-15 2008-12-18 Microsoft Corporation Multiple sensor input data synthesis

Cited By (294)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9576369B2 (en) 2008-05-20 2017-02-21 Fotonation Cayman Limited Systems and methods for generating depth maps using images captured by camera arrays incorporating cameras having different fields of view
US9049381B2 (en) 2008-05-20 2015-06-02 Pelican Imaging Corporation Systems and methods for normalizing image data captured by camera arrays
US11412158B2 (en) 2008-05-20 2022-08-09 Fotonation Limited Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US9060124B2 (en) 2008-05-20 2015-06-16 Pelican Imaging Corporation Capturing and processing of images using non-monolithic camera arrays
US10027901B2 (en) 2008-05-20 2018-07-17 Fotonation Cayman Limited Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras
US9077893B2 (en) 2008-05-20 2015-07-07 Pelican Imaging Corporation Capturing and processing of images captured by non-grid camera arrays
US10142560B2 (en) 2008-05-20 2018-11-27 Fotonation Limited Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US9060142B2 (en) 2008-05-20 2015-06-16 Pelican Imaging Corporation Capturing and processing of images captured by camera arrays including heterogeneous optics
US9485496B2 (en) 2008-05-20 2016-11-01 Pelican Imaging Corporation Systems and methods for measuring depth using images captured by a camera array including cameras surrounding a central camera
US11792538B2 (en) 2008-05-20 2023-10-17 Adeia Imaging Llc Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US9060121B2 (en) 2008-05-20 2015-06-16 Pelican Imaging Corporation Capturing and processing of images captured by camera arrays including cameras dedicated to sampling luma and cameras dedicated to sampling chroma
US8885059B1 (en) 2008-05-20 2014-11-11 Pelican Imaging Corporation Systems and methods for measuring depth using images captured by camera arrays
US9060120B2 (en) 2008-05-20 2015-06-16 Pelican Imaging Corporation Systems and methods for generating depth maps using images captured by camera arrays
US9055213B2 (en) 2008-05-20 2015-06-09 Pelican Imaging Corporation Systems and methods for measuring depth using images captured by monolithic camera arrays including at least one bayer camera
US8896719B1 (en) 2008-05-20 2014-11-25 Pelican Imaging Corporation Systems and methods for parallax measurement using camera arrays incorporating 3 x 3 camera configurations
US9094661B2 (en) 2008-05-20 2015-07-28 Pelican Imaging Corporation Systems and methods for generating depth maps using a set of images containing a baseline image
US9055233B2 (en) 2008-05-20 2015-06-09 Pelican Imaging Corporation Systems and methods for synthesizing higher resolution images using a set of images containing a baseline image
US9049391B2 (en) 2008-05-20 2015-06-02 Pelican Imaging Corporation Capturing and processing of near-IR images including occlusions using camera arrays incorporating near-IR light sources
US9049411B2 (en) 2008-05-20 2015-06-02 Pelican Imaging Corporation Camera arrays incorporating 3×3 imager configurations
US12041360B2 (en) 2008-05-20 2024-07-16 Adeia Imaging Llc Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US9124815B2 (en) 2008-05-20 2015-09-01 Pelican Imaging Corporation Capturing and processing of images including occlusions captured by arrays of luma and chroma cameras
US9712759B2 (en) 2008-05-20 2017-07-18 Fotonation Cayman Limited Systems and methods for generating depth maps using a camera arrays incorporating monochrome and color cameras
US9049367B2 (en) 2008-05-20 2015-06-02 Pelican Imaging Corporation Systems and methods for synthesizing higher resolution images using images captured by camera arrays
US9188765B2 (en) 2008-05-20 2015-11-17 Pelican Imaging Corporation Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US9191580B2 (en) 2008-05-20 2015-11-17 Pelican Imaging Corporation Capturing and processing of images including occlusions captured by camera arrays
US9749547B2 (en) 2008-05-20 2017-08-29 Fotonation Cayman Limited Capturing and processing of images using camera array incorperating Bayer cameras having different fields of view
US12022207B2 (en) 2008-05-20 2024-06-25 Adeia Imaging Llc Capturing and processing of images including occlusions focused on an image sensor by a lens stack array
US9041823B2 (en) 2008-05-20 2015-05-26 Pelican Imaging Corporation Systems and methods for performing post capture refocus using images captured by camera arrays
US9049390B2 (en) 2008-05-20 2015-06-02 Pelican Imaging Corporation Capturing and processing of images captured by arrays including polychromatic cameras
US9041829B2 (en) 2008-05-20 2015-05-26 Pelican Imaging Corporation Capturing and processing of high dynamic range images using camera arrays
US9264610B2 (en) 2009-11-20 2016-02-16 Pelican Imaging Corporation Capturing and processing of images including occlusions captured by heterogeneous camera arrays
US10306120B2 (en) 2009-11-20 2019-05-28 Fotonation Limited Capturing and processing of images captured by camera arrays incorporating cameras with telephoto and conventional lenses to generate depth maps
US9001227B2 (en) 2010-04-05 2015-04-07 Qualcomm Incorporated Combining data from multiple image sensors
US8896668B2 (en) 2010-04-05 2014-11-25 Qualcomm Incorporated Combining data from multiple image sensors
US10455168B2 (en) 2010-05-12 2019-10-22 Fotonation Limited Imager array interfaces
US8928793B2 (en) 2010-05-12 2015-01-06 Pelican Imaging Corporation Imager array interfaces
US9936148B2 (en) 2010-05-12 2018-04-03 Fotonation Cayman Limited Imager array interfaces
US8890938B2 (en) * 2010-11-04 2014-11-18 Samsung Electronics Co., Ltd. Digital photographing apparatus and method of controlling the same
US20120113230A1 (en) * 2010-11-04 2012-05-10 Samsung Electronics Co., Ltd. Digital photographing apparatus and method of controlling the same
US11423513B2 (en) 2010-12-14 2022-08-23 Fotonation Limited Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers
US9047684B2 (en) 2010-12-14 2015-06-02 Pelican Imaging Corporation Systems and methods for synthesizing high resolution images using a set of geometrically registered images
US11875475B2 (en) 2010-12-14 2024-01-16 Adeia Imaging Llc Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers
US10366472B2 (en) 2010-12-14 2019-07-30 Fotonation Limited Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers
US9041824B2 (en) 2010-12-14 2015-05-26 Pelican Imaging Corporation Systems and methods for dynamic refocusing of high resolution images generated using images captured by a plurality of imagers
US9361662B2 (en) 2010-12-14 2016-06-07 Pelican Imaging Corporation Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers
US9197821B2 (en) * 2011-05-11 2015-11-24 Pelican Imaging Corporation Systems and methods for transmitting and receiving array camera image data
US10218889B2 (en) 2011-05-11 2019-02-26 Fotonation Limited Systems and methods for transmitting and receiving array camera image data
US9866739B2 (en) 2011-05-11 2018-01-09 Fotonation Cayman Limited Systems and methods for transmitting and receiving array camera image data
US20130057710A1 (en) * 2011-05-11 2013-03-07 Pelican Imaging Corporation Systems and methods for transmitting and receiving array camera image data
US20190174040A1 (en) * 2011-05-11 2019-06-06 Fotonation Limited Systems and Methods for Transmitting and Receiving Array Camera Image Data
US10742861B2 (en) * 2011-05-11 2020-08-11 Fotonation Limited Systems and methods for transmitting and receiving array camera image data
US8692893B2 (en) * 2011-05-11 2014-04-08 Pelican Imaging Corporation Systems and methods for transmitting and receiving array camera image data
US20140218546A1 (en) * 2011-05-11 2014-08-07 Pelican Imaging Corporation Systems and Methods for Transmitting and Receiving Array Camera Image Data
US9338436B2 (en) * 2011-06-07 2016-05-10 Sony Corporation Imaging device and imaging method
US10045009B2 (en) 2011-06-07 2018-08-07 Sony Corporation Imaging device and imaging control method with adjustable frame frequency
US10194141B2 (en) 2011-06-07 2019-01-29 Sony Corporation Imaging device and imaging method
US10595009B2 (en) 2011-06-07 2020-03-17 Sony Corporation Imaging device and imaging method
US20120314101A1 (en) * 2011-06-07 2012-12-13 Ooba Yuuji Imaging device and imaging method
US9516222B2 (en) 2011-06-28 2016-12-06 Kip Peli P1 Lp Array cameras incorporating monolithic array camera modules with high MTF lens stacks for capture of images used in super-resolution processing
US9578237B2 (en) 2011-06-28 2017-02-21 Fotonation Cayman Limited Array cameras incorporating optics with modulation transfer functions greater than sensor Nyquist frequency for capture of images used in super-resolution processing
US9794476B2 (en) 2011-09-19 2017-10-17 Fotonation Cayman Limited Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures
US10375302B2 (en) 2011-09-19 2019-08-06 Fotonation Limited Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures
US12052409B2 (en) 2011-09-28 2024-07-30 Adela Imaging LLC Systems and methods for encoding image files containing depth maps stored as metadata
US9036931B2 (en) 2011-09-28 2015-05-19 Pelican Imaging Corporation Systems and methods for decoding structured light field image files
US9031335B2 (en) 2011-09-28 2015-05-12 Pelican Imaging Corporation Systems and methods for encoding light field image files having depth and confidence maps
US9031343B2 (en) 2011-09-28 2015-05-12 Pelican Imaging Corporation Systems and methods for encoding light field image files having a depth map
US9536166B2 (en) 2011-09-28 2017-01-03 Kip Peli P1 Lp Systems and methods for decoding image files containing depth maps stored as metadata
US9031342B2 (en) 2011-09-28 2015-05-12 Pelican Imaging Corporation Systems and methods for encoding refocusable light field image files
US9036928B2 (en) 2011-09-28 2015-05-19 Pelican Imaging Corporation Systems and methods for encoding structured light field image files
US9025895B2 (en) 2011-09-28 2015-05-05 Pelican Imaging Corporation Systems and methods for decoding refocusable light field image files
US9864921B2 (en) 2011-09-28 2018-01-09 Fotonation Cayman Limited Systems and methods for encoding image files containing depth maps stored as metadata
US10430682B2 (en) 2011-09-28 2019-10-01 Fotonation Limited Systems and methods for decoding image files containing depth maps stored as metadata
US9025894B2 (en) 2011-09-28 2015-05-05 Pelican Imaging Corporation Systems and methods for decoding light field image files having depth and confidence maps
US9811753B2 (en) 2011-09-28 2017-11-07 Fotonation Cayman Limited Systems and methods for encoding light field image files
US11729365B2 (en) 2011-09-28 2023-08-15 Adela Imaging LLC Systems and methods for encoding image files containing depth maps stored as metadata
US9042667B2 (en) 2011-09-28 2015-05-26 Pelican Imaging Corporation Systems and methods for decoding light field image files using a depth map
US20180197035A1 (en) 2011-09-28 2018-07-12 Fotonation Cayman Limited Systems and Methods for Encoding Image Files Containing Depth Maps Stored as Metadata
US10019816B2 (en) 2011-09-28 2018-07-10 Fotonation Cayman Limited Systems and methods for decoding image files containing depth maps stored as metadata
US10984276B2 (en) 2011-09-28 2021-04-20 Fotonation Limited Systems and methods for encoding image files containing depth maps stored as metadata
US10275676B2 (en) 2011-09-28 2019-04-30 Fotonation Limited Systems and methods for encoding image files containing depth maps stored as metadata
US9454245B2 (en) 2011-11-01 2016-09-27 Qualcomm Incorporated System and method for improving orientation data
US9995575B2 (en) 2011-11-01 2018-06-12 Qualcomm Incorporated System and method for improving orientation data
US9495018B2 (en) 2011-11-01 2016-11-15 Qualcomm Incorporated System and method for improving orientation data
US9785254B2 (en) 2011-11-01 2017-10-10 Qualcomm Incorporated System and method for improving orientation data
DE102012003127A1 (de) * 2012-02-16 2013-09-05 Leica Camera Ag Verfahren für eine Autofokuseinrichtung
US9754422B2 (en) 2012-02-21 2017-09-05 Fotonation Cayman Limited Systems and method for performing depth based image editing
US10311649B2 (en) 2012-02-21 2019-06-04 Fotonation Limited Systems and method for performing depth based image editing
US9412206B2 (en) 2012-02-21 2016-08-09 Pelican Imaging Corporation Systems and methods for the manipulation of captured light field image data
US9253450B2 (en) 2012-04-18 2016-02-02 Utc Fire & Security Americas Corporation, Inc. Total bus surveillance system
US9210392B2 (en) 2012-05-01 2015-12-08 Pelican Imaging Coporation Camera modules patterned with pi filter groups
US9706132B2 (en) 2012-05-01 2017-07-11 Fotonation Cayman Limited Camera modules patterned with pi filter groups
US20130327831A1 (en) * 2012-06-11 2013-12-12 Datalogic ADC, Inc. Dynamic imager switching
US10334241B2 (en) 2012-06-28 2019-06-25 Fotonation Limited Systems and methods for detecting defective camera arrays and optic arrays
US9807382B2 (en) 2012-06-28 2017-10-31 Fotonation Cayman Limited Systems and methods for detecting defective camera arrays and optic arrays
US9766380B2 (en) 2012-06-30 2017-09-19 Fotonation Cayman Limited Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors
US11022725B2 (en) 2012-06-30 2021-06-01 Fotonation Limited Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors
US10261219B2 (en) 2012-06-30 2019-04-16 Fotonation Limited Systems and methods for manufacturing camera modules using active alignment of lens stack arrays and sensors
US9858673B2 (en) 2012-08-21 2018-01-02 Fotonation Cayman Limited Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints
US9240049B2 (en) 2012-08-21 2016-01-19 Pelican Imaging Corporation Systems and methods for measuring depth using an array of independently controllable cameras
US9235900B2 (en) 2012-08-21 2016-01-12 Pelican Imaging Corporation Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints
US12002233B2 (en) 2012-08-21 2024-06-04 Adeia Imaging Llc Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints
US9129377B2 (en) 2012-08-21 2015-09-08 Pelican Imaging Corporation Systems and methods for measuring depth based upon occlusion patterns in images
US9123117B2 (en) 2012-08-21 2015-09-01 Pelican Imaging Corporation Systems and methods for generating depth maps and corresponding confidence maps indicating depth estimation reliability
US9123118B2 (en) 2012-08-21 2015-09-01 Pelican Imaging Corporation System and methods for measuring depth using an array camera employing a bayer filter
US9147254B2 (en) 2012-08-21 2015-09-29 Pelican Imaging Corporation Systems and methods for measuring depth in the presence of occlusions using a subset of images
US10380752B2 (en) 2012-08-21 2019-08-13 Fotonation Limited Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints
US9813616B2 (en) 2012-08-23 2017-11-07 Fotonation Cayman Limited Feature based high resolution motion estimation from low resolution images captured using an array source
US10462362B2 (en) 2012-08-23 2019-10-29 Fotonation Limited Feature based high resolution motion estimation from low resolution images captured using an array source
US9214013B2 (en) 2012-09-14 2015-12-15 Pelican Imaging Corporation Systems and methods for correcting user identified artifacts in light field images
CN103685978A (zh) * 2012-09-26 2014-03-26 全视技术有限公司 用于同步多个视频传感器的装置及方法
US20140085497A1 (en) * 2012-09-26 2014-03-27 Omnivision Technologies, Inc. Systems And Methods For Sychronizing Multiple Video Sensors
US20140092439A1 (en) * 2012-09-28 2014-04-03 Scott A. Krig Encoding images using a 3d mesh of polygons and corresponding textures
US10390005B2 (en) 2012-09-28 2019-08-20 Fotonation Limited Generating images from light fields utilizing virtual viewpoints
US10104364B2 (en) * 2012-10-05 2018-10-16 Qualcomm Incorporated Method and apparatus for bus sharing by multiple imaging sensors
US20160182890A1 (en) * 2012-10-05 2016-06-23 Qualcomm Incorporated Method and apparatus for bus sharing by multiple imaging sensors
US9143711B2 (en) 2012-11-13 2015-09-22 Pelican Imaging Corporation Systems and methods for array camera focal plane control
US9749568B2 (en) 2012-11-13 2017-08-29 Fotonation Cayman Limited Systems and methods for array camera focal plane control
US9195254B2 (en) * 2012-12-21 2015-11-24 Qualcomm, Incorporated Method and apparatus for multi-level de-emphasis
JP2015535673A (ja) * 2012-12-27 2015-12-14 インテル・コーポレーション ゲットアンドセットアーキテクチャに基づくトランスポートメカニズム内のコマンド実行
US9600296B2 (en) 2012-12-27 2017-03-21 Intel Corporation Executing a command within a transport mechanism based on a get and set architecture
JP2016028521A (ja) * 2012-12-27 2016-02-25 インテル・コーポレーション ゲットアンドセットアーキテクチャに基づくトランスポートメカニズム内のコマンド実行
US9547160B2 (en) 2013-01-05 2017-01-17 Light Labs Inc. Methods and apparatus for capturing and/or processing images
US9568713B2 (en) 2013-01-05 2017-02-14 Light Labs Inc. Methods and apparatus for using multiple optical chains in parallel to support separate color-capture
US9671595B2 (en) 2013-01-05 2017-06-06 Light Labs Inc. Methods and apparatus for using multiple optical chains in paralell
US9690079B2 (en) 2013-01-05 2017-06-27 Light Labs Inc. Camera methods and apparatus using optical chain modules which alter the direction of received light
US8908041B2 (en) 2013-01-15 2014-12-09 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US10764517B2 (en) 2013-01-15 2020-09-01 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US10200638B2 (en) 2013-01-15 2019-02-05 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US9286522B2 (en) 2013-01-15 2016-03-15 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US9854185B2 (en) 2013-01-15 2017-12-26 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US9531966B2 (en) 2013-01-15 2016-12-27 Mobileye Vision Technologies Ltd. Stereo assist with rolling shutters
US10009538B2 (en) 2013-02-21 2018-06-26 Fotonation Cayman Limited Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information
US9743051B2 (en) 2013-02-24 2017-08-22 Fotonation Cayman Limited Thin form factor computational array cameras and modular array cameras
US9253380B2 (en) 2013-02-24 2016-02-02 Pelican Imaging Corporation Thin form factor computational array cameras and modular array cameras
US9374512B2 (en) 2013-02-24 2016-06-21 Pelican Imaging Corporation Thin form factor computational array cameras and modular array cameras
US9774831B2 (en) 2013-02-24 2017-09-26 Fotonation Cayman Limited Thin form factor computational array cameras and modular array cameras
US9774789B2 (en) 2013-03-08 2017-09-26 Fotonation Cayman Limited Systems and methods for high dynamic range imaging using array cameras
US9917998B2 (en) 2013-03-08 2018-03-13 Fotonation Cayman Limited Systems and methods for measuring scene information while capturing images using array cameras
US10958892B2 (en) 2013-03-10 2021-03-23 Fotonation Limited System and methods for calibration of an array camera
US11272161B2 (en) 2013-03-10 2022-03-08 Fotonation Limited System and methods for calibration of an array camera
US11570423B2 (en) 2013-03-10 2023-01-31 Adeia Imaging Llc System and methods for calibration of an array camera
US11985293B2 (en) 2013-03-10 2024-05-14 Adeia Imaging Llc System and methods for calibration of an array camera
US9986224B2 (en) 2013-03-10 2018-05-29 Fotonation Cayman Limited System and methods for calibration of an array camera
US10225543B2 (en) 2013-03-10 2019-03-05 Fotonation Limited System and methods for calibration of an array camera
US9124864B2 (en) 2013-03-10 2015-09-01 Pelican Imaging Corporation System and methods for calibration of an array camera
US9800856B2 (en) 2013-03-13 2017-10-24 Fotonation Cayman Limited Systems and methods for synthesizing images from image data captured by an array camera using restricted depth of field depth maps in which depth estimation precision varies
US9733486B2 (en) 2013-03-13 2017-08-15 Fotonation Cayman Limited Systems and methods for controlling aliasing in images captured by an array camera for use in super-resolution processing
US9888194B2 (en) 2013-03-13 2018-02-06 Fotonation Cayman Limited Array camera architecture implementing quantum film image sensors
US9741118B2 (en) 2013-03-13 2017-08-22 Fotonation Cayman Limited System and methods for calibration of an array camera
US10127682B2 (en) 2013-03-13 2018-11-13 Fotonation Limited System and methods for calibration of an array camera
US10091405B2 (en) 2013-03-14 2018-10-02 Fotonation Cayman Limited Systems and methods for reducing motion blur in images or video in ultra low light with array cameras
US10412314B2 (en) 2013-03-14 2019-09-10 Fotonation Limited Systems and methods for photometric normalization in array cameras
US9100586B2 (en) 2013-03-14 2015-08-04 Pelican Imaging Corporation Systems and methods for photometric normalization in array cameras
US9787911B2 (en) 2013-03-14 2017-10-10 Fotonation Cayman Limited Systems and methods for photometric normalization in array cameras
US9578259B2 (en) 2013-03-14 2017-02-21 Fotonation Cayman Limited Systems and methods for reducing motion blur in images or video in ultra low light with array cameras
US10547772B2 (en) 2013-03-14 2020-01-28 Fotonation Limited Systems and methods for reducing motion blur in images or video in ultra low light with array cameras
US10122993B2 (en) 2013-03-15 2018-11-06 Fotonation Limited Autofocus system for a conventional camera that uses depth information from an array camera
US10455218B2 (en) 2013-03-15 2019-10-22 Fotonation Limited Systems and methods for estimating depth using stereo array cameras
US10542208B2 (en) 2013-03-15 2020-01-21 Fotonation Limited Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information
US9602805B2 (en) 2013-03-15 2017-03-21 Fotonation Cayman Limited Systems and methods for estimating depth using ad hoc stereo array cameras
US10182216B2 (en) 2013-03-15 2019-01-15 Fotonation Limited Extended color processing on pelican array cameras
US9800859B2 (en) 2013-03-15 2017-10-24 Fotonation Cayman Limited Systems and methods for estimating depth using stereo array cameras
US10674138B2 (en) 2013-03-15 2020-06-02 Fotonation Limited Autofocus system for a conventional camera that uses depth information from an array camera
US9438888B2 (en) 2013-03-15 2016-09-06 Pelican Imaging Corporation Systems and methods for stereo imaging with camera arrays
US9497429B2 (en) 2013-03-15 2016-11-15 Pelican Imaging Corporation Extended color processing on pelican array cameras
US9955070B2 (en) 2013-03-15 2018-04-24 Fotonation Cayman Limited Systems and methods for synthesizing high resolution images using image deconvolution based on motion and depth information
US9497370B2 (en) 2013-03-15 2016-11-15 Pelican Imaging Corporation Array camera architecture implementing quantum dot color filters
US10638099B2 (en) 2013-03-15 2020-04-28 Fotonation Limited Extended color processing on pelican array cameras
US9633442B2 (en) 2013-03-15 2017-04-25 Fotonation Cayman Limited Array cameras including an array camera module augmented with a separate camera
US20150009351A1 (en) * 2013-07-04 2015-01-08 Olympus Corporation Image acquisition apparatus
US9571728B2 (en) * 2013-07-04 2017-02-14 Olympus Corporation Image acquisition apparatus
US20160103209A1 (en) * 2013-07-16 2016-04-14 Fujifilm Corporation Imaging device and three-dimensional-measurement device
US10540806B2 (en) 2013-09-27 2020-01-21 Fotonation Limited Systems and methods for depth-assisted perspective distortion correction
US9898856B2 (en) 2013-09-27 2018-02-20 Fotonation Cayman Limited Systems and methods for depth-assisted perspective distortion correction
US9325906B2 (en) 2013-10-18 2016-04-26 The Lightco Inc. Methods and apparatus relating to a thin camera device
US9557519B2 (en) 2013-10-18 2017-01-31 Light Labs Inc. Methods and apparatus for implementing a camera device supporting a number of different focal lengths
US9374514B2 (en) 2013-10-18 2016-06-21 The Lightco Inc. Methods and apparatus relating to a camera including multiple optical chains
US9749511B2 (en) 2013-10-18 2017-08-29 Light Labs Inc. Methods and apparatus relating to a camera including multiple optical chains
US9578252B2 (en) 2013-10-18 2017-02-21 Light Labs Inc. Methods and apparatus for capturing images using optical chains and/or for using captured images
US10038860B2 (en) * 2013-10-18 2018-07-31 Light Labs Inc. Methods and apparatus for controlling sensors to capture images in a synchronized manner
US9423588B2 (en) 2013-10-18 2016-08-23 The Lightco Inc. Methods and apparatus for supporting zoom operations
US9563033B2 (en) 2013-10-18 2017-02-07 Light Labs Inc. Methods and apparatus for capturing images and/or for using captured images
US9451171B2 (en) 2013-10-18 2016-09-20 The Lightco Inc. Zoom related methods and apparatus
US9557520B2 (en) * 2013-10-18 2017-01-31 Light Labs Inc. Synchronized image capture methods and apparatus
US9851527B2 (en) 2013-10-18 2017-12-26 Light Labs Inc. Methods and apparatus for capturing and/or combining images
US10009530B2 (en) * 2013-10-18 2018-06-26 Light Labs Inc. Methods and apparatus for synchronized image capture using camera modules with different focal lengths
US20150296154A1 (en) * 2013-10-18 2015-10-15 The Lightco Inc. Image capture related methods and apparatus
US9544501B2 (en) 2013-10-18 2017-01-10 Light Labs Inc. Methods and apparatus for implementing and/or using a camera device
US10120159B2 (en) 2013-10-18 2018-11-06 Light Labs Inc. Methods and apparatus for supporting zoom operations
US9549127B2 (en) * 2013-10-18 2017-01-17 Light Labs Inc. Image capture control methods and apparatus
US10509208B2 (en) * 2013-10-18 2019-12-17 Light Labs Inc. Methods and apparatus for implementing and/or using a camera device
US20150109468A1 (en) * 2013-10-18 2015-04-23 The Lightco Inc. Image capture control methods and apparatus
US9551854B2 (en) 2013-10-18 2017-01-24 Light Labs Inc. Methods and apparatus for controlling sensors to capture images in a synchronized manner
US9467627B2 (en) 2013-10-26 2016-10-11 The Lightco Inc. Methods and apparatus for use with multiple optical chains
US9736365B2 (en) 2013-10-26 2017-08-15 Light Labs Inc. Zoom related methods and apparatus
US9426365B2 (en) 2013-11-01 2016-08-23 The Lightco Inc. Image stabilization related methods and apparatus
US9686471B2 (en) 2013-11-01 2017-06-20 Light Labs Inc. Methods and apparatus relating to image stabilization
US9185276B2 (en) 2013-11-07 2015-11-10 Pelican Imaging Corporation Methods of manufacturing array camera modules incorporating independently aligned lens stacks
US9924092B2 (en) 2013-11-07 2018-03-20 Fotonation Cayman Limited Array cameras incorporating independently aligned lens stacks
US9264592B2 (en) 2013-11-07 2016-02-16 Pelican Imaging Corporation Array camera modules incorporating independently aligned lens stacks
US9426343B2 (en) 2013-11-07 2016-08-23 Pelican Imaging Corporation Array cameras incorporating independently aligned lens stacks
US10767981B2 (en) 2013-11-18 2020-09-08 Fotonation Limited Systems and methods for estimating depth from projected texture using camera arrays
US11486698B2 (en) 2013-11-18 2022-11-01 Fotonation Limited Systems and methods for estimating depth from projected texture using camera arrays
US10119808B2 (en) 2013-11-18 2018-11-06 Fotonation Limited Systems and methods for estimating depth from projected texture using camera arrays
US9456134B2 (en) 2013-11-26 2016-09-27 Pelican Imaging Corporation Array camera configurations incorporating constituent array cameras and constituent cameras
US9426361B2 (en) 2013-11-26 2016-08-23 Pelican Imaging Corporation Array camera configurations incorporating multiple constituent array cameras
US9813617B2 (en) 2013-11-26 2017-11-07 Fotonation Cayman Limited Array camera configurations incorporating constituent array cameras and constituent cameras
US10708492B2 (en) 2013-11-26 2020-07-07 Fotonation Limited Array camera configurations incorporating constituent array cameras and constituent cameras
US9554031B2 (en) 2013-12-31 2017-01-24 Light Labs Inc. Camera focusing related methods and apparatus
US9979878B2 (en) 2014-02-21 2018-05-22 Light Labs Inc. Intuitive camera user interface methods and apparatus
US9462170B2 (en) 2014-02-21 2016-10-04 The Lightco Inc. Lighting methods and apparatus
US10574905B2 (en) 2014-03-07 2020-02-25 Fotonation Limited System and methods for depth regularization and semiautomatic interactive matting using RGB-D images
US10089740B2 (en) 2014-03-07 2018-10-02 Fotonation Limited System and methods for depth regularization and semiautomatic interactive matting using RGB-D images
CN103916498A (zh) * 2014-03-28 2014-07-09 宁波萨瑞通讯有限公司 串口摄像头和并口摄像头兼容方法
US9247117B2 (en) 2014-04-07 2016-01-26 Pelican Imaging Corporation Systems and methods for correcting for warpage of a sensor array in an array camera module by introducing warpage into a focal plane of a lens stack array
US9521319B2 (en) 2014-06-18 2016-12-13 Pelican Imaging Corporation Array cameras and array camera modules including spectral filters disposed outside of a constituent image sensor
US10191356B2 (en) 2014-07-04 2019-01-29 Light Labs Inc. Methods and apparatus relating to detection and/or indicating a dirty lens condition
US10110794B2 (en) 2014-07-09 2018-10-23 Light Labs Inc. Camera device including multiple optical chains and related methods
CN105335779A (zh) * 2014-08-14 2016-02-17 夏志刚 数卡器及使用数卡器进行数卡的方法
EP3189659A4 (en) * 2014-09-03 2018-05-02 Intel Corporation Imaging architecture for depth camera mode with mode switching
US11546576B2 (en) 2014-09-29 2023-01-03 Adeia Imaging Llc Systems and methods for dynamic calibration of array cameras
US10250871B2 (en) 2014-09-29 2019-04-02 Fotonation Limited Systems and methods for dynamic calibration of array cameras
US9912865B2 (en) 2014-10-17 2018-03-06 Light Labs Inc. Methods and apparatus for supporting burst modes of camera operation
US9912864B2 (en) 2014-10-17 2018-03-06 Light Labs Inc. Methods and apparatus for using a camera device to support multiple modes of operation
US10944911B2 (en) * 2014-10-24 2021-03-09 Texas Instruments Incorporated Image data processing for digital overlap wide dynamic range sensors
US20210160418A1 (en) * 2014-10-24 2021-05-27 Texas Instruments Incorporated Image data processing for digital overlap wide dynamic range sensors
US20160119575A1 (en) * 2014-10-24 2016-04-28 Texas Instruments Incorporated Image data processing for digital overlap wide dynamic range sensors
US11962914B2 (en) * 2014-10-24 2024-04-16 Texas Instruments Incorporated Image data processing for digital overlap wide dynamic range sensors
US9998638B2 (en) 2014-12-17 2018-06-12 Light Labs Inc. Methods and apparatus for implementing and using camera devices
US9544503B2 (en) 2014-12-30 2017-01-10 Light Labs Inc. Exposure control methods and apparatus
US9824427B2 (en) 2015-04-15 2017-11-21 Light Labs Inc. Methods and apparatus for generating a sharp image
US9942474B2 (en) 2015-04-17 2018-04-10 Fotonation Cayman Limited Systems and methods for performing high speed video capture and depth estimation using array cameras
US10091447B2 (en) 2015-04-17 2018-10-02 Light Labs Inc. Methods and apparatus for synchronizing readout of multiple image sensors
US9967535B2 (en) 2015-04-17 2018-05-08 Light Labs Inc. Methods and apparatus for reducing noise in images
WO2016168781A1 (en) * 2015-04-17 2016-10-20 The Lightco Inc. Methods and apparatus for syncronizing readout of multiple image sensors
US10075651B2 (en) 2015-04-17 2018-09-11 Light Labs Inc. Methods and apparatus for capturing images using multiple camera modules in an efficient manner
US9857584B2 (en) 2015-04-17 2018-01-02 Light Labs Inc. Camera device methods, apparatus and components
US9930233B2 (en) 2015-04-22 2018-03-27 Light Labs Inc. Filter mounting methods and apparatus and related camera apparatus
CN107615748A (zh) * 2015-06-10 2018-01-19 索尼公司 图像处理装置和方法
US20180213139A1 (en) * 2015-06-10 2018-07-26 Sony Corporation Image processing apparatus and method
US10129483B2 (en) 2015-06-23 2018-11-13 Light Labs Inc. Methods and apparatus for implementing zoom using one or more moveable camera modules
US10491806B2 (en) 2015-08-03 2019-11-26 Light Labs Inc. Camera device control related methods and apparatus
US10365480B2 (en) 2015-08-27 2019-07-30 Light Labs Inc. Methods and apparatus for implementing and/or using camera devices with one or more light redirection devices
US9749549B2 (en) 2015-10-06 2017-08-29 Light Labs Inc. Methods and apparatus for facilitating selective blurring of one or more image portions
US10003738B2 (en) 2015-12-18 2018-06-19 Light Labs Inc. Methods and apparatus for detecting and/or indicating a blocked sensor or camera module
US10225445B2 (en) 2015-12-18 2019-03-05 Light Labs Inc. Methods and apparatus for providing a camera lens or viewing point indicator
EP3185538A1 (en) * 2015-12-24 2017-06-28 Samsung Electronics Co., Ltd Electronic device and method of controlling the same
US10250842B2 (en) 2015-12-24 2019-04-02 Samsung Electronics Co., Ltd. Electronic device and method of controlling the same
US10701283B2 (en) 2015-12-24 2020-06-30 Samsung Electronics Co., Ltd. Digital photographing apparatus and method of controlling the same
EP3386185A4 (en) * 2015-12-24 2018-11-14 Samsung Electronics Co., Ltd. Electronic device and control method for electronic device
US10306218B2 (en) 2016-03-22 2019-05-28 Light Labs Inc. Camera calibration apparatus and methods
US10825156B2 (en) 2016-05-26 2020-11-03 Sony Semiconductor Solutions Corporation Processing apparatus, image sensor, and system
TWI744315B (zh) * 2016-05-26 2021-11-01 日商索尼半導體解決方案公司 處理裝置，影像感測器及電子系統
EP4375843A1 (en) * 2016-05-26 2024-05-29 Sony Semiconductor Solutions Corporation Processing apparatus, image sensor, and system
WO2017203901A1 (en) * 2016-05-26 2017-11-30 Sony Semiconductor Solutions Corporation Processing apparatus, image sensor, and system
US11557024B2 (en) 2016-05-26 2023-01-17 Sony Semiconductor Solutions Corporation Processing apparatus, image sensor, and system
US9948832B2 (en) 2016-06-22 2018-04-17 Light Labs Inc. Methods and apparatus for synchronized image capture in a device including optical chains with different orientations
WO2018062599A1 (ko) * 2016-09-27 2018-04-05 주식회사 켐트로닉스 Svm 시스템 및 그의 영상입력 및 처리방법
US10911739B2 (en) 2016-10-20 2021-02-02 Hitachi Automotive Systems, Ltd. Camera device
EP3531296A4 (en) * 2016-10-20 2020-05-27 Hitachi Automotive Systems, Ltd. CAMERA DEVICE
US10692262B2 (en) 2017-01-12 2020-06-23 Electronics And Telecommunications Research Institute Apparatus and method for processing information of multiple cameras
US10554958B2 (en) 2017-03-13 2020-02-04 Microsoft Technology Licensing, Llc Systems and methods for interleaving multiple active camera frames
CN109309784A (zh) * 2017-07-28 2019-02-05 展讯通信（上海）有限公司 移动终端
US10818026B2 (en) 2017-08-21 2020-10-27 Fotonation Limited Systems and methods for hybrid depth regularization
US11562498B2 (en) 2017-08-21 2023-01-24 Adela Imaging LLC Systems and methods for hybrid depth regularization
US10482618B2 (en) 2017-08-21 2019-11-19 Fotonation Limited Systems and methods for hybrid depth regularization
US11983893B2 (en) 2017-08-21 2024-05-14 Adeia Imaging Llc Systems and methods for hybrid depth regularization
US20200202196A1 (en) * 2018-12-21 2020-06-25 Waymo Llc Searching an autonomous vehicle sensor data repository
US11861481B2 (en) * 2018-12-21 2024-01-02 Waymo Llc Searching an autonomous vehicle sensor data repository
US11699273B2 (en) 2019-09-17 2023-07-11 Intrinsic Innovation Llc Systems and methods for surface modeling using polarization cues
US11270110B2 (en) 2019-09-17 2022-03-08 Boston Polarimetrics, Inc. Systems and methods for surface modeling using polarization cues
US11525906B2 (en) 2019-10-07 2022-12-13 Intrinsic Innovation Llc Systems and methods for augmentation of sensor systems and imaging systems with polarization
US12099148B2 (en) 2019-10-07 2024-09-24 Intrinsic Innovation Llc Systems and methods for surface normals sensing with polarization
US11982775B2 (en) 2019-10-07 2024-05-14 Intrinsic Innovation Llc Systems and methods for augmentation of sensor systems and imaging systems with polarization
US11302012B2 (en) 2019-11-30 2022-04-12 Boston Polarimetrics, Inc. Systems and methods for transparent object segmentation using polarization cues
US11842495B2 (en) 2019-11-30 2023-12-12 Intrinsic Innovation Llc Systems and methods for transparent object segmentation using polarization cues
US11580667B2 (en) 2020-01-29 2023-02-14 Intrinsic Innovation Llc Systems and methods for characterizing object pose detection and measurement systems
US11797863B2 (en) 2020-01-30 2023-10-24 Intrinsic Innovation Llc Systems and methods for synthesizing data for training statistical models on different imaging modalities including polarized images
US11953700B2 (en) 2020-05-27 2024-04-09 Intrinsic Innovation Llc Multi-aperture polarization optical systems using beam splitters
EP4184913A4 (en) * 2020-07-29 2023-12-20 Huawei Technologies Co., Ltd. FUSION APPARATUS FOR MULTIPLE DATA TRANSMISSION CHANNELS AND ELECTRONIC DEVICE
US11948225B2 (en) * 2020-09-18 2024-04-02 Kabushiki Kaisha Toshiba Image processing apparatus
US20220092726A1 (en) * 2020-09-18 2022-03-24 Kabushiki Kaisha Toshiba Image processing apparatus
US20220164350A1 (en) * 2020-11-25 2022-05-26 Waymo Llc Searching an autonomous vehicle sensor data repository based on context embedding
US12067471B2 (en) * 2020-11-25 2024-08-20 Waymo Llc Searching an autonomous vehicle sensor data repository based on context embedding
US11627257B2 (en) 2020-11-26 2023-04-11 Samsung Electronics Co., Ltd. Electronic device including image sensor having multi-crop function
US12069227B2 (en) 2021-03-10 2024-08-20 Intrinsic Innovation Llc Multi-modal and multi-spectral stereo camera arrays
US12020455B2 (en) 2021-03-10 2024-06-25 Intrinsic Innovation Llc Systems and methods for high dynamic range image reconstruction
US11954886B2 (en) 2021-04-15 2024-04-09 Intrinsic Innovation Llc Systems and methods for six-degree of freedom pose estimation of deformable objects
US11290658B1 (en) 2021-04-15 2022-03-29 Boston Polarimetrics, Inc. Systems and methods for camera exposure control
US11683594B2 (en) 2021-04-15 2023-06-20 Intrinsic Innovation Llc Systems and methods for camera exposure control
US12067746B2 (en) 2021-05-07 2024-08-20 Intrinsic Innovation Llc Systems and methods for using computer vision to pick up small objects
US11689813B2 (en) 2021-07-01 2023-06-27 Intrinsic Innovation Llc Systems and methods for high dynamic range imaging using crossed polarizers
US20230275716A1 (en) * 2022-02-28 2023-08-31 e-con Systems India Private Limited System and method for assisting data transmission over virtual channels
EP4369717A1 (en) * 2022-11-14 2024-05-15 Samsung Electronics Co., Ltd. Image processing apparatus including line interleaving controller for a plurality of image sensors and operating method thereof

Also Published As

Publication number Publication date
KR101512222B1 (ko) 2015-04-14
EP2556658A1 (en) 2013-02-13
CN102870401A (zh) 2013-01-09
AU2011237801A1 (en) 2012-11-01
TW201210314A (en) 2012-03-01
TW201205308A (en) 2012-02-01
US20110242355A1 (en) 2011-10-06
RU2012146968A (ru) 2014-05-20
KR20130028096A (ko) 2013-03-18
US20110242356A1 (en) 2011-10-06
KR20130006503A (ko) 2013-01-16
JP5675959B2 (ja) 2015-02-25
TWI442766B (zh) 2014-06-21
BR112012025366A2 (pt) 2016-06-28
WO2011127076A1 (en) 2011-10-13
EP2556657A1 (en) 2013-02-13
CN102884783A (zh) 2013-01-16
CN102884783B (zh) 2017-03-08
EP2556656A1 (en) 2013-02-13
JP5559416B2 (ja) 2014-07-23
TW201212643A (en) 2012-03-16
AR081486A1 (es) 2012-09-19
CN102870401B (zh) 2016-04-06
US9001227B2 (en) 2015-04-07
WO2011127077A1 (en) 2011-10-13
AR081160A1 (es) 2012-07-04
KR101399718B1 (ko) 2014-05-27
IL222289A0 (en) 2012-12-31
AR081161A1 (es) 2012-07-04
SG184225A1 (en) 2012-11-29
TWI508520B (zh) 2015-11-11
ZA201208298B (en) 2013-07-31
WO2011127078A1 (en) 2011-10-13
KR20120137432A (ko) 2012-12-20
BR112012025182A2 (pt) 2016-06-21
JP2013530558A (ja) 2013-07-25
CN102859987A (zh) 2013-01-02
KR101517661B1 (ko) 2015-05-04
JP2013528021A (ja) 2013-07-04
CA2794682A1 (en) 2011-10-13
JP2013539608A (ja) 2013-10-24

Similar Documents

Publication Publication Date Title
US8896668B2 (en) Combining data from multiple image sensors
US9001227B2 (en) Combining data from multiple image sensors
US8885067B2 (en) Multocular image pickup apparatus and multocular image pickup method
EP2720455B1 (en) Image pickup device imaging three-dimensional moving image and two-dimensional moving image, and image pickup apparatus mounting image pickup device
KR20110109905A (ko) 카메라 시스템, 영상 처리 장치 및 카메라 장치
WO2019167571A1 (ja) 画像処理装置及び画像処理方法
JP2003348605A (ja) カラービデオカメラ

Legal Events

Date Code Title Description
AS Assignment

Owner name: QUALCOMM INCORPORATED, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOMA, SERGIU R.;CHEUNG, JOSEPH;HWANG, HAU;AND OTHERS;REEL/FRAME:026229/0385

Effective date: 20110427

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION