US20130021504A1 - Multiple image processing - Google Patents
Multiple image processing Download PDFInfo
- Publication number
- US20130021504A1 US20130021504A1 US13/235,975 US201113235975A US2013021504A1 US 20130021504 A1 US20130021504 A1 US 20130021504A1 US 201113235975 A US201113235975 A US 201113235975A US 2013021504 A1 US2013021504 A1 US 2013021504A1
- Authority
- US
- United States
- Prior art keywords
- image data
- image
- processing
- processed
- pipeline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000003384 imaging method Methods 0.000 claims description 34
- 238000000034 method Methods 0.000 claims description 30
- 230000008569 process Effects 0.000 claims description 18
- 238000001914 filtration Methods 0.000 claims description 7
- 238000005457 optimization Methods 0.000 claims description 6
- 238000003672 processing method Methods 0.000 claims 6
- 238000010586 diagram Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 3
- 238000003705 background correction Methods 0.000 description 3
- 239000000872 buffer Substances 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000003707 image sharpening Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000000153 supplemental effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000011143 downstream manufacturing Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/537—Motion estimation other than block-based
- H04N19/54—Motion estimation other than block-based using feature points or meshes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/56—Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/681—Motion detection
- H04N23/6811—Motion detection based on the image signal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20201—Motion blur correction
Definitions
- FIG. 1 is a block diagram of one embodiment of an image processing circuitry according to the present disclosure.
- FIGS. 2-7 are block diagrams of embodiments of an image signal processing pipeline implemented by the pipeline processing logic from the image processing circuitry of FIG. 1 .
- FIG. 8 is a block diagram illustrating an embodiment of an electronic device employing the image processing circuitry of FIG. 1 .
- FIGS. 9-13 are flow chart diagrams depicting various functionalities of embodiments of image processing circuitry of FIG. 1 .
- This disclosure pertains to a device, method, computer useable medium, and processor programmed to automatically utilize parallel image captures in an image processing pipeline in a digital camera, digital video camera, or other imaging device.
- a device, method, computer useable medium, and processor programmed to automatically utilize parallel image captures in an image processing pipeline in a digital camera, digital video camera, or other imaging device.
- One of ordinary skill in the art would recognize that the techniques disclosed may also be applied to other contexts and applications as well.
- a digital imaging device may include an image sensor that provides a number of light-detecting elements (e.g., photodetectors) configured to convert light detected by the image sensor into an electrical signal.
- An image sensor may also include a color filter array that filters light captured by the image sensor to capture color information.
- the image data captured by the image sensor may then be processed by an image processing pipeline circuitry, which may apply a number of various image processing operations to the image data to generate a full color image that may be displayed for viewing on a display device, such as a monitor.
- the illustrated imaging device 150 may be provided as a digital camera configured to acquire both still images and moving images (e.g., video).
- the device 150 may include lens(es) 110 and one or more image sensors 101 configured to capture and convert light into electrical signals.
- the image sensor may include a CMOS (complementary metal-oxide-semiconductor) image sensor (e.g., a CMOS active-pixel sensor (APS)) or a CCD (charge-coupled device) sensor.
- CMOS complementary metal-oxide-semiconductor
- APS CMOS active-pixel sensor
- CCD charge-coupled device
- the image processing circuitry 100 may include various subcomponents and/or discrete units of logic that collectively form an image processing “pipeline” for performing each of the various image processing steps. These subcomponents may be implemented using hardware (e.g., digital signal processors or ASICs (application-specific integrated circuits)) or software, or via a combination of hardware and software components. The various image processing operations may be provided by the image processing circuitry 100 .
- the image processing circuitry 100 may include front-end processing logic 103 , pipeline processing logic 104 , and control logic 105 , among others.
- the image sensor(s) 101 may include a color filter array (e.g., a Bayer filter) and may thus provide both light intensity and wavelength information captured by each imaging pixel of the image sensors 101 to provide for a set of raw image data that may be processed by the front-end processing logic 103 .
- a color filter array e.g., a Bayer filter
- a single lens 110 and a single image sensor 101 may be employed in the image processing circuitry. While in other embodiments, multiple lens 110 and multiple image sensors 101 may be employed, such as for a stereoscopy uses, among others.
- the front-end processing logic 103 may also receive pixel data from memory 108 .
- the raw pixel data may be sent to memory 108 from the image sensor 101 .
- the raw pixel data residing in the memory 108 may then be provided to the front-end processing logic 103 for processing.
- the front-end processing logic 103 may perform one or more image processing operations.
- the processed image data may then be provided to the pipeline processing logic 104 for additional processing prior to being displayed (e.g., on display device 106 ), or may be sent to the memory 108 .
- the pipeline processing logic 104 receives the “front-end” processed data, either directly from the front-end processing logic 103 or from memory 108 , and may provide for additional processing of the image data in the raw domain, as well as in the RGB and YCbCr color spaces, as the case may be.
- Image data processed by the pipeline processing logic 104 may then be output to the display 106 (or viewfinder) for viewing by a user and/or may be further processed by a graphics engine. Additionally, output from the pipeline processing logic 104 may be sent to memory 108 and the display 106 may read the image data from memory 108 . Further, in some implementations, the pipeline processing logic 104 may also include an encoder 107 , such as a compression engine, for encoding the image data prior to being read by the display 106 .
- an encoder 107 such as a compression engine
- the encoder 107 may be a JPEG (Joint Photographic Experts Group) compression engine for encoding still images, or an H.264 compression engine for encoding video images, or some combination thereof. Also, it should be noted that the pipeline processing logic 104 may also receive raw image data from the memory 108 .
- JPEG Joint Photographic Experts Group
- the control logic 105 may include a processor 820 ( FIG. 8 ) and/or microcontroller configured to execute one or more routines (e.g., firmware) that may be configured to determine control parameters for the imaging device 150 , as well as control parameters for the pipeline processing logic 104 .
- the control parameters may include sensor control parameters, camera flash control parameters, lens control parameters (e.g., focal length for focusing or zoom), or a combination of such parameters for the image sensor(s) 101 .
- the control parameters may also include image processing commands, such as autowhite balance, autofocus, autoexposure, and color adjustments, as well as lens shading correction parameters for the pipeline processing logic 104 .
- the control parameters may further comprise multiplexing signals or commands for the pipeline processing logic 104 .
- one embodiment of the pipeline processing logic 104 may perform processes of an image signal processing pipeline by first sending image information to a first process element 201 which may take the raw data produced by the image sensor 101 ( FIG. 1 ) and generate a digital image that will be viewed by a user or undergo further processing by a downstream process element.
- the image signal processing pipeline may be considered as a series of specialized algorithms that adjusts image data in real-time and is often implemented as an integrated component of a system-on-chip (SoC) image processor.
- SoC system-on-chip
- the first process element 201 of an image signal processing pipeline could perform a particular image process such as noise reduction, defective pixel detection/correction, lens shading correction, lens distortion correction, demosaicing, image sharpening, color uniformity, RGB (red, green, blue) contrast, saturation boost process, etc.
- the pipeline may include a second process element 202 .
- the second process element 202 could perform a particular and different image process such as noise reduction, defective pixel detection/correction, lens shading correction, demosaicing, image sharpening, color uniformity, RGB contrast, saturation boost process etc.
- the image data may then be sent to additional element(s) of the pipeline as the case may be, saved to memory 108 ( FIG. 1 ), and/or input for display 106 ( FIG. 1 ).
- the image signal processing pipeline performed by pipeline processing logic 104 contains parallel paths instead of a single linear path.
- the parallel paths may provide a first path and a second path.
- the first path comprises a main processing path and the second path comprises a supplemental processing path. Therefore, while raw image data is being processed in the first path to generate a high-resolution image output suitable for storage, the raw image data is processed in the second and parallel path to generate a lower resolution image that can be generated more quickly (as compared to the first path) and be displayed in the camera viewfinder or display 106 . It may be that the second path contains fewer stages or elements 321 , 322 than the first path.
- the first path may contain the same number of or less number of stages or elements 311 , 312 as compared to the second path.
- the second path may involve resolution down-conversion of the image to lessen the amount of pixels that need to be processed during image processing, such as for image analysis, in the pipeline.
- the benefits of the parallel paths may apply to still images as well as video images captured by the image sensor(s) 101 ( FIG. 1 ). It is noted that some embodiments of the pipeline processing logic 104 utilizes a single image sensor 101 that provides raw data to the first and second paths, where the first path may process the raw data relatively carefully and more slowly than the second path that can generate an image available to be previewed more quickly.
- each stage in the pipeline may begin processing as soon as image data is available so the entire image does not have to be received from the previous sensor or stage before processing is started.
- multiple imagers or image sensors 101 may be utilized, as shown in FIG. 4 .
- one imager or sensor 101 a may provide raw data at a lower resolution than a second image sensor 101 b , where the lower resolution raw data feeds a pipeline path to the display 106 and the higher resolution data feeds a path used for encoding and/or for storage in memory 108 .
- a secondary or supplemental image may be used in image analysis that can help subsequent image analysis operations for the main image.
- a secondary image at a smaller size or resolution than the main image might undergo facial recognition algorithms (or other object recognition algorithm) and output of positive results may be used to identify facial structures (or other objects) in the main image. Therefore, the secondary image may be produced in a format that is more suited for some of the applicable states or processing elements in its path. Accordingly, processing elements 411 , 412 , may be divided up between elements that are suited for the main image and processing elements 421 , 422 that are suited for the secondary image. Accordingly, a secondary image may be initially processed, such as being made smaller or scaled, for the benefit of downstream elements.
- the path of the secondary image may contain a noise filtering element due to a downstream element needed for the secondary image to have undergone noise reduction.
- the different paths or elements in the different paths may also use different imaging formats. For example, one of the paths may use an integral image format whereas a standard image format is used in the other path. Accordingly, downstream elements in the integral image path may need an integral image format as opposed to a standard image format and vice versa.
- the images generated by the first and second paths may be stored in memory 108 and made available for subsequent use by other procedures and elements that follow. Accordingly, in one embodiment, while a main image is being processed in a main path of the pipeline, another image which might be downsized or scaled of that image or a previous image may be read by the main path. This may enable more powerful processing in the pipeline, such as during noise filtering.
- neighboring pixels are analyzed. This process of denoising the pixel may have a stronger effect with the more pixels that are able to be analyzed further away from the pixel being processed. Due to hardware constraints, such as memory buffer size used by processing logic, there is a limit in how far away from the current pixel that the process can analyze neighboring pixels.
- a downscaled version of the main image is generated in a second path and the noise filter in the main path reads the downscaled version of the image and stores those pixels for noise analysis. Since there are the same number of line buffers but the image is downscaled, this effectively allows the noise filter to see further away in the original image because the second image is at a reduced scale.
- another embodiment utilizes a downscaled version of an image to assist in dynamic range optimization processing.
- a dynamic range optimization process is provided a way to see further away from a current pixel than would be available by only considering the full resolution image.
- a high dynamic range imaging process or element also reads a downscaled version of a main image to see further away from the current pixel.
- raw image data (from an image sensor 101 ) may be provided to the front-end processing logic 103 and processed on a pixel-by-pixel basis in a number of formats.
- raw pixel data received by the front-end processing logic 103 may be up-sampled for image processing purposes.
- raw image or pixel data may be down-sampled or scaled.
- down-sampling of image data may reduce hardware size (e.g., area) and also reduce processing/computational complexity.
- the front-end processing logic 103 generates two distinct kinds of images for the pipeline processing logic 104 .
- the imaging device 150 may be capturing video images and the user or the device itself determines to also capture a still image in addition to the video or moving images.
- a problem to overcome with this task in conventional cameras is that the video images are being generated at a resolution that is less than desired for still images.
- a potential solution would be to record the video images at the higher resolution desired for the still image, but this would require the pipeline processing logic 104 to undergo processing of the higher resolution video images.
- one embodiment of the present disclosure captures the raw image data by the sensor 101 at the higher resolution suitable for still image photography. Then, the front-end pipeline processing logic 103 scales down the size of the captured images to a resolution size suitable for video processing before feeding the image data to the appropriate pipeline processing logic 104 . When the user or the imaging device 150 decides to capture an image still, for this one frame, the front-end pipeline processing logic 103 will receive instructions from the control logic 105 and store the desired frame in memory 108 at the higher resolution. Further, in one embodiment, although a main imaging path of the pipeline is handling the video processing, as processing time allows, the main imaging path can be provided the still image from memory 108 .
- the video processing is assigned a higher priority than the still image processing by the pipeline processing logic 104 .
- the pipeline processing logic 104 features a single pipeline for processing captured images but has the capability to multiplex the single pipeline between different input images. Therefore, the single pipeline may switch from processing an image or series of images having a high priority to an image or series of images having a lower priority as processing time allows.
- Multiplexing of the imaging pipeline is also implemented in an embodiment utilizing multiple image sensors 101 .
- a stereoscopic image device that delivers a left image and a right image of an object to a single image pipeline, as represented in FIG. 5 .
- the single image pipeline in pipeline processing logic 104 can therefore be multiplexed between the left and right images that are being input in parallel to the image signal processing pipeline so that the pipeline is shared.
- the images can be processed concurrently by switching processing of the images between one another as processing time allows by front-end processing logic 103 . This reduces latency by not delaying processing of an image until completion of the other image, and processing of the two images will finish more quickly.
- one embodiment utilizes multiple image sensors 101 that produce multiple inputs for the pipeline processing logic 104 .
- one of the image sensors 101 b may capture a low resolution image that is fed as a preview of an image recently captured, where the other image sensor 101 a captures a high resolution image of the subject of the picture that is processed in parallel. Otherwise, the low resolution image may be used for framing a shot to be captured, where the subsequent captured shot or image is at a higher resolution and may undergo additional processing. Therefore, this embodiment features an imaging device with two fully parallel image capture and processing pipeline paths.
- a single image sensor 101 is utilized to capture image information and provide the information to the front-end processing logic 103 , whereby the front-end processing logic 103 may generate two input images for parallel paths in the pipeline processing logic 104 (as represented in FIG. 7 ). Also, in some embodiments, a single image sensor 101 is utilized to capture image information and provide the information to the front-end processing logic 103 , whereby the front-end processing logic 103 may generate two input images for multiplexed input into a single path of the pipeline processing logic 104 (as represented in FIG. 6 ).
- FIG. 8 is a block diagram illustrating an example of an electronic device 805 that may provide for the processing of image data using one or more of the image processing techniques briefly mentioned above.
- the electronic device 805 may be any type of electronic device, such as a laptop or desktop computer, a mobile phone, tablet, a digital media player, or the like, that is configured to receive and process image data, such as data acquired using one or more image sensing components.
- the electronic device 805 may provide for the processing of image data using one or more of the image processing techniques briefly discussed above, among others.
- the electronic device 805 may apply such image processing techniques to image data stored in a memory 830 of the electronic device 805 .
- the electronic device 805 may include one or more imaging devices 880 , such as an integrated or external digital camera, configured to acquire image data, which may then be processed by the electronic device 805 using one or more of the above-mentioned image processing techniques.
- the electronic device 805 may include various internal and/or external components which contribute to the function of the device 805 .
- the various functional blocks shown in FIG. 8 may comprise hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium) or a combination of both hardware and software elements.
- the electronic device 805 may include input/output (I/O) ports 810 , one or more processors 820 , memory device 830 , non-volatile storage 840 , networking device 850 , power source 860 , and display 870 .
- I/O input/output
- the electronic device 10 may include one or more imaging devices 880 , such as a digital camera, and image processing circuitry 890 .
- the image processing circuitry 890 may be configured implement one or more of the above-discussed image processing techniques when processing image data.
- image data processed by image processing circuitry 890 may be retrieved from the memory 830 and/or the non-volatile storage device(s) 840 , or may be acquired using the imaging device 880 .
- the system block diagram of the device 805 shown in FIG. 8 is intended to be a high-level control diagram depicting various components that may be included in such a device 805 . That is, the connection lines between each individual component shown in FIG. 1 may not necessarily represent paths or directions through which data flows or is transmitted between various components of the device 805 .
- the depicted processor(s) 820 may, in some embodiments, include multiple processors, such as a main processor (e.g., CPU), and dedicated image and/or video processors. In such embodiments, the processing of image data may be primarily handled by these dedicated processors, thus effectively offloading such tasks from a main processor (CPU).
- main processor e.g., CPU
- dedicated image and/or video processors dedicated image and/or video processors.
- FIG. 9 shown is a flowchart that provides one example of the operation of a portion of the image processing circuitry 100 according to various embodiments. It is understood that the flowchart of FIG. 9 provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of the image processing circuitry 100 as described herein. As an alternative, the flowchart of FIG. 9 may be viewed as depicting an example of steps of a method implemented in the electronic device 805 ( FIG. 8 ) according to one or more embodiments.
- imaging processing circuitry 100 provides an imaging pipeline for processing images captured from one or more image sensors 101 , where the imaging single processing pipeline features two parallel paths for processing the images.
- an input image obtained from the image sensor(s) 101 is processed at full-resolution.
- an input image obtained from the image sensor(s) 101 is processed at a down-scaled resolution, as depicted in step 906 .
- the down-scaled resolution version of the input image is output from the second parallel path of the pipeline before completion of processing of the input image at full-resolution and is provided for display, in step 908 .
- imaging processing circuitry 100 provides an imaging pipeline for processing images captured from one or more image sensors 101 , where the image signal processing pipeline features two parallel paths for processing the images.
- image signal processing pipeline features two parallel paths for processing the images.
- an input image obtained from the image sensor(s) 101 is processed at full-resolution.
- an input image obtained from the image sensor(s) 101 is processed at a down-scaled resolution, as depicted in step 1006 .
- the down-scaled resolution version of the input image undergoes image enhancement analysis in the second parallel path that is applied to the full-resolution version of the image in the first parallel path, in step 1008 .
- image enhancement analysis may include noise filtering, dynamic range optimization, high dynamic range imaging, facial or object recognition, among others.
- imaging processing circuitry 100 provides an image signal processing pipeline for processing images captured from one or more image sensors 101 , where the pipeline features a single pipeline path for processing the images.
- multiple input images may be fed into the single pipeline path by multiplexing the different images by front-end circuitry (e.g., front-end processing logic 103 ).
- front-end circuitry e.g., front-end processing logic 103 .
- front-end processing logic 103 For example, consider a stereoscopic image device that delivers a left image and a right image of an object to a single image pipeline, as represented in FIG. 5 .
- the single image pipeline in pipeline processing logic 104 can therefore be multiplexed between the left and right images that are being input in parallel to the pipeline via the front-end circuitry. Instead of processing one of the images in its entirety after the other has been processed in its entirety, the images can be processed concurrently by switching processing of the images between one another as processing time allows by front-end processing circuitry.
- front-end processing circuitry can receive a single input image from an image sensor 101 .
- the front-end processing circuitry may then generate two or more input images for multiplexed input into a single path of an image signal processing pipeline of pipeline processing logic 104 (as represented in FIG. 6 ).
- the single pipeline in pipeline processing logic 104 can therefore be multiplexed between the multiple images that have been generated by the front-end circuitry, in step 1206 .
- the images can be processed concurrently by switching processing of the images between one another as processing time allows by front-end processing circuitry.
- one embodiment of the present disclosure captures the raw image data by the sensor 101 at a high resolution suitable for still image photography, in step 1302 . Then, the front-end pipeline processing logic 103 scales down the size of the captured images to a resolution size suitable for video processing, in step 1304 , before feeding the image data to the appropriate pipeline processing logic 104 , in step 1306 . When the user or the imaging device 150 decides to capture an image still, for this one frame, the front-end pipeline processing logic 103 will receive instructions from the control logic 105 and store the desired frame in memory 108 at the higher resolution, in step 1308 . Further, in one embodiment, although a main imaging path of the pipeline is handling the video processing, as processing time allows, the main imaging path can be provided the still image from memory 108 , in step 1310 .
- a “computer readable medium” can be any means that can contain, store, communicate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- the computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device.
- the computer readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical).
- an electrical connection having one or more wires
- a portable computer diskette magnetic
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- CDROM portable compact disc read-only memory
- the scope of certain embodiments includes embodying the functionality of the embodiments in logic embodied in hardware or software-configured mediums.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- This application claims priority to copending U.S. provisional application entitled, “Image Capture Device Systems and Methods,” having Ser. No. 61/509,747, filed Jul. 20, 2011, which is entirely incorporated herein by reference.
- With current cameras, there is a significant delay between the capture of an image and the subsequent display of a framed image to the user via a viewfinder. Accordingly, advances in image processing may allow for improvements, such as shorter latency.
- Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
-
FIG. 1 is a block diagram of one embodiment of an image processing circuitry according to the present disclosure. -
FIGS. 2-7 are block diagrams of embodiments of an image signal processing pipeline implemented by the pipeline processing logic from the image processing circuitry ofFIG. 1 . -
FIG. 8 is a block diagram illustrating an embodiment of an electronic device employing the image processing circuitry ofFIG. 1 . -
FIGS. 9-13 are flow chart diagrams depicting various functionalities of embodiments of image processing circuitry ofFIG. 1 . - This disclosure pertains to a device, method, computer useable medium, and processor programmed to automatically utilize parallel image captures in an image processing pipeline in a digital camera, digital video camera, or other imaging device. One of ordinary skill in the art would recognize that the techniques disclosed may also be applied to other contexts and applications as well.
- For cameras in embedded devices, e.g., digital cameras, digital video cameras, mobile phones, personal data assistants (PDAs), tablets, portable music players, and desktop or laptop computers, to produce more visually pleasing images, techniques such as those disclosed herein can improve image quality without incurring significant computational overhead or power costs.
- To acquire image data, a digital imaging device may include an image sensor that provides a number of light-detecting elements (e.g., photodetectors) configured to convert light detected by the image sensor into an electrical signal. An image sensor may also include a color filter array that filters light captured by the image sensor to capture color information. The image data captured by the image sensor may then be processed by an image processing pipeline circuitry, which may apply a number of various image processing operations to the image data to generate a full color image that may be displayed for viewing on a display device, such as a monitor.
- Referring to
FIG. 1 , a block diagram of one embodiment of animage processing circuitry 100 is shown for animaging device 150. The illustratedimaging device 150 may be provided as a digital camera configured to acquire both still images and moving images (e.g., video). Thedevice 150 may include lens(es) 110 and one ormore image sensors 101 configured to capture and convert light into electrical signals. By way of example only, the image sensor may include a CMOS (complementary metal-oxide-semiconductor) image sensor (e.g., a CMOS active-pixel sensor (APS)) or a CCD (charge-coupled device) sensor. - In some embodiments, the
image processing circuitry 100 may include various subcomponents and/or discrete units of logic that collectively form an image processing “pipeline” for performing each of the various image processing steps. These subcomponents may be implemented using hardware (e.g., digital signal processors or ASICs (application-specific integrated circuits)) or software, or via a combination of hardware and software components. The various image processing operations may be provided by theimage processing circuitry 100. - The
image processing circuitry 100 may include front-end processing logic 103,pipeline processing logic 104, andcontrol logic 105, among others. The image sensor(s) 101 may include a color filter array (e.g., a Bayer filter) and may thus provide both light intensity and wavelength information captured by each imaging pixel of theimage sensors 101 to provide for a set of raw image data that may be processed by the front-end processing logic 103. - In some embodiments, a
single lens 110 and asingle image sensor 101 may be employed in the image processing circuitry. While in other embodiments,multiple lens 110 andmultiple image sensors 101 may be employed, such as for a stereoscopy uses, among others. - The front-
end processing logic 103 may also receive pixel data frommemory 108. For instance, the raw pixel data may be sent tomemory 108 from theimage sensor 101. The raw pixel data residing in thememory 108 may then be provided to the front-end processing logic 103 for processing. - Upon receiving the raw image data (from
image sensor 101 or from memory 108), the front-end processing logic 103 may perform one or more image processing operations. The processed image data may then be provided to thepipeline processing logic 104 for additional processing prior to being displayed (e.g., on display device 106), or may be sent to thememory 108. Thepipeline processing logic 104 receives the “front-end” processed data, either directly from the front-end processing logic 103 or frommemory 108, and may provide for additional processing of the image data in the raw domain, as well as in the RGB and YCbCr color spaces, as the case may be. Image data processed by thepipeline processing logic 104 may then be output to the display 106 (or viewfinder) for viewing by a user and/or may be further processed by a graphics engine. Additionally, output from thepipeline processing logic 104 may be sent tomemory 108 and thedisplay 106 may read the image data frommemory 108. Further, in some implementations, thepipeline processing logic 104 may also include anencoder 107, such as a compression engine, for encoding the image data prior to being read by thedisplay 106. - The
encoder 107 may be a JPEG (Joint Photographic Experts Group) compression engine for encoding still images, or an H.264 compression engine for encoding video images, or some combination thereof. Also, it should be noted that thepipeline processing logic 104 may also receive raw image data from thememory 108. - The
control logic 105 may include a processor 820 (FIG. 8 ) and/or microcontroller configured to execute one or more routines (e.g., firmware) that may be configured to determine control parameters for theimaging device 150, as well as control parameters for thepipeline processing logic 104. By way of example only, the control parameters may include sensor control parameters, camera flash control parameters, lens control parameters (e.g., focal length for focusing or zoom), or a combination of such parameters for the image sensor(s) 101. The control parameters may also include image processing commands, such as autowhite balance, autofocus, autoexposure, and color adjustments, as well as lens shading correction parameters for thepipeline processing logic 104. The control parameters may further comprise multiplexing signals or commands for thepipeline processing logic 104. - Referring now to
FIG. 2 , one embodiment of thepipeline processing logic 104 may perform processes of an image signal processing pipeline by first sending image information to afirst process element 201 which may take the raw data produced by the image sensor 101 (FIG. 1 ) and generate a digital image that will be viewed by a user or undergo further processing by a downstream process element. Accordingly, the image signal processing pipeline may be considered as a series of specialized algorithms that adjusts image data in real-time and is often implemented as an integrated component of a system-on-chip (SoC) image processor. With an image signal processing pipeline implemented in hardware, front-end image processing can be completed without placing any processing burden on the main application processor 820 (FIG. 8 ). - In one embodiment, the
first process element 201 of an image signal processing pipeline could perform a particular image process such as noise reduction, defective pixel detection/correction, lens shading correction, lens distortion correction, demosaicing, image sharpening, color uniformity, RGB (red, green, blue) contrast, saturation boost process, etc. As discussed above, the pipeline may include asecond process element 202. In one embodiment, thesecond process element 202 could perform a particular and different image process such as noise reduction, defective pixel detection/correction, lens shading correction, demosaicing, image sharpening, color uniformity, RGB contrast, saturation boost process etc. The image data may then be sent to additional element(s) of the pipeline as the case may be, saved to memory 108 (FIG. 1 ), and/or input for display 106 (FIG. 1 ). - Referring next to
FIG. 3 , in one embodiment, the image signal processing pipeline performed bypipeline processing logic 104 contains parallel paths instead of a single linear path. For example, the parallel paths may provide a first path and a second path. Further, in one embodiment, the first path comprises a main processing path and the second path comprises a supplemental processing path. Therefore, while raw image data is being processed in the first path to generate a high-resolution image output suitable for storage, the raw image data is processed in the second and parallel path to generate a lower resolution image that can be generated more quickly (as compared to the first path) and be displayed in the camera viewfinder ordisplay 106. It may be that the second path contains fewer stages orelements elements - The benefits of the parallel paths may apply to still images as well as video images captured by the image sensor(s) 101 (
FIG. 1 ). It is noted that some embodiments of thepipeline processing logic 104 utilizes asingle image sensor 101 that provides raw data to the first and second paths, where the first path may process the raw data relatively carefully and more slowly than the second path that can generate an image available to be previewed more quickly. - Use of parallel paths in the image signal processing pipeline may enable processing of multiple image data simultaneously while maximizing final image quality. Additionally, each stage in the pipeline may begin processing as soon as image data is available so the entire image does not have to be received from the previous sensor or stage before processing is started.
- In an alternative embodiment, multiple imagers or
image sensors 101 may be utilized, as shown inFIG. 4 . For example, one imager orsensor 101 a may provide raw data at a lower resolution than asecond image sensor 101 b, where the lower resolution raw data feeds a pipeline path to thedisplay 106 and the higher resolution data feeds a path used for encoding and/or for storage inmemory 108. - Further, in some embodiments, a secondary or supplemental image may be used in image analysis that can help subsequent image analysis operations for the main image. As an example, a secondary image at a smaller size or resolution than the main image might undergo facial recognition algorithms (or other object recognition algorithm) and output of positive results may be used to identify facial structures (or other objects) in the main image. Therefore, the secondary image may be produced in a format that is more suited for some of the applicable states or processing elements in its path. Accordingly, processing
elements processing elements - In some embodiments, the images generated by the first and second paths may be stored in
memory 108 and made available for subsequent use by other procedures and elements that follow. Accordingly, in one embodiment, while a main image is being processed in a main path of the pipeline, another image which might be downsized or scaled of that image or a previous image may be read by the main path. This may enable more powerful processing in the pipeline, such as during noise filtering. - For example, during noise filtering, for any given pixel being processed, neighboring pixels are analyzed. This process of denoising the pixel may have a stronger effect with the more pixels that are able to be analyzed further away from the pixel being processed. Due to hardware constraints, such as memory buffer size used by processing logic, there is a limit in how far away from the current pixel that the process can analyze neighboring pixels.
- Accordingly, in one embodiment, a downscaled version of the main image is generated in a second path and the noise filter in the main path reads the downscaled version of the image and stores those pixels for noise analysis. Since there are the same number of line buffers but the image is downscaled, this effectively allows the noise filter to see further away in the original image because the second image is at a reduced scale.
- Accordingly, another embodiment utilizes a downscaled version of an image to assist in dynamic range optimization processing. By having available a downscaled version of an image alongside a full resolution image in
memory 108, a dynamic range optimization process is provided a way to see further away from a current pixel than would be available by only considering the full resolution image. In a similar manner, a high dynamic range imaging process or element also reads a downscaled version of a main image to see further away from the current pixel. - Referring back to
FIG. 1 , in one embodiment, raw image data (from an image sensor 101) may be provided to the front-end processing logic 103 and processed on a pixel-by-pixel basis in a number of formats. For example, in one embodiment, raw pixel data received by the front-end processing logic 103 may be up-sampled for image processing purposes. In another embodiment, raw image or pixel data may be down-sampled or scaled. As will be appreciated, down-sampling of image data may reduce hardware size (e.g., area) and also reduce processing/computational complexity. - In some embodiments, the front-
end processing logic 103 generates two distinct kinds of images for thepipeline processing logic 104. As an example, theimaging device 150 may be capturing video images and the user or the device itself determines to also capture a still image in addition to the video or moving images. A problem to overcome with this task in conventional cameras is that the video images are being generated at a resolution that is less than desired for still images. A potential solution would be to record the video images at the higher resolution desired for the still image, but this would require thepipeline processing logic 104 to undergo processing of the higher resolution video images. However, it is difficult to encode video at a high resolution (e.g., 8 megapixels) and it is also impractical, since video images do not necessarily require a very high resolution. - Accordingly, one embodiment of the present disclosure captures the raw image data by the
sensor 101 at the higher resolution suitable for still image photography. Then, the front-endpipeline processing logic 103 scales down the size of the captured images to a resolution size suitable for video processing before feeding the image data to the appropriatepipeline processing logic 104. When the user or theimaging device 150 decides to capture an image still, for this one frame, the front-endpipeline processing logic 103 will receive instructions from thecontrol logic 105 and store the desired frame inmemory 108 at the higher resolution. Further, in one embodiment, although a main imaging path of the pipeline is handling the video processing, as processing time allows, the main imaging path can be provided the still image frommemory 108. - Accordingly, in one embodiment, the video processing is assigned a higher priority than the still image processing by the
pipeline processing logic 104. In such an embodiment, thepipeline processing logic 104 features a single pipeline for processing captured images but has the capability to multiplex the single pipeline between different input images. Therefore, the single pipeline may switch from processing an image or series of images having a high priority to an image or series of images having a lower priority as processing time allows. - Multiplexing of the imaging pipeline is also implemented in an embodiment utilizing
multiple image sensors 101. For example, consider a stereoscopic image device that delivers a left image and a right image of an object to a single image pipeline, as represented inFIG. 5 . The single image pipeline inpipeline processing logic 104 can therefore be multiplexed between the left and right images that are being input in parallel to the image signal processing pipeline so that the pipeline is shared. Instead of processing one of the images in its entirety after the other has been processed in its entirety, the images can be processed concurrently by switching processing of the images between one another as processing time allows by front-end processing logic 103. This reduces latency by not delaying processing of an image until completion of the other image, and processing of the two images will finish more quickly. - Alternatively, one embodiment utilizes
multiple image sensors 101 that produce multiple inputs for thepipeline processing logic 104. Referring now toFIG. 6 , in one scenario, one of theimage sensors 101 b may capture a low resolution image that is fed as a preview of an image recently captured, where theother image sensor 101 a captures a high resolution image of the subject of the picture that is processed in parallel. Otherwise, the low resolution image may be used for framing a shot to be captured, where the subsequent captured shot or image is at a higher resolution and may undergo additional processing. Therefore, this embodiment features an imaging device with two fully parallel image capture and processing pipeline paths. - Further, in some embodiments, a
single image sensor 101 is utilized to capture image information and provide the information to the front-end processing logic 103, whereby the front-end processing logic 103 may generate two input images for parallel paths in the pipeline processing logic 104 (as represented inFIG. 7 ). Also, in some embodiments, asingle image sensor 101 is utilized to capture image information and provide the information to the front-end processing logic 103, whereby the front-end processing logic 103 may generate two input images for multiplexed input into a single path of the pipeline processing logic 104 (as represented inFIG. 6 ). - Keeping the above points in mind,
FIG. 8 is a block diagram illustrating an example of anelectronic device 805 that may provide for the processing of image data using one or more of the image processing techniques briefly mentioned above. Theelectronic device 805 may be any type of electronic device, such as a laptop or desktop computer, a mobile phone, tablet, a digital media player, or the like, that is configured to receive and process image data, such as data acquired using one or more image sensing components. - Regardless of its form (e.g., portable or non-portable), it should be understood that the
electronic device 805 may provide for the processing of image data using one or more of the image processing techniques briefly discussed above, among others. In some embodiments, theelectronic device 805 may apply such image processing techniques to image data stored in amemory 830 of theelectronic device 805. In further embodiments, theelectronic device 805 may include one ormore imaging devices 880, such as an integrated or external digital camera, configured to acquire image data, which may then be processed by theelectronic device 805 using one or more of the above-mentioned image processing techniques. - As shown in
FIG. 8 , theelectronic device 805 may include various internal and/or external components which contribute to the function of thedevice 805. Those of ordinary skill in the art will appreciate that the various functional blocks shown inFIG. 8 may comprise hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium) or a combination of both hardware and software elements. For example, in the presently illustrated embodiment, theelectronic device 805 may include input/output (I/O)ports 810, one ormore processors 820,memory device 830,non-volatile storage 840,networking device 850,power source 860, anddisplay 870. Additionally, the electronic device 10 may include one ormore imaging devices 880, such as a digital camera, andimage processing circuitry 890. As will be discussed further below, theimage processing circuitry 890 may be configured implement one or more of the above-discussed image processing techniques when processing image data. As can be appreciated, image data processed byimage processing circuitry 890 may be retrieved from thememory 830 and/or the non-volatile storage device(s) 840, or may be acquired using theimaging device 880. - Before continuing, it should be understood that the system block diagram of the
device 805 shown inFIG. 8 is intended to be a high-level control diagram depicting various components that may be included in such adevice 805. That is, the connection lines between each individual component shown inFIG. 1 may not necessarily represent paths or directions through which data flows or is transmitted between various components of thedevice 805. Indeed, as discussed below, the depicted processor(s) 820 may, in some embodiments, include multiple processors, such as a main processor (e.g., CPU), and dedicated image and/or video processors. In such embodiments, the processing of image data may be primarily handled by these dedicated processors, thus effectively offloading such tasks from a main processor (CPU). - Referring next to
FIG. 9 , shown is a flowchart that provides one example of the operation of a portion of theimage processing circuitry 100 according to various embodiments. It is understood that the flowchart ofFIG. 9 provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of theimage processing circuitry 100 as described herein. As an alternative, the flowchart ofFIG. 9 may be viewed as depicting an example of steps of a method implemented in the electronic device 805 (FIG. 8 ) according to one or more embodiments. - Beginning in
step 902,imaging processing circuitry 100 provides an imaging pipeline for processing images captured from one ormore image sensors 101, where the imaging single processing pipeline features two parallel paths for processing the images. As described instep 904, in a first parallel path of the pipeline, an input image obtained from the image sensor(s) 101 is processed at full-resolution. Additionally, in a second parallel path of the pipeline, an input image obtained from the image sensor(s) 101 is processed at a down-scaled resolution, as depicted instep 906. The down-scaled resolution version of the input image is output from the second parallel path of the pipeline before completion of processing of the input image at full-resolution and is provided for display, instep 908. - Next, referring to
FIG. 10 , shown is a flowchart that provides an additional example of the operation of a portion of theimage processing circuitry 100 according to various embodiments. Beginning instep 1002,imaging processing circuitry 100 provides an imaging pipeline for processing images captured from one ormore image sensors 101, where the image signal processing pipeline features two parallel paths for processing the images. As described instep 1004, in a first parallel path of the pipeline, an input image obtained from the image sensor(s) 101 is processed at full-resolution. Additionally, in a second parallel path of the pipeline, an input image obtained from the image sensor(s) 101 is processed at a down-scaled resolution, as depicted instep 1006. The down-scaled resolution version of the input image undergoes image enhancement analysis in the second parallel path that is applied to the full-resolution version of the image in the first parallel path, instep 1008. In particular, pixels are able to be analyzed in the down-scaled resolution version of the input image that may not be able to be analyzed as efficiently in the full-resolution version of the input image due to buffer limitations or other hardware restraints. In various embodiments, the type of image enhancement analysis may include noise filtering, dynamic range optimization, high dynamic range imaging, facial or object recognition, among others. - In
FIG. 11 , a flow chart is shown that provides an additional example of the operation of a portion of theimage processing circuitry 100 according to various embodiments. Beginning instep 1102,imaging processing circuitry 100 provides an image signal processing pipeline for processing images captured from one ormore image sensors 101, where the pipeline features a single pipeline path for processing the images. As described instep 1104, multiple input images may be fed into the single pipeline path by multiplexing the different images by front-end circuitry (e.g., front-end processing logic 103). For example, consider a stereoscopic image device that delivers a left image and a right image of an object to a single image pipeline, as represented inFIG. 5 . The single image pipeline inpipeline processing logic 104 can therefore be multiplexed between the left and right images that are being input in parallel to the pipeline via the front-end circuitry. Instead of processing one of the images in its entirety after the other has been processed in its entirety, the images can be processed concurrently by switching processing of the images between one another as processing time allows by front-end processing circuitry. - Further, in
FIG. 12 , a flow chart is shown that provides an additional example of the operation of a portion of theimage processing circuitry 100 according to various embodiments. Beginning instep 1202, front-end processing circuitry can receive a single input image from animage sensor 101. Instep 1204, the front-end processing circuitry may then generate two or more input images for multiplexed input into a single path of an image signal processing pipeline of pipeline processing logic 104 (as represented inFIG. 6 ). The single pipeline inpipeline processing logic 104 can therefore be multiplexed between the multiple images that have been generated by the front-end circuitry, instep 1206. Instead of processing one of the images in its entirety after the other has been processed in its entirety, the images can be processed concurrently by switching processing of the images between one another as processing time allows by front-end processing circuitry. - Next, in
FIG. 13 , a flow chart is shown that provides an additional example of the operation of a portion of theimage processing circuitry 100 according to various embodiments. Accordingly, one embodiment of the present disclosure captures the raw image data by thesensor 101 at a high resolution suitable for still image photography, instep 1302. Then, the front-endpipeline processing logic 103 scales down the size of the captured images to a resolution size suitable for video processing, instep 1304, before feeding the image data to the appropriatepipeline processing logic 104, instep 1306. When the user or theimaging device 150 decides to capture an image still, for this one frame, the front-endpipeline processing logic 103 will receive instructions from thecontrol logic 105 and store the desired frame inmemory 108 at the higher resolution, instep 1308. Further, in one embodiment, although a main imaging path of the pipeline is handling the video processing, as processing time allows, the main imaging path can be provided the still image frommemory 108, in step 1310. - Any process descriptions or blocks in flow charts should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of embodiments of the present disclosure in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art.
- In the context of this document, a “computer readable medium” can be any means that can contain, store, communicate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a nonexhaustive list) of the computer readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical). In addition, the scope of certain embodiments includes embodying the functionality of the embodiments in logic embodied in hardware or software-configured mediums.
- It should be emphasized that the above-described embodiments are merely possible examples of implementations, merely set forth for a clear understanding of the principles of the disclosure. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
Claims (20)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/235,975 US20130021504A1 (en) | 2011-07-20 | 2011-09-19 | Multiple image processing |
US13/431,064 US8553109B2 (en) | 2011-07-20 | 2012-03-27 | Concurrent image processing for generating an output image |
US14/039,922 US20140028870A1 (en) | 2011-07-20 | 2013-09-27 | Concurrent image processing for generating an output image |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161509747P | 2011-07-20 | 2011-07-20 | |
US13/235,975 US20130021504A1 (en) | 2011-07-20 | 2011-09-19 | Multiple image processing |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/431,064 Continuation-In-Part US8553109B2 (en) | 2011-07-20 | 2012-03-27 | Concurrent image processing for generating an output image |
Publications (1)
Publication Number | Publication Date |
---|---|
US20130021504A1 true US20130021504A1 (en) | 2013-01-24 |
Family
ID=47555520
Family Applications (9)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/232,045 Abandoned US20130021488A1 (en) | 2011-07-20 | 2011-09-14 | Adjusting Image Capture Device Settings |
US13/232,052 Abandoned US20130021512A1 (en) | 2011-07-20 | 2011-09-14 | Framing of Images in an Image Capture Device |
US13/235,975 Abandoned US20130021504A1 (en) | 2011-07-20 | 2011-09-19 | Multiple image processing |
US13/245,941 Abandoned US20130021489A1 (en) | 2011-07-20 | 2011-09-27 | Regional Image Processing in an Image Capture Device |
US13/281,521 Abandoned US20130021490A1 (en) | 2011-07-20 | 2011-10-26 | Facial Image Processing in an Image Capture Device |
US13/313,345 Abandoned US20130022116A1 (en) | 2011-07-20 | 2011-12-07 | Camera tap transcoder architecture with feed forward encode data |
US13/313,352 Active 2032-01-11 US9092861B2 (en) | 2011-07-20 | 2011-12-07 | Using motion information to assist in image processing |
US13/330,047 Abandoned US20130021484A1 (en) | 2011-07-20 | 2011-12-19 | Dynamic computation of lens shading |
US13/413,863 Abandoned US20130021491A1 (en) | 2011-07-20 | 2012-03-07 | Camera Device Systems and Methods |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/232,045 Abandoned US20130021488A1 (en) | 2011-07-20 | 2011-09-14 | Adjusting Image Capture Device Settings |
US13/232,052 Abandoned US20130021512A1 (en) | 2011-07-20 | 2011-09-14 | Framing of Images in an Image Capture Device |
Family Applications After (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/245,941 Abandoned US20130021489A1 (en) | 2011-07-20 | 2011-09-27 | Regional Image Processing in an Image Capture Device |
US13/281,521 Abandoned US20130021490A1 (en) | 2011-07-20 | 2011-10-26 | Facial Image Processing in an Image Capture Device |
US13/313,345 Abandoned US20130022116A1 (en) | 2011-07-20 | 2011-12-07 | Camera tap transcoder architecture with feed forward encode data |
US13/313,352 Active 2032-01-11 US9092861B2 (en) | 2011-07-20 | 2011-12-07 | Using motion information to assist in image processing |
US13/330,047 Abandoned US20130021484A1 (en) | 2011-07-20 | 2011-12-19 | Dynamic computation of lens shading |
US13/413,863 Abandoned US20130021491A1 (en) | 2011-07-20 | 2012-03-07 | Camera Device Systems and Methods |
Country Status (1)
Country | Link |
---|---|
US (9) | US20130021488A1 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140036108A1 (en) * | 2012-08-03 | 2014-02-06 | Samsung Electronics Co., Ltd. | Image processing method and apparatus |
US20140368514A1 (en) * | 2013-06-12 | 2014-12-18 | Infineon Technologies Ag | Device, method and system for processing an image data stream |
US20150237280A1 (en) * | 2014-02-19 | 2015-08-20 | Samsung Electronics Co., Ltd. | Image processing device with multiple image signal processors and image processing method |
US20160366341A1 (en) * | 2014-02-27 | 2016-12-15 | Nubia Technology Co., Ltd. | Image processing method and imaging device |
US10861127B1 (en) * | 2019-09-17 | 2020-12-08 | Gopro, Inc. | Image and video processing using multiple pipelines |
US11006044B1 (en) * | 2020-03-03 | 2021-05-11 | Qualcomm Incorporated | Power-efficient dynamic electronic image stabilization |
US11405565B2 (en) * | 2014-05-08 | 2022-08-02 | Sony Group Corporation | Information processing device and information processing method |
Families Citing this family (78)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
BR9714213A (en) | 1997-01-27 | 2000-02-29 | Peter D Haaland | Coatings, processes and apparatus to reduce the reflection of optical substrates. |
US10116839B2 (en) * | 2014-08-14 | 2018-10-30 | Atheer Labs, Inc. | Methods for camera movement compensation for gesture detection and object recognition |
JP5781351B2 (en) * | 2011-03-30 | 2015-09-24 | 日本アビオニクス株式会社 | Imaging apparatus, pixel output level correction method thereof, infrared camera system, and interchangeable lens system |
JP5778469B2 (en) | 2011-04-28 | 2015-09-16 | 日本アビオニクス株式会社 | Imaging apparatus, image generation method, infrared camera system, and interchangeable lens system |
KR101796481B1 (en) * | 2011-11-28 | 2017-12-04 | 삼성전자주식회사 | Method of eliminating shutter-lags with low power consumption, camera module, and mobile device having the same |
US9118876B2 (en) * | 2012-03-30 | 2015-08-25 | Verizon Patent And Licensing Inc. | Automatic skin tone calibration for camera images |
US9472005B1 (en) * | 2012-04-18 | 2016-10-18 | Amazon Technologies, Inc. | Projection and camera system for augmented reality environment |
US9619036B2 (en) | 2012-05-11 | 2017-04-11 | Comcast Cable Communications, Llc | System and methods for controlling a user experience |
US9438805B2 (en) * | 2012-06-08 | 2016-09-06 | Sony Corporation | Terminal device and image capturing method |
US8957973B2 (en) * | 2012-06-11 | 2015-02-17 | Omnivision Technologies, Inc. | Shutter release using secondary camera |
US20130335587A1 (en) * | 2012-06-14 | 2013-12-19 | Sony Mobile Communications, Inc. | Terminal device and image capturing method |
TWI498771B (en) * | 2012-07-06 | 2015-09-01 | Pixart Imaging Inc | Gesture recognition system and glasses with gesture recognition function |
US9554042B2 (en) * | 2012-09-24 | 2017-01-24 | Google Technology Holdings LLC | Preventing motion artifacts by intelligently disabling video stabilization |
US9286509B1 (en) * | 2012-10-19 | 2016-03-15 | Google Inc. | Image optimization during facial recognition |
JP2014086849A (en) * | 2012-10-23 | 2014-05-12 | Sony Corp | Content acquisition device and program |
US9060127B2 (en) * | 2013-01-23 | 2015-06-16 | Orcam Technologies Ltd. | Apparatus for adjusting image capture settings |
JP2014176034A (en) * | 2013-03-12 | 2014-09-22 | Ricoh Co Ltd | Video transmission device |
US9552630B2 (en) * | 2013-04-09 | 2017-01-24 | Honeywell International Inc. | Motion deblurring |
US9595083B1 (en) * | 2013-04-16 | 2017-03-14 | Lockheed Martin Corporation | Method and apparatus for image producing with predictions of future positions |
US8738629B1 (en) | 2013-05-03 | 2014-05-27 | Splunk Inc. | External Result Provided process for retrieving data stored using a different configuration or protocol |
US9916367B2 (en) | 2013-05-03 | 2018-03-13 | Splunk Inc. | Processing system search requests from multiple data stores with overlapping data |
WO2014190468A1 (en) | 2013-05-27 | 2014-12-04 | Microsoft Corporation | Video encoder for images |
US9529513B2 (en) * | 2013-08-05 | 2016-12-27 | Microsoft Technology Licensing, Llc | Two-hand interaction with natural user interface |
US9270959B2 (en) | 2013-08-07 | 2016-02-23 | Qualcomm Incorporated | Dynamic color shading correction |
CN105612083B (en) * | 2013-10-09 | 2018-10-23 | 麦格纳覆盖件有限公司 | To the system and method for the control that vehicle window is shown |
EP3067746B1 (en) | 2013-12-06 | 2019-08-21 | Huawei Device Co., Ltd. | Photographing method for dual-camera device and dual-camera device |
US10931866B2 (en) | 2014-01-05 | 2021-02-23 | Light Labs Inc. | Methods and apparatus for receiving and storing in a camera a user controllable setting that is used to control composite image generation performed after image capture |
US9245347B2 (en) * | 2014-01-30 | 2016-01-26 | Adobe Systems Incorporated | Image Cropping suggestion |
US9251594B2 (en) * | 2014-01-30 | 2016-02-02 | Adobe Systems Incorporated | Cropping boundary simplicity |
US10121060B2 (en) * | 2014-02-13 | 2018-11-06 | Oath Inc. | Automatic group formation and group detection through media recognition |
CN105359531B (en) | 2014-03-17 | 2019-08-06 | 微软技术许可有限责任公司 | Method and system for determining for the coder side of screen content coding |
US20150297986A1 (en) * | 2014-04-18 | 2015-10-22 | Aquifi, Inc. | Systems and methods for interactive video games with motion dependent gesture inputs |
US10051196B2 (en) * | 2014-05-20 | 2018-08-14 | Lenovo (Singapore) Pte. Ltd. | Projecting light at angle corresponding to the field of view of a camera |
WO2016004278A1 (en) * | 2014-07-03 | 2016-01-07 | Brady Worldwide, Inc. | Lockout/tagout device with non-volatile memory and related system |
US10031400B2 (en) * | 2014-08-06 | 2018-07-24 | Kevin J. WARRIAN | Orientation system for image recording device |
KR102225947B1 (en) * | 2014-10-24 | 2021-03-10 | 엘지전자 주식회사 | Mobile terminal and method for controlling the same |
CN105549302B (en) | 2014-10-31 | 2018-05-08 | 国际商业机器公司 | The coverage suggestion device of photography and vedio recording equipment |
US10334158B2 (en) * | 2014-11-03 | 2019-06-25 | Robert John Gove | Autonomous media capturing |
US20160148648A1 (en) * | 2014-11-20 | 2016-05-26 | Facebook, Inc. | Systems and methods for improving stabilization in time-lapse media content |
US10924743B2 (en) | 2015-02-06 | 2021-02-16 | Microsoft Technology Licensing, Llc | Skipping evaluation stages during media encoding |
US11721414B2 (en) * | 2015-03-12 | 2023-08-08 | Walmart Apollo, Llc | Importing structured prescription records from a prescription label on a medication package |
WO2016183380A1 (en) * | 2015-05-12 | 2016-11-17 | Mine One Gmbh | Facial signature methods, systems and software |
US10551913B2 (en) | 2015-03-21 | 2020-02-04 | Mine One Gmbh | Virtual 3D methods, systems and software |
US10853625B2 (en) | 2015-03-21 | 2020-12-01 | Mine One Gmbh | Facial signature methods, systems and software |
US20160316220A1 (en) * | 2015-04-21 | 2016-10-27 | Microsoft Technology Licensing, Llc | Video encoder management strategies |
US10447926B1 (en) | 2015-06-19 | 2019-10-15 | Amazon Technologies, Inc. | Motion estimation based video compression and encoding |
US10165186B1 (en) * | 2015-06-19 | 2018-12-25 | Amazon Technologies, Inc. | Motion estimation based video stabilization for panoramic video from multi-camera capture device |
US10136132B2 (en) | 2015-07-21 | 2018-11-20 | Microsoft Technology Licensing, Llc | Adaptive skip or zero block detection combined with transform size decision |
EP3136726B1 (en) * | 2015-08-27 | 2018-03-07 | Axis AB | Pre-processing of digital images |
US9648223B2 (en) * | 2015-09-04 | 2017-05-09 | Microvision, Inc. | Laser beam scanning assisted autofocus |
US9456195B1 (en) | 2015-10-08 | 2016-09-27 | Dual Aperture International Co. Ltd. | Application programming interface for multi-aperture imaging systems |
US9578221B1 (en) * | 2016-01-05 | 2017-02-21 | International Business Machines Corporation | Camera field of view visualizer |
JP6514140B2 (en) * | 2016-03-17 | 2019-05-15 | 株式会社東芝 | Imaging support apparatus, method and program |
US9639935B1 (en) * | 2016-05-25 | 2017-05-02 | Gopro, Inc. | Apparatus and methods for camera alignment model calibration |
EP3466051A1 (en) | 2016-05-25 | 2019-04-10 | GoPro, Inc. | Three-dimensional noise reduction |
WO2017205597A1 (en) * | 2016-05-25 | 2017-11-30 | Gopro, Inc. | Image signal processing-based encoding hints for motion estimation |
US10140776B2 (en) * | 2016-06-13 | 2018-11-27 | Microsoft Technology Licensing, Llc | Altering properties of rendered objects via control points |
US9851842B1 (en) * | 2016-08-10 | 2017-12-26 | Rovi Guides, Inc. | Systems and methods for adjusting display characteristics |
US10366122B2 (en) * | 2016-09-14 | 2019-07-30 | Ants Technology (Hk) Limited. | Methods circuits devices systems and functionally associated machine executable code for generating a searchable real-scene database |
CN110084088A (en) * | 2016-10-26 | 2019-08-02 | 奥康科技有限公司 | For analyzing image and providing the wearable device and method of feedback |
CN106550227B (en) * | 2016-10-27 | 2019-02-22 | 成都西纬科技有限公司 | A kind of image saturation method of adjustment and device |
US10477064B2 (en) | 2017-08-21 | 2019-11-12 | Gopro, Inc. | Image stitching with electronic rolling shutter correction |
US10791265B1 (en) | 2017-10-13 | 2020-09-29 | State Farm Mutual Automobile Insurance Company | Systems and methods for model-based analysis of damage to a vehicle |
US11587046B1 (en) | 2017-10-25 | 2023-02-21 | State Farm Mutual Automobile Insurance Company | Systems and methods for performing repairs to a vehicle |
JP7004736B2 (en) * | 2017-10-26 | 2022-01-21 | 京セラ株式会社 | Image processing equipment, imaging equipment, driving support equipment, mobile objects, and image processing methods |
JP2020507094A (en) * | 2017-12-25 | 2020-03-05 | 歌爾科技有限公司GoerTek Technology Co., Ltd. | Laser beam scanning display device and augmented reality glasses |
KR20200029270A (en) | 2018-09-10 | 2020-03-18 | 삼성전자주식회사 | Electronic apparatus for recognizing an object and controlling method thereof |
JP7456385B2 (en) * | 2018-10-25 | 2024-03-27 | ソニーグループ株式会社 | Image processing device, image processing method, and program |
US10771696B2 (en) * | 2018-11-26 | 2020-09-08 | Sony Corporation | Physically based camera motion compensation |
WO2020142471A1 (en) * | 2018-12-30 | 2020-07-09 | Sang Chul Kwon | Foldable mobile phone |
US11289078B2 (en) * | 2019-06-28 | 2022-03-29 | Intel Corporation | Voice controlled camera with AI scene detection for precise focusing |
US11064118B1 (en) * | 2019-12-18 | 2021-07-13 | Gopro, Inc. | Systems and methods for dynamic stabilization adjustment |
US11284157B2 (en) | 2020-06-11 | 2022-03-22 | Rovi Guides, Inc. | Methods and systems facilitating adjustment of multiple variables via a content guidance application |
TWI774039B (en) * | 2020-08-12 | 2022-08-11 | 瑞昱半導體股份有限公司 | System for compensating image with fixed pattern noise |
US11563899B2 (en) * | 2020-08-14 | 2023-01-24 | Raytheon Company | Parallelization technique for gain map generation using overlapping sub-images |
CN114079735B (en) * | 2020-08-19 | 2024-02-23 | 瑞昱半导体股份有限公司 | Image compensation system for fixed image noise |
US11902671B2 (en) * | 2021-12-09 | 2024-02-13 | Fotonation Limited | Vehicle occupant monitoring system including an image acquisition device with a rolling shutter image sensor |
WO2023150800A1 (en) * | 2022-02-07 | 2023-08-10 | Gopro, Inc. | Methods and apparatus for real-time guided encoding |
Family Cites Families (48)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100325253B1 (en) * | 1998-05-19 | 2002-03-04 | 미야즈 준이치롯 | Motion vector search method and apparatus |
US6486908B1 (en) * | 1998-05-27 | 2002-11-26 | Industrial Technology Research Institute | Image-based method and system for building spherical panoramas |
US20010047517A1 (en) * | 2000-02-10 | 2001-11-29 | Charilaos Christopoulos | Method and apparatus for intelligent transcoding of multimedia data |
JP2001245303A (en) * | 2000-02-29 | 2001-09-07 | Toshiba Corp | Moving picture coder and moving picture coding method |
US6407680B1 (en) * | 2000-12-22 | 2002-06-18 | Generic Media, Inc. | Distributed on-demand media transcoding system and method |
US7034848B2 (en) * | 2001-01-05 | 2006-04-25 | Hewlett-Packard Development Company, L.P. | System and method for automatically cropping graphical images |
CN1286326C (en) * | 2001-05-31 | 2006-11-22 | 佳能株式会社 | Information storing apparatus and method thereof |
US7801215B2 (en) * | 2001-07-24 | 2010-09-21 | Sasken Communication Technologies Limited | Motion estimation technique for digital video encoding applications |
US20030126622A1 (en) * | 2001-12-27 | 2003-07-03 | Koninklijke Philips Electronics N.V. | Method for efficiently storing the trajectory of tracked objects in video |
KR100850705B1 (en) * | 2002-03-09 | 2008-08-06 | 삼성전자주식회사 | Method for adaptive encoding motion image based on the temperal and spatial complexity and apparatus thereof |
JP4275358B2 (en) * | 2002-06-11 | 2009-06-10 | 株式会社日立製作所 | Image information conversion apparatus, bit stream converter, and image information conversion transmission method |
US7259784B2 (en) * | 2002-06-21 | 2007-08-21 | Microsoft Corporation | System and method for camera color calibration and image stitching |
US20040131276A1 (en) * | 2002-12-23 | 2004-07-08 | John Hudson | Region-based image processor |
AU2003296127A1 (en) * | 2002-12-25 | 2004-07-22 | Nikon Corporation | Blur correction camera system |
KR100566290B1 (en) * | 2003-09-18 | 2006-03-30 | 삼성전자주식회사 | Image Scanning Method By Using Scan Table and Discrete Cosine Transform Apparatus adapted it |
JP4123171B2 (en) * | 2004-03-08 | 2008-07-23 | ソニー株式会社 | Method for manufacturing vibration type gyro sensor element, vibration type gyro sensor element, and method for adjusting vibration direction |
WO2005094270A2 (en) * | 2004-03-24 | 2005-10-13 | Sharp Laboratories Of America, Inc. | Methods and systems for a/v input device to diplay networking |
US8315307B2 (en) * | 2004-04-07 | 2012-11-20 | Qualcomm Incorporated | Method and apparatus for frame prediction in hybrid video compression to enable temporal scalability |
US20060109900A1 (en) * | 2004-11-23 | 2006-05-25 | Bo Shen | Image data transcoding |
JP2006203682A (en) * | 2005-01-21 | 2006-08-03 | Nec Corp | Converting device of compression encoding bit stream for moving image at syntax level and moving image communication system |
WO2007044556A2 (en) * | 2005-10-07 | 2007-04-19 | Innovation Management Sciences, L.L.C. | Method and apparatus for scalable video decoder using an enhancement stream |
TW200816798A (en) * | 2006-09-22 | 2008-04-01 | Altek Corp | Method of automatic shooting by using an image recognition technology |
US7843824B2 (en) * | 2007-01-08 | 2010-11-30 | General Instrument Corporation | Method and apparatus for statistically multiplexing services |
US7924316B2 (en) * | 2007-03-14 | 2011-04-12 | Aptina Imaging Corporation | Image feature identification and motion compensation apparatus, systems, and methods |
WO2008142948A1 (en) * | 2007-05-23 | 2008-11-27 | Nec Corporation | Dynamic image distribution system, conversion device, and dynamic image distribution method |
WO2009017105A1 (en) * | 2007-07-30 | 2009-02-05 | Nec Corporation | Connection terminal, distribution system, conversion method, and program |
US20090060039A1 (en) * | 2007-09-05 | 2009-03-05 | Yasuharu Tanaka | Method and apparatus for compression-encoding moving image |
US8098732B2 (en) * | 2007-10-10 | 2012-01-17 | Sony Corporation | System for and method of transcoding video sequences from a first format to a second format |
US8063942B2 (en) * | 2007-10-19 | 2011-11-22 | Qualcomm Incorporated | Motion assisted image sensor configuration |
US8170342B2 (en) * | 2007-11-07 | 2012-05-01 | Microsoft Corporation | Image recognition of content |
JP2009152672A (en) * | 2007-12-18 | 2009-07-09 | Samsung Techwin Co Ltd | Recording apparatus, reproducing apparatus, recording method, reproducing method, and program |
JP5242151B2 (en) * | 2007-12-21 | 2013-07-24 | セミコンダクター・コンポーネンツ・インダストリーズ・リミテッド・ライアビリティ・カンパニー | Vibration correction control circuit and imaging apparatus including the same |
JP2009159359A (en) * | 2007-12-27 | 2009-07-16 | Samsung Techwin Co Ltd | Moving image data encoding apparatus, moving image data decoding apparatus, moving image data encoding method, moving image data decoding method and program |
US20090217338A1 (en) * | 2008-02-25 | 2009-08-27 | Broadcom Corporation | Reception verification/non-reception verification of base/enhancement video layers |
US20090323810A1 (en) * | 2008-06-26 | 2009-12-31 | Mediatek Inc. | Video encoding apparatuses and methods with decoupled data dependency |
US7990421B2 (en) * | 2008-07-18 | 2011-08-02 | Sony Ericsson Mobile Communications Ab | Arrangement and method relating to an image recording device |
JP2010039788A (en) * | 2008-08-05 | 2010-02-18 | Toshiba Corp | Image processing apparatus and method thereof, and image processing program |
JP2010147808A (en) * | 2008-12-18 | 2010-07-01 | Olympus Imaging Corp | Imaging apparatus and image processing method in same |
US8311115B2 (en) * | 2009-01-29 | 2012-11-13 | Microsoft Corporation | Video encoding using previously calculated motion information |
US20100194851A1 (en) * | 2009-02-03 | 2010-08-05 | Aricent Inc. | Panorama image stitching |
US20100229206A1 (en) * | 2009-03-03 | 2010-09-09 | Viasat, Inc. | Space shifting over forward satellite communication channels |
US8520083B2 (en) * | 2009-03-27 | 2013-08-27 | Canon Kabushiki Kaisha | Method of removing an artefact from an image |
US20100309975A1 (en) * | 2009-06-05 | 2010-12-09 | Apple Inc. | Image acquisition and transcoding system |
JP5473536B2 (en) * | 2009-10-28 | 2014-04-16 | 京セラ株式会社 | Portable imaging device with projector function |
US20110170608A1 (en) * | 2010-01-08 | 2011-07-14 | Xun Shi | Method and device for video transcoding using quad-tree based mode selection |
US8681255B2 (en) * | 2010-09-28 | 2014-03-25 | Microsoft Corporation | Integrated low power depth camera and projection device |
US9007428B2 (en) * | 2011-06-01 | 2015-04-14 | Apple Inc. | Motion-based image stitching |
US8554011B2 (en) * | 2011-06-07 | 2013-10-08 | Microsoft Corporation | Automatic exposure correction of images |
-
2011
- 2011-09-14 US US13/232,045 patent/US20130021488A1/en not_active Abandoned
- 2011-09-14 US US13/232,052 patent/US20130021512A1/en not_active Abandoned
- 2011-09-19 US US13/235,975 patent/US20130021504A1/en not_active Abandoned
- 2011-09-27 US US13/245,941 patent/US20130021489A1/en not_active Abandoned
- 2011-10-26 US US13/281,521 patent/US20130021490A1/en not_active Abandoned
- 2011-12-07 US US13/313,345 patent/US20130022116A1/en not_active Abandoned
- 2011-12-07 US US13/313,352 patent/US9092861B2/en active Active
- 2011-12-19 US US13/330,047 patent/US20130021484A1/en not_active Abandoned
-
2012
- 2012-03-07 US US13/413,863 patent/US20130021491A1/en not_active Abandoned
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9225905B2 (en) * | 2012-08-03 | 2015-12-29 | Samsung Electronics Co., Ltd. | Image processing method and apparatus |
US20140036108A1 (en) * | 2012-08-03 | 2014-02-06 | Samsung Electronics Co., Ltd. | Image processing method and apparatus |
US20140368514A1 (en) * | 2013-06-12 | 2014-12-18 | Infineon Technologies Ag | Device, method and system for processing an image data stream |
US10796617B2 (en) * | 2013-06-12 | 2020-10-06 | Infineon Technologies Ag | Device, method and system for processing an image data stream |
KR20150098094A (en) * | 2014-02-19 | 2015-08-27 | 삼성전자주식회사 | Image Processing Device and Method including a plurality of image signal processors |
US9538087B2 (en) * | 2014-02-19 | 2017-01-03 | Samsung Electronics Co., Ltd. | Image processing device with multiple image signal processors and image processing method |
KR102128468B1 (en) * | 2014-02-19 | 2020-06-30 | 삼성전자주식회사 | Image Processing Device and Method including a plurality of image signal processors |
US20150237280A1 (en) * | 2014-02-19 | 2015-08-20 | Samsung Electronics Co., Ltd. | Image processing device with multiple image signal processors and image processing method |
US20160366341A1 (en) * | 2014-02-27 | 2016-12-15 | Nubia Technology Co., Ltd. | Image processing method and imaging device |
US10194086B2 (en) * | 2014-02-27 | 2019-01-29 | Nubia Technology Co., Ltd. | Image processing method and imaging device |
US11405565B2 (en) * | 2014-05-08 | 2022-08-02 | Sony Group Corporation | Information processing device and information processing method |
US10861127B1 (en) * | 2019-09-17 | 2020-12-08 | Gopro, Inc. | Image and video processing using multiple pipelines |
US11443403B2 (en) | 2019-09-17 | 2022-09-13 | Gopro, Inc. | Image and video processing using multiple pipelines |
US11006044B1 (en) * | 2020-03-03 | 2021-05-11 | Qualcomm Incorporated | Power-efficient dynamic electronic image stabilization |
Also Published As
Publication number | Publication date |
---|---|
US20130021488A1 (en) | 2013-01-24 |
US20130021512A1 (en) | 2013-01-24 |
US20130021484A1 (en) | 2013-01-24 |
US9092861B2 (en) | 2015-07-28 |
US20130021490A1 (en) | 2013-01-24 |
US20130022116A1 (en) | 2013-01-24 |
US20130021483A1 (en) | 2013-01-24 |
US20130021489A1 (en) | 2013-01-24 |
US20130021491A1 (en) | 2013-01-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20130021504A1 (en) | Multiple image processing | |
US8553109B2 (en) | Concurrent image processing for generating an output image | |
US9270875B2 (en) | Dual image capture processing | |
JP5845464B2 (en) | Image processing apparatus, image processing method, and digital camera | |
US8199222B2 (en) | Low-light video frame enhancement | |
JP5853166B2 (en) | Image processing apparatus, image processing method, and digital camera | |
US9961272B2 (en) | Image capturing apparatus and method of controlling the same | |
US20120224766A1 (en) | Image processing apparatus, image processing method, and program | |
US8854503B2 (en) | Image enhancements through multi-image processing | |
US9774799B2 (en) | Method and apparatus for image data transfer in digital photographing | |
KR20160044945A (en) | Image photographing appratus | |
US8681235B2 (en) | Apparatus for processing digital image signal that obtains still image at desired point in time and method of controlling the apparatus | |
JP6702792B2 (en) | Image processing apparatus and control method thereof | |
JP2008072501A (en) | Imaging apparatus, imaging method, program, and recording medium | |
KR102014444B1 (en) | Photographing apparatus, method for controlling the same, and computer-readable recording medium | |
US20240179425A1 (en) | Image sensor with multiple image readout | |
US11843871B1 (en) | Smart high dynamic range image clamping | |
US20240078635A1 (en) | Compression of images for generating combined images | |
US20200204730A1 (en) | Imaging apparatus | |
WO2024118403A1 (en) | Image sensor with multiple image readout |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BROADCOM CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PLOWMAN, DAVID;PATUCK, NAUSHIR;SEWELL, BENJAMIN;REEL/FRAME:026928/0138 Effective date: 20110919 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH CAROLINA Free format text: PATENT SECURITY AGREEMENT;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:037806/0001 Effective date: 20160201 Owner name: BANK OF AMERICA, N.A., AS COLLATERAL AGENT, NORTH Free format text: PATENT SECURITY AGREEMENT;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:037806/0001 Effective date: 20160201 |
|
AS | Assignment |
Owner name: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD., SINGAPORE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:041706/0001 Effective date: 20170120 Owner name: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BROADCOM CORPORATION;REEL/FRAME:041706/0001 Effective date: 20170120 |
|
AS | Assignment |
Owner name: BROADCOM CORPORATION, CALIFORNIA Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS;ASSIGNOR:BANK OF AMERICA, N.A., AS COLLATERAL AGENT;REEL/FRAME:041712/0001 Effective date: 20170119 |