WO2015165528A1 - Method and a device for generating a high quality image using pixel shifted low resolution images - Google Patents
Method and a device for generating a high quality image using pixel shifted low resolution images Download PDFInfo
- Publication number
- WO2015165528A1 WO2015165528A1 PCT/EP2014/058849 EP2014058849W WO2015165528A1 WO 2015165528 A1 WO2015165528 A1 WO 2015165528A1 EP 2014058849 W EP2014058849 W EP 2014058849W WO 2015165528 A1 WO2015165528 A1 WO 2015165528A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- image sensor
- image
- pixels
- imaging device
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000003384 imaging method Methods 0.000 claims description 56
- 230000006870 function Effects 0.000 claims 2
- 230000008569 process Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000002059 diagnostic imaging Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002596 correlated effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 229920000642 polymer Polymers 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 101100504379 Mus musculus Gfral gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000007175 bidirectional communication Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4069—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution by subpixel displacements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N25/46—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by combining or binning pixels
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/48—Increasing resolution by shifting the sensor relative to the scene
Definitions
- the present invention relates to the field of image processing.
- the invention relates to a method of capturing high quality images using pixel shifted low resolution images without moving an image sensor.
- Background Imaging systems capture images using image sensors, which include a plurality of pixels which capture information associated with the object that is exposed.
- the imaging systems are equipped with image sensors of different sizes and capacity.
- the size and the ca- pacity of the image sensor can pose problems in capturing an object or a scene. For example, if the imaging sensor is large and has a high number of pixels, them the imaging system requires a long time for reading out the information captured by the image sensor and further process the image. This leads to a reduction in frame rate, which is a measure of how frequently the images are captured.
- Pixel binning is a technique wherein pixel values captured by a plurality of pixels on the image sensor are combined into a single value and an image is generated based on the combined pixel values. In this case, though the frame rate of the imaging system may increase the resolution of the result- ing image is low.
- Some imaging systems such as X-ray imaging systems, have flat panel detectors as image sensors.
- Flat panel detec- tors are generally large in size and have very low frame rates.
- X-ray imaging systems require more time to capture moving scenes thus resulting in the X-ray images having low frame rate.
- the frame rate of the X-ray imaging system can be increased to by using pixel binning technique. In such a case, the pixel binning results in the acquired image having a low resolution.
- a plurality of low resolution images are processed to generate a high resolution image, this technique is called super resolution enhancement.
- the plurality of low resolution images are integrated by averaging the low resolution frames thereby increasing the resolution.
- the aforementioned technique requires a large number of low resolution frames to reconstruct a super resolution frame resulting in the consumption of a lot of time and resources .
- the super resolution image is reconstructed using shift or spatial modulation.
- the spatially modulated images are reconstructed using a plurality of low resolution images which are acquired by physically moving the image sensor while capturing the low resolution frames.
- the plurality of low resolution images are pixel shifted from one another. Thereafter, the plurality of low resolution images are correlated with one another and a super resolution image is generated.
- X-ray detectors such as flat panel detectors (FPDs) are very bulky.
- FPDs flat panel detectors
- Such movement of the image sensor is not practical in case of bulky detectors like FPDs. Therefore, there is a need to accommodate the feature of spatial shift while generating low resolution images without moving the image sensor. Further, it is required that the frame rate of the image sensor is also increased for faster generation of super resolution images.
- the imaging systems not offer the flexibility to medical personnel to change a pixel binning size and a step size at which a pixel group, which is used to capture the image, is shifted on the image sensor. Therefore, there is a need for a system that allows medical imaging technicians to change parameters such as pixel binning size, pixel shift step value and a quality of the final image .
- the underlying objective of the invention is to provide an imaging device and a method for providing a user with an option to generate images having one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images, without having to move an image sensor.
- the imaging device comprises an image sensor, which includes a plurality of active elements known as pixels for capturing a view which is exposed to the image sensor.
- the plurality of pixels is ar- ranged in a plurality of rows and columns on the image sensor .
- the image sensor is a Flat Panel Detec- tor (FPD) for detecting X-rays.
- the image sensor may be a sensor used in radiographic and fluoroscopic imaging.
- the image sensors may also include Charge Coupled devices used in cameras and video recorders .
- the imaging device comprises the readout module configured to read out a plurality of super-pixel values from the image sensor, wherein the super-pixel values comprises a summation of the pixel values of pixels in at least a portion of the plurality of pixels, and wherein the pixel subsets are grouped into a pixel group.
- the portion of the plurality of pixels of the pixel subset is contiguous pixels on the image sensor.
- the pixel values captured by several pixels of the pixel subset are combined together into the super-pixel value.
- the pixel subset has a size such as, for exam- pie, 2X2, 3X3 and 4X4.
- the size of the pixel subset depends on factors such as, a desired resolution of a reconstructed image and a size of the image sensor.
- the super- pixel value is used to generate a low resolution image. Due to pixel binning, it is possible to increase the read- out time of the image thereby increasing the frame rate of the image sensor.
- the shift control module is configured to shift a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension.
- the readout patterns such as polygonal patterns, are shifted to acquire a plurality pixel shifted low resolution images.
- the readout pattern may be a square pattern, which can be construed as a footprint of the pixel group on the image sensor.
- the shift control module directs the readout module to change the position of the pixel group. The readout patterns are shifted in accordance with a predefined shift value.
- the predefined shift value is always maintained to be an inte- gral multiple of a pixel dimension.
- the pixel dimension is a dimension, for example a length and a breadth, of a single pixel on the image sensor.
- shifting the readout pattern results in the generation of pixel shifted low resolution images which can be used in reconstructing a final image having at least one of, a high resolution image and a high Signal to Noise Ratio (SNR) image, as compared to the low resolution images .
- the predefined step size is set by a user.
- the imaging device provides flexibility to the user to define the predefined step size which is considered by the imaging device for shifting the pixel group across the image sensor.
- the predefined shift value is also used during reconstruction of the final image.
- the shift control module is communicatively coupled to the readout module and the reconstruction module.
- the shift control module directs the readout module to read out the pixel in a particular format which includes pixel subsets and pixel groups.
- the user can modify the settings at any point by providing the necessary instructions to the shift control module. Further, the predefined step size by which the group is shifted across the image sensor needs to be communicated to the readout module.
- a reconstruction module is configured to generate a final image which has one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, based on a predefined set- ting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image.
- the reconstructed image is generated by using one or more estimation algorithms which correlate the plurality of pixel shifted low resolution images.
- reconstruction module retrieves the predefined step size from the shift control module to generate the final image.
- the predefined step size determines the reso- lution of a maximum resolution of the final image.
- the predefined step size determines the processing time and processing resources required by the reconstruction module .
- the final image has one of, a high resolution and a high SNR, making the final image useful in the field of X-ray imaging.
- the high SNR and high resolution can be achieved with minimum number of frames, as the plurality of low resolution images is pixel shifted.
- the pixel shifted low resolution images are correlated using optimization algorithms to generate the final image.
- An advantage of the imaging device disclosed herein having a predefined shift value is that, the user can define the step size at which the pixel group is shifted on the image sensor. By allowing a user to specify the pixel shift step size, the quality and the speed at which the final image is generated can be controlled effectively. Further, the readout pattern is shifted based on a predefined shift value.
- the predefined shift value determines the resolution level of the low resolution images that are captured.
- the predefined shift value also determines a maximum resolution of the reconstructed final image.
- the predefined shift value determines the frame rate of the imaging device.
- the imaging device and method are advantageous by having a predefined setting in that, a user can set the predefined setting to opt either a higher SNR or a higher resolution in the final image.
- the user can set the predefined setting to opt for the final image to have either a high SNR or a high resolution.
- the invention is also based on the object of specifying a method for generating images having one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images without having to move an image sensor, wherein the image sensor comprises a plurality of pixels arranged in a plurality of rows and columns.
- the method comprises the step of exposing an image sensor to capture an image, wherein the image sensor comprises a plurality of pixels arranged in a plu- rality of rows and columns.
- the method comprises the step of reading out a plurality of super-pixel values from the image sensor, wherein the super-pixel values comprises the pixel values of pixels in at least one pixel subset, wherein the pixel subset includes at least a portion of the plurality of pixels, and wherein the pixel subsets are grouped into a pixel group.
- the method comprises the step of shifting a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension .
- the method comprises the step of generating a final image which has one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, based on a predefined setting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image.
- SNR Signal to Noise Ratio
- the reconstruction module generates the final image based on optimization algorithms.
- the one or more algorithms include, for example, least square regularization and total variation minimization and the like. Further, the one or more algorithms can be updated by the user whenever there is an improvement in the algorithm. Furthermore, new algorithms can be added to increase the processing capability of the reconstruction module.
- t final image has at least one of, a SNR and a resolution, substantially higher than that of the plurality of low resolution images.
- the imaging device generates a final image using a minimum number of low resolution pixel shifted images.
- FIG 1 shows a block diagram of an exemplary imaging device for generating a final image using a plurality of low resolution images according to an embodiment of the invention
- FIG 2 shows a block diagram of an exemplary module constituting the imaging device in accordance with an embodiment of the invention
- FIG 3 shows an exemplary arrangement of pixel subset and a pixel group on an image sensor
- FIG 4A shows an initial position of the pixel group on the image sensor
- FIG 4B shows the pixel group shifted along a horizontal axis on the image sensor
- FIG 4C shows the pixel group shifted along a diagonal axis on the image sensor
- FIG 4D shows the pixel group shifted along a vertical axis on the image sensor
- FIG 5 shows a flowchart of an exemplary method steps in accordance with an embodiment of the invention
- FIG 1. An exemplary imaging device for generating images having at least one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images without having to move an image sensor according to one or more embodiments of the present invention is depicted in FIG 1.
- FIG 1 depicts an overview of an imaging device, the various components thereof, according to the present invention. It may be noted herein that cross-references will be made to other figures during the elucidation of a certain figure for the purpose of elucidation of the various components of the imaging device and/or the imaging device according to one or more embodiments of the present invention.
- FIG 1 illustrates a block diagram of an exemplary imaging device capable of generating images having at least one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images without having to move an image sensor according to the present invention.
- the imaging device 1 may be a personal computer, a laptop computer, a server computer, a tablet and the like. In FIG 1, the imaging device 1 is communicatively coupled to an image sensor 3.
- the imag- ing device includes a processor 5, a memory 7, a storage unit 9, and input/output devices 11.
- the input/output devices 11 includes a display unit 22.
- the processor 2 means any type of computa- tional circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing microprocessor, a reduced instruction set computing microprocessor, a very long instruction word microprocessor, an explicitly parallel instruction computing microprocessor, a graphics processor, a digital signal processor, or any other type of processing circuit.
- the processor 110 may also include embedded controllers, such as generic or programmable logic devices or arrays, application specific integrated circuits, single-chip computers, and the like.
- the memory 7 may be volatile memory and non-volatile memory.
- a variety of computer-readable storage media may be stored in and accessed from the memory elements.
- Memory elements may include any suitable memory device (s) for storing data and machine-readable instructions, such as read only memory, random access memory, erasable programmable read only memory, electrically erasable programmable read only memory, hard drive, removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges, memory cards, and the like.
- FIG 1 shows an exemplary arrangement of the imaging device.
- the imaging device 1 includes an image sensor 3 having a plurality of pixels.
- the memory 7 includes a readout module 14, a shift control module 16 and a reconstruction module 20.
- the modules in the memory 7 may interact as illustrated in FIG. 2.
- the modules may be stored in the memory 7 in the form of computer readable instructions or as an executable source code file.
- the image sensor 3 detects incident light impinging on a sur- face of the image sensor 3.
- the photo sensor includes a plurality of picture elements, referred herein as pixels, to detect the amount of light incident on the image sensor 3.
- the pixels are arranged in a plurality of rows and columns on the image sensor 3. When the image sensor 3 is exposed, the pixels detect the light incident on the image sensor 3 and store a charge value corresponding to an intensity of the incident light.
- the plurality of pixels of the image sensor 3 captures an image to which the image sensor 3 is exposed.
- the image sensor 3 is a flat panel detector (FPD) which is widely used in radiographic and fluoroscopic imaging.
- the image sensor 1 can include other sensors such as charge coupled devices (CCDs) .
- the memory 7 of the imaging device 1 includes the readout module 14 to read out the light intensity values captured by the plurality of pix- els in the image sensor 3.
- the readout module 14 reads out the pixel values of the plurality of pixels while capturing an image.
- a certain amount of time is required for reading out the plurality of pixels on the image sensor 3.
- the time required for reading out a single frame from the im- age sensor 3 is known as frame rate.
- the frame rate depends on the size and a readout speed of the image sensor 3. In cases where the image sensor 3 is very large, for example FPDs, the readout module 14 has to readout a large number of pixels therefore resulting in a low frame rate.
- the image sensor 3 and the readout module 14 may not be used in the imaging device 1.
- the images required for generating the final image may be stored in the storage unit 9 of the imaging device 1.
- the images may be retrieved directly from the storage unit 9 without using the image sensor 3 and the readout module 14.
- a high frame rate for capturing an image is desired.
- a process known a pixel binning is used to increase the frame rate of FPDs . The details associated with pixel binning process are explained further in conjunction with the shift control module 3.
- FIG 1 illustrates the shift control module 16 which directs the readout module 14 to read out the pixel values of the image sensor 3 in a particular pattern.
- the shift control module 16 is communicatively coupled to the readout module 14 in order to enable bidirectional communication with the readout module 14.
- the shift control module 16 is a part of the readout module 14 and the readout module 14 may perform the pixel binning process.
- the shift control module 16 increases the frame rate of the image sensor by combining pixel values of multiple pixels into a single pixel value, known as a super-pixel value.
- the shift control module 16 groups a number of pixels into a pixel subset. Further, the pixel values captured by each pixel in the pixel subset are added to generate a single super-pixel value. Furthermore, a plurality of pixel subsets are grouped together resulting in a pixel group.
- the shift control module 16 directs the readout module 14 to shift a readout pattern for reading pixel values.
- the readout pattern is a footprint of the pixel group on the image sensor.
- the readout module reads out the super-pixel values in a square pattern.
- the readout module 14 reads the values of all the pixels on the image sensor.
- the shift control module 16 directs the readout module 14 to read the pixel values in a particular pattern. Firstly, the shift control module 16 groups the pixel values in accordance with the pixel subsets and pixel group as explained earlier. Thereafter, the shift control module 16 shifts a position of a readout pattern of the readout module 14 across the image sensor 3. The shift is performed in accordance with a predefined shift value 17 specified in the shift control module 4. The predefined shift value is set by the shift control module 16.
- the shift control module 16 includes a provision to accept user inputs regarding the predefined shift value 17.
- the predefined shift value 17 determines the frame rate of the imaging device 1 as well as the maximum resolution of a final image that is generated.
- the imaging device 1 determines the predefined shift value 17 based on the desired qualities of a final image, for example, a high resolution or a high Signal to Noise (SNR) .
- SNR Signal to Noise
- FIG 3 shows an exemplary arrangement of pixel subset and a pixel group on an image sensor.
- the shift control module 16 directs the readout module 14 to read the pixel values by grouping the pixels as shown in FIG 3.
- FIG 3 shows a portion of the image sensor 3 having a plurality of pixels, such as pixel 25.
- the shift control module 16 may transmit instruction to the readout module to group pixel values into pixel subsets and pixel groups.
- the pixel subset 32 is a group of 2X2 pixels on the image sensor.
- the pixel subset 34 may be a group of 4X4 pixels on the image sensor.
- pixel group is a group of 4 pixel subsets.
- the pixel group can extend across the entire im- age sensor 3 or cover at least a portion of the image sensor 3.
- the pixel group is shifted across the image sensor 3 according to the predefined shift value 17.
- the predefined shift value 17 is maintained to be an integral multiple of single pixel dimension.
- the predefined shift value 17 may be set as 2 by the imaging device 1.
- the pixel group is moved in steps of 2 pixels across the image sensor.
- the pixel group 29 may be shifted along at least one of a vertical, horizontal and a diagonal direction across the image sensor 3. Further details regard- ing the pixel subsets and pixel group is provided in conjunction with FIG 4A- FIG 4D.
- provision may be provided for the user to specify a pixel subset size and a pixel group size in or- der to provide additional control to the user to manipulate the quality of the final image.
- the plurality of low resolution images is stored in the storage unit 9.
- the storage unit 9 of FIG 1 can be a volatile memory or a nonvolatile memory.
- a variety of computer-readable storage media may be stored in the memory unit and/or accessed from the memory unit.
- the storage unit 9 may include any suitable elements for storing data and machine-readable instructions, such as Read Only Memory (ROM) , Random Access Memory (RAM) , Erasable Programmable Read Only Memory (EPROM) , Electrically Erasable Programmable Read Only Memory (EEPROM) , Hard Disk Drive, removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges, memory cards, Universal Serial Buses (USBs) , et cetera.
- ROM Read Only Memory
- RAM Random Access Memory
- EPROM Erasable Programmable Read Only Memory
- EEPROM Electrically Erasable Programmable Read Only Memory
- Hard Disk Drive removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges
- the reconstruction module 20 includes one or more processing algorithms to process the plurality of low resolution images which are pixel shifted.
- the reconstruction module 20 includes algorithms to correlate or interpolate the pixel shifted low resolution images.
- the reconstruction module 20 generates a final image having either a high resolution or a high SNR.
- the user using the imaging device 1 can specify the type of the image that the reconstruction module 7 will generate by specifying the predetermined setting 21.
- the predefined setting 21 determines the properties of the final images that are generated.
- the predefined setting 21 determines if the final image has at least one of, a high resolution image and a high SNR image or a combination of varying levels of resolution and SNR.
- the display unit 22 includes one of an LCD (Liquid Crystal Diode) display, a TFT (Thin Film Transistor) LCD display, an LEP (Light
- Emitting Polymer or PLED Polymer Light Emitting Diode
- OLED Organic Light Emitting Diode
- CRT cathode ray tube
- the display unit 22 may be backlit via a backlight such that it may be viewed in the dark or other low-light environments.
- the shift control module 16 virtually groups the plurality of pixel on the image sensor into pixel subsets and pixel groups, as shown in FIG 4A.
- FIG 4A illustrates an initial position of the pixel group 34 on the image sensor 3.
- 2X2 pixels of the image sensor 3 are grouped into a pixel subset, such as pixel subset 32.
- the 4X4 pixels of the image sensor 3 are grouped into a pixel subset 34.
- the 4X4 pixels on the image sensor 3 are contiguous pixels. Further, such 2X2 pixel subsets are grouped into a pixel group 36.
- the readout module 14 reads out the super-pixel values sensed by the image sensor 3.
- the shift control module 16 directs the readout module 14 to combine the pixel values of individual pixels in the 4X4 pixel subset into a single super-pixel value. Thereafter, the shift control module 16 shifts the pixel group in at least of a, horizontal, vertical or a di- agonal direction on the image sensor based on the predefined step size.
- FIGS 4B, 4C and 4D show the pixel group 34 shifted along a horizontal, a diagonal and a vertical axes respectively on the image sensor 3. It can be further noted that the pixel group 34 is shifted based on the predefined shift value 17 that is 1 in this case.
- the plurality of low resolution images is stored in the memory 7.
- the storage in the memory 7 may be optional .
- the low resolution images are generated quickly as the numbers of pixels are substantially reduced due to pixel binning.
- an image sensor may have 100 pixels and may take 1 second to generate an image reading out all the 100 pixels. Due to pixel binning, if a group of 2X2 pixels are binned together into one super-pixel value, then the time to read the pixel values is reduced to 0.25 seconds as the number of pixels values to be read is reduced to 25.
- the reconstruction module 20 accesses the plurality of low resolution images in order to generate the final image.
- the final image has at least one of, a substantially high SNR or a substantially high resolution, based on a predefined setting 21 which is selected by the user. Based on the selection of the predefined setting 21, the reconstruction module 20 generates the final image with the desired qualities.
- the qualities desired by the user may include a high resolution, a high SNR or varying combination of both.
- the reconstruction module 20 includes one or more algorithms such as, least square minimization and total variation minimization, which are used to generate the final image. How- ever, the algorithms may be added, deleted and modified whenever necessary.
- the final image may be reconstructed using a plurality of pixel shifted low resolution images stored in the storage unit 9 of the imaging device 1, without using the image sensor 3 to capture the im- age .
- FIG 5 shows the flow diagram illustrating an exemplary method for genereating a final image, using a processor, with at least one of a high SNR and a high resolution from a plurali- ty of pixel shifted low resolution images.
- an image sensor having a plurality of pixels, is exposed to capture an image .
- a plurality of super-pixel values are read out from an image sensor, wherein the super-pixel value is generated by combining pixel values of pixels in the at least one pixel subset, wherein the pixel subset includes at least a portion of the plurality of pixels and wherein a plurality of pixel subsets is grouped into a pixel group.
- a position of the pixel group is shifted across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension.
- a final image which has at least one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, is generated based on a predefined setting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image.
- SNR Signal to Noise Ratio
- the said final image is displayed on a display module.
- the present invention can take a form of a computer program product comprising program modules accessible from computer- usable or computer-readable medium storing program code for use by or in connection with one or more computers, processors, or instruction execution system.
- a computer-usable or computer-readable me- dium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- the medium can be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or appa- ratus or device) or a propagation medium (through propagation mediums in and of themselves as signal carriers are not included in the definition of physical computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM) , a read-only memory (ROM) , a rigid magnetic disk and an optical disk such as compact disk read-only memory (CD-ROM) , compact disk-read/write) and Digital Versatile/Video Disc (DVD) .
- Both processors and program code for implementing each aspect of the technology can be centralized or distributed (or a combination thereof) as known to those skilled in the art.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The present invention relates to a method and a device for generating a high quality image with at least one of a high Signal to Noise Ratio (SNR) and a high resolution using low resolution pixel shifted images. In an embodiment, the method includes, reading out a plurality of super-pixel values from a pixel group on an image sensor. Further, the method includes generating low resolution pixel shifted images by shifting a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size. Furthermore, the method includes generating a final image which has one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, wherein the final image is generated using the low resolution images and the predefined step size.
Description
Description
Method and a Device for Generating a High Quality Image Using Pixel Shifted Low Resolution Images
Field of Invention
The present invention relates to the field of image processing. In particular, the invention relates to a method of capturing high quality images using pixel shifted low resolution images without moving an image sensor.
Background Imaging systems capture images using image sensors, which include a plurality of pixels which capture information associated with the object that is exposed. The imaging systems are equipped with image sensors of different sizes and capacity. In some applications, the size and the ca- pacity of the image sensor can pose problems in capturing an object or a scene. For example, if the imaging sensor is large and has a high number of pixels, them the imaging system requires a long time for reading out the information captured by the image sensor and further process the image. This leads to a reduction in frame rate, which is a measure of how frequently the images are captured.
However, methods such as pixel binning are used to increase the frame rate of imaging systems. Pixel binning is a technique wherein pixel values captured by a plurality of pixels on the image sensor are combined into a single value and an image is generated based on the combined pixel values. In this case, though the frame rate of the imaging system may increase the resolution of the result- ing image is low.
Some imaging systems, such as X-ray imaging systems, have flat panel detectors as image sensors. Flat panel detec-
tors are generally large in size and have very low frame rates. As a result, X-ray imaging systems require more time to capture moving scenes thus resulting in the X-ray images having low frame rate. The frame rate of the X-ray imaging system can be increased to by using pixel binning technique. In such a case, the pixel binning results in the acquired image having a low resolution.
In order to mitigate the loss of resolution due to pixel binning, a plurality of low resolution images are processed to generate a high resolution image, this technique is called super resolution enhancement. Typically, while performing resolution enhancement, the plurality of low resolution images are integrated by averaging the low resolution frames thereby increasing the resolution. The aforementioned technique requires a large number of low resolution frames to reconstruct a super resolution frame resulting in the consumption of a lot of time and resources .
In some cases, the super resolution image is reconstructed using shift or spatial modulation. The spatially modulated images are reconstructed using a plurality of low resolution images which are acquired by physically moving the image sensor while capturing the low resolution frames.
Due to the movement of the image sensor the plurality of low resolution images are pixel shifted from one another. Thereafter, the plurality of low resolution images are correlated with one another and a super resolution image is generated.
Generally, X-ray detectors such as flat panel detectors (FPDs) are very bulky. Such movement of the image sensor is not practical in case of bulky detectors like FPDs. Therefore, there is a need to accommodate the feature of spatial shift while generating low resolution images without moving the image sensor. Further, it is required that
the frame rate of the image sensor is also increased for faster generation of super resolution images.
Currently, there are systems where pixel shifted low reso- lution images are used for generating high resolution images. However, in medical imaging industry which includes radiographic and fluoroscopic imaging, there may be instances when merely a high resolution image may not suffice. In an exemplary scenario, a detailed analysis of an anatomy of a subject may be required. In such a case, there may be a need for increased Signal to Noise ratio (SNR) in a final X-ray image rather than a higher resolution. Currently, there are no methods to balance between a requirement of a high SNR and a high resolution.
Further, in medical imaging there may be a need of changing the frame rate of the imaging system in order to change the speed at which the images are captured. Currently, the imaging systems not offer the flexibility to medical personnel to change a pixel binning size and a step size at which a pixel group, which is used to capture the image, is shifted on the image sensor. Therefore, there is a need for a system that allows medical imaging technicians to change parameters such as pixel binning size, pixel shift step value and a quality of the final image .
The underlying objective of the invention is to provide an imaging device and a method for providing a user with an option to generate images having one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images, without having to move an image sensor.
According to the invention, the imaging device comprises an image sensor, which includes a plurality of active elements known as pixels for capturing a view which is exposed to the image sensor. The plurality of pixels is ar-
ranged in a plurality of rows and columns on the image sensor .
In an embodiment, the image sensor is a Flat Panel Detec- tor (FPD) for detecting X-rays. Further, the image sensor may be a sensor used in radiographic and fluoroscopic imaging. In some exemplary embodiments, the image sensors may also include Charge Coupled devices used in cameras and video recorders .
In accordance with the invention, the imaging device comprises the readout module configured to read out a plurality of super-pixel values from the image sensor, wherein the super-pixel values comprises a summation of the pixel values of pixels in at least a portion of the plurality of pixels, and wherein the pixel subsets are grouped into a pixel group. In an embodiment, the portion of the plurality of pixels of the pixel subset is contiguous pixels on the image sensor.
In a process known as pixel binning, the pixel values captured by several pixels of the pixel subset are combined together into the super-pixel value. In an exemplary embodiment, the pixel subset has a size such as, for exam- pie, 2X2, 3X3 and 4X4. The size of the pixel subset depends on factors such as, a desired resolution of a reconstructed image and a size of the image sensor. The super- pixel value is used to generate a low resolution image. Due to pixel binning, it is possible to increase the read- out time of the image thereby increasing the frame rate of the image sensor.
In accordance with the invention, the shift control module is configured to shift a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension. In accordance with the embodiment,
the readout patterns, such as polygonal patterns, are shifted to acquire a plurality pixel shifted low resolution images. In an example, the readout pattern may be a square pattern, which can be construed as a footprint of the pixel group on the image sensor. Further, the shift control module directs the readout module to change the position of the pixel group. The readout patterns are shifted in accordance with a predefined shift value. The predefined shift value is always maintained to be an inte- gral multiple of a pixel dimension. The pixel dimension is a dimension, for example a length and a breadth, of a single pixel on the image sensor. In accordance with the embodiment, shifting the readout pattern results in the generation of pixel shifted low resolution images which can be used in reconstructing a final image having at least one of, a high resolution image and a high Signal to Noise Ratio (SNR) image, as compared to the low resolution images . In accordance with the invention, the predefined step size is set by a user. The imaging device provides flexibility to the user to define the predefined step size which is considered by the imaging device for shifting the pixel group across the image sensor. The predefined shift value is also used during reconstruction of the final image.
In accordance with the invention, the shift control module is communicatively coupled to the readout module and the reconstruction module. The shift control module directs the readout module to read out the pixel in a particular format which includes pixel subsets and pixel groups. The user can modify the settings at any point by providing the necessary instructions to the shift control module. Further, the predefined step size by which the group is shifted across the image sensor needs to be communicated to the readout module.
r
In accordance with the invention, a reconstruction module is configured to generate a final image which has one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, based on a predefined set- ting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image. The reconstructed image is generated by using one or more estimation algorithms which correlate the plurality of pixel shifted low resolution images.
Further, reconstruction module retrieves the predefined step size from the shift control module to generate the final image. The predefined step size determines the reso- lution of a maximum resolution of the final image. The predefined step size determines the processing time and processing resources required by the reconstruction module .
The final image has one of, a high resolution and a high SNR, making the final image useful in the field of X-ray imaging. The high SNR and high resolution can be achieved with minimum number of frames, as the plurality of low resolution images is pixel shifted. The pixel shifted low resolution images are correlated using optimization algorithms to generate the final image.
An advantage of the imaging device disclosed herein having a predefined shift value is that, the user can define the step size at which the pixel group is shifted on the image sensor. By allowing a user to specify the pixel shift step size, the quality and the speed at which the final image is generated can be controlled effectively. Further, the readout pattern is shifted based on a predefined shift value. The predefined shift value determines the resolution level of the low resolution images that are captured. The predefined shift value also determines a
maximum resolution of the reconstructed final image. The predefined shift value determines the frame rate of the imaging device. The imaging device and method are advantageous by having a predefined setting in that, a user can set the predefined setting to opt either a higher SNR or a higher resolution in the final image. In medical imaging such as, X-ray imaging, there can be instances when a higher resolution im- age is required with an acceptable level of SNR and vice versa. In such a case, the user can set the predefined setting to opt for the final image to have either a high SNR or a high resolution.
The invention is also based on the object of specifying a method for generating images having one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images without having to move an image sensor, wherein the image sensor comprises a plurality of pixels arranged in a plurality of rows and columns.
In an embodiment, the method comprises the step of exposing an image sensor to capture an image, wherein the image sensor comprises a plurality of pixels arranged in a plu- rality of rows and columns.
In another embodiment, the method comprises the step of reading out a plurality of super-pixel values from the image sensor, wherein the super-pixel values comprises the pixel values of pixels in at least one pixel subset, wherein the pixel subset includes at least a portion of the plurality of pixels, and wherein the pixel subsets are grouped into a pixel group. In yet another embodiment, the method comprises the step of shifting a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined
step size is equal to an integral multiple of a pixel dimension .
In yet another embodiment, the method comprises the step of generating a final image which has one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, based on a predefined setting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image.
According to the invention, the reconstruction module generates the final image based on optimization algorithms. The one or more algorithms include, for example, least square regularization and total variation minimization and the like. Further, the one or more algorithms can be updated by the user whenever there is an improvement in the algorithm. Furthermore, new algorithms can be added to increase the processing capability of the reconstruction module.
Another advantageous feature of the invention is that, t final image has at least one of, a SNR and a resolution, substantially higher than that of the plurality of low resolution images. The imaging device generates a final image using a minimum number of low resolution pixel shifted images.
For further elucidation of the invention,
FIG 1 shows a block diagram of an exemplary imaging device for generating a final image using a plurality of low resolution images according to an embodiment of the invention;
FIG 2 shows a block diagram of an exemplary module constituting the imaging device in accordance with an embodiment of the invention,
FIG 3 shows an exemplary arrangement of pixel subset and a pixel group on an image sensor,
FIG 4A shows an initial position of the pixel group on the image sensor;
FIG 4B shows the pixel group shifted along a horizontal axis on the image sensor;
FIG 4C shows the pixel group shifted along a diagonal axis on the image sensor;
FIG 4D shows the pixel group shifted along a vertical axis on the image sensor;
FIG 5 shows a flowchart of an exemplary method steps in accordance with an embodiment of the invention;
An exemplary imaging device for generating images having at least one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images without having to move an image sensor according to one or more embodiments of the present invention is depicted in FIG 1.
FIG 1 depicts an overview of an imaging device, the various components thereof, according to the present invention. It may be noted herein that cross-references will be made to other figures during the elucidation of a certain figure for the purpose of elucidation of the various components of the imaging device and/or the imaging device according to one or more embodiments of the present invention. FIG 1 illustrates a block diagram of an exemplary imaging device capable of generating images having at least one of, a high SNR and a high resolution, using a plurality of pixel shifted low resolution images without having to move an image
sensor according to the present invention. The imaging device 1 may be a personal computer, a laptop computer, a server computer, a tablet and the like. In FIG 1, the imaging device 1 is communicatively coupled to an image sensor 3. The imag- ing device includes a processor 5, a memory 7, a storage unit 9, and input/output devices 11. The input/output devices 11 includes a display unit 22.
The processor 2, as used herein, means any type of computa- tional circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing microprocessor, a reduced instruction set computing microprocessor, a very long instruction word microprocessor, an explicitly parallel instruction computing microprocessor, a graphics processor, a digital signal processor, or any other type of processing circuit. The processor 110 may also include embedded controllers, such as generic or programmable logic devices or arrays, application specific integrated circuits, single-chip computers, and the like.
The memory 7 may be volatile memory and non-volatile memory. A variety of computer-readable storage media may be stored in and accessed from the memory elements. Memory elements may include any suitable memory device (s) for storing data and machine-readable instructions, such as read only memory, random access memory, erasable programmable read only memory, electrically erasable programmable read only memory, hard drive, removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges, memory cards, and the like.
FIG 1 shows an exemplary arrangement of the imaging device. The imaging device 1 includes an image sensor 3 having a plurality of pixels. The memory 7 includes a readout module 14, a shift control module 16 and a reconstruction module 20. The modules in the memory 7 may interact as illustrated in FIG. 2. The modules may be stored in the memory 7 in the form of
computer readable instructions or as an executable source code file.
The image sensor 3 detects incident light impinging on a sur- face of the image sensor 3. The photo sensor includes a plurality of picture elements, referred herein as pixels, to detect the amount of light incident on the image sensor 3. In an embodiment, the pixels are arranged in a plurality of rows and columns on the image sensor 3. When the image sensor 3 is exposed, the pixels detect the light incident on the image sensor 3 and store a charge value corresponding to an intensity of the incident light. The plurality of pixels of the image sensor 3 captures an image to which the image sensor 3 is exposed.
In a preferred embodiment, the image sensor 3 is a flat panel detector (FPD) which is widely used in radiographic and fluoroscopic imaging. In some alternate embodiments, the image sensor 1 can include other sensors such as charge coupled devices (CCDs) .
In the embodiment illustrated in FIG 1, the memory 7 of the imaging device 1 includes the readout module 14 to read out the light intensity values captured by the plurality of pix- els in the image sensor 3. In general, the readout module 14 reads out the pixel values of the plurality of pixels while capturing an image. A certain amount of time is required for reading out the plurality of pixels on the image sensor 3. The time required for reading out a single frame from the im- age sensor 3 is known as frame rate. The frame rate depends on the size and a readout speed of the image sensor 3. In cases where the image sensor 3 is very large, for example FPDs, the readout module 14 has to readout a large number of pixels therefore resulting in a low frame rate.
In an alternate embodiment, the image sensor 3 and the readout module 14 may not be used in the imaging device 1. The images required for generating the final image may be stored
in the storage unit 9 of the imaging device 1. The images may be retrieved directly from the storage unit 9 without using the image sensor 3 and the readout module 14. In some medical applications, a high frame rate for capturing an image is desired. In such a scenario, a process known a pixel binning is used to increase the frame rate of FPDs . The details associated with pixel binning process are explained further in conjunction with the shift control module 3.
FIG 1 illustrates the shift control module 16 which directs the readout module 14 to read out the pixel values of the image sensor 3 in a particular pattern. The shift control module 16 is communicatively coupled to the readout module 14 in order to enable bidirectional communication with the readout module 14. In some embodiments, the shift control module 16 is a part of the readout module 14 and the readout module 14 may perform the pixel binning process. In an embodiment, the shift control module 16 increases the frame rate of the image sensor by combining pixel values of multiple pixels into a single pixel value, known as a super-pixel value. The shift control module 16 groups a number of pixels into a pixel subset. Further, the pixel values captured by each pixel in the pixel subset are added to generate a single super-pixel value. Furthermore, a plurality of pixel subsets are grouped together resulting in a pixel group.
In an embodiment, the shift control module 16 directs the readout module 14 to shift a readout pattern for reading pixel values. The readout pattern is a footprint of the pixel group on the image sensor. In an example, the readout module reads out the super-pixel values in a square pattern. Conventionally, the readout module 14 reads the values of all the pixels on the image sensor. In the preferred embodiment as shown in FIG 1, the shift control module 16 directs the readout module 14 to read the pixel values in a particular pattern. Firstly, the shift control module 16 groups the pixel values in accordance with the pixel subsets and pixel group
as explained earlier. Thereafter, the shift control module 16 shifts a position of a readout pattern of the readout module 14 across the image sensor 3. The shift is performed in accordance with a predefined shift value 17 specified in the shift control module 4. The predefined shift value is set by the shift control module 16.
In some embodiments, the shift control module 16 includes a provision to accept user inputs regarding the predefined shift value 17. The predefined shift value 17 determines the frame rate of the imaging device 1 as well as the maximum resolution of a final image that is generated. The imaging device 1 determines the predefined shift value 17 based on the desired qualities of a final image, for example, a high resolution or a high Signal to Noise (SNR) .
FIG 3 shows an exemplary arrangement of pixel subset and a pixel group on an image sensor. The shift control module 16 directs the readout module 14 to read the pixel values by grouping the pixels as shown in FIG 3. FIG 3 shows a portion of the image sensor 3 having a plurality of pixels, such as pixel 25. The shift control module 16 may transmit instruction to the readout module to group pixel values into pixel subsets and pixel groups. For example, as shown in FIG. 3, the pixel subset 32 is a group of 2X2 pixels on the image sensor. In another instance, the pixel subset 34 may be a group of 4X4 pixels on the image sensor. In an embodiment, pixel group is a group of 4 pixel subsets. In an exemplary embodiment, the pixel group can extend across the entire im- age sensor 3 or cover at least a portion of the image sensor 3.
Now referring back to the shift control module 16, the pixel group is shifted across the image sensor 3 according to the predefined shift value 17. In an exemplary embodiment, the predefined shift value 17 is maintained to be an integral multiple of single pixel dimension. For example, the predefined shift value 17 may be set as 2 by the imaging device 1.
In such a case, the pixel group is moved in steps of 2 pixels across the image sensor. The pixel group 29 may be shifted along at least one of a vertical, horizontal and a diagonal direction across the image sensor 3. Further details regard- ing the pixel subsets and pixel group is provided in conjunction with FIG 4A- FIG 4D.
In some embodiments, provision may be provided for the user to specify a pixel subset size and a pixel group size in or- der to provide additional control to the user to manipulate the quality of the final image.
In an embodiment, the plurality of low resolution images is stored in the storage unit 9. It may be noted herein that the storage unit 9 of FIG 1 can be a volatile memory or a nonvolatile memory. A variety of computer-readable storage media may be stored in the memory unit and/or accessed from the memory unit. The storage unit 9 may include any suitable elements for storing data and machine-readable instructions, such as Read Only Memory (ROM) , Random Access Memory (RAM) , Erasable Programmable Read Only Memory (EPROM) , Electrically Erasable Programmable Read Only Memory (EEPROM) , Hard Disk Drive, removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges, memory cards, Universal Serial Buses (USBs) , et cetera.
In the next stage, the plurality of low resolution images is received by the reconstruction module 20, as shown in FIG 2. The reconstruction module 20 includes one or more processing algorithms to process the plurality of low resolution images which are pixel shifted. The reconstruction module 20 includes algorithms to correlate or interpolate the pixel shifted low resolution images. The reconstruction module 20 generates a final image having either a high resolution or a high SNR. The user using the imaging device 1 can specify the type of the image that the reconstruction module 7 will generate by specifying the predetermined setting 21.
In one exemplary embodiment, the predefined setting 21 determines the properties of the final images that are generated. The predefined setting 21 determines if the final image has at least one of, a high resolution image and a high SNR image or a combination of varying levels of resolution and SNR.
Further, the final image generated by the reconstruction module 20 is displayed on the display unit 22. The display unit 22 includes one of an LCD (Liquid Crystal Diode) display, a TFT (Thin Film Transistor) LCD display, an LEP (Light
Emitting Polymer or PLED (Polymer Light Emitting Diode) display, Organic Light Emitting Diode (OLED) , a cathode ray tube (CRT) , and so forth, capable of displaying text and graphical information. The display unit 22 may be backlit via a backlight such that it may be viewed in the dark or other low-light environments.
Now, the functioning of the shift control module 16 and the subsequent generation of the pixel shifted low resolution im- ages is explained. Firstly, the shift control module 16 virtually groups the plurality of pixel on the image sensor into pixel subsets and pixel groups, as shown in FIG 4A.
FIG 4A illustrates an initial position of the pixel group 34 on the image sensor 3. As shown in FIG 4A, 2X2 pixels of the image sensor 3 are grouped into a pixel subset, such as pixel subset 32. In an embodiment, the 4X4 pixels of the image sensor 3 are grouped into a pixel subset 34. The 4X4 pixels on the image sensor 3 are contiguous pixels. Further, such 2X2 pixel subsets are grouped into a pixel group 36.
When the image sensor 3 is exposed to capture an image, the readout module 14 reads out the super-pixel values sensed by the image sensor 3. The shift control module 16 directs the readout module 14 to combine the pixel values of individual pixels in the 4X4 pixel subset into a single super-pixel value. Thereafter, the shift control module 16 shifts the pixel group in at least of a, horizontal, vertical or a di-
agonal direction on the image sensor based on the predefined step size. FIGS 4B, 4C and 4D show the pixel group 34 shifted along a horizontal, a diagonal and a vertical axes respectively on the image sensor 3. It can be further noted that the pixel group 34 is shifted based on the predefined shift value 17 that is 1 in this case.
Subsequently, super-pixel values generated by the readout module 14 and a plurality of low resolution pixel shifted im- ages are generated. In an embodiment of the invention, the plurality of low resolution images is stored in the memory 7. However, a person skilled in the art may acknowledge that the storage in the memory 7 may be optional . The low resolution images are generated quickly as the numbers of pixels are substantially reduced due to pixel binning. In an example, an image sensor may have 100 pixels and may take 1 second to generate an image reading out all the 100 pixels. Due to pixel binning, if a group of 2X2 pixels are binned together into one super-pixel value, then the time to read the pixel values is reduced to 0.25 seconds as the number of pixels values to be read is reduced to 25.
Thereafter, the reconstruction module 20 accesses the plurality of low resolution images in order to generate the final image. The final image has at least one of, a substantially high SNR or a substantially high resolution, based on a predefined setting 21 which is selected by the user. Based on the selection of the predefined setting 21, the reconstruction module 20 generates the final image with the desired qualities. The qualities desired by the user may include a high resolution, a high SNR or varying combination of both. The reconstruction module 20 includes one or more algorithms such as, least square minimization and total variation minimization, which are used to generate the final image. How- ever, the algorithms may be added, deleted and modified whenever necessary.
In an alternate embodiment of the invention, the final image may be reconstructed using a plurality of pixel shifted low resolution images stored in the storage unit 9 of the imaging device 1, without using the image sensor 3 to capture the im- age .
FIG 5 shows the flow diagram illustrating an exemplary method for genereating a final image, using a processor, with at least one of a high SNR and a high resolution from a plurali- ty of pixel shifted low resolution images. At step 40, an image sensor, having a plurality of pixels, is exposed to capture an image .
At step 42, a plurality of super-pixel values are read out from an image sensor, wherein the super-pixel value is generated by combining pixel values of pixels in the at least one pixel subset, wherein the pixel subset includes at least a portion of the plurality of pixels and wherein a plurality of pixel subsets is grouped into a pixel group.
At step 44, a position of the pixel group is shifted across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension.
At step 46, a final image which has at least one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, is generated based on a predefined setting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image.
At step 48, the said final image is displayed on a display module.
The present invention can take a form of a computer program product comprising program modules accessible from computer-
usable or computer-readable medium storing program code for use by or in connection with one or more computers, processors, or instruction execution system. For the purpose of this description, a computer-usable or computer-readable me- dium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or appa- ratus or device) or a propagation medium (through propagation mediums in and of themselves as signal carriers are not included in the definition of physical computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM) , a read-only memory (ROM) , a rigid magnetic disk and an optical disk such as compact disk read-only memory (CD-ROM) , compact disk-read/write) and Digital Versatile/Video Disc (DVD) . Both processors and program code for implementing each aspect of the technology can be centralized or distributed (or a combination thereof) as known to those skilled in the art.
While the present invention has been described in detail with reference to certain embodiments, it should be appreciated that the present invention is not limited to those embodi- ments. In view of the present disclosure, many modifications and variations would be present themselves, to those skilled in the art without departing from the scope of the various embodiments of the present invention, as described herein. The scope of the present invention is, therefore, indicated by the following claims rather than by the foregoing description. All changes, modifications, and variations coming within the meaning and range of equivalency of the claims are to be considered within their scope.
List of Reference Signs
I Imaging device
3 Image sensor
5 Processor
7 Memory
9 Storage Unit
II Input/Output Unit
14 Readout Module
16 Shift Control Module
20 Reconstruction Module
22 Display Unit
25 A single pixel
32 2X2 Pixel Subset
34 4X4 Pixel subset
36 Pixel group
40 Step of exposing an image sensor
42 Step of reading out super-pixel values from the image sensor
44 Step of shifting a position of the pixel group
46 Step of generating a final image
48 Step of displaying the final image
Claims
1. An imaging device (1) for maximizing at least one of, a Signal to Noise Ratio (SNR) and a resolution of an image, the imaging device (1) comprising:
an image sensor (3) , wherein the image sensor comprises a plurality of pixels, wherein the image sensor is exposed to capture an image;
a processor (5) connected to the image sensor;
a memory (7) coupled to the processor, wherein the memory includes:
a readout module, wherein the readout module is configured to read out a plurality of super- pixel values from the image sensor (3) , wherein the super-pixel values comprises the pixel values of pixels in at least one pixel subset, wherein the pixel subset includes at least a portion of the plurality of pixels, and wherein the pixel subsets are grouped into a pixel group;
a shift control module, wherein the shift control module is configured to shift a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension;
a reconstruction module, wherein the reconstruction module is configured to generate a final image which has at least one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, based on a predefined setting, wherein the reconstruction module uses a plurality of low resolution images and the predefined step size to generate the final image.
The imaging device (1) according to claim 1, wherein the image sensor is a Flat Panel Detector (FPD) .
The imaging device (1) according to claim 1, wherein the portion of plurality of pixels in the pixel subset are contiguous .
The imaging device (1) according to claim 1, wherein the shift control module is configured to direct the readout module to shift the pixel group.
The imaging device (1) according to claim 1, wherein the predefined setting is user settable.
The imaging device (1) according to any of the claims 1- 3 wherein, the readout module is configured to read out a single value from the pixel subset.
The imaging device (1) according to claim 1 wherein, the resolution of the final image is a function of the predefined step size.
The imaging device (1) according to claim 1, wherein the final image has at least one of, a SNR and a resolution, substantially higher than each of the plurality of low resolution images.
A method for maximizing at least one of a Signal to Noise Ratio (SNR) and a resolution of an image, the method comprising:
exposing an image sensor to capture an image, wherein the image sensor comprises a plurality of pixels;
reading out, using a processor, a plurality of super-pixel values from the image sensor, wherein the super-pixel values comprises the pixel values of pixels in at least one pixel subset, wherein the pixel subset includes at least a portion of the plurality of pixels,
and wherein the pixel subsets are grouped into a pixel group ;
shifting a position of the pixel group across the image sensor, wherein the at least one pixel group is shifted based on a predefined step size, wherein the predefined step size is equal to an integral multiple of a pixel dimension;
generating a final image which has at least one of, a substantially high Signal to Noise Ratio (SNR) and a substantially high resolution, based on a predefined setting, wherein the final image is generated using a plurality of low resolution images and the predefined step size.
O.The method according to claim 9, wherein values of the pixels in the pixel subset are combined into a single super-pixel value.
1. The method according to claim 9, wherein the pixel group is shifted without moving the image sensor.
2. The method according to claim 9, wherein the resolution of the high resolution image is a function of the predefined step size.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2014/058849 WO2015165528A1 (en) | 2014-04-30 | 2014-04-30 | Method and a device for generating a high quality image using pixel shifted low resolution images |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2014/058849 WO2015165528A1 (en) | 2014-04-30 | 2014-04-30 | Method and a device for generating a high quality image using pixel shifted low resolution images |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015165528A1 true WO2015165528A1 (en) | 2015-11-05 |
Family
ID=50685898
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2014/058849 WO2015165528A1 (en) | 2014-04-30 | 2014-04-30 | Method and a device for generating a high quality image using pixel shifted low resolution images |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2015165528A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230088836A1 (en) * | 2020-03-31 | 2023-03-23 | Sony Group Corporation | Image processing device and method, and program |
WO2024018238A1 (en) * | 2022-07-22 | 2024-01-25 | Touch Biometrix Limited | Sensor and method of operating a sensor |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090268055A1 (en) * | 2008-04-29 | 2009-10-29 | Hamilton Jr John F | Concentric exposure sequence for image sensor |
EP2528320A1 (en) * | 2010-01-21 | 2012-11-28 | Olympus Corporation | Image processing device, imaging device, program, and image processing method |
US20130057743A1 (en) * | 2011-09-02 | 2013-03-07 | Sony Corporation | Solid-state imaging device and camera system |
WO2014018949A2 (en) * | 2012-07-26 | 2014-01-30 | Olive Medical Corporation | Wide dynamic range using monochromatic sensor |
-
2014
- 2014-04-30 WO PCT/EP2014/058849 patent/WO2015165528A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090268055A1 (en) * | 2008-04-29 | 2009-10-29 | Hamilton Jr John F | Concentric exposure sequence for image sensor |
EP2528320A1 (en) * | 2010-01-21 | 2012-11-28 | Olympus Corporation | Image processing device, imaging device, program, and image processing method |
US20130057743A1 (en) * | 2011-09-02 | 2013-03-07 | Sony Corporation | Solid-state imaging device and camera system |
WO2014018949A2 (en) * | 2012-07-26 | 2014-01-30 | Olive Medical Corporation | Wide dynamic range using monochromatic sensor |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230088836A1 (en) * | 2020-03-31 | 2023-03-23 | Sony Group Corporation | Image processing device and method, and program |
US11770614B2 (en) * | 2020-03-31 | 2023-09-26 | Sony Group Corporation | Image processing device and method, and program |
WO2024018238A1 (en) * | 2022-07-22 | 2024-01-25 | Touch Biometrix Limited | Sensor and method of operating a sensor |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11830161B2 (en) | Dynamically cropping digital content for display in any aspect ratio | |
US10902557B2 (en) | Method and apparatus for restoring image | |
US20160119606A1 (en) | Image processing apparatus and image processing method | |
CN102985942A (en) | Zooming-in a displayed image | |
KR20140004592A (en) | Image blur based on 3d depth information | |
US20200257361A1 (en) | Three-dimensional (3d) rendering method and apparatus for user' eyes | |
US20120068996A1 (en) | Safe mode transition in 3d content rendering | |
US9955111B2 (en) | Electronic apparatus and display control method | |
CN106289158A (en) | Distance detection device and include the camera model of this distance detection device | |
CN108348207A (en) | Radiation imaging system | |
JP2013258586A5 (en) | ||
EP3633969A1 (en) | Image sensor and image sensing method | |
EP2702450A1 (en) | System and method for correction of vignetting effect in multi-camera flat panel x-ray detectors | |
US20210266490A1 (en) | Lens array camera and method of driving lens array camera | |
WO2015165528A1 (en) | Method and a device for generating a high quality image using pixel shifted low resolution images | |
US20200177794A1 (en) | Calibration method and apparatus | |
US20170161941A1 (en) | Method and apparatus for processing three-dimensional (3d) image, and graphics processing unit (gpu) | |
US20140152804A1 (en) | Sub-pixel imaging for enhanced pixel resolution | |
CN104756478B (en) | Resolution ratio and focus enhancing | |
CN102342842A (en) | Radiological image radiographing and displaying method and apparatus | |
EP3013034A1 (en) | Apparatus and method of detecting x-ray, and x-ray imaging system | |
JP2018066931A5 (en) | ||
JP2013255040A (en) | Image pick-up device, image pick-up system, control method and program of image pick-up device | |
EP4366323A1 (en) | Method and apparatus with hyperlapse video generation | |
CN104604216A (en) | Circuitry and method for collecting image array data with separate addressing and dynamic clamping of pixels to allow for faster pixel data readout and full removal of pixel charges |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14722629 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14722629 Country of ref document: EP Kind code of ref document: A1 |