CN115460362A - Visual sensor with reconfigurable resolution - Google Patents
Visual sensor with reconfigurable resolution Download PDFInfo
- Publication number
- CN115460362A CN115460362A CN202211390953.4A CN202211390953A CN115460362A CN 115460362 A CN115460362 A CN 115460362A CN 202211390953 A CN202211390953 A CN 202211390953A CN 115460362 A CN115460362 A CN 115460362A
- Authority
- CN
- China
- Prior art keywords
- sensor
- resolution
- signal
- pixel
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title abstract description 27
- 238000003384 imaging method Methods 0.000 claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 26
- 230000003287 optical effect Effects 0.000 claims abstract description 21
- 239000011664 nicotinic acid Substances 0.000 claims abstract description 16
- 230000008054 signal transmission Effects 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 238000004458 analytical method Methods 0.000 claims description 19
- 239000003990 capacitor Substances 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 230000005669 field effect Effects 0.000 claims description 6
- 230000003592 biomimetic effect Effects 0.000 claims description 2
- 230000007246 mechanism Effects 0.000 abstract description 8
- 210000001525 retina Anatomy 0.000 abstract description 8
- 238000010223 real-time analysis Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 19
- 238000000034 method Methods 0.000 description 15
- 210000004027 cell Anatomy 0.000 description 14
- 238000004364 calculation method Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 4
- 238000013144 data compression Methods 0.000 description 4
- 238000005265 energy consumption Methods 0.000 description 4
- 210000001508 eye Anatomy 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 210000001519 tissue Anatomy 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 210000005252 bulbus oculi Anatomy 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002207 retinal effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- BQCADISMDOOEFD-UHFFFAOYSA-N Silver Chemical compound [Ag] BQCADISMDOOEFD-UHFFFAOYSA-N 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 210000003855 cell nucleus Anatomy 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 210000000695 crystalline len Anatomy 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005684 electric field Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Images
Landscapes
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The invention discloses a visual sensor with reconfigurable resolution, which comprises: the pixel structure comprises a plurality of pixels, wherein each pixel comprises a sensor sensing module, the sensor sensing module is used for converting an optical signal into an electric signal, and an analog signal is converted into a digital signal to be output. The sensor signal transmission module and the sensor signal processing module are used for processing the data transmitted by the signal transmission module. The vision sensor in the invention is based on the distribution of cells in human retina and vision attention mechanism, and can control the output mode of pixels through the pixel control module on the basis of image acquisition, thereby realizing a low-resolution real-time monitoring mode and a bionic local high-resolution real-time analysis mode. The bionic local high-resolution imaging of a plurality of targets can be realized under the condition of not reducing the detailed information of the targets, so that the information redundancy is reduced, and the imaging speed is increased.
Description
Technical Field
The invention relates to the field of visual sensing, in particular to a visual sensor with reconfigurable resolution.
Background
The future of the manufacturing industry is intelligentized, and the basis of the intelligentization is a sensor; the internet is oriented to the internet of things, and the foundation of the internet of things is also a sensor. Taking a person as an example, the person obtains visual information of the outside world through eyes, and the visual information accounts for more than 80% of the total amount of information obtained by the person from the outside world. The vision sensor is a source of visual information of the machine system, and the main function is to acquire the most original image information enough for the machine system to use.
PMT (Photo Multiplier Tube) appeared in the 50 s of the 20 th century was the earliest visual sensor, but its cost was quite high; CCD (Charge Coupled Device) was developed in Bell laboratory in 1969, and has been rapidly popularized due to its small volume and low cost; the CMOS (Complementary Metal Oxide Semiconductor) age is entered at the end of the 20 th century and is not used for manufacturing a vision sensor until 1998, and the CMOS has the advantages of simpler structure than a CCD, about 1/3 of the power consumption of a common CCD and lower manufacturing cost than the CCD. With the increase of the level of semiconductor manufacturing process, nowadays, advanced CMOS vision sensors are the mainstream sensors for digital image information acquisition.
The CMOS vision sensor collects the photoelectric information in each pixel and converts the photoelectric information into digital information to be output. The method and the device collect images of each frame at a fixed frequency, the data volume is huge, a large amount of redundant information exists, the difficulty of calculation is improved, the processing efficiency is reduced, and the energy consumption is increased. The current vision sensor develops towards the direction of miniaturization, large visual field and high resolution, tens of millions of pixels or even hundreds of millions of pixels are often generated, particularly the high-precision application field at the front edge, such as the field of clinical medicine and military defense, focuses on real-time tracking of dynamic targets, and requires high-resolution imaging of the targets, so that the computing demand is huge.
In view of the above problems, the conventional solution is to increase the computational power of a computer and optimize an algorithm, thereby increasing the processing efficiency, but this method has limitations. Many applications benefit from a biomimetic approach, and the eye is an important organ for human to obtain visual information, so that research on the human visual system provides a new idea for solving problems.
The cone cells of the human retina are normally distributed, densely distributed in the fovea, and rapidly attenuated outward along the fovea. The central fovea is the optical focus of the eyeball, so that high-resolution imaging is performed in the central fovea, low-resolution imaging is performed on the periphery of the central fovea, and the function of dynamic early warning is kept. The high-resolution imaging is performed right in front of the sight line in the visual field, the low-resolution imaging is performed on the periphery of the visual field, and the dynamic early warning function is achieved. Visual attention mechanism means that a person will selectively focus on an area of interest while ignoring information from other areas. When a certain target object in a visual field is to be observed in detail, the eyeball position is adjusted through eye muscles, that is, the approximate characteristics of surrounding targets are not lost, and high-resolution imaging of the target object can be realized. The retina imaging method can be applied to a visual sensor to greatly compress redundant information, improve the effectiveness of the information, reduce the calculation power, accelerate the processing efficiency and reduce the energy consumption.
In 2022, a Liquid Crystal Optical Phased Array (LCOPA) -based retina-like scanning method was proposed in the literature (Tang M, cao J, hao Q, et al. Wide range scanning based on liquid crystal optical phase 194, optics and Lasers in Engineering, 2022, 151: 106885), which compresses the peripheral redundant data and maintains the high imaging quality of the fovea. And the fixation control function of the retina, the crystalline lens and the fovea of the human eye is realized by the optimally designed scanning strategy, namely the number of scanning rings and the number of sectors of each ring. The beam splitting method and the angle amplification system are adopted to meet the requirements of high-efficiency scanning and large field of view. Experiments prove that the flexible retina-like strategy has good scanning performance, particularly the scanning of the eccentric fovea. The scanning visual field is enlarged by 2.2 times, and the average scanning error is less than 5 percent. A time-varying class retinal computed ghosting Imaging strategy for Axially Moving objects to suppress image degradation caused by object motion in single pixel Imaging applications is proposed and demonstrated in the literature (Zhang Y, cao J, cui H, et al. The method can maintain the inherent advantages of computed ghost imaging and reconstruct objects in a single-arm system without the need for redundant hardware. By using the time-varying retinal-like pattern and the compressive sensing algorithm, high quality imaging results are obtained. The illumination pattern projected by the DMD is designed with a structure resembling the retina and the radius of the foveal area can be modified according to the course of the movement of the target. The experimental result shows that the proposed method has better performance in reconstructing the axial moving object than the traditional method. It does not enable retina-like imaging based on visual attention mechanism for multiple objects.
As described above, although the current retinal vision-like sensor can expand the visual field and achieve data compression to a certain extent, the data received at the rear end is still information of all pixels, and the data amount is not actually compressed at the sensor hardware layer. And the current retina-like vision sensor can not perform retina-like high-resolution imaging on a plurality of target objects in the visual field under the condition of data compression.
Disclosure of Invention
The invention aims to provide a visual sensor with reconfigurable resolution.
The vision sensor in the invention is based on the visual attention mechanism of the retina, realizes the compression of information while ensuring the large visual field and high-resolution imaging, and can selectively perform high-resolution imaging aiming at a plurality of targets. Therefore, redundant information is reduced, imaging efficiency is improved, computing power requirements are reduced, and energy consumption is reduced.
The invention adopts the following specific technical scheme:
a resolution reconfigurable vision sensor, comprising:
each pixel comprises a sensor sensing module, the sensor sensing module is used for converting optical signals at the corresponding pixel into electric signals, and converting analog signals into digital signals to be output;
a sensor pixel control module for controlling the output of the sensor sensing module at each pixel; realizing selective resolution of output image information;
the sensor signal transmission module is used for signal transmission among different modules;
the sensor signal processing module is used for receiving the output of the sensor sensing module according to the sensor pixel control module and processing the received signal;
the sensor sensing module comprises a photosensitive element, a selection control element, an amplifier element and an analog-to-digital conversion element; the optical sensing element is used for converting an optical signal into an electric signal, the amplifier element is used for amplifying the electric signal, and the analog-to-digital conversion element is used for converting an analog signal of the amplified electric signal into a digital signal and outputting the digital signal; the selective control element is electrically connected with the sensor pixel control module and used for controlling the on-off of the sensor sensing module circuit and the resetting of the photosensitive element according to the sensor pixel control module.
The sensor sensing module may be a CCD, CMOS or other sensing module. May be integrated or may be independent.
The photosensitive element may be a photoresistor, a photodiode, or the like, which can perform photoelectric conversion.
The selection control element comprises a column selection control element, a row selection control element, a reset switch and a source follower; the circuit structure of the sensor sensing module further comprises a power supply and a capacitor, and the circuit structure specifically comprises:
the positive electrode of the photosensitive element is grounded, and the negative electrode of the photosensitive element is connected with the source electrode of the row selection control element; the drain electrode of the row selection control element is connected with the source electrode of the reset switch and the grid electrode of the source follower; the drain electrode of the reset switch is connected with the power supply; the source electrode of the column selection control element is connected with one end of the capacitor, and the other end of the capacitor is grounded; the drain electrode of the column selection control element is connected with the source electrode of the source follower; the drain electrode of the source follower is connected with a power supply; the grid electrodes of the row selection control element, the reset switch and the column selection control element are used as control signal input and connected with the sensor pixel control module, and the amplifier element is connected between the capacitor and the source electrode of the column selection control element and used as signal output.
The column selection control element, the row selection control element and the reset switch can be field effect transistors, and can also be other elements or circuits which can be switched on and off through an electric signal control circuit.
The amplifier element of the sensor can be a field effect transistor, a triode, an operational amplifier, or other elements or structures capable of realizing electric signal amplification.
The analog-to-digital conversion element of the sensor is an element that can convert an analog signal of a continuous variable into a discrete digital signal.
And the pixel control module is used for controlling the output of the pixels in the sensing module to realize the selective high-resolution output of image information.
The pixel control module of the sensor may be an FPGA (Field Programmable Gate Array) or other real-time control circuit.
The imaging mode controlled by the pixel control module of the sensor can be a high-resolution analysis mode, a low-resolution real-time monitoring mode, a bionic local high-resolution analysis mode and the like. In the high-resolution analysis mode, the sensor pixel control module controls the sensor sensing module at each pixel to output all the signals according to time sequence;
in the low-resolution real-time monitoring mode, the sensor pixel control module controls the sensor sensing modules at a plurality of pixels in different areas to be merged and output according to time sequence;
and in the bionic local high-resolution analysis mode, the sensor pixel control module controls the sensor sensing modules at a plurality of pixels of the region without the target object to be output in a combined mode according to time sequence, and the sensor sensing module at each pixel of the region containing the target object outputs independently.
The target object of the bionic local high-resolution analysis mode can be one, two, three, etc. in multiple numbers.
The combined output of the sensor sensing modules at the multiple pixels can be 2 pixels combined into one pixel, 3 pixels combined into one pixel, 4 pixels combined into one pixel, and the like.
The signal transmission module is used for transmitting the information of the sensor sensing module to the pixel control module; transmitting a control signal of the pixel control module to a sensor sensing module; and transmitting the information of the sensor sensing module to the signal processing module.
The transmission line of the signal transmission module can be a connecting line such as a copper wire, a silver wire and the like, and can also be a connecting line on a printed circuit board.
And the signal processing module is used for processing the data transmitted by the signal transmission module.
Preferably, a CMOS is used as the sensor sensing module, and an FPGA is used as the pixel control module. The FPGA can realize time sequence control and process signals. The method has the advantages that the human vision attention mechanism and the distribution of photoreceptors on the retina are combined, the human vision imaging mechanism is simulated, the pixels of the CMOS vision sensor are controlled in real time by using the FPGA, the single-pixel high-resolution imaging of a plurality of concerned target areas is realized, the multi-pixel low-resolution imaging of other areas is realized, the effect of the vision sensor with reconfigurable resolution is achieved, data is really compressed on a hardware layer, the data volume is compressed while the resolution of a target object is not lost, the data redundancy is reduced, the data processing speed is increased, the calculation power requirement is reduced, and the energy consumption is reduced.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention provides a new data compression method for high-resolution and large-field visual imaging through a visual imaging simulating mechanism, and the data compression of a real sensor hardware layer is realized.
2. The invention realizes the large-scale compression of data, reduces redundant information and accelerates the processing speed without reducing the resolution ratio of the target object and losing the detailed characteristics of the target object.
3. Compared with the existing high-resolution imaging of a single target object, the method can realize the high-resolution imaging of a plurality of target objects by real-time reconstruction.
Drawings
FIG. 1 shows a block diagram of a sensor module of the present invention.
Fig. 2 illustrates a circuit portion structure of an exemplary sensor sensing module of the present invention.
Fig. 3 shows a schematic diagram of an imaging circuit of the vision sensor of the present invention.
Fig. 4 shows a timing diagram of one imaging circuit output of the vision sensor of the present invention.
Fig. 5 shows another imaging circuit schematic diagram of the vision sensor of the present invention.
Fig. 6 shows another imaging circuit output timing diagram of the vision sensor of the present invention.
Fig. 7 shows another imaging circuit pattern diagram of the vision sensor of the present invention.
Fig. 8 shows another imaging circuit output timing diagram of the vision sensor of the present invention.
Fig. 9 shows a block diagram illustrating the operation of the sensor according to the present invention.
FIG. 10 is a diagram showing a model of the sensor resolution reconstruction operation principle of the present invention.
Fig. 11 is a graph showing the results of a pre-and post-comparative test using a picture of cells from the sensor.
In the figure:
1-sensor sensing module, 2-sensor signal transmission module, 3-sensor signal processing module, 4-sensor pixel control module, 5-power supply Vdd, 6-source follower, 7-column selection transistor, 8-capacitance, 9-signal amplifier, 10-photodiode PD, 11-row selection transistor, 12-reset switch transistor, 14-ADC circuit, 54-timing signal of FPGA, 56-signal output of new round, 57-output signal of other pixels, 32-FPGA pixel control module, 33-multi-pixel combination as one pixel, 58-pixel output signal of high resolution analysis mode number 6, 59-output of low resolution real time monitoring mode single pixel, 60-signal output of no target area random representative pixel, 61-target area single pixel output, 18-no target area, 19-random representative output pixel of no target area, 20-small range area of no target area, 21-target, 22-target area, 29-high resolution cell tissue map, 30-low resolution cell tissue map, 31-local bionic map.
Detailed Description
The following provides a detailed description of the preferred embodiments of the present invention with reference to the accompanying drawings.
As shown in fig. 1, the present invention discloses a block diagram of a resolution reconfigurable vision sensor. The vision sensor with the reconfigurable resolution mainly comprises a plurality of pixels, wherein each pixel comprises a sensor sensing module 1, a sensor signal transmission module 2, a sensor signal processing module 3 and a sensor pixel control module 4. The method comprises the steps that firstly, a sensor sensing module 1 collects signals of object images corresponding to pixels, then the signals are transmitted to a sensor signal processing module 3 through a sensor signal transmission module 2, the sensor signal processing module 3 processes the signals, the processed signals are transmitted to a sensor pixel control module 4, and the sensor pixel control module 4 can control the output of the sensor sensing module at each pixel.
The sensor sensing module comprises a photosensitive element, a selection control element, an amplifier element and an analog-to-digital conversion element; the optical sensing element is used for converting an optical signal into an electric signal, the amplifier element is used for amplifying the electric signal, and the analog-to-digital conversion element is used for converting an analog signal of the amplified electric signal into a digital signal and outputting the digital signal; the selective control element is electrically connected with the sensor pixel control module and used for controlling the on-off of the sensor sensing module circuit and the resetting of the photosensitive element according to the sensor pixel control module. Preferably, the selection control element comprises a column selection control element, a row selection control element, a reset switch and a source follower; the circuit structure of the sensor sensing module further comprises a power supply and a capacitor, and the circuit structure specifically comprises:
the positive electrode of the photosensitive element is grounded, and the negative electrode of the photosensitive element is connected with the source electrode of the row selection control element; the drain electrode of the row selection control element is connected with the source electrode of the reset switch and the grid electrode of the source follower; the drain electrode of the reset switch is connected with the power supply; the source electrode of the column selection control element is connected with one end of the capacitor, and the other end of the capacitor is grounded; the drain electrode of the column selection control element is connected with the source electrode of the source follower; the drain electrode of the source follower is connected with a power supply. The grid electrodes of the row selection control element, the reset switch and the column selection control element are used as control signal input and connected with the sensor pixel control module, and the amplifier element is connected between the capacitor and the source electrode of the column selection control element and used as signal output.
The photosensitive element may be a photoresistor, a photodiode, or the like, which can perform photoelectric conversion.
The column selection control element, the row selection control element and the reset switch can be field effect transistors, and can also be other elements or circuits which can be switched on and off through an electric signal control circuit.
The amplifier element can be a field effect transistor, a triode, an operational amplifier, or other elements or structures capable of realizing electric signal amplification.
An analog-to-digital conversion element is an element that can convert a continuously variable analog signal into a discrete digital signal.
Fig. 2 shows a circuit portion structure of an exemplary sensor sensing module according to the present invention. A simple 3T pixel structure with two pixel circuits connected in parallel is shown, and similarly, the circuits of the respective pixels, i.e., the sensor sensing modules, are connected in parallel, and the circuit of each sensor sensing module is divided into the following elements: a photodiode PD10 for performing photoelectric conversion; a reset switching transistor 12 for resetting a voltage signal in the circuit; a source follower 6 for controlling the transmission of photo-generated electrons in the photodiode PD10 to the switch of the signal line; a row selection transistor 11 and a column selection transistor 7, which are used for controlling the switch of signal output, and which pixel signal is output first and which pixel signal is output later can be controlled by the switch. As shown in fig. 2, the circuit of the sensor sensing module is that a photodiode PD10 is connected in series with a row selection transistor 11, a reset switch transistor 12, and a power supply Vdd5, and a capacitor 8 is connected in series with a column selection transistor 7, a source follower 6, and the power supply Vdd 5. The anode of the photodiode PD10 is grounded, and the cathode is connected to the source of the row selection transistor 11; the drain electrode of the row selection transistor 11 is connected with the source electrode of the reset switch transistor 12 and the grid electrode of the source follower 6; the drain electrode of the reset switch transistor 12 is connected with a power supply Vdd5; one end of the capacitor 8 is grounded, and the other end is connected with the source electrode of the column selection transistor 7; the drain electrode of the column selection transistor 7 is connected with the source electrode of the source follower 6; the drain of the source follower 6 is connected to a power supply Vdd 5. The gates of the row selection transistor 11, the reset switch transistor 12, and the column selection transistor 7 are input as control signals, and the signal amplifier 9 connects between the capacitor 8 and the source of the column selection transistor 7 as a signal output.
The working principle of the sensor sensing module, namely the pixel circuit is as follows:
for example, in a single pixel circuit, the photodiode PD10 is first reset by applying a reverse voltage to the photodiode PD10, or activating the reset switch transistor 12 and the row select transistor 11 to reset the photodiode PD 10. After the reset is completed, the reset switching transistor 12 is no longer turned on.
And then exposed to light. In the process, all transistors are turned off, photons strike the PN junction and the silicon substrate of the photodiode PD10, and electron-hole pairs are generated after the photons are absorbed. These electron-hole pairs, after moving through the built-in electric field, are collected at both ends of the PN junction of the photodiode PD10, forming a potential difference.
And then read out. After the exposure is completed, the row selection transistor 11 and the column selection transistor 7 of the selected pixel are turned on, and the electrical signal in the photodiode PD10 passes through the source follower 6, is amplified by the amplifier 9, and then continues to be transmitted.
As shown in fig. 3, the present invention discloses a schematic diagram of an imaging circuit of the vision sensor of the present invention. This mode is a high resolution analysis mode, and all pixels output signals with high resolution. Taking a 4 × 4 array as a representative example, the photosensitive elements detect object image information, convert optical signals into electrical signals, and a sensor pixel control module (shown as an FPGA module 32 in the figure) provides logic for controlling row selection control elements and logic for controlling column selection control elements in a time sequence, so that output signals are controlled to be read out sequentially according to the numbered sequence. The output analog signal is converted into a digital signal through an ADC circuit, and the digital signal is transmitted to the FPGA module 32 for primary processing of the signal, so that partial calculation force is reduced or some functions without complex calculation are directly realized.
As shown in fig. 4, the present invention discloses a timing diagram of an imaging circuit of the vision sensor of the present invention. The time sequence of the invention can be controlled by FPGA, and the FPGA can be programmed with different circuits and can change the time sequence output. The timing chart shows the output timing of each pixel after reset and exposure. This mode is a high-resolution analysis mode, and all pixels output signals with high resolution. The pixel units numbered 6, 7, 10, 11 adjacent to the center are selected as representative as the sensor array in fig. 3, illustrating one imaging circuit timing principle of the vision sensor of the present invention. On the basis of the timing signal 54 of the FPGA, when a timing signal pulse comes, first controlling the pixel unit with the number 6 to output the acquired pixel output signal 58 of the high resolution analysis mode number 6; when the next time sequence signal pulse comes, controlling the pixel unit with the number of 7 to output the acquired signal; similarly, the pixel units numbered 10 and 11 are controlled to output the acquired signals. The readout of the acquired signals from the pixel numbered 1, and the output signals 57 from the other pixels numbered 2, 3, etc. in this order, is seen from the timing chart for the entire array. When the signals of all pixels are output in sequence, a new round of reset and exposure operations is started, followed by a new round of signal output 56, i.e. a new frame of image.
As shown in fig. 5, the present invention discloses another imaging circuit pattern diagram of the vision sensor of the present invention. The requirements for image resolution are different because of the difference in functionality. When the image quality is too poor, no problem can be explained; when the image quality is poor, whether a target of interest exists in the image can be barely distinguished; when the image quality is general, the details provided by the image can only help to distinguish the morphological characteristics, but the full details cannot be judged; when the image quality is good, the detail provided by the image is enough to judge whether the target in the image and the real target are the same target; when the image quality is excellent, sufficient detail information is provided in the image, the target can be completely determined, and other possibilities are excluded.
According to the function requirement, several pixels are determined to be combined into one pixel, which can be two pixels, three pixels and the like, and the data is greatly compressed while the function is realized. The mode is a low-resolution real-time monitoring mode, multiple pixels are combined into one pixel 33, four pixels are combined into one pixel as an example, the photosensitive element detects object image information, an optical signal is converted into an electric signal, the logic of a row selection control element and the logic of a column selection control element are provided by the FPGA module 32 through time sequence control, the pixels numbered 1, 2, 5 and 6 are simultaneously output, and the output end of the circuit is connected with a capacitor in parallel, so that the output is an average voltage value. The output analog signal is converted into a digital signal through an ADC circuit, and the digital signal is transmitted to the FPGA module 32 for primary processing of the signal, so that partial calculation force is reduced or some functions without complex calculation are directly realized. This one average voltage value represents the four voltage values of the pixels numbered 1, 2, 5, 6, implementing 4:1, thereby reducing the output at the hardware level, reducing the data volume and realizing the large-scale compression of the data.
As shown in fig. 6, the present invention discloses another imaging circuit timing diagram of the vision sensor of the present invention. Like fig. 4, the circuit timing can be controlled by FPGA which has been programmed with different circuits to change the timing output. Such as the sensor array in fig. 5, illustrates one imaging circuit timing principle of the vision sensor of the present invention. The timing chart shows the output timing of each pixel after reset and exposure. Because the mode is a low-resolution real-time monitoring mode, four pixels are combined into one pixel, when the output is started, firstly, the FPGA control signal simultaneously gates the row selection transistor 11 and the column selection transistor 7 with the numbers of 1, 2, 5 and 6 pixels, and closes the reset transistor 12, namely, a high-level row selection signal and a column selection signal are given, and a low-level reset signal is given. The signals of these four pixels are output simultaneously and averaged to form the overall output, i.e., the low resolution real-time monitor mode single pixel output 59. Followed in turn by the total output of 3, 4, 7, 8 pixels and the total output of 9, 10, 13, 14 pixels, etc. When the signals of all the pixels are sequentially output, a new round of reset and exposure operation is started, and then a new round of signal output, namely a new frame of image, is started. The resolution of the mode is reduced to one fourth of a high-resolution analysis mode, the requirements of partial functions can be met, the data size is reduced, and the data is greatly compressed.
As shown in fig. 7, the present invention discloses another imaging circuit pattern diagram of the vision sensor of the present invention. The mode is a bionic local high-resolution analysis mode, and the pixel output of the next frame is controlled after the simple processing by the FPGA according to the image of the previous frame. For example, it is determined that there is a sharp change in the luminance value of the target object region and the gradation values of the non-target object regions are almost the same. The no-target object area may be a plurality of pixels combined into one pixel, but the number is not recommended to be too large. When the target object moves to the previous frame and the output value of the area A is greatly different from the output values of other non-target object areas in the current frame, the next frame area A is determined as the target object area, and all pixels in the area output signals independently. The position of the object may be determined based on the boundary, shape, etc. of the object.
Take four pixels as an example. In the figure, the regions where the pixels numbered 3, 4, 7 and 8 are located are target regions, and the regions where the other pixels are located are non-target regions. The photosensitive element detects the object image information, converts the optical signal into an electrical signal, and the FPGA module 32 provides the logic of the time sequence control row selection control element and the logic of the column selection control element to control the output of a certain pixel. The output analog signals are converted into digital signals through an ADC circuit, the digital signals are transmitted to an FPGA module 32 for primary processing of the signals, and then pixels of target area numbers 3, 4, 7 and 8 are controlled to be output in a single pixel, and four pixels of other areas are combined into one pixel for output. The distribution of human retina cells and a visual attention mechanism are similar, the function of bionic local high-resolution analysis is achieved, detailed information of a target area is not reduced, redundant information is reduced, and data are compressed greatly.
As shown in fig. 8, the present invention discloses another imaging circuit timing diagram of the vision sensor of the present invention. As in fig. 4 and 6, the circuit timing can be controlled by the FPGA, which has been programmed with different circuits to vary the timing output. Such as the sensor array in fig. 7, illustrates one imaging circuit timing principle of the vision sensor of the present invention. The timing chart shows the output timing of each pixel after reset and exposure. Because the mode is a bionic local high-resolution analysis mode, a target area and a non-target area need to be monitored in the last frame of picture, and the time sequence output is controlled by the FPGA. Taking fig. 7 as an example, the areas where the pixels with numbers 3, 4, 7, and 8 are located are target areas, the areas where other pixels are located are non-target areas, four pixels without target areas can be selected to be combined into one pixel output, and the FPGA controls to randomly select one of the pixels as the random representative pixel output of the four pixels to represent the output of the combined pixel. When the output is started, the row selection transistor 11 and the column selection transistor 7 of one of the pixels with the numbers of 1, 2, 5 and 6 are firstly randomly gated, the reset transistor 12 is turned off, namely, a high-level row selection signal and a high-level column selection signal are given, a low-level reset signal is given, and the transistors in other unselected pixel circuits are all turned off, namely, the signals are all in a low level. The signal output 60 of the pixel randomly represented by the non-target area is output at this time, then the row selection transistor 11 and the column selection transistor 7 of the 3, 4, 7, 8 pixels are individually gated in sequence through the FPGA control, and the reset transistor 12 is turned off, that is, the row selection signal and the column selection signal with high level and the reset signal with low level are given. The signals of these four pixels are individually output in sequence, i.e. the target area single-pixel output 61. And then sequentially outputting signals of the remaining pixels. When the signals of all the pixels are sequentially output, a new round of reset and exposure operation is started, and then a new round of signal output, namely a new frame image, is started. And adjusting the FPGA control time sequence of the next frame in real time according to the dynamic condition of the target object in the previous frame image, such as the boundary of the target object, the color range and the gray value range of the target object, and the like, so as to realize real-time high-resolution imaging of the target object. The redundant information is reduced and the data is greatly compressed while the target detailed information is not reduced.
As shown in FIG. 9, the present invention discloses a block diagram of the working contents of the sensor of the present invention. The block diagram contains the contents of three modes of the sensor of the present invention. The first mode is a low-resolution real-time monitoring mode, and some functions do not require high-resolution imaging, and can be realized by relatively fuzzy imaging, such as real-time monitoring and tracking of an object. An optical signal of the object image is received by a light receiving element (PD shown in the figure), and the optical signal is converted into an electric signal, which is amplified by a pixel circuit and the like, and is output under control. The pixel circuit is output by FPGA time sequence control and pixel control. The analog signal is converted into a digital signal after passing through an A/D conversion circuit, and is transmitted to the FPGA for processing through a Logic circuit. In the process of controlling the pixel circuit by the FPGA, a plurality of pixels are combined into one pixel to be output, and the data volume is greatly reduced. Suppose that now there are output signals of one hundred thousand, which correspond to signals of one hundred thousand pixels.
The second working mode is a bionic local high-resolution analysis mode, an optical signal of an object image is received by the photosensitive element, the optical signal is converted into an electric signal, and the signal is amplified and the like through the pixel circuit and is controlled to be output. The pixel circuit is output by FPGA time sequence control and pixel control. The analog signal is converted into a digital signal after passing through an A/D conversion circuit, and is transmitted to the FPGA for processing through a Logic circuit. And then the FPGA feeds back the signal to the pixel circuit to control the output of the pixel, the area with the target object controls the output of a single pixel in a specific area, the area without the target object controls the combination of a small range of the pixel into one pixel, one pixel is taken as a representative random pixel to output, and the output in the range is represented, so that local high-resolution imaging is realized. The mode is partial pixel output, and the data is greatly compressed under the condition of realizing high-resolution imaging of a target object on the assumption that one hundred million pixels of information are transmitted on the circuit at present. And the bionic local high-resolution imaging of a plurality of targets can be realized through the pixel control module FPGA 37.
The third mode of operation is a high resolution analysis mode. The optical signal of the object image is received by the photosensitive element, the optical signal is converted into an electrical signal, and the signal is amplified and the like by the pixel circuit and is controlled to be output. The pixel control circuit is output by FPGA time sequence control and pixel control. The analog signal is converted into a digital signal after passing through an A/D conversion circuit, and is transmitted to the FPGA for processing through a Logic circuit. This mode is a full-pixel output, assuming that one billion pixels of information are now being transferred on the circuit for use in a scenario requiring global high-resolution imaging.
As shown in FIG. 10, the invention discloses a model diagram of the sensor resolution reconstruction working principle of the invention. In the imaging field of view, the target object area 18 and the target object area 22 are separated, and the output of a single pixel is controlled in the target object area 22, thereby realizing high-resolution imaging of the target object 21. In the non-target area 18, a plurality of pixels are controlled to be combined into one pixel, and the output of one pixel is taken as a random representative output pixel 19 of the non-target area to represent the output of the range pixel. Therefore, the data is greatly compressed under the condition of not losing all information of the target object. And the bionic local high-resolution imaging of a plurality of targets can be realized through the pixel control module FPGA 37.
As shown in FIG. 11, the present invention discloses a front-back comparison test result chart of the cell picture of the sensor of the present invention. The pictorial object is a high-resolution cell tissue map 29, a low-resolution cell tissue map 30 in a low-resolution real-time monitoring mode, and a bionic local high-resolution map 31 of target cells in a bionic local high-resolution real-time analysis mode. In the low-resolution cell organization chart 30, because the cell nucleus of the abnormal cell is relatively large, the abnormal cell can be distinguished according to the size of the cell, the data volume is reduced, the imaging speed is increased, and the functions of counting, tracking and the like can be realized. In the target cell bionic local high-resolution map 31, the target object region of interest is output with high resolution to facilitate the analysis of the specific details of the target object, and the region not of interest is output with low resolution.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. It is not necessary or exhaustive to mention all embodiments. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.
Claims (10)
1. A resolution reconfigurable vision sensor, comprising:
each pixel comprises a sensor sensing module, the sensor sensing module is used for converting optical signals at the corresponding pixel into electric signals, and converting analog signals into digital signals to be output;
a sensor pixel control module for controlling the output of the sensor sensing module at each pixel; realizing selective resolution of output image information;
the sensor signal transmission module is used for signal transmission among different modules;
the sensor signal processing module is used for receiving the output of the sensor sensing module according to the sensor pixel control module and processing the received signal;
the sensor sensing module comprises a photosensitive element, a selection control element, an amplifier element and an analog-to-digital conversion element; the optical sensing element is used for converting an optical signal into an electric signal, the amplifier element is used for amplifying the electric signal, and the analog-to-digital conversion element is used for converting an analog signal of the amplified electric signal into a digital signal and outputting the digital signal; the selective control element is electrically connected with the sensor pixel control module and used for controlling the on-off of the sensor sensing module circuit and the resetting of the photosensitive element according to the sensor pixel control module.
2. The resolution reconfigurable vision sensor according to claim 1, wherein the light sensing element is a photo resistor or a photodiode.
3. The resolution reconfigurable vision sensor according to claim 1, wherein the amplifier element is a field effect transistor, a triode, or an operational amplifier.
4. The resolution reconfigurable vision sensor according to claim 1, wherein the selection control elements include a column selection control element, a row selection control element, a reset switch, and a source follower; the circuit structure of the sensor sensing module further comprises a power supply and a capacitor, and the circuit structure specifically comprises:
the positive electrode of the photosensitive element is grounded, and the negative electrode of the photosensitive element is connected with the source electrode of the row selection control element; the drain electrode of the row selection control element is connected with the source electrode of the reset switch and the grid electrode of the source follower; the drain electrode of the reset switch is connected with the power supply; the source electrode of the column selection control element is connected with one end of the capacitor, and the other end of the capacitor is grounded; the drain electrode of the column selection control element is connected with the source electrode of the source follower; the drain electrode of the source follower is connected with a power supply; the grid electrodes of the row selection control element, the reset switch and the column selection control element are used as control signal input and connected with the sensor pixel control module, and the amplifier element is connected between the capacitor and the source electrode of the column selection control element and used as signal output.
5. The resolution reconfigurable vision sensor according to claim 4, wherein the column selection control elements, the row selection control elements and the reset switches are field effect transistors.
6. The resolution reconfigurable vision sensor of claim 1, wherein the sensor pixel control module is an FPGA.
7. The resolution reconfigurable vision sensor according to claim 1, wherein the imaging modes controlled by the sensor pixel control module include:
in the high-resolution analysis mode, the sensor pixel control module controls the sensor sensing module at each pixel to output all the signals according to time sequence;
in the low-resolution real-time monitoring mode, the sensor pixel control module controls the sensor sensing modules at a plurality of pixels in different areas to be combined and output according to time sequence;
and in the bionic local high-resolution analysis mode, the sensor pixel control module controls the sensor sensing modules at a plurality of pixels of the region without the target object to be combined and output according to time sequence, and the sensor sensing module at each pixel of the region containing the target object outputs independently.
8. The resolution reconfigurable vision sensor according to claim 7, wherein the target of the biomimetic local high resolution analysis mode is one or more.
9. The resolution reconfigurable vision sensor of claim 7, wherein the number of sensor sensing block outputs combined at the plurality of pixels is 2, 3 or 4.
10. The vision sensor with reconfigurable resolution according to claim 1, wherein the sensor signal transmission module is configured to transmit information of the sensor sensing module to the sensor pixel control module; transmitting a control signal of the sensor pixel control module to the sensor sensing module; and transmitting the information of the sensing module to the signal processing module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211390953.4A CN115460362A (en) | 2022-11-08 | 2022-11-08 | Visual sensor with reconfigurable resolution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211390953.4A CN115460362A (en) | 2022-11-08 | 2022-11-08 | Visual sensor with reconfigurable resolution |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115460362A true CN115460362A (en) | 2022-12-09 |
Family
ID=84310716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211390953.4A Pending CN115460362A (en) | 2022-11-08 | 2022-11-08 | Visual sensor with reconfigurable resolution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115460362A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1518346A (en) * | 2003-01-24 | 2004-08-04 | ���µ�����ҵ��ʽ���� | MOS solid imaging element and imaging device with it |
| CN1533150A (en) * | 2003-03-26 | 2004-09-29 | �ֵܹ�ҵ��ʽ���� | Image sensor, image reader and method for setting resolution |
| CN101047798A (en) * | 2006-03-31 | 2007-10-03 | 佳能株式会社 | Image sensor |
| CN102663758A (en) * | 2012-04-20 | 2012-09-12 | 北京工业大学 | Image acquiring and processing method based on FPGA (field programmable gate array) serving as control core |
| CN103179332A (en) * | 2011-12-22 | 2013-06-26 | 洛阳理工学院 | Visual target self-adaption detection controller based on field programmable gate array (FPGA) technology |
| CN104125419A (en) * | 2014-06-24 | 2014-10-29 | 北京理工大学 | Self-adaptive resolution ratio implementing method based on CMOS image sensor |
| CN109314749A (en) * | 2016-06-28 | 2019-02-05 | 索尼公司 | Imaging device, imaging method and program |
| CN110073654A (en) * | 2017-03-23 | 2019-07-30 | 索尼半导体解决方案公司 | Control equipment, control method, program and electronic apparatus system |
| WO2019208412A1 (en) * | 2018-04-27 | 2019-10-31 | ソニーセミコンダクタソリューションズ株式会社 | Imaging device and driving method of imaging device |
| CN111164670A (en) * | 2017-08-10 | 2020-05-15 | 罗伯特·博世有限公司 | Low power and low data rate imager |
| CN111527741A (en) * | 2018-01-29 | 2020-08-11 | 索尼半导体解决方案公司 | Image capturing apparatus and method of controlling image capturing element |
-
2022
- 2022-11-08 CN CN202211390953.4A patent/CN115460362A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1518346A (en) * | 2003-01-24 | 2004-08-04 | ���µ�����ҵ��ʽ���� | MOS solid imaging element and imaging device with it |
| CN1533150A (en) * | 2003-03-26 | 2004-09-29 | �ֵܹ�ҵ��ʽ���� | Image sensor, image reader and method for setting resolution |
| CN101047798A (en) * | 2006-03-31 | 2007-10-03 | 佳能株式会社 | Image sensor |
| CN103179332A (en) * | 2011-12-22 | 2013-06-26 | 洛阳理工学院 | Visual target self-adaption detection controller based on field programmable gate array (FPGA) technology |
| CN102663758A (en) * | 2012-04-20 | 2012-09-12 | 北京工业大学 | Image acquiring and processing method based on FPGA (field programmable gate array) serving as control core |
| CN104125419A (en) * | 2014-06-24 | 2014-10-29 | 北京理工大学 | Self-adaptive resolution ratio implementing method based on CMOS image sensor |
| CN109314749A (en) * | 2016-06-28 | 2019-02-05 | 索尼公司 | Imaging device, imaging method and program |
| CN110073654A (en) * | 2017-03-23 | 2019-07-30 | 索尼半导体解决方案公司 | Control equipment, control method, program and electronic apparatus system |
| CN111164670A (en) * | 2017-08-10 | 2020-05-15 | 罗伯特·博世有限公司 | Low power and low data rate imager |
| CN111527741A (en) * | 2018-01-29 | 2020-08-11 | 索尼半导体解决方案公司 | Image capturing apparatus and method of controlling image capturing element |
| WO2019208412A1 (en) * | 2018-04-27 | 2019-10-31 | ソニーセミコンダクタソリューションズ株式会社 | Imaging device and driving method of imaging device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Culurciello et al. | A biomorphic digital image sensor | |
| Culurciello et al. | Arbitrated address event representation digital image sensor | |
| Chen et al. | Pulse-modulation imaging—Review and performance analysis | |
| CN206596105U (en) | Image element circuit | |
| US7843554B2 (en) | High dynamic range sensor system and method | |
| DK2098062T3 (en) | Method for generating an image in electronic form, image element for an image sensor for image generation and image sensor | |
| CN102685403B (en) | Method for expanding dynamic range of time-delay integration-complementary metal oxide semiconductor (TDI-CMOS) image sensor | |
| CN106454148B (en) | Piecemeal separate exposures CMOS image sensor pixel structure and its control method | |
| CN111510651B (en) | Image sensing circuit, image sensor and terminal equipment | |
| JP2008017019A (en) | Solid state imaging device | |
| US20200228743A1 (en) | Image sensor and image capturing apparatus | |
| CN113242344B (en) | Image sensor, camera assembly and mobile terminal | |
| WO2000051345A1 (en) | Method and apparatus for dual resolution foveating display output | |
| JP6918390B2 (en) | Image sensor, image sensor and image information processing method | |
| RU2723645C1 (en) | High-resolution panorama television surveillance computer system device | |
| JP7091052B2 (en) | Image sensor and image sensor | |
| CN115460362A (en) | Visual sensor with reconfigurable resolution | |
| CN107040733B (en) | CMOS image sensor | |
| US11343454B2 (en) | Imaging systems and methods for performing pixel binning and variable integration for analog domain regional feature extraction | |
| CN106982336B (en) | Implement the shared method of dynamic earth fault and imaging sensor in the imaging sensor with pipeline architecture | |
| Yin et al. | A 0.5 V 34.4 uW 14.28 kfps 105dB smart image sensor with array-level analog signal processing | |
| Kim et al. | A compact-pixel tri-mode vision sensor | |
| Singh et al. | 34.2 a 21pJ/frame/pixel imager and 34pJ/frame/pixel image processor for a low-vision augmented-reality smart contact lens | |
| JPH0921879A (en) | Radiation plane detection device and radiation image pickup device | |
| JP7164579B2 (en) | Image sensors and electronic devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20221209 |
|
| RJ01 | Rejection of invention patent application after publication |