CN104065937B - For the real time high-speed image pre-processing method of cmos image sensor - Google Patents
For the real time high-speed image pre-processing method of cmos image sensor Download PDFInfo
- Publication number
- CN104065937B CN104065937B CN201410276634.XA CN201410276634A CN104065937B CN 104065937 B CN104065937 B CN 104065937B CN 201410276634 A CN201410276634 A CN 201410276634A CN 104065937 B CN104065937 B CN 104065937B
- Authority
- CN
- China
- Prior art keywords
- data
- group
- static memory
- data line
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
A kind of real time high-speed image pre-processing method for cmos image sensor, comprise the Bayer data exported by the photographic department of cmos image sensor, the line number of the single frames array of described Bayer data is M, columns is N, its innovation is: adopt the piecemeal of pipeline system, the mode of operation of classifying type access data, by the mode of piecemeal, classifying type, data are broken the whole up into parts, adopt the relatively less register of quantity to process the data after point zero simultaneously.Advantageous Effects of the present invention is: by changing caching process mode, use and take buffer structure on the less sheet of hardware resource, achieve high speed processing more data cached, significantly improve integrated level and the chip processing speed of cmos image sensor.
Description
Technical field
The present invention relates to a kind of photo-sensor signal will treatment technology, particularly relate to a kind of real time high-speed image pre-processing method for cmos image sensor.
Background technology
Along with raising and the informationalized high speed development of technological level, cmos image sensor replaces ccd sensor gradually with the advantage of its high integration, high speed and low-power consumption, low cost, is widely used in digital camera, video camera and the first-class multiple field of security protection shooting.
In prior art, there is correlation in image quality and the process of the cmos image sensor produced based on existing technique, the less image quality of process is poorer, therefore, how making cmos image sensor under less process condition, obtain high-quality imaging is the hot issue studied at present.
The data mode that the photographic department of cmos image sensor exports is generally Bayer form (calling Bayer data in the following text), Bayer data generally do not do direct application, processed by successive image Processing Algorithm again after needing that preliminary treatment is carried out to it, when carrying out preliminary treatment to Bayer data, prior art generally adopts memory and register serial mode to process, its principle as shown in Figure 2, capable for a frame M() × N(arranges) the Bayer data of array, if color interpolation algorithm needs buffer memory M, capable data process, M × N number of register and M-1 RAM memory is just needed during process, when Bayer data input register, first clock cycle, data are read into A
mb
nin individual register, when next clock arrives, while reading in new data, by original A
mb
nin a upper beat data serial move into A
mb
n-1in, in subsequent process, data are transmitted successively in register and RAM memory, after first order register has all read in data, after following clock cycle arrives, the data of one-level register are delivered to successively second level register and carry out algorithm process (in whole process, RAM plays data cached effect), at least need (2M-1) × N+M clock cycle by M × N number of Bayer data all move into second level buffer.Aforementioned processing mode Problems existing is: if subsequent algorithm requires that the pel array of process is larger, when data are more, existing preprocess method will need a large amount of RSs, data serial transmission also can need the more time, area and the power consumption of chip will be increased like this, and the processing speed of algorithm can be reduced, simultaneously because the sequential serial line by line of data inputs, the order of type of pixel maintains the type of pixel order of original Bayer form, and this also causes the difficulty on codes implement to strengthen.
Summary of the invention
For the problem in background technology, the present invention proposes a kind of real time high-speed image pre-processing method for cmos image sensor, comprise the Bayer data exported by the photographic department of cmos image sensor, the line number of the single frames array of described Bayer data is M, columns is N, its innovation is:
Build cache module: adopt K static memory to build cache module, 10≤K < M, and K is even number, the storage depth of single static memory is J(this paper indication storage depth is exactly memory capacity, and its unit is byte); K static memory is divided into L process group, L=K/2, all corresponding two static memories of each process group;
Bayer data are classified: the single file data in described single frames array are designated as data line, multiple data during individual data is capable are divided into two groups by the odd, even relation of the columns sequence number corresponding to each data: in same data line, columns sequence number is that multiple data of odd number form data group one, and columns sequence number is that multiple data of even number form data group two; Then each data line is all to there being a data group one and data group two, and the data amount check in data group one and data group two is N/2, N/2=J; Classifying rules is designated as to the mode that Bayer data are classified;
During process, carry out as follows:
1) by classifying rules, the data in 1 to L in single frames array data line are classified, obtains 2L data group;
2) by the row order in single frames array, by in the static memory in the data group in each data line successively input buffer module, the corresponding data group of each static memory, two data groups in same data line input in two static memories of same process group; Multiple data in individual data group input in static memory in a serial fashion; If the corresponding 1st process group of the 1st data line, the corresponding 2nd process group of the 2nd data line ... L data line corresponding L process group;
3) after the data of L data line have inputted, the outside parallel output of multiple data that static memory has started buffer memory;
While the outside parallel output of static memory, after the multiple data in L+1 data line being pressed classifying rules classification, serial input the 1st processes in static memory corresponding in group, in order to replace the multiple data in the 1st data line; The processing mode that aforesaid new data replaces legacy data in process group is designated as update process, in the process of follow-up parallel output static memory data, continues to carry out update process with new data line successively to all the other process groups;
4) after L process group is completed update process, by mode in step 3), with new data line, again successively update process is carried out to L process group, until cache module is completed buffer memory, output processing to all data lines.
Principle of the present invention is: the solution of the present invention defines in fact a kind of piecemeal of pipeline system, the mode of operation of classifying type access data, pass through piecemeal, the mode of classifying type makes data break the whole up into parts, adopt the relatively less register of quantity to process the data after point zero simultaneously, not only greatly reduce processing delay, improve processing speed, and save chip area, improve chip integration, specifically, the present invention is achieved in that " piecemeal " namely carries out serial input buffer memory one by one in units of data line, data line is divided into two pieces to store, output processing is for the data in all piecemeals and by its parallel output.Store in the data completing first all memories, carry out more new data time, serial input and parallel output will synchronously carry out, " classifying type " is divided into two data groups by the data in each data line by columns is odd, even, certainly, why can do such classification, it is the inherent characteristic having relied on Bayer data, individual data in Bayer data is capable, that R and G is alternately arranged, that G and B is alternately arranged (RGB is red-green-blue), and the odd, even relation of R, G, B is constant in different pieces of information row, this characteristic of Bayer data is make use of fully in the present invention, classified by odd, even relation, data line is broken the whole up into parts, to coordinate the enforcement of the present invention program, after " streamline " has namely carried out buffer memory, output processing to the data in a certain data line, continue to process in new data line input buffer module, the most direct effect that the mode of operation that aforementioned three kinds of means are formed is brought, register number required in preprocessing process is made significantly to reduce exactly, from the data scale of process, in the single treatment of prior art, need to process the M in single frames array × N number of data simultaneously, therefore it needs M-1 RAM memory and M × N number of register, and in the present invention, owing to having carried out " piecemeal " to single frames array, " classification ", thus the degree of depth of the register number making single buffer memory need and single static memory is significantly reduced, be conducive to carrying out layout on limited chip area, simultaneously due to the parallel output of data, reduce design difficulty and processing delay, improve the efficiency of process.
In the present invention, the means of control data transfer are same as the prior art, its Control timing sequence adopts conventional clock CLK to control, in order to make sequencing control more simplify, the present invention has also done following improvement: two static memories in single process group and individual data capable in data group one and the corresponding relation of data group two keep constant.
Based on the reduction to static memory physical requirement that aforementioned schemes is brought, the invention allows for following preferred version to improve the dimensional parameters of device: described static memory adopts on-chip memory to realize.On-chip memory is due to size restrictions, its disposal ability is not as the buffer storage of peripheral hardware, but after employing the solution of the present invention, a small amount of on-chip memory just can bear the buffer memory work of larger data amount, this means a great for the integrated level improving cmos image sensor.
Preferably, when M get 512, N get 512 time, K gets 10, then the storage depth J of static memory is 256.
Advantageous Effects of the present invention is: by changing caching process mode, use and take buffer structure on the less sheet of hardware resource, achieve high speed processing more data cached, significantly improve integrated level and the chip processing speed of cmos image sensor.
Accompanying drawing explanation
Fig. 1, handling principle schematic diagram of the present invention (in figure, the quantity of static memory is 10);
The handling principle schematic diagram of Fig. 2, prior art;
Data array schematic diagram in Fig. 3, Bayer data;
Fig. 4, cmos sensor system on chip structure schematic diagram.
Embodiment
For a real time high-speed image pre-processing method for cmos image sensor, comprise the Bayer data exported by the photographic department of cmos image sensor, the line number of the single frames array of described Bayer data is M, columns is N, its innovation is:
Build cache module: adopt K static memory to build cache module, 10≤K < M, and K is even number, the storage depth of single static memory is J; K static memory is divided into L process group, L=K/2, all corresponding two static memories of each process group;
Bayer data are classified: the single file data in described single frames array are designated as data line, multiple data during individual data is capable are divided into two groups by the odd, even relation of the columns sequence number corresponding to each data: in same data line, columns sequence number is that multiple data of odd number form data group one, and columns sequence number is that multiple data of even number form data group two; Then each data line is all to having a data group one and a data group two, and the data amount check in data group one and data group two is N/2, N/2=J; Classifying rules is designated as to the mode that Bayer data are classified;
During process, carry out as follows:
1) by classifying rules, the data in 1 to L in single frames array data line are classified, obtains 2L data group;
2) by the row order in single frames array, by in the static memory in the data group in each data line successively input buffer module, the corresponding data group of each static memory, two data groups in same data line input in two static memories of same process group; Multiple data in individual data group input in static memory in a serial fashion; If the corresponding 1st process group of the 1st data line, the corresponding 2nd process group of the 2nd data line ... L data line corresponding L process group;
3) after the data of L data line have inputted, the outside parallel output of multiple data that static memory has started buffer memory;
While the outside parallel output of static memory, after the multiple data in L+1 data line being pressed classifying rules classification, serial input the 1st processes in static memory corresponding in group, in order to replace the multiple data in the 1st data line; The processing mode that aforesaid new data replaces legacy data in process group is designated as update process, in the process of follow-up parallel output static memory data, continues to carry out update process with new data line successively to all the other process groups;
4) after L process group is completed update process, by mode in step 3), with new data line, again successively update process is carried out to L process group, until cache module is completed buffer memory, output processing to all data lines.
Further, two static memories in single process group and individual data capable in data group one and the corresponding relation of data group two keep constant.
Further, described static memory adopts on-chip memory to realize.
Further, when M get 512, N get 512 time, K gets 10, then the storage depth J of static memory is 256.
Capable with a frame 512(below) × 512(arranges) the Bayer data instance of array, the solution of the present invention and prior art are carried out quantification and compare:
Based on the present invention program, when adopting 10 static memories to process 512 × 512 array datas, when buffer memory 5 row data process, need 10 degree of depth to be the buffer memory computing that the static memory of 256 words and 10 registers carry out data, the whole serial of frame data is read in needs 262144 clock cycle with parallel read-out.
According to prior art, prior art is when processing 512 × 512 data, when buffer memory 5 row data process, need 5 degree of depth to be the buffer memory computing that the static memory of 512 words and 3072 registers carry out data, the clock cycle needed for the access of total data is 524288.
The present invention and prior art have greater advantage in chip area and layout, the thought of piecemeal is utilized to be divided into 10 degree of depth to be the static memory of 256 words the static memory that 5 degree of depth are 512 words, this integral layout being conducive to SOC (system on a chip) with and reduce design difficulty, and for same process conditions, the register number that the present invention uses is significantly less than the register number (capacity of the two static memory is identical) of prior art, and this can saving chip area and power consumption.
Claims (4)
1. for a real time high-speed image pre-processing method for cmos image sensor, comprise the Bayer data exported by the photographic department of cmos image sensor, the line number of the single frames array of described Bayer data be M, columns is N, it is characterized in that:
Build cache module: adopt K static memory to build cache module, 10≤K < M, and K is even number, the storage depth of single static memory is J, and described storage depth is exactly memory capacity, and its unit is byte; K static memory is divided into L process group, L=K/2, all corresponding two static memories of each process group;
Bayer data are classified: the single file data in described single frames array are designated as data line, multiple data during individual data is capable are divided into two groups by the odd, even relation of the columns sequence number corresponding to each data: in same data line, columns sequence number is that multiple data of odd number form data group one, and columns sequence number is that multiple data of even number form data group two; Then each data line is all to there being a data group one and data group two, and the data amount check in data group one and data group two is N/2, N/2=J; Classifying rules is designated as to the mode that Bayer data are classified;
During process, carry out as follows:
1) by classifying rules, the data in 1 to L in single frames array data line are classified, obtains 2L data group;
2) by the row order in single frames array, by in the static memory in the data group in each data line successively input buffer module, the corresponding data group of each static memory, two data groups in same data line input in two static memories of same process group; Multiple data in individual data group input in static memory in a serial fashion; If the corresponding 1st process group of the 1st data line, the corresponding 2nd process group of the 2nd data line ... L data line corresponding L process group;
3) after the data of L data line have inputted, the outside parallel output of multiple data that static memory has started buffer memory;
While the outside parallel output of static memory, after the multiple data in L+1 data line being pressed classifying rules classification, serial input the 1st processes in static memory corresponding in group, in order to replace the multiple data in the 1st data line; The processing mode that aforesaid new data replaces legacy data in process group is designated as update process, in the process of follow-up parallel output static memory data, continues to carry out update process with new data line successively to all the other process groups;
4) after L process group is completed update process, by mode in step 3), with new data line, again successively update process is carried out to L process group, until cache module is completed buffer memory, output processing to all data lines.
2. the real time high-speed image pre-processing method for cmos image sensor according to claim 1, is characterized in that: two static memories in single process group and individual data capable in data group one and the corresponding relation of data group two keep constant.
3. the real time high-speed image pre-processing method for cmos image sensor according to claim 1, is characterized in that: described static memory adopts on-chip memory to realize.
4. the real time high-speed image pre-processing method for cmos image sensor according to claim 1, is characterized in that: when M get 512, N get 512 time, K gets 10, then the storage depth J of static memory is 256.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410276634.XA CN104065937B (en) | 2014-06-20 | 2014-06-20 | For the real time high-speed image pre-processing method of cmos image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410276634.XA CN104065937B (en) | 2014-06-20 | 2014-06-20 | For the real time high-speed image pre-processing method of cmos image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104065937A CN104065937A (en) | 2014-09-24 |
| CN104065937B true CN104065937B (en) | 2016-01-13 |
Family
ID=51553442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410276634.XA Active CN104065937B (en) | 2014-06-20 | 2014-06-20 | For the real time high-speed image pre-processing method of cmos image sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104065937B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376534B (en) * | 2014-10-17 | 2017-05-24 | 中国电子科技集团公司第四十四研究所 | Low-power-consumption real-time noise-reduction and sharpening merged preprocessing method for CMOS image sensor |
| CN106060432B (en) * | 2016-06-02 | 2018-11-23 | 西北核技术研究所 | A kind of processing method of unstringing in real time of cmos image sensor data-signal |
| CN108829628B (en) * | 2018-05-30 | 2021-03-02 | 江南大学 | Real-time control method and device for double jacquard of warp knitting machine |
| CN109741237B (en) * | 2018-12-28 | 2020-10-23 | 中国科学院半导体研究所 | Large-scale image data processing system and method |
| CN111193869B (en) * | 2020-01-09 | 2021-08-06 | Oppo广东移动通信有限公司 | Image data processing method, image data processing device and mobile terminal |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101119490A (en) * | 2006-08-02 | 2008-02-06 | 索尼株式会社 | Image signal processing device and method |
| CN201937742U (en) * | 2010-12-14 | 2011-08-17 | 深圳市视鑫数码有限公司 | High-speed image acquisition system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101094246B1 (en) * | 2009-03-16 | 2011-12-19 | 이재웅 | Wide Dynamic Range CMOS Image Sensor |
| US8599292B2 (en) * | 2010-08-18 | 2013-12-03 | Taiwan Semiconductor Manufacturing Co., Ltd. | CMOS sensor with low partition noise and low disturbance between adjacent row control signals in a pixel array |
-
2014
- 2014-06-20 CN CN201410276634.XA patent/CN104065937B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101119490A (en) * | 2006-08-02 | 2008-02-06 | 索尼株式会社 | Image signal processing device and method |
| CN201937742U (en) * | 2010-12-14 | 2011-08-17 | 深圳市视鑫数码有限公司 | High-speed image acquisition system |
Non-Patent Citations (1)
| Title |
|---|
| 一种应用于CMOS图像传感器的电容阵列PGA研究;吴治军等;《半导体光电》;20111031;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104065937A (en) | 2014-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104065937B (en) | For the real time high-speed image pre-processing method of cmos image sensor | |
| CN108701236B (en) | Convolutional neural network | |
| US20210019594A1 (en) | Convolutional neural network accelerating device and method | |
| JP2020516981A (en) | Configurable and programmable image processor unit | |
| US7409528B2 (en) | Digital signal processing architecture with a wide memory bandwidth and a memory mapping method thereof | |
| CN102687162B (en) | The method and apparatus of image procossing is carried out with pixel rate | |
| US8989509B2 (en) | Streaming wavelet transform | |
| CN103098077A (en) | Real-time video frame pre-processing hardware | |
| US11709911B2 (en) | Energy-efficient memory systems and methods | |
| EP0306305B1 (en) | Image processor with free flow pipeline bus | |
| US20220084159A1 (en) | Image signal processor, method of operating the image signal processor, and application processor including the image signal processor | |
| CN102131053A (en) | Data acquisition, coding and storage method applied to high speed imaging system | |
| CN104717485A (en) | VGA interface naked-eye 3D display system based on FPGA | |
| CN111340835A (en) | FPGA-based video image edge detection system | |
| CN106067955A (en) | Reading circuit for imageing sensor | |
| US20120327260A1 (en) | Parallel operation histogramming device and microcomputer | |
| CN103400153A (en) | Serial filtering matching method and system for real-time image identification | |
| CN108024074A (en) | A kind of miniaturization infrared imaging method based on SOPC | |
| US11194490B1 (en) | Data formatter for convolution | |
| CN106708457A (en) | FPGA (Field Programmable Gate Array) processing module and method used for DMD (Digital Micromirror Device) dynamic area selection | |
| US11341603B2 (en) | Electronic image processing device | |
| Chang et al. | Integrated operation of image capturing and processing in FPGA | |
| CN102707919A (en) | Device and method for controlling FIFO (First In First Out) read-write by using finite state machine (FSM) | |
| CN101312499B (en) | Video processor capable of recombination and application apparatus thereof | |
| CN109218636A (en) | The binaryzation data output method of imaging sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191028 Address after: 401332 No. 367 Xiyong Road, Xiyong Town Micropower Park, Shapingba District, Chongqing Patentee after: China Electric Technology Group Chongqing acoustic photoelectric Co., Ltd. Address before: 400060 Chongqing District, Huayuan Road City, No. 14, No. 44 Patentee before: The 44th Research Institute of China Electronic Science and Technology Group Corporation |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201125 Address after: No. 28-2, Xiyuan 1st Road, Shapingba District, Chongqing 400030 Patentee after: UNITED MICROELECTRONICS CENTER Co.,Ltd. Address before: 401332 No. 367 Xiyong Road, Xiyong Town Micropower Park, Shapingba District, Chongqing Patentee before: CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION CHONGQING ACOUSTIC-OPTIC-ELECTRONIC Co.,Ltd. |
|
| TR01 | Transfer of patent right |