CN104065937A - Real-time high-speed image pre-processing method for CMOS image sensor - Google Patents
Real-time high-speed image pre-processing method for CMOS image sensor Download PDFInfo
- Publication number
- CN104065937A CN104065937A CN201410276634.XA CN201410276634A CN104065937A CN 104065937 A CN104065937 A CN 104065937A CN 201410276634 A CN201410276634 A CN 201410276634A CN 104065937 A CN104065937 A CN 104065937A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- group
- static memory
- data line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Processing (AREA)
- Image Input (AREA)
Abstract
A real-time high-speed image pre-processing method for a CMOS image sensor is characterized in that Bayer data output by a light-sensing part of the CMOS image sensor are included, the row number of a single-frame array of the Bayer data is M, and the column number of the single-frame array of the Bayer data is N. According to the innovation of the method, an assembly-line type working mode storing data in the modes of block-split and classification is adopted, the data are broken up into parts in the modes of block-split and classification, and relative less registers are adopted to process the data which have been broken up into parts. The real-time high-speed image pre-processing method has the advantages that the cache processing mode is changed, an on-chip cache structure which occupies less hardware resources is adopted, high-speed processing of larger cache data is achieved, and the integration level and on-chip processing speed of the CMOS image sensor are increased greatly.
Description
Technical field
The present invention relates to a kind of photo-sensor signal will treatment technology, relate in particular to a kind of real time high-speed image pre-processing method for cmos image sensor.
Background technology
Along with raising and the informationalized high speed development of technological level, CMOS imageing sensor with its high integration, high-speed and low-power consumption, advantage replaces ccd sensor gradually cheaply, is widely used in digital camera, video camera and the first-class multiple fields of security protection shooting.
In prior art, there is correlation in image quality and the process of the cmos image sensor of producing based on existing technique, the less image quality of process is poorer, therefore, how making cmos image sensor under less process condition, obtain high-quality imaging is the hot issue of studying at present.
The data mode of the photographic department output of cmos image sensor is generally Bayer form (calling Bayer data in the following text), Bayer data are not generally done direct application, need to carry out being processed by successive image Processing Algorithm again after preliminary treatment to it, in the time that Bayer data are carried out to preliminary treatment, prior art generally adopts memory and register serial mode to process, its principle as shown in Figure 2, capable for a frame M() × N(row) the Bayer data of array, if color interpolation algorithm needs buffer memory M, capable data are processed, when processing, just need M × N register and M-1 RAM memory, in the time of Bayer data input register, in first clock cycle, data are read into A
mb
nin individual register, when next clock arrives, when reading in new data, by original A
mb
nin a upper beat data serial move into A
mb
n-1in, in subsequent process, data are transmitted successively in register and RAM memory, when first order register has all read in after data, after next clock cycle arrives, the data of one-level register are delivered to second level register successively and carry out algorithm process (in whole process, RAM plays data cached effect), M × N Bayer data are all moved into second level buffer at least needs (2M-1) × N+M clock cycle.The problem that aforementioned processing mode exists is: if subsequent algorithm requires the pel array of processing larger, when data are more, existing preprocess method will need a large amount of RSs, data serial transmission also can need the more time, will increase like this area and the power consumption of chip, and can reduce the processing speed of algorithm, simultaneously because the sequential serial line by line of data is inputted, the order of type of pixel maintains the type of pixel order of original Bayer form, and this difficulty that also causes code to be realized strengthens.
Summary of the invention
For the problem in background technology, the present invention proposes a kind of real time high-speed image pre-processing method for cmos image sensor, comprise that the line number of the single frames array of described Bayer data is that M, columns are N by the Bayer data of the photographic department output of cmos image sensor, its innovation is:
Build cache module: adopt K static memory to build cache module, 10≤K < M, and K is even number, the storage depth of single static memory be J(herein indication storage depth be exactly memory capacity, its unit is byte); K static memory is divided into L processing group, L=K/2, all corresponding two static memories of each processing group;
Bayer data are classified: the single file data in described single frames array are designated as data line, multiple data during individual data is capable are divided into two groups by strange, the even relation of the corresponding columns sequence number of each data: in same data line, columns sequence number is multiple data formation data groups one of odd number, multiple data formation data groups two that columns sequence number is even number; Each data line is all to there being a data group one and one data group two, and data amount check in data group one and data group two is N/2, N/2=J; The mode that Bayer data are classified is designated as classifying rules;
When processing, carry out as follows:
1) press classifying rules, the 1st in single frames array to the data in L data line are classified, obtain 2L data group;
2) press the row order in single frames array, in the static memory in the data group in each data line successively input buffer module, the corresponding data group of each static memory, two data groups in same data line are inputted in two static memories of same processing group; Multiple data in individual data group are with in serial mode input static memory; If corresponding the 1st processing group of the 1st data line, corresponding the 2nd processing group of the 2nd data line ... the corresponding L of L data line processes group;
3) after the data of L data line have been inputted, the multiple data outside parallel output that static memory has started buffer memory;
When the outside parallel output of static memory, the multiple data in L+1 data line are pressed after classifying rules classification, serial input the 1st processing group in the static memory of correspondence, in order to replace the multiple data in the 1st data line; The aforesaid processing mode with legacy data in new data replacement processing group is designated as renewal processing, in the process of follow-up parallel output static memory data, continues, with new data line, all the other processing groups are upgraded to processing successively;
4) after L processing group all completed and upgrade and process, by mode in step 3), L processing group upgraded to processing again successively with new data line, until cache module has completed buffer memory, exported processing all data provisional capital.
Principle of the present invention is: the solution of the present invention has formed in fact a kind of piecemeal of pipeline system, the mode of operation of classifying type access data, pass through piecemeal, the mode of classifying type breaks the whole up into parts data, adopt the relatively less register of quantity to dividing the data after zero to process simultaneously, not only greatly reduce processing delay, improve processing speed, and save chip area, improve chip integration, specifically, the present invention is achieved in that " piecemeal " carries out serial input buffer memory one by one with data behavior unit, data line is divided into two to be stored, output is processed for the data in all piecemeals and by its parallel output.Complete first all data storages of memories, while carrying out more new data, serial input and parallel output will synchronously carry out, " classifying type " is strange by columns by the data in each data line, idol is divided into two data groups, certainly, why can do such classification, it is the inherent characteristic that has relied on Bayer data, individual data in Bayer data is capable, R and G alternative arrangement, G and B alternative arrangement (RGB is red-green-blue), and in different pieces of information row, strange, the even relation of R, G, B is constant, in the present invention, utilize fully this specific character of Bayer data, classified by strange, even relation, data line is broken the whole up into parts, to coordinate the present invention program's enforcement, " streamline " carried out, after buffer memory, output processing, continuing in new data line input buffer module, to process to the data in a certain data line, the direct effect that the mode of operation that aforementioned three kinds of means form is brought, make exactly register quantity required in preprocessing process significantly reduce, from the data scale of processing, in the single treatment of prior art, need to process the data of the M × N in single frames array simultaneously, therefore it needs M-1 RAM memory and M × N register, and in the present invention, due to single frames array has been carried out to " piecemeal ", " classification ", thereby the register quantity of single buffer memory needs and the degree of depth of single static memory are significantly reduced, be conducive to carry out layout on limited chip area, while is due to the parallel output of data, design difficulty and processing delay are reduced, improve the efficiency of processing.
The means of controlling data transfer in the present invention are same as the prior art, it is controlled sequential and adopts conventional clock CLK to control, for sequencing control is more simplified, the present invention has also done following improvement: the data group one in two static memories in single processing group and individual data are capable and the corresponding relation of data group two keep constant.
Bring based on aforementioned schemes to the reducing of static memory physical requirement, the invention allows for following preferred version and improve the dimensional parameters of device: described static memory adopts on-chip memory to realize.On-chip memory is due to size restrictions, its disposal ability is not as the buffer storage of peripheral hardware, but after employing the solution of the present invention, a small amount of on-chip memory just can be born the buffer memory work of larger data amount, this means a great for the integrated level that improves CMOS imageing sensor.
Preferably, when M gets 512, N is while getting 512, K gets 10, and the storage depth J of static memory is 256.
Useful technique effect of the present invention is: by caching process mode is changed, use takies buffer structure on the less sheet of hardware resource, realize large data cached high speed processing, significantly improved processing speed on the integrated level of cmos image sensor and sheet.
Brief description of the drawings
Fig. 1, handling principle schematic diagram of the present invention (in figure, the quantity of static memory is 10);
The handling principle schematic diagram of Fig. 2, prior art;
Data array schematic diagram in Fig. 3, Bayer data;
Fig. 4, cmos sensor system on chip structure schematic diagram.
Embodiment
For a real time high-speed image pre-processing method for cmos image sensor, comprise that the line number of the single frames array of described Bayer data is that M, columns are N by the Bayer data of the photographic department output of cmos image sensor, its innovation is:
Build cache module: adopt K static memory to build cache module, 10≤K < M, and K is even number, the storage depth of single static memory is J; K static memory is divided into L processing group, L=K/2, all corresponding two static memories of each processing group;
Bayer data are classified: the single file data in described single frames array are designated as data line, multiple data during individual data is capable are divided into two groups by strange, the even relation of the corresponding columns sequence number of each data: in same data line, columns sequence number is multiple data formation data groups one of odd number, multiple data formation data groups two that columns sequence number is even number; Each data line is all to there being one and data group two of a data group, and data amount check in data group one and data group two is N/2, N/2=J; The mode that Bayer data are classified is designated as classifying rules;
When processing, carry out as follows:
1) press classifying rules, the 1st in single frames array to the data in L data line are classified, obtain 2L data group;
2) press the row order in single frames array, in the static memory in the data group in each data line successively input buffer module, the corresponding data group of each static memory, two data groups in same data line are inputted in two static memories of same processing group; Multiple data in individual data group are with in serial mode input static memory; If corresponding the 1st processing group of the 1st data line, corresponding the 2nd processing group of the 2nd data line ... the corresponding L of L data line processes group;
3) after the data of L data line have been inputted, the multiple data outside parallel output that static memory has started buffer memory;
When the outside parallel output of static memory, the multiple data in L+1 data line are pressed after classifying rules classification, serial input the 1st processing group in the static memory of correspondence, in order to replace the multiple data in the 1st data line; The aforesaid processing mode with legacy data in new data replacement processing group is designated as renewal processing, in the process of follow-up parallel output static memory data, continues, with new data line, all the other processing groups are upgraded to processing successively;
4) after L processing group all completed and upgrade and process, by mode in step 3), L processing group upgraded to processing again successively with new data line, until cache module has completed buffer memory, exported processing all data provisional capital.
Further, data group one in capable of two static memories in single processing group and individual data and the corresponding relation of data group two keep constant.
Further, described static memory adopts on-chip memory to realize.
Further, when M gets 512, N is while getting 512, K gets 10, and the storage depth J of static memory is 256.
Capable with a frame 512(below) × 512(row) the Bayer data instance of array, the solution of the present invention and prior art are quantized to comparison:
Based on the present invention program, while adopting 10 static memories to process 512 × 512 array datas, in the time that buffer memory 5 row data are processed, need static memory and 10 registers that 10 degree of depth are 256 words to carry out the buffer memory computing of data, the whole serials of frame data are read in parallel read-out needs 262144 clock cycle.
Known according to prior art, prior art is in the time processing 512 × 512 data, in the time that buffer memory 5 row data are processed, need static memory and 3072 registers that 5 degree of depth are 512 words to carry out the buffer memory computing of data, the required clock cycle of access of total data is 524288.
The present invention and prior art have greater advantage aspect chip area and layout, the static memory that the thought of utilizing piecemeal is 512 words by 5 degree of depth is divided into the static memory that 10 degree of depth are 256 words, this integral layout that is conducive to SOC (system on a chip) with and reduce design difficulty, and for same process conditions, the register quantity that the present invention uses is significantly less than the register quantity (capacity of the two static memory is identical) of prior art, and this can saving chip area and power consumption.
Claims (4)
1. for a real time high-speed image pre-processing method for cmos image sensor, comprise that the line number of the single frames array of described Bayer data is that M, columns are N, it is characterized in that by the Bayer data of the photographic department output of cmos image sensor:
Build cache module: adopt K static memory to build cache module, 10≤K < M, and K is even number, the storage depth of single static memory is J; K static memory is divided into L processing group, L=K/2, all corresponding two static memories of each processing group;
Bayer data are classified: the single file data in described single frames array are designated as data line, multiple data during individual data is capable are divided into two groups by strange, the even relation of the corresponding columns sequence number of each data: in same data line, columns sequence number is multiple data formation data groups one of odd number, multiple data formation data groups two that columns sequence number is even number; Each data line is all to there being a data group one and one data group two, and data amount check in data group one and data group two is N/2, N/2=J; The mode that Bayer data are classified is designated as classifying rules;
When processing, carry out as follows:
1) press classifying rules, the 1st in single frames array to the data in L data line are classified, obtain 2L data group;
2) press the row order in single frames array, in the static memory in the data group in each data line successively input buffer module, the corresponding data group of each static memory, two data groups in same data line are inputted in two static memories of same processing group; Multiple data in individual data group are with in serial mode input static memory; If corresponding the 1st processing group of the 1st data line, corresponding the 2nd processing group of the 2nd data line ... the corresponding L of L data line processes group;
3) after the data of L data line have been inputted, the multiple data outside parallel output that static memory has started buffer memory;
When the outside parallel output of static memory, the multiple data in L+1 data line are pressed after classifying rules classification, serial input the 1st processing group in the static memory of correspondence, in order to replace the multiple data in the 1st data line; The aforesaid processing mode with legacy data in new data replacement processing group is designated as renewal processing, in the process of follow-up parallel output static memory data, continues, with new data line, all the other processing groups are upgraded to processing successively;
4) after L processing group all completed and upgrade and process, by mode in step 3), L processing group upgraded to processing again successively with new data line, until cache module has completed buffer memory, exported processing all data provisional capital.
2. the real time high-speed image pre-processing method for cmos image sensor according to claim 1, is characterized in that: the data group one in two static memories in single processing group and individual data are capable and the corresponding relation of data group two keep constant.
3. the real time high-speed image pre-processing method for cmos image sensor according to claim 1, is characterized in that: described static memory adopts on-chip memory to realize.
4. the real time high-speed image pre-processing method for cmos image sensor according to claim 1, is characterized in that: when M gets 512, N is while getting 512, K gets 10, and the storage depth J of static memory is 256.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410276634.XA CN104065937B (en) | 2014-06-20 | 2014-06-20 | For the real time high-speed image pre-processing method of cmos image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410276634.XA CN104065937B (en) | 2014-06-20 | 2014-06-20 | For the real time high-speed image pre-processing method of cmos image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104065937A true CN104065937A (en) | 2014-09-24 |
| CN104065937B CN104065937B (en) | 2016-01-13 |
Family
ID=51553442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410276634.XA Active CN104065937B (en) | 2014-06-20 | 2014-06-20 | For the real time high-speed image pre-processing method of cmos image sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104065937B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376534A (en) * | 2014-10-17 | 2015-02-25 | 中国电子科技集团公司第四十四研究所 | Low-power-consumption real-time noise-reduction and sharpening merged preprocessing algorithm for CMOS image sensor |
| CN106060432A (en) * | 2016-06-02 | 2016-10-26 | 西北核技术研究所 | Real-time deserializing method for CMOS image sensor data signals |
| CN108829628A (en) * | 2018-05-30 | 2018-11-16 | 江南大学 | A kind of tricot machine double Jacquard real-time control methods and device |
| CN109741237A (en) * | 2018-12-28 | 2019-05-10 | 中国科学院半导体研究所 | Large-scale image data processing system and method |
| CN111193869A (en) * | 2020-01-09 | 2020-05-22 | Oppo广东移动通信有限公司 | Image data processing method, image data processing device and mobile terminal |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101119490A (en) * | 2006-08-02 | 2008-02-06 | 索尼株式会社 | Image signal processing device and method |
| CN201937742U (en) * | 2010-12-14 | 2011-08-17 | 深圳市视鑫数码有限公司 | High-speed image acquisition system |
| US20120033118A1 (en) * | 2009-03-16 | 2012-02-09 | Zeeann Co., Ltd. | Cmos image sensor having wide dynamic range and sensing method thereof |
| US20140042303A1 (en) * | 2010-08-18 | 2014-02-13 | Taiwan Semiconductor Manufacturing Co., Ltd. | Cmos sensor with low partition noise and low disturbance between adjacent row control signals in a pixel array |
-
2014
- 2014-06-20 CN CN201410276634.XA patent/CN104065937B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101119490A (en) * | 2006-08-02 | 2008-02-06 | 索尼株式会社 | Image signal processing device and method |
| US20120033118A1 (en) * | 2009-03-16 | 2012-02-09 | Zeeann Co., Ltd. | Cmos image sensor having wide dynamic range and sensing method thereof |
| US20140042303A1 (en) * | 2010-08-18 | 2014-02-13 | Taiwan Semiconductor Manufacturing Co., Ltd. | Cmos sensor with low partition noise and low disturbance between adjacent row control signals in a pixel array |
| CN201937742U (en) * | 2010-12-14 | 2011-08-17 | 深圳市视鑫数码有限公司 | High-speed image acquisition system |
Non-Patent Citations (1)
| Title |
|---|
| 吴治军等: "一种应用于CMOS图像传感器的电容阵列PGA研究", 《半导体光电》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376534A (en) * | 2014-10-17 | 2015-02-25 | 中国电子科技集团公司第四十四研究所 | Low-power-consumption real-time noise-reduction and sharpening merged preprocessing algorithm for CMOS image sensor |
| CN104376534B (en) * | 2014-10-17 | 2017-05-24 | 中国电子科技集团公司第四十四研究所 | Low-power-consumption real-time noise-reduction and sharpening merged preprocessing method for CMOS image sensor |
| CN106060432A (en) * | 2016-06-02 | 2016-10-26 | 西北核技术研究所 | Real-time deserializing method for CMOS image sensor data signals |
| CN106060432B (en) * | 2016-06-02 | 2018-11-23 | 西北核技术研究所 | A kind of processing method of unstringing in real time of cmos image sensor data-signal |
| CN108829628A (en) * | 2018-05-30 | 2018-11-16 | 江南大学 | A kind of tricot machine double Jacquard real-time control methods and device |
| CN109741237A (en) * | 2018-12-28 | 2019-05-10 | 中国科学院半导体研究所 | Large-scale image data processing system and method |
| CN109741237B (en) * | 2018-12-28 | 2020-10-23 | 中国科学院半导体研究所 | Large-scale image data processing system and method |
| CN111193869A (en) * | 2020-01-09 | 2020-05-22 | Oppo广东移动通信有限公司 | Image data processing method, image data processing device and mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104065937B (en) | 2016-01-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3346423B1 (en) | Deep convolutional network heterogeneous architecture system and device | |
| US10726177B2 (en) | Reconfigurable interconnect | |
| EP3346425B1 (en) | Hardware accelerator engine and method | |
| US20200167890A1 (en) | Configurable and programmable image processor unit | |
| CN102665049B (en) | Programmable visual chip-based visual image processing system | |
| Negi et al. | Deep pipelined one-chip FPGA implementation of a real-time image-based human detection algorithm | |
| EP3346426A1 (en) | Reconfigurable interconnect, corresponding system and method | |
| CN104065937B (en) | For the real time high-speed image pre-processing method of cmos image sensor | |
| EP3346427B1 (en) | Configurable accelerator framework, system and method | |
| US7409528B2 (en) | Digital signal processing architecture with a wide memory bandwidth and a memory mapping method thereof | |
| US8989509B2 (en) | Streaming wavelet transform | |
| KR20160038741A (en) | System having low power computation architecture | |
| US20060161720A1 (en) | Image data transmission method and system with DMAC | |
| CN105611114B (en) | Digital multireel for AER imaging sensors accumulates nuclear convolution processing chip | |
| US11816871B2 (en) | Real-time low latency computer vision/machine learning compute accelerator with smart convolutional neural network scheduler | |
| CN112235579B (en) | Video processing method, computer-readable storage medium and electronic device | |
| CN102131053A (en) | Data acquisition, coding and storage method applied to high speed imaging system | |
| CN111340835A (en) | FPGA-based video image edge detection system | |
| US20200342285A1 (en) | Data processing using a neural network system | |
| JP2020042774A (en) | Artificial intelligence inference computing device | |
| US9030570B2 (en) | Parallel operation histogramming device and microcomputer | |
| US11048650B1 (en) | Method and system for integrating processing-in-sensor unit and in-memory computing unit | |
| US11341603B2 (en) | Electronic image processing device | |
| CN106708457A (en) | FPGA (Field Programmable Gate Array) processing module and method used for DMD (Digital Micromirror Device) dynamic area selection | |
| CN102707919A (en) | Device and method for controlling FIFO (First In First Out) read-write by using finite state machine (FSM) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191028 Address after: 401332 No. 367 Xiyong Road, Xiyong Town Micropower Park, Shapingba District, Chongqing Patentee after: China Electric Technology Group Chongqing acoustic photoelectric Co., Ltd. Address before: 400060 Chongqing District, Huayuan Road City, No. 14, No. 44 Patentee before: The 44th Research Institute of China Electronic Science and Technology Group Corporation |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201125 Address after: No. 28-2, Xiyuan 1st Road, Shapingba District, Chongqing 400030 Patentee after: UNITED MICROELECTRONICS CENTER Co.,Ltd. Address before: 401332 No. 367 Xiyong Road, Xiyong Town Micropower Park, Shapingba District, Chongqing Patentee before: CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION CHONGQING ACOUSTIC-OPTIC-ELECTRONIC Co.,Ltd. |