CN114051088A - High-speed camera module method based on field decomposition - Google Patents
High-speed camera module method based on field decomposition Download PDFInfo
- Publication number
- CN114051088A CN114051088A CN202210033366.3A CN202210033366A CN114051088A CN 114051088 A CN114051088 A CN 114051088A CN 202210033366 A CN202210033366 A CN 202210033366A CN 114051088 A CN114051088 A CN 114051088A
- Authority
- CN
- China
- Prior art keywords
- frame
- camera module
- image
- speed camera
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a high-speed camera module method based on field decomposition, which relates to the technical field of photography, wherein a camera module acquires a time sequence of single-frame images through a lens array and arranges the single-frame images on a CMOS (complementary metal oxide semiconductor) to form X subframe images, wherein the number of lenses in the lens array is the same as that of subframes, the lenses select the same cylindrical lenses, and the frame number of the camera module is Y; sequentially reading the arranged sub-frame images through the CMOS to obtain each frame of imaged image, respectively carrying out correction calculation on each frame of imaged image, and restoring the frame of imaged image into a sub-frame image sequence in the optical axis direction; and carrying out image fusion on the transverse pixels of the subframe image sequence to obtain the image sequence with the enhanced light receiving quantity.
Description
Technical Field
The invention relates to the technical field of photography, in particular to a high-speed camera module method based on field decomposition.
Background
The high-speed camera shooting measurement equipment can provide measurement data with wider dynamic range, higher accuracy, more comprehensive parameters and richer content compared with the conventional camera shooting measurement equipment. In terms of the present, the high-speed camera not only consumes very much power, but also a common battery cannot support the problem of the power consumption of the high-speed camera; in addition, the sensor of the high-speed camera is too large compared with the sensor of the mobile phone, if the camera of the mobile phone is selected as the high-speed camera, large-area light sensing cannot be achieved due to the problem of the space size of the mobile phone, if the camera frame rate of the mobile phone is forcibly increased, an image picture is dark, and if a special light sensor is used, the cost is too high.
Therefore, a method for increasing the photosensitive area of an image during image capturing without increasing the hardware space is required.
Disclosure of Invention
In view of the technical defects, the invention provides a high-speed camera module method based on field decomposition.
In order to solve the problems proposed by the background art, the technical scheme of the invention is as follows:
a high-speed camera module method based on field decomposition comprises the following steps:
s1, the camera module acquires single frame images of time sequence through the lens array and forms X sub-frame images on the CMOS, wherein the number of the lenses in the lens array is the same as that of the sub-frames, the lenses select the same cylindrical lenses, and the frame number of the camera module is Y;
s2, reading the arranged sub-frame images in sequence through the CMOS to obtain each frame of imaged image, and respectively carrying out correction calculation on each frame of image to restore the frame of imaged image into a sub-frame image sequence in the optical axis direction;
and S3, carrying out image fusion on the transverse pixels of the sub-frame image sequence to obtain the image sequence with the enhanced light receiving quantity.
Preferably, the lens array described in step S1 is arranged in order from top to bottom, and the X sub-frame images are arranged in order from top to bottom on the CMOS, resulting in a single frame image.
Preferably, the number of frames Y of the image pickup module in step S1 is 60 frames.
Preferably, the image pickup module described in step S1 is an image pickup module having 6400 × 4800 pixels.
Preferably, the lens array described in step S1 is specifically composed of 20 axially slightly curved cylindrical lenses.
Preferably, the image pickup module described in step S1 further includes an image sensor array, and the image sensor array is a two-dimensional pixel array.
Preferably, the CMOS in step S2 reads out the arranged sub-frame images sequentially, specifically, sequentially from top to bottom.
Preferably, the correction calculation is performed according to the difference of the incident angle of the light for each frame of the image sequence described in step S2.
The invention has the beneficial effects that: the invention provides a high-speed camera module method based on field decomposition, which can select the existing mobile phone camera module as a unit module, has low cost, greatly improves the number of mobile phone camera frames and has good market prospect.
Drawings
Fig. 1 is a schematic view of a lens array according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to fig. 1 of the present invention, and other advantages and effects of the present invention can be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, a high-speed camera module method based on field decomposition includes the following steps:
(1) a single module of 6400 × 4800 (= 3072 ten thousand) pixels, 60 frames/sec was used;
(2) adding an array formed by 20 axial micro-curved cylindrical lenses in front of the module lens to enable the same view field scene to form 20 images 6400 x 240 from top to bottom on the CMOS image plane, which are respectively called as sub-frames 1-20; this does not significantly increase the module volume due to the thin thickness of the axially micro-curved cylindrical array.
(3) Thus, since the data readout of the CMOS exposed pixels is performed by (or optionally) scanning from top to bottom and from left to right, and the frame rate of 60fps is such that the result of reading out 20 sub-frames one by one from top to bottom over the entire CMOS in 1/60s is exactly 20 images of the same field of view scene each at (1/60s)/20=1/1200 s;
(4) now, within 1 second, an image sequence of field scenes evenly divided into 60 frames and 1200 subframes according to time is obtained; because the light incidence angles of the scene light of the field of view of 20 subframes in the same frame are different, the scene light needs to be respectively restored to the image in the original standard optical axis direction through correction calculation;
standard optical axis: the center line of the beam (column), or the axis of symmetry of the optical system. The beam rotates around this axis without any change in optical properties. The standard optical axis is the center line of the beam passing through the center point of the camera lens.
Standard angle: typical viewing angles are equivalent to 35-40mm lenses, around 65 °.
The invention can utilize orthodontic algorithm to carry out correction calculation, and the flow of the orthodontic algorithm is as follows:
step 1: the characteristic point sets of the reference image and the registration image which acquire a single frame are respectively
And
；
step 2: randomly extracting 4 groups of feature points in the feature point set, and calculating a second-order tensor by the following formula:
wherein
，
；
Step 3: the error norm is calculated as follows:
wherein the content of the first and second substances,
；
Step 4: recording the obtained error norm as a random variable
Then circulating from Step2 to Step3
Then, a random variable sequence is obtained
。
As can be seen from the above, for random sequences
For all
Presence of a positive integer
All of
So that
，
The minimum error norm theoretically obtainable is recorded as a random sequence
Converge according to the probability so as to
When the size of the particles is large enough,
in close proximity to
Therefore, the orthodontic algorithm can effectively improve the orthodontic effect and reduce the orthodontic error.
And the homography matrix of each sub-frame and the standard image can be calculated through the orthodontic algorithm, the homography matrix is the transformation of the pixel space of the two pictures, and the deviation angle can be calculated through the homography matrix.
(5) Then, since each sub-frame is composed of 6400 × 240 pixels, that is, the number of horizontal pixels is 20 times of the required number of horizontal pixels 320, although the light receiving amount of each sub-frame is only 1/20 of the original light receiving amount, the number of horizontal pixels of 20 times of the sub-frame is merged into 320 pixels through image fusion calculation, not only the light receiving amount of the new frame of 320 × 240 obtained is still 20 × (1/20) =100%, but also the horizontal pixels of the new frame image are all merged into 20 times of pixels, and the quality is obviously higher; thus obtaining the correct video sequence of 1200 frames/second images.
The CMOS data reading steps are as follows:
the first step is as follows: the pixel array is irradiated by the external light to generate a photoelectric effect, and corresponding charges are generated in the pixel units.
The scene is focused by an imaging lens onto an image sensor array, which is a two-dimensional array of pixels, each pixel including a photodiode, the photodiode in each pixel converting the intensity of light at the surface of the array into an electrical signal.
The second step is that: the pixel desired to be operated is selected by the row selection circuit and the column selection circuit, and an electric signal on the pixel is read out.
In the gating process, the row selection logic unit can scan the pixel array line by line or scan the pixel array in an interlaced mode, and the same principle is applied to the rows. The row selection logic unit and the column selection logic unit are matched for use, so that the window extraction function of the image can be realized.
The third step: and carrying out signal processing on the corresponding pixel unit.
The image signals in the row pixel units are transmitted to the corresponding analog signal processing units and A/D converters through the signal buses of the columns where the image signals are located, and the image signals are converted into digital image signals to be output. The analog signal processing unit has the main functions of amplifying signals and improving the signal-to-noise ratio.
The field scenes obtained by 20 subframes of the CMOS exposure pixels due to different incident angles are different, the degree of angle change between each subframe and the adjacent subframe is consistent, the deviation angles of two pictures can be continuously estimated through the picture pairs of the adjacent frames, scanning from top to bottom from left to right is more beneficial to estimating the deviation angle of each subframe from the whole, reordering is not needed, and the subsequent correction calculation is facilitated to restore the images to be in the original standard optical axis direction.
The high-speed camera module technology can be applied to the fields of scientific research, military test, industrial production evaluation and the like. For example, the phenomena of automobile collision test, electric arc generation in the welding process and organic liquid splashing in the battery explosion process can be clearly captured only by means of high-speed camera shooting. High-speed photography can complete quick and multiple sampling of a high-speed target in a short time, and when the high-speed target is projected at a normal speed, the change process of the recorded target is clearly and slowly presented to the eyes of people. The application of the high-speed camera shooting technology in the field of aerospace mainly shows trajectory tracking, missile launching, combustion testing, material testing, PIV testing and the like. The high-speed shooting technology is used for capturing the moving track of a high-speed object which cannot be captured by a common camera by means of the high-frequency shooting speed of the high-speed shooting technology, accurately tracking the motion of an object, measuring the size, distance, change and speed of the object, and then playing the object at the picture frequency which can be seen by human eyes.
The high-speed camera module method based on the visual field decomposition, which is provided by the invention, can select the existing mobile phone camera module as a 'unit module', has lower cost and can simultaneously achieve the following functions:
(1) the height and the width of the high-speed camera module are respectively not more than 2 times of those of the existing mainstream mobile phone, and the thickness of the high-speed camera module is not more than 8 cm;
(2) the shooting resolution of the high-speed camera module is not lower than 320 x 240 pixels;
(3) the shooting frame rate of the high-speed camera module is not lower than 1200 frames/second;
(4) the shooting distance of the high-speed camera module is 1-5 m;
(5) the high-speed camera module can shoot black and white images with 8 bits and 256 gray scales;
(6) the image shooting and data processing of the high-speed camera module can be carried out in a time mode of mutually staggered head-to-tail connection, namely a mode of 'shooting one section completely → processing one section → shooting one section again → processing one section again', and the shooting time of each section is not shorter than 10 minutes;
(7) the cell module may be selected at a frame rate of 60 frames/second.
The foregoing is illustrative of the preferred embodiments of this invention, and it is to be understood that the invention is not limited to the precise form disclosed herein and that various other combinations, modifications, and environments may be resorted to, falling within the scope of the concept as disclosed herein, either as described above or as apparent to those skilled in the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. A high-speed camera module method based on field decomposition is characterized by comprising the following steps:
s1, the camera module acquires single frame images of time sequence through the lens array and forms X sub-frame images on the CMOS, wherein the number of the lenses in the lens array is the same as that of the sub-frames, the lenses all adopt the same cylindrical lenses, and the frame number of the camera module is Y;
s2, reading the arranged sub-frame images in sequence through the CMOS to obtain each frame of imaged image, and respectively carrying out correction calculation on each frame of image to restore the frame of imaged image into a sub-frame image sequence in the optical axis direction;
and S3, carrying out image fusion on the transverse pixels of the sub-frame image sequence to obtain the image sequence with the enhanced light receiving quantity.
2. The field decomposition-based high-speed camera module method according to claim 1, wherein the lens arrays in step S1 are arranged in the order from top to bottom, and X sub-frame images are arranged in the order from top to bottom on the CMOS to obtain a single frame image.
3. The field-of-view decomposition-based high-speed camera module method according to claim 1, wherein the frame number Y of the camera module in step S1 is 60 frames.
4. The method according to claim 1, wherein the camera module of step S1 is 6400 × 4800 pixels.
5. The field-of-view decomposition-based high-speed camera module method according to claim 1, wherein the lens array in step S1 is specifically composed of 20 axially slightly curved cylindrical lenses.
6. The field decomposition-based high-speed camera module method according to claim 1, wherein an image sensor array is further disposed in the camera module in step S1, and the image sensor array is a two-dimensional pixel array.
7. The field decomposition-based high-speed camera module method according to claim 1, wherein the CMOS in step S2 reads the arranged subframe images sequentially, specifically, sequentially from top to bottom.
8. The field-of-view decomposition-based high-speed camera module method according to claim 1, wherein the correction calculation is performed according to the difference of the incident angles of the light rays for each frame of image sequence in step S2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210033366.3A CN114051088B (en) | 2022-01-12 | 2022-01-12 | High-speed camera module shooting method based on field decomposition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210033366.3A CN114051088B (en) | 2022-01-12 | 2022-01-12 | High-speed camera module shooting method based on field decomposition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114051088A true CN114051088A (en) | 2022-02-15 |
| CN114051088B CN114051088B (en) | 2022-03-25 |
Family
ID=80196324
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210033366.3A Active CN114051088B (en) | 2022-01-12 | 2022-01-12 | High-speed camera module shooting method based on field decomposition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114051088B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080025571A1 (en) * | 2006-07-26 | 2008-01-31 | Funai Electric Co., Ltd. | Motion Detection Imaging Device |
| CN104350732A (en) * | 2012-05-28 | 2015-02-11 | 株式会社尼康 | Imaging device |
| CN106803889A (en) * | 2017-03-08 | 2017-06-06 | 吕盼稂 | A kind of intelligent high-speed image capture method and its device in blast process |
| WO2017204172A1 (en) * | 2016-05-25 | 2017-11-30 | Canon Kabushiki Kaisha | Method and apparatus for generating a virtual image from a viewpoint selected by the user, from a camera array with default parameters associated to the selected type of sport event |
| CN111614876A (en) * | 2020-05-19 | 2020-09-01 | 北京海月水母科技有限公司 | Real-time remote high-speed intelligent follow-up shooting method based on galvanometer camera array |
| CN113900079A (en) * | 2021-09-14 | 2022-01-07 | 武汉大学 | Compensation correction device and method for all-optical ultrafast imaging |
-
2022
- 2022-01-12 CN CN202210033366.3A patent/CN114051088B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080025571A1 (en) * | 2006-07-26 | 2008-01-31 | Funai Electric Co., Ltd. | Motion Detection Imaging Device |
| CN104350732A (en) * | 2012-05-28 | 2015-02-11 | 株式会社尼康 | Imaging device |
| WO2017204172A1 (en) * | 2016-05-25 | 2017-11-30 | Canon Kabushiki Kaisha | Method and apparatus for generating a virtual image from a viewpoint selected by the user, from a camera array with default parameters associated to the selected type of sport event |
| CN106803889A (en) * | 2017-03-08 | 2017-06-06 | 吕盼稂 | A kind of intelligent high-speed image capture method and its device in blast process |
| CN111614876A (en) * | 2020-05-19 | 2020-09-01 | 北京海月水母科技有限公司 | Real-time remote high-speed intelligent follow-up shooting method based on galvanometer camera array |
| CN113900079A (en) * | 2021-09-14 | 2022-01-07 | 武汉大学 | Compensation correction device and method for all-optical ultrafast imaging |
Non-Patent Citations (1)
| Title |
|---|
| 于雷: "数字高速相机的光学分幅系统设计", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114051088B (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101888487B (en) | High dynamic range video imaging system and image generating method | |
| KR100315897B1 (en) | Adaptive non-uniformity compensation algorithm | |
| CN103973999B (en) | Camera device and its control method | |
| US9615039B2 (en) | Systems and methods for reducing noise in video streams | |
| CN102726037B (en) | Image processing apparatus, camera head and image processing method | |
| FR2882160A1 (en) | Video image capturing method for e.g. portable telephone, involves measuring local movements on edges of images and estimating parameters of global movement model in utilizing result of measurement of local movements | |
| CN109828285B (en) | Dual-band time domain compressed sensing high-speed imaging method and device | |
| Messikommer et al. | Multi-bracket high dynamic range imaging with event cameras | |
| Yan et al. | High dynamic range imaging via gradient-aware context aggregation network | |
| Zhao et al. | Reconstructing clear image for high-speed motion scene with a retina-inspired spike camera | |
| CN111182238A (en) | High-resolution mobile electronic equipment imaging device and method based on scanning light field | |
| Wang et al. | Neural global shutter: Learn to restore video from a rolling shutter camera with global reset feature | |
| Purohit et al. | Image processing framework for performance enhancement of low-light image sensors | |
| JP2003283907A (en) | Imaging device | |
| CN116489525A (en) | Image processing method and implementation device | |
| Blommel et al. | Effects of microscan operation on staring infrared sensor imagery | |
| CN114051088B (en) | High-speed camera module shooting method based on field decomposition | |
| CN110930440B (en) | Image alignment method, device, storage medium and electronic equipment | |
| Xu et al. | Image lag elimination algorithm for pulse-sequence-based high-speed image sensor | |
| CN107152971B (en) | A method of improving linear array infrared exoelectron enlarged drawing level detail resolution ratio | |
| CN113393379B (en) | High-resolution imaging method for large F number diffraction real-time correction based on pixel coding | |
| CN110809102B (en) | Imaging acceleration method and device based on binary modulation | |
| CN114842046A (en) | Device and method for measuring high-dynamic scene image of moving object | |
| Zhang et al. | Computational Super-Resolution Imaging With a Sparse Rotational Camera Array | |
| CN115967852B (en) | Image processing method based on optical compensation optimization algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |