CN114371471A - Photosensitive chip capable of realizing laser radar point cloud and image fusion - Google Patents
Photosensitive chip capable of realizing laser radar point cloud and image fusion Download PDFInfo
- Publication number
- CN114371471A CN114371471A CN202210068527.2A CN202210068527A CN114371471A CN 114371471 A CN114371471 A CN 114371471A CN 202210068527 A CN202210068527 A CN 202210068527A CN 114371471 A CN114371471 A CN 114371471A
- Authority
- CN
- China
- Prior art keywords
- laser
- pixels
- photosensitive
- visible light
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The invention relates to the technical field of fusion of laser radar depth information and image information, in particular to a photosensitive chip capable of realizing fusion of laser radar point cloud and an image, which comprises a plurality of photosensitive pixel units distributed in a matrix form, wherein the photosensitive pixel units comprise a plurality of visible light photosensitive pixels and laser photosensitive pixels distributed in the matrix form, the laser photosensitive pixels are embedded among the plurality of visible light photosensitive pixels distributed in the matrix form, the laser photosensitive pixels provide depth detection information for the surrounding visible light photosensitive pixels, and the visible light photosensitive pixels provide object characteristic information for the laser photosensitive pixels. Compared with the prior art, the photosensitive chip capable of realizing the fusion of the point cloud and the image of the laser radar realizes the fusion of depth measurement and the image, can simplify the structure of the system, improve the stability of the system, can fuse the depth information measured by the laser photosensitive pixels and the color information measured by the visible light pixels in real time, and avoids a complex fusion algorithm.
Description
[ technical field ] A method for producing a semiconductor device
The invention relates to the technical field of fusion of laser radar depth information and image information, in particular to a photosensitive chip capable of realizing the fusion of laser radar point cloud and an image.
[ background of the invention ]
The laser radar can detect three-dimensional information of an object and generate a depth point cloud picture of the object; the image information of the object provides characteristic information such as texture, color and the like of the object, and the identification is convenient. The depth information of the laser radar is fused with the characteristic information of the image to form a color three-dimensional image, and the identification, processing and application of the three-dimensional image are facilitated.
In the fusion scheme in the prior art, a laser radar system, a camera and two independent component systems are used, so that the structural design is complicated, the stability requirement of the design is high, the laser radar system and the camera image system need to be calibrated, then the point cloud and the image of the measured object are respectively obtained, the algorithm of image fusion is complex, and meanwhile, the cost is increased due to the two systems.
[ summary of the invention ]
In order to overcome the problems, the invention provides a photosensitive chip capable of realizing the fusion of the point cloud and the image of the laser radar, which can effectively solve the problems.
The invention provides a technical scheme for solving the technical problems, which comprises the following steps: the utility model provides a can realize sensitization chip of laser radar point cloud and image fusion, including a plurality of sensitization pixel units that are the matrix and distribute, including a plurality of visible light sense light pixel and the laser sensitive pixel that are the matrix and distribute in the sensitization pixel unit, the laser sensitive pixel inlays between a plurality of visible light sense light pixel that are the matrix and distribute, the laser sensitive pixel provides degree of depth detection information for visible light sense light pixel on every side, and visible light sense light pixel provides object characteristic information for the laser sensitive pixel.
Preferably, the visible light sensitive pixels comprise visible light sensitive areas and laser sensitive areas, and laser sensitive pixels are arranged in the laser sensitive areas.
Preferably, the number of the laser sensitive pixels accounts for ten percent of the sum of the number of the visible light sensitive pixels and the number of the laser sensitive pixels.
Preferably, the number of the laser sensitive pixels accounts for twenty percent of the sum of the number of the visible light sensitive pixels and the number of the laser sensitive pixels.
Preferably, the number of the laser sensitive pixels accounts for thirty percent of the sum of the number of the visible light sensitive pixels and the number of the laser sensitive pixels.
Preferably, the number of the laser sensitive pixels is N percent of the sum of the number of the visible light sensitive pixels and the number of the laser sensitive pixels, wherein N is greater than zero and less than or equal to fifty.
Compared with the prior art, the photosensitive chip capable of realizing the fusion of the point cloud and the image of the laser radar integrates the distance measurement and the image test chip of the laser radar on the same chip, realizes the fusion of the depth measurement and the image, can simplify the structure of the system and improve the stability of the system; by means of interpolation, depth information measured by the laser photosensitive pixels and color information measured by the visible light pixels can be fused in real time, and a complex fusion algorithm is avoided.
[ description of the drawings ]
Fig. 1 is a schematic structural diagram of a conventional 4000 x 3000 visible light CMOS chip in the prior art;
fig. 2 is a schematic diagram illustrating a conventional 4000 x 3000 visible light CMOS chip in the prior art, in which each pixel uses an interpolation algorithm to obtain signals except for its own received color signal;
FIG. 3 is a schematic diagram of the area division of the photosensitive chip capable of realizing the fusion of the point cloud and the image of the laser radar according to the present invention;
FIG. 4 is a schematic diagram of a first embodiment of a photosensitive chip capable of achieving laser radar point cloud and image fusion according to the present invention;
FIG. 5 is an enlarged view of a single light-sensitive pixel cell of FIG. 4;
FIG. 6 is a schematic diagram of a second embodiment of the photosensitive chip capable of achieving the fusion of the laser radar point cloud and the image according to the present invention;
fig. 7 is an enlarged view of a single light-sensitive pixel cell of fig. 6.
[ detailed description ] embodiments
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It should be noted that all directional indications (such as up, down, left, right, front, and back … …) in the embodiments of the present invention are limited to relative positions on a given view, not absolute positions.
In addition, the descriptions related to "first", "second", etc. in the present invention are only for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Referring to fig. 1 and 2, a conventional 4000 x 3000 visible CMOS chip structure is shown for this Bayer pattern sensor. Each of the R (red light-sensitive pixel), G (green light-sensitive pixel), and B (blue light-sensitive pixel) pixels can only receive corresponding red, green, and blue monochromatic light, and in order to recover a color image, an interpolation algorithm is required for each pixel to obtain signals other than the self-received color signal, for example, for the B6 pixel channel in fig. 2, the interpolation algorithm is required to obtain red and green information, for example:
the R color information is obtained by averaging adjacent four R pixels:
RB6＝(R1+R3+R9+R11)/4
the G color information is obtained by averaging adjacent four R pixels:
GB6＝(G2+G5+G7+G10)/4
the conventional visible light photosensitive pixels can collect characteristic information of texture, color and the like of an object by receiving visible light reflected or emitted by the object, but cannot detect depth information; the laser radar collects depth information of an object by emitting infrared laser to the object and receiving laser signals reflected by the object, but cannot collect color information of the object.
With the above teachings, referring to fig. 3 to 7, the photosensitive chip capable of fusing the point cloud and the image of the laser radar according to the present invention includes a plurality of photosensitive pixel units distributed in a matrix form, the photosensitive pixel units include a plurality of visible light sensing pixels and laser light sensing pixels distributed in a matrix form, the laser light sensing pixels are embedded among the plurality of visible light sensing pixels distributed in a matrix form, the laser light sensing pixels provide depth detection information for the surrounding visible light sensing pixels, and the visible light sensing pixels provide object characteristic information for the laser light sensing pixels. The visible light sensitive pixels include three pixels of R (red light sensitive pixel), G (green light sensitive pixel) and B (blue light sensitive pixel).
The number of the laser photosensitive pixels accounts for ten percent of the sum of the number of the visible light photosensitive pixels and the number of the laser photosensitive pixels.
In the first embodiment, for a 4000 × 3000 photo-sensing chip, 100 regions are divided according to 400 × 300, a laser-sensitive pixel X is inserted into a pixel position in the middle of each region, the distance information measured by the laser-sensitive pixel X can be copied to other visible light-sensitive pixels in the 400 × 300 region, and the R, G, B information of the laser-sensitive pixel X can be obtained by interpolation, for example:
Rx＝(R1+R2+R3+R4+R5+R6+R7+R8)/8
Gx＝(G1+G2+G3+G4+G5+G6+G7+G8)/8
Bx＝(B1+B2+B3+B4+B5+B6+B7+B8)/8
after Rx, Gx, Bx are obtained, R, G, B information can be obtained by the remaining visible light sensing pixels through the interpolation method described above.
In addition, the number of the laser sensitive pixels may also account for twenty percent, thirty percent, or N percent of the sum of the number of the visible light sensitive pixels and the number of the laser sensitive pixels, where N is greater than zero and equal to or less than fifty. The insertion mode is similar to the above mode, the higher the insertion ratio of the laser photosensitive pixels is, the more accurate the obtained distance information is, the closer the distance information is to the true value, but as the number of the inserted laser photosensitive pixels increases, the number of the visible light photosensitive pixels decreases, the acquired color information will decrease, the color reduction effect is reduced, and a proper insertion ratio needs to be selected according to different test requirements.
In a second embodiment, the photosensitive chip includes a plurality of photosensitive pixel units that are distributed in a matrix form, each photosensitive pixel unit includes a plurality of visible light photosensitive pixels that are distributed in a matrix form, the visible light photosensitive pixels include visible light photosensitive areas and laser photosensitive areas, and the laser photosensitive areas are provided with laser photosensitive pixels. Each laser photosensitive area acquires distance information, and the visible light photosensitive area acquires color information, so that each photosensitive pixel unit can provide the distance information and the color information at the same time, and the improvement of the accuracy of visible light images and distance detection is facilitated.
The photosensitive chip capable of realizing the point cloud and image fusion of the laser radar can be used in a solid-state laser and can also be used in a mixed solid-state and mechanical laser radar system.
Compared with the prior art, the photosensitive chip capable of realizing the fusion of the point cloud and the image of the laser radar integrates the distance measurement and the image test chip of the laser radar on the same chip, realizes the fusion of the depth measurement and the image, can simplify the structure of the system and improve the stability of the system; by means of interpolation, depth information measured by the laser photosensitive pixels and color information measured by the visible light pixels can be fused in real time, and a complex fusion algorithm is avoided.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and any modifications, equivalents, improvements, etc. made within the spirit of the present invention should be included in the scope of the present invention.
Claims (6)
1. Can realize the sensitization chip of laser radar point cloud and image fusion, its characterized in that includes a plurality of sensitization pixel units that are the matrix and distribute, including a plurality of visible light sense light pixel and the laser sensitive pixel that are the matrix and distribute in the sensitization pixel unit, the laser sensitive pixel inlays between a plurality of visible light sense light pixel that are the matrix and distribute, the laser sensitive pixel provides degree of depth detection information for visible light sense light pixel on every side, and visible light sense light pixel provides object characteristic information for the laser sensitive pixel.
2. The photosensitive chip capable of achieving the point cloud and image fusion of the lidar according to claim 1, wherein the visible light photosensitive pixels comprise visible light photosensitive areas and laser photosensitive areas, and laser photosensitive pixels are arranged in the laser photosensitive areas.
3. The photo chip for achieving laser radar point cloud and image fusion as claimed in claim 1, wherein the number of the laser photo pixels is ten percent of the sum of the number of the visible light photo pixels and the number of the laser photo pixels.
4. The photo chip for achieving laser radar point cloud and image fusion as claimed in claim 1, wherein the number of the laser photo pixels is twenty percent of the sum of the number of the visible light photo pixels and the number of the laser photo pixels.
5. The photo chip for achieving laser radar point cloud and image fusion as claimed in claim 1, wherein the number of the laser photo pixels is thirty percent of the sum of the number of the visible light photo pixels and the number of the laser photo pixels.
6. The photosensitive chip capable of realizing laser radar point cloud and image fusion according to claim 1, wherein the number of the laser photosensitive pixels accounts for N percent of the sum of the number of the visible photosensitive pixels and the number of the laser photosensitive pixels, wherein N is greater than zero and less than or equal to fifty.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210068527.2A CN114371471A (en) | 2022-01-20 | 2022-01-20 | Photosensitive chip capable of realizing laser radar point cloud and image fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210068527.2A CN114371471A (en) | 2022-01-20 | 2022-01-20 | Photosensitive chip capable of realizing laser radar point cloud and image fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114371471A true CN114371471A (en) | 2022-04-19 |
Family
ID=81145630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210068527.2A Pending CN114371471A (en) | 2022-01-20 | 2022-01-20 | Photosensitive chip capable of realizing laser radar point cloud and image fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114371471A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115902818A (en) * | 2023-02-21 | 2023-04-04 | 探维科技(北京)有限公司 | Signal detection system, radar system and detection method of image fusion laser |
-
2022
- 2022-01-20 CN CN202210068527.2A patent/CN114371471A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115902818A (en) * | 2023-02-21 | 2023-04-04 | 探维科技(北京)有限公司 | Signal detection system, radar system and detection method of image fusion laser |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106127745B (en) | The combined calibrating method and device of structure light 3 D vision system and line-scan digital camera | |
| CN102378015B (en) | Use the image capture of brightness and chromaticity transducer | |
| CN102801929B (en) | Imageing sensor and picture pick-up device | |
| CN102449548B (en) | Image capturing apparatus | |
| US8830382B2 (en) | Image pickup apparatus and image processing apparatus | |
| US6924841B2 (en) | System and method for capturing color images that extends the dynamic range of an image sensor using first and second groups of pixels | |
| US9048153B2 (en) | Three-dimensional image sensor | |
| KR101517704B1 (en) | Image recording device and method for recording an image | |
| CN101971072A (en) | Image sensor and focus detection apparatus | |
| CN102484721A (en) | Four-channel color filter array pattern | |
| CN107851322A (en) | Method and apparatus for determining depth map for image | |
| CN102598682B (en) | Three-dimensional Imaging Device | |
| US20020122117A1 (en) | Camera device, camera system and image processing method | |
| WO2021174425A1 (en) | Image sensor and image sensitization method | |
| EP3513550B1 (en) | Flat digital image sensor | |
| CN103444183A (en) | Color imaging element, imaging device, and imaging program | |
| WO2023016144A1 (en) | Focusing control method and apparatus, imaging device, electronic device, and computer readable storage medium | |
| CN114371471A (en) | Photosensitive chip capable of realizing laser radar point cloud and image fusion | |
| CN106067937A (en) | Camera lens module array, image sensering device and digital zooming image interfusion method | |
| CN104427329B (en) | Camera device and system and its manufacture method based on single imaging sensor | |
| CN104935793A (en) | Filter-array-equipped microlens and solid-state imaging device | |
| CN108495115A (en) | Imaging sensor and its pixel groups and pel array, the method for obtaining image information | |
| CN103621078A (en) | Image processing apparatus and image processing program | |
| JPH08166237A (en) | Method of detecting phase difference between a pair of images | |
| CN106352847B (en) | Distance-measuring device and distance measurement method based on phase difference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Liang Yanhong Inventor after: Yan Xueliang Inventor after: Yu Yanwu Inventor before: Liang Yanhong |