CN115696077A - Imaging device and imaging control method - Google Patents
Imaging device and imaging control method Download PDFInfo
- Publication number
- CN115696077A CN115696077A CN202110791480.8A CN202110791480A CN115696077A CN 115696077 A CN115696077 A CN 115696077A CN 202110791480 A CN202110791480 A CN 202110791480A CN 115696077 A CN115696077 A CN 115696077A
- Authority
- CN
- China
- Prior art keywords
- pixel
- macro
- pixels
- sub
- spad
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 81
- 238000005259 measurement Methods 0.000 claims description 33
- 230000002194 synthesizing effect Effects 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims 2
- 238000001514 detection method Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Landscapes
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
Description
技术领域technical field
本申请涉及光电探测技术领域,特别涉及一种成像装置及成像控制方法。The present application relates to the field of photoelectric detection technology, in particular to an imaging device and an imaging control method.
背景技术Background technique
基于SPAD(Single Photon Avalanche Diode,单光子雪崩二极管)阵列的3D成像技术是近年来的研究热点,在消费电子、安防、机器人、自动驾驶等领域有巨大的应用前景。3D imaging technology based on SPAD (Single Photon Avalanche Diode) array is a research hotspot in recent years, and has great application prospects in consumer electronics, security, robotics, autonomous driving and other fields.
在相关技术中,为了节省电路面积,将SPAD阵列中空间上彼此相邻的n个SPAD子像素划分为一个宏像素,每个宏像素内的n个SPAD子像素共用同一套处理电路,这样,每个宏像素将对应一个像素数据。相应地,最终输出的图像的分辨率也将降低为未划分宏像素之前的分辨率的1/n。例如,参见图1,将空间上相邻的2*2个SPAD子像素划分为一个宏像素,输出一个像素数据,在这种情况下,当SPAD阵列中包括320*240个SPAD子像素时,输出的图像的分辨率将为160*120,由此可见,相关技术中的成像方法使得SPAD阵列的输出图像的平面分辨率损失较大。In the related art, in order to save the circuit area, the n SPAD sub-pixels spatially adjacent to each other in the SPAD array are divided into one macro-pixel, and the n SPAD sub-pixels in each macro-pixel share the same set of processing circuits, thus, Each macropixel will correspond to one pixel data. Correspondingly, the resolution of the final output image will also be reduced to 1/n of the resolution before the macro pixels are not divided. For example, referring to FIG. 1 , spatially adjacent 2*2 SPAD sub-pixels are divided into a macro-pixel, and a pixel data is output. In this case, when 320*240 SPAD sub-pixels are included in the SPAD array, The resolution of the output image will be 160*120. It can be seen that the imaging method in the related art causes a large loss in the planar resolution of the output image of the SPAD array.
发明内容Contents of the invention
本申请实施例提供了一种成像装置及成像控制方法,可以在SPAD阵列的子像素个数、后续电路不变的基础上,提高平面分辨率。所述技术方案如下:The embodiment of the present application provides an imaging device and an imaging control method, which can improve the planar resolution on the basis that the number of sub-pixels of the SPAD array and subsequent circuits remain unchanged. Described technical scheme is as follows:
一方面,提供了一种成像装置,所述成像装置包括:成像单元和处理单元,所述成像单元包括单光子雪崩二极管SPAD阵列、开关阵列和多路选择器,所述SPAD阵列包括多个宏像素,所述多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素;In one aspect, an imaging device is provided, the imaging device includes: an imaging unit and a processing unit, the imaging unit includes a single photon avalanche diode SPAD array, a switch array and a multiplexer, and the SPAD array includes a plurality of macro A pixel, each macro pixel in the plurality of macro pixels includes (N×M) SPAD sub-pixels, and there are overlapping SPAD sub-pixels between every two adjacent macro pixels;
所述开关阵列用于接收所述多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据;The switch array is used to receive the sub-pixel data respectively output by the (N×M) SPAD sub-pixels included in each macro-pixel in the plurality of macro-pixels, and the (N×M) SPADs included in each macro-pixel Synthesizing the sub-pixel data respectively output by the sub-pixels to obtain one channel of first image data corresponding to the corresponding macro-pixel, and outputting one channel of first image data corresponding to each macro-pixel;
所述多路选择器用于接收所述开关阵列输出的多路第一图像数据,并将所述多路第一图像数据依次输出至所述处理单元;The multiplexer is used to receive multiple channels of first image data output by the switch array, and sequentially output the multiple channels of first image data to the processing unit;
所述处理单元用于接收所述多路选择器依次输出的各个宏像素对应的第一图像数据,并根据各个宏像素的位置和对应的第一图像数据和相应宏像素,生成融合图像。The processing unit is configured to receive the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generate a fused image according to the position of each macro pixel and the corresponding first image data and the corresponding macro pixel.
可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一行SPAD子像素连接一个第一级逻辑或门,第一宏像素中多行SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logic OR gate and a second-level logic OR gate, a row of SPAD sub-pixels in the first macro pixel is connected to a first-level logic OR gate, and multiple rows of SPAD sub-pixels in the first macro pixel A plurality of first-level logic OR gates connected to sub-pixels is connected to a second-level logic OR gate, and the first macro pixel is any one of the plurality of macro pixels;
所述第一宏像素连接的每个第一级逻辑或门用于接收自身连接的一行SPAD子像素中每个SPAD子像素输出的子像素数据,并将接收到的子像素数据合成为一路行像素数据,输出所述行像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive the sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected to itself, and synthesize the received sub-pixel data into one row Pixel data, outputting the row of pixel data;
所述第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的行像素数据,将接收到的行像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and synthesize the received row pixel data to obtain the first the first image data corresponding to the macro pixels, and output the first image data corresponding to the first macro pixels.
可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a column of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro pixel A plurality of first-level logic OR gates connected to sub-pixels is connected to a second-level logic OR gate, and the first macro pixel is any one of the plurality of macro pixels;
所述第一宏像素连接的每个第一级逻辑或门用于接收连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,并将接收到的子像素数据合成为一路列像素数据,输出所述列像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive one path of sub-pixel data output by each SPAD sub-pixel in a connected column of SPAD sub-pixels, and synthesize the received sub-pixel data into one path of column Pixel data, outputting the column of pixel data;
所述第一宏像素连接的第二级逻辑或门用于接收连接的多个第一级逻辑或门分别输出的列像素数据,将接收到的列像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive the column pixel data respectively output by the connected multiple first-level logical OR gates, and synthesize the received column pixel data to obtain the first macro pixel the first image data corresponding to the pixel, and output the first image data corresponding to the first macro pixel.
可选地,所述多路选择器包括多个输入端、一个输出端和一个控制端,所述多路选择器的多个输入端与所述开关阵列连接,所述多路选择器的控制端与所述处理单元的控制端连接,所述多路选择器的输出端与所述处理单元的输入端连接;Optionally, the multiplexer includes a plurality of input terminals, an output terminal and a control terminal, the multiple input terminals of the multiplexer are connected to the switch array, and the control of the multiplexer The terminal is connected to the control terminal of the processing unit, and the output terminal of the multiplexer is connected to the input terminal of the processing unit;
所述多路选择器的每个输入端用于接收所述开关阵列输出的一路第一图像数据;Each input terminal of the multiplexer is used to receive one channel of first image data output by the switch array;
所述多路选择器的控制端用于在每个测量时刻接收所述处理单元的控制端输出的选通信号;The control terminal of the multiplexer is used to receive the strobe signal output by the control terminal of the processing unit at each measurement moment;
所述多路选择器的输出端用于在每个测量时刻输出与相应测量时刻接收到的选通信号对应的一路第一图像数据。The output terminal of the multiplexer is used for outputting one path of first image data corresponding to the strobe signal received at the corresponding measurement moment at each measurement moment.
可选地,所述处理单元用于根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置,并将相应选通信号对应的第一图像数据作为确定出的宏像素的位置上的像素数据,以生成所述融合图像。Optionally, the processing unit is configured to determine the position of the macropixel corresponding to the first image data corresponding to the received corresponding strobe signal according to the strobe signal output at each measurement moment, and correspond the corresponding strobe signal to The first image data is used as the determined pixel data at the position of the macro pixel to generate the fused image.
可选地,所述多个宏像素中每相邻的两个宏像素具有(N-1)行重叠的SPAD子像素,或者,所述多个宏像素中每相邻的两个宏像素具有(M-1)列重叠的SPAD子像素。Optionally, every two adjacent macro pixels in the multiple macro pixels have (N-1) rows of overlapping SPAD sub-pixels, or, every two adjacent macro pixels in the multiple macro pixels have (M-1) columns of overlapping SPAD sub-pixels.
另一方面,提供了一种成像控制方法,所述方法应用于所述方法应用于成像装置的处理单元中,所述成像装置还包括成像单元,所述成像单元包括单光子雪崩二极管SPAD阵列、开关阵列和多路选择器,所述SPAD阵列包括多个宏像素,所述多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素,其中,所述开关阵列用于接收所述多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据至所述多路选择器;所述方法包括:In another aspect, an imaging control method is provided, the method is applied to a processing unit of an imaging device, the imaging device further includes an imaging unit, and the imaging unit includes a single photon avalanche diode SPAD array, A switch array and a multiplexer, the SPAD array includes a plurality of macro pixels, each macro pixel in the plurality of macro pixels includes (N×M) SPAD sub-pixels, and every two adjacent macro pixels There are overlapping SPAD sub-pixels between them, wherein the switch array is used to receive sub-pixel data respectively output by (N×M) SPAD sub-pixels included in each macro-pixel in the plurality of macro-pixels, for each Combining the sub-pixel data respectively output by the (N×M) SPAD sub-pixels included in the macro-pixel to obtain one path of first image data corresponding to the corresponding macro-pixel, and outputting one path of first image data corresponding to each macro-pixel to the multiple a way selector; the method comprising:
所述处理单元按照预设时序向所述多路选择器依次输出不同种类的选通信号,以使所述多路选择器在每接收到所述处理单元输出的一种选通信号时,输出与相应选通信号对应的第一图像数据;The processing unit sequentially outputs different types of strobe signals to the multiplexer according to a preset timing, so that the multiplexer outputs each time a strobe signal output by the processing unit is received. first image data corresponding to the corresponding strobe signal;
所述处理单元接收所述多路选择器依次输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。The processing unit receives the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.
可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,所述第一宏像素中的一行SPAD子像素连接一个第一级逻辑或门,第一宏像素中多行SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a row of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple A plurality of first-level logical OR gates connected to row SPAD sub-pixels are connected to a second-level logical OR gate, and the first macro pixel is any one of the plurality of macro pixels;
所述第一宏像素连接的每个第一级逻辑或门用于接收自身连接的一行SPAD子像素中每个SPAD子像素输出的子像素数据,并将接收到的子像素数据合成为一路行像素数据,输出所述行像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive the sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected to itself, and synthesize the received sub-pixel data into one row Pixel data, outputting the row of pixel data;
所述第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的行像素数据,将接收到的行像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and synthesize the received row pixel data to obtain the first the first image data corresponding to the macro pixels, and output the first image data corresponding to the first macro pixels.
可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a column of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro pixel A plurality of first-level logic OR gates connected to sub-pixels is connected to a second-level logic OR gate, and the first macro pixel is any one of the plurality of macro pixels;
所述第一宏像素连接的每个第一级逻辑或门用于接收连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,并将接收到的子像素数据合成为一路列像素数据,输出所述列像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive one path of sub-pixel data output by each SPAD sub-pixel in a connected column of SPAD sub-pixels, and synthesize the received sub-pixel data into one path of column Pixel data, outputting the column of pixel data;
所述第一宏像素连接的第二级逻辑或门用于接收连接的多个第一级逻辑或门分别输出的列像素数据,将接收到的列像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive the column pixel data respectively output by the connected multiple first-level logical OR gates, and synthesize the received column pixel data to obtain the first macro pixel the first image data corresponding to the pixel, and output the first image data corresponding to the first macro pixel.
可选地,所述多路选择器包括多个输入端、一个输出端和一个控制端,所述多路选择器的多个输入端与所述开关阵列连接,所述多路选择器的控制端与所述处理单元的控制端连接,所述多路选择器的输出端与所述处理单元的输入端连接,所述多路选择器的每个输入端用于接收所述开关阵列输出的一路第一图像数据;Optionally, the multiplexer includes a plurality of input terminals, an output terminal and a control terminal, the multiple input terminals of the multiplexer are connected to the switch array, and the control of the multiplexer The terminal is connected to the control terminal of the processing unit, the output terminal of the multiplexer is connected to the input terminal of the processing unit, and each input terminal of the multiplexer is used to receive the output of the switch array. One channel of first image data;
所述处理单元按照预设时序向所述多路选择器依次输出不同种类的选通信号,包括:The processing unit sequentially outputs different types of strobe signals to the multiplexer according to a preset timing sequence, including:
所述处理单元的控制端在第一测量时刻向所述多路选择器的控制端输出第一选通信号,以使所述多路选择器的输出端在所述第一测量时刻输出与所述第一选通信号对应的一路第一图像数据,所述第一测量时刻为多个测量时刻中的任一测量时刻。The control terminal of the processing unit outputs a first gating signal to the control terminal of the multiplexer at the first measurement moment, so that the output terminal of the multiplexer outputs the same signal at the first measurement moment. One path of first image data corresponding to the first gating signal, and the first measurement moment is any one of the plurality of measurement moments.
可选地,所述处理单元根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像,包括:Optionally, the processing unit generates a fused image according to the first image data corresponding to each macropixel and the position of the corresponding macropixel, including:
根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置;According to the gate signal output at each measurement moment, determine the position of the macro pixel corresponding to the first image data corresponding to the received corresponding gate signal;
将相应选通信号对应的的第一图像数据作为相应测量时刻确定的宏像素的位置上的像素数据,以生成所述融合图像。The first image data corresponding to the corresponding gate signal is used as the pixel data at the position of the macro pixel determined at the corresponding measurement moment, so as to generate the fused image.
可选地,所述多个宏像素中每相邻的两个宏像素具有(N-1)行重叠的SPAD子像素,或者,所述多个宏像素中每相邻的两个宏像素具有(M-1)列重叠的SPAD子像素。Optionally, every two adjacent macro pixels in the multiple macro pixels have (N-1) rows of overlapping SPAD sub-pixels, or, every two adjacent macro pixels in the multiple macro pixels have (M-1) columns of overlapping SPAD sub-pixels.
另一方面,提供了一种成像装置,所述装置包括:In another aspect, an imaging device is provided, the device comprising:
处理器;processor;
用于存储处理器可执行指令的存储器;memory for storing processor-executable instructions;
其中,所述处理器执行所述存储器中的可执行指令来执行上述成像控制方法。Wherein, the processor executes the executable instructions in the memory to implement the above imaging control method.
另一方面,提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述的成像控制方法的步骤。In another aspect, there is provided a computer program product containing instructions, which, when run on a computer, causes the computer to execute the steps of the above-mentioned imaging control method.
本申请实施例提供的技术方案带来的有益效果至少包括:The beneficial effects brought by the technical solutions provided by the embodiments of the present application at least include:
在本申请实施例中,通过成像单元中的开关阵列和多路选择器可以依次输出多个宏像素中每个宏像素对应的一路第一图像数据,处理单元接收每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。由于成像单元中SPAD阵列包括的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素,因此,通过开关阵列和多路选择器使得成像单元输出的宏像素对应的图像数据的数量增加,从而在SPAD子像素个数、后续存储不增加的情况下,提高了输出图像的平面分辨率。In the embodiment of the present application, through the switch array and the multiplexer in the imaging unit, one channel of first image data corresponding to each macro pixel in the plurality of macro pixels can be sequentially output, and the processing unit receives the first image data corresponding to each macro pixel. image data, and generate a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel. Since there are overlapping SPAD sub-pixels between every two adjacent macro-pixels in the multiple macro-pixels included in the SPAD array in the imaging unit, the image corresponding to the macro-pixels output by the imaging unit is made through the switch array and the multiplexer The amount of data increases, so that the planar resolution of the output image is improved without increasing the number of SPAD sub-pixels and subsequent storage.
附图说明Description of drawings
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.
图1是一种SPAD阵列的结构图;Fig. 1 is a structural diagram of a SPAD array;
图2是本申请实施例提供的一种成像装置所涉及的系统架构图;FIG. 2 is a system architecture diagram related to an imaging device provided by an embodiment of the present application;
图3是本申请实施例提供的一种成像装置的结构示意图;FIG. 3 is a schematic structural diagram of an imaging device provided in an embodiment of the present application;
图4是本申请实施例提供的一种SPAD阵列中宏像素的示意图;FIG. 4 is a schematic diagram of a macro pixel in a SPAD array provided by an embodiment of the present application;
图5是本申请实施例提供的一种宏像素与开关阵列之间连接关系的示意图;FIG. 5 is a schematic diagram of a connection relationship between a macro pixel and a switch array provided by an embodiment of the present application;
图6是本申请实施例提供的另一种宏像素与开关阵列之间连接关系的示意图;FIG. 6 is a schematic diagram of another connection relationship between a macro pixel and a switch array provided by an embodiment of the present application;
图7是本申请实施例提供的另一种成像装置的结构示意图;FIG. 7 is a schematic structural diagram of another imaging device provided by an embodiment of the present application;
图8是本申请实施例提供的一种成像装置生成的融合图像的示意图;FIG. 8 is a schematic diagram of a fused image generated by an imaging device provided in an embodiment of the present application;
图9是本申请实施例提供的一种成像控制方法的流程图。FIG. 9 is a flowchart of an imaging control method provided by an embodiment of the present application.
具体实施方式Detailed ways
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.
在对本申请实施例进行详细的解释说明之前,先对本申请实施例涉及的系统架构进行介绍。Before explaining the embodiment of the present application in detail, the system architecture involved in the embodiment of the present application is firstly introduced.
图2是本申请实施例提供的一种成像装置所涉及的系统架构图。如图2所示,该系统包括成像装置10、路由器20和服务器30。成像装置10和路由器20之间相连,路由器20和服务器30之间相连。其中,成像装置10用于生成图像,并将生成的图像输出至路由器20;路由器20用于传输成像装置10生成的图像至服务器30;服务器30用于接收路由器20传输的图像,并将接收到的图像进行存储。FIG. 2 is a system architecture diagram related to an imaging device provided by an embodiment of the present application. As shown in FIG. 2 , the system includes an
其中,成像装置10包括处理单元101和成像单元102。该成像单元102包括SPAD阵列。其中,该SPAD阵列包括多个宏像素,每个宏像素包括多个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素。该成像单元102用于输出每个宏像素对应的第一图像数据,处理单元101用于接收成像单元102输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。需要说明的是,该成像单元102的详细实现方式可以参见下文中的介绍,在此不再赘述。Wherein, the
除此之外,该成像装置10还可以包括激光器驱动103、激光器104、发射镜头105和接收镜头106。In addition, the
激光驱动器103可以分别与成像单元102和激光器104连接。成像单元102可以向激光器驱动103发送发射开始指令,并在发送发射开始指令的同时开始计时。激光器驱动103在接收到发射开始指令后,驱动激光器104发出激光。The
发射镜头105用于将激光器104发出的激光发射出去,照射到目标物体上。发射镜头105可以固定于激光器104之上,也可以装设于激光器104之内。The emitting
接收镜头106用于接收目标物体反射的光,进而将接收到的反射光发射到成像单元102中的SPAD阵列的各个SPAD子像素上,以便成像单元102中的SPAD阵列的各个SPAD子像素根据探测到的反射光信号输出子像素数据,进而使得成像单元102根据各个子像素数据输出每个宏像素对应的第一图像数据。接收镜头106可以固定于成像单元102中的SPAD阵列之上,也可以装设于SPAD阵列102之内。The receiving
可选地,在一些可能的实现方式中,该成像装置10在得到融合图像之后,还可以基于该融合图像进行智能分析,之后,将该融合图像和智能分析结果经过路由器20一起发送至服务器30,以便服务器30对该融合图像和智能分析结果进行存储,供后续其他业务进行使用。Optionally, in some possible implementations, after the
接下来对本申请实施例提供的成像装置进行介绍。Next, the imaging device provided by the embodiment of the present application will be introduced.
图3是本申请实施例提供的一种成像装置的示意图。如图3所示,该装置包括:成像单元301和处理单元302,该成像单元301包括SPAD阵列3011、开关阵列3012和多路选择器3013,该SPAD阵列3011包括多个宏像素,多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素。其中,开关阵列3012用于接收多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据;多路选择器3013用于接收开关阵列3012输出的多路第一图像数据,并将多路第一图像数据依次输出至处理单元302;处理单元302用于接收SPAD阵列输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。FIG. 3 is a schematic diagram of an imaging device provided by an embodiment of the present application. As shown in Figure 3, the device includes: an imaging unit 301 and a
在本申请实施例中,SPAD阵列3011中包含有呈矩形阵列排布的多个SPAD子像素,可以将SPAD阵列3011中(N×M)个SPAD子像素组成一个宏像素。则该SPAD阵列3011中包含有多个由(N×M)个SPAD子像素组成的宏像素。其中,N为宏像素的行数,M为宏像素的列数,且N与M可以相等,也可以不相等,本申请实施例对此不作限定。In the embodiment of the present application, the
需要说明的是,在本申请实施例中,SPAD阵列3011中的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素。It should be noted that, in the embodiment of the present application, among the plurality of macro-pixels in the
示例性地,当各个宏像素包括(N×M)个SPAD子像素时,上下相邻的两个宏像素可以具有多行重叠的SPAD子像素,左右相邻的两个宏像素可以具有多列重叠的SPAD子像素。例如,上下相邻的两个宏像素可以具有(N-1)行重叠的SPAD子像素,也可以具有(N-2)行重叠的SPAD子像素重叠,本申请实施例对此不作限定。左右相邻的两个宏像素具有(M-1)列重叠的SPAD子像素,也可以具有(M-2)列重叠的SPAD子像素,本申请实施例对此不做限定。Exemplarily, when each macro pixel includes (N×M) SPAD sub-pixels, two adjacent macro pixels up and down may have multiple rows of overlapping SPAD sub-pixels, and two macro pixels adjacent to left and right may have multiple columns Overlapping SPAD subpixels. For example, two vertically adjacent macro pixels may have (N-1) rows of overlapping SPAD sub-pixels, or may have (N-2) rows of overlapping SPAD sub-pixels, which is not limited in this embodiment of the present application. Two adjacent left and right macro pixels have (M-1) columns of overlapping SPAD sub-pixels, and may also have (M-2) columns of overlapping SPAD sub-pixels, which is not limited in this embodiment of the present application.
例如,参见图4,一个4×4共有16个SPAD子像素的SPAD阵列,以N=M=3为例,一个宏像素包括3×3个SPAD子像素。其中,上下相邻的宏像素1和宏像素2之间重叠2行,第一行重叠的SPAD子像素包括SPAD5~SPAD7,第二行重叠的SPAD子像素包括SPAD9~SPAD11;左右相邻的宏像素2和宏像素3重叠2列,第一列重叠的SPAD子像素包括SPAD6、SPAD10、SPAD14,第二列重叠的SPAD子像素包括SPAD7、SPAD11、SPAD15。For example, referring to FIG. 4 , a 4×4 SPAD array with 16 SPAD sub-pixels in total, taking N=M=3 as an example, a macro pixel includes 3×3 SPAD sub-pixels. Among them, 2 rows overlap between the upper and lower adjacent
在本申请实施例中,开关阵列3012包括第一级逻辑或门和第二级逻辑或门。在一种可能的实现方式中,以SPAD阵列3011中多个宏像素中任一宏像素为例,将其称为第一宏像素,该第一宏像素中的每一行SPAD子像素分别连接一个第一级逻辑或门,每个宏像素中多行SPAD子像素对应的多个第一级逻辑或门连接一个第二级逻辑或门。In the embodiment of the present application, the
其中,该第一宏像素连接的每个第一级逻辑或门用于接收连接的一行SPAD子像素中每个SPAD子像素输出的一路子像素数据,之后,第一级逻辑或门将接收到的子像素数据进行合成,得到一路行像素数据,输出该行像素数据。例如,对于一个由(N×M)个SPAD子像素组成的宏像素,该宏像素连接N个第一级逻辑或门,每个第一级逻辑或门将输出一个行像素数据,这样,N个第一级逻辑或门将输出N个行像素数据。Wherein, each first-level logical OR gate connected to the first macro-pixel is used to receive one path of sub-pixel data output by each SPAD sub-pixel in the connected row of SPAD sub-pixels, and then the first-level logical OR gate will receive the The sub-pixel data is synthesized to obtain a row of pixel data, and the row of pixel data is output. For example, for a macro pixel composed of (N×M) SPAD sub-pixels, the macro pixel is connected with N first-level logical OR gates, and each first-level logical OR gate will output a row of pixel data, so that N The first logical OR gate will output N rows of pixel data.
第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的行像素数据,之后,第二级逻辑或门将接收到的多路行像素数据进行合成,得到一路第一图像数据,并输出该第一图像数据。The second-level logical OR gate connected to the first macro-pixel is used to receive row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and then, the second-level logical OR gate performs multi-channel row pixel data received by the second-level logical OR gate. Synthesize to obtain a channel of first image data, and output the first image data.
需要说明的是,在一种可能的情况中,当第一宏像素包括4个SPAD子像素时,则一个第一级逻辑或门可以为一个二输入或门。相应地,一个第二级逻辑或门也可以为一个二输入或门。It should be noted that, in a possible situation, when the first macro-pixel includes 4 SPAD sub-pixels, a first-level logical OR gate may be a two-input OR gate. Correspondingly, a second-level logical OR gate can also be a two-input OR gate.
例如,参见图5,假设SPAD阵列包括3×3个SPAD子像素,SPAD1、SPAD2、SPAD4、SPAD5组成宏像素1,该宏像素1中的第一行的SPAD1和SPAD2连接一个第一级逻辑或门,该第一级逻辑或门为一个二输入或门。在这种情况下,该第一级逻辑或门接收SPAD1和SPAD2分别输出的子像素数据,并将这两路子像素数据合成为一路行像素数据1+2;同理,第二行的SPAD4和SPAD5连接一个第一级逻辑或门,该第一级逻辑或门接收SPAD4和SPAD5分别输出的子像素数据,并将这两路子像素数据合成为一路行像素数据4+5。For example, referring to FIG. 5 , assuming that the SPAD array includes 3×3 SPAD sub-pixels, SPAD1, SPAD2, SPAD4, and SPAD5 form a
进一步地,宏像素1的两行SPAD子像素对应的两个第一级逻辑或门连接一个第二级逻辑或门,该第二逻辑或门也为一个二输入或门,这样,该第二级逻辑或门接收这两个第一级逻辑或门输出的行像素数据1+2和行像素数据4+5,将行像素数据1+2和行像素数据4+5合成为一路第一图像数据1245,并输出第一图像数据1245。Further, the two first-level logical OR gates corresponding to the two rows of SPAD sub-pixels of the
在另一种可能的情况中,当第一宏像素包括X个SPAD子像素,且X大于4时,则第一宏像素连接的每个第一级逻辑或门可能包括一个或多个二输入或门,第一宏像素连接的第二逻辑或门也可能包括一个或多个二输入或门。In another possible situation, when the first macro pixel includes X SPAD sub-pixels, and X is greater than 4, each first-level logical OR gate connected to the first macro pixel may include one or more two-input The OR gate, the second logical OR gate connected to the first macro pixel may also include one or more two-input OR gates.
例如,当X为9时,根据R=ceiling(log2X)计算得到R=4,其中,ceiling(*)是指向上取整。在这种情况下,该开关阵列将包括4层二输入或门,其中,每个第一级逻辑或门可以由两层二输入或门组成,每个第二级逻辑或门也可以由两层二输入或门组成。For example, when X is 9, R=4 is calculated according to R=ceiling(log 2 X), where ceiling(*) refers to rounding up. In this case, the switch array will include 4 layers of two-input OR gates, wherein each first-level logical OR gate can be composed of two layers of two-input OR gates, and each second-level logical OR gate can also be composed of two layers of two-input OR gates.
例如,参见图6,假设SPAD阵列包括4×4个SPAD子像素,SPAD1、SPAD2、SPAD3、SPAD5、SPAD6、SPAD7、SPAD9、SPAD10以及SPAD11组成宏像素1,在这种情况下,该宏像素1中的第一行中每相邻的两个SPAD子像素可以连接一个二输入或门,这样,这2个二输入或门即为该宏像素1的第一行SPAD子像素连接的第一级逻辑或门中的第一层二输入或门。在此基础上,该第一级逻辑或门中相邻的第一层输入或门连接一个二输入或门,这样,这个二输入或门即为第一行SPAD子像素连接的第一级逻辑或门中的第二层输入或门。这样,宏像素1的各行SPAD子像素连接的第一级逻辑或门中每个第一层二输入或门接收连接的两个SPAD子像素输出的子像素数据,并将两路子像素数据进行合并后输出。每个第二层二输入或门接收连接的两个第一层输入或门输出的两路数据,并将接收到的两路数据进行合并,以得到一路行像素数据。For example, referring to FIG. 6 , assuming that the SPAD array includes 4×4 SPAD sub-pixels, SPAD1, SPAD2, SPAD3, SPAD5, SPAD6, SPAD7, SPAD9, SPAD10 and SPAD11 form a
在通过上述方法得到宏像素中的每行SPAD子像素对应的行像素数据之后,对于这3个行像素数据,可以将输出相邻两行的行像素数据的两个第二层二输入或门连接一个二输入或门,此时,连接的这个二输入或门即为第二级逻辑或门中的第一层二输入或门。之后,第二级逻辑或门中的两个第一层二输入或门再连接一个二输入或门,此时,这个二输入或门即为第二级逻辑或门中的第二层二输入或门。这样,第二级逻辑或门中的每个第一层二输入或门可以将接收到的相邻两行的行像素数据进行合并,输出一路数据,之后,第二层二输入或门将两个第一层二输入或门输入的两路数据进行合并,从而得到第一图像数据。After the row pixel data corresponding to each row of SPAD sub-pixels in the macro pixel is obtained by the above method, for these three rows of pixel data, the two second-layer two-input OR gates that output the row pixel data of two adjacent rows can be A two-input OR gate is connected. At this time, the connected two-input OR gate is the first layer two-input OR gate in the second-level logic OR gate. After that, the two first-level two-input OR gates in the second-level logical OR gate are connected to a two-input OR gate. At this time, this two-input OR gate is the second-level two-input OR gate in the second-level logical OR gate. OR gate. In this way, each first-layer two-input OR gate in the second-level logical OR gate can combine the received row pixel data of two adjacent rows, and output one road of data, after that, the second layer two-input OR gate combines two The two channels of data input by the two-input OR gate of the first layer are combined to obtain the first image data.
上述仅是以宏像素包括9个SPAD子像素为例进行示例性说明,当宏像素包括8个、6个或者更多SPAD子像素,可以参考上述方法进行处理,本申请实施例在此不再赘述。The above is only an example of a macro pixel including 9 SPAD sub-pixels. When a macro pixel includes 8, 6 or more SPAD sub-pixels, the above method can be referred to for processing. repeat.
另外,对于多个宏像素中的每个宏像素,其均可以参考上述第一宏像素,连接多个第一级逻辑或门以及一个第二级逻辑或门。这样,该开关阵列3012中将包括多个第一级逻辑或门和多个第二级逻辑或门。In addition, for each macro pixel among the multiple macro pixels, it can refer to the above-mentioned first macro pixel, and connect multiple first-level logical OR gates and one second-level logical OR gate. In this way, the
可选地,在另一种可能的实现方式中,仍以第一宏像素为例,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门。Optionally, in another possible implementation, still taking the first macro pixel as an example, a column of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro pixel Multiple first-level logical OR gates connected to the sub-pixels are connected to one second-level logical OR gate.
其中,第一宏像素连接的每个第一级逻辑或门可以用于接收自身连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,之后,第一级逻辑或门将接收到的子像素数据进行合成,得到一路列像素数据,输出该列像素数据。例如,在一个(N×M)个SPAD子像素组成的一个宏像素,该宏像素连接M个第一级逻辑或门,每个第一级逻辑或门将输出一个列像素数据,这样,M个第一级逻辑或门将输出M个列像素数据。Wherein, each first-level logical OR gate connected to the first macro-pixel can be used to receive one path of sub-pixel data output by each SPAD sub-pixel in a column of SPAD sub-pixels connected to itself, and then the first-level logical OR gate will receive Synthesize the sub-pixel data to obtain a column of pixel data, and output the column of pixel data. For example, in a macro pixel composed of (N×M) SPAD sub-pixels, the macro pixel is connected with M first-level logical OR gates, and each first-level logical OR gate will output a column of pixel data, so that M The first level logic OR gate will output M columns of pixel data.
第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的列像素数据,之后,第二级逻辑或门将接收到的列像素数据进行合成,得到一路第一图像数据,并输出第一图像数据。The second level logical OR gate connected to the first macro pixel is used to receive column pixel data respectively output by multiple first level logical OR gates connected to itself, and then the second level logical OR gate synthesizes the received column pixel data, Obtain one channel of first image data, and output the first image data.
需要说明的是,当宏像素包括的SPAD子像素为4时,每个第一级逻辑或门和第二级逻辑或门均可以通过一个二输入或门来实现。可选地,当宏像素包括更多的SPAD子像素时,则可以参考前文中介绍的方法,每个第一级逻辑或门通过一个或多个二输入或门来实现,每个第二级逻辑或门也通过一个或多个二输入或门来实现,本申请实施例在此不再赘述。It should be noted that when the number of SPAD sub-pixels included in the macro-pixel is 4, each of the first-level logic OR gate and the second-level logic OR gate can be realized by a two-input OR gate. Optionally, when the macro-pixel includes more SPAD sub-pixels, you can refer to the method introduced above, each first-level logical OR gate is realized by one or more two-input OR gates, each second-level The logical OR gate is also implemented by one or more two-input OR gates, which will not be repeated in this embodiment of the present application.
开关阵列3012中的每个第二级逻辑或门在输出一路第一图像数据之后,多路选择器3013接收多个第二级逻辑或门分别输出的一路第一图像数据,并将接收到的多路第一图像数据依次输出至处理单元302。After each second-level logical OR gate in the
在本申请实施例中,多路选择器3013包括多个输入端、一个输出端和一个控制端,多路选择器3013的多个输入端与开关阵列3012连接,多路选择器3013的控制端与处理单元302的控制端连接,多路选择器3013的输出端与处理单元302的输入端连接。In the embodiment of the present application, the
其中,多路选择器3013的每个输入端用于接收开关阵列3012输出的一路第一图像数据;多路选择器3013的控制端用于在每个测量时刻接收处理单元302的控制端输出的选通信号;多路选择器3013的输出端用于在每个测量时刻输出与相应测量时刻接收到的选通信号对应的一路第一图像数据。Wherein, each input terminal of the
需要说明的是,多路选择器3013的输入端的数量与开关阵列3012包括的第二级逻辑或门的数量相同,这样,多路选择器3013的每个输入端连接开关阵列3012中的一个第二级逻辑或门,以接收连接的第二级逻辑或门输出的第一图像数据。It should be noted that the number of input terminals of the
其中,在一种可能的实现方式中,可以按照开关阵列3012中的每个第二级逻辑或门输出的第一图像数据所对应的宏像素在SPAD阵列3011中的位置,将多路选择器3013中的各个输入端依次与各个第二级逻辑或门进行连接。例如,用于输出SPAD阵列3011的多个宏像素中的第一行第一列的宏像素对应的第一图像数据的第二级逻辑或门可以与多路选择器3013中的第一个输入端连接,用于输出SPAD阵列3011的多个宏像素中的第一行第二列的宏像素对应的第一图像数据的第二级逻辑或门可以与多路选择器3013中的第二个输入端连接,以此类推。Wherein, in a possible implementation manner, according to the position in the
在接收多路第一图像数据的同时,多路选择器3013的控制端还可以接收处理单元302的控制端按照预设时序输出的选通信号。其中,处理单元302可以按照SPAD阵列3011中的多个宏像素的位置以及多路选择器3013中用于接收每个宏像素对应的第一图像数据的输入端,在不同的测量时刻通过自身的控制端依次输出用于选通多路选择器中的不同输入端的选通信号。相应地,多路选择器3013可以通过自身的控制端在每个测量时刻接收处理单元302的控制端输出的选通信号,并根据该选通信号输出对应的输入端输入的第一图像数据。While receiving multiple channels of first image data, the control terminal of the
例如,处理单元确定SPAD阵列中第一行第一列的宏像素1对应的第一图像数据将输入至多路选择器的输入端1,第一行第二列的宏像素2对应的第一图像数据将输入至多路选择器的输入端2,第二行第一列的宏像素3对应的第一图像数据将输入至多路选择器的输入端3,第二行第二列的宏像素4对应的第一图像数据将输入至多路选择器的输入端4,在此基础上,在第一个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端1的选通信号0000。多路选择器的控制端在接收到选通信号0000之后,将多路选择器的输入端1输入的宏像素1的第一图像数据输出至处理单元。在第二个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端2的选通信号0001。多路选择器的控制端在接收到选通信号0001之后,将多路选择器的输入端2输入的宏像素2的第一图像数据输出至处理单元。在第三个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端3的选通信号0010,多路选择器的控制端在接收到选通信号0010之后,将多路选择器的输入端3输入的宏像素3的第一图像数据输出至处理单元。在第四个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端4的选通信号0011,多路选择器的控制端在接收到选通信号0011之后,将多路选择器的输入端4输入的宏像素4的第一图像数据输出至处理单元。For example, the processing unit determines that the first image data corresponding to the
处理单元302在接收到每个选通信号对应的第一图像数据之后,可以根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置,并将相应选通信号对应的第一图像数据作为确定出的宏像素的位置上的像素数据,以生成融合图像。After receiving the first image data corresponding to each gate signal, the
需要说明的是,上述相邻的两个测量时刻之间可以间隔一定时长,在该时长内,处理单元302可以对接收到的第一图像数据进行处理。It should be noted that there may be a certain period of time between the above two adjacent measurement moments, and within this period of time, the
示例性地,处理单元302在某个测量时刻通过自身控制端输出选通信号之后,可以接收多路选择器3013根据该选通信号输出的第一图像数据。由前述介绍可知,处理单元302在发送选通信号时是根据宏像素的位置和多路选择器3013中用于输出相应宏像素的图像数据的输入端来发送的,基于此,处理单元302可以根据在该测量时刻输出的选通信号所对应的宏像素的位置,确定接收到的第一图像数据为哪个宏像素对应的图像数据,进而将该第一图像数据作为该宏像素的位置上的像素数据。之后,处理单元302可以在下一个测量时刻到达之后,继续通过控制端向多路选择器3013发送选通信号,并对接收到的相应选通信号对应的第一图像数据进行处理。如此,处理单元302通过接收并处理多路选择器依次输出的多路第一图像数据,得到各个宏像素的位置上的像素数据,从而生成融合图像。Exemplarily, after the
为了更好的理解本申请实施例的装置,现以一个3×3的SPAD阵列为例对本申请实施例提供的成像装置进行详细说明,其中,参见图7,该SPAD阵列包括9个SPAD子像素,分别为SPAD1至SPAD9。另外,该SPAD阵列中的每个宏像素包括2×2个SPAD子像素。其中,SPAD1、SPAD2、SPAD4、SPAD5组成宏像素1,SPAD2、SPAD3、SPAD5、SPAD6组成宏像素2,SPAD4、SPAD5、SPAD7、SPAD8组成宏像素3,SPAD5、SPAD6、SPAD8、SPAD9组成宏像素4。宏像素1和宏像素2中重叠的SPAD子像素为SPAD2和SPAD5,宏像素3和宏像素4中重叠的SPAD子像素为SPAD5和SPAD8,宏像素1和宏像素3中重叠的SPAD子像素为SPAD4和SPAD5,宏像素2和宏像素4中重叠的SPAD子像素为SPAD5和SPAD6。In order to better understand the device of the embodiment of the present application, a 3×3 SPAD array is taken as an example to describe the imaging device provided by the embodiment of the present application in detail, wherein, referring to FIG. 7, the SPAD array includes 9 SPAD sub-pixels , SPAD1 to SPAD9, respectively. In addition, each macro pixel in the SPAD array includes 2×2 SPAD sub-pixels. Among them, SPAD1, SPAD2, SPAD4, SPAD5 form
上述4个宏像素中每个宏像素的一行SPAD子像素连接一个第一级逻辑或门,这样,每个宏像素的一行SPAD子像素中的两个SPAD子像素将输出两路子像素数据至连接的第一级逻辑或门,例如,宏像素1的第一行的SPAD1、SPAD2子像素连接一个第一级逻辑或门,这样,SPAD1输出子像素数据1至连接的第一级逻辑或门,SPDA2输出子像素数据2至连接的第一级逻辑或门,该第一级逻辑或门在接收到子像素数据1和2之后,将两路子像素数据合成为一路行像素数据1+2,并输出该行像素数据1+2。对于其他宏像素中的各行SPAD子像素,每行SPAD子像素均连接有一个第一级逻辑或门,这样,通过6个第一级逻辑或门,将输出6个行像素数据。另外,每个宏像素中的多行SPAD子像素连接的多个第一级逻辑或门将连接一个第二级逻辑或门,例如,宏像素1的第一行的SPAD1和SPAD2两个子像素连接的第一级逻辑或门以及第二行的SPAD4和SPAD5两个子像素连接的第一级逻辑或门将连接一个第二级逻辑或门,这样,这个第二级逻辑或门将接收到行像素数据1+2和行像素数据4+5,之后,该第二级逻辑或门将行像素数据1+2和行像素数据4+5合成为第一图像数据1245。对于其他宏像素的各行SPAD子像素连接的第一级逻辑或门,也均可以连接一个第二级逻辑或门,如此,每个第二级逻辑或门可以连接一个宏像素中的两行SPAD子像素对应的两个第一级逻辑或门,并将连接的两个第一级逻辑或门输出的两个行像素数据合成为一路第一图像数据,从而得到四路第一图像数据,分别为第一图像数据1245、第一图像数据4578、第一图像数据2356和第一图像数据5689。A row of SPAD subpixels of each of the above four macropixels is connected to a first-level logical OR gate, so that two SPAD subpixels in a row of SPAD subpixels of each macropixel will output two channels of subpixel data to the connection For example, the SPAD1 and SPAD2 sub-pixels of the first row of the
每个第二级逻辑或门可以将自身得到的一路第一图像数据输出至多路选择器,多路选择器可以根据接收到的选通信号按顺序依次将各路第一图像数据输出至处理单元。Each second-level logical OR gate can output one channel of first image data obtained by itself to the multiplexer, and the multiplexer can sequentially output each channel of first image data to the processing unit according to the received strobe signal .
其中,宏像素1对应的第二级逻辑或门可以连接多路选择器的输入端1,宏像素2对应的第二级逻辑或门可以连接多路选择器的输入端2,宏像素3对应的第二级逻辑或门连接多路选择器的输入端3,宏像素4对应的第二级逻辑或门连接多路选择器的输入端4。相应地,处理单元的控制端可以依次输出选通信号00、01、10和11。多路选择器在接收到选通信号00时,输出第一图像数据1245,处理单元在接收到第一图像数据1245之后,将第一图像数据1245作为宏像素1所在位置处的像素数据。在接收到选通信号01时,输出第一图像数据2356,处理单元在接收到第一图像数据2356之后,将第一图像数据2356作为宏像素2所在位置处的像素数据。以此类推,最终得到的融合图像如图8所示。Among them, the second level logic OR gate corresponding to
在本申请实施例中,通过成像单元中的开关阵列和多路选择器可以依次输出多个宏像素中每个宏像素对应的一路第一图像数据,处理单元接收每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。由于成像单元中SPAD阵列包括的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素,因此,通过开关阵列和多路选择器使得成像单元输出的宏像素的图像数据的数量增加,而该SPAD阵列的TDC和直方图计算电路均未增加,由此可见,本申请实施例提供的图像处理方法能够在SPAD子像素个数、后续存储、主要电路面积不增加的情况下,提高了SPAD阵列的输出图像的平面分辨率。In the embodiment of the present application, through the switch array and the multiplexer in the imaging unit, one channel of first image data corresponding to each macro pixel in the plurality of macro pixels can be sequentially output, and the processing unit receives the first image data corresponding to each macro pixel. image data, and generate a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel. Since there are overlapping SPAD sub-pixels between every two adjacent macro-pixels in the multiple macro-pixels included in the SPAD array in the imaging unit, the image data of the macro-pixels output by the imaging unit are output by the switch array and the multiplexer. The number increases, but the TDC and histogram calculation circuits of the SPAD array do not increase. It can be seen that the image processing method provided by the embodiment of the present application can be used without increasing the number of SPAD sub-pixels, subsequent storage, and main circuit area. Next, the planar resolution of the output image of the SPAD array is improved.
接下来,对本申请实施例提供的成像方法进行介绍。Next, the imaging method provided in the embodiment of the present application is introduced.
图9是本申请实施例提供的一种成像控制方法的流程图,该方法应用于前述实施例介绍的成像装置的处理单元中,该成像装置还包括成像单元,成像单元包括单光子雪崩二极管SPAD阵列,SPAD阵列包括多个宏像素,多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素,其中,开关阵列用于接收多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据至多路选择器,该方法包括以下步骤:Fig. 9 is a flow chart of an imaging control method provided by an embodiment of the present application, the method is applied to the processing unit of the imaging device introduced in the foregoing embodiments, the imaging device also includes an imaging unit, and the imaging unit includes a single photon avalanche diode SPAD Array, the SPAD array includes a plurality of macro pixels, each macro pixel in the plurality of macro pixels includes (N×M) SPAD sub-pixels, and there are overlapping SPAD sub-pixels between every two adjacent macro pixels, wherein , the switch array is used to receive sub-pixel data respectively output by (N×M) SPAD sub-pixels included in each macro-pixel in a plurality of macro-pixels, and the (N×M) SPAD sub-pixels included in each macro-pixel are respectively The output sub-pixel data is synthesized to obtain one path of first image data corresponding to the corresponding macro pixel, and one path of first image data corresponding to each macro pixel is output to the multiplexer. The method includes the following steps:
步骤901:处理单元按照预设时序向多路选择器依次输出不同种类的选通信号,以使多路选择器在每接收到处理单元输出的一种选通信号时,输出与相应选通信号对应的第一图像数据。Step 901: The processing unit sequentially outputs different types of strobe signals to the multiplexer according to the preset timing, so that the multiplexer outputs a corresponding strobe signal every time it receives a strobe signal output by the processing unit. corresponding to the first image data.
其中,处理单元按照预设时序依次输出不同种类的选通信号的实现过程可以参考前述实施例中介绍的实现方式,本申请实施例在此不再赘述。Wherein, the implementation process of the processing unit sequentially outputting different types of strobe signals according to the preset time sequence can refer to the implementation manners introduced in the foregoing embodiments, and the embodiments of the present application will not repeat them here.
步骤902:处理单元接收多路选择器依次输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。Step 902: The processing unit receives the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.
其中,多路选择器根据处理单元发送的选通信号输出对应的第一图像数据的过程可以参考前述实施例中的实现方式。另外,处理单元生成融合图像的过程也可以参考前述实现方式,本申请实施例在此不再赘述。Wherein, the process of outputting the corresponding first image data by the multiplexer according to the strobe signal sent by the processing unit may refer to the implementation manners in the foregoing embodiments. In addition, the process of generating the fused image by the processing unit may also refer to the foregoing implementation manners, which will not be repeated in this embodiment of the present application.
可选地,开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一行SPAD子像素连接一个第一级逻辑或门,第一宏像素中多行SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,第一宏像素为多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a row of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple rows of SPAD sub-pixels in the first macro pixel A plurality of connected first-level logical OR gates are connected to a second-level logical OR gate, and the first macro pixel is any one of the plurality of macro pixels;
其中,第一宏像素连接的每个第一级逻辑或门接收自身连接的一行SPAD子像素中每个SPAD子像素输出的一路子像素数据,将接收到的子像素数据合成为一路行像素数据,输出行像素数据;Wherein, each first-stage logical OR gate connected to the first macro pixel receives a road of sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected to itself, and synthesizes the received sub-pixel data into a road of pixel data , output row pixel data;
第一宏像素连接的第二级逻辑或门接收自身连接的多个第一级逻辑或门分别输出的行像素数据,将接收到的行像素数据进行合成,得到第一宏像素对应的第一图像数据,输出第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel receives row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and synthesizes the received row pixel data to obtain the first pixel corresponding to the first macro pixel. Image data, outputting first image data corresponding to the first macro pixel.
可选地,开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,第一宏像素为多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a column of SPAD sub-pixels in the first macro-pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro-pixel A plurality of connected first-level logical OR gates are connected to a second-level logical OR gate, and the first macro pixel is any one of the plurality of macro pixels;
第一宏像素连接的每个第一级逻辑或门用于接收连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,并将接收到的子像素数据合成为一路列像素数据,输出列像素数据;Each first-level logical OR gate connected to the first macro-pixel is used to receive a channel of sub-pixel data output by each SPAD sub-pixel in a connected column of SPAD sub-pixels, and synthesize the received sub-pixel data into a column of pixel data , output column pixel data;
第一宏像素连接的第二级逻辑或门用于接收连接的多个第一级逻辑或门分别输出的列像素数据,将接收到的列像素数据进行合成,得到第一宏像素对应的第一图像数据,输出第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive column pixel data respectively output by the connected multiple first-level logical OR gates, and synthesize the received column pixel data to obtain the first macro pixel corresponding to the first macro pixel. Image data, outputting the first image data corresponding to the first macro pixel.
可选地,多路选择器包括多个输入端、一个输出端和一个控制端,多路选择器的多个输入端与开关阵列连接,多路选择器的控制端与处理单元的控制端连接,多路选择器的输出端与处理单元的输入端连接,多路选择器的每个输入端接收开关阵列输出的一路第一图像数据;Optionally, the multiplexer includes multiple input terminals, an output terminal and a control terminal, multiple input terminals of the multiplexer are connected to the switch array, and the control terminal of the multiplexer is connected to the control terminal of the processing unit , the output end of the multiplexer is connected to the input end of the processing unit, and each input end of the multiplexer receives one path of first image data output by the switch array;
处理单元按照预设时序向多路选择器依次输出不同种类的选通信号,包括:The processing unit sequentially outputs different types of strobe signals to the multiplexer according to the preset timing, including:
处理单元的控制端在第一测量时刻向多路选择器的控制端输出的第一选通信号,以使多路选择器的输出端在第一测量时刻输出与第一选通信号对应的一路第一图像数据,第一测量时刻为多个测量时刻中的任一测量时刻。The control terminal of the processing unit outputs the first strobe signal to the control terminal of the multiplexer at the first measurement moment, so that the output terminal of the multiplexer outputs one channel corresponding to the first strobe signal at the first measurement moment. For the first image data, the first measurement moment is any one of the plurality of measurement moments.
可选地,处理单元根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像,包括:Optionally, the processing unit generates a fused image according to the first image data corresponding to each macropixel and the position of the corresponding macropixel, including:
根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置;According to the gate signal output at each measurement moment, determine the position of the macro pixel corresponding to the first image data corresponding to the received corresponding gate signal;
并将相应选通信号对应的第一图像数据作为相应测量时刻确定出的宏像素的位置上的像素值,以生成融合图像。The first image data corresponding to the corresponding gate signal is used as the pixel value at the position of the macro pixel determined at the corresponding measurement time, so as to generate a fused image.
可选地,多个宏像素中每相邻的两个宏像素具有(N-1)行重叠的SPAD子像素,或者,多个宏像素中每相邻的两个宏像素具有(M-1)列重叠的SPAD子像素。Optionally, every two adjacent macro pixels in the multiple macro pixels have (N-1) rows of overlapping SPAD sub-pixels, or, every two adjacent macro pixels in the multiple macro pixels have (M-1 ) columns of overlapping SPAD sub-pixels.
需要说明的是,上述各个步骤的实现方式可以参考前述实施例中关于成像装置的实现方式,本申请实施例在此不再赘述。It should be noted that, for implementation manners of the foregoing steps, reference may be made to the implementation manners of the imaging device in the foregoing embodiments, and details are not repeated in this embodiment of the present application.
综上所述,在本申请实施例中,通过成像单元中的开关阵列和多路选择器可以依次输出多个宏像素中每个宏像素对应的一路第一图像数据,处理单元接收每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。由于成像单元中SPAD阵列包括的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素,因此,通过开关阵列和多路选择器使得成像单元输出的宏像素的图像数据的数量增加,而该SPAD阵列的TDC和直方图计算电路均未增加,由此可见,本申请实施例提供的图像处理方法能够在SPAD子像素个数、后续存储、主要电路面积不增加的情况下,提高了SPAD阵列的输出图像的平面分辨率。To sum up, in the embodiment of the present application, through the switch array and multiplexer in the imaging unit, one channel of first image data corresponding to each macro pixel among the multiple macro pixels can be sequentially output, and the processing unit receives each macro pixel The pixel corresponds to the first image data, and according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel, a fused image is generated. Since there are overlapping SPAD sub-pixels between every two adjacent macro-pixels in the multiple macro-pixels included in the SPAD array in the imaging unit, the image data of the macro-pixels output by the imaging unit are output by the switch array and the multiplexer. The number increases, but the TDC and histogram calculation circuits of the SPAD array do not increase. It can be seen that the image processing method provided by the embodiment of the present application can be used without increasing the number of SPAD sub-pixels, subsequent storage, and main circuit area. Next, the planar resolution of the output image of the SPAD array is improved.
本申请实施例还提供了一种计算机可读存储介质,该存储介质内存储有计算机程序,该计算机程序被前述成像装置执行时可以实现图9所示的成像控制方法。The embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by the aforementioned imaging device, the imaging control method shown in FIG. 9 can be realized.
本申请实施例还提供了一种包含指令的计算机程序产品,当其在前述的成像装置上运行时,使得成像装置执行上述图9所示实施例提供的成像控制方法。The embodiment of the present application also provides a computer program product including instructions, which, when running on the aforementioned imaging device, causes the imaging device to execute the imaging control method provided in the embodiment shown in FIG. 9 above.
以上所述并不用以限制本申请实施例,凡在本申请实施例的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请实施例的保护范围之内。The above description is not intended to limit the embodiments of the present application, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included within the scope of protection of the embodiments of the present application.
Claims (12)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110791480.8A CN115696077A (en) | 2021-07-13 | 2021-07-13 | Imaging device and imaging control method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110791480.8A CN115696077A (en) | 2021-07-13 | 2021-07-13 | Imaging device and imaging control method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115696077A true CN115696077A (en) | 2023-02-03 |
Family
ID=85044434
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110791480.8A Pending CN115696077A (en) | 2021-07-13 | 2021-07-13 | Imaging device and imaging control method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115696077A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2024199986A1 (en) * | 2023-03-24 | 2024-10-03 | Sony Semiconductor Solutions Corporation | Image sensor control circuitry, image sensor, image sensor control method |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109799496A (en) * | 2018-11-14 | 2019-05-24 | 深圳市光微科技有限公司 | Concurrency detection circuit, photon detector, pulsed TOF sensor and its implementation |
CN109932852A (en) * | 2019-04-28 | 2019-06-25 | 清华大学深圳研究生院 | One kind making the maximized method of light-field camera sensor utilization |
CN111624615A (en) * | 2019-02-28 | 2020-09-04 | 意法半导体(R&D)有限公司 | Optical sensor |
CN111771168A (en) * | 2018-02-22 | 2020-10-13 | Imec 非营利协会 | Optical devices, systems and methods for forming three-dimensional light field distributions |
US20210208257A1 (en) * | 2020-01-07 | 2021-07-08 | Liturex (Guangzhou) Co. Ltd | Spad array with ambient light suppression for solid-state lidar |
-
2021
- 2021-07-13 CN CN202110791480.8A patent/CN115696077A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111771168A (en) * | 2018-02-22 | 2020-10-13 | Imec 非营利协会 | Optical devices, systems and methods for forming three-dimensional light field distributions |
CN109799496A (en) * | 2018-11-14 | 2019-05-24 | 深圳市光微科技有限公司 | Concurrency detection circuit, photon detector, pulsed TOF sensor and its implementation |
CN111624615A (en) * | 2019-02-28 | 2020-09-04 | 意法半导体(R&D)有限公司 | Optical sensor |
CN109932852A (en) * | 2019-04-28 | 2019-06-25 | 清华大学深圳研究生院 | One kind making the maximized method of light-field camera sensor utilization |
US20210208257A1 (en) * | 2020-01-07 | 2021-07-08 | Liturex (Guangzhou) Co. Ltd | Spad array with ambient light suppression for solid-state lidar |
Non-Patent Citations (1)
Title |
---|
康岩 等: "基于像素复用的SPAD阵列连续扫描三维成像", 红外与激光工程, 30 December 2020 (2020-12-30) * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2024199986A1 (en) * | 2023-03-24 | 2024-10-03 | Sony Semiconductor Solutions Corporation | Image sensor control circuitry, image sensor, image sensor control method |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11997397B2 (en) | Method, apparatus, and device for processing images, and storage medium | |
KR102354260B1 (en) | Method and device for processing lightfield data | |
US10484627B2 (en) | Image sensor with improved micro-lens, related imaging method and electronic device | |
US11107392B2 (en) | Display panel and electronic device | |
US11037968B2 (en) | Image sensor architecture | |
US10671218B2 (en) | Floating touch control panel, touch control method of the same, and display device | |
US20110202310A1 (en) | Depth sensor, depth estimation method using the same, and depth estimation device including the same | |
US20250117640A1 (en) | High bandwidth camera data transmission | |
US11328645B2 (en) | Display control method and device for N-primary-color display panel, and display device | |
JP7450809B2 (en) | DTOF detection module, terminal device, and ranging method | |
CN115862515B (en) | Source driving circuit, display device, electronic device and driving chip | |
US20160372020A1 (en) | Display substrate and method for driving the same, and display apparatus | |
US20190096360A1 (en) | Display method, image processing device, display device, and storage medium | |
CN115696077A (en) | Imaging device and imaging control method | |
US20230204741A1 (en) | Ranging method and device, storage medium, and lidar | |
WO2023060938A1 (en) | Debugging method for display panel, display device, and storage medium | |
KR20150063010A (en) | Method and device for estimating disparity associated with views of a scene acquired with a plenoptic camera | |
KR102211862B1 (en) | Image sensor and image sensor system including the same | |
US11011095B2 (en) | Display panel, and image control device and method thereof | |
US20210011167A1 (en) | Solid-state imaging device, solid-state imaging system, and method of driving the solid-state imaging device | |
US10127004B2 (en) | Display device and display signal input system and display signal input method thereof | |
US20230132616A1 (en) | Laser radar system and detection method | |
WO2019085489A1 (en) | Display screen, pixel driving method, and display device | |
WO2020224426A1 (en) | Fingerprint sensor, fingerprint module, and terminal device | |
WO2020186509A1 (en) | A scalable data fusion architecture and related products |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |