CN115696077A - Imaging device and imaging control method - Google Patents

Imaging device and imaging control method Download PDF

Info

Publication number
CN115696077A
CN115696077A CN202110791480.8A CN202110791480A CN115696077A CN 115696077 A CN115696077 A CN 115696077A CN 202110791480 A CN202110791480 A CN 202110791480A CN 115696077 A CN115696077 A CN 115696077A
Authority
CN
China
Prior art keywords
pixel
macro
pixels
sub
spad
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110791480.8A
Other languages
Chinese (zh)
Inventor
苏星
程卫亮
张泽鑫
沈林杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN202110791480.8A priority Critical patent/CN115696077A/en
Publication of CN115696077A publication Critical patent/CN115696077A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

The embodiment of the application discloses an imaging device and an imaging control method, and belongs to the technical field of photoelectric detection. In the embodiment of the application, a switch array and a multiplexer in an imaging unit may sequentially output one path of first image data corresponding to each macro pixel in a plurality of macro pixels, and a processing unit receives the first image data corresponding to each macro pixel and generates a fused image according to the first image data corresponding to each macro pixel and a position of the corresponding macro pixel. Because the overlapped SPAD sub-pixels are arranged between every two adjacent macro-pixels in the plurality of macro-pixels included in the SPAD array in the imaging unit, the quantity of image data corresponding to the macro-pixels output by the imaging unit is increased through the switch array and the multiplexer, and therefore the plane resolution of an output image is improved under the condition that the quantity of the SPAD sub-pixels and subsequent storage are not increased.

Description

成像装置及成像控制方法Imaging device and imaging control method

技术领域technical field

本申请涉及光电探测技术领域,特别涉及一种成像装置及成像控制方法。The present application relates to the field of photoelectric detection technology, in particular to an imaging device and an imaging control method.

背景技术Background technique

基于SPAD(Single Photon Avalanche Diode,单光子雪崩二极管)阵列的3D成像技术是近年来的研究热点,在消费电子、安防、机器人、自动驾驶等领域有巨大的应用前景。3D imaging technology based on SPAD (Single Photon Avalanche Diode) array is a research hotspot in recent years, and has great application prospects in consumer electronics, security, robotics, autonomous driving and other fields.

在相关技术中,为了节省电路面积,将SPAD阵列中空间上彼此相邻的n个SPAD子像素划分为一个宏像素,每个宏像素内的n个SPAD子像素共用同一套处理电路,这样,每个宏像素将对应一个像素数据。相应地,最终输出的图像的分辨率也将降低为未划分宏像素之前的分辨率的1/n。例如,参见图1,将空间上相邻的2*2个SPAD子像素划分为一个宏像素,输出一个像素数据,在这种情况下,当SPAD阵列中包括320*240个SPAD子像素时,输出的图像的分辨率将为160*120,由此可见,相关技术中的成像方法使得SPAD阵列的输出图像的平面分辨率损失较大。In the related art, in order to save the circuit area, the n SPAD sub-pixels spatially adjacent to each other in the SPAD array are divided into one macro-pixel, and the n SPAD sub-pixels in each macro-pixel share the same set of processing circuits, thus, Each macropixel will correspond to one pixel data. Correspondingly, the resolution of the final output image will also be reduced to 1/n of the resolution before the macro pixels are not divided. For example, referring to FIG. 1 , spatially adjacent 2*2 SPAD sub-pixels are divided into a macro-pixel, and a pixel data is output. In this case, when 320*240 SPAD sub-pixels are included in the SPAD array, The resolution of the output image will be 160*120. It can be seen that the imaging method in the related art causes a large loss in the planar resolution of the output image of the SPAD array.

发明内容Contents of the invention

本申请实施例提供了一种成像装置及成像控制方法,可以在SPAD阵列的子像素个数、后续电路不变的基础上,提高平面分辨率。所述技术方案如下:The embodiment of the present application provides an imaging device and an imaging control method, which can improve the planar resolution on the basis that the number of sub-pixels of the SPAD array and subsequent circuits remain unchanged. Described technical scheme is as follows:

一方面,提供了一种成像装置,所述成像装置包括:成像单元和处理单元,所述成像单元包括单光子雪崩二极管SPAD阵列、开关阵列和多路选择器,所述SPAD阵列包括多个宏像素,所述多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素;In one aspect, an imaging device is provided, the imaging device includes: an imaging unit and a processing unit, the imaging unit includes a single photon avalanche diode SPAD array, a switch array and a multiplexer, and the SPAD array includes a plurality of macro A pixel, each macro pixel in the plurality of macro pixels includes (N×M) SPAD sub-pixels, and there are overlapping SPAD sub-pixels between every two adjacent macro pixels;

所述开关阵列用于接收所述多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据;The switch array is used to receive the sub-pixel data respectively output by the (N×M) SPAD sub-pixels included in each macro-pixel in the plurality of macro-pixels, and the (N×M) SPADs included in each macro-pixel Synthesizing the sub-pixel data respectively output by the sub-pixels to obtain one channel of first image data corresponding to the corresponding macro-pixel, and outputting one channel of first image data corresponding to each macro-pixel;

所述多路选择器用于接收所述开关阵列输出的多路第一图像数据,并将所述多路第一图像数据依次输出至所述处理单元;The multiplexer is used to receive multiple channels of first image data output by the switch array, and sequentially output the multiple channels of first image data to the processing unit;

所述处理单元用于接收所述多路选择器依次输出的各个宏像素对应的第一图像数据,并根据各个宏像素的位置和对应的第一图像数据和相应宏像素,生成融合图像。The processing unit is configured to receive the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generate a fused image according to the position of each macro pixel and the corresponding first image data and the corresponding macro pixel.

可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一行SPAD子像素连接一个第一级逻辑或门,第一宏像素中多行SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logic OR gate and a second-level logic OR gate, a row of SPAD sub-pixels in the first macro pixel is connected to a first-level logic OR gate, and multiple rows of SPAD sub-pixels in the first macro pixel A plurality of first-level logic OR gates connected to sub-pixels is connected to a second-level logic OR gate, and the first macro pixel is any one of the plurality of macro pixels;

所述第一宏像素连接的每个第一级逻辑或门用于接收自身连接的一行SPAD子像素中每个SPAD子像素输出的子像素数据,并将接收到的子像素数据合成为一路行像素数据,输出所述行像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive the sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected to itself, and synthesize the received sub-pixel data into one row Pixel data, outputting the row of pixel data;

所述第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的行像素数据,将接收到的行像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and synthesize the received row pixel data to obtain the first the first image data corresponding to the macro pixels, and output the first image data corresponding to the first macro pixels.

可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a column of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro pixel A plurality of first-level logic OR gates connected to sub-pixels is connected to a second-level logic OR gate, and the first macro pixel is any one of the plurality of macro pixels;

所述第一宏像素连接的每个第一级逻辑或门用于接收连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,并将接收到的子像素数据合成为一路列像素数据,输出所述列像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive one path of sub-pixel data output by each SPAD sub-pixel in a connected column of SPAD sub-pixels, and synthesize the received sub-pixel data into one path of column Pixel data, outputting the column of pixel data;

所述第一宏像素连接的第二级逻辑或门用于接收连接的多个第一级逻辑或门分别输出的列像素数据,将接收到的列像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive the column pixel data respectively output by the connected multiple first-level logical OR gates, and synthesize the received column pixel data to obtain the first macro pixel the first image data corresponding to the pixel, and output the first image data corresponding to the first macro pixel.

可选地,所述多路选择器包括多个输入端、一个输出端和一个控制端,所述多路选择器的多个输入端与所述开关阵列连接,所述多路选择器的控制端与所述处理单元的控制端连接,所述多路选择器的输出端与所述处理单元的输入端连接;Optionally, the multiplexer includes a plurality of input terminals, an output terminal and a control terminal, the multiple input terminals of the multiplexer are connected to the switch array, and the control of the multiplexer The terminal is connected to the control terminal of the processing unit, and the output terminal of the multiplexer is connected to the input terminal of the processing unit;

所述多路选择器的每个输入端用于接收所述开关阵列输出的一路第一图像数据;Each input terminal of the multiplexer is used to receive one channel of first image data output by the switch array;

所述多路选择器的控制端用于在每个测量时刻接收所述处理单元的控制端输出的选通信号;The control terminal of the multiplexer is used to receive the strobe signal output by the control terminal of the processing unit at each measurement moment;

所述多路选择器的输出端用于在每个测量时刻输出与相应测量时刻接收到的选通信号对应的一路第一图像数据。The output terminal of the multiplexer is used for outputting one path of first image data corresponding to the strobe signal received at the corresponding measurement moment at each measurement moment.

可选地,所述处理单元用于根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置,并将相应选通信号对应的第一图像数据作为确定出的宏像素的位置上的像素数据,以生成所述融合图像。Optionally, the processing unit is configured to determine the position of the macropixel corresponding to the first image data corresponding to the received corresponding strobe signal according to the strobe signal output at each measurement moment, and correspond the corresponding strobe signal to The first image data is used as the determined pixel data at the position of the macro pixel to generate the fused image.

可选地,所述多个宏像素中每相邻的两个宏像素具有(N-1)行重叠的SPAD子像素,或者,所述多个宏像素中每相邻的两个宏像素具有(M-1)列重叠的SPAD子像素。Optionally, every two adjacent macro pixels in the multiple macro pixels have (N-1) rows of overlapping SPAD sub-pixels, or, every two adjacent macro pixels in the multiple macro pixels have (M-1) columns of overlapping SPAD sub-pixels.

另一方面,提供了一种成像控制方法,所述方法应用于所述方法应用于成像装置的处理单元中,所述成像装置还包括成像单元,所述成像单元包括单光子雪崩二极管SPAD阵列、开关阵列和多路选择器,所述SPAD阵列包括多个宏像素,所述多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素,其中,所述开关阵列用于接收所述多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据至所述多路选择器;所述方法包括:In another aspect, an imaging control method is provided, the method is applied to a processing unit of an imaging device, the imaging device further includes an imaging unit, and the imaging unit includes a single photon avalanche diode SPAD array, A switch array and a multiplexer, the SPAD array includes a plurality of macro pixels, each macro pixel in the plurality of macro pixels includes (N×M) SPAD sub-pixels, and every two adjacent macro pixels There are overlapping SPAD sub-pixels between them, wherein the switch array is used to receive sub-pixel data respectively output by (N×M) SPAD sub-pixels included in each macro-pixel in the plurality of macro-pixels, for each Combining the sub-pixel data respectively output by the (N×M) SPAD sub-pixels included in the macro-pixel to obtain one path of first image data corresponding to the corresponding macro-pixel, and outputting one path of first image data corresponding to each macro-pixel to the multiple a way selector; the method comprising:

所述处理单元按照预设时序向所述多路选择器依次输出不同种类的选通信号,以使所述多路选择器在每接收到所述处理单元输出的一种选通信号时,输出与相应选通信号对应的第一图像数据;The processing unit sequentially outputs different types of strobe signals to the multiplexer according to a preset timing, so that the multiplexer outputs each time a strobe signal output by the processing unit is received. first image data corresponding to the corresponding strobe signal;

所述处理单元接收所述多路选择器依次输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。The processing unit receives the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.

可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,所述第一宏像素中的一行SPAD子像素连接一个第一级逻辑或门,第一宏像素中多行SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a row of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple A plurality of first-level logical OR gates connected to row SPAD sub-pixels are connected to a second-level logical OR gate, and the first macro pixel is any one of the plurality of macro pixels;

所述第一宏像素连接的每个第一级逻辑或门用于接收自身连接的一行SPAD子像素中每个SPAD子像素输出的子像素数据,并将接收到的子像素数据合成为一路行像素数据,输出所述行像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive the sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected to itself, and synthesize the received sub-pixel data into one row Pixel data, outputting the row of pixel data;

所述第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的行像素数据,将接收到的行像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and synthesize the received row pixel data to obtain the first the first image data corresponding to the macro pixels, and output the first image data corresponding to the first macro pixels.

可选地,所述开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,所述第一宏像素为所述多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a column of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro pixel A plurality of first-level logic OR gates connected to sub-pixels is connected to a second-level logic OR gate, and the first macro pixel is any one of the plurality of macro pixels;

所述第一宏像素连接的每个第一级逻辑或门用于接收连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,并将接收到的子像素数据合成为一路列像素数据,输出所述列像素数据;Each first-stage logical OR gate connected to the first macro-pixel is used to receive one path of sub-pixel data output by each SPAD sub-pixel in a connected column of SPAD sub-pixels, and synthesize the received sub-pixel data into one path of column Pixel data, outputting the column of pixel data;

所述第一宏像素连接的第二级逻辑或门用于接收连接的多个第一级逻辑或门分别输出的列像素数据,将接收到的列像素数据进行合成,得到所述第一宏像素对应的第一图像数据,输出所述第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive the column pixel data respectively output by the connected multiple first-level logical OR gates, and synthesize the received column pixel data to obtain the first macro pixel the first image data corresponding to the pixel, and output the first image data corresponding to the first macro pixel.

可选地,所述多路选择器包括多个输入端、一个输出端和一个控制端,所述多路选择器的多个输入端与所述开关阵列连接,所述多路选择器的控制端与所述处理单元的控制端连接,所述多路选择器的输出端与所述处理单元的输入端连接,所述多路选择器的每个输入端用于接收所述开关阵列输出的一路第一图像数据;Optionally, the multiplexer includes a plurality of input terminals, an output terminal and a control terminal, the multiple input terminals of the multiplexer are connected to the switch array, and the control of the multiplexer The terminal is connected to the control terminal of the processing unit, the output terminal of the multiplexer is connected to the input terminal of the processing unit, and each input terminal of the multiplexer is used to receive the output of the switch array. One channel of first image data;

所述处理单元按照预设时序向所述多路选择器依次输出不同种类的选通信号,包括:The processing unit sequentially outputs different types of strobe signals to the multiplexer according to a preset timing sequence, including:

所述处理单元的控制端在第一测量时刻向所述多路选择器的控制端输出第一选通信号,以使所述多路选择器的输出端在所述第一测量时刻输出与所述第一选通信号对应的一路第一图像数据,所述第一测量时刻为多个测量时刻中的任一测量时刻。The control terminal of the processing unit outputs a first gating signal to the control terminal of the multiplexer at the first measurement moment, so that the output terminal of the multiplexer outputs the same signal at the first measurement moment. One path of first image data corresponding to the first gating signal, and the first measurement moment is any one of the plurality of measurement moments.

可选地,所述处理单元根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像,包括:Optionally, the processing unit generates a fused image according to the first image data corresponding to each macropixel and the position of the corresponding macropixel, including:

根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置;According to the gate signal output at each measurement moment, determine the position of the macro pixel corresponding to the first image data corresponding to the received corresponding gate signal;

将相应选通信号对应的的第一图像数据作为相应测量时刻确定的宏像素的位置上的像素数据,以生成所述融合图像。The first image data corresponding to the corresponding gate signal is used as the pixel data at the position of the macro pixel determined at the corresponding measurement moment, so as to generate the fused image.

可选地,所述多个宏像素中每相邻的两个宏像素具有(N-1)行重叠的SPAD子像素,或者,所述多个宏像素中每相邻的两个宏像素具有(M-1)列重叠的SPAD子像素。Optionally, every two adjacent macro pixels in the multiple macro pixels have (N-1) rows of overlapping SPAD sub-pixels, or, every two adjacent macro pixels in the multiple macro pixels have (M-1) columns of overlapping SPAD sub-pixels.

另一方面,提供了一种成像装置,所述装置包括:In another aspect, an imaging device is provided, the device comprising:

处理器;processor;

用于存储处理器可执行指令的存储器;memory for storing processor-executable instructions;

其中,所述处理器执行所述存储器中的可执行指令来执行上述成像控制方法。Wherein, the processor executes the executable instructions in the memory to implement the above imaging control method.

另一方面,提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述的成像控制方法的步骤。In another aspect, there is provided a computer program product containing instructions, which, when run on a computer, causes the computer to execute the steps of the above-mentioned imaging control method.

本申请实施例提供的技术方案带来的有益效果至少包括:The beneficial effects brought by the technical solutions provided by the embodiments of the present application at least include:

在本申请实施例中,通过成像单元中的开关阵列和多路选择器可以依次输出多个宏像素中每个宏像素对应的一路第一图像数据,处理单元接收每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。由于成像单元中SPAD阵列包括的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素,因此,通过开关阵列和多路选择器使得成像单元输出的宏像素对应的图像数据的数量增加,从而在SPAD子像素个数、后续存储不增加的情况下,提高了输出图像的平面分辨率。In the embodiment of the present application, through the switch array and the multiplexer in the imaging unit, one channel of first image data corresponding to each macro pixel in the plurality of macro pixels can be sequentially output, and the processing unit receives the first image data corresponding to each macro pixel. image data, and generate a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel. Since there are overlapping SPAD sub-pixels between every two adjacent macro-pixels in the multiple macro-pixels included in the SPAD array in the imaging unit, the image corresponding to the macro-pixels output by the imaging unit is made through the switch array and the multiplexer The amount of data increases, so that the planar resolution of the output image is improved without increasing the number of SPAD sub-pixels and subsequent storage.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

图1是一种SPAD阵列的结构图;Fig. 1 is a structural diagram of a SPAD array;

图2是本申请实施例提供的一种成像装置所涉及的系统架构图;FIG. 2 is a system architecture diagram related to an imaging device provided by an embodiment of the present application;

图3是本申请实施例提供的一种成像装置的结构示意图;FIG. 3 is a schematic structural diagram of an imaging device provided in an embodiment of the present application;

图4是本申请实施例提供的一种SPAD阵列中宏像素的示意图;FIG. 4 is a schematic diagram of a macro pixel in a SPAD array provided by an embodiment of the present application;

图5是本申请实施例提供的一种宏像素与开关阵列之间连接关系的示意图;FIG. 5 is a schematic diagram of a connection relationship between a macro pixel and a switch array provided by an embodiment of the present application;

图6是本申请实施例提供的另一种宏像素与开关阵列之间连接关系的示意图;FIG. 6 is a schematic diagram of another connection relationship between a macro pixel and a switch array provided by an embodiment of the present application;

图7是本申请实施例提供的另一种成像装置的结构示意图;FIG. 7 is a schematic structural diagram of another imaging device provided by an embodiment of the present application;

图8是本申请实施例提供的一种成像装置生成的融合图像的示意图;FIG. 8 is a schematic diagram of a fused image generated by an imaging device provided in an embodiment of the present application;

图9是本申请实施例提供的一种成像控制方法的流程图。FIG. 9 is a flowchart of an imaging control method provided by an embodiment of the present application.

具体实施方式Detailed ways

为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

在对本申请实施例进行详细的解释说明之前,先对本申请实施例涉及的系统架构进行介绍。Before explaining the embodiment of the present application in detail, the system architecture involved in the embodiment of the present application is firstly introduced.

图2是本申请实施例提供的一种成像装置所涉及的系统架构图。如图2所示,该系统包括成像装置10、路由器20和服务器30。成像装置10和路由器20之间相连,路由器20和服务器30之间相连。其中,成像装置10用于生成图像,并将生成的图像输出至路由器20;路由器20用于传输成像装置10生成的图像至服务器30;服务器30用于接收路由器20传输的图像,并将接收到的图像进行存储。FIG. 2 is a system architecture diagram related to an imaging device provided by an embodiment of the present application. As shown in FIG. 2 , the system includes an imaging device 10 , a router 20 and a server 30 . The imaging device 10 is connected to the router 20 , and the router 20 is connected to the server 30 . Wherein, the imaging device 10 is used to generate an image, and output the generated image to the router 20; the router 20 is used to transmit the image generated by the imaging device 10 to the server 30; the server 30 is used to receive the image transmitted by the router 20, and receive images are stored.

其中,成像装置10包括处理单元101和成像单元102。该成像单元102包括SPAD阵列。其中,该SPAD阵列包括多个宏像素,每个宏像素包括多个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素。该成像单元102用于输出每个宏像素对应的第一图像数据,处理单元101用于接收成像单元102输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。需要说明的是,该成像单元102的详细实现方式可以参见下文中的介绍,在此不再赘述。Wherein, the imaging device 10 includes a processing unit 101 and an imaging unit 102 . The imaging unit 102 includes a SPAD array. Wherein, the SPAD array includes a plurality of macro pixels, each macro pixel includes a plurality of SPAD sub-pixels, and there are overlapping SPAD sub-pixels between every two adjacent macro pixels. The imaging unit 102 is used to output the first image data corresponding to each macro pixel, and the processing unit 101 is used to receive the first image data corresponding to each macro pixel output by the imaging unit 102, and according to the first image data corresponding to each macro pixel The image data and the locations of the corresponding macropixels are used to generate a fused image. It should be noted that, for the detailed implementation of the imaging unit 102 , reference may be made to the introduction below, which will not be repeated here.

除此之外,该成像装置10还可以包括激光器驱动103、激光器104、发射镜头105和接收镜头106。In addition, the imaging device 10 may further include a laser driver 103 , a laser 104 , a transmitting lens 105 and a receiving lens 106 .

激光驱动器103可以分别与成像单元102和激光器104连接。成像单元102可以向激光器驱动103发送发射开始指令,并在发送发射开始指令的同时开始计时。激光器驱动103在接收到发射开始指令后,驱动激光器104发出激光。The laser driver 103 can be connected to the imaging unit 102 and the laser 104 respectively. The imaging unit 102 may send an emission start instruction to the laser driver 103, and start timing while sending the emission start instruction. The laser driver 103 drives the laser 104 to emit laser light after receiving the emission start instruction.

发射镜头105用于将激光器104发出的激光发射出去,照射到目标物体上。发射镜头105可以固定于激光器104之上,也可以装设于激光器104之内。The emitting lens 105 is used for emitting the laser light emitted by the laser 104 to irradiate the target object. The emitting lens 105 can be fixed on the laser 104 or installed in the laser 104 .

接收镜头106用于接收目标物体反射的光,进而将接收到的反射光发射到成像单元102中的SPAD阵列的各个SPAD子像素上,以便成像单元102中的SPAD阵列的各个SPAD子像素根据探测到的反射光信号输出子像素数据,进而使得成像单元102根据各个子像素数据输出每个宏像素对应的第一图像数据。接收镜头106可以固定于成像单元102中的SPAD阵列之上,也可以装设于SPAD阵列102之内。The receiving lens 106 is used to receive the light reflected by the target object, and then transmit the received reflected light to each SPAD sub-pixel of the SPAD array in the imaging unit 102, so that each SPAD sub-pixel of the SPAD array in the imaging unit 102 can detect The received reflected light signal outputs sub-pixel data, so that the imaging unit 102 outputs the first image data corresponding to each macro-pixel according to each sub-pixel data. The receiving lens 106 can be fixed on the SPAD array in the imaging unit 102 , or installed inside the SPAD array 102 .

可选地,在一些可能的实现方式中,该成像装置10在得到融合图像之后,还可以基于该融合图像进行智能分析,之后,将该融合图像和智能分析结果经过路由器20一起发送至服务器30,以便服务器30对该融合图像和智能分析结果进行存储,供后续其他业务进行使用。Optionally, in some possible implementations, after the imaging device 10 obtains the fused image, it can also perform intelligent analysis based on the fused image, and then send the fused image and the intelligent analysis result to the server 30 through the router 20 , so that the server 30 stores the fused image and the intelligent analysis result for subsequent use by other businesses.

接下来对本申请实施例提供的成像装置进行介绍。Next, the imaging device provided by the embodiment of the present application will be introduced.

图3是本申请实施例提供的一种成像装置的示意图。如图3所示,该装置包括:成像单元301和处理单元302,该成像单元301包括SPAD阵列3011、开关阵列3012和多路选择器3013,该SPAD阵列3011包括多个宏像素,多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素。其中,开关阵列3012用于接收多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据;多路选择器3013用于接收开关阵列3012输出的多路第一图像数据,并将多路第一图像数据依次输出至处理单元302;处理单元302用于接收SPAD阵列输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。FIG. 3 is a schematic diagram of an imaging device provided by an embodiment of the present application. As shown in Figure 3, the device includes: an imaging unit 301 and a processing unit 302, the imaging unit 301 includes a SPAD array 3011, a switch array 3012 and a multiplexer 3013, the SPAD array 3011 includes a plurality of macro pixels, a plurality of macro pixels Each macro pixel in the pixel includes (N×M) SPAD sub-pixels, and there are overlapping SPAD sub-pixels between every two adjacent macro pixels. Among them, the switch array 3012 is used to receive the sub-pixel data respectively outputted by (N×M) SPAD sub-pixels included in each macro-pixel in multiple macro-pixels, and the (N×M) SPAD sub-pixels included in each macro-pixel The sub-pixel data respectively output by the pixels are synthesized to obtain one channel of first image data corresponding to the corresponding macro pixel, and output one channel of first image data corresponding to each macro pixel; the multiplexer 3013 is used to receive the multiple channels output by the switch array 3012 The first image data, and multiple channels of first image data are sequentially output to the processing unit 302; the processing unit 302 is used to receive the first image data corresponding to each macro pixel output by the SPAD array, and according to the corresponding first image data of each macro pixel An image data and the location of the corresponding macro-pixels are used to generate a fused image.

在本申请实施例中,SPAD阵列3011中包含有呈矩形阵列排布的多个SPAD子像素,可以将SPAD阵列3011中(N×M)个SPAD子像素组成一个宏像素。则该SPAD阵列3011中包含有多个由(N×M)个SPAD子像素组成的宏像素。其中,N为宏像素的行数,M为宏像素的列数,且N与M可以相等,也可以不相等,本申请实施例对此不作限定。In the embodiment of the present application, the SPAD array 3011 includes a plurality of SPAD sub-pixels arranged in a rectangular array, and (N×M) SPAD sub-pixels in the SPAD array 3011 can be formed into a macro pixel. Then the SPAD array 3011 includes a plurality of macro pixels composed of (N×M) SPAD sub-pixels. Wherein, N is the number of rows of macro pixels, and M is the number of columns of macro pixels, and N and M may or may not be equal, which is not limited in this embodiment of the present application.

需要说明的是,在本申请实施例中,SPAD阵列3011中的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素。It should be noted that, in the embodiment of the present application, among the plurality of macro-pixels in the SPAD array 3011 , there are overlapping SPAD sub-pixels between every two adjacent macro-pixels.

示例性地,当各个宏像素包括(N×M)个SPAD子像素时,上下相邻的两个宏像素可以具有多行重叠的SPAD子像素,左右相邻的两个宏像素可以具有多列重叠的SPAD子像素。例如,上下相邻的两个宏像素可以具有(N-1)行重叠的SPAD子像素,也可以具有(N-2)行重叠的SPAD子像素重叠,本申请实施例对此不作限定。左右相邻的两个宏像素具有(M-1)列重叠的SPAD子像素,也可以具有(M-2)列重叠的SPAD子像素,本申请实施例对此不做限定。Exemplarily, when each macro pixel includes (N×M) SPAD sub-pixels, two adjacent macro pixels up and down may have multiple rows of overlapping SPAD sub-pixels, and two macro pixels adjacent to left and right may have multiple columns Overlapping SPAD subpixels. For example, two vertically adjacent macro pixels may have (N-1) rows of overlapping SPAD sub-pixels, or may have (N-2) rows of overlapping SPAD sub-pixels, which is not limited in this embodiment of the present application. Two adjacent left and right macro pixels have (M-1) columns of overlapping SPAD sub-pixels, and may also have (M-2) columns of overlapping SPAD sub-pixels, which is not limited in this embodiment of the present application.

例如,参见图4,一个4×4共有16个SPAD子像素的SPAD阵列,以N=M=3为例,一个宏像素包括3×3个SPAD子像素。其中,上下相邻的宏像素1和宏像素2之间重叠2行,第一行重叠的SPAD子像素包括SPAD5~SPAD7,第二行重叠的SPAD子像素包括SPAD9~SPAD11;左右相邻的宏像素2和宏像素3重叠2列,第一列重叠的SPAD子像素包括SPAD6、SPAD10、SPAD14,第二列重叠的SPAD子像素包括SPAD7、SPAD11、SPAD15。For example, referring to FIG. 4 , a 4×4 SPAD array with 16 SPAD sub-pixels in total, taking N=M=3 as an example, a macro pixel includes 3×3 SPAD sub-pixels. Among them, 2 rows overlap between the upper and lower adjacent macro pixels 1 and 2, the SPAD sub-pixels overlapping in the first row include SPAD5-SPAD7, and the overlapping SPAD sub-pixels in the second row include SPAD9-SPAD11; the left and right adjacent macro pixels 2 and macro pixel 3 are overlapped by 2 columns, the overlapping SPAD sub-pixels in the first column include SPAD6, SPAD10, SPAD14, and the overlapping SPAD sub-pixels in the second column include SPAD7, SPAD11, SPAD15.

在本申请实施例中,开关阵列3012包括第一级逻辑或门和第二级逻辑或门。在一种可能的实现方式中,以SPAD阵列3011中多个宏像素中任一宏像素为例,将其称为第一宏像素,该第一宏像素中的每一行SPAD子像素分别连接一个第一级逻辑或门,每个宏像素中多行SPAD子像素对应的多个第一级逻辑或门连接一个第二级逻辑或门。In the embodiment of the present application, the switch array 3012 includes a first-level logic OR gate and a second-level logic OR gate. In a possible implementation manner, taking any macro pixel in the plurality of macro pixels in the SPAD array 3011 as an example, it is called the first macro pixel, and each row of SPAD sub-pixels in the first macro pixel is respectively connected to a A first-level logic OR gate, a plurality of first-level logic OR gates corresponding to multiple rows of SPAD sub-pixels in each macro pixel is connected to a second-level logic OR gate.

其中,该第一宏像素连接的每个第一级逻辑或门用于接收连接的一行SPAD子像素中每个SPAD子像素输出的一路子像素数据,之后,第一级逻辑或门将接收到的子像素数据进行合成,得到一路行像素数据,输出该行像素数据。例如,对于一个由(N×M)个SPAD子像素组成的宏像素,该宏像素连接N个第一级逻辑或门,每个第一级逻辑或门将输出一个行像素数据,这样,N个第一级逻辑或门将输出N个行像素数据。Wherein, each first-level logical OR gate connected to the first macro-pixel is used to receive one path of sub-pixel data output by each SPAD sub-pixel in the connected row of SPAD sub-pixels, and then the first-level logical OR gate will receive the The sub-pixel data is synthesized to obtain a row of pixel data, and the row of pixel data is output. For example, for a macro pixel composed of (N×M) SPAD sub-pixels, the macro pixel is connected with N first-level logical OR gates, and each first-level logical OR gate will output a row of pixel data, so that N The first logical OR gate will output N rows of pixel data.

第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的行像素数据,之后,第二级逻辑或门将接收到的多路行像素数据进行合成,得到一路第一图像数据,并输出该第一图像数据。The second-level logical OR gate connected to the first macro-pixel is used to receive row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and then, the second-level logical OR gate performs multi-channel row pixel data received by the second-level logical OR gate. Synthesize to obtain a channel of first image data, and output the first image data.

需要说明的是,在一种可能的情况中,当第一宏像素包括4个SPAD子像素时,则一个第一级逻辑或门可以为一个二输入或门。相应地,一个第二级逻辑或门也可以为一个二输入或门。It should be noted that, in a possible situation, when the first macro-pixel includes 4 SPAD sub-pixels, a first-level logical OR gate may be a two-input OR gate. Correspondingly, a second-level logical OR gate can also be a two-input OR gate.

例如,参见图5,假设SPAD阵列包括3×3个SPAD子像素,SPAD1、SPAD2、SPAD4、SPAD5组成宏像素1,该宏像素1中的第一行的SPAD1和SPAD2连接一个第一级逻辑或门,该第一级逻辑或门为一个二输入或门。在这种情况下,该第一级逻辑或门接收SPAD1和SPAD2分别输出的子像素数据,并将这两路子像素数据合成为一路行像素数据1+2;同理,第二行的SPAD4和SPAD5连接一个第一级逻辑或门,该第一级逻辑或门接收SPAD4和SPAD5分别输出的子像素数据,并将这两路子像素数据合成为一路行像素数据4+5。For example, referring to FIG. 5 , assuming that the SPAD array includes 3×3 SPAD sub-pixels, SPAD1, SPAD2, SPAD4, and SPAD5 form a macro pixel 1, and the SPAD1 and SPAD2 of the first row in the macro pixel 1 are connected to a first-level logical OR gate, the first-level logical OR gate is a two-input OR gate. In this case, the first-level logical OR gate receives the sub-pixel data respectively output by SPAD1 and SPAD2, and synthesizes the two-way sub-pixel data into one line of pixel data 1+2; similarly, SPAD4 and SPAD2 in the second line SPAD5 is connected to a first-level logical OR gate, which receives the sub-pixel data output by SPAD4 and SPAD5 respectively, and synthesizes the two channels of sub-pixel data into one channel of row pixel data 4+5.

进一步地,宏像素1的两行SPAD子像素对应的两个第一级逻辑或门连接一个第二级逻辑或门,该第二逻辑或门也为一个二输入或门,这样,该第二级逻辑或门接收这两个第一级逻辑或门输出的行像素数据1+2和行像素数据4+5,将行像素数据1+2和行像素数据4+5合成为一路第一图像数据1245,并输出第一图像数据1245。Further, the two first-level logical OR gates corresponding to the two rows of SPAD sub-pixels of the macro pixel 1 are connected to a second-level logical OR gate, and the second logical OR gate is also a two-input OR gate. In this way, the second logical OR gate The first-level logical OR gate receives the row pixel data 1+2 and the row pixel data 4+5 output by the two first-level logical OR gates, and synthesizes the row pixel data 1+2 and the row pixel data 4+5 into a first image data 1245, and output the first image data 1245.

在另一种可能的情况中,当第一宏像素包括X个SPAD子像素,且X大于4时,则第一宏像素连接的每个第一级逻辑或门可能包括一个或多个二输入或门,第一宏像素连接的第二逻辑或门也可能包括一个或多个二输入或门。In another possible situation, when the first macro pixel includes X SPAD sub-pixels, and X is greater than 4, each first-level logical OR gate connected to the first macro pixel may include one or more two-input The OR gate, the second logical OR gate connected to the first macro pixel may also include one or more two-input OR gates.

例如,当X为9时,根据R=ceiling(log2X)计算得到R=4,其中,ceiling(*)是指向上取整。在这种情况下,该开关阵列将包括4层二输入或门,其中,每个第一级逻辑或门可以由两层二输入或门组成,每个第二级逻辑或门也可以由两层二输入或门组成。For example, when X is 9, R=4 is calculated according to R=ceiling(log 2 X), where ceiling(*) refers to rounding up. In this case, the switch array will include 4 layers of two-input OR gates, wherein each first-level logical OR gate can be composed of two layers of two-input OR gates, and each second-level logical OR gate can also be composed of two layers of two-input OR gates. Layer 2 is composed of input OR gates.

例如,参见图6,假设SPAD阵列包括4×4个SPAD子像素,SPAD1、SPAD2、SPAD3、SPAD5、SPAD6、SPAD7、SPAD9、SPAD10以及SPAD11组成宏像素1,在这种情况下,该宏像素1中的第一行中每相邻的两个SPAD子像素可以连接一个二输入或门,这样,这2个二输入或门即为该宏像素1的第一行SPAD子像素连接的第一级逻辑或门中的第一层二输入或门。在此基础上,该第一级逻辑或门中相邻的第一层输入或门连接一个二输入或门,这样,这个二输入或门即为第一行SPAD子像素连接的第一级逻辑或门中的第二层输入或门。这样,宏像素1的各行SPAD子像素连接的第一级逻辑或门中每个第一层二输入或门接收连接的两个SPAD子像素输出的子像素数据,并将两路子像素数据进行合并后输出。每个第二层二输入或门接收连接的两个第一层输入或门输出的两路数据,并将接收到的两路数据进行合并,以得到一路行像素数据。For example, referring to FIG. 6 , assuming that the SPAD array includes 4×4 SPAD sub-pixels, SPAD1, SPAD2, SPAD3, SPAD5, SPAD6, SPAD7, SPAD9, SPAD10 and SPAD11 form a macro pixel 1, in this case, the macro pixel 1 Every two adjacent SPAD sub-pixels in the first row can be connected with a two-input OR gate, so that these two two-input OR gates are the first level of connection between the SPAD sub-pixels in the first row of the macro pixel 1 The first layer of two-input OR gates in logical OR gates. On this basis, the adjacent first-level input OR gate in the first-level logic OR gate is connected to a two-input OR gate, so that this two-input OR gate is the first-level logic connected to the SPAD sub-pixels in the first row The second layer of OR gates is an input OR gate. In this way, each of the first-level two-input OR gates in the first-level logical OR gates connected to the SPAD sub-pixels in each row of the macro pixel 1 receives the sub-pixel data output by the connected two SPAD sub-pixels, and combines the two-way sub-pixel data After output. Each second-layer two-input OR gate receives two channels of data output by the connected two first-layer input OR gates, and combines the received two channels of data to obtain one channel of row pixel data.

在通过上述方法得到宏像素中的每行SPAD子像素对应的行像素数据之后,对于这3个行像素数据,可以将输出相邻两行的行像素数据的两个第二层二输入或门连接一个二输入或门,此时,连接的这个二输入或门即为第二级逻辑或门中的第一层二输入或门。之后,第二级逻辑或门中的两个第一层二输入或门再连接一个二输入或门,此时,这个二输入或门即为第二级逻辑或门中的第二层二输入或门。这样,第二级逻辑或门中的每个第一层二输入或门可以将接收到的相邻两行的行像素数据进行合并,输出一路数据,之后,第二层二输入或门将两个第一层二输入或门输入的两路数据进行合并,从而得到第一图像数据。After the row pixel data corresponding to each row of SPAD sub-pixels in the macro pixel is obtained by the above method, for these three rows of pixel data, the two second-layer two-input OR gates that output the row pixel data of two adjacent rows can be A two-input OR gate is connected. At this time, the connected two-input OR gate is the first layer two-input OR gate in the second-level logic OR gate. After that, the two first-level two-input OR gates in the second-level logical OR gate are connected to a two-input OR gate. At this time, this two-input OR gate is the second-level two-input OR gate in the second-level logical OR gate. OR gate. In this way, each first-layer two-input OR gate in the second-level logical OR gate can combine the received row pixel data of two adjacent rows, and output one road of data, after that, the second layer two-input OR gate combines two The two channels of data input by the two-input OR gate of the first layer are combined to obtain the first image data.

上述仅是以宏像素包括9个SPAD子像素为例进行示例性说明,当宏像素包括8个、6个或者更多SPAD子像素,可以参考上述方法进行处理,本申请实施例在此不再赘述。The above is only an example of a macro pixel including 9 SPAD sub-pixels. When a macro pixel includes 8, 6 or more SPAD sub-pixels, the above method can be referred to for processing. repeat.

另外,对于多个宏像素中的每个宏像素,其均可以参考上述第一宏像素,连接多个第一级逻辑或门以及一个第二级逻辑或门。这样,该开关阵列3012中将包括多个第一级逻辑或门和多个第二级逻辑或门。In addition, for each macro pixel among the multiple macro pixels, it can refer to the above-mentioned first macro pixel, and connect multiple first-level logical OR gates and one second-level logical OR gate. In this way, the switch array 3012 will include multiple first-level logic OR gates and multiple second-level logic OR gates.

可选地,在另一种可能的实现方式中,仍以第一宏像素为例,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门。Optionally, in another possible implementation, still taking the first macro pixel as an example, a column of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro pixel Multiple first-level logical OR gates connected to the sub-pixels are connected to one second-level logical OR gate.

其中,第一宏像素连接的每个第一级逻辑或门可以用于接收自身连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,之后,第一级逻辑或门将接收到的子像素数据进行合成,得到一路列像素数据,输出该列像素数据。例如,在一个(N×M)个SPAD子像素组成的一个宏像素,该宏像素连接M个第一级逻辑或门,每个第一级逻辑或门将输出一个列像素数据,这样,M个第一级逻辑或门将输出M个列像素数据。Wherein, each first-level logical OR gate connected to the first macro-pixel can be used to receive one path of sub-pixel data output by each SPAD sub-pixel in a column of SPAD sub-pixels connected to itself, and then the first-level logical OR gate will receive Synthesize the sub-pixel data to obtain a column of pixel data, and output the column of pixel data. For example, in a macro pixel composed of (N×M) SPAD sub-pixels, the macro pixel is connected with M first-level logical OR gates, and each first-level logical OR gate will output a column of pixel data, so that M The first level logic OR gate will output M columns of pixel data.

第一宏像素连接的第二级逻辑或门用于接收自身连接的多个第一级逻辑或门分别输出的列像素数据,之后,第二级逻辑或门将接收到的列像素数据进行合成,得到一路第一图像数据,并输出第一图像数据。The second level logical OR gate connected to the first macro pixel is used to receive column pixel data respectively output by multiple first level logical OR gates connected to itself, and then the second level logical OR gate synthesizes the received column pixel data, Obtain one channel of first image data, and output the first image data.

需要说明的是,当宏像素包括的SPAD子像素为4时,每个第一级逻辑或门和第二级逻辑或门均可以通过一个二输入或门来实现。可选地,当宏像素包括更多的SPAD子像素时,则可以参考前文中介绍的方法,每个第一级逻辑或门通过一个或多个二输入或门来实现,每个第二级逻辑或门也通过一个或多个二输入或门来实现,本申请实施例在此不再赘述。It should be noted that when the number of SPAD sub-pixels included in the macro-pixel is 4, each of the first-level logic OR gate and the second-level logic OR gate can be realized by a two-input OR gate. Optionally, when the macro-pixel includes more SPAD sub-pixels, you can refer to the method introduced above, each first-level logical OR gate is realized by one or more two-input OR gates, each second-level The logical OR gate is also implemented by one or more two-input OR gates, which will not be repeated in this embodiment of the present application.

开关阵列3012中的每个第二级逻辑或门在输出一路第一图像数据之后,多路选择器3013接收多个第二级逻辑或门分别输出的一路第一图像数据,并将接收到的多路第一图像数据依次输出至处理单元302。After each second-level logical OR gate in the switch array 3012 outputs one channel of first image data, the multiplexer 3013 receives one channel of first image data respectively output by multiple second-level logical OR gates, and converts the received The multiple channels of first image data are sequentially output to the processing unit 302 .

在本申请实施例中,多路选择器3013包括多个输入端、一个输出端和一个控制端,多路选择器3013的多个输入端与开关阵列3012连接,多路选择器3013的控制端与处理单元302的控制端连接,多路选择器3013的输出端与处理单元302的输入端连接。In the embodiment of the present application, the multiplexer 3013 includes a plurality of input terminals, an output terminal and a control terminal, the multiple input terminals of the multiplexer 3013 are connected to the switch array 3012, and the control terminal of the multiplexer 3013 It is connected with the control terminal of the processing unit 302 , and the output terminal of the multiplexer 3013 is connected with the input terminal of the processing unit 302 .

其中,多路选择器3013的每个输入端用于接收开关阵列3012输出的一路第一图像数据;多路选择器3013的控制端用于在每个测量时刻接收处理单元302的控制端输出的选通信号;多路选择器3013的输出端用于在每个测量时刻输出与相应测量时刻接收到的选通信号对应的一路第一图像数据。Wherein, each input terminal of the multiplexer 3013 is used to receive one path of first image data output by the switch array 3012; Strobe signal; the output terminal of the multiplexer 3013 is used to output a path of first image data corresponding to the strobe signal received at the corresponding measurement moment at each measurement moment.

需要说明的是,多路选择器3013的输入端的数量与开关阵列3012包括的第二级逻辑或门的数量相同,这样,多路选择器3013的每个输入端连接开关阵列3012中的一个第二级逻辑或门,以接收连接的第二级逻辑或门输出的第一图像数据。It should be noted that the number of input terminals of the multiplexer 3013 is the same as the number of the second-stage logical OR gates included in the switch array 3012, so that each input terminal of the multiplexer 3013 is connected to a first-order logic gate in the switch array 3012. a second-level logical OR gate to receive the first image data output by the connected second-level logical OR gate.

其中,在一种可能的实现方式中,可以按照开关阵列3012中的每个第二级逻辑或门输出的第一图像数据所对应的宏像素在SPAD阵列3011中的位置,将多路选择器3013中的各个输入端依次与各个第二级逻辑或门进行连接。例如,用于输出SPAD阵列3011的多个宏像素中的第一行第一列的宏像素对应的第一图像数据的第二级逻辑或门可以与多路选择器3013中的第一个输入端连接,用于输出SPAD阵列3011的多个宏像素中的第一行第二列的宏像素对应的第一图像数据的第二级逻辑或门可以与多路选择器3013中的第二个输入端连接,以此类推。Wherein, in a possible implementation manner, according to the position in the SPAD array 3011 of the macro pixel corresponding to the first image data output by each second-level logical OR gate in the switch array 3012, the multiplexer Each input terminal in 3013 is connected with each second-level logical OR gate in turn. For example, the second level logical OR gate for outputting the first image data corresponding to the macropixels in the first row and first column among the plurality of macropixels of the SPAD array 3011 can be combined with the first input in the multiplexer 3013 terminal connection, the second level logical OR gate for outputting the first image data corresponding to the macropixels in the first row and the second column among the plurality of macropixels of the SPAD array 3011 can be connected with the second one in the multiplexer 3013 input connections, and so on.

在接收多路第一图像数据的同时,多路选择器3013的控制端还可以接收处理单元302的控制端按照预设时序输出的选通信号。其中,处理单元302可以按照SPAD阵列3011中的多个宏像素的位置以及多路选择器3013中用于接收每个宏像素对应的第一图像数据的输入端,在不同的测量时刻通过自身的控制端依次输出用于选通多路选择器中的不同输入端的选通信号。相应地,多路选择器3013可以通过自身的控制端在每个测量时刻接收处理单元302的控制端输出的选通信号,并根据该选通信号输出对应的输入端输入的第一图像数据。While receiving multiple channels of first image data, the control terminal of the multiplexer 3013 may also receive a gate signal output by the control terminal of the processing unit 302 according to a preset timing. Wherein, the processing unit 302 can pass the position of the plurality of macro pixels in the SPAD array 3011 and the input terminal for receiving the first image data corresponding to each macro pixel in the multiplexer 3013, through its own The control terminal sequentially outputs gating signals for gating different input terminals in the multiplexer. Correspondingly, the multiplexer 3013 can receive the gate signal output by the control terminal of the processing unit 302 at each measurement moment through its own control terminal, and output the first image data input by the corresponding input terminal according to the gate signal.

例如,处理单元确定SPAD阵列中第一行第一列的宏像素1对应的第一图像数据将输入至多路选择器的输入端1,第一行第二列的宏像素2对应的第一图像数据将输入至多路选择器的输入端2,第二行第一列的宏像素3对应的第一图像数据将输入至多路选择器的输入端3,第二行第二列的宏像素4对应的第一图像数据将输入至多路选择器的输入端4,在此基础上,在第一个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端1的选通信号0000。多路选择器的控制端在接收到选通信号0000之后,将多路选择器的输入端1输入的宏像素1的第一图像数据输出至处理单元。在第二个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端2的选通信号0001。多路选择器的控制端在接收到选通信号0001之后,将多路选择器的输入端2输入的宏像素2的第一图像数据输出至处理单元。在第三个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端3的选通信号0010,多路选择器的控制端在接收到选通信号0010之后,将多路选择器的输入端3输入的宏像素3的第一图像数据输出至处理单元。在第四个测量时刻,处理单元的控制端输出用于选通多路选择器的输入端4的选通信号0011,多路选择器的控制端在接收到选通信号0011之后,将多路选择器的输入端4输入的宏像素4的第一图像数据输出至处理单元。For example, the processing unit determines that the first image data corresponding to the macro pixel 1 in the first row and the first column in the SPAD array will be input to the input terminal 1 of the multiplexer, and the first image data corresponding to the macro pixel 2 in the first row and the second column The data will be input to the input terminal 2 of the multiplexer, the first image data corresponding to the macro pixel 3 in the second row and the first column will be input to the input terminal 3 of the multiplexer, and the macro pixel 4 in the second row and the second column corresponds to The first image data will be input to the input terminal 4 of the multiplexer, on this basis, at the first measurement moment, the control terminal of the processing unit outputs a strobe signal for gating the input terminal 1 of the multiplexer 0000. After receiving the strobe signal 0000, the control terminal of the multiplexer outputs the first image data of the macro-pixel 1 input from the input terminal 1 of the multiplexer to the processing unit. At the second measurement instant, the control terminal of the processing unit outputs a gate signal 0001 for gate input 2 of the multiplexer. After receiving the strobe signal 0001, the control terminal of the multiplexer outputs the first image data of the macro-pixel 2 input from the input terminal 2 of the multiplexer to the processing unit. At the third measurement moment, the control terminal of the processing unit outputs a gating signal 0010 for gating the input terminal 3 of the multiplexer. After receiving the gating signal 0010, the control terminal of the multiplexer switches the multiplexer The first image data of the macro pixel 3 input by the input terminal 3 of the selector is output to the processing unit. At the fourth measurement moment, the control terminal of the processing unit outputs a gating signal 0011 for gating the input terminal 4 of the multiplexer, and after receiving the gating signal 0011, the control terminal of the multiplexer switches the multiplexer The first image data of the macro pixel 4 input by the input terminal 4 of the selector is output to the processing unit.

处理单元302在接收到每个选通信号对应的第一图像数据之后,可以根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置,并将相应选通信号对应的第一图像数据作为确定出的宏像素的位置上的像素数据,以生成融合图像。After receiving the first image data corresponding to each gate signal, the processing unit 302 can determine the macro pixel corresponding to the first image data corresponding to the received corresponding gate signal according to the gate signal output at each measurement moment and using the first image data corresponding to the corresponding gate signal as the determined pixel data at the position of the macro pixel to generate a fused image.

需要说明的是,上述相邻的两个测量时刻之间可以间隔一定时长,在该时长内,处理单元302可以对接收到的第一图像数据进行处理。It should be noted that there may be a certain period of time between the above two adjacent measurement moments, and within this period of time, the processing unit 302 may process the received first image data.

示例性地,处理单元302在某个测量时刻通过自身控制端输出选通信号之后,可以接收多路选择器3013根据该选通信号输出的第一图像数据。由前述介绍可知,处理单元302在发送选通信号时是根据宏像素的位置和多路选择器3013中用于输出相应宏像素的图像数据的输入端来发送的,基于此,处理单元302可以根据在该测量时刻输出的选通信号所对应的宏像素的位置,确定接收到的第一图像数据为哪个宏像素对应的图像数据,进而将该第一图像数据作为该宏像素的位置上的像素数据。之后,处理单元302可以在下一个测量时刻到达之后,继续通过控制端向多路选择器3013发送选通信号,并对接收到的相应选通信号对应的第一图像数据进行处理。如此,处理单元302通过接收并处理多路选择器依次输出的多路第一图像数据,得到各个宏像素的位置上的像素数据,从而生成融合图像。Exemplarily, after the processing unit 302 outputs a gating signal through its own control terminal at a certain measurement moment, it may receive the first image data output by the multiplexer 3013 according to the gating signal. It can be seen from the foregoing introduction that when the processing unit 302 sends the strobe signal, it sends the strobe signal according to the position of the macro pixel and the input terminal for outputting the image data of the corresponding macro pixel in the multiplexer 3013. Based on this, the processing unit 302 can According to the position of the macro pixel corresponding to the strobe signal output at the measurement moment, determine which macro pixel the received first image data corresponds to, and then use the first image data as the position of the macro pixel. pixel data. Afterwards, the processing unit 302 may continue to send the gate signal to the multiplexer 3013 through the control terminal after the next measurement time arrives, and process the first image data corresponding to the received corresponding gate signal. In this way, the processing unit 302 receives and processes the multiple channels of first image data sequentially output by the multiplexer to obtain pixel data at the position of each macro pixel, thereby generating a fused image.

为了更好的理解本申请实施例的装置,现以一个3×3的SPAD阵列为例对本申请实施例提供的成像装置进行详细说明,其中,参见图7,该SPAD阵列包括9个SPAD子像素,分别为SPAD1至SPAD9。另外,该SPAD阵列中的每个宏像素包括2×2个SPAD子像素。其中,SPAD1、SPAD2、SPAD4、SPAD5组成宏像素1,SPAD2、SPAD3、SPAD5、SPAD6组成宏像素2,SPAD4、SPAD5、SPAD7、SPAD8组成宏像素3,SPAD5、SPAD6、SPAD8、SPAD9组成宏像素4。宏像素1和宏像素2中重叠的SPAD子像素为SPAD2和SPAD5,宏像素3和宏像素4中重叠的SPAD子像素为SPAD5和SPAD8,宏像素1和宏像素3中重叠的SPAD子像素为SPAD4和SPAD5,宏像素2和宏像素4中重叠的SPAD子像素为SPAD5和SPAD6。In order to better understand the device of the embodiment of the present application, a 3×3 SPAD array is taken as an example to describe the imaging device provided by the embodiment of the present application in detail, wherein, referring to FIG. 7, the SPAD array includes 9 SPAD sub-pixels , SPAD1 to SPAD9, respectively. In addition, each macro pixel in the SPAD array includes 2×2 SPAD sub-pixels. Among them, SPAD1, SPAD2, SPAD4, SPAD5 form macro pixel 1, SPAD2, SPAD3, SPAD5, SPAD6 form macro pixel 2, SPAD4, SPAD5, SPAD7, SPAD8 form macro pixel 3, and SPAD5, SPAD6, SPAD8, SPAD9 form macro pixel 4. The overlapping SPAD sub-pixels in macro pixel 1 and macro pixel 2 are SPAD2 and SPAD5, the overlapping SPAD sub-pixels in macro pixel 3 and macro pixel 4 are SPAD5 and SPAD8, and the overlapping SPAD sub-pixels in macro pixel 1 and macro pixel 3 are SPAD4 and SPAD5 , the overlapping SPAD sub-pixels in macro pixel 2 and macro pixel 4 are SPAD5 and SPAD6.

上述4个宏像素中每个宏像素的一行SPAD子像素连接一个第一级逻辑或门,这样,每个宏像素的一行SPAD子像素中的两个SPAD子像素将输出两路子像素数据至连接的第一级逻辑或门,例如,宏像素1的第一行的SPAD1、SPAD2子像素连接一个第一级逻辑或门,这样,SPAD1输出子像素数据1至连接的第一级逻辑或门,SPDA2输出子像素数据2至连接的第一级逻辑或门,该第一级逻辑或门在接收到子像素数据1和2之后,将两路子像素数据合成为一路行像素数据1+2,并输出该行像素数据1+2。对于其他宏像素中的各行SPAD子像素,每行SPAD子像素均连接有一个第一级逻辑或门,这样,通过6个第一级逻辑或门,将输出6个行像素数据。另外,每个宏像素中的多行SPAD子像素连接的多个第一级逻辑或门将连接一个第二级逻辑或门,例如,宏像素1的第一行的SPAD1和SPAD2两个子像素连接的第一级逻辑或门以及第二行的SPAD4和SPAD5两个子像素连接的第一级逻辑或门将连接一个第二级逻辑或门,这样,这个第二级逻辑或门将接收到行像素数据1+2和行像素数据4+5,之后,该第二级逻辑或门将行像素数据1+2和行像素数据4+5合成为第一图像数据1245。对于其他宏像素的各行SPAD子像素连接的第一级逻辑或门,也均可以连接一个第二级逻辑或门,如此,每个第二级逻辑或门可以连接一个宏像素中的两行SPAD子像素对应的两个第一级逻辑或门,并将连接的两个第一级逻辑或门输出的两个行像素数据合成为一路第一图像数据,从而得到四路第一图像数据,分别为第一图像数据1245、第一图像数据4578、第一图像数据2356和第一图像数据5689。A row of SPAD subpixels of each of the above four macropixels is connected to a first-level logical OR gate, so that two SPAD subpixels in a row of SPAD subpixels of each macropixel will output two channels of subpixel data to the connection For example, the SPAD1 and SPAD2 sub-pixels of the first row of the macro pixel 1 are connected to a first-level logical OR gate, so that SPAD1 outputs sub-pixel data 1 to the connected first-level logical OR gate, SPDA2 outputs the sub-pixel data 2 to the connected first-level logic OR gate, and after receiving the sub-pixel data 1 and 2, the first-level logic OR gate synthesizes the two-way sub-pixel data into one line of pixel data 1+2, and Output the row of pixel data 1+2. For each row of SPAD sub-pixels in other macro pixels, each row of SPAD sub-pixels is connected with a first-level logic OR gate, so that 6 rows of pixel data will be output through 6 first-level logic OR gates. In addition, multiple first-level logic OR gates connected to multiple rows of SPAD sub-pixels in each macro pixel will be connected to a second-level logic OR gate, for example, the two sub-pixels of SPAD1 and SPAD2 in the first row of macro pixel 1 are connected The first-level logic OR gate and the first-level logic OR gate connected to the two subpixels of SPAD4 and SPAD5 in the second row will be connected to a second-level logic OR gate, so that this second-level logic OR gate will receive row pixel data 1+ 2 and row pixel data 4+5, and then the second-level logical OR gate synthesizes row pixel data 1+2 and row pixel data 4+5 into first image data 1245. For the first-level logical OR gate connected to each row of SPAD sub-pixels of other macro pixels, a second-level logical OR gate can also be connected, so that each second-level logical OR gate can be connected to two rows of SPAD in a macro pixel The two first-level logic OR gates corresponding to the sub-pixels, and the two row pixel data output by the connected two first-level logic OR gates are synthesized into one path of first image data, thereby obtaining four paths of first image data, respectively These are the first image data 1245 , the first image data 4578 , the first image data 2356 and the first image data 5689 .

每个第二级逻辑或门可以将自身得到的一路第一图像数据输出至多路选择器,多路选择器可以根据接收到的选通信号按顺序依次将各路第一图像数据输出至处理单元。Each second-level logical OR gate can output one channel of first image data obtained by itself to the multiplexer, and the multiplexer can sequentially output each channel of first image data to the processing unit according to the received strobe signal .

其中,宏像素1对应的第二级逻辑或门可以连接多路选择器的输入端1,宏像素2对应的第二级逻辑或门可以连接多路选择器的输入端2,宏像素3对应的第二级逻辑或门连接多路选择器的输入端3,宏像素4对应的第二级逻辑或门连接多路选择器的输入端4。相应地,处理单元的控制端可以依次输出选通信号00、01、10和11。多路选择器在接收到选通信号00时,输出第一图像数据1245,处理单元在接收到第一图像数据1245之后,将第一图像数据1245作为宏像素1所在位置处的像素数据。在接收到选通信号01时,输出第一图像数据2356,处理单元在接收到第一图像数据2356之后,将第一图像数据2356作为宏像素2所在位置处的像素数据。以此类推,最终得到的融合图像如图8所示。Among them, the second level logic OR gate corresponding to macro pixel 1 can be connected to the input terminal 1 of the multiplexer, the second level logic OR gate corresponding to macro pixel 2 can be connected to the input terminal 2 of the multiplexer, and the macro pixel 3 corresponds to The second level logical OR gate of the macro pixel 4 is connected to the input terminal 3 of the multiplexer, and the second level logical OR gate corresponding to the macro pixel 4 is connected to the input terminal 4 of the multiplexer. Correspondingly, the control terminal of the processing unit may output strobe signals 00, 01, 10 and 11 in sequence. When the multiplexer receives the strobe signal 00, it outputs the first image data 1245. After receiving the first image data 1245, the processing unit uses the first image data 1245 as the pixel data at the position of the macro pixel 1. When the gate signal 01 is received, the first image data 2356 is output, and the processing unit takes the first image data 2356 as the pixel data at the position of the macro pixel 2 after receiving the first image data 2356 . By analogy, the final fused image is shown in Figure 8.

在本申请实施例中,通过成像单元中的开关阵列和多路选择器可以依次输出多个宏像素中每个宏像素对应的一路第一图像数据,处理单元接收每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。由于成像单元中SPAD阵列包括的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素,因此,通过开关阵列和多路选择器使得成像单元输出的宏像素的图像数据的数量增加,而该SPAD阵列的TDC和直方图计算电路均未增加,由此可见,本申请实施例提供的图像处理方法能够在SPAD子像素个数、后续存储、主要电路面积不增加的情况下,提高了SPAD阵列的输出图像的平面分辨率。In the embodiment of the present application, through the switch array and the multiplexer in the imaging unit, one channel of first image data corresponding to each macro pixel in the plurality of macro pixels can be sequentially output, and the processing unit receives the first image data corresponding to each macro pixel. image data, and generate a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel. Since there are overlapping SPAD sub-pixels between every two adjacent macro-pixels in the multiple macro-pixels included in the SPAD array in the imaging unit, the image data of the macro-pixels output by the imaging unit are output by the switch array and the multiplexer. The number increases, but the TDC and histogram calculation circuits of the SPAD array do not increase. It can be seen that the image processing method provided by the embodiment of the present application can be used without increasing the number of SPAD sub-pixels, subsequent storage, and main circuit area. Next, the planar resolution of the output image of the SPAD array is improved.

接下来,对本申请实施例提供的成像方法进行介绍。Next, the imaging method provided in the embodiment of the present application is introduced.

图9是本申请实施例提供的一种成像控制方法的流程图,该方法应用于前述实施例介绍的成像装置的处理单元中,该成像装置还包括成像单元,成像单元包括单光子雪崩二极管SPAD阵列,SPAD阵列包括多个宏像素,多个宏像素中的每个宏像素包括(N×M)个SPAD子像素,且每相邻的两个宏像素之间具有重叠的SPAD子像素,其中,开关阵列用于接收多个宏像素中每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据,对每个宏像素包括的(N×M)个SPAD子像素分别输出的子像素数据进行合成,得到相应宏像素对应的一路第一图像数据,输出每个宏像素对应的一路第一图像数据至多路选择器,该方法包括以下步骤:Fig. 9 is a flow chart of an imaging control method provided by an embodiment of the present application, the method is applied to the processing unit of the imaging device introduced in the foregoing embodiments, the imaging device also includes an imaging unit, and the imaging unit includes a single photon avalanche diode SPAD Array, the SPAD array includes a plurality of macro pixels, each macro pixel in the plurality of macro pixels includes (N×M) SPAD sub-pixels, and there are overlapping SPAD sub-pixels between every two adjacent macro pixels, wherein , the switch array is used to receive sub-pixel data respectively output by (N×M) SPAD sub-pixels included in each macro-pixel in a plurality of macro-pixels, and the (N×M) SPAD sub-pixels included in each macro-pixel are respectively The output sub-pixel data is synthesized to obtain one path of first image data corresponding to the corresponding macro pixel, and one path of first image data corresponding to each macro pixel is output to the multiplexer. The method includes the following steps:

步骤901:处理单元按照预设时序向多路选择器依次输出不同种类的选通信号,以使多路选择器在每接收到处理单元输出的一种选通信号时,输出与相应选通信号对应的第一图像数据。Step 901: The processing unit sequentially outputs different types of strobe signals to the multiplexer according to the preset timing, so that the multiplexer outputs a corresponding strobe signal every time it receives a strobe signal output by the processing unit. corresponding to the first image data.

其中,处理单元按照预设时序依次输出不同种类的选通信号的实现过程可以参考前述实施例中介绍的实现方式,本申请实施例在此不再赘述。Wherein, the implementation process of the processing unit sequentially outputting different types of strobe signals according to the preset time sequence can refer to the implementation manners introduced in the foregoing embodiments, and the embodiments of the present application will not repeat them here.

步骤902:处理单元接收多路选择器依次输出的每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。Step 902: The processing unit receives the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.

其中,多路选择器根据处理单元发送的选通信号输出对应的第一图像数据的过程可以参考前述实施例中的实现方式。另外,处理单元生成融合图像的过程也可以参考前述实现方式,本申请实施例在此不再赘述。Wherein, the process of outputting the corresponding first image data by the multiplexer according to the strobe signal sent by the processing unit may refer to the implementation manners in the foregoing embodiments. In addition, the process of generating the fused image by the processing unit may also refer to the foregoing implementation manners, which will not be repeated in this embodiment of the present application.

可选地,开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一行SPAD子像素连接一个第一级逻辑或门,第一宏像素中多行SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,第一宏像素为多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a row of SPAD sub-pixels in the first macro pixel is connected to a first-level logical OR gate, and multiple rows of SPAD sub-pixels in the first macro pixel A plurality of connected first-level logical OR gates are connected to a second-level logical OR gate, and the first macro pixel is any one of the plurality of macro pixels;

其中,第一宏像素连接的每个第一级逻辑或门接收自身连接的一行SPAD子像素中每个SPAD子像素输出的一路子像素数据,将接收到的子像素数据合成为一路行像素数据,输出行像素数据;Wherein, each first-stage logical OR gate connected to the first macro pixel receives a road of sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected to itself, and synthesizes the received sub-pixel data into a road of pixel data , output row pixel data;

第一宏像素连接的第二级逻辑或门接收自身连接的多个第一级逻辑或门分别输出的行像素数据,将接收到的行像素数据进行合成,得到第一宏像素对应的第一图像数据,输出第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel receives row pixel data respectively outputted by multiple first-level logical OR gates connected to itself, and synthesizes the received row pixel data to obtain the first pixel corresponding to the first macro pixel. Image data, outputting first image data corresponding to the first macro pixel.

可选地,开关阵列包括第一级逻辑或门和第二级逻辑或门,第一宏像素中的一列SPAD子像素连接一个第一级逻辑或门,第一宏像素中多列SPAD子像素连接的多个第一级逻辑或门连接一个第二级逻辑或门,第一宏像素为多个宏像素中的任一个宏像素;Optionally, the switch array includes a first-level logical OR gate and a second-level logical OR gate, a column of SPAD sub-pixels in the first macro-pixel is connected to a first-level logical OR gate, and multiple columns of SPAD sub-pixels in the first macro-pixel A plurality of connected first-level logical OR gates are connected to a second-level logical OR gate, and the first macro pixel is any one of the plurality of macro pixels;

第一宏像素连接的每个第一级逻辑或门用于接收连接的一列SPAD子像素中每个SPAD子像素输出的一路子像素数据,并将接收到的子像素数据合成为一路列像素数据,输出列像素数据;Each first-level logical OR gate connected to the first macro-pixel is used to receive a channel of sub-pixel data output by each SPAD sub-pixel in a connected column of SPAD sub-pixels, and synthesize the received sub-pixel data into a column of pixel data , output column pixel data;

第一宏像素连接的第二级逻辑或门用于接收连接的多个第一级逻辑或门分别输出的列像素数据,将接收到的列像素数据进行合成,得到第一宏像素对应的第一图像数据,输出第一宏像素对应的第一图像数据。The second-level logical OR gate connected to the first macro pixel is used to receive column pixel data respectively output by the connected multiple first-level logical OR gates, and synthesize the received column pixel data to obtain the first macro pixel corresponding to the first macro pixel. Image data, outputting the first image data corresponding to the first macro pixel.

可选地,多路选择器包括多个输入端、一个输出端和一个控制端,多路选择器的多个输入端与开关阵列连接,多路选择器的控制端与处理单元的控制端连接,多路选择器的输出端与处理单元的输入端连接,多路选择器的每个输入端接收开关阵列输出的一路第一图像数据;Optionally, the multiplexer includes multiple input terminals, an output terminal and a control terminal, multiple input terminals of the multiplexer are connected to the switch array, and the control terminal of the multiplexer is connected to the control terminal of the processing unit , the output end of the multiplexer is connected to the input end of the processing unit, and each input end of the multiplexer receives one path of first image data output by the switch array;

处理单元按照预设时序向多路选择器依次输出不同种类的选通信号,包括:The processing unit sequentially outputs different types of strobe signals to the multiplexer according to the preset timing, including:

处理单元的控制端在第一测量时刻向多路选择器的控制端输出的第一选通信号,以使多路选择器的输出端在第一测量时刻输出与第一选通信号对应的一路第一图像数据,第一测量时刻为多个测量时刻中的任一测量时刻。The control terminal of the processing unit outputs the first strobe signal to the control terminal of the multiplexer at the first measurement moment, so that the output terminal of the multiplexer outputs one channel corresponding to the first strobe signal at the first measurement moment. For the first image data, the first measurement moment is any one of the plurality of measurement moments.

可选地,处理单元根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像,包括:Optionally, the processing unit generates a fused image according to the first image data corresponding to each macropixel and the position of the corresponding macropixel, including:

根据每个测量时刻输出的选通信号,确定接收到的相应选通信号对应的第一图像数据所对应的宏像素的位置;According to the gate signal output at each measurement moment, determine the position of the macro pixel corresponding to the first image data corresponding to the received corresponding gate signal;

并将相应选通信号对应的第一图像数据作为相应测量时刻确定出的宏像素的位置上的像素值,以生成融合图像。The first image data corresponding to the corresponding gate signal is used as the pixel value at the position of the macro pixel determined at the corresponding measurement time, so as to generate a fused image.

可选地,多个宏像素中每相邻的两个宏像素具有(N-1)行重叠的SPAD子像素,或者,多个宏像素中每相邻的两个宏像素具有(M-1)列重叠的SPAD子像素。Optionally, every two adjacent macro pixels in the multiple macro pixels have (N-1) rows of overlapping SPAD sub-pixels, or, every two adjacent macro pixels in the multiple macro pixels have (M-1 ) columns of overlapping SPAD sub-pixels.

需要说明的是,上述各个步骤的实现方式可以参考前述实施例中关于成像装置的实现方式,本申请实施例在此不再赘述。It should be noted that, for implementation manners of the foregoing steps, reference may be made to the implementation manners of the imaging device in the foregoing embodiments, and details are not repeated in this embodiment of the present application.

综上所述,在本申请实施例中,通过成像单元中的开关阵列和多路选择器可以依次输出多个宏像素中每个宏像素对应的一路第一图像数据,处理单元接收每个宏像素对应的第一图像数据,并根据每个宏像素对应的第一图像数据和相应宏像素的位置,生成融合图像。由于成像单元中SPAD阵列包括的多个宏像素中每相邻的两个宏像素之间具有重叠的SPAD子像素,因此,通过开关阵列和多路选择器使得成像单元输出的宏像素的图像数据的数量增加,而该SPAD阵列的TDC和直方图计算电路均未增加,由此可见,本申请实施例提供的图像处理方法能够在SPAD子像素个数、后续存储、主要电路面积不增加的情况下,提高了SPAD阵列的输出图像的平面分辨率。To sum up, in the embodiment of the present application, through the switch array and multiplexer in the imaging unit, one channel of first image data corresponding to each macro pixel among the multiple macro pixels can be sequentially output, and the processing unit receives each macro pixel The pixel corresponds to the first image data, and according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel, a fused image is generated. Since there are overlapping SPAD sub-pixels between every two adjacent macro-pixels in the multiple macro-pixels included in the SPAD array in the imaging unit, the image data of the macro-pixels output by the imaging unit are output by the switch array and the multiplexer. The number increases, but the TDC and histogram calculation circuits of the SPAD array do not increase. It can be seen that the image processing method provided by the embodiment of the present application can be used without increasing the number of SPAD sub-pixels, subsequent storage, and main circuit area. Next, the planar resolution of the output image of the SPAD array is improved.

本申请实施例还提供了一种计算机可读存储介质,该存储介质内存储有计算机程序,该计算机程序被前述成像装置执行时可以实现图9所示的成像控制方法。The embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by the aforementioned imaging device, the imaging control method shown in FIG. 9 can be realized.

本申请实施例还提供了一种包含指令的计算机程序产品,当其在前述的成像装置上运行时,使得成像装置执行上述图9所示实施例提供的成像控制方法。The embodiment of the present application also provides a computer program product including instructions, which, when running on the aforementioned imaging device, causes the imaging device to execute the imaging control method provided in the embodiment shown in FIG. 9 above.

以上所述并不用以限制本申请实施例,凡在本申请实施例的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请实施例的保护范围之内。The above description is not intended to limit the embodiments of the present application, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included within the scope of protection of the embodiments of the present application.

Claims (12)

1. An image forming apparatus, characterized in that the image forming apparatus comprises: the device comprises an imaging unit and a processing unit, wherein the imaging unit comprises a single photon avalanche diode SPAD array, a switch array and a multiplexer, the SPAD array comprises a plurality of macro pixels, each macro pixel in the plurality of macro pixels comprises (N multiplied by M) SPAD sub-pixels, and each two adjacent macro pixels have overlapped SPAD sub-pixels;
the switch array is used for receiving sub-pixel data respectively output by (N multiplied by M) SPAD sub-pixels included by each macro-pixel in the macro-pixels, synthesizing the sub-pixel data respectively output by (N multiplied by M) SPAD sub-pixels included by each macro-pixel to obtain a path of first image data corresponding to the corresponding macro-pixel, and outputting a path of first image data corresponding to each macro-pixel;
the multiplexer is used for receiving the multiple paths of first image data output by the switch array and sequentially outputting the multiple paths of first image data to the processing unit;
the processing unit is used for receiving the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generating a fusion image according to the position of each macro pixel, the corresponding first image data and the corresponding macro pixel.
2. The imaging device of claim 1, wherein the switch array comprises a first stage logical OR gate and a second stage logical OR gate, one row of SPAD sub-pixels in a first macro-pixel is connected with one first stage logical OR gate, a plurality of first stage logical OR gates connected with a plurality of rows of SPAD sub-pixels in the first macro-pixel are connected with one second stage logical OR gate, and the first macro-pixel is any one macro-pixel in the plurality of macro-pixels;
each first-stage logic or gate connected with the first macro pixel is used for receiving sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected with the first macro pixel, synthesizing the received sub-pixel data into a row of pixel data, and outputting the row of pixel data;
the second-level logic or gate connected with the first macro pixel is used for receiving row pixel data respectively output by the plurality of first-level logic or gates connected with the second macro pixel, synthesizing the received row pixel data to obtain first image data corresponding to the first macro pixel, and outputting the first image data corresponding to the first macro pixel.
3. The imaging device of claim 1, wherein the switch array comprises a first stage logical OR gate and a second stage logical OR gate, one column of SPAD sub-pixels in the first macro-pixel is connected with one first stage logical OR gate, a plurality of first stage logical OR gates connected with a plurality of columns of SPAD sub-pixels in the first macro-pixel are connected with one second stage logical OR gate, and the first macro-pixel is any one macro-pixel in the plurality of macro-pixels;
each first-level logic or gate connected with the first macro-pixel is used for receiving one path of sub-pixel data output by each SPAD sub-pixel in a connected row of SPAD sub-pixels, synthesizing the received sub-pixel data into one path of row pixel data and outputting the row of pixel data;
the second-level logic or gate connected with the first macro-pixel is used for receiving column pixel data respectively output by the plurality of first-level logic or gates connected with the first macro-pixel, synthesizing the received column pixel data to obtain first image data corresponding to the first macro-pixel, and outputting the first image data corresponding to the first macro-pixel.
4. The imaging apparatus of claim 1, wherein the multiplexer includes a plurality of input terminals, an output terminal, and a control terminal, the plurality of input terminals of the multiplexer being connected to the switch array, the control terminal of the multiplexer being connected to the control terminal of the processing unit, the output terminal of the multiplexer being connected to the input terminal of the processing unit;
each input end of the multiplexer is used for receiving one path of first image data output by the switch array;
the control end of the multiplexer is used for receiving the gating signal output by the control end of the processing unit at each measuring moment;
and the output end of the multiplexer is used for outputting one path of first image data corresponding to the gating signal received at the corresponding measuring moment at each measuring moment.
5. The imaging apparatus according to claim 4, wherein the processing unit is configured to determine, according to the gating signal output at each measurement time, a position of a macro pixel corresponding to the first image data corresponding to the received corresponding gating signal, and use the first image data corresponding to the corresponding gating signal as the pixel data at the determined position of the macro pixel to generate the fused image.
6. The imaging device of any of claims 1-5, wherein each adjacent two of the plurality of macropixels have (N-1) rows of overlapping SPAD subpixels, or wherein each adjacent two of the plurality of macropixels have (M-1) columns of overlapping SPAD subpixels.
7. An imaging control method is applied to a processing unit of an imaging device, the imaging device further comprises an imaging unit, the imaging unit comprises a single-photon avalanche diode SPAD array, a switch array and a multiplexer, the SPAD array comprises a plurality of macro pixels, each macro pixel in the macro pixels comprises (N × M) SPAD sub-pixels, and overlapped SPAD sub-pixels are arranged between every two adjacent macro pixels, wherein the switch array is used for receiving sub-pixel data respectively output by the (N × M) SPAD sub-pixels included by each macro pixel in the macro pixels, synthesizing the sub-pixel data respectively output by the (N × M) SPAD sub-pixels included by each macro pixel to obtain a path of first image data corresponding to the corresponding macro pixel, and outputting the path of first image data corresponding to each macro pixel to the multiplexer; the method comprises the following steps:
the processing unit sequentially outputs different types of gating signals to the multiplexer according to a preset time sequence, so that the multiplexer outputs first image data corresponding to the corresponding gating signals when receiving one type of gating signal output by the processing unit;
and the processing unit receives the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generates a fusion image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.
8. The method of claim 7, wherein the switch array comprises a first-stage logical OR gate and a second-stage logical OR gate, one row of SPAD sub-pixels in the first macro-pixel is connected with one first-stage logical OR gate, a plurality of first-stage logical OR gates connected with a plurality of rows of SPAD sub-pixels in the first macro-pixel are connected with one second-stage logical OR gate, and the first macro-pixel is any one of the plurality of macro-pixels;
each first-stage logic or gate connected with the first macro-pixel is used for receiving sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected with the first macro-pixel, synthesizing the received sub-pixel data into a row of pixel data, and outputting the row of pixel data;
the second-level logic or gate connected with the first macro pixel is used for receiving row pixel data respectively output by a plurality of first-level logic or gates connected with the second macro pixel, synthesizing the received row pixel data to obtain first image data corresponding to the first macro pixel, and outputting the first image data corresponding to the first macro pixel.
9. The method of claim 7, wherein the switch array comprises a first stage logical OR gate and a second stage logical OR gate, one column of SPAD sub-pixels in the first macro-pixel is connected with one first stage logical OR gate, a plurality of first stage logical OR gates connected with a plurality of columns of SPAD sub-pixels in the first macro-pixel are connected with one second stage logical OR gate, and the first macro-pixel is any one macro-pixel in the plurality of macro-pixels;
each first-stage logic or gate connected with the first macro pixel is used for receiving one path of sub-pixel data output by each SPAD sub-pixel in a connected line of SPAD sub-pixels, synthesizing the received sub-pixel data into one line of pixel data and outputting the line of pixel data;
the second-level logic or gate connected with the first macro-pixel is used for receiving column pixel data respectively output by the plurality of first-level logic or gates connected with the first macro-pixel, synthesizing the received column pixel data to obtain first image data corresponding to the first macro-pixel, and outputting the first image data corresponding to the first macro-pixel.
10. The method according to claim 7, wherein the multiplexer comprises a plurality of input terminals, an output terminal and a control terminal, the plurality of input terminals of the multiplexer are connected to the switch array, the control terminal of the multiplexer is connected to the control terminal of the processing unit, the output terminal of the multiplexer is connected to the input terminal of the processing unit, and each input terminal of the multiplexer is configured to receive a path of the first image data output by the switch array;
the processing unit sequentially outputs different kinds of gating signals to the multiplexer according to a preset time sequence, and the gating signals comprise:
the control end of the processing unit outputs a first gating signal to the control end of the multiplexer at a first measurement time, so that the output end of the multiplexer outputs one path of first image data corresponding to the first gating signal at the first measurement time, and the first measurement time is any one of a plurality of measurement times.
11. The method of claim 10, wherein the processing unit generates the fused image according to the first image data corresponding to each macro-pixel and the position of the corresponding macro-pixel, and comprises:
determining the position of a macro pixel corresponding to first image data corresponding to the received corresponding gating signal according to the gating signal output at each measuring moment;
and taking the first image data corresponding to the corresponding gating signal as pixel data at the position of the macro pixel determined at the corresponding measuring moment to generate the fused image.
12. The method of any of claims 7-11, wherein each adjacent two of the macropixels have (N-1) rows of overlapping SPAD subpixels, or wherein each adjacent two of the macropixels have (M-1) columns of overlapping SPAD subpixels.
CN202110791480.8A 2021-07-13 2021-07-13 Imaging device and imaging control method Pending CN115696077A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110791480.8A CN115696077A (en) 2021-07-13 2021-07-13 Imaging device and imaging control method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110791480.8A CN115696077A (en) 2021-07-13 2021-07-13 Imaging device and imaging control method

Publications (1)

Publication Number Publication Date
CN115696077A true CN115696077A (en) 2023-02-03

Family

ID=85044434

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110791480.8A Pending CN115696077A (en) 2021-07-13 2021-07-13 Imaging device and imaging control method

Country Status (1)

Country Link
CN (1) CN115696077A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024199986A1 (en) * 2023-03-24 2024-10-03 Sony Semiconductor Solutions Corporation Image sensor control circuitry, image sensor, image sensor control method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109799496A (en) * 2018-11-14 2019-05-24 深圳市光微科技有限公司 Concurrency detection circuit, photon detector, pulsed TOF sensor and its implementation
CN109932852A (en) * 2019-04-28 2019-06-25 清华大学深圳研究生院 One kind making the maximized method of light-field camera sensor utilization
CN111624615A (en) * 2019-02-28 2020-09-04 意法半导体(R&D)有限公司 Optical sensor
CN111771168A (en) * 2018-02-22 2020-10-13 Imec 非营利协会 Optical devices, systems and methods for forming three-dimensional light field distributions
US20210208257A1 (en) * 2020-01-07 2021-07-08 Liturex (Guangzhou) Co. Ltd Spad array with ambient light suppression for solid-state lidar

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111771168A (en) * 2018-02-22 2020-10-13 Imec 非营利协会 Optical devices, systems and methods for forming three-dimensional light field distributions
CN109799496A (en) * 2018-11-14 2019-05-24 深圳市光微科技有限公司 Concurrency detection circuit, photon detector, pulsed TOF sensor and its implementation
CN111624615A (en) * 2019-02-28 2020-09-04 意法半导体(R&D)有限公司 Optical sensor
CN109932852A (en) * 2019-04-28 2019-06-25 清华大学深圳研究生院 One kind making the maximized method of light-field camera sensor utilization
US20210208257A1 (en) * 2020-01-07 2021-07-08 Liturex (Guangzhou) Co. Ltd Spad array with ambient light suppression for solid-state lidar

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
康岩 等: "基于像素复用的SPAD阵列连续扫描三维成像", 红外与激光工程, 30 December 2020 (2020-12-30) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024199986A1 (en) * 2023-03-24 2024-10-03 Sony Semiconductor Solutions Corporation Image sensor control circuitry, image sensor, image sensor control method

Similar Documents

Publication Publication Date Title
US11997397B2 (en) Method, apparatus, and device for processing images, and storage medium
KR102354260B1 (en) Method and device for processing lightfield data
US10484627B2 (en) Image sensor with improved micro-lens, related imaging method and electronic device
US11107392B2 (en) Display panel and electronic device
US11037968B2 (en) Image sensor architecture
US10671218B2 (en) Floating touch control panel, touch control method of the same, and display device
US20110202310A1 (en) Depth sensor, depth estimation method using the same, and depth estimation device including the same
US20250117640A1 (en) High bandwidth camera data transmission
US11328645B2 (en) Display control method and device for N-primary-color display panel, and display device
JP7450809B2 (en) DTOF detection module, terminal device, and ranging method
CN115862515B (en) Source driving circuit, display device, electronic device and driving chip
US20160372020A1 (en) Display substrate and method for driving the same, and display apparatus
US20190096360A1 (en) Display method, image processing device, display device, and storage medium
CN115696077A (en) Imaging device and imaging control method
US20230204741A1 (en) Ranging method and device, storage medium, and lidar
WO2023060938A1 (en) Debugging method for display panel, display device, and storage medium
KR20150063010A (en) Method and device for estimating disparity associated with views of a scene acquired with a plenoptic camera
KR102211862B1 (en) Image sensor and image sensor system including the same
US11011095B2 (en) Display panel, and image control device and method thereof
US20210011167A1 (en) Solid-state imaging device, solid-state imaging system, and method of driving the solid-state imaging device
US10127004B2 (en) Display device and display signal input system and display signal input method thereof
US20230132616A1 (en) Laser radar system and detection method
WO2019085489A1 (en) Display screen, pixel driving method, and display device
WO2020224426A1 (en) Fingerprint sensor, fingerprint module, and terminal device
WO2020186509A1 (en) A scalable data fusion architecture and related products

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination