CN115696077A

CN115696077A - Imaging apparatus and imaging control method

Info

Publication number: CN115696077A
Application number: CN202110791480.8A
Authority: CN
Inventors: 苏星; 程卫亮; 张泽鑫; 沈林杰
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2021-07-13
Filing date: 2021-07-13
Publication date: 2023-02-03

Abstract

The embodiment of the application discloses an imaging device and an imaging control method, and belongs to the technical field of photoelectric detection. In the embodiment of the application, a switch array and a multiplexer in an imaging unit may sequentially output one path of first image data corresponding to each macro pixel in a plurality of macro pixels, and a processing unit receives the first image data corresponding to each macro pixel and generates a fused image according to the first image data corresponding to each macro pixel and a position of the corresponding macro pixel. Because the overlapped SPAD sub-pixels are arranged between every two adjacent macro-pixels in the plurality of macro-pixels included in the SPAD array in the imaging unit, the quantity of image data corresponding to the macro-pixels output by the imaging unit is increased through the switch array and the multiplexer, and therefore the plane resolution of an output image is improved under the condition that the quantity of the SPAD sub-pixels and subsequent storage are not increased.

Description

Imaging apparatus and imaging control method

Technical Field

The present disclosure relates to the field of photoelectric detection technologies, and in particular, to an imaging device and an imaging control method.

Background

The 3D imaging technology based on the SPAD (Single Photon Avalanche Diode) array is a research hotspot in recent years, and has great application prospects in the fields of consumer electronics, security, robots, automatic driving and the like.

In the related art, in order to save circuit area, n SPAD sub-pixels spatially adjacent to each other in the SPAD array are divided into one macro-pixel, and the n SPAD sub-pixels in each macro-pixel share the same set of processing circuits, so that each macro-pixel corresponds to one pixel data. Accordingly, the resolution of the finally output image will also be reduced to 1/n of the resolution before the macro-pixels are not divided. For example, referring to fig. 1, while the spatially adjacent 2*2 SPAD subpixels are divided into one macropixel and one pixel data is output, in this case, when 320 × 240 SPAD subpixels are included in the SPAD array, the resolution of the output image will be 160 × 120, and thus it can be seen that the imaging method in the related art causes a large loss in planar resolution of the output image of the SPAD array.

Disclosure of Invention

The embodiment of the application provides an imaging device and an imaging control method, which can improve the plane resolution on the basis of unchanged number of sub-pixels and follow-up circuits of an SPAD array. The technical scheme is as follows:

in one aspect, there is provided an image forming apparatus including: the device comprises an imaging unit and a processing unit, wherein the imaging unit comprises a single photon avalanche diode SPAD array, a switch array and a multiplexer, the SPAD array comprises a plurality of macro-pixels, each macro-pixel in the macro-pixels comprises (N multiplied by M) SPAD sub-pixels, and each two adjacent macro-pixels are provided with overlapped SPAD sub-pixels;

the switch array is used for receiving sub-pixel data respectively output by (NxM) SPAD sub-pixels included by each macro-pixel in the macro-pixels, synthesizing the sub-pixel data respectively output by (NxM) SPAD sub-pixels included by each macro-pixel to obtain a path of first image data corresponding to the corresponding macro-pixel, and outputting a path of first image data corresponding to each macro-pixel;

the multiplexer is used for receiving the multiple paths of first image data output by the switch array and sequentially outputting the multiple paths of first image data to the processing unit;

the processing unit is used for receiving the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generating a fusion image according to the position of each macro pixel, the corresponding first image data and the corresponding macro pixel.

Optionally, the switch array includes a first-stage logical or gate and a second-stage logical or gate, a row of SPAD subpixels in the first macro pixel is connected with one first-stage logical or gate, a plurality of first-stage logical or gates connected with a plurality of rows of SPAD subpixels in the first macro pixel are connected with one second-stage logical or gate, and the first macro pixel is any one of the macro pixels;

each first-stage logic or gate connected with the first macro pixel is used for receiving sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected with the first macro pixel, synthesizing the received sub-pixel data into a row of pixel data, and outputting the row of pixel data;

the second-level logic or gate connected with the first macro pixel is used for receiving row pixel data respectively output by the plurality of first-level logic or gates connected with the second macro pixel, synthesizing the received row pixel data to obtain first image data corresponding to the first macro pixel, and outputting the first image data corresponding to the first macro pixel.

Optionally, the switch array includes a first-stage logical or gate and a second-stage logical or gate, a column of SPAD sub-pixels in the first macro-pixel is connected with a first-stage logical or gate, a plurality of first-stage logical or gates connected with a plurality of columns of SPAD sub-pixels in the first macro-pixel are connected with a second-stage logical or gate, and the first macro-pixel is any one of the macro-pixels;

each first-stage logic or gate connected with the first macro pixel is used for receiving one path of sub-pixel data output by each SPAD sub-pixel in a connected line of SPAD sub-pixels, synthesizing the received sub-pixel data into one line of pixel data and outputting the line of pixel data;

the second-level logic OR gate connected with the first macro pixel is used for receiving column pixel data output by the plurality of first-level logic OR gates respectively, synthesizing the received column pixel data to obtain first image data corresponding to the first macro pixel, and outputting the first image data corresponding to the first macro pixel.

Optionally, the multiplexer includes a plurality of input terminals, an output terminal and a control terminal, the plurality of input terminals of the multiplexer are connected to the switch array, the control terminal of the multiplexer is connected to the control terminal of the processing unit, and the output terminal of the multiplexer is connected to the input terminal of the processing unit;

each input end of the multiplexer is used for receiving one path of first image data output by the switch array;

the control end of the multiplexer is used for receiving the gating signal output by the control end of the processing unit at each measuring moment;

and the output end of the multiplexer is used for outputting one path of first image data corresponding to the gating signal received at the corresponding measuring moment at each measuring moment.

Optionally, the processing unit is configured to determine, according to the gating signal output at each measurement time, a position of a macro pixel corresponding to the first image data corresponding to the received corresponding gating signal, and use the first image data corresponding to the corresponding gating signal as pixel data at the determined position of the macro pixel to generate the fused image.

Optionally, each adjacent two of the macro-pixels of the plurality of macro-pixels have (N-1) rows of overlapping SPAD sub-pixels, or each adjacent two of the macro-pixels of the plurality of macro-pixels have (M-1) columns of overlapping SPAD sub-pixels.

In another aspect, an imaging control method is provided, and is applied to a processing unit of an imaging device, where the imaging device further includes an imaging unit, where the imaging unit includes a single-photon avalanche diode SPAD array, a switch array, and a multiplexer, where the SPAD array includes a plurality of macro pixels, each macro pixel in the plurality of macro pixels includes (N × M) SPAD sub-pixels, and each two adjacent macro pixels have overlapping SPAD sub-pixels therebetween, where the switch array is configured to receive sub-pixel data respectively output by the (N × M) SPAD sub-pixels included in each macro pixel in the plurality of macro pixels, synthesize sub-pixel data respectively output by the (N × M) SPAD sub-pixels included in each macro pixel, obtain one path of first image data corresponding to the corresponding macro pixel, and output one path of first image data corresponding to each macro pixel to the multiplexer; the method comprises the following steps:

the processing unit sequentially outputs different types of gating signals to the multiplexer according to a preset time sequence, so that the multiplexer outputs first image data corresponding to the corresponding gating signals when receiving one type of gating signals output by the processing unit;

and the processing unit receives the first image data corresponding to each macro pixel sequentially output by the multiplexer, and generates a fusion image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.

Optionally, the switch array comprises a first-stage logical or gate and a second-stage logical or gate, one row of SPAD sub-pixels in the first macro-pixel is connected with one first-stage logical or gate, a plurality of first-stage logical or gates connected with a plurality of rows of SPAD sub-pixels in the first macro-pixel are connected with one second-stage logical or gate, and the first macro-pixel is any one of the macro-pixels;

each first-stage logic or gate connected with the first macro-pixel is used for receiving sub-pixel data output by each SPAD sub-pixel in a row of SPAD sub-pixels connected with the first macro-pixel, synthesizing the received sub-pixel data into a row of pixel data, and outputting the row of pixel data;

the second-level logic or gate connected with the first macro pixel is used for receiving row pixel data respectively output by a plurality of first-level logic or gates connected with the second macro pixel, synthesizing the received row pixel data to obtain first image data corresponding to the first macro pixel, and outputting the first image data corresponding to the first macro pixel.

each first-level logic or gate connected with the first macro-pixel is used for receiving one path of sub-pixel data output by each SPAD sub-pixel in a connected row of SPAD sub-pixels, synthesizing the received sub-pixel data into one path of row pixel data and outputting the row of pixel data;

the second-level logic or gate connected with the first macro-pixel is used for receiving column pixel data respectively output by the plurality of first-level logic or gates connected with the first macro-pixel, synthesizing the received column pixel data to obtain first image data corresponding to the first macro-pixel, and outputting the first image data corresponding to the first macro-pixel.

Optionally, the multiplexer includes a plurality of input terminals, an output terminal, and a control terminal, the plurality of input terminals of the multiplexer are connected to the switch array, the control terminal of the multiplexer is connected to the control terminal of the processing unit, the output terminal of the multiplexer is connected to the input terminal of the processing unit, and each input terminal of the multiplexer is configured to receive one path of first image data output by the switch array;

the processing unit sequentially outputs different kinds of gating signals to the multiplexer according to a preset time sequence, and the gating signals comprise:

the control end of the processing unit outputs a first gating signal to the control end of the multiplexer at a first measurement time, so that the output end of the multiplexer outputs one path of first image data corresponding to the first gating signal at the first measurement time, and the first measurement time is any one of a plurality of measurement times.

Optionally, the processing unit generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel, and includes:

determining the position of a macro pixel corresponding to first image data corresponding to the received corresponding gating signal according to the gating signal output at each measuring moment;

and taking the first image data corresponding to the corresponding gating signal as pixel data at the position of the macro pixel determined at the corresponding measuring moment to generate the fused image.

In another aspect, there is provided an image forming apparatus, the apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor executes executable instructions in the memory to perform the imaging control method described above.

In another aspect, a computer program product is provided comprising instructions which, when run on a computer, cause the computer to perform the steps of the imaging control method described above.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

in the embodiment of the application, a switch array and a multiplexer in an imaging unit can sequentially output one path of first image data corresponding to each macro pixel in a plurality of macro pixels, and a processing unit receives the first image data corresponding to each macro pixel and generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel. Because the overlapped SPAD sub-pixels are arranged between every two adjacent macro-pixels in the plurality of macro-pixels included in the SPAD array in the imaging unit, the quantity of image data corresponding to the macro-pixels output by the imaging unit is increased through the switch array and the multiplexer, and therefore the plane resolution of an output image is improved under the condition that the quantity of the SPAD sub-pixels and subsequent storage are not increased.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a block diagram of a SPAD array;

fig. 2 is a system architecture diagram of an imaging device according to an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of an imaging device provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a macropixel in a SPAD array according to an embodiment of the present application;

fig. 5 is a schematic diagram illustrating a connection relationship between a macro-pixel and a switch array according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of another connection relationship between a macro-pixel and a switch array according to an embodiment of the present disclosure;

FIG. 7 is a schematic view of another imaging device according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a fused image generated by an imaging device according to an embodiment of the present disclosure;

fig. 9 is a flowchart of an imaging control method according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Before explaining the embodiments of the present application in detail, a system architecture related to the embodiments of the present application will be described.

Fig. 2 is a system architecture diagram of an imaging device according to an embodiment of the present disclosure. As shown in fig. 2, the system includes an image forming apparatus 10, a router 20, and a server 30. The image forming apparatus 10 is connected to the router 20, and the router 20 is connected to the server 30. Wherein, the imaging device 10 is used for generating an image and outputting the generated image to the router 20; the router 20 is used for transmitting the image generated by the imaging device 10 to the server 30; the server 30 is configured to receive the image transmitted by the router 20 and store the received image.

The imaging apparatus 10 includes, among other things, a processing unit 101 and an imaging unit 102. The imaging unit 102 comprises an array of SPADs. The SPAD array comprises a plurality of macro-pixels, each macro-pixel comprises a plurality of SPAD sub-pixels, and overlapped SPAD sub-pixels are arranged between every two adjacent macro-pixels. The imaging unit 102 is configured to output first image data corresponding to each macro-pixel, and the processing unit 101 is configured to receive the first image data corresponding to each macro-pixel output by the imaging unit 102, and generate a fused image according to the first image data corresponding to each macro-pixel and a position of the corresponding macro-pixel. It should be noted that, the detailed implementation of the imaging unit 102 can be referred to the following description, and is not described herein again.

In addition to this, the imaging apparatus 10 may further include a laser driver 103, a laser 104, an emission lens 105, and a reception lens 106.

The laser driver 103 may be connected to the imaging unit 102 and the laser 104, respectively. The image forming unit 102 may send an emission start instruction to the laser driver 103 and start timing at the same time as sending the emission start instruction. The laser driver 103 drives the laser 104 to emit laser light upon receiving the emission start instruction.

The emission lens 105 is used to emit laser light emitted from the laser 104 to irradiate a target object. The emission lens 105 may be fixed on the laser 104 or may be disposed in the laser 104.

The receiving lens 106 is configured to receive light reflected by the target object, and further emit the received reflected light to each SPAD subpixel of the SPAD array in the imaging unit 102, so that each SPAD subpixel of the SPAD array in the imaging unit 102 outputs subpixel data according to the detected reflected light signal, and further, the imaging unit 102 outputs first image data corresponding to each macro pixel according to each subpixel data. The receiving lens 106 may be fixed on the SPAD array in the imaging unit 102, or may be installed in the SPAD array 102.

Optionally, in some possible implementations, after obtaining the fused image, the imaging device 10 may further perform intelligent analysis based on the fused image, and then send the fused image and the intelligent analysis result to the server 30 through the router 20, so that the server 30 stores the fused image and the intelligent analysis result for use by other subsequent services.

Next, an image forming apparatus provided in an embodiment of the present application will be described.

Fig. 3 is a schematic view of an imaging device according to an embodiment of the present application. As shown in fig. 3, the apparatus includes: the imaging unit 301 comprises an SPAD array 3011, a switch array 3012 and a multiplexer 3013, wherein the SPAD array 3011 comprises a plurality of macro-pixels, each macro-pixel in the macro-pixels comprises (N × M) SPAD sub-pixels, and each two adjacent macro-pixels have overlapped SPAD sub-pixels. The switch array 3012 is configured to receive sub-pixel data respectively output by (N × M) SPAD sub-pixels included in each macro pixel of the multiple macro pixels, synthesize the sub-pixel data respectively output by (N × M) SPAD sub-pixels included in each macro pixel, obtain one path of first image data corresponding to the corresponding macro pixel, and output one path of first image data corresponding to each macro pixel; the multiplexer 3013 is configured to receive multiple paths of first image data output by the switch array 3012, and sequentially output the multiple paths of first image data to the processing unit 302; the processing unit 302 is configured to receive the first image data corresponding to each macro pixel output by the SPAD array, and generate a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel.

In this embodiment, the SPAD array 3011 includes a plurality of SPAD sub-pixels arranged in a rectangular array, and (N × M) SPAD sub-pixels in the SPAD array 3011 may constitute a macro-pixel. The SPAD array 3011 contains a plurality of macropixels consisting of (N × M) SPAD subpixels. N is a row number of the macro-pixel, M is a column number of the macro-pixel, and N and M may be equal or unequal, which is not limited in the embodiments of the present application.

It should be noted that, in the embodiment of the present application, each adjacent two macro pixels in the plurality of macro pixels in the SPAD array 3011 have overlapping SPAD sub-pixels therebetween.

For example, when each macro-pixel includes (N × M) SPAD sub-pixels, two macro-pixels adjacent to each other up and down may have multiple rows of overlapping SPAD sub-pixels, and two macro-pixels adjacent to each other left and right may have multiple columns of overlapping SPAD sub-pixels. For example, two adjacent macro-pixels above and below may have overlapping SPAD sub-pixels in (N-1) lines, or may have overlapping SPAD sub-pixels in (N-2) lines, which is not limited in this embodiment of the present application. Two adjacent left and right macro pixels have (M-1) overlapped SPAD sub-pixels, and may also have (M-2) overlapped SPAD sub-pixels, which is not limited in this embodiment of the application.

For example, referring to fig. 4, a SPAD array of 4 × 4 with 16 SPAD subpixels, for example, N = M =3, and a macropixel includes 3 × 3 SPAD subpixels. 2 lines are overlapped between the macro-pixel 1 and the macro-pixel 2 which are adjacent up and down, the SPAD sub-pixels overlapped in the first line comprise SPAD 5-SPAD 7, and the SPAD sub-pixels overlapped in the second line comprise SPAD 9-SPAD 11; the macro-pixel 2 and the macro-pixel 3 adjacent to each other on the left and the right are overlapped by 2 columns, the SPAD sub-pixels overlapped by the first column comprise SPAD6, SPAD10 and SPAD14, and the SPAD sub-pixels overlapped by the second column comprise SPAD7, SPAD11 and SPAD15.

In an embodiment of the present application, the switch array 3012 includes a first level logic or gate and a second level logic or gate. In a possible implementation manner, taking any macro pixel of multiple macro pixels in the SPAD array 3011 as an example, it is referred to as a first macro pixel, each row of SPAD sub pixels in the first macro pixel is connected to a first-level logical or gate, and multiple first-level logical or gates corresponding to multiple rows of SPAD sub pixels in each macro pixel are connected to a second-level logical or gate.

Each first-stage logic or gate connected with the first macro-pixel is used for receiving a path of sub-pixel data output by each SPAD sub-pixel in a row of connected SPAD sub-pixels, and then the first-stage logic or gates synthesize the received sub-pixel data to obtain a path of row pixel data and output the row pixel data. For example, for a macropixel consisting of (N × M) SPAD subpixels, the macropixel is connected to N first-level logical or gates, each of which will output one row of pixel data, so that the N first-level logical or gates will output N rows of pixel data.

And the second-level logic OR gate connected with the first macro pixel is used for receiving line pixel data respectively output by the plurality of first-level logic OR gates connected with the second macro pixel, and then the second-level logic OR gate synthesizes the received multiple lines of line pixel data to obtain one path of first image data and outputs the first image data.

It should be noted that, in a possible case, when the first macro-pixel includes 4 SPAD sub-pixels, then a first-stage logical or gate may be a two-input or gate. Accordingly, a second stage logical OR gate may also be a two input OR gate.

For example, referring to fig. 5, assuming that the SPAD array includes 3 × 3 SPAD sub-pixels, SPAD1, SPAD2, SPAD4, and SPAD5 constitute a macro-pixel 1, SPAD1 and SPAD2 in the first row of the macro-pixel 1 are connected to a first stage logical or gate, which is a two-input or gate. In this case, the first stage logical or gate receives the sub-pixel data respectively output by SPAD1 and SPAD2, and synthesizes the two sub-pixel data into one row of pixel data 1+2; similarly, SPAD4 and SPAD5 in the second row are connected to a first-stage logical or gate, which receives the sub-pixel data output by SPAD4 and SPAD5, and synthesizes the two sub-pixel data into a row of pixel data 4+5.

Further, two first-stage logical or gates corresponding to two rows of SPAD subpixels of the macro pixel 1 are connected to a second-stage logical or gate, and the second logical or gate is also a two-input or gate, so that the second-stage logical or gate receives the row pixel data 1+2 and the row pixel data 4+5 output by the two first-stage logical or gates, synthesizes the row pixel data 1+2 and the row pixel data 4+5 into one path of first image data 1245, and outputs the first image data 1245.

In another possible scenario, when the first macro-pixel comprises X SPAD sub-pixels, and X is greater than 4, then each first level logical or gate of the first macro-pixel connection may comprise one or more two-input or gates, and the second logical or gate of the first macro-pixel connection may also comprise one or more two-input or gates.

For example, when X is 9, according to R = ceiling (log) ₂ X) to get R =4, wherein ceiling (X) refers to rounding up. In this case, the switch array would include 4 layers of two-input OR gates, where each first level logic OR gate may be comprised of two layers of two-input OR gates, and each second level logic OR gate may also be comprised of two layers of two-input OR gates.

For example, referring to fig. 6, assuming that the SPAD array includes 4 × 4 SPAD subpixels, SPAD1, SPAD2, SPAD3, SPAD5, SPAD6, SPAD7, SPAD9, SPAD10, and SPAD11 constitute a macro pixel 1, in this case, each two adjacent SPAD subpixels in the first row in the macro pixel 1 may be connected with a two-input or gate, so that these 2 two-input or gates are the first-level two-input or gates in the first-level logical or gates connected with the SPAD subpixels in the first row of the macro pixel 1. On the basis, the adjacent first-level input OR gate in the first-level logic OR gate is connected with a two-input OR gate, so that the two-input OR gate is the second-level input OR gate in the first-level logic OR gate connected with the first row of SPAD sub-pixels. In this way, each first-layer second-input or gate in the first-level logic or gate connected with each row of SPAD subpixels of the macro-pixel 1 receives subpixel data output by two connected SPAD subpixels, and outputs the two paths of subpixel data after merging. And each second-layer two-input OR gate receives two paths of data output by the two first-layer input OR gates connected with each other, and combines the two received paths of data to obtain one path of row pixel data.

After the row pixel data corresponding to each row of SPAD sub-pixels in the macro-pixel is obtained by the above method, for the 3 rows of pixel data, two second-layer second input or gates outputting the row pixel data of two adjacent rows may be connected to one second-layer input or gate, and at this time, the connected second-layer input or gate is the first-layer second input or gate in the second-layer logical or gate. And then, two first-layer two-input OR gates in the second-level logic OR gate are connected with one two-input OR gate, and at the moment, the two-input OR gate is the second-layer two-input OR gate in the second-level logic OR gate. In this way, each first-layer two-input or gate in the second-level logical or gate can combine the received row pixel data of two adjacent rows to output one path of data, and then the second-layer two-input or gate combines the two paths of data input by the two first-layer two-input or gates to obtain the first image data.

The above is only an example that the macro pixel includes 9 SPAD sub-pixels, and when the macro pixel includes 8, 6, or more SPAD sub-pixels, the processing may be performed by referring to the above method, and details of the embodiment of the present application are not repeated herein.

In addition, for each macro-pixel in the plurality of macro-pixels, a plurality of first-level logic or gates and a second-level logic or gate may be connected with reference to the first macro-pixel. Thus, a plurality of first level logic or gates and a plurality of second level logic or gates will be included in the switch array 3012.

Optionally, in another possible implementation manner, still taking the first macro-pixel as an example, a column of SPAD sub-pixels in the first macro-pixel is connected to a first-stage logical or gate, and a plurality of first-stage logical or gates connected to a plurality of columns of SPAD sub-pixels in the first macro-pixel are connected to a second-stage logical or gate.

Each first-stage logic or gate connected to the first macro-pixel may be configured to receive a path of sub-pixel data output by each SPAD sub-pixel in a column of SPAD sub-pixels connected to the first macro-pixel, and then synthesize the received sub-pixel data to obtain a path of column of pixel data, and output the column of pixel data. For example, in a macro-pixel consisting of (N × M) SPAD sub-pixels, the macro-pixel is connected to M first-level logic or gates, each of which will output one column of pixel data, so that the M first-level logic or gates will output M columns of pixel data.

The second-level logic or gate connected with the first macro pixel is used for receiving column pixel data respectively output by the plurality of first-level logic or gates connected with the second macro pixel, and then the second-level logic or gate synthesizes the received column pixel data to obtain a path of first image data and outputs the first image data.

It should be noted that, when the macropixel includes a SPAD subpixel of 4, each of the first-stage logical or gate and the second-stage logical or gate may be implemented by a two-input or gate. Optionally, when the macro pixel includes more SPAD sub-pixels, reference may be made to the method described in the foregoing, where each first-level logic or gate is implemented by one or more two-input or gates, and each second-level logic or gate is also implemented by one or more two-input or gates, and details of the embodiment of the present application are not repeated herein.

After each second-level logical or gate in the switch array 3012 outputs one path of first image data, the multiplexer 3013 receives one path of first image data output by a plurality of second-level logical or gates, and sequentially outputs the received multiple paths of first image data to the processing unit 302.

In the embodiment of the present application, the multiplexer 3013 includes a plurality of input terminals, an output terminal, and a control terminal, the plurality of input terminals of the multiplexer 3013 are connected to the switch array 3012, the control terminal of the multiplexer 3013 is connected to the control terminal of the processing unit 302, and the output terminal of the multiplexer 3013 is connected to the input terminal of the processing unit 302.

Each input end of the multiplexer 3013 is configured to receive one path of first image data output by the switch array 3012; the control terminal of the multiplexer 3013 is configured to receive the gating signal output by the control terminal of the processing unit 302 at each measurement time; an output end of the multiplexer 3013 is configured to output, at each measurement time, one path of first image data corresponding to the strobe signal received at the corresponding measurement time.

It should be noted that the number of the inputs of the multiplexer 3013 is the same as the number of second-stage logic or gates included in the switch array 3012, so that each input of the multiplexer 3013 is connected to one second-stage logic or gate in the switch array 3012 to receive the first image data output by the connected second-stage logic or gate.

In a possible implementation manner, the input terminals of the multiplexer 3013 may be sequentially connected to the second-stage logical or gates according to the position of the macropixel corresponding to the first image data output by each second-stage logical or gate in the switch array 3012 in the SPAD array 3011. For example, a second stage logical or gate for outputting first image data corresponding to a macropixel in a first row and a first column of the plurality of macropixels of SPAD array 3011 may be connected to a first input of multiplexer 3013, a second stage logical or gate for outputting first image data corresponding to a macropixel in a first row and a second column of the plurality of macropixels of SPAD array 3011 may be connected to a second input of multiplexer 3013, and so on.

The control terminal of the multiplexer 3013 may also receive a gating signal output by the control terminal of the processing unit 302 according to a preset timing while receiving multiple paths of the first image data. The processing unit 302 may sequentially output, at different measurement times, gating signals for gating different input terminals in the multiplexer through its own control terminal according to the positions of the macro pixels in the SPAD array 3011 and the input terminal of the multiplexer 3013, which is used for receiving the first image data corresponding to each macro pixel. Accordingly, the multiplexer 3013 may receive the strobe signal output from the control terminal of the processing unit 302 at each measurement time via its own control terminal, and output the first image data input from the corresponding input terminal according to the strobe signal.

For example, the processing unit determines that the first image data corresponding to the macro-pixel 1 in the first row and the first column in the SPAD array is to be input to the input terminal 1 of the multiplexer, the first image data corresponding to the macro-pixel 2 in the first row and the second column is to be input to the input terminal 2 of the multiplexer, the first image data corresponding to the macro-pixel 3 in the second row and the first column is to be input to the input terminal 3 of the multiplexer, and the first image data corresponding to the macro-pixel 4 in the second row and the second column is to be input to the input terminal 4 of the multiplexer, and on the basis, at the first measurement time, the control terminal of the processing unit outputs the gate signal 0000 for gating the input terminal 1 of the multiplexer. The control terminal of the multiplexer outputs the first image data of the macropixel 1 input from the input terminal 1 of the multiplexer to the processing unit after receiving the gate signal 0000. At the second measurement instant, the control terminal of the processing unit outputs a gating signal 0001 for gating input terminal 2 of the multiplexer. The control terminal of the multiplexer outputs the first image data of the macro-pixel 2 input from the input terminal 2 of the multiplexer to the processing unit after receiving the gate signal 0001. At the third measurement time, the control terminal of the processing unit outputs a gate signal 0010 for gating the input terminal 3 of the multiplexer, and the control terminal of the multiplexer outputs the first image data of the macro-pixel 3 input from the input terminal 3 of the multiplexer to the processing unit after receiving the gate signal 0010. At the fourth measurement time, the control terminal of the processing unit outputs a gate signal 0011 for gating the input terminal 4 of the multiplexer, and the control terminal of the multiplexer outputs the first image data of the macro-pixel 4 input from the input terminal 4 of the multiplexer to the processing unit after receiving the gate signal 0011.

After receiving the first image data corresponding to each gating signal, the processing unit 302 may determine, according to the gating signal output at each measurement time, a position of a macro pixel corresponding to the first image data corresponding to the received corresponding gating signal, and use the first image data corresponding to the corresponding gating signal as pixel data at the determined position of the macro pixel to generate a fused image.

It should be noted that, the two adjacent measurement time points may be separated by a certain time period, and the processing unit 302 may process the received first image data within the time period.

Illustratively, after the processing unit 302 outputs a strobe signal through its own control terminal at a certain measurement timing, it may receive first image data output by the multiplexer 3013 according to the strobe signal. As can be seen from the foregoing description, when the processing unit 302 transmits the strobe signal, the strobe signal is transmitted according to the position of the macro-pixel and the input terminal of the multiplexer 3013 for outputting the image data of the corresponding macro-pixel, and based on this, the processing unit 302 can determine, according to the position of the macro-pixel corresponding to the strobe signal output at the measurement time, which image data corresponds to which macro-pixel the received first image data corresponds, and further use the first image data as the pixel data at the position of the macro-pixel. Thereafter, the processing unit 302 may continue to send the gating signal to the multiplexer 3013 through the control terminal after the next measurement time arrives, and process the received first image data corresponding to the corresponding gating signal. In this manner, the processing unit 302 receives and processes the plurality of first image data sequentially output by the multiplexer, obtains pixel data at the position of each macro pixel, and generates a fused image.

For better understanding of the device of the embodiment of the present application, the imaging device provided in the embodiment of the present application is now described in detail by taking a 3 × 3 SPAD array as an example, where, referring to fig. 7, the SPAD array includes 9 SPAD sub-pixels, SPAD1 to SPAD9 respectively. In addition, each macro-pixel in the SPAD array includes 2 × 2 SPAD sub-pixels. Wherein, SPAD1, SPAD2, SPAD4 and SPAD5 form a macro-pixel 1, SPAD2, SPAD3, SPAD5 and SPAD6 form a macro-pixel 2, SPAD4, SPAD5, SPAD7 and SPAD8 form a macro-pixel 3, SPAD5, SPAD6, SPAD8 and SPAD9 form a macro-pixel 4. The overlapped SPAD sub-pixels in the macro-pixel 1 and the macro-pixel 2 are SPAD2 and SPAD5, the overlapped SPAD sub-pixels in the macro-pixel 3 and the macro-pixel 4 are SPAD5 and SPAD8, the overlapped SPAD sub-pixels in the macro-pixel 1 and the macro-pixel 3 are SPAD4 and SPAD5, and the overlapped SPAD sub-pixels in the macro-pixel 2 and the macro-pixel 4 are SPAD5 and SPAD6.

One row of SPAD sub-pixels of each macro-pixel in the 4 macro-pixels is connected with one first-stage logical or gate, so that two SPAD sub-pixels in one row of SPAD sub-pixels of each macro-pixel output two sub-pixel data to the connected first-stage logical or gate, for example, SPAD1 and SPAD2 sub-pixels in the first row of macro-pixel 1 are connected with one first-stage logical or gate, so that SPAD1 outputs sub-pixel data 1 to the connected first-stage logical or gate, and SPDA2 outputs sub-pixel data 2 to the connected first-stage logical or gate, and after receiving

sub-pixel data

1 and 2, the first-stage logical or gate synthesizes the two sub-pixel data into one row of pixel data 1+2 and outputs the row of pixel data 1+2. For each row of SPAD sub-pixels in other macro-pixels, each row of SPAD sub-pixels is connected with a first-stage logical or gate, so that 6 rows of pixel data are output through 6 first-stage logical or gates. In addition, the multiple first level logical or gates connected by the multiple rows of SPAD subpixels in each macro-pixel will connect one second level logical or gate, for example, the first level logical or gate connected by two subpixels of SPAD1 and SPAD2 of the first row of macro-pixel 1 and the first level logical or gate connected by two subpixels of SPAD4 and SPAD5 of the second row will connect one second level logical or gate, so that this second level logical or gate will receive row pixel data 1+2 and row pixel data 4+5, and then the second level logical or gate will synthesize row pixel data 1+2 and row pixel data 4+5 into first image data 1245. For the first-level logical or gates connected to SPAD sub-pixels in each row of other macro-pixels, a second-level logical or gate may also be connected, so that each second-level logical or gate may be connected to two first-level logical or gates corresponding to two SPAD sub-pixels in one macro-pixel, and combine two rows of pixel data output by the two connected first-level logical or gates into one path of first image data, thereby obtaining four paths of first image data, which are first image data 1245, first image data 4578, first image data 2356, and first image data 5689.

Each second-level logic or gate can output one path of the first image data obtained by the second-level logic or gate to the multi-path selector, and the multi-path selector can sequentially output each path of the first image data to the processing unit according to the received gating signals.

The second-level logical or gate corresponding to the macro pixel 1 may be connected to the input terminal 1 of the multiplexer, the second-level logical or gate corresponding to the macro pixel 2 may be connected to the input terminal 2 of the multiplexer, the second-level logical or gate corresponding to the macro pixel 3 is connected to the input terminal 3 of the multiplexer, and the second-level logical or gate corresponding to the macro pixel 4 is connected to the input terminal 4 of the multiplexer. Accordingly, the control terminal of the processing unit may sequentially output the gate signals 00, 01, 10, and 11. The multiplexer outputs first image data 1245 upon receiving the gate signal 00, and the processing unit takes the first image data 1245 as pixel data at the position of the macro pixel 1 after receiving the first image data 1245. Upon receiving the strobe signal 01, the first image data 2356 is output, and the processing unit takes the first image data 2356 as the pixel data at the position where the macro pixel 2 is located after receiving the first image data 2356. By analogy, the finally obtained fusion image is shown in fig. 8.

In the embodiment of the application, a switch array and a multiplexer in an imaging unit can sequentially output one path of first image data corresponding to each macro pixel in a plurality of macro pixels, and a processing unit receives the first image data corresponding to each macro pixel and generates a fused image according to the first image data corresponding to each macro pixel and the position of the corresponding macro pixel. Because the overlapped SPAD sub-pixels are arranged between every two adjacent macro-pixels in the plurality of macro-pixels included in the SPAD array in the imaging unit, the quantity of image data of the macro-pixels output by the imaging unit is increased through the switch array and the multiplexer, and the TDC and the histogram calculation circuit of the SPAD array are not increased, so that the image processing method provided by the embodiment of the application can improve the plane resolution of an output image of the SPAD array under the condition that the quantity of the SPAD sub-pixels, subsequent storage and the area of a main circuit are not increased.

Next, an imaging method provided in an embodiment of the present application will be described.

Fig. 9 is a flowchart of an imaging control method provided in an embodiment of the present application, where the method is applied to a processing unit of an imaging apparatus described in the foregoing embodiment, and the imaging apparatus further includes an imaging unit, where the imaging unit includes a single-photon avalanche diode SPAD array, the SPAD array includes a plurality of macro pixels, each macro pixel in the plurality of macro pixels includes (N × M) SPAD sub-pixels, and each two adjacent macro pixels have overlapping SPAD sub-pixels therebetween, where the switch array is configured to receive sub-pixel data respectively output by (N × M) SPAD sub-pixels included in each macro pixel in the plurality of macro pixels, synthesize sub-pixel data respectively output by (N × M) SPAD sub-pixels included in each macro pixel, obtain one path of first image data corresponding to the corresponding macro pixel, and output one path of first image data corresponding to each macro pixel to a multiplexer, and the method includes the following steps:

step 901: the processing unit sequentially outputs different types of gating signals to the multiplexer according to a preset time sequence, so that the multiplexer outputs first image data corresponding to the corresponding gating signals when receiving one type of gating signals output by the processing unit.

The implementation process that the processing unit sequentially outputs different types of gating signals according to the preset time sequence may refer to the implementation manner described in the foregoing embodiments, and details of the embodiments of the present application are not described herein again.

Step 902: the processing unit receives the first image data corresponding to each macro-pixel output by the multiplexer in sequence, and generates a fusion image according to the first image data corresponding to each macro-pixel and the position of the corresponding macro-pixel.

The process of the multiplexer outputting the corresponding first image data according to the strobe signal sent by the processing unit may refer to the implementation manner in the foregoing embodiment. In addition, the process of generating the fusion image by the processing unit may also refer to the foregoing implementation manner, and details are not described herein in this embodiment of the application.

Optionally, the switch array includes a first-stage logical or gate and a second-stage logical or gate, a row of SPAD subpixels in the first macro pixel is connected to one first-stage logical or gate, a plurality of first-stage logical or gates connected to the plurality of rows of SPAD subpixels in the first macro pixel are connected to one second-stage logical or gate, and the first macro pixel is any one of the plurality of macro pixels;

each first-stage logic OR gate connected with the first macro pixel receives one path of sub-pixel data output by each SPAD sub-pixel in one line of SPAD sub-pixels connected with the first macro pixel, synthesizes the received sub-pixel data into one path of line pixel data, and outputs the line pixel data;

the second-level logical OR gate connected with the first macro pixel receives row pixel data respectively output by a plurality of first-level logical OR gates connected with the second macro pixel, synthesizes the received row pixel data to obtain first image data corresponding to the first macro pixel, and outputs the first image data corresponding to the first macro pixel.

Optionally, the switch array includes a first-stage logical or gate and a second-stage logical or gate, a column of SPAD sub-pixels in the first macro-pixel is connected with one first-stage logical or gate, a plurality of first-stage logical or gates connected with the columns of SPAD sub-pixels in the first macro-pixel are connected with one second-stage logical or gate, and the first macro-pixel is any one of the macro-pixels;

each first-level logic or gate connected with the first macro-pixel is used for receiving one path of sub-pixel data output by each SPAD sub-pixel in a connected row of SPAD sub-pixels, synthesizing the received sub-pixel data into one path of row pixel data and outputting the row pixel data;

Optionally, the multiplexer includes a plurality of input terminals, an output terminal and a control terminal, the plurality of input terminals of the multiplexer are connected to the switch array, the control terminal of the multiplexer is connected to the control terminal of the processing unit, the output terminal of the multiplexer is connected to the input terminal of the processing unit, and each input terminal of the multiplexer receives one path of the first image data output by the switch array;

the processing unit outputs different kinds of gating signals to the multiplexer in sequence according to a preset time sequence, and the method comprises the following steps:

the control end of the processing unit outputs a first gating signal to the control end of the multiplexer at a first measurement time, so that the output end of the multiplexer outputs one path of first image data corresponding to the first gating signal at the first measurement time, and the first measurement time is any one of the multiple measurement times.

Optionally, the processing unit generates a fused image according to the first image data corresponding to each macro-pixel and the position of the corresponding macro-pixel, including:

and taking the first image data corresponding to the corresponding gating signal as the pixel value of the macro pixel position determined at the corresponding measuring moment so as to generate a fusion image.

Optionally, each adjacent two of the macro-pixels in the plurality of macro-pixels have (N-1) rows of overlapping SPAD sub-pixels, or each adjacent two of the macro-pixels in the plurality of macro-pixels have (M-1) columns of overlapping SPAD sub-pixels.

It should be noted that, for implementation of the above steps, reference may be made to implementation of the imaging device in the foregoing embodiments, and details of the embodiments of the present application are not described herein again.

In summary, in the embodiment of the present application, a switch array and a multiplexer in an imaging unit may sequentially output one path of first image data corresponding to each macro pixel in a plurality of macro pixels, and a processing unit receives the first image data corresponding to each macro pixel and generates a fused image according to the first image data corresponding to each macro pixel and a position of the corresponding macro pixel. Because the overlapped SPAD sub-pixels are arranged between every two adjacent macro-pixels in the plurality of macro-pixels included in the SPAD array in the imaging unit, the quantity of image data of the macro-pixels output by the imaging unit is increased through the switch array and the multiplexer, and the TDC and the histogram calculation circuit of the SPAD array are not increased, so that the image processing method provided by the embodiment of the application can improve the plane resolution of an output image of the SPAD array under the condition that the quantity of the SPAD sub-pixels, subsequent storage and the area of a main circuit are not increased.

An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by the foregoing imaging apparatus, the imaging control method shown in fig. 9 can be implemented.

The embodiment of the present application further provides a computer program product containing instructions, which when run on the aforementioned imaging apparatus, causes the imaging apparatus to execute the imaging control method provided in the embodiment shown in fig. 9.

The above description should not be taken as limiting the embodiments of the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the embodiments of the present application.

Claims

1. An image forming apparatus, characterized in that the image forming apparatus comprises: the device comprises an imaging unit and a processing unit, wherein the imaging unit comprises a single photon avalanche diode SPAD array, a switch array and a multiplexer, the SPAD array comprises a plurality of macro pixels, each macro pixel in the plurality of macro pixels comprises (N multiplied by M) SPAD sub-pixels, and each two adjacent macro pixels have overlapped SPAD sub-pixels;

the switch array is used for receiving sub-pixel data respectively output by (N multiplied by M) SPAD sub-pixels included by each macro-pixel in the macro-pixels, synthesizing the sub-pixel data respectively output by (N multiplied by M) SPAD sub-pixels included by each macro-pixel to obtain a path of first image data corresponding to the corresponding macro-pixel, and outputting a path of first image data corresponding to each macro-pixel;

2. The imaging device of claim 1, wherein the switch array comprises a first stage logical OR gate and a second stage logical OR gate, one row of SPAD sub-pixels in a first macro-pixel is connected with one first stage logical OR gate, a plurality of first stage logical OR gates connected with a plurality of rows of SPAD sub-pixels in the first macro-pixel are connected with one second stage logical OR gate, and the first macro-pixel is any one macro-pixel in the plurality of macro-pixels;

3. The imaging device of claim 1, wherein the switch array comprises a first stage logical OR gate and a second stage logical OR gate, one column of SPAD sub-pixels in the first macro-pixel is connected with one first stage logical OR gate, a plurality of first stage logical OR gates connected with a plurality of columns of SPAD sub-pixels in the first macro-pixel are connected with one second stage logical OR gate, and the first macro-pixel is any one macro-pixel in the plurality of macro-pixels;

4. The imaging apparatus of claim 1, wherein the multiplexer includes a plurality of input terminals, an output terminal, and a control terminal, the plurality of input terminals of the multiplexer being connected to the switch array, the control terminal of the multiplexer being connected to the control terminal of the processing unit, the output terminal of the multiplexer being connected to the input terminal of the processing unit;

5. The imaging apparatus according to claim 4, wherein the processing unit is configured to determine, according to the gating signal output at each measurement time, a position of a macro pixel corresponding to the first image data corresponding to the received corresponding gating signal, and use the first image data corresponding to the corresponding gating signal as the pixel data at the determined position of the macro pixel to generate the fused image.

6. The imaging device of any of claims 1-5, wherein each adjacent two of the plurality of macropixels have (N-1) rows of overlapping SPAD subpixels, or wherein each adjacent two of the plurality of macropixels have (M-1) columns of overlapping SPAD subpixels.

7. An imaging control method is applied to a processing unit of an imaging device, the imaging device further comprises an imaging unit, the imaging unit comprises a single-photon avalanche diode SPAD array, a switch array and a multiplexer, the SPAD array comprises a plurality of macro pixels, each macro pixel in the macro pixels comprises (N × M) SPAD sub-pixels, and overlapped SPAD sub-pixels are arranged between every two adjacent macro pixels, wherein the switch array is used for receiving sub-pixel data respectively output by the (N × M) SPAD sub-pixels included by each macro pixel in the macro pixels, synthesizing the sub-pixel data respectively output by the (N × M) SPAD sub-pixels included by each macro pixel to obtain a path of first image data corresponding to the corresponding macro pixel, and outputting the path of first image data corresponding to each macro pixel to the multiplexer; the method comprises the following steps:

the processing unit sequentially outputs different types of gating signals to the multiplexer according to a preset time sequence, so that the multiplexer outputs first image data corresponding to the corresponding gating signals when receiving one type of gating signal output by the processing unit;

8. The method of claim 7, wherein the switch array comprises a first-stage logical OR gate and a second-stage logical OR gate, one row of SPAD sub-pixels in the first macro-pixel is connected with one first-stage logical OR gate, a plurality of first-stage logical OR gates connected with a plurality of rows of SPAD sub-pixels in the first macro-pixel are connected with one second-stage logical OR gate, and the first macro-pixel is any one of the plurality of macro-pixels;

9. The method of claim 7, wherein the switch array comprises a first stage logical OR gate and a second stage logical OR gate, one column of SPAD sub-pixels in the first macro-pixel is connected with one first stage logical OR gate, a plurality of first stage logical OR gates connected with a plurality of columns of SPAD sub-pixels in the first macro-pixel are connected with one second stage logical OR gate, and the first macro-pixel is any one macro-pixel in the plurality of macro-pixels;

10. The method according to claim 7, wherein the multiplexer comprises a plurality of input terminals, an output terminal and a control terminal, the plurality of input terminals of the multiplexer are connected to the switch array, the control terminal of the multiplexer is connected to the control terminal of the processing unit, the output terminal of the multiplexer is connected to the input terminal of the processing unit, and each input terminal of the multiplexer is configured to receive a path of the first image data output by the switch array;

11. The method of claim 10, wherein the processing unit generates the fused image according to the first image data corresponding to each macro-pixel and the position of the corresponding macro-pixel, and comprises:

12. The method of any of claims 7-11, wherein each adjacent two of the macropixels have (N-1) rows of overlapping SPAD subpixels, or wherein each adjacent two of the macropixels have (M-1) columns of overlapping SPAD subpixels.