CN116233393A

CN116233393A - Depth map generation method, detection system and SPAD sensor

Info

Publication number: CN116233393A
Application number: CN202111442873.4A
Authority: CN
Inventors: 苏星; 张泽鑫; 沈林杰
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2021-11-30
Filing date: 2021-11-30
Publication date: 2023-06-06

Abstract

The embodiment of the invention provides a depth map generation method, a detection system and a SPAD sensor, which are applied to the technical field of imaging. The method is applied to a main control unit, the main control unit is connected with the SPAD sensor, and the method comprises the following steps: determining a target subarray in a SPAD array in a SPAD sensor; wherein, the target subarray is: subarrays in the SPAD array matched with the appointed subareas in the target detection area; the target detection area is an area detected by the SPAD sensor during detection; designating a sub-region as determined based on an image acquired by the RGB sensor or the IR sensor; controlling the SPAD sensor to detect depth data in a target detection area; storing depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor; based on the depth data stored in the memory unit, a depth map specifying the sub-region is generated. By the scheme, limitation of data storage on the improvement of the array scale of the SPAD array can be avoided.

Description

Depth map generation method, detection system and SPAD sensor

Technical Field

The present invention relates to the field of imaging technologies, and in particular, to a depth map generating method, a detection system, and a SPAD sensor.

Background

The three-dimensional imaging technology based on the SPAD (Single Photon Avalanche Diode) array is a research hot spot in recent years, and has a huge application prospect in the fields of consumer electronics, security protection, robots, automatic driving and the like.

For a SPAD sensor, the larger the array size of the SPAD array within the SPAD sensor, the better the performance of the SPAD sensor, while increasing the size of the SPAD array within the SPAD sensor would result in a multiplication of the data storage requirements, which means that a larger storage area is required, which is difficult to achieve in a SPAD sensor with limited chip area, making data storage an important factor limiting the array size increase of the SPAD array.

Therefore, how to avoid the limitation of data storage on the array size of the SPAD array is a technical problem to be solved.

Disclosure of Invention

The embodiment of the invention aims to provide a depth map generation method, a detection system and a SPAD sensor, so that limitation of data storage on improving the array scale of the SPAD array can be avoided. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a depth map generating method, applied to a main control unit, where the main control unit is connected to a SPAD sensor, the method includes:

Determining a target subarray in a SPAD array in the SPAD sensor; wherein the target subarray is: subarrays matched with a designated subarea in a target detection area in the SPAD array; the target detection area is an area detected when the SPAD sensor detects; the specified sub-area is determined based on images acquired by an RGB sensor or an IR sensor;

controlling the SPAD sensor to detect depth data in the target detection area;

storing depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor;

and generating a depth map for the specified sub-region based on the depth data stored in the storage unit.

Optionally, the determining a target subarray in a SPAD array in the SPAD sensor includes:

acquiring a two-dimensional image of the target detection area, and identifying an interested target on the two-dimensional image;

when an object of interest is detected in the two-dimensional image, determining a region in which the detected object of interest is located as a specified sub-region;

and mapping the determined position information of the appointed subarea based on the pre-constructed position mapping relation between the two-dimensional image and the SPAD array to obtain subarray position information aiming at the SPAD array in the SPAD sensor.

And determining the subarray indicated by the subarray position information in the SPAD array as a target subarray.

Optionally, the acquiring the two-dimensional image of the target detection area includes:

and acquiring a two-dimensional image of the target detection area through the RGB sensor and/or the IR sensor.

Optionally, the position mapping relationship is: based on the internal and external parameters of the SPAD sensor, and the internal and external parameters of the RGB sensor or the IR sensor.

Optionally, the storing only the depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor includes:

according to a preset SPAD unit selection sequence, selecting a SPAD unit from all SPAD units in the target subarray as a target SPAD unit;

storing the depth data detected by the target SPAD unit into a storage unit in the SPAD sensor;

and returning to the step of executing the step of selecting a SPAD unit from the SPAD units in the target subarray according to the preset SPAD unit selection sequence until the detection data in the SPAD units in the target subarray are stored.

Optionally, the SPAD sensor further includes a selection circuit, the selection circuit including: a row decoder for outputting a row signal to each SPAD cell and a column decoder for outputting a column signal to each SPAD cell;

storing the depth data detected by the target SPAD unit into a storage unit in the SPAD sensor, including:

controlling the row decoder to output a row signal to the target SPAD cell, and controlling the column decoder to output a column signal to the target SPAD cell so as to configure the output state of the target SPAD cell into an outputtable state;

and controlling the target SPAD unit in an outputtable state, and outputting the detected depth data to a storage unit in the SPAD sensor.

Optionally, before the generating the depth map for the specified sub-region based on the depth data stored in the storage unit, the method further includes:

judging whether the frequency of the SPAD sensor for detecting the depth data reaches a preset detection frequency threshold value or not;

if not, returning to the step of controlling the SPAD sensor to detect the depth data in the target detection area;

And if so, executing the step of generating a depth map for the specified sub-region based on the depth data stored in the storage unit.

Optionally, after storing the depth data detected by each SPAD unit in the target subarray in the storage unit in the SPAD sensor, the method further includes:

constructing a histogram corresponding to each SPAD unit based on the depth data detected by each SPAD unit in the storage unit, wherein the histogram corresponding to each SPAD unit is used for storing the depth data detected by the SPAD unit;

the generating a depth map for the specified sub-region based on the depth data stored in the storage unit includes:

selecting detection data with the largest detection ratio of the SPAD unit from a histogram corresponding to the SPAD unit as target detection data of the SPAD unit aiming at each SPAD unit in the target subarray;

and constructing a depth map aiming at the appointed subarea based on the target detection data of each SPAD unit in the target subarea.

Optionally, the target detection area is divided into a plurality of designated sub-areas;

before the determining the target subarray in the SPAD array within the SPAD sensor, the method further comprises:

Determining a designated subarea from the designated subareas, and taking subareas matched with the designated subarea in the SPAD array as target subareas;

after the generating of the depth map for the specified sub-region based on the depth data stored in the storage unit, the method further comprises:

and determining a new designated subarea from the plurality of designated subareas, and returning to the subarea matched with the designated subarea in the SPAD array as a target subarea until a depth map is generated for each designated subarea.

In a second aspect, an embodiment of the present invention provides a SPAD sensor, the SPAD sensor including: SPAD array, selection circuit and memory cell, the selection circuit includes: the row decoder is used for outputting row signals to each SPAD unit in the SPAD array, and the column decoder is used for outputting column signals to each SPAD unit;

the SPAD sensor is used for detecting depth data of a target detection area under the control of the main control unit, wherein the target detection area is the area detected by the SPAD sensor during detection;

The row decoder is used for outputting a row signal to a target SPAD unit indicated by the received output instruction when receiving the output instruction of the main control unit; the target SPAD unit is a SPAD unit determined by the main control unit in each SPAD unit in a target subarray, and the target subarray is a subarray which is determined by the main control unit in the SPAD array and matched with a designated subarea in the target detection area;

the column decoder is used for outputting a column signal to a target SPAD unit indicated by the received output instruction when receiving the output instruction of the main control unit;

any SPAD unit in the SPAD array converts an output state into an outputtable state when receiving a row signal and a column signal at the same time;

the SPAD unit in an output state is used for writing the detected detection data into the storage unit when receiving the storage instruction of the main control unit;

the storage unit is used for outputting the stored detection data when receiving the control instruction of the main control unit, so that the main control unit generates a depth map for the specified subarea based on the depth data output by the storage unit.

Optionally, the row decoder includes a plurality of row output lines, where a row output line of the row decoder is connected to one row SPAD cell in the SPAD array, and each row output line is used to output a row signal to the SPAD cell connected thereto;

the column decoder comprises a plurality of column output lines, one column output line of the column decoder is connected with one column of SPAD units in the SPAD array, and each column output line is used for outputting column signals to the SPAD units connected with the column output line.

In a third aspect, an embodiment of the present invention provides a detection system, where the depth map generating system includes: the device comprises a main control unit, a SPAD sensor, an RGB sensor or an IR sensor;

the RGB sensor or the IR sensor is used for collecting an image of a target detection area, determining a specified subarea based on the collected image and transmitting the determined specified subarea to the main control unit;

the main control unit is used for receiving the appointed subarea, determining a target subarray in a SPAD array in the SPAD sensor based on the appointed subarea, and sending a control signal to the SPAD sensor based on the target subarray;

the SPAD sensor is used for detecting depth data in the target detection area after receiving the control signal, and storing the depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor;

The main control unit is further used for generating a depth map for the specified sub-region based on the depth data stored in the storage unit.

In a fourth aspect, an embodiment of the present invention provides an electronic device, including a main control unit, a communication interface, a memory, a communication bus, a SPAD sensor, an RGB sensor, or an IR sensor according to any one of the methods of the first aspect, where the main control unit, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

the main control unit is used for implementing any one of the method steps in the first aspect when executing the program stored in the memory.

In a fifth aspect, embodiments of the present invention provide a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the method steps of any of the first aspects.

The embodiment of the invention has the beneficial effects that:

according to the depth map generation method provided by the embodiment of the invention, the target subarray corresponding to the appointed subarea in the view range in the SPAD array can be determined, and then only the depth data of each SPAD unit in the target subarray is stored in the storage unit, and the depth map of the appointed subarea is generated based on the stored depth data. After the detection, only the depth data of the SPAD units in the target subarray are stored in the storage unit, instead of storing the depth data of all SPAD units in the SPAD array in the storage unit, so that the data required to be stored in the storage unit can be smaller than the depth data of all SPAD units in the SPAD array, the array scale of the SPAD array is improved, the storage area is not required to be improved, and the limitation of data storage on the improvement of the array scale of the SPAD array can be avoided.

Further, since the target subarray in the SPAD array corresponds to the designated subarea in the visual field range, this means that only the depth map in the designated subarea is generated when the depth map is generated, and the depth maps of other areas in the visual field range are not required to be generated, so that the calculation amount required for generating the depth map each time can be reduced.

Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic illustration of a SPAD array;

FIG. 2 is a flowchart of a depth map generating method according to an embodiment of the present invention;

FIG. 3 is another flowchart of a depth map generating method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a target sub-array according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a SPAD array incorporating a decoder according to an embodiment of the present invention;

FIG. 6 is another flowchart of a depth map generating method according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a histogram according to an embodiment of the present invention;

FIG. 8 is a schematic block diagram of a depth map generation method for an RGB/IR sensor according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a SPAD sensor according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of another embodiment of a SPAD sensor according to the present invention;

fig. 11 is a schematic structural diagram of a depth map generating apparatus according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

A SPAD sensor is a sensor that detects target depth data through a SPAD array. A SPAD array is an array of a plurality of SPAD cells, each SPAD cell comprising a SPAD and a TDC (Time Digital Converter, time-to-digital converter). Exemplary, as shown in fig. 1, a SPAD array is schematically illustrated, where the SPAD array in fig. 1 includes m×n SPAD cells.

In the related art, when using a depth map generated by a SPAD sensor, a storage unit in the SPAD sensor is required to store depth data detected by all SPAD units in the SPAD array. This means that with the increasing array size of the SPAD array, after each detection, the data that needs to be stored by the storage unit is increased, and further the more the chip area that needs to be occupied by the storage unit is, the more the chip area in the SPAD sensor is limited, so that the chip area occupied by the storage unit is difficult to be increased, so that data storage becomes an important factor for limiting the increase of the array size of the SPAD array.

Since for many monitoring scenes the image frames of the majority of the monitoring areas are stationary and different, the frames of the local areas may change only when a target object is present in the monitoring scene. Compared with the whole monitoring picture, the local area is generally far smaller than the original picture in size, so the invention uses the characteristic that the local area is used as the appointed subarea, only 3D information of the appointed subarea is stored, processed and generated, the data required to be stored and processed is reduced, and the storage area is further reduced, thereby avoiding the limitation of data storage on the increase of the array scale of the SPAD array.

Based on the above, the embodiment of the invention provides a depth map generation method applied to a main control unit.

It should be noted that the above-mentioned main control unit may be a computing processing unit in the image capturing device where the SPAD sensor is located, and may include electrical components such as a processor and a memory, where the main control unit includes, but is not limited to, a CPU (Central Processing Unit ), a DSP (Digital Signal Processing, digital signal processing) unit, an ARM (Advanced RISC Machines, advanced reduced instruction set processor), and an ASIC (Application Specific Integrated Circuit ), or the above-mentioned main control unit may also be various electronic devices, for example, a computer, a server, and other devices with data processing capabilities. In addition, the depth map generating method provided by the embodiment of the invention can be realized by software, hardware or a combination of software and hardware. Furthermore, the main control unit needs to be connected with the SPAD sensor, and the main control unit and the SPAD sensor can be directly and electrically connected, or indirectly connected through intermediate equipment.

The depth map generating method provided by the embodiment of the invention can comprise the following steps:

determining a target subarray in a SPAD array in a SPAD sensor; wherein, the target subarray is: subarrays in the SPAD array matched with the appointed subareas in the target detection area; the target detection area is an area detected by the SPAD sensor during detection; designating a sub-region as determined based on an image acquired by the RGB sensor or the IR sensor;

controlling the SPAD sensor to detect depth data in a target detection area;

based on the depth data stored in the memory unit, a depth map specifying the sub-region is generated.

According to the depth map generation method provided by the embodiment of the invention, after detection, only the depth data of the SPAD units in the target subarray are stored in the storage unit, instead of storing the depth data of all the SPAD units in the SPAD array in the storage unit, so that the data required to be stored in the storage unit can be smaller than the depth data of all the SPAD units in the SPAD array, the array scale of the SPAD array can be improved, the storage area can not be improved, and the limitation of data storage on the improvement of the array scale of the SPAD array can be avoided.

The depth map generating method, the detecting system and the SPAD sensor provided by the embodiment of the invention are described in detail below with reference to the accompanying drawings.

As shown in fig. 2, the depth map generating method provided by the embodiment of the present invention is applied to a main control unit, where the main control unit is connected to a SPAD sensor, and may include the following steps:

s201, determining a target subarray in a SPAD array in the SPAD sensor; wherein, the target subarray is: subarrays in the SPAD array matched with the appointed subareas in the target detection area; the target detection area is an area detected by the SPAD sensor during detection; designating a sub-region as determined based on an image acquired by the RGB sensor or the IR sensor;

the region detected during detection by the SPAD sensor is a region in which the SPAD sensor acquires depth data, for example, when an indoor depth map is constructed, the SPAD sensor needs to be used for acquiring the depth data in the room to be constructed, namely, the SPAD sensor is used for detecting the room to be constructed, and the region detected during detection by the SPAD sensor is the target detection region in the room to be constructed.

The designated subarea in the target detection area can be determined based on the image acquired by the RGB sensor or the IR sensor, and the target subarea matched with the designated subarea is: and the subarray in the SPAD array is used for detecting the depth data of the appointed subarea. Illustratively, the SPAD array sub-array 1 is used for detecting depth data of a sub-region 1 in a target detection region, the SPAD array sub-array 2 is used for detecting depth data of a sub-region 2 in the target detection region, the SPAD array sub-array 3 is used for detecting depth data of a sub-region 3 in the target detection region, and when the sub-region 2 is a designated sub-region, the SPAD array sub-array 2 is a target sub-array.

Because the target subarray is matched with the specified subarea, the target subarray is determined from the SPAD array based on the predetermined specified subarea. For example, the position information of the designated sub-area in the target detection area may be determined first, and then, based on the determined position information, a target sub-array matching the designated sub-area may be determined from the SPAD array.

At this time, in this step, determining the target subarray in the SPAD array in the SPAD sensor may include:

sub-array position information aiming at a SPAD array in the SPAD sensor is obtained, and a sub-array indicated by the sub-array position information in the SPAD array is determined to be a target sub-array; wherein the sub-array position information is determined based on the position information of the specified sub-area.

The sub-array position information of the SPAD array in the SPAD sensor can be obtained in various ways, and the sub-array position information can be obtained at least by any one of the following two ways:

the first seed array position information determining mode is an automatic generation mode and comprises the following steps:

step 1: acquiring a two-dimensional image of a target detection area, and identifying an interested target on the two-dimensional image;

the two-dimensional image may be an image obtained by a CIS (Contact Image Sensor ) such as an RGB sensor or an IR (Infrared Radiation, infrared) sensor. At this time, step 1 may include: a two-dimensional image of the target detection area is acquired by means of an RGB sensor and/or an IR sensor. Alternatively, to reduce errors, the distance between the sensor for acquiring the two-dimensional image and the SPAD sensor should be as small as possible, such as 1cm or less, on the structural layout.

The object of interest may be determined in connection with the actual scene and the requirements, for example the object of interest may be a moving object or an object comprising specific features, such as wearing black clothing.

The two-dimensional image can be subjected to interested target detection in various modes, for example, an interested target detection algorithm such as a frame subtraction method, a background subtraction method, an optical flow method and the like can be adopted to carry out the interested target detection on the two-dimensional image, or a neural network model can be trained in advance to obtain an interested target detection model for the interested target detection, and then the interested target detection model is utilized to carry out the interested target identification on the two-dimensional image.

Step 2: when an object of interest is detected in the two-dimensional image, determining a region in which the detected object of interest is located as a specified sub-region;

the method comprises the steps that an interested target is detected in a two-dimensional image, namely, in a target detection area, the area where the interested target is located is changed in a picture, and pictures of other areas are not changed, so that when the interested target is detected, the area where the interested target is located can be determined to be a specified subarea.

Step 3: and mapping the determined position information of the designated subarea based on a position mapping relation between the pre-constructed two-dimensional image and the SPAD array to obtain subarray position information aiming at the SPAD array in the SPAD sensor.

The position information of the designated sub-area may be information indicating the position of the designated sub-area in the target detection area, and there may exist various expression forms, for example, in one implementation manner, a set of pixel coordinates of each pixel point in the designated sub-area may be used, or in another implementation manner, a diagonal coordinate manner of the designated sub-area may be used, for example, a form of { pixel coordinates of an upper left corner pixel point, pixel coordinates of a lower right corner pixel point }, or { pixel coordinates of an upper left corner pixel point, pixel coordinates of an upper right corner pixel point } may be used, and in yet another implementation manner, a form of extending an angular coordinate by a wide may be used, for example, a form of { angular pixel coordinates, a length, a width } may be used.

Similar to the position information of the designated sub-area, there may be various kinds of performance information of the sub-array, and the specific performance form may be referred to the above examples of the performance form of the position information of the designated sub-area, which will not be described herein.

The above-mentioned position mapping relationship may be: based on the internal and external parameters of the SPAD sensor, and the internal and external parameters of the RGB sensor or the IR sensor.

There are various manners of the pre-constructed position mapping relationship between the two-dimensional image and the SPAD array, for example, in one implementation manner, the position mapping relationship between the two-dimensional image and the SPAD array may be determined according to the internal and external parameters of the sensor for acquiring the two-dimensional image and the SPAD sensor.

Or in another implementation manner, the position mapping relation can also obtain multiple groups of calibration data between the sensor for acquiring the two-dimensional image and the SPAD sensor in a calibration manner, and then fit the position mapping relation between the two-dimensional image and the SPAD array by using the calibration data.

After the position mapping relation between the two-dimensional image and the SPAD array is constructed, the determined position information of the designated subarea can be mapped based on the position mapping relation between the two-dimensional image and the SPAD array, so that the subarea position information of the SPAD array in the SPAD sensor can be obtained.

The second seed array position information determining mode is a manual configuration mode. Specifically, when a configuration operation for sub-array position information is received, the position information configured by the configuration operation may be used as sub-array position information, where the sub-array position information configured by the configuration operation may be determined by a user according to a requirement or position information of a specified sub-area, and the configuration operation may be a selection operation, an input operation, or the like.

S202, controlling the SPAD sensor to detect depth data in a target detection area;

in this step, the laser may be controlled to emit an optical signal to the target detection area, and the reflected light is received by the SPAD array in the SPAD sensor, and each SPAD unit in the SPAD array may record the time difference between the emitted light and the returned light.

In general, the larger the distance between the SAPD sensor and the measurement point, the larger the time difference recorded by each SPAD cell, and thus, the time difference can be used as depth data of each measurement point. Alternatively, the distance between each measurement point and the SPAD sensor may be calculated based on the propagation speed of light and the time difference, or the normalized value may be used as the depth data to be detected after the normalization processing is performed on the detected time difference or the calculated distance, which is not particularly limited in the embodiment of the present invention.

S203, storing only the depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor;

after the SPAD sensor detects, only each SPAD unit in the SPAD sensor can record the depth data detected by the SPAD unit, and further the depth data detected by each SPAD unit can be stored in a storage unit in the SPAD sensor.

Specifically, the main control unit may send a data storage instruction to the SPAD sensor to control the SPAD sensor to store depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor.

There are many ways in which the SPAD sensor stores the depth data detected by each SPAD unit in the target subarray into the storage unit in the SPAD sensor, for example, in one implementation, the depth data detected by each SPAD unit in the target subarray may be written into the storage unit at the same time, or in another implementation, the depth data detected by each SPAD unit in the target subarray may be sequentially stored into the storage unit in the SPAD sensor.

S204, generating a depth map of the appointed subarea based on the depth data stored in the storage unit.

Because the storage unit only stores the depth data detected by each SPAD unit in the target subarray, the main control unit can generate a depth map of the appointed subarea by using the depth data stored in the storage unit. Specifically, the main control unit may determine a depth value of each measurement point in the specified sub-area by using the depth data stored in the storage unit, so as to construct a depth map of the specified sub-area.

Based on the embodiment shown in fig. 2, the embodiment of the present invention further provides a depth map generating method, as shown in fig. 3, where storing depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor may include:

s301, selecting a SPAD unit from all SPAD units in a target subarray according to a preset SPAD unit selection sequence to serve as a target SPAD unit;

in order to store the depth data in each SPAD unit in the target subarray into the storage unit, the depth data in each SPAD unit in the target subarray may be sequentially stored, and based on this, a SPAD unit may be selected from the SPAD units in the target subarray according to a preset SPAD unit selection order, and may be used as the target SPAD unit. The target SPAD unit selected at each time is the SPAD unit which needs to store data at the time.

The selection sequence of the SPAD units can be determined according to the position of the SPAD units in the target subarray, and in one implementation manner, the selection sequence of each SPAD unit can be determined according to the sequence of left, right, top and bottom.

For example, as shown in fig. 4, which is a schematic diagram of a target subarray, the gray part in fig. 4 is a target subarray in a SPAD array, and the target subarray includes 9 SPAD units, namely, SPAD unit 1-SPAD unit 9, and if the order of SPAD unit 1, SPAD unit 2, SPAD unit 3, SPAD unit 4, SPAD unit 5, SPAD unit 6, SPAD unit 7, SPAD unit 8, SPAD unit 9 is in the order of left, right, top, bottom, the order table { SPAD unit 1, SPAD unit 2, SPAD unit 3, SPAD unit 4, SPAD unit 5, SPAD unit 7, SPAD unit 8, SPAD unit 9}.

It should be noted that, in addition to selecting the target SPAD unit from the SPAD units in the target subarray according to the preset SPAD unit selection sequence, other modes may be adopted to select the target SPAD unit according to actual requirements. For example, it is also possible to randomly select a target SPAD cell from among the SPAD cells within the target sub-array.

S302, storing depth data detected by a target SPAD unit into a storage unit in the SPAD sensor;

in this step, after the target SPAD unit is determined, the depth data detected by the target SPAD unit may be stored in a storage unit in the SPAD sensor. Optionally, the main control unit may send a storage instruction for the target SPAD unit to the SPAD sensor, so as to control the SPAD sensor to store the depth data detected by the target SPAD unit into the storage unit.

Specifically, the storing the depth data detected by the target SPAD unit into the storage unit in the SPAD sensor may include a step two:

step one: configuring the output state of the target SPAD cell into an outputtable state;

the output state of each SPAD unit is used for representing whether the SPAD unit can output the detected depth data, if the output state of the SPAD unit is the outputtable state, the SPAD unit can output the detected detection data, and if the output state of the SPAD unit is the non-outputtable state, the SPAD unit can not output the detected detection data.

In order to ensure that only depth data of one SPAD unit is stored at a time, the master control unit may configure the output state of each SPAD unit in the SPAD array to a non-outputtable state before performing the data storage step. After the target SPAD cell is determined, only the output state of the SPAD cell may be configured to be an outputtable state, so that only depth data within the target SPAD cell is stored when data storage is performed.

The above-described process of configuring the output state of the target SPAD cell to an outputtable state may be performed in combination with the hardware structure of the SPAD sensor. As shown in fig. 5, which is a schematic view of a SPAD array combined with a decoder, in fig. 5, the SPAD sensor further includes a selection circuit, where the selection circuit is composed of a row decoder and a column decoder, the row decoder is used for outputting a row signal to each SPAD unit, and the column decoder is used for outputting a column signal to each SPAD unit. For each SPAD cell, the SPAD cell changes its output state to an outputtable state only when the row signal and the column signal are received simultaneously.

At this time, in this step, configuring the output state of the target SPAD cell to be an outputtable state may include:

the control row decoder outputs a row signal to the target SPAD cell and the control column decoder outputs a column signal to the target SPAD cell to configure an output state of the target SPAD cell to an outputtable state.

Specifically, after determining the target SPAD unit, the main control unit may send signal control instructions to the row decoder and the inner decoder, respectively, based on the position information of the target SPAD unit, to control the row decoder to output a row signal to the target SPAD unit, and to control the column decoder to output a column signal to the target SPAD unit.

Illustratively, the target SPAD element is SPAD element 5 in fig. 5, and the position information of SPAD element 5 is the array coordinates (3, 3) of SPAD element 5 in the SPAD array, which means that SPAD element 5 is located at row 3 and column 3 in the array. After determining that the SPAD unit 5 is the target SPAD unit, the main control unit may send a signal control instruction for the 3 rd row to the row decoder, control the row decoder to output a row signal to the SPAD unit of the 3 rd row, and similarly, the main control unit may send a signal control instruction for the 3 rd column to the column decoder, and control the column decoder to output a column signal to the SPAD unit of the 3 rd column, so that the SPAD unit 5 may receive the row signal and the column signal at the same time, and the output state is configured to be an outputtable state.

Step two: and controlling the target SPAD unit in the outputtable state, and outputting the detected depth data to a storage unit in the SPAD sensor.

In this step, after the output state of the target SPAD unit is configured to be the outputtable state, the target SPAD unit in the outputtable state can be controlled, and the detected depth data is output to the storage unit in the SPAD sensor.

Specifically, the main control unit may send a data storage instruction for the entire SPAD array to the SPAD sensor, and since only the output state of the target SPAD unit in the entire SPAD array is in an outputtable state, the data storage instruction for the entire SPAD array can only control the target SPAD unit to output the detected depth data to the storage unit in the SPAD sensor.

S303, judging whether the detection data in each SPAD unit in the target subarray is stored or not, if not, returning to the step S301, and if so, ending.

If the detected data in each SPAD unit in the target sub-array is not stored completely, S301 needs to be executed, and the process of determining the target SPAD unit from each SPAD unit in the target sub-array and storing the SPAD unit data is executed until all the detected data in the SPAD units in the target sub-array are stored, so that the process of storing the depth data detected by each SPAD unit in the target sub-array into the storage unit in the SPAD sensor is completed.

The depth map generation method provided by the embodiment of the invention can avoid the limitation of data storage on the increase of the array scale of the SPAD array, and can reduce the calculation amount required for each time of depth map generation. Furthermore, as the target SPAD unit can be selected and only the depth data of the SPAD unit is stored in the storage unit, abnormal data storage caused by simultaneous storage of the depth data in the SPAD unit can be avoided, so that the storage stability can be improved, and meanwhile, an implementation basis is provided for avoiding limitation of data storage on the increase of the array scale of the SPAD array.

Because the depth data obtained by single measurement is often incomplete or has errors, in order to provide the quality of the generated depth map, the depth map is often required to be measured for multiple times to generate the depth map based on the depth data detected for multiple times, and based on the depth map, as shown in fig. 6, the embodiment of the invention further provides a depth map generation method, which comprises the following steps:

s601, determining a target subarray in a SPAD array in the SPAD sensor; wherein, the target subarray is: subarrays in the SPAD array matched with the appointed subareas in the target detection area; the target detection area is an area detected by the SPAD sensor during detection; designating a sub-region as determined based on an image acquired by the RGB sensor or the IR sensor;

In this step, the same as step S201 is referred to in step S201, and detailed description is omitted again.

S602, controlling the SPAD sensor to detect depth data in a target detection area;

in this step, the same as step S202 is referred to as step S203, and detailed description thereof is omitted.

S603, storing the depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor;

in this step, the same as step S203 is referred to as step S203, and detailed description is omitted again.

S604, judging whether the number of times of depth data detection by the SPAD sensor reaches a preset detection number threshold, if not, executing the step S602, otherwise, executing the step S605.

The preset detection frequency threshold value can be determined according to experience and requirements, and can be 1000, 5000 and the like. If the number of times of depth data detection by the SPAD sensor does not reach the detection number threshold, the depth data detection is still needed, that is, the step S602 is executed again, if the number of times of depth data detection by the SPAD sensor reaches the detection number threshold, it is indicated that the data acquisition result can be the result, and the step S605 is executed.

In this step, the judgment of whether the number of times of depth data detection by the SPAD sensor reaches the preset detection number threshold may be implemented by using a counter, that is, after each time of depth data detection by the SPAD sensor is completed, the counter performs a 1 adding operation, so that it may be judged whether the number of times of depth data detection by the SPAD sensor reaches the preset detection number threshold by comparing the number of times recorded by the counter with the detection number threshold.

S605 generates a depth map specifying a sub-region based on the depth data stored in the storage unit.

Because the storage unit stores the depth data detected by the target subarray with the threshold number of times of detection, the optimal depth data can be obtained by utilizing the depth data detected by the target subarray for multiple times.

For example, for each SPAD element in the target subarray, the depth data with the largest proportion may be selected from the depth data measured by the SPAD element as the depth data of the SPAD element.

Further, in the case where the depth data is stored in the form of a histogram, for each SPAD cell in the target subarray, the abscissa (represented distance) corresponding to the data having the largest value may be selected from the histogram in the storage unit as the target detection data of the SPAD cell, and further, the depth map of the specified subarray may be constructed based on the target detection data of each SPAD cell in the target subarray.

The depth map generation method provided by the embodiment of the invention can avoid the limitation of data storage on the increase of the array scale of the SPAD array, and can reduce the calculation amount required for each time of depth map generation. Further, since the depth map is constructed using the depth data detected by the plurality of times of depth data, the quality of the depth map can be improved.

Optionally, in the method for generating a depth map according to the embodiment of the present invention, in order to improve efficiency of final generation of the depth map, after storing the depth data detected by each SPAD unit in the target subarray into the storage unit in the SPAD sensor, the method may include:

and constructing a histogram corresponding to each SPAD unit based on the depth data detected by each SPAD unit in the storage unit, wherein the histogram corresponding to each SPAD unit is used for storing the depth data detected by the SPAD unit.

The Histogram (Histogram), also called quality distribution map, is a statistical report map. For each SPAD cell, the number of each depth data may be counted based on the depth data detected by the SPAD cell in the storage cell, as shown in fig. 7, which is an exemplary histogram diagram provided in an embodiment of the present invention, wherein the abscissa in the histogram diagram is a depth value corresponding to the depth data detected by the SPAD cell corresponding to the histogram, the ordinate is the number of times each depth data is detected by the SPAD cell, taking the depth value 1 in fig. 7 as an example, the number of depth values corresponding to 1 is 10, which indicates that in the SPAD cells corresponding to the target subarray, the depth value corresponding to the 10 detection results is 1.

At this time, the generating the depth map for the specified sub-region based on the depth data stored in the storage unit may include steps a to B:

step A: selecting detection data with the largest detection ratio of the SPAD unit from a histogram corresponding to the SPAD unit aiming at each SPAD unit in the target subarray as target detection data of the SPAD unit;

since the histogram of each SPAD unit records the depth data detected by the SPAD unit in the whole detection period, when the detection times in each detection period are enough, the detection data with the largest proportion in the histogram corresponding to each SPAD unit can be considered as accurate detection data, and therefore, after the histograms corresponding to the SPAD units are obtained, the detection data with the largest proportion detected by the SPAD unit can be selected from the histograms corresponding to the SPAD units as the target detection data of the SPAD units.

Illustratively, the histogram shown in fig. 7 contains probe data: the method comprises the steps of detecting a depth value 1, detecting a depth value 2, detecting a depth value 3 and detecting a depth value 4, wherein the detection of the depth value 1 is 10 times, the detection of the depth value 2 is 15 times, the detection of the depth value 4 is 8 times, the detection of the depth value 4 is 20 times, and the depth value 4 is taken as target detection data.

And (B) step (B): and constructing a depth map aiming at the appointed subarea based on the target detection data of each SPAD unit in the target subarea.

After the target detection data of each SPAD unit is obtained, the target detection data of each SPAD unit can be used as depth data of a pixel point corresponding to the SPAD unit in a depth map to be generated, and each SPAD unit is traversed in sequence to obtain the depth map aiming at the appointed subarea.

According to the depth map generation method provided by the embodiment of the invention, through the depth map, the target depth data corresponding to each SPAD unit can be conveniently and accurately determined, and further the generation efficiency and accuracy of the depth map are improved.

Optionally, in the depth map generating method provided by the embodiment of the present invention, the target detection area is divided into a plurality of designated subareas, at this time, before determining the target subarea in the SPAD array in the SPAD sensor, the method further includes determining a designated subarea from the plurality of designated subareas, and using the subarea matched with the designated subarea in the SPAD array as the target subarea, further after generating the depth map for the designated subarea based on the depth data stored in the storage unit, determining a new designated subarea from the plurality of designated subareas, and returning the subarea matched with the designated subarea in the SPAD array as the target subarea, until generating the depth map for each designated subarea, thereby generating the depth map for the target detection area. Therefore, by adopting the technical scheme of the embodiment of the invention, the generated data volume can be reduced, and the capability of monitoring in a large range can be maintained.

In order to more clearly illustrate the depth map generation method provided by the embodiment of the present invention, as shown in fig. 8, the embodiment of the present invention provides a depth map generation schematic block diagram combined with an RGB/IR sensor. In the figure, an RGB/IR sensor is an RGB/IR sensor, a SPAD sensor is a SPAD sensor, each spad+tdc forms a SPAD unit, and an SRAM (Static Random-Access Memory) is a storage unit in the SPAD sensor.

The RGB/IR sensor transmits the acquired two-dimensional image to the main control unit; the main control unit performs target detection on the two-dimensional image to obtain a target ROI, wherein the ROI is in the two-dimensional image, so that the main control unit needs to perform ROI according to the inner parameter and the outer parameter of the RGB/IR sensor and the SPAD sensor _{_RGB/IR} Conversion to ROI _{_SPAD} Wherein, ROI _{_RGB/IR} For the position information of the ROI in the two-dimensional image, i.e. the position information of the specified sub-region mentioned above in the embodiment of the invention, the ROI _{_SPAD} The position information of the ROI in the SPAD array is the sub-array position information of the target sub-array mentioned above in the embodiment of the present invention. Further, the central control unit will ROI _{_SPAD} To the SPAD sensor. Wherein the ROI is _{_SPAD} Or may be generated by the master control unit based on a configuration between received configuration operations.

SPAD sensor based on entered ROI _{_SPAD} By selecting the electrical network through P.Q, the gating paths in the TDC and the SARM in each SPAD unit are switched, and only the depth map of the ROI is stored, processed and generated, so that P.Q pixels can be selected from M.N pixels and stored in P.Q storage units, limitation of data storage on the increase of the array scale of the SPAD array can be avoided, and the calculated amount required for each time of depth map generation is reduced.

As shown in fig. 9, an embodiment of the present invention further provides a SPAD sensor, including: SPAD array 901, selection circuitry 902, and memory cells 903, selection circuitry 902 includes: a row decoder 9021 and a column decoder 9022, the row decoder 9021 is configured to output a row signal to each SPAD cell 9011 in the SPAD array 901, and the column decoder 9022 is configured to output a column signal to each SPAD cell 9011;

the SPAD sensor is used for detecting depth data of a target detection area under the control of the main control unit, wherein the target detection area is an area detected by the SPAD sensor during detection;

the row decoder 9021 is configured to, when receiving an output instruction of the main control unit, output a row signal to a target SPAD unit indicated by the received output instruction; the target SPAD unit is a SPAD unit 9011 determined by the main control unit in each SPAD unit 9011 in a target subarray, and the target subarray is a subarray which is determined by the main control unit in the SPAD array and matched with a designated subarea in a target detection area;

A column decoder 9022, configured to, when receiving an output instruction of the main control unit, output a row signal to a target SPAD unit 9011 indicated by the received output instruction;

any SPAD unit 9011 in the SPAD array converts the output state into an outputtable state when receiving the row signal and the column signal at the same time;

the SPAD unit 9011 in an outputtable state is configured to write detected detection data into the storage unit 903 when receiving a storage instruction of the main control unit;

the storage unit 903 is configured to output the stored probe data when receiving a control instruction of the main control unit, so that the main control unit generates a depth map of the specified sub-region based on the depth data output by the storage unit.

According to the SPAD sensor provided by the embodiment of the invention, after detection, only the depth data of the SPAD units in the target subarray are stored in the storage unit, instead of storing the depth data of all the SPAD units in the SPAD array in the storage unit, so that the data required to be stored in the storage unit can be smaller than the depth data of all the SPAD units in the SPAD array, the array scale of the SPAD array can be improved, the storage area can not be improved, and the limitation of data storage on the improvement of the array scale of the SPAD array can be avoided.

Fig. 10 is a schematic structural diagram of another SPAD sensor according to the present embodiment. Each TDC square in the figure represents a SPAD unit 9011, and a ram (Random Access Memory ) module is a storage unit.

The row decoder 9021 may include a plurality of row output lines, a row output line of the row decoder 9021 being connected to one row SPAD cell 9011 in the SPAD array, each row output line being for outputting a row signal to the SPAD cell 9011 connected thereto;

the column decoder 9022 includes a plurality of column output lines, one column output line of the column decoder 9022 is connected to one column SPAD cell 9011 in the SPAD array, and each column output line is for outputting a column signal to the SPAD cell connected thereto.

The SPAD sensor provided by the embodiment of the invention can avoid the limitation of data storage on the increase of the array scale of the SPAD array, can reduce the calculation amount required by each time of generating the depth map, and further can realize flexible mapping and switching between pixels and a storage module in a digital domain by utilizing a row decoder and a column decoder without being limited by a region, and has higher storage utilization rate.

Corresponding to the depth map generating method provided by the embodiment of the present invention, as shown in fig. 11, the embodiment of the present invention further provides a detection system, where the depth map generating system includes: a main control unit 1101, SPAD sensor 1102, RGB sensor or IR sensor 1103;

an RGB sensor or IR sensor 1103 for acquiring an image of the target detection area, determining a specified sub-area based on the acquired image, and transmitting the determined specified sub-area to the main control unit 1101;

the main control unit 1101 is configured to receive the specified sub-area, determine a target sub-array in the SPAD sensor 1102 based on the specified sub-area, and send a control signal to the SPAD sensor 1102 based on the target sub-array;

the SPAD sensor 1102 is configured to detect depth data in a target detection area after receiving a control signal, and store only the depth data detected by each SPAD unit in the target subarray into a storage unit in the SPAD sensor 1102;

the main control unit 1101 is further configured to generate a depth map for the specified sub-region based on the depth data stored in the storage unit.

Alternatively, in one implementation, the RGB sensor or the IR sensor 1103 may acquire a two-dimensional image of the target detection area and perform target recognition of interest on the two-dimensional image, and when the target of interest is detected in the two-dimensional image, the RGB sensor or the IR sensor 1103 may determine the area in which the detected target of interest is located as the specified sub-area.

The main control unit 1101 may map the determined position information of the designated sub-area based on a position mapping relationship between the two-dimensional image and the SPAD array, so as to obtain sub-array position information for the SPAD array in the SPAD sensor, and further determine the sub-array indicated by the sub-array position information in the SPAD array as the target sub-array.

Alternatively, in one implementation, the main control unit 1101 may select a SPAD unit from the SPAD units in the target subarray of the SPAD sensor 1102 according to a preset SPAD unit selection order, and use the SPAD unit as the target SPAD unit. After determining the target SPAD cell, a storage instruction for the target SPAD can be sent to the SPAD sensor 1102, so that the SPAD sensor 1102 stores the depth data detected by the target SPAD cell into the storage cell;

when the depth data detected by the target SPAD unit is stored in the storage unit, the main control unit 1101 may continue to return to select a SPAD unit from the SPAD units in the target subarray of the SPAD sensor 1102 according to the preset selection order of the SPAD units, as a step of the target SPAD unit, until the detected data of the SPAD units are stored.

Optionally, the SPAD sensor 1102 further includes a selection circuit, where the selection circuit includes: the row decoder is used for outputting row signals to each SPAD unit, and the column decoder is used for outputting column signals to each SPAD unit;

the main control unit 1101 may control the row decoder to output a row signal to the target SPAD unit and control the column decoder to output a column signal to the target SPAD unit to configure an output state of the target SPAD unit to an outputtable state; and controlling the target SPAD unit in the outputtable state, and outputting the detected depth data to a storage unit in the SPAD sensor.

The main control unit 1101 may determine whether the number of times the SPAD sensor detects depth data reaches a preset detection number threshold before generating a depth map for a specified sub-region based on the depth data stored in the storage unit; if not, returning to the step of controlling the SPAD sensor to detect the depth data in the target detection area; if so, performing a related step of generating a depth map for the specified sub-region.

In one implementation, the main control unit 1101 further includes, after storing depth data detected by each SPAD unit in the target subarray in a storage unit in the SPAD sensor: based on the depth data detected by each SPAD unit in the storage unit, constructing a histogram corresponding to each SPAD unit, wherein the histogram corresponding to each SPAD unit is used for storing the depth data detected by the SPAD unit, further, according to each SPAD unit in the target subarray, the detection data with the largest detection ratio detected by the SPAD unit is selected from the histograms corresponding to the SPAD units and used as the target detection data of the SPAD unit, and based on the target detection data of each SPAD unit in the target subarray, a depth map aiming at the appointed subarray is constructed.

Optionally, if the target detection area is divided into a plurality of designated subareas, the main control unit 1101 may determine a designated subarea from the plurality of designated subareas before determining the target subarea in the SPAD array in the SPAD sensor 1102, and use the subarea matched with the designated subarea in the SPAD array as the target subarea, further determine a new designated subarea from the plurality of designated subareas after generating the depth map for the designated subarea based on the depth data stored in the storage unit, and return the subarea matched with the designated subarea in the SPAD array as the target subarea until the depth map is generated for each designated subarea.

According to the detection system provided by the embodiment of the invention, after detection, only the depth data of the SPAD units in the target subarray are stored in the storage unit, instead of storing the depth data of all the SPAD units in the SPAD array in the storage unit, so that the data required to be stored in the storage unit can be smaller than the depth data of all the SPAD units in the SPAD array, the array scale of the SPAD array can be improved, the storage area can not be improved, and the limitation of data storage on the improvement of the array scale of the SPAD array can be avoided.

The embodiment of the present invention further provides an electronic device, as shown in fig. 12, including a main control unit 1201, a communication interface 1202, a memory 1203, a communication bus 1204, a SPAD sensor 1205, an RGB sensor or an IR sensor 1206, where the main control unit 1201, the communication interface 1202, and the memory 1203 complete communication with each other through the communication bus 1204,

a memory 1203 for storing a computer program;

the main control unit 1201 is configured to implement the steps of the depth map generating method according to the embodiment of the present invention when executing the program stored in the memory 1203.

The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The main control unit may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In yet another embodiment of the present invention, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor, implements the steps of any of the depth map generation methods described above.

In a further embodiment of the present invention, a computer program product comprising instructions which, when run on a computer, cause the computer to perform the depth map generating method of any of the above embodiments is also provided.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system, apparatus embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. The depth map generation method is characterized by being applied to a main control unit, wherein the main control unit is connected with a SPAD sensor, and the method comprises the following steps:

determining a target subarray in a SPAD array in the SPAD sensor; wherein the target subarray is: subarrays matched with a designated subarea in a target detection area in the SPAD array; the target detection area is as follows: the SPAD sensor detects the detected area; the specified sub-area is determined based on images acquired by an RGB sensor or an IR sensor;

controlling the SPAD sensor to detect depth data in the target detection area;

2. The method of claim 1, wherein the determining a target sub-array in a SPAD array within the SPAD sensor comprises:

mapping the determined position information of the appointed sub-region based on a pre-constructed position mapping relation between the two-dimensional image and the SPAD array to obtain sub-array position information aiming at the SPAD array in the SPAD sensor;

3. The method of claim 2, wherein the acquiring a two-dimensional image of the target detection zone comprises:

4. A method according to claim 3, wherein the location mapping relationship is: based on the internal and external parameters of the SPAD sensor, and the internal and external parameters of the RGB sensor or the IR sensor.

5. The method of claim 1, wherein storing only depth data detected by SPAD cells within the target subarray into memory cells within the SPAD sensor comprises:

6. The method of claim 5, wherein the SPAD sensor further comprises a selection circuit, the selection circuit comprising: a row decoder for outputting a row signal to each SPAD cell and a column decoder for outputting a column signal to each SPAD cell;

7. The method of claim 1, wherein prior to the generating a depth map for the specified sub-region based on the depth data stored in the storage unit, the method further comprises:

8. The method of claim 7, further comprising, after storing depth data detected by each SPAD element within the target subarray in a storage element within the SPAD sensor:

9. The method of claim 1, wherein the target detection zone is divided into a plurality of designated sub-zones;

10. A SPAD sensor, said SPAD sensor comprising: SPAD array, selection circuit and memory cell, the selection circuit includes: the row decoder is used for outputting row signals to each SPAD unit in the SPAD array, and the column decoder is used for outputting column signals to each SPAD unit;

11. The SPAD sensor according to claim 10, wherein said row decoder comprises a plurality of row output lines, a row output line of said row decoder being connected to a row SPAD cell of said SPAD array, each row output line for outputting a row signal to a SPAD cell connected thereto;

12. A detection system, wherein the depth map generation system comprises: the device comprises a main control unit, a SPAD sensor, an RGB sensor or an IR sensor;

13. An electronic device comprising a master control unit, a communication interface, a memory, a communication bus, a SPAD sensor, an RGB sensor or an IR sensor according to the method of any one of claims 1-9, wherein the master control unit, the communication interface, the memory complete the communication with each other via the communication bus;

A memory for storing a computer program;

the main control unit is used for implementing the method steps of any one of claims 1-9 when executing the program stored on the memory.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a main control unit, implements the method steps of any of claims 1-9.