CN117579952A - Image acquisition device, data reading method, control assembly and storage medium - Google Patents

Abstract

The application discloses an image acquisition device, a data reading method, a control assembly and a storage medium, and relates to the technical field of image sensors. For efficient readout and transfer of pixel data. The image acquisition apparatus includes: the photosensitive pixel array unit is used for exposing according to the exposure control signal; the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit is used for reading out pixel data generated by exposure of the photosensitive pixel array unit; the data reading unit comprises N groups of reading lines, wherein the N groups of reading lines are used for reading pixel data generated by the N pixel groups; a group of readout lines connected to a pixel group; and the data transmission unit is used for transmitting the pixel data acquired by the data reading unit through a transmission interface of the data transmission unit.

Description

Image acquisition device, data reading method, control assembly and storage medium

Technical Field

The present disclosure relates to the field of image sensors, and more particularly, to an image acquisition device, a data readout method, a control module, and a storage medium.

Background

The image sensor collects image signals using an array of photosensitive pixels. In the process of collecting image signals, the photosensitive component (photosensitive element) of each pixel responds to incident light under the corresponding exposure time length, so that the light signals sensed by the point are converted into electric signals, and then the electric signals are collected and amplified by a reading circuit and then converted into digital signals through an analog-to-digital converter (ADC).

In the related art, the exposure mode of the image sensor includes long exposure, short exposure, and the like, if the pixels in the pixel array of the image sensor all adopt the form of long exposure, the obtained image has a higher signal-to-noise ratio, but the condition of blurring or overexposure may occur; if the pixels in the pixel array of the image sensor are all in the form of short exposure, the obtained image has no blurred signal, but the signal-to-noise ratio of the image is lower.

It can be understood that, for the same pixel, the perceived light signal amplitude will also change along with the change of the surrounding light intensity, so if all pixels adopt uniform and fixed exposure time, it is difficult to ensure that each pixel in the pixel matrix will not cause overexposure of the photosensitive element at each moment due to overlong exposure time; or, the exposure time is too short, so that the photosensitive element is underexposed, and the subsequent image processing effect is not facilitated. If the exposure time of each pixel is not uniform, it is difficult to avoid the transmission of invalid data bits of the pixels which are not exposed during the readout, which causes redundancy of the data bits and reduces the data transmission efficiency.

Disclosure of Invention

The application provides an image acquisition device, a data reading method, a control assembly and a storage medium, which are used for efficiently reading and transmitting pixel data.

In a first aspect, the present application provides an image acquisition apparatus comprising: the device comprises a photosensitive pixel array unit, a data reading unit and a data transmission unit; the photosensitive pixel array unit is used for exposing according to the exposure control signal; the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit is used for reading out pixel data generated by exposure of the photosensitive pixel array unit; the data reading unit comprises N groups of reading lines, wherein the N groups of reading lines are used for reading pixel data generated by the N pixel groups; a group of readout lines connected to a pixel group; and the data transmission unit is used for transmitting the pixel data acquired by the data reading unit through a transmission interface of the data transmission unit.

It can be understood that, in the image acquisition device provided in the present application, the photosensitive pixel array unit includes N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit is used for collecting pixel data generated by the photosensitive pixel array unit; the data reading unit comprises N groups of reading lines, and the N groups of reading lines are in one-to-one correspondence with the N pixel groups; and the data transmission unit is used for transmitting the pixel data acquired by the data reading unit through a transmission interface of the data transmission unit. In this way, under the condition that the exposure modes of the pixels in the photosensitive pixel array are different, the corresponding pixel data can be read out through the corresponding readout lines of each pixel group, so that on one hand, the pixel data generated by each pixel group can be read out in time, the pixel data generated by the pixel group does not need to be cached, and the other pixel groups do not need to wait for exposure, and the data reading efficiency is improved; on the other hand, one group of readout lines corresponds to one pixel group (the exposure time of pixels in one pixel group is the same), redundant data can be prevented from being read, the data volume entering a data transmission channel in unit time is reduced, the bandwidth requirement of data transmission is reduced, and the efficiency of data transmission can be effectively improved.

In a possible implementation, the data readout unit is specifically configured to read out the first pixel data generated by the first pixel group through the first group of readout lines; reading out second pixel data generated by the second pixel group through the second group read-out line; wherein the first group of sense lines and the second group of sense lines are any two groups of sense lines in the N groups of sense lines; the first pixel group and the second pixel group are any two pixel groups of the N pixel groups.

In another possible implementation manner, the image acquisition apparatus further includes: a bit width conversion unit; the bit width conversion unit is used for acquiring first pixel data, converting the original bit width of the first pixel data into the first bit width and transmitting the first bit width through the data transmission unit; the first pixel data is pixel data generated by exposing a first pixel group in the N pixel groups; acquiring second pixel data, converting the original bit width of the second pixel data into a second bit width, and transmitting the second bit width through a data transmission unit; the second pixel data is pixel data generated by exposing a second pixel group in the N pixel groups; n pixel groups correspond to N groups of bit widths; the first bit width and the second bit width are any two groups of bit widths of the N bit widths.

In another possible implementation, the N groups of bit widths are independent of each other; the first bit width is determined by the exposure time length of the first pixel group and/or the bandwidth of the data transmission unit; the second bit width is determined by the exposure time of the second pixel group and/or the bandwidth of the data transfer unit.

In another possible implementation, the data transmission unit includes one or more transmission channels; the data transmission unit is specifically configured to transmit the pixel data collected by the data readout unit to the transmission interface through one or more transmission channels, and then transmit the pixel data through the transmission interface.

In another possible implementation manner, the image acquisition apparatus further includes: a data processing unit; the data processing unit is used for acquiring the pixel data acquired by the data reading unit through the transmission interface, recovering the pixel data into a signal to be processed, and performing image processing on the signal to be processed.

In another possible implementation manner, the data processing unit is specifically configured to reorganize the pixel data according to one or more of an exposure duration of the pixel data, a readout time of the pixel data, an image channel where the pixel data is located, or position information of the pixel data, so as to obtain a signal to be processed, and perform image processing on the signal to be processed.

In another possible implementation manner, the data processing unit is specifically configured to perform image processing on the signal to be processed by using a neural network.

In another possible implementation manner, the image acquisition apparatus further includes: an exposure control unit; the exposure control unit comprises N groups of control lines, wherein the N groups of control lines are used for transmitting exposure control signals to the N pixel groups; a group of control lines is connected with a pixel group; an exposure control unit for transmitting a first exposure control signal to the first pixel group through the first group control line so that the first pixel group is exposed based on the first exposure control signal; transmitting a second exposure control signal to the second pixel group through the second group control line so that the second pixel group is exposed based on the second exposure control signal; the first set of control lines and the second set of control lines are any two sets of readout lines in the N sets of control lines; the first pixel group and the second pixel group are any two pixel groups of the N pixel groups.

In another possible implementation manner, the first exposure control signal is used to control a first exposure start time and a first exposure end time of the first pixel group; the second exposure control signal is used for controlling a second exposure start time and a second exposure end time of the second pixel group; and/or the first exposure control signal is used for controlling the times and the exposure time of the first pixel group for executing the exposure operation in the exposure period; the second exposure control signal is used for controlling the times and the exposure time of the second pixel group to execute the exposure operation in the exposure period.

In another possible implementation manner, the image acquisition apparatus further includes: a gain control unit; the gain control unit comprises N groups of signal lines; the N groups of signal lines are used for transmitting gain control signals to the N pixel groups; a group of signal lines is connected with one pixel group; a gain control unit for transmitting a first gain control signal to the first pixel group through the first group signal line so that the first pixel group adjusts a luminance value in the first pixel data based on the first gain control signal; the first pixel data is pixel data generated by exposing the first pixel group; transmitting a second gain control signal to the second pixel group through the second group signal line so that the second pixel group adjusts a luminance value in the second pixel data based on the second gain control signal; the second pixel data is pixel data generated by exposing the second pixel group; the first group of signal lines and the second group of signal lines are any two groups of signal lines in the N groups of signal lines; the first pixel group and the second pixel group are any two pixel groups of the N pixel groups.

In a second aspect, the present application provides a data readout method applied to an image acquisition apparatus, the image acquisition apparatus including: the device comprises a photosensitive pixel array unit, a data reading unit, a data transmission unit, a data processing unit and a control component, wherein the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit comprises N groups of reading lines, wherein the N groups of reading lines are used for reading pixel data generated by the N pixel groups; a group of readout lines connected to a pixel group; the method is applied to the control assembly; the method comprises the following steps: the control data reading unit reads out pixel data generated by the exposure operation of the N pixel groups through N groups of reading lines respectively; the control pixel data is transmitted by the data transmission unit.

In one possible implementation manner, the control data readout unit reads out pixel data generated by performing an exposure operation on N pixel groups through N groups of readout lines, respectively, and includes: the control data reading unit reads out first pixel data generated by the first pixel group through the first group of reading lines; the control data reading unit reads out second pixel data generated by the second pixel group through a second group of reading lines; wherein the first group of sense lines and the second group of sense lines are any two groups of sense lines in the N groups of sense lines; the first pixel group and the second pixel group are any two pixel groups of the N pixel groups.

In another possible implementation manner, before the transmission of the upper control pixel data through the data transmission unit, the method further includes: acquiring first pixel data, and converting the original bit width of the first pixel data into a first bit width; the first pixel data is pixel data generated by exposing a first pixel group in the N pixel groups; acquiring second pixel data, and converting the original bit width of the second pixel data into a second bit width; the second pixel data is pixel data generated by exposing a second pixel group in the N pixel groups; n pixel groups correspond to N groups of bit widths; the first bit width and the second bit width are any two groups of bit widths in the N bit widths; the control pixel data is transmitted by a data transmission unit, and includes: converting the original bit width of the first pixel data into a first bit width, and transmitting the first bit width through a data transmission unit; and converting the original bit width of the second pixel data into a second bit width, and transmitting the second pixel data through a data transmission unit.

In another possible implementation, the N groups of bit widths are independent of each other; the first bit width is determined by the exposure time length of the first pixel group and/or the bandwidth of the data transmission unit; the second bit width is determined by the exposure time of the second pixel group and/or the bandwidth of the data transfer unit.

In another possible implementation manner, the method further includes: the control data transmission unit transmits the pixel data acquired by the data reading unit to the data processing unit, so that the data processing unit restores the pixel data into a signal to be processed and performs image processing on the signal to be processed.

In another possible implementation manner, the signal to be processed is obtained by reorganizing the pixel data by the data processing unit according to one or more of an exposure time of the pixel data, a readout time of the pixel data, and a position information of an image channel or the pixel data where the pixel data is located.

In another possible implementation manner, the control component includes N groups of control lines; the N groups of control lines are used for transmitting exposure control signals to the N pixel groups; a group of control lines is connected with a pixel group; the method further comprises the steps of: transmitting a first exposure control signal to the first pixel group through the first group control line so that the first pixel group is exposed based on the first exposure control signal; transmitting a second exposure control signal to the second pixel group through the second group control line so that the second pixel group is exposed based on the second exposure control signal; the first set of control lines and the second set of control lines are any two sets of readout lines in the N sets of control lines; the first pixel group and the second pixel group are any two pixel groups of the N pixel groups.

In another possible implementation manner, the first exposure control signal is used to control a first exposure start time and a first exposure end time of the first pixel group; the second exposure control signal is used for controlling a second exposure start time and a second exposure end time of the second pixel group; and/or the first exposure control signal is used for controlling the times and the exposure time of the first pixel group for executing the exposure operation in the exposure period; the second exposure control signal is used for controlling the times and the exposure time of the second pixel group to execute the exposure operation in the exposure period.

In another possible implementation manner, the control component further includes N groups of signal lines; the N groups of signal lines are used for transmitting gain control signals to the N pixel groups; a group of signal lines is connected with one pixel group; the method further comprises the following steps: transmitting a first gain control signal to the first pixel group through the first group signal line so that the first pixel group adjusts a luminance value in the first pixel data based on the first gain control signal; the first pixel data is pixel data generated by exposing the first pixel group; transmitting a second gain control signal to the second pixel group through the second group signal line so that the second pixel group adjusts a luminance value in the second pixel data based on the second gain control signal; the second pixel data is pixel data generated by exposing the second pixel group; the first group of signal lines and the second group of signal lines are any two groups of signal lines in the N groups of signal lines; the first pixel group and the second pixel group are any two pixel groups of the N pixel groups.

In a third aspect, the present application provides a control assembly comprising: one or more processors; one or more memories; wherein the one or more memories are adapted to store computer program code comprising computer instructions which, when executed by the one or more processors, control the components to perform the data readout method provided in any of the possible implementations of the second aspect described above.

In a fourth aspect, the present application provides a computer-readable storage medium storing computer-executable instructions that, when run on a computer, cause the computer to perform the data readout method provided in any one of the possible implementations of the second aspect described above.

The description of the second aspect to the fourth aspect in the present application may refer to the detailed description of the first aspect; also, the advantageous effects described in the second aspect to the fourth aspect may refer to the advantageous effect analysis of the first aspect, and are not described herein.

Drawings

Fig. 1 is a schematic structural diagram of an image capturing device according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a photosensitive pixel array according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating an exposure operation according to an embodiment of the present disclosure;

FIG. 4 is a second schematic diagram of an exposure operation according to an embodiment of the present disclosure;

FIG. 5 is a third schematic diagram of an exposure operation according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of an exposure operation according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of an exposure mode according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram I of pixel data according to an embodiment of the present disclosure;

fig. 9 is a schematic diagram two of pixel data according to an embodiment of the present application;

FIG. 10 is a schematic diagram I of a reorganized pixel data according to an embodiment of the present application;

FIG. 11 is a second schematic diagram of a reorganized pixel data according to an embodiment of the present disclosure;

FIG. 12 is a schematic diagram I of a data bit width conversion according to an embodiment of the present application;

FIG. 13 is a second schematic diagram of data bit width conversion according to an embodiment of the present disclosure;

FIG. 14 is a flowchart showing a data reading method according to an embodiment of the present application;

fig. 15 is a second flowchart of a data readout method according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of a control component according to an embodiment of the present application.

Detailed Description

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone.

The terms "first" and "second" and the like in the description and in the drawings are used for distinguishing between different objects or for distinguishing between different processes of the same object and not for describing a particular sequential order of objects.

Furthermore, references to the terms "comprising" and "having" and any variations thereof in the description of the present application are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or apparatus.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In the description of the present application, unless otherwise indicated, the meaning of "a plurality" means two or more.

In the related art, the exposure mode of the image sensor includes long exposure, short exposure, and the like, if the pixels in the pixel array of the image sensor all adopt the form of long exposure, the obtained image has a higher signal-to-noise ratio, but the condition of blurring or overexposure may occur; if the pixels in the pixel array of the image sensor are all in the form of short exposure, the obtained image has no blurred signal, but the signal-to-noise ratio of the image is lower.

It can be understood that, for the same pixel, the perceived light signal amplitude will also change along with the change of the surrounding light intensity, so if all pixels adopt uniform and fixed exposure time, it is difficult to ensure that each pixel in the pixel matrix will not cause overexposure of the photosensitive element at each moment due to overlong exposure time; or, the exposure time is too short, so that the photosensitive element is underexposed, and the subsequent image processing effect is not facilitated. If the exposure time of each pixel is not uniform, it is difficult to avoid the transmission of invalid data bits of the pixels which are not exposed during the readout, which causes redundancy of the data bits and reduces the data transmission efficiency.

In view of the above technical problems, an embodiment of the present application provides an image acquisition apparatus, including: the device comprises a photosensitive pixel array unit, a data reading unit and a data transmission unit; the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit is used for collecting pixel data generated by the photosensitive pixel array unit; the data reading unit comprises N groups of reading lines, and the N groups of reading lines are in one-to-one correspondence with the N pixel groups; and the data transmission unit is used for transmitting the pixel data acquired by the data reading unit through a transmission interface of the data transmission unit. In this way, under the condition that the exposure modes of the pixels in the photosensitive pixel array are different, the corresponding pixel data can be read out through the corresponding readout lines of each pixel group, so that on one hand, the pixel data generated by each pixel group can be read out in time, the pixel data generated by the pixel group does not need to be cached, and the other pixel groups do not need to wait for exposure, and the data reading efficiency is improved; on the other hand, one group of readout lines corresponds to one pixel group (the exposure time of pixels in one pixel group is the same), redundant data can be prevented from being read, the data volume entering a data transmission channel in unit time is reduced, the bandwidth requirement of data transmission is reduced, and the efficiency of data transmission can be effectively improved.

In addition, the N pixel groups are respectively controlled by different exposure control signals, so that the pixels in the photosensitive pixel array unit can be exposed according to different exposure modes, and a high-quality image with higher signal-to-noise ratio and no blurring signal can be obtained through the mutual combination of the short exposure signals and the long exposure signals.

The embodiments provided in the present application are specifically described below with reference to the drawings attached to the specification.

Referring to fig. 1, a schematic structural diagram of an image capturing device according to an embodiment of the present application is shown. As shown in fig. 1, the image acquisition apparatus may include: a photosensitive pixel array unit 110, an exposure control unit 120, a data readout unit 130, and a data transmission unit 140.

In some embodiments, as shown in fig. 1, the photosensitive pixel array unit 110 is connected to the exposure control unit 120 through a control line, the photosensitive pixel array unit 110 is connected to the data readout unit 130 through a readout line, the data transmission unit 140 includes a transmission channel, and the data readout unit 130 is connected to the data transmission unit 140 through the transmission channel.

The photosensitive pixel array unit 110 is used for performing exposure according to the exposure control signal.

The pixel data includes pixel values of pixels in the photosensitive pixel array unit 110.

In some embodiments, the photosensitive pixel array unit 110 includes a plurality of pixels arranged in a pixel matrix of x×y; wherein X and Y are integers greater than 1.

In some embodiments, each pixel in the photosensitive pixel array unit 110 includes a photosensitive element for sensing an optical signal. The exposure of the photosensitive pixel array unit 110 means that the photosensitive components of the pixels in the photosensitive pixel array unit 110 respond to incident light for a corresponding exposure time period and generate an electrical signal according to a photoelectric effect. In general, the longer the exposure time of a pixel, the more electrical signals (e.g., charges) the pixel generates according to the photoelectric effect, and the larger the pixel value the pixel corresponds to.

In some embodiments, the photosensitive pixel array unit 110 includes N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals. Wherein N is an integer greater than or equal to 2.

It should be noted that, for ease of understanding, only the first pixel group and the second pixel group (the first pixel group and the second pixel group are any two pixel groups of the N pixel groups) of the N pixel groups are shown in fig. 1, but the photosensitive pixel array unit 110 is not represented to include only the first pixel group and the second pixel group. In other embodiments, the photosensitive pixel array unit 110 may further include a third pixel group and a fourth pixel group, which is not limited in the embodiments of the present application.

In some embodiments, the division of the pixel groups in the photosensitive pixel array unit 110 has a variety, and the pixels in each pixel group may be discretely distributed at different positions of the photosensitive pixel array unit 110. For example, assuming that the pixels in the photosensitive pixel array unit 110 may be divided into A, B, C, D four pixel groups, the positions of the pixels in the four pixel groups in the photosensitive pixel array unit 110 may be in the form as shown in fig. 2.

For example, the pixel groups may be divided according to the light incident amounts of the color channels corresponding to the pixels in the photosensitive pixel array unit 110 in different brightness environments. For example, according to the color distribution of the preset color image, a color filter array is disposed on the light incident surface of the photosensitive pixel array unit 110 during exposure, so that the ambient light can be filtered by the color filter array and then converted into monochromatic light of different color channels, and the monochromatic light is respectively projected to corresponding pixels in the photosensitive pixel array unit 110, so that different pixels in the photosensitive pixel array unit 110 can respectively sense the monochromatic light of different color channels, and the exposure data output by the photosensitive pixel array unit 110 can include color information. For example, in the above case, the pixels in the photosensitive pixel array unit 110 may be divided into a plurality of pixel groups according to, for example, the light quantity, and the exposure may be controlled independently of each other, and differential exposure may be achieved for different environmental brightnesses and different visual field ranges, for example, if the environmental brightness of the visual field range corresponding to one region of the photosensitive pixel array unit 110 is higher, the exposure duration of the pixel group included in the region may be shorter; conversely, if the ambient brightness of the field of view corresponding to a region of the photosensitive pixel array unit 110 is lower, the exposure time of the pixel group included in the region may be longer.

The exposure control unit 120 is configured to send an exposure control signal to the photosensitive pixel array unit 110.

In some embodiments, the exposure control unit 120 transmits an exposure control signal to the photosensitive pixel array unit 110 through a control line. Specifically, the pixels on each row in the horizontal direction of the photosensitive pixel array unit 110 share the same set of control lines, that is, the exposure control is performed row by row, so that the exposure control unit 120 sends an exposure control signal to the pixels on each row through the control lines of each row.

In some embodiments, the exposure control unit 120 includes N sets of control lines for transmitting exposure control signals to N pixel sets; a set of control lines is connected to a set of pixels. Specifically, taking an example that the N groups of control lines include a first group of control lines and a second group of control lines, the exposure control unit 120 is specifically configured to send a first exposure control signal to the first pixel group through the first group of control lines, so that the first pixel group performs exposure based on the first exposure control signal to obtain first pixel data; and sending a second exposure control signal to a second pixel group in the N pixel groups through a second group control line in the N groups of control lines, so that the second pixel group is exposed based on the second exposure control signal, and second pixel data are obtained.

Illustratively, as shown in fig. 1, two sets of control lines are disposed for each row in the horizontal direction of the photosensitive pixel array unit 110, wherein a first set of control lines is used to transmit a first exposure control signal to a first pixel group; the second set of control lines is used to send exposure control signals to the second image array. For example, the first row ofGroup control line R _{1_0} For feeding pixels R in a first row ₁ C ₂ And pixel R ₁ C ₄ Transmitting an exposure control signal, wherein the pixel R ₁ C ₂ And pixel R ₁ C ₄ Are all pixels in the first pixel group; second group of control lines R of first row _{1_1} For feeding pixels R in a first row ₁ C ₁ And pixel R ₁ C ₃ Transmitting an exposure control signal, wherein the pixel R ₁ C ₁ And pixel R ₁ C ₃ Are all pixels in the second pixel group. Wherein R is _x Represent rows, C _y Representation column, R _{x_0} A first group of control lines representing each row, R _{x_1} A second set of control lines representing each row.

It should be noted that, for ease of understanding, only the first and second sets of control lines of the N sets of control lines are shown in fig. 1, but it is not represented that the exposure control unit 120 includes only the first and second sets of control lines. In other embodiments, the exposure control unit 120 may further include a third set of control lines, a fourth set of control lines, and the like, which is not limited in the embodiments of the present application.

In some embodiments, the first exposure control signal is used to control a first exposure start time and a first exposure end time of the first pixel group; the second exposure control signal is used for controlling a second exposure start time and a second exposure end time of the second pixel group; wherein the first exposure start time is different from the second exposure start time; and/or the first exposure end time is different from the second exposure end time.

Wherein the first exposure start time and the first exposure end time determine the exposure time of the first pixel group; the second exposure start time and the second exposure end time determine the exposure time period of the second pixel group.

In some embodiments, the first exposure control signal is used to control the number of times the first pixel group performs the exposure operation and the exposure duration in the exposure period; the second exposure control signal is used for controlling the times and the exposure time of the second pixel group to execute the exposure operation in the exposure period.

As a possible implementation manner, the first exposure control signal is used for controlling the first pixel group to execute an exposure operation according to the first exposure duration in the exposure period; the second exposure control signal is used for controlling the second pixel group to execute one exposure operation according to the second exposure duration in the exposure period.

Wherein the first exposure time period and the second exposure time period are different.

For example, assuming that the first exposure period is S and the second exposure period is L, as shown in fig. 3, in one exposure period, the pixels in the first pixel group each perform one exposure operation with the first exposure period S, and the pixels in the second pixel group each perform one exposure operation with the second exposure period L.

As another possible implementation manner, the first exposure control signal is used for controlling the first pixel group to execute a plurality of exposure operations according to the first exposure duration in the exposure period; the second exposure control signal is used for controlling the second pixel group to execute a plurality of exposure operations according to a second exposure duration in the exposure period.

Wherein the first exposure time period and the second exposure time period are different.

For example, assuming that the first exposure period is S and the second exposure period is L, as shown in fig. 4, in one exposure period, the pixels in the first pixel group each perform a plurality of exposure operations using the first exposure period S, and the pixels in the second pixel group each perform a plurality of exposure operations using the second exposure period L.

As another possible implementation manner, the first exposure control signal is used for controlling the first pixel group to perform multiple exposure operations according to the first exposure duration combination in the exposure period; the second exposure control signal is used for controlling the second pixel group to execute a plurality of exposure operations according to the second exposure time duration combination in the exposure period.

The first exposure time length combination and the second exposure time length combination comprise a plurality of exposure time lengths, and the first exposure time length combination and the second exposure time length combination are different. It will be understood that, in the above exposure mode, a group of pixels in the photosensitive pixel array unit 110 are exposed according to the exposure duration combination in the time domain, which means that the pixels in the photosensitive pixel array unit 110 are alternately exposed by using a plurality of exposure durations in the exposure duration combination in one exposure period, that is, the exposure strategy of each exposure in the exposure period is periodically changed.

Illustratively, the first exposure time period combination is assumed to include: the exposure time period S, the exposure time period M and the exposure time period M, and the second exposure time period combination comprises: exposure time period L, and exposure time period S; then, as shown in fig. 5, the exposure strategy of the 1 st exposure in the exposure period is that the first pixel group adopts the exposure time length S, and the second pixel group adopts the exposure time length L; the exposure strategy of the 2 nd exposure in the exposure period is that the first pixel group adopts exposure time length M, and the second pixel group adopts exposure time length L; the exposure strategy of the 3 rd exposure in the exposure period is that the first pixel group adopts exposure time length M, and the second pixel group adopts exposure time length S; the exposure strategy for the 4 th exposure in the exposure period is the same as that for the 1 st exposure.

As yet another possible implementation manner, the first exposure control signal is used to control the first pixel group to perform multiple exposure operations in accordance with the first exposure duration combination in the exposure period; the second exposure control signal is used for controlling the second pixel group to execute a plurality of exposure operations according to a second exposure duration in the exposure period.

Wherein the first exposure time period combination includes a plurality of exposure time periods. It can be appreciated that the implementation manner is as follows: the first pixel group is alternately exposed in the exposure period according to the exposure time length in the first exposure time length combination of each frame of image; the second pixel group has the same exposure time length in each exposure in the exposure period.

Illustratively, the first exposure time period combination is assumed to include: the exposure time length S, the exposure time length M and the exposure time length M, wherein the second exposure time length is L; then, as shown in fig. 6, the exposure strategy of the 1 st exposure in the exposure period is that the first pixel group adopts the exposure time length S, and the second pixel group adopts the exposure time length L; the exposure strategy of the 2 nd exposure in the exposure period is that the first pixel group adopts exposure time length M, and the second pixel group adopts exposure time length L; the exposure strategy of the 3 rd exposure in the exposure period is that the first pixel group adopts exposure time length M, and the second pixel group adopts exposure time length L; the exposure strategy for the 4 th exposure in the exposure period is the same as that for the 1 st exposure.

It can be understood that, for the pixels in the photosensitive pixel array unit 110, the perceived light signal amplitude will also change along with the change of the surrounding light intensity, so if the pixels in the photosensitive pixel array unit 110 all adopt uniform and fixed exposure time, it is difficult to ensure that each pixel in the pixel matrix will not be overexposed by the photosensitive element due to the overlong exposure time at each moment; or, the exposure time is too short, so that the photosensitive element is underexposed, and the subsequent image processing effect is not facilitated. Therefore, according to the method provided by the embodiment of the present application, the pixels in the photosensitive pixel array unit 110 can be controlled to be exposed according to different exposure modes through the multiple groups of mutually independent control lines, and a high-quality image with a high signal-to-noise ratio and no blurring signal can be obtained through the mutual combination of the short exposure signal and the long exposure signal.

The data reading unit 130 is configured to read out pixel data generated by exposure performed by the photosensitive pixel array unit 110.

In some embodiments, the data readout unit 130 reads out pixel data generated by the photosensitive pixel array unit 110 by performing exposure through a readout line. Specifically, the pixels on each column of the photosensitive pixel array unit 110 share the same set of readout lines, that is, the data readout is performed by columns, so that the data readout unit 130 reads out the pixel data generated by the pixels on each column through the readout lines of each column.

In some embodiments, the data readout unit 130 includes N sets of readout lines for reading out pixel data generated by the N pixel groups; a set of readout lines is connected to a set of pixels. Specifically, taking an example that the N sets of readout lines include a first set of readout lines and a second set of readout lines (the first set of readout lines and the second set of readout lines are any two sets of readout lines in the N sets of readout lines), the data readout unit 130 is specifically configured to read out first pixel data generated by exposing the first pixel group through the first set of readout lines; and reading out second pixel data generated by exposing the second pixel group through the second group of reading lines.

Illustratively, as shown in fig. 1, two sets of readout lines are disposed for each column in the vertical direction of the photosensitive pixel array unit 110, wherein a first set of readout lines is used for reading out first pixel data generated by a first pixel group; the second group of readout lines are used for reading out second pixel data generated by the second image array. For example, a first set of sense lines C of a first column _{1_0} For reading out the pixels R in the first column ₂ C ₁ And pixel R ₄ C ₁ The generated pixel data; wherein, the pixel R ₂ C ₁ And pixel R ₄ C ₁ Pixels which are all the first pixel group; second group of sense lines C of first column _{1_1} For reading out the pixels R in the first column ₁ C ₁ And pixel R ₃ C ₁ The generated pixel data; wherein, the pixel R ₁ C ₁ And pixel R ₃ C ₁ Are all pixels of the second pixel group. Wherein R is _x Represent rows, C _y Representation column, C _{y_0} A first group of sense lines representing each column, C _{y_1} Representing a second set of sense lines for each column.

It will be appreciated that, due to the different exposure durations corresponding to the different exposure modes, it may happen that one group of pixels has completed exposure, while another group of pixels has not completed exposure, which may cause confusion in the readout timing, and that the readout data is empty or redundant. Therefore, the data readout unit 130 provided in the embodiment of the present application is configured with multiple groups of independent readout lines, which are respectively used for reading pixel data generated by pixels in different exposure modes (i.e. different exposure durations), so that on one hand, pixel data generated by each pixel group can be timely read out, no buffering is needed for the pixel data generated by the pixel group, and no waiting is needed for other pixel groups to complete exposure, thereby improving the efficiency of data readout; on the other hand, one group of readout lines corresponds to one pixel group (the exposure time of pixels in one pixel group is the same), redundant data can be prevented from being read, the data volume entering a data transmission channel in unit time is reduced, the bandwidth requirement of data transmission is reduced, and the efficiency of data transmission can be effectively improved.

For example, assuming that the exposure time corresponding to the first exposure mode is 3s, the exposure time corresponding to the second exposure mode is 5s, and the exposure time of the two groups is different, the first pixel group can directly read out the exposure data through the first readout line after the exposure is completed, and then the exposure data is transmitted through the first transmission channel, so that the pixel data can be read out in time without waiting for the completion of the exposure of the second pixel group, and the reading and transmission efficiency of the pixel data can be effectively improved; furthermore, after the exposure of the second pixel group is completed, the second pixel data can be directly read out through the second read-out line and then transmitted through the second transmission channel, and no empty space exists in the transmitted pixel data, so that the bandwidth requirements of data reading and data transmission are reduced.

It should be noted that, for ease of understanding, only the first and second sets of the N sets of sense lines are shown in fig. 1, but it is not represented that the data sensing unit 130 includes only the first and second sets of sense lines. In other embodiments, the data sensing unit 130 may further include a third set of sensing lines and a fourth set of sensing lines, etc., which are not limited in this embodiment.

And a data transmission unit 140 for acquiring the pixel data read by the data reading unit 130 and transmitting the pixel data through a transmission interface of the data transmission unit 140.

In some embodiments, the data transmission unit 140 receives and transmits the pixel data read out by the data readout unit 130 through a transmission channel, and then transmits the pixel data through a transmission interface. Alternatively, the transmission channel may be one or more transmission channels.

It should be noted that, in the embodiment of the present application, the correspondence between the N groups of readout lines of the data readout unit 130 and one or more transmission channels of the data transmission unit 140 is not limited. For ease of understanding, the first set of readout lines is shown in fig. 1 as being connected to the first transmission channel, and the second set of readout lines is shown as being connected to the second transmission channel, but it is not meant that the correspondence between the N sets of readout lines of the data readout unit 130 and one or more transmission channels of the data transmission unit 140 may only be a one-to-one correspondence.

Optionally, the correspondence between the N groups of readout lines of the data readout unit 130 and one or more transmission channels of the data transmission unit 140 includes: a group of readout lines corresponds to a plurality of transmission channels; or, a group of readout lines corresponds to a transmission channel; alternatively, the N sets of readout lines correspond to one transmission channel.

In some embodiments, the transmission channel may be a physical transmission channel or a virtual transmission channel.

For example, assume that the first exposure control signal is to control the first pixel group to perform an exposure operation with a first exposure period; the second exposure control signal is used for controlling the second pixel group to execute a plurality of exposure operations in an exposure period by adopting a second exposure time length, wherein, as shown in fig. 7, the first exposure time length is L (Long), the second exposure time length is S (Short), and L > S; then, as shown in (a) of fig. 8, the pixel data acquired by the data transmission unit 140 at time t1 is the second pixel data generated by the second pixel group (because the first pixel group has not completed exposure at time t 1); as shown in (b) of fig. 8, the pixel data acquired by the data transmission unit 140 at time t2 is the first pixel data and the second pixel data.

As shown in fig. 9, the first pixel data generated by the first pixel group read out by the first group read out line at time t1 is shown in (a) of fig. 9; at time t2, the first pixel data generated by the first pixel group read out by the first group read out line and the second pixel data generated by the second pixel group read out by the second group read out line are as shown in (b) of fig. 9.

As can be seen from the above examples, since the image acquisition device provided in the embodiment of the present application adopts multiple groups of independent readout lines, even if the exposure modes of the pixels in the photosensitive pixel array 110 are different, the pixel data generated by each pixel group can be timely read out through the corresponding readout line of the pixel group, and then transmitted to the data transmission unit 140 through the transmission channel, so that the transmission interface of the data transmission unit 140 can timely transmit the pixel data. Therefore, pixel data generated by the pixel group does not need to be cached, other pixel groups do not need to be waited for completing exposure, redundant data generated additionally does not need to be removed, the data quantity entering a data transmission channel in unit time is reduced, the bandwidth requirement of data transmission is reduced, and the efficiency of data transmission can be effectively improved.

In some embodiments, the transmission interface is configured to package the received pixel data for transmission, for example, the transmission interface may package the pixel data into a form of byte data or a form of data block for transmission. Illustratively, the transmission interface may be a mobile industry processor interface (mobile industry processor interface, MIPI), which is a standardized interface protocol for defining internal interface standards for electronic devices. For example, the MIPI data transmission unit receives the pixel data and the clock synchronization signal of the pixel data amount, and then divides the 8-bit pixel data specified according to the MIPI protocol into output channels for output.

Optionally, the data transmission unit 140 may support both: multiple data types such as RAW6, RAW7, RAW8, RAW10, RAW12, RAW14, YUV422 8bit, YUV422 10bit and the like are input.

In some embodiments, the image acquisition apparatus further comprises: a data processing unit (not shown in fig. 1) for acquiring the pixel data acquired by the data readout unit 130 through the transmission interface of the data transmission unit 140, recovering the pixel data into a signal to be processed, and performing image processing on the signal to be processed.

The form of the signal to be processed is not limited in the embodiment of the present application, for example, the signal to be processed may be an image signal or a data stream signal. The image signals may be image signals ordered according to exposure time length; alternatively, the image signal may be an image of different channels, including but not limited to the following: luminance channels or chrominance channels.

In some embodiments, the data processing unit is specifically configured to reorganize the pixel data according to one or more of an exposure duration of the pixel data, a readout time of the pixel data, and position information of an image channel in which the pixel data is located or the pixel data, so as to obtain a signal to be processed, and perform image processing on the signal to be processed. Wherein, the image channel includes: luminance channel, color channel, etc.

For example, the data processing unit may construct image sequences corresponding to times t0, t1, and t2, respectively, according to the readout time sequence of the pixel data; or the data processing unit can respectively construct image sequences of the first pixel group, the second pixel group and the third pixel group according to pixel data of different pixel groups corresponding to different exposure control signals; alternatively, the data processing unit may construct the image sequences of the luminance channel and the color channel, respectively, according to the image channel in which the pixel data is located. The color channels in different image formats are different, for example, in a Bayer (Red) format, and may be R (Red) channel, G1 (Green) channel, G2 (Green) channel, B (Blue) channel, and the like, respectively.

Illustratively, as shown in fig. 10, the data processing unit acquires the first pixel data and the second pixel data through the output interface of the data transmission unit 140; furthermore, the data processing unit may reorganize the first pixel data and the second pixel data according to the exposure time and the position information of the first pixel data and the second pixel data, to obtain a signal to be processed, and perform image processing on the signal to be processed.

Illustratively, as shown in fig. 11, the data processing unit acquires the first pixel data and the second pixel data through the output interface of the data transmission unit 140; furthermore, the data processing unit may reorganize the first pixel data and the second pixel data according to the position information of the first pixel data and the second pixel data in different channels, to obtain a signal to be processed, and perform image processing on the signal to be processed.

In some embodiments, the data processing unit is specifically configured to perform image processing on the signal to be processed by using a neural network.

Among them, image processing includes, but is not limited to: image signal processing (Image Signal Processor, ISP) operation, intelligent processing, dyeing processing, enhancement processing, or the like.

Wherein the ISP operations include: performing operations such as dead pixel correction, color interpolation, gamma correction, color correction, converting RGB into YUV, noise reduction, sharpening and the like; the intelligent processing comprises the following steps: target recognition, target segmentation, target detection, etc.; the dyeing treatment comprises the following steps: fluorescent staining, spectral staining, electronic staining, etc.; the enhancement process includes: the sharpness is enhanced and the brightness is enhanced.

In some embodiments, the image acquisition apparatus further comprises: a bit width conversion unit (not shown in fig. 1) between the data readout unit 130 and the data transmission unit 140 for performing bit width conversion of the pixel data read out by the data readout unit 130, for example, compressing the bit width of the pixel data read out by the data readout unit 130.

In some embodiments, the bit width conversion unit is specifically configured to obtain the first pixel data, convert the original bit width of the first pixel data into the first bit width, and then transmit the first bit width through the data transmission unit 140.

The first pixel data is pixel data generated by exposing a first pixel group in the N pixel groups.

In some embodiments, the N pixel groups correspond to N groups of bit widths, the N groups of bit widths being independent of each other. The first bit width is any one of the N bit widths. The first bit width is determined by the exposure time of the first pixel group and/or the bandwidth of the data transfer unit 140.

For example, the first bit width is positively correlated with the exposure time of the first pixel group, for example, if the exposure time of the first pixel group is shorter, the first bit width corresponding to the first pixel data is smaller; the first bit width is positively correlated with the bandwidth of the data transmission unit 140, and if the bandwidth of the data transmission unit 140 is smaller, the first bit width corresponding to the first pixel data is smaller.

In some embodiments, the bit width conversion unit is specifically configured to obtain the second pixel data, and after converting the original bit width of the second pixel data into the second bit width, the second pixel data is transmitted through the data transmission unit 140.

The second pixel data is pixel data generated by exposing a second pixel group in the N pixel groups.

In some embodiments, the second bit width is any one of the group of N bit widths. The second bit width is determined by the exposure time of the second pixel group and/or the bandwidth of the data transfer unit 140.

For example, the second bit width is positively correlated with the exposure time of the second pixel group, for example, if the exposure time of the second pixel group is shorter, the second bit width corresponding to the second pixel data is smaller; the second bit width is positively correlated with the bandwidth of the data transmission unit 140, and if the bandwidth of the data transmission unit 140 is smaller, the second bit width corresponding to the second pixel data is smaller.

For example, it is assumed that the original bit width of the first pixel data and the original bit width of the second pixel data obtained by the bit width conversion unit are both 12 bits, wherein the exposure time period of the first pixel data is longer than the exposure time period of the second pixel data. Then, as shown in fig. 12, the data bit width conversion unit is used to adjust the bit width of the first pixel data and the bit width of the second pixel data, respectively, that is, the pixels in one frame of image may have different bit widths. For example, the bit width of the second pixel data may be converted to 10 bits, the bit width of the first pixel data still remaining 12 bits.

In some embodiments, the bit width conversion unit is further configured to compress the bit width of the data points that do not require the high bit width to reduce the data transmission pressure.

As an example, as shown in fig. 13, it is assumed that the original bit width of the first pixel data and the original bit width of the second pixel data acquired by the bit width conversion unit are both 12 bits; after the bit width conversion unit performs bit width conversion, the first bit width corresponding to the first pixel data is 12bits, and the second bit width corresponding to the second pixel data is 10bits, so that the bit width conversion unit can perform data compression on the data point under the condition that part of pixels in the first pixel data and the second pixel data do not need high bit width. For example, as shown in fig. 13, assuming that the pixel data in the second column and the fourth column do not need to have an upper bit width, the bit width conversion unit may compress the bit widths of the pixel data in the second column and the fourth column to 8 bits.

It can be understood that, compared with the method of directly adopting unified data bit width (for example, unified as 8bits, 10bits or 12bits, etc.) to perform data transmission in the related art, the bit width conversion unit provided in the embodiment of the present application can adopt multiple data bit widths for pixels in different exposure modes and different light-sensitive pixels, and convert pixel data obtained according to different exposure modes into different data bit widths respectively, and then perform transmission, so that data transmission pressure can be effectively reduced.

In some embodiments, the image acquisition apparatus further comprises: a gain control unit (not shown in fig. 1) for transmitting a gain control signal to the pixels in the photosensitive pixel array unit 110 so that the pixels in the photosensitive pixel array unit 110 adjust the luminance value in the pixel data based on the gain control signal.

In some embodiments, the gain control unit transmits a gain control signal to the pixels in the photosensitive pixel array unit 110 through signal lines.

The arrangement form of the signal line is not limited in the embodiments of the present application, and for example, the signal line may depend on the design type of a complementary metal Oxide Semiconductor (cmos) sensor. In one possible implementation, the signal lines are connected to column amplifiers (e.g., analog programmable amplifiers) of the photosensitive pixel array unit 110, one for each column. Analog amplifiers of the same gain are connected using the same signal line.

Optionally, the gain control unit includes N groups of signal lines; the N groups of signal lines are used for sending gain control signals to the N pixel groups; a group of signal lines is connected to an array of pixels.

Taking an example that the N groups of signal lines include a first group of signal lines and a second group of signal lines (the first group of signal lines and the second group of signal lines are any two groups of signal lines in the N groups of signal lines), the gain control unit is specifically configured to send a first gain control signal to a first pixel group in the N pixel groups through the first group of signal lines, so that the first pixel group adjusts a luminance value in the first pixel data based on the first gain control signal; and transmitting a second gain control signal to a second pixel group of the N pixel groups through the second group signal line, so that the second pixel group adjusts the brightness value in the second pixel data based on the second gain control signal.

The second pixel data is pixel data generated by exposing the second pixel group; the first pixel data is pixel data generated by exposing the first pixel group.

In some embodiments, the gain value corresponding to the first gain control signal is determined by the light sensing amount of the first pixel group; the gain value corresponding to the second gain control signal is determined by the light sensing amount of the second pixel group.

It can be understood that in the actual use process, the difference of the sensitization amount of each channel may be larger due to the spectral response characteristics of different channels; in addition, the exposure time may be different between different channels, and if the sensor can only configure the gain suitable for the sensitization amount of one channel, then the images of other channels are underexposed or overexposed, which is unfavorable for the subsequent image processing effect. Therefore, the embodiment of the application configures a gain control unit for the image acquisition device, and outputs different gain control signals to different pixel groups so as to enable pixels with different channels and different light sensing characteristics to achieve proper exposure.

A data readout method provided in the embodiment of the present application is described in detail below.

The data readout method provided in the embodiment of the present application is applied to an image acquisition device (for example, the image acquisition device shown in fig. 1), where the image acquisition device includes: the device comprises a photosensitive pixel array unit, a data reading unit, a data transmission unit, a data processing unit and a control component, wherein the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit comprises N groups of reading lines, wherein the N groups of reading lines are used for reading out the pixel numbers generated by the N pixel groups; a set of readout lines is connected to a set of pixels.

Alternatively, the method provided in the embodiments of the present application may be performed by the control component described above. Illustratively, the control component may be a server; alternatively, the control component may be a central processing unit (central processing unit, CPU), an image processing unit (Graphics Processing Unit, GPU), a general purpose processor network processor (network processor, NP), a digital signal processor (digital signal processing, DSP), a microprocessor, a microcontroller, a programmable logic device (programmable logic device, PLD), or any combination thereof. The control component may also be other means having processing functions, such as a circuit, device or software module, which is not limited in any way by this application.

Still further exemplary, the control component may include one or more of an exposure control unit, a bit width conversion unit, or a gain control unit as shown in fig. 1.

As shown in fig. 14, the data readout method provided in the embodiment of the present application includes the following steps:

s301, a control data reading unit reads out pixel data generated by exposure operation of N pixel groups through N groups of reading lines.

The pixel data is the pixel value of the pixel in the photosensitive pixel array unit.

It should be noted that, for convenience of description, the following N groups of sense lines include: the first set of sense lines and the second set of sense lines (the first set of sense lines and the second set of sense lines being any two sets of sense lines of the N sets of sense lines), the N pixel sets comprising: the first pixel group and the second pixel group (the first pixel group and the second pixel group are any two pixel groups of the N pixel groups) are described as an example, but the N groups of readout lines do not represent that the N groups of readout lines include only the first group of readout lines and the second group of readout lines; nor does it represent that the N pixel groups include only the first pixel group and the second pixel group. In other embodiments, the N sets of sense lines may also include a third set of sense lines or a fourth set of sense lines, etc.; the N pixel groups may further include a third pixel group or a fourth pixel group, etc., which is not limited in the embodiment of the present application.

For example, the above step S301 may be implemented as: the control data reading unit reads out first pixel data generated by exposing the first pixel group through a first group of reading lines; the control data reading unit reads out second pixel data generated by exposing the second pixel group through the second group of reading lines.

For example, as shown in FIG. 1, a first set of sense lines C of a first column _{1_0} For reading out the pixels R in the first column ₂ C ₁ And pixel R ₄ C ₁ The generated pixel data; wherein, the pixel R ₂ C ₁ And pixel R ₄ C ₁ Pixels which are all the first pixel group; second group of sense lines C of first column _{1_1} For reading out the pixels R in the first column ₁ C ₁ And pixel R ₃ C ₁ The generated pixel data; wherein, the pixel R ₁ C ₁ And pixel R ₃ C ₁ Are all pixels of the second pixel group. Wherein R is _x Represent rows, C _y Representation column, C _{y_0} A first group of sense lines representing each column, C _{y_1} Representing a second set of sense lines for each column.

It will be appreciated that, due to the different exposure durations corresponding to the different exposure modes, it may happen that one group of pixels has completed exposure, while another group of pixels has not completed exposure, which may cause confusion in the readout timing, and that the readout data is empty or redundant. Therefore, the data readout unit 130 provided in the embodiment of the present application is configured with multiple groups of independent readout lines, which are respectively used for reading out pixel data generated by pixels in different exposure modes (i.e. different exposure durations), so that readout timing conflicts can be effectively avoided.

S302, controlling pixel data to be transmitted through a data transmission unit.

In some embodiments, before performing step S302, the method further includes: the original bit width of the first pixel data is converted into a first bit width, and the original bit width of the second pixel data is converted into a second bit width.

The first pixel data is pixel data generated by exposing a first pixel group in the N pixel groups; the second pixel data is pixel data generated by exposing a second pixel group of the N pixel groups.

Optionally, the N pixel groups correspond to N groups of bit widths; the first bit width and the second bit width are any two groups of bit widths of the N bit widths.

The first bit width is determined by the exposure time of the first pixel group and/or the bandwidth of the data transmission unit. Illustratively, the first bit width is positively correlated with the exposure time of the first pixel data, e.g., if the exposure time of the first pixel data is shorter, the first bit width corresponding to the first pixel data is smaller; the first bit width is positively correlated with the bandwidth of the first transmission channel, and if the bandwidth of the first transmission channel is smaller, the first bit width corresponding to the first pixel data is smaller.

The second bit width is determined by the exposure time of the second pixel group and/or the bandwidth of the data transmission unit; illustratively, the second bit width is positively correlated with the exposure time of the second pixel data, e.g., if the exposure time of the second pixel data is shorter, the second bit width corresponding to the second pixel data is smaller; the second bit width is positively correlated with the bandwidth of the second transmission channel, and if the bandwidth of the second transmission channel is smaller, the second bit width corresponding to the second pixel data is smaller.

Thus, the step S302 may be implemented as follows: converting the original bit width of the first pixel data into a first bit width, and transmitting the first bit width through a data transmission unit; and converting the original bit width of the second pixel data into a second bit width, and transmitting the second pixel data through a data transmission unit.

It can be understood that, compared with the method of directly adopting a unified data bit width (for example, unified to 8bits, 10bits or 12bits, etc.) to perform data transmission in the related art, the method provided by the embodiment of the application can adopt a plurality of data bit widths for pixels in different exposure modes and different light-sensitive pixels, respectively convert pixel data obtained according to different exposure modes into different data bit widths, and then perform transmission, so that the data transmission pressure can be effectively reduced.

In some embodiments, the data transmission unit comprises: one or more transmission channels, and a transmission interface, the step S302 may be implemented as: pixel data is received via one or more transmission channels and transmitted to the transmission interface, which then transmits the pixel data.

The transmission channel may be a physical transmission channel or a virtual transmission channel.

The transmission interface is used for packaging and transmitting the received pixel data, for example, the transmission interface can package the pixel data into a byte data form or a data block form and then transmit the pixel data.

In some embodiments, the above method further comprises: and transmitting the pixel data to the data processing unit through the data transmission unit so that the data processing unit restores the pixel data into a signal to be processed and performs image processing on the signal to be processed.

The signal to be processed is obtained by recombining the pixel data by the data processing unit according to one or more of exposure time of the pixel data, readout time of the pixel data, and position information of an image channel or the pixel data where the pixel data is located.

Illustratively, the image processing of the signal to be processed includes: and adopting a neural network to perform image processing on the signal to be processed.

It can be understood that, based on the method provided by the embodiment of the application, under the condition that the exposure modes of the pixels in the photosensitive pixel array are different, the corresponding pixel data can be read out through the corresponding readout lines of each pixel group, so that on one hand, the pixel data generated by each pixel group can be read out in time, the pixel data generated by the pixel group does not need to be cached, and the exposure of other pixel groups does not need to be waited, thereby improving the data reading efficiency; on the other hand, one group of readout lines corresponds to one pixel group (the exposure time of pixels in one pixel group is the same), redundant data can be prevented from being read out, so that the data volume entering a data transmission channel in unit time is reduced, the bandwidth requirement of data transmission is reduced, and the efficiency of data transmission can be effectively improved.

In some embodiments, the control assembly includes N sets of control lines for sending exposure control signals to the N sets of pixels; a set of control lines is connected to a set of pixels. Illustratively, a first one of the N sets of control lines is connected to a first one of the N pixel sets, and a second one of the N sets of control lines is connected to a second one of the N pixel sets.

Thus, before step S301, as shown in fig. 15, the above method further includes: steps S201 to S202 are as follows.

S201, a first exposure control signal is sent to the first pixel group through a first control group control line, so that the first pixel group is exposed based on the first exposure control signal, and first pixel data are obtained.

In some embodiments, the first exposure control signal is used to control a first exposure start time and a first exposure end time of the first pixel group.

In some embodiments, the first exposure control signal is used to control the number of times the first pixel group performs the exposure operation and the exposure time period within the exposure period.

The first exposure control signal is used for controlling the mode that the pixels in the first pixel group are exposed by adopting the first exposure time period or the first exposure time period combination. Under the condition that the pixels in the first pixel group adopt the first exposure time length, the exposure strategies of the pixels in the first pixel group in the exposure period are the same, and each exposure adopts the first exposure time length. The first exposure time period combination may include a plurality of exposure time periods, and in the case that the pixels in the first pixel group adopt the first exposure time period combination, it is explained that the exposure policy of the pixels in the first pixel group in the exposure period is periodically changed, that is, the pixels in the first pixel group may be sequentially exposed by adopting the exposure time periods in the first exposure time period combination. For example, if the first exposure duration combination includes: the exposure time length S, the exposure time length L and the exposure time length M, when the pixels in the first pixel group are exposed, the exposure time length of the first exposure is S, the exposure time length of the second exposure is L, the exposure time length of the third exposure is M, then the exposure time length of the fourth exposure is S, and the exposure time lengths of the second exposure and the third exposure are adopted in a circulating mode in sequence in the fifth and sixth times.

Exemplary, as shown in FIG. 1, a first set of control lines R for a first row _{1_0} For feeding pixels R in a first row ₁ C ₂ And pixel R ₁ C ₄ Transmitting an exposure control signal, wherein the pixel R ₁ C ₂ And pixel R ₁ C ₄ Are all pixels in the first pixel group. Wherein R is _x Represent rows, C _y Representation column, R _{x_0} A first set of control lines representing each row.

In some embodiments, the control assembly further comprises N sets of signal lines; the N groups of signal lines are used for sending gain control signals to the N pixel groups, and one group of signal lines are connected with one pixel group. Based on this, the above method further comprises: and transmitting a first gain control signal to a first pixel group of the N pixel groups through a first group of signal lines of the N groups of signal lines, so that the first pixel group adjusts the brightness value in the first pixel data based on the first gain control signal.

The first pixel data is pixel data generated by exposing the first pixel group.

S202, a second exposure control signal is sent to the second pixel group through the second group control line, so that the second pixel group is exposed based on the second exposure control signal, and second pixel data are obtained.

In some embodiments, the second exposure control signal is used to control a second exposure start time and a second exposure end time for the second pixel group; wherein the first exposure start time is different from the second exposure start time; and/or the first exposure end time is different from the second exposure end time.

In some embodiments, the second exposure control signal is used to control the number of times the second pixel group performs the exposure operation and the exposure time period within the exposure period.

The second exposure control signal is used for controlling the mode that the pixels in the second pixel group are exposed by adopting the second exposure time period or the second exposure time period combination. And under the condition that the pixels in the second pixel group adopt the second exposure time length, the exposure strategies of the pixels in the second pixel group in the exposure period are the same, and each exposure adopts the second exposure time length. The second exposure time period combination may include a plurality of exposure time periods, and in the case that the pixels in the second pixel group adopt the two exposure time period combination, it is explained that the exposure policy of the pixels in the second pixel group in the exposure period is periodically changed, that is, the pixels in the second pixel group may be sequentially exposed by adopting the exposure time periods in the second exposure time period combination. For example, if the second exposure duration combination includes: the exposure time length L, the exposure time length L and the exposure time length M, when the pixels in the second pixel group are exposed, the exposure time length of the first exposure is L, the exposure time length of the second exposure is L, the exposure time length of the third exposure is M, then the exposure time length of the fourth exposure is L, and the exposure time lengths of the fifth exposure and the sixth exposure are sequentially and circularly adopted.

Exemplary, as shown in FIG. 1, the second set of control lines R of the first row _{1_1} For feeding pixels R in a first row ₁ C ₁ And pixel R ₁ C ₃ Transmitting an exposure control signal, wherein the pixel R ₁ C ₁ And pixel R ₁ C ₃ Are all pixels in the second pixel group. R is R _{x_1} A second group of control lines representing each row

In some embodiments, the above method further comprises: and transmitting a second gain control signal to a second pixel group of the N pixel groups through a second group of signal lines of the N groups of signal lines, so that the second pixel group adjusts the brightness value in the second pixel data based on the second gain control signal.

The second pixel data is pixel data generated by exposing the second pixel group.

It can be understood that in the actual use process, the difference of the sensitization amount of each channel may be larger due to the spectral response characteristics of different channels; in addition, the exposure time may be different between different channels, and if the sensor can only configure the gain suitable for the sensitization amount of one channel, then the images of other channels are underexposed or overexposed, which is unfavorable for the subsequent image processing effect. Therefore, the embodiment of the application is to output different gain control signals to different pixel groups so as to enable pixels with different channels and different light sensing characteristics to achieve proper exposure.

It can be understood that, for the pixels in the pixel array, the perceived light signal amplitude will also change along with the change of the surrounding light intensity, so if the pixels in the pixel array all adopt uniform and fixed exposure time, it is difficult to ensure that each pixel in the pixel matrix will not cause overexposure of the photosensitive element due to the overlong exposure time at each moment; or, the exposure time is too short, so that the photosensitive element is underexposed, and the subsequent image processing effect is not facilitated. Therefore, according to the method provided by the embodiment of the application, the pixels in the pixel array can be respectively controlled to be exposed according to different exposure modes through a plurality of groups of mutually independent control lines, and a high-quality image with high signal-to-noise ratio and no fuzzy signal can be obtained through the mutual combination of the short exposure signal and the long exposure signal.

The embodiment of the application provides a schematic structural diagram of the control assembly in the embodiment. As shown in fig. 16, the control assembly 400 includes: a processor 402, a communication interface 403, a bus 404. Optionally, the control assembly 400 may also include a memory 401.

The processor 402 may be any logic block, module, and circuitry that implements or performs the various examples described in connection with the present disclosure. The processor 402 may be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 402 may also be a combination that implements computing functionality, such as a combination comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

A communication interface 403 for connecting with other devices via a communication network. The communication network may be an ethernet, a radio access network, a wireless local area network (wireless local area networks, WLAN), etc.

The memory 401 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

As a possible implementation, the memory 401 may exist separately from the processor 402, and the memory 401 may be connected to the processor 402 by a bus 404, for storing instructions or program codes. The processor 402, when calling and executing instructions or program codes stored in the memory 401, can implement the data readout method provided in the embodiment of the present application.

In another possible implementation, the memory 401 may also be integrated with the processor 402.

Bus 404, which may be an extended industry standard architecture (extended industry standard architecture, EISA) bus, or the like. The bus 404 may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 16, but not only one bus or one type of bus.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the control module is divided into different functional modules to implement all or part of the functions described above.

Embodiments of the present application also provide a computer-readable storage medium. All or part of the flow in the above method embodiments may be implemented by computer instructions to instruct related hardware, and the program may be stored in the above computer readable storage medium, and the program may include the flow in the above method embodiments when executed. The computer readable storage medium may be any of the foregoing embodiments or memory. The computer readable storage medium may also be an external storage device of the control component, for example, a plug-in hard disk provided on the control component, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card, or the like. Further, the computer readable storage medium may also include both an internal storage unit and an external storage device of the control component. The computer readable storage medium is used for storing the computer program and other programs and data required by the control component. The above-described computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

The present application also provides a computer program product comprising a computer program which, when run on a computer, causes the computer to perform any of the generated data readout methods provided in the above embodiments.

Although the present application has been described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the figures, the disclosure, and the appended claims. In the claims, the word "Comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Although the present application has been described in connection with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely exemplary illustrations of the present application as defined in the appended claims and are considered to cover any and all modifications, variations, combinations, or equivalents that fall within the scope of the present application. It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (22)

1. An image acquisition apparatus, comprising: the device comprises a photosensitive pixel array unit, a data reading unit and a data transmission unit;

the photosensitive pixel array unit is used for exposing according to an exposure control signal; the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2;

the data reading unit is used for reading out pixel data generated by exposure of the photosensitive pixel array unit; the data reading unit comprises N groups of reading lines, wherein the N groups of reading lines are used for reading pixel data generated by the N pixel groups; a group of readout lines connected to a pixel group;

the data transmission unit is used for transmitting the pixel data acquired by the data reading unit through a transmission interface of the data transmission unit.

2. The device according to claim 1, wherein the data readout unit is in particular configured to read out first pixel data generated by the first pixel group via the first set of readout lines; reading out second pixel data generated by the second pixel group through the second group read-out line; wherein the first set of sense lines and the second set of sense lines are any two sets of sense lines of the N sets of sense lines; the first pixel group and the second pixel group are any two pixel groups in the N pixel groups.

3. The apparatus of claim 1, wherein the image acquisition apparatus further comprises: a bit width conversion unit;

the bit width conversion unit is used for obtaining first pixel data, converting the original bit width of the first pixel data into the first bit width and transmitting the first bit width through the data transmission unit; the first pixel data is pixel data generated by exposing a first pixel group in the N pixel groups;

acquiring second pixel data, converting the original bit width of the second pixel data into a second bit width, and transmitting the second bit width through the data transmission unit; the second pixel data is pixel data generated by exposing a second pixel group in the N pixel groups; the N pixel groups correspond to N groups of bit widths; the first bit width and the second bit width are any two groups of bit widths of the N bit widths.

4. The apparatus of claim 3, wherein the N groups of bit widths are independent of each other; wherein the first bit width is determined by the exposure time length of the first pixel group and/or the bandwidth of the data transmission unit; the second bit width is determined by the exposure time of the second pixel group and/or the bandwidth of the data transmission unit.

5. The apparatus of claim 1, wherein the data transmission unit comprises one or more transmission channels; the data transmission unit is specifically configured to transmit, through the one or more transmission channels, pixel data collected by the data readout unit to the transmission interface, and then transmit the pixel data by the transmission interface.

6. The apparatus of claim 1, wherein the image acquisition apparatus further comprises: a data processing unit;

the data processing unit is used for acquiring the pixel data acquired by the data reading unit through the transmission interface, recovering the pixel data into a signal to be processed, and performing image processing on the signal to be processed.

7. The apparatus according to claim 6, wherein the data processing unit is specifically configured to reorganize the pixel data according to one or more of an exposure time of the pixel data, a readout time of the pixel data, an image channel in which the pixel data is located, or position information of the pixel data, to obtain a signal to be processed, and perform image processing on the signal to be processed.

8. The apparatus according to claim 6, wherein the data processing unit is configured to perform image processing on the signal to be processed using a neural network.

9. The apparatus of claim 1, wherein the image acquisition apparatus further comprises: an exposure control unit; the exposure control unit comprises N groups of control lines, wherein the N groups of control lines are used for transmitting exposure control signals to the N pixel groups; a group of control lines is connected with a pixel group;

the exposure control unit is used for sending a first exposure control signal to a first pixel group through a first group of control lines so that the first pixel group is exposed based on the first exposure control signal; transmitting a second exposure control signal to a second pixel group through a second group control line so that the second pixel group is exposed based on the second exposure control signal; the first set of control lines and the second set of control lines are any two sets of sense lines of the N sets of control lines; the first pixel group and the second pixel group are any two pixel groups in the N pixel groups.

10. The apparatus of claim 9, wherein the first exposure control signal is configured to control a first exposure start time and a first exposure end time of the first pixel group; the second exposure control signal is used for controlling a second exposure start time and a second exposure end time of the second pixel group;

And/or the first exposure control signal is used for controlling the times and the exposure time of the first pixel group for executing the exposure operation in the exposure period; the second exposure control signal is used for controlling the times and the exposure time of the second pixel group for executing the exposure operation in the exposure period.

11. The apparatus of claim 1, wherein the image acquisition apparatus further comprises: a gain control unit; the gain control unit comprises N groups of signal lines; the N groups of signal lines are used for transmitting gain control signals to the N pixel groups; a group of signal lines is connected with one pixel group;

the gain control unit is used for sending a first gain control signal to a first pixel group through a first group of signal lines so that the first pixel group adjusts the brightness value in first pixel data based on the first gain control signal; the first pixel data is pixel data generated by exposing the first pixel group;

transmitting a second gain control signal to a second pixel group through a second group signal line so that the second pixel group adjusts a luminance value in second pixel data based on the second gain control signal; the second pixel data is pixel data generated by exposing the second pixel group; the first group of signal lines and the second group of signal lines are any two groups of signal lines in the N groups of signal lines; the first pixel group and the second pixel group are any two pixel groups in the N pixel groups.

12. A data reading method, characterized by being applied to an image acquisition apparatus, the image acquisition apparatus comprising: the device comprises a photosensitive pixel array unit, a data reading unit, a data transmission unit, a data processing unit and a control component, wherein the photosensitive pixel array unit comprises N pixel groups; the N pixel groups are independent from each other and are respectively controlled by different exposure control signals; n is an integer greater than or equal to 2; the data reading unit comprises N groups of reading lines, wherein the N groups of reading lines are used for reading pixel data generated by the N pixel groups; a group of readout lines connected to a pixel group; the method is applied to the control assembly; the method comprises the following steps:

the data reading unit is controlled to read pixel data generated by the exposure operation of the N pixel groups through the N groups of reading lines;

and controlling the pixel data to be transmitted through the data transmission unit.

13. The method according to claim 12, wherein controlling the data readout unit to read out pixel data generated by performing an exposure operation on the N pixel groups through the N groups of readout lines, respectively, includes:

controlling the data reading unit to read out first pixel data generated by the first pixel group through a first group of reading lines;

Controlling the data reading unit to read out second pixel data generated by the second pixel group through a second group of reading lines; wherein the first set of sense lines and the second set of sense lines are any two sets of sense lines of the N sets of sense lines; the first pixel group and the second pixel group are any two pixel groups in the N pixel groups.

14. The method of claim 12, wherein prior to said controlling the transmission of the pixel data by the data transmission unit, the method further comprises:

acquiring first pixel data, and converting the original bit width of the first pixel data into a first bit width; the first pixel data is pixel data generated by exposing a first pixel group in the N pixel groups;

acquiring second pixel data, and converting the original bit width of the second pixel data into a second bit width; the second pixel data is pixel data generated by exposing a second pixel group in the N pixel groups; the N pixel groups correspond to N groups of bit widths; the first bit width and the second bit width are any two groups of bit widths in the N bit widths;

the controlling the pixel data to be transmitted by the data transmission unit includes:

Converting the original bit width of the first pixel data into a first bit width, and transmitting the first pixel data through the data transmission unit;

and converting the original bit width of the second pixel data into a second bit width, and transmitting the second pixel data through the data transmission unit.

15. The method of claim 14, wherein the N groups of bit widths are independent of each other; wherein the first bit width is determined by the exposure time length of the first pixel group and/or the bandwidth of the data transmission unit; the second bit width is determined by the exposure time of the second pixel group and/or the bandwidth of the data transmission unit.

16. The method according to claim 12, wherein the method further comprises:

and controlling the data transmission unit to transmit the pixel data acquired by the data reading unit to the data processing unit so that the data processing unit restores the pixel data into a signal to be processed and performs image processing on the signal to be processed.

17. The method according to claim 16, wherein the signal to be processed is obtained by the data processing unit by reorganizing the pixel data according to one or more of an exposure time of the pixel data, a readout time of the pixel data, an image channel in which the pixel data is located, or position information of the pixel data.

18. The method of claim 12, wherein the control assembly comprises N sets of control lines; the N groups of control lines are used for transmitting exposure control signals to the N pixel groups; a group of control lines is connected with a pixel group; before the controlling the data readout unit to read out the pixel data generated by the exposure operation performed by the N pixel groups through the N groups of readout lines, respectively, the method further includes:

transmitting a first exposure control signal to a first pixel group through a first group control line, so that the first pixel group is exposed based on the first exposure control signal;

transmitting a second exposure control signal to a second pixel group through a second group control line so that the second pixel group is exposed based on the second exposure control signal; the first set of control lines and the second set of control lines are any two sets of sense lines of the N sets of control lines; the first pixel group and the second pixel group are any two pixel groups in the N pixel groups.

19. The method of claim 18, wherein the first exposure control signal is used to control a first exposure start time and a first exposure end time of the first pixel group; the second exposure control signal is used for controlling a second exposure start time and a second exposure end time of the second pixel group;

And/or the first exposure control signal is used for controlling the times and the exposure time of the first pixel group for executing the exposure operation in the exposure period; the second exposure control signal is used for controlling the times and the exposure time of the second pixel group for executing the exposure operation in the exposure period.

20. The method of claim 12, wherein the control assembly further comprises N sets of signal lines; the N groups of signal lines are used for transmitting gain control signals to the N pixel groups; a group of signal lines is connected with one pixel group; the method further comprises the steps of:

transmitting a first gain control signal to a first pixel group through a first group signal line so that the first pixel group adjusts a luminance value in first pixel data based on the first gain control signal; the first pixel data is pixel data generated by exposing the first pixel group;

transmitting a second gain control signal to a second pixel group through a second group signal line so that the second pixel group adjusts a luminance value in second pixel data based on the second gain control signal; the second pixel data is pixel data generated by exposing the second pixel group; the first group of signal lines and the second group of signal lines are any two groups of signal lines in the N groups of signal lines; the first pixel group and the second pixel group are any two pixel groups in the N pixel groups.

21. A control assembly, comprising:

one or more processors;

one or more memories;

wherein the one or more memories are configured to store computer program code comprising computer instructions that, when executed by the one or more processors, perform the data readout method of any of claims 12 to 20.

22. A computer-readable storage medium storing computer-executable instructions that, when executed on a computer, cause the computer to perform the data readout method of any one of claims 12 to 20.