CN116367001A - Image sensor serial data dynamic training method - Google Patents

Abstract

The invention provides a serial data dynamic training method of an image sensor, which comprises the following steps: setting a parameter register training word related part, setting a training control signal to be effective and lasting more than 2 time sequence driving clock cycles in a data transmission idle period or a specific time interval of an image sensor, continuously transmitting a corresponding number of special training words with asymmetry to a data receiving module by the image sensor according to the training control signal, comparing the received bytes with the special training words or the shift varieties thereof by the data receiving module, judging a data shift offset, determining a serial-parallel adjustment quantity and a serial-parallel adjustment direction according to the quantized bit number of the image sensor and the data shift offset, and adjusting a serial-parallel conversion node to finish dynamic training. The invention can realize real-time dynamic training of asynchronous serial image data on the premise of not influencing normal imaging of the image sensor, ensure correct receiving of the image data of the sensor and improve the reliability and stability of long-time working of the image sensor.

Description

Image sensor serial data dynamic training method

Technical Field

The invention relates to the technical field of image sensors, in particular to a serial data dynamic training method of an image sensor.

Background

Image sensors are mainly classified into two types by structures, charge-Coupled Device (CCD) and complementary metal oxide field effect transistor (Complementary Metal Oxide Semiconductor, CMOS) image sensors. The CCD image sensor outputs analog signals, the CCD image sensor is used together with a video processor, and the video processor converts the analog signals into digital signals; the CMOS image sensor internally integrates a mode conversion function and directly outputs a digital signal. The digital signals of the image sensor are divided into two types, serial data and parallel data: the parallel data can form synchronous transmission with a clock signal, the interface protocol is simple, but the bus width is large, a plurality of transmission lines with bit width are needed, and the transmission speed is limited due to high synchronism requirement during high-speed transmission; the serial data is asynchronous transmission, the bus width is small, only 2 transmission lines are needed, and the transmission rate is far higher than that of parallel data, but the serial data transmission needs to carry out a certain data synchronous handshake or data training process, so that the protocol and the synchronous process are complex, and a certain difficulty is brought to engineering realization.

With the development of technology, the performance of the image sensor is remarkably improved, the number of pixels is increased, the working frequency is improved, the data rate of image data is higher and higher, the transmission precision of a parallel mode cannot meet the requirement of high-speed data flow, and serial data transmission is applied more, so that image training becomes an important technology. Since the traversal comparison is required during training and a sufficient number of traversal positions are required for training accuracy, the asynchronous image training of the complete serial data requires a certain time, so that the asynchronous image training is often performed before the start of the image capturing or in an image capturing gap enough to complete the training, and the interval gap of only a few clock cycles cannot be implemented during continuous imaging. However, in the actual engineering project, the working state of the counter in the image sensor is affected due to the influence of space environmental radiation or electromagnetic interference, so that the problem that the output image is in data serial position is caused, and the like, so that the serial data which is stably received through image training is abnormal, and a dynamic training method which can be completed in a short time is needed to be capable of receiving and correcting the image data in real time, so that the image data of the sensor is correctly received.

Disclosure of Invention

Aiming at the problem of serial data serial bit of the image sensor caused by space environment radiation or electromagnetic interference and the like, the invention provides an asynchronous serial image data real-time dynamic training method which is based on special training words and does not influence the normal imaging of the image sensor.

In order to solve the problems, the invention adopts the following technical scheme:

a serial data dynamic training method of an image sensor comprises the following steps:

step 1: setting a training word related part of an image sensor parameter register through a communication data bus;

step 2: setting a training control signal to be effective in a data transmission idle period or a specific time interval of an image sensor, wherein the training control signal lasts for more than 2 time sequence driving clock cycles, and the image sensor continuously transmits a corresponding number of special training words with asymmetry to a data receiving module according to the training control signal;

step 3: the data receiving module receives at least more than 1 byte corresponding to a special training word or a shift variation thereof, compares the received byte with the special training word or the shift variation thereof, judges a data shift offset, and then determines a serial-parallel adjustment amount and a serial-parallel adjustment direction according to the quantized bit number of the image sensor and the data shift offset;

step 4: and the data receiving module adjusts the serial-parallel conversion node according to the serial-parallel adjustment quantity and the serial-parallel adjustment direction, so as to complete the serial data dynamic training of the image sensor.

The beneficial effects of the invention are as follows:

the invention provides a dynamic training method for serial data of an image sensor, which can solve the problem of serial data of the image sensor caused by space environment radiation or electromagnetic interference and the like.

Drawings

FIG. 1 is a timing diagram of a normal data reception state after completion of asynchronous data training;

FIG. 2 is a timing diagram of a data receiving state after a single event upset occurs in a frequency division counter of an image sensor;

FIG. 3 is a flowchart of a method for dynamic training of serial data of an image sensor according to an embodiment of the present invention;

FIG. 4 is a program flow chart of a method for dynamically training serial data of an image sensor according to an embodiment of the present invention;

FIG. 5 is a timing diagram of a dynamic training state during normal operation of the image sensor;

FIG. 6 is a timing diagram of a dynamic training state when a frequency division counter inside an image sensor turns over and the count value becomes smaller;

FIG. 7 is a timing diagram of the normal data receiving state after training according to FIG. 6;

FIG. 8 is a timing diagram of a dynamic training state when the internal frequency division counter of the image sensor is turned over by a single event and the count value becomes large;

fig. 9 is a timing chart of the normal data receiving state after training corresponding to fig. 8 is completed.

Detailed Description

The technical scheme of the present invention will be described in detail with reference to the accompanying drawings and preferred embodiments.

The image sensor can perform a complete training in an initialization stage, so that the camera system can correctly receive the serial image data of the sensor, as shown in fig. 1, the image sensor receives an external high-speed clock, generates an internal working clock (frequency division multiple is the quantization bit number of the image of the sensor) of the device through an internal frequency division counter, and simultaneously, the image sensor can output the serial image data outwards according to the frequency division counter. The camera core processor (such as FPGA, ISP chip, etc.) also has a corresponding serial-parallel conversion counter, and when performing image training, the serial-parallel conversion node is adjusted, so that the image sensor and the core processor serial-parallel conversion module are synchronized, the serial-parallel converted image data is an independent and complete byte, and then the subsequent operation processing is performed after the internal clock synchronization of the core processor.

When external disturbance such as space environment radiation or electromagnetic interference occurs, the level of an internal counter of an image sensor or a core processor is possibly turned over, so that the synchronization failure of a serial-parallel conversion module of the image sensor and the core processor is caused, and the mismatch occurs in linkage of the image sensor and the serial-parallel conversion module, so that the serial data of the image sensor is caused to have a serial-to-bit problem. As shown in fig. 2, in order to cause a single event phenomenon, a frequency division counter inside the image sensor turns over a single event, so that a serial bit phenomenon occurs in data reception.

Aiming at the problem of serial data serial bit of the image sensor caused by space environment radiation or electromagnetic interference and the like, the embodiment provides a method for dynamically training the serial data of the image sensor, as shown in fig. 3, which specifically comprises the following steps:

step 1: setting a training word related part of an image sensor parameter register through a communication data bus (such as an SPI bus and the like), so that the image sensor outputs a special training word with asymmetry when a training control signal is effective;

step 2: setting a training control signal to be effective in a data transmission idle period or a specific time interval of an image sensor, continuously transmitting a corresponding number (2, 3 and …) of special training words with asymmetry to a data receiving module by the image sensor according to the training control signal, wherein when the received image data has a data serial phenomenon, the special training words are received, but the serial phenomenon can occur, the data received by the data receiving module can be completely received by the 2 or more continuously transmitted special training words, and at least 1 or more special training words or bytes corresponding to shift varieties of the special training words can be completely received by the data receiving module;

step 3: the data receiving module compares the received bytes with the special training word or the shift variation thereof, can judge the data shift offset, and determines the final serial-parallel adjustment quantity and the serial-parallel adjustment direction according to the data shift offset, the quantization bit number of the image sensor and the multiple comparison result of the bytes and the special training word or the shift variation thereof;

step 4: the data receiving module adjusts the serial-parallel conversion node according to the serial-parallel adjustment quantity and the serial-parallel adjustment direction, so that the image data of the sensor can be correctly received, and the effect of realizing real-time dynamic training of the asynchronous serial image data on the premise of not influencing the normal imaging of the image sensor is achieved.

In order to realize real-time dynamic training of asynchronous serial image data, the invention adopts a special training word as a reference object, and the special training word has asymmetric characteristics, namely the training word is not symmetrical, such as binary numbers 010 and 110011, and belongs to the training word with symmetry, and the symmetry number can cause difficult boundary distinction during serial transmission and can not accurately position the data through searching, so the special training word used by the invention is required to be asymmetric. The embodiment provides two special training word construction methods: one is binary code of 010011000111, 01 staggered format, under which the structure of the special training word is that from 10 and 1, 2 0 and 2 1, 3 0 and 3 1, & ltCHEM & gt, n 0 and n 1 (n is a positive integer), and the data length is determined by the quantization bit number of the image sensor data; the two are binary codes of 101100111000, 10 staggered formats, and under the formats, the structure of the special training word is that the special training word starts from 1 to 0, and is sequentially 2 to 1 to 2 to 0, 3 to 1 to 3 to 0, n to 1 to n to 0 (n is a positive integer), and the data length is also determined by the quantization bit number of the image sensor data. Both configurations can ensure that more than 3bit training words are asymmetric. For example, a special training word of a twelve-bit quantized image may be 0x4C7 (type 1) or 0xB38 (type 2), or roller-shifted variant data of these two data: 0x98E (type 1 shift variant) or 0xCE2 (type 2 shift variant); the rest of the quantized bit numbers are specially trained for handwriting and so on.

The training control signal is the smallest in 2 time sequence driving clock cycles, the required resource is the smallest, the scheme is the simplest, but for 3 time sequence driving clock cycles and more, the invention can also be realized in a similar way, for example, for the condition that the training control signal is in 3 time sequence driving clock cycles, the data receiving module receives and identifies bytes corresponding to continuous 2 or 3 special training words and shift varieties thereof, then carries out corresponding comparison judgment, and finally determines the serial-parallel adjustment quantity and the serial-parallel adjustment direction.

The process of the image sensor serial data dynamic training method provided by the invention is explained by taking the specific implementation mode that the effective time of the training control signal is set to 2 time sequence driving clock cycles, and after the relevant part of the training word of the image sensor parameter register is set, as shown in fig. 4, the specific process of the steps 2 to 4 is as follows:

a) Judging whether the dynamic training enabling is started or not, if the dynamic training enabling is not started, repeating the step a, and if the dynamic training enabling is started, jumping to the step b;

b) C, judging whether the image sensor is in a data transmission idle period or a specific time interval, if so, jumping to the step c, otherwise, repeating the step b;

c) Generating an image sensor training control signal with 2 time sequence driving clock cycles, and jumping to the step d;

d) Judging the 1 st byte corresponding to the training control signal of the image sensor, judging whether the training control signal is a special training word or a shift variation of the training word, and jumping to the step e after the judgment is finished;

e) Judging the 2 nd byte corresponding to the training control signal of the image sensor, judging whether the training control signal is a special training word or a shift variation of the training word, and jumping to the step f after the judgment is finished;

f) Judging whether the 1 st byte and the 2 nd byte are special training words or shift variants thereof, if so, jumping to the step g, otherwise jumping to the step n;

g) Judging whether the data shift offset is 0, if so, jumping to the step h, otherwise jumping to the step i;

h) The serial-parallel adjustment quantity is 0, the serial-parallel adjustment direction is unchanged, and the step s is skipped;

i) Judging whether the data shift offset is larger than 1 and smaller than 6, if so, jumping to step j, otherwise jumping to step k;

j) The serial-parallel adjustment quantity is the quantized bit number of the image sensor minus the data shift offset, the serial-parallel adjustment direction is the subtraction, and the step s is skipped;

k) Judging whether the data shift offset is larger than 7 and smaller than 11, if yes, jumping to step l, otherwise jumping to step m;

l) the serial-parallel adjustment quantity is a data shift offset quantity, the serial-parallel adjustment direction is added, and the step s is skipped;

m) the serial-parallel adjustment quantity is 0, the serial-parallel adjustment direction is unchanged, and the step s is skipped;

n) judging whether the 1 st byte is a special training word or a shift variant thereof and the 2 nd byte is not, if yes, jumping to the step o, otherwise jumping to the step p;

o) the serial-parallel adjustment quantity is a data shift offset quantity, the serial-parallel adjustment direction is added, and the step s is skipped;

p) judging whether the 2 nd byte is a special training word or a shift variant thereof and the 1 st byte is not, if yes, jumping to the step q, otherwise jumping to the step r;

q) the serial-parallel adjustment quantity is the quantized bit number of the image sensor minus the data shift offset, the serial-parallel adjustment direction is the subtraction, and the step s is skipped;

r) the serial-parallel adjustment quantity is 0, the serial-parallel adjustment direction is unchanged, and the step s is skipped;

s) adjusting the serial-parallel conversion node according to the serial-parallel adjustment quantity and the serial-parallel adjustment direction, and jumping to the step t;

t) judging whether the serial-parallel conversion node is smaller than 0, if yes, jumping to step u, otherwise jumping to step v;

u) adjusting the serial-parallel conversion node into the serial-parallel conversion node plus the image sensor quantization bit number, and jumping to the step a;

v) judging whether the serial-parallel conversion node is larger than or equal to the quantized bit number of the image sensor, if yes, jumping to the step w, otherwise jumping to the step a;

w) adjusting the serial-parallel conversion node to be the serial-parallel conversion node minus the image sensor quantization bit number, and jumping to the step a.

The data shift offset in the present invention refers to the number of bits of the data shifted to the left with respect to the special training word on the premise that the data collected by the data receiving module is the special training word or the shift variation thereof, taking the special training word 0x98E (this is the type 1 special training word shift variation) under 12-bit quantization as an example, and the data shift offset between the data receiving module and each variation is shown in the following table.

Table 1 data shift offset correspondence table

Sequence number	Sampling data	Data shift offset
			1	0x98E	0
2	0x31D	1
			3	0x63A	2
4	0xC74	3
			5	0x8E9	4
6	0x1D3	5
			7	0x3A6	6
8	0x74C	7
			9	0xE98	8
10	0xD31	9
			11	0xA63	10
12	0x4C7	11

The following describes the dynamic training method of the image sensor data under the conditions of normal serial communication and abnormal serial data, assuming that the quantization bit number of the image sensor is 12 bits, the special training word uses 0x98E, and the non-image filling redundant data of the image sensor is 0xD56. As shown in fig. 5, in the dynamic training state timing diagram of the image sensor during normal operation, a training control signal of 2 time sequence driving clock cycles is sent in an idle period or a specific time interval of data transmission of the image sensor, then the image sensor sends 2 special training words, after initialization training, the system receives data normally, and then the system receives 0x98E and 0x98E respectively in 2 periods in which the training control signal is valid after serial-parallel conversion, and according to the program flow, the adjustment amount of the image data receiving counter of the FPGA is 0, i.e. no adjustment is made, and the system keeps the current state to continue normal operation. In fig. 5 to 9, train Pattern is a special training word, dummy Data is padding Data, the padding Data can be set through a register, the detector (i.e., image sensor) outputs Train Pattern when the training control signal is valid (pulled high), i.e., the special training word, the detector (i.e., image sensor) outputs Dummy Data when the training control signal is invalid (pulled low), i.e., the padding Data, dummy Data Mix is mixed padding Data, i.e., the mixture of two padding Data, and the trace Pattern byte1 part+dummy DataPart indicates that the corresponding received Data is a concatenation of a partial bit of the 1 st byte of the special training word and a partial bit of the padding Data.

The image sensor can perform a complete training in an initialization stage, so that the camera system can correctly receive serial image data of the sensor, and when the image sensor is abnormal due to space environment radiation or electromagnetic interference, the received image data can have data serial positions. When the level of the counter inside the image sensor is turned over due to external disturbance such as space environment radiation or electromagnetic interference, the counter value may become larger or smaller.

As shown in FIG. 6, the timing diagram of the dynamic training state when the single event of the internal frequency division counter of the image sensor is turned over and the count value becomes smaller, the external interference causes the internal frequency division counter of the image sensor to change from 4 to 0, which causes the internal working state of the image sensor to change, the output data sequence and the time to change, which causes the serial bit of the data received by the core processor to appear, which is the mixing of the data of the front and back 2 bytes of the image sensor, namely the aliasing data formed by splicing the 7 th to 0 th bit of the n+1th byte and the 11 th to 8 th bit of the N byte, so that the 2 corresponding bytes actually collected after the training control signal of 2 time sequence driving clock cycles is sent in the idle period or the specific time interval of the data transmission of the image sensor are respectively 0x8ED and 0x8E9, wherein 0x8ED is obtained by splicing the 7 th to 0 th bit of the special training word 0x98E and the 11 th to 8 th bits of the filling data 0xD55, 0x8E9 is obtained by splicing the 7 th to 0 th bit of the two special training words and the 11 th to 8 th bits, the 1 st byte is not the special training word or the shift variation thereof, the 2 nd byte is the shift variation of the special training word, the data shift offset is 4, the FPGA image data receiving counter serial-parallel adjustment amount is 8 when the judgment result of the 2 bytes is synthesized, the adjustment direction is reduced, the FPGA image serial-parallel node is adjusted to 1 from 9, the dynamic training is completed, and the data serial-bit phenomenon is corrected, as shown in figure 7.

As shown in FIG. 8, the timing diagram of the dynamic training state when the single event of the internal frequency division counter of the image sensor is turned over and the count value becomes large, the external interference causes the internal frequency division counter of the image sensor to change from 3 to 7, which causes the internal working state of the image sensor to change, the output data sequence and the timing to change, which causes the serial bit of the data received by the core processor to appear, which is the mixing of the data of the front and rear 2 bytes of the image sensor, namely the mixing data formed by splicing the 3 rd bit to 0 th bit of the n+2th byte and the 11 th bit to 4 th bit of the n+1th byte, so that the 2 corresponding bytes actually collected after the training control signal of 2 time sequence driving clock cycles is sent in the idle period or the specific time interval of the data transmission of the image sensor are respectively 0xE98 and 0x598, wherein, 0xE98 is obtained by splicing the 3 rd bit to 0 th bit+11 th bit to 4 th bit of two special training words, 0x598 is obtained by splicing the 3 rd bit to 0 th bit of filling data 0xD 55+11 th bit to 4 th bit of special training word 0x98E, 1 st byte is a shift variant of the special training word, 2 nd byte is not the special training word or the shift variant thereof, the data shift offset is 8, the adjustment amount of the FPGA image data receiving counter is 8 when the judgment result of 2 bytes is synthesized, the adjustment direction is added, the FPGA image serial-parallel conversion node is adjusted to 17 from being reduced to be 5 when the quantization bit is larger than 12, the dynamic training is completed, and the data serial-bit phenomenon is corrected, as shown in fig. 9.

The invention provides a dynamic training method for serial data of an image sensor, which can solve the problem of serial data of the image sensor caused by space environment radiation or electromagnetic interference and the like.

The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims (3)

1. The serial data dynamic training method for the image sensor is characterized by comprising the following steps of:

step 1: setting a training word related part of an image sensor parameter register through a communication data bus;

step 2: setting a training control signal to be effective in a data transmission idle period or a specific time interval of an image sensor, wherein the training control signal lasts for more than 2 time sequence driving clock cycles, and the image sensor continuously transmits a corresponding number of special training words with asymmetry to a data receiving module according to the training control signal;

step 3: the data receiving module receives at least more than 1 byte corresponding to a special training word or a shift variation thereof, compares the received byte with the special training word or the shift variation thereof, judges a data shift offset, and then determines a serial-parallel adjustment amount and a serial-parallel adjustment direction according to the quantized bit number of the image sensor and the data shift offset;

step 4: and the data receiving module adjusts the serial-parallel conversion node according to the serial-parallel adjustment quantity and the serial-parallel adjustment direction, so as to complete the serial data dynamic training of the image sensor.

2. The method for dynamically training serial data of an image sensor according to claim 1, wherein when the special training word adopts a 01-interleaved format, the special training word has a structure of from 10 to 1, and sequentially from 2 0 to 2 1, from 3 0 to 3 1, & & gtand from n 0 to n 1; when the special training word adopts a 10-interlace format, the structure of the special training word is from 1 to 1 and 1 to 0, sequentially 2 pieces of 1 and 2 pieces of 0, 3 pieces of 1 and 3 pieces of 0, & ltS & gt, n pieces of 1 and n pieces of 0; wherein n is a positive integer.

3. The method according to claim 1 or 2, wherein the specific procedure of steps 2 to 4 is as follows when the training control signal lasts for 2 time-series driving clock cycles:

a) Judging whether the dynamic training enabling is started, if yes, jumping to the step b, otherwise repeating the step a;

b) Judging whether the image sensor is in a data transmission idle period or a specific time interval, if so, jumping to the step c, otherwise, repeating the step b;

c) Generating an image sensor training control signal with 2 time sequence driving clock cycles, and jumping to the step d;

d) Judging whether the 1 st byte corresponding to the training control signal of the image sensor is a special training word or a shift variation thereof, and jumping to the step e after the judgment is finished;

e) Judging whether the 2 nd byte corresponding to the training control signal of the image sensor is a special training word or a shift variation thereof, and jumping to the step f after the judgment is finished;

f) Judging whether the 1 st byte and the 2 nd byte are special training words or shift variants thereof, if yes, jumping to the step g, otherwise jumping to the step n;

g) Judging whether the data shift offset is 0, if so, jumping to the step h, otherwise jumping to the step i;

h) The serial-parallel adjustment quantity is 0, the serial-parallel adjustment direction is unchanged, and the step s is skipped;

i) Judging whether the data shift offset is larger than 1 and smaller than 6, if yes, jumping to step j, otherwise jumping to step k;

j) The serial-parallel adjustment quantity is the quantized bit number of the image sensor minus the data shift offset, the serial-parallel adjustment direction is the subtraction, and the step s is skipped;

k) Judging whether the data shift offset is larger than 7 and smaller than 11, if yes, jumping to step l, otherwise jumping to step m;

l) the serial-parallel adjustment quantity is a data shift offset quantity, the serial-parallel adjustment direction is added, and the step s is skipped;

m) the serial-parallel adjustment quantity is 0, the serial-parallel adjustment direction is unchanged, and the step s is skipped;

n) judging whether the 1 st byte is a special training word or a shift variant thereof and the 2 nd byte is not, if yes, jumping to the step o, otherwise jumping to the step p;

o) the serial-parallel adjustment quantity is a data shift offset quantity, the serial-parallel adjustment direction is added, and the step s is skipped;

p) judging whether the 2 nd byte is a training word or a shift variant thereof and the 1 st byte is not, if yes, jumping to the step q, otherwise jumping to the step r;

q) the serial-parallel adjustment quantity is the quantized bit number of the image sensor minus the data shift offset, the serial-parallel adjustment direction is the subtraction, and the step s is skipped;

r) the serial-parallel adjustment quantity is 0, the serial-parallel adjustment direction is unchanged, and the step s is skipped;

s) adjusting the serial-parallel conversion node according to the serial-parallel adjustment quantity and the serial-parallel adjustment direction, and jumping to the step t;

t) judging whether the serial-parallel conversion node is smaller than 0, if yes, jumping to step u, otherwise jumping to step v;

u) adjusting the serial-parallel conversion node into the serial-parallel conversion node plus the image sensor quantization bit number, and jumping to the step a;

v) judging whether the serial-parallel conversion node is larger than or equal to the quantized bit number of the image sensor, if so, jumping to the step w, otherwise jumping to the step a;

w) adjusting the serial-parallel conversion node to be the serial-parallel conversion node minus the image sensor quantization bit number, and jumping to the step a.

Images