JP2013070128A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a camera adopting an imaging device with a plurality of outputs execute switching of a frame rate and vertical and horizontal inversion outputs with a small-scale line memory alone without using a frame memory or the like, and to perform defective pixel interpolation suitable to an output mode.SOLUTION: The number of output channels of an imaging device is switched because of a change of a frame rate. By switching a compositing method for channels according to the number of output channels and establishing a camera link format suited to the frame rate, switching of the frame rate as an imaging apparatus can be achieved. In addition, an interpolation coordinate is re-calculated for each output mode switching from a coordinate of a registered defect pixel.

Description

  The present invention relates to an imaging apparatus using an imaging device having a plurality of imaging regions and a plurality of output units, and more particularly to a method of switching frame rates and switching inverted output and interpolation of defective pixels. .

In recent years, an imaging device having a large number of effective pixels (high pixels) can be produced, and an imaging device capable of acquiring a high-definition image has been realized. However, if the number of pixels is large, it takes time to read out the video signal from each pixel at the conventional reading speed, and the imaging apparatus has a low frame rate. Therefore, in order to realize a high frame rate of an imaging apparatus using a high-pixel imaging device, an imaging device having a plurality of outputs has been commercialized. For example, a high frame rate is realized by dividing the output of an imaging device having 2048 × 2048 effective pixels into 16 channels (ch) in units of 128 × 2048 pixels. It is also possible to reduce the frame rate by reducing the number of output channels using a part of the channels of 16 channels (for example, 8 channels, 4 channels, 2 channels).
In addition, in the camera link interface which is one of machine vision interfaces, the frame rate can be changed by selecting the number of TAPs and the output frequency. By increasing the number of TAPs, a higher frame rate can be realized. However, the number of cables increases and the number of capture boards is limited. Increasing the output frequency also contributes to higher frame rates, but the cable length and capture board are limited.
In addition, there is an imaging apparatus that performs interpolation of defective pixels using registered coordinates of defective pixels.

JP 2008-42838 A

Patent Document 1 discloses a technique for selecting the position of a pixel region for partial scanning and switching the scanning speed of a horizontal pulse and a vertical pulse for driving an image sensor. However, Patent Document 1 is a technology related to partial scan that sets a plurality of pixel areas that require image information and a plurality of pixel areas that do not require image information, and uses a high-pixel imaging device. It is not effective for increasing the frame rate of the imaging apparatus.
An object of the present invention is to enable switching of a frame rate and up / down and left / right reversal output with only a small line memory without using a frame memory or the like in a camera employing an image sensor having a plurality of outputs. Further, appropriate defective pixel interpolation is performed according to the output mode.

In order to achieve the above object, an image pickup apparatus according to the present invention includes an image pickup device having an image pickup region divided into a plurality of portions and an output channel for outputting a video signal for each of the plurality of image pickup regions. An image sensor setting unit for switching the number and output order of output channels for outputting signals, a synthesis processing unit for synthesizing the plurality of video signals output from the image sensor, and a camera link for the synthesized signals An image output unit for converting to an output format is provided, and the first feature is that the frame rate is switched by a combination of switching of the number of output channels of the image sensor and a combining method of the combining processing unit.
The image pickup apparatus having the first feature has a second feature that the image pickup device has an up / down and left / right inversion output function by a combination of up / down and left / right output switching of the image pickup device and the composition method of the synthesis processing unit.
In the imaging device of the second feature, the third feature is that switching of the frame rate, switching of the up / down and left / right inversion outputs, switching of partial scan, etc., and appropriate defective pixel interpolation are performed. .

  According to the present invention, in a camera that employs an imaging device having a plurality of outputs, frame rates can be switched and up / down and left / right inverted output can be performed using only a small line memory without using a frame memory or the like. Further, it is possible to perform appropriate defective pixel interpolation according to the output mode.

It is a block diagram which shows the structure of one Example of the imaging device of this invention. 4 is a table showing the relationship between the number of output channels of the image sensor 101 and the camera link output mode in an embodiment of the imaging apparatus of the present invention. It is a figure for demonstrating the synthetic | combination processing method for every output mode in one Example of the imaging device of this invention.

According to the present invention, in a camera that employs an imaging device having a plurality of outputs, frame rates can be switched and up / down and left / right reversed output can be performed using only a small line memory without using a frame memory or the like. Further, the imaging apparatus performs appropriate defective pixel interpolation according to the output mode.
For this reason, the imaging apparatus of the present invention switches the number of output channels of the imaging device in order to change or switch the frame rate. In addition, the channel combining method is switched according to the number of output channels of the image sensor, and the camera link format is set according to the frame rate. As a result, an imaging apparatus capable of switching the frame rate and switching the output mode is realized. Further, the imaging apparatus of the present invention recalculates the interpolation coordinates every time the output mode is switched from the coordinates of the registered defective pixels.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
In addition, the following description is for describing one embodiment of the present invention, and does not limit the scope of the present invention. Accordingly, those skilled in the art can employ embodiments in which these elements or all of the elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.
In the description of each drawing, the same reference numerals are assigned to components having the same function, and the description is omitted as much as possible to avoid duplication.

  An embodiment of the imaging apparatus of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an embodiment of an imaging apparatus of the present invention. 10 is an image pickup device, 101 is an image sensor, 103 is a serial-parallel conversion unit that converts a serial signal into a parallel signal, 104 is a synthesis processing unit, 105 is a defective pixel interpolation unit, 106 is an image output unit, and 108 is a frame rate switching unit. is there. In the synthesis processing unit, 141 is a line memory write channel selector, 143 is a line memory (line memory 1 to line memory 16), and 145 is a TAP selector (TAP1 to TAP8). For example, the effective pixel number of the image sensor 101 is 2048 × 2048 pixels. In addition, the frame rate switching unit 108 performs imaging device control, synthesis processing control, defective pixel interpolation control, and output method switching of the imaging device 101, the synthesis processing unit 104, the defective pixel interpolation unit 105, and the image output unit 106. The process is executed in accordance with conditions set by an instruction from an operation unit (not shown).

In the imaging apparatus 10 of FIG. 1, the imaging element 101 divides the channel into 16 channels of ch1 to ch16 and outputs video signals of pixel areas of 128 × 2048 pixels to the serial-parallel conversion unit 103, respectively.
The serial-parallel converter 103 converts the 16-channel video signals input from the image sensor 101 into parallel data, and outputs the parallel data to the synthesis processor 104.
The composition processing unit 104 converts the 16-channel video signals input from the serial-parallel conversion unit 103 into parallel data by the line memory write channel selectors 141 and 16, the line memory 143 of the 16 lines, and the eight TAP selectors 145, respectively. Each is output to the composition processing unit 105. Specifically, the video signals of a plurality of channels input from the serial-parallel converter 103 are output to the line memory 143 according to the selection of the line memory write ch selector 141, and are combined by the TAP selector 145, and the defective pixel interpolator To 105.
The defective pixel interpolation unit 105 performs defective pixel interpolation, which will be described later, on each input TAP video signal, and outputs it to the image output unit 106 for each TAP video signal.
The image output unit 106 performs known video processing on each input TAP video signal, and outputs the video signal from the camera link terminal of the imaging apparatus 10 to the outside for each TAP video signal.

The camera link output mode in an embodiment of the imaging apparatus of the present invention will be described with reference to FIGS.
FIG. 2 is a table showing the relationship between the number of output channels of the image sensor 101 and the camera link output mode in an embodiment of the imaging apparatus of the present invention.
In FIG. 2, when the image sensor output is “40 MHz × 2 ch”, the frame rate is “18.75 fps”, and the camera link output mode is “40 MHz × 2 TAP (basic setting)”. When the image sensor output is “40 MHz × 4 ch”, the frame rate is “37.5 fps”, the camera link output mode is “80 MHz × 2 TAP (basic setting)”, or the camera link output mode is “ 40 MHz × 4 TAP (medium setting) ”. Furthermore, when the image sensor output is “40 MHz × 8 ch”, the frame rate is “75.0 fps”, the camera link output mode is “80 MHz × 4 TAP (medium setting)”, or the camera link output mode is “ 40 MHz × 8 TAP (maximum setting) ”. Furthermore, when the image sensor output is “40 MHz × 16 ch”, the frame rate is “150.0 fps”, and the camera link output mode is “80 MHz × 8 TAP (maximum setting)”.
FIG. 3 is a diagram for explaining a composition processing method for each output mode in an embodiment of the imaging apparatus of the present invention.

  2 and 3, when the camera link output mode is 40 MHz × 2 TAP, the output of the image sensor 101 is 40 MHz × 2 ch. In this case, video signals of 1024 × 2048 pixels are output from ch1 and ch9 of the image sensor 101, respectively. The video signals output from ch1 and ch9 are combined by the combining processing unit 104. Finally, video signals of 1024 × 2048 pixels are output at 40 MHz from TAP1 and TAP2 of the image output unit 106, respectively.

  When the camera link output mode is 80 MHz × 2 TAP, the output of the image sensor 101 is 40 MHz × 4 ch. In this case, 512 × 2048 pixel video signals are output from ch1, ch5, ch9, and ch13 of the image sensor 101, respectively. The video signals output from ch1 and ch5 are combined by the combining processing unit 104 and output from TAP1 of the image output unit 106 at 80 MHz. Similarly, the video signals output from ch9 and ch13 are combined by the combining processing unit 104 and output from TAP2 of the image output unit 106 at 80 MHz.

  When the camera link output mode is 40 MHz × 4 TAP, the output of the image sensor 101 is 40 MHz × 4 ch. In this case, 512 × 2048 pixel video signals are output from ch1, ch5, ch9, and ch13 of the image sensor 101, respectively. The video signal output from ch1 is combined by the combining processing unit 104 and output from TAP1 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch5 is combined by the combining processing unit 104 and output from TAP2 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch9 is combined by the combining processing unit 104 and output from TAP3 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch 13 is combined by the combining processing unit 104 and output from TAP 4 of the image output unit 106 at 40 MHz.

  When the camera link output mode is 80 MHz × 4 TAP, the output of the image sensor 101 is 40 MHz × 8 ch. In this case, video signals of 256 × 2048 pixels are output from ch1, ch3, ch5, ch7, ch9, ch11, ch13, and ch15 of the image sensor 101, respectively. The video signal output from ch1 and the video signal output from ch3 are combined by the combining processing unit 104 and output from TAP1 of the image output unit 106 at 80 MHz. Similarly, the video signal output from ch5 and the video signal output from ch7 are combined by the combining processing unit 104 and output from TAP2 of the image output unit 106 at 80 MHz. Similarly, the video signal output from ch9 and the video signal output from ch11 are combined by the combining processing unit 104 and output from TAP3 of the image output unit 106 at 80 MHz. Similarly, the video signal output from ch13 and the video signal output from ch15 are combined by the combining processing unit 104 and output from TAP4 of the image output unit 106 at 80 MHz.

  When the camera link output mode is 40 MHz × 8 TAP, the output of the image sensor 101 is 40 MHz × 8 ch. In this case, video signals of 256 × 2048 pixels are output from ch1, ch3, ch5, ch7, ch9, ch11, and ch13 of the image sensor 101, respectively. The video signal output from ch1 is combined by the combining processing unit 104 and output from TAP1 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch3 is combined by the combining processing unit 104 and output from TAP2 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch5 is combined by the combining processing unit 104 and output from TAP3 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch7 is combined by the combining processing unit 104 and output from TAP4 of the image output unit 106 at 40 MHz. The video signal output from ch9 is combined by the combining processing unit 104 and output from TAP5 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch11 is combined by the combining processing unit 104 and output from TAP6 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch 13 is combined by the combining processing unit 104 and output from TAP 7 of the image output unit 106 at 40 MHz. Similarly, the video signal output from ch 15 is combined by the combining processing unit 104 and output from TAP 8 of the image output unit 106 at 40 MHz.

When the camera link output mode is 80 MHz × 8 TAP, the output of the image sensor 101 is 40 MHz × 16 ch. In this case, video signals of 128 × 2048 pixels are output from the ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8, ch9, ch10, ch11, ch12, ch13, ch14, ch15, and ch16 of the image sensor 101, respectively. Is output.
The video signals output from ch1 and ch2 are combined by the combining processing unit 104 and output from TAP1 of the image output unit 106 at 80 MHz. The video signals output from ch3 and ch4 are combined by the combining processing unit 104 and output from TAP2 of the image output unit 106 at 80 MHz. The video signals output from ch5 and ch6 are combined by the combining processing unit 104 and output from TAP3 of the image output unit 106 at 80 MHz. The video signals output from ch7 and ch8 are combined by the combining processing unit 104 and output from TAP4 of the image output unit 106 at 80 MHz.
The video signals output from ch9 and ch10 are combined by the combining processing unit 104, and output from TAP5 of the image output unit 106 at 80 MHz. The video signals output from ch11 and ch12 are combined by the combining processing unit 104 and output from TAP6 of the image output unit 106 at 80 MHz. The video signals output from ch13 and ch14 are combined by the combining processing unit 104 and output from the TAP 7 of the image output unit 106 at 80 MHz. The video signals output from ch15 and ch16 are combined by the combining processing unit 104 and output from the TAP 8 of the image output unit 106 at 80 MHz.

Note that the imaging device of the imaging apparatus of the present invention has a left-right inversion output function. However, the data in the channel is inverted, and in order to perform the inverted output for one screen, it is necessary to replace the image data of the entire channel. For example, in the case of 80 MHz × 8 TAP, the video signals output from ch16 and ch15 are combined and the video signals output from TAP1 and ch14 and ch13 are combined to output the video output from TAP2 and from ch12 and ch11. The video signals output from ch10 and ch9 are synthesized by combining the signals, the video signals output from ch8 and ch7 are synthesized from TAP4, and the video signals output from ch6 and ch5 are synthesized from TAP5. The video signals output from ch4 and ch3 are synthesized from TAP6 and synthesized from TAP7, and the video signals output from ch2 and ch1 are synthesized and output from TAP8 at 80 MHz respectively.
Similar replacement is performed for other output modes.

Next, a defective pixel interpolation method in the defective pixel interpolation unit 105 will be described. When the output mode is switched as described above, it is necessary to convert the interpolation position of the defective pixel for each output TAP using the registered coordinates of the defective pixel. If the registered coordinates are (Xin, Yin), and the interpolated coordinates after considering the up / down / left / right inversion setting are (Xflip, Yflip),
(1) When there is no up / down and left / right reversal,
Xflip = Xin, Yflip = Yin
(2) In the case of only horizontal reversal,
Xflip = 2049−Xin, Yflip = Yin
(3) If only upside down,
Xflip = Xin, Yflip = 2049−Yin
(4) In case of vertical and horizontal flipping,
Xflip = 2049−Xin, Yflip = 2049−Yin
It becomes.
If the interpolation coordinates after considering the partial scan setting are (Xpar, Ypar), the partial scan start position is Ystart, and the width is Ywidth,
Xpar = Xflip, Ypar = Yflip-Ystart + 1 (Yflip is Ystart or more and less than Ystart + Ywidth)
It becomes.
TAP1 interpolation coordinates (Xtap1, Ytap1), TAP2 interpolation coordinates (Xtap2, Ytap2), TAP3 interpolation coordinates (Xtap3, Ytap3), TAP4 interpolation coordinates (Xtap4, Ytap4), and TAP5 interpolation coordinates ( Xtap5, Ytap5), TAP6 interpolation coordinates (Xtap6, Ytap6), TAP7 interpolation coordinates (Xtap7, Ytap7), and TAP8 interpolation coordinates (Xtap8, Ytap8).

In the case of 2TAP output, (Xtap1, Ytap1) and (Xtap2, Ytap2) are set.
Xpar from 1 to 1024
Xtap1 = Xpar, Ytap1 = Ypar
Xpar from 1025 to 2048
Xtap2 = Xpar-1024, Ytap2 = Ypar
It becomes.

In the case of 4TAP output, (Xtap1, Ytap1), (Xtap2, Ytap2), (Xtap3, Ytap3), (Xtap4, Ytap4) are set.
Xpar from 1 to 512
Xtap1 = Xpar, Ytap1 = Ypar
From Xpar 513 to 1024,
Xtap2 = Xpar-512, Ytap2 = Ypar
Xpar from 1025 to 1536
Xtap3 = Xpar-1024, Ytap3 = Ypar
From Xpar 1537 to 2048,
Xtap4 = Xpar-1536, Ytap4 = Ypar
It becomes.

In the case of 8TAP output, (Xtap1, Ytap1), (Xtap2, Ytap2), (Xtap3, Ytap3), (Xtap4, Ytap4), (Xtap5, Ytap5), (Xtap6, Ytap6), (Xtap7, Ytap7), ( Xtap8, Ytap8) are set.
Xpar from 1 to 256
Xtap1 = Xpar, Ytap1 = Ypar
From Xpar 257 to 512,
Xtap2 = Xpar-256, Ytap2 = Ypar
From Xpar 513 to 768,
Xtap3 = Xpar-512, Ytap3 = Ypar
Xpar from 769 to 1024
Xtap4 = Xpar-768, Ytap4 = Ypar
When Xpar is from 1025 to 1280,
Xtap5 = Xpar-1024, Ytap5 = Ypar
From Xpar 1281 to 1536,
Xtap6 = Xpar-1280, Ytap6 = Ypar
Xpar from 1537 to 1792
Xtap7 = Xpar-1526, Ytap7 = Ypar
Xpar from 1792 to 2048
Xtap8 = Xpar-1792, Ytap8 = Ypar
It becomes.

  According to the above-described embodiment, it is possible to switch the frame rate and to output vertically and horizontally reversed with only a small line memory. In addition, by using the coordinates of the defect image that has been registered once to recalculate and set the interpolation pixel for each output mode, it is possible to perform appropriate defect pixel interpolation.

  DESCRIPTION OF SYMBOLS 10: Image pick-up device 101: Image pick-up element 102: Image pick-up element setting part 103: Serial-parallel conversion part 104: Compositing process part 105: Defect pixel interpolation part 106: Image output part 108: Frame rate switching part 141 : Line memory write ch selector, 143: Line memory, 145: TAP selector.

Claims (1)

  An imaging device having an imaging region divided into a plurality of regions and an output channel for outputting a video signal for each of the plurality of imaging regions, and switching the number and output order of output channels for outputting the video signals from the imaging device An image sensor setting unit; a synthesis processing unit that synthesizes the plurality of video signals output from the image sensor; and an image output unit that converts the synthesized signal into a camera link output format. An image pickup apparatus, wherein the frame rate is switched by a combination of switching of the number of channels and a combining method of the combining processing unit.

Images