JP2010283588A - High-definition video signal processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a high-definition image signal processor for sharing correction processing by FPGAs even for changing an image processing system. <P>SOLUTION: High-speed cross point switches 8 and 16 are provided at input sides and output sides of signal processing circuits 10 to 15 each including FPGA1 to FPGAn. Connection at the input sides and the output sides of the signal processing circuits 10 to 15 to be needed for switching an image processing system of a processing unit 7 as a whole is changed by the selection of cross points of the cross point switches 8 and 16. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a processing device for a video signal output from an image sensor, and in particular, video signal processing in a system that obtains a high-definition image using an image sensor having a plurality of outputs or an image sensor by combining a plurality of outputs. Relates to the device.

Conventionally, a television camera or the like having a pixel number exceeding the number of pixels of HDTV (High Definition Television) has been put into practical use (see, for example, Patent Documents 1 to 5).
In such a camera or the like, there is a limit to the readout speed of the video signal from the imaging device and the speed of signal processing after readout, so that it is difficult to use an imaging device having a plurality of outputs or a plurality of pasted imaging devices. A fine image is obtained or a video signal is divided into a plurality of pieces for parallel processing.
In other words, the pixel region of the image sensor is divided into a plurality of areas at substantially constant intervals, the plurality of areas are driven at substantially the same timing, a parallel readout operation is performed, and the read video signals are processed in a parallel state. .

FIG. 10 is a block diagram considered as an example of signal processing corresponding to a high-definition image. Various correction processes such as white / black correction, aberration correction, shading correction, knee correction, γ correction, and detail (DTL) correction are performed on each output signal of the parallel output image sensor that is vertically divided into regions.
Since the output signal of the image sensor is divided for each divided region and each RGB color channel, if the correction amount becomes discontinuous at the boundary of the region, degradation that can be recognized visually occurs in the image. Then, when performing aberration correction and shading correction, the signal of the adjacent area is also input, which is called margin processing.
When handling ultra-high-definition video called 4K2K or 8K4K, the amount of processing for this purpose is enormous, so it is often composed of a plurality of substrates connected to each other by high-speed serial transmission. During the system development process, the system developer needs to consider the method of signal division and the order of various processes in order to suppress the increase in hardware scale and overhead due to parallel processing. However, it is very laborious to raise the circuit board for each method.

In recent years, programmable LSI devices called FPGAs (Field Programmable Gate Arrays) have come to be provided on the market. According to this FPGA, the necessary logic information is stored in the built-in SRAM type memory cell. The circuit can be operated as designed simply by loading the wiring information.
In other words, if a board on which a plurality of FPGAs are mounted as necessary is prepared, it is easy and quick to prototype, verify, and evaluate a system without causing a large number of boards for individual circuits. This can greatly contribute to the performance improvement and cost reduction of the high-definition video signal processing apparatus.

JP 2005-130331 A JP 2003-143562 A JP 2000-312311 A JP 2005-269163 A JP 2005-333526 A

By the way, in high-definition video signal processing, consideration must be given to the signal format (how to divide by area and color, etc.) and the image processing method as a whole. It is not easy to share correction processing simply by using it.
The image processing method as a whole can be broadly divided into single-color image processing (a method for processing each single-color image as it is, which requires a small number of area divisions) and RGB image processing (a method for collectively processing RGB images). Correction is easy) or dedicated image processing (image processing requested by a user or the like).

In order to enable connection with a circuit board of a different image processing system, it is necessary to rebind all the parallel signals of the image sensor input to the circuit board or the parallel signals from the previous circuit board.
For this re-bundling, it is necessary for each FPGA to prepare connections for all the circuit boards in the previous stage in preparation for all connection combinations, but at the present time, it has such a large number of high-speed serial signal interfaces. The device is not available.
For this reason, it is necessary to switch the wiring not in the signal processing circuit but outside, and even if the signal processing circuit is an FPGA, it cannot be handled by loading the logic information and the wiring information.

Therefore, in the prior art, for changing the image processing method, it is necessary to generate separate circuit boards for the monochrome image processing unit, the RGB image processing unit, and the dedicated image processing unit, respectively. Problems arise in the sharing of processing.
An object of the present invention is to provide a high-definition video signal processing apparatus capable of sharing correction processing by an FPGA even when changing an image processing method.

  The above object is to provide a high-definition video signal processing apparatus for processing a video signal output from an image sensor with a signal processing circuit made of FPGA, and to provide cross point switches on the input side and the output side of the signal processing circuit, respectively. This is achieved by configuring so that the connection change between the input side and the output side of the signal processing circuit necessary for switching the image processing system as a whole unit can be obtained by selecting the cross point of the cross point switch.

  At this time, selection of the cross point of the cross point switch may be given by loading logic information into the FPGA of the signal processing circuit.

  According to the present invention, since the switching of the image processing system as a whole unit can be obtained simply by selecting the cross point of the cross point switch, it is not necessary to raise another circuit board even if the image processing system is different. In this development process, it is possible to easily change the image processing method and greatly contribute to the improvement of system reliability by diversifying test items.

It is a block diagram of a processing unit of a high-definition video signal processing apparatus according to the present invention. It is a block diagram of a processing unit of a high-definition video signal processing apparatus according to the present invention. It is explanatory drawing which shows an example of the crosspoint switch in one embodiment of this invention. It is the 1st explanatory view showing other examples of the crosspoint switch in one embodiment of the present invention. It is the 2nd explanatory view showing other examples of the crosspoint switch in one embodiment of the present invention. It is 3rd explanatory drawing which shows another example of the crosspoint switch in one embodiment of this invention. It is a 4th explanatory view showing other examples of a crosspoint switch in an embodiment of the invention. It is a 5th explanatory view showing other examples of a crosspoint switch in an embodiment of the invention. It is 6th explanatory drawing which shows another example of the crosspoint switch in one embodiment of this invention. It is a block diagram which shows an example of the signal processing corresponding to a high definition image. It is a block diagram of the high-definition video signal processing apparatus which concerns on other embodiment of this invention.

Hereinafter, a high-definition video signal processing apparatus according to the present invention will be described in detail by embodiments.
First, FIG. 1 is a block diagram of a processing unit 7 according to an embodiment of the present invention, and also schematically shows an operation when monochromatic image processing is performed.
The processing unit 7 of this example has, as main components, a high-speed crosspoint switch 8 on the input side, six signal processing circuits 10 to 15 and a high-speed crosspoint switch 16 on the output side mounted on a single board. It is a thing.
The R (red) sensor 1, the B (blue) sensor 2, and the G (green) sensor 3 are, for example, sensors of parallel output image sensors (so-called three-plate camera) as shown in FIG. In each sensor, the imaging area is divided into eight parts, which are output by eight high-speed serial interfaces (N = 8) such as LVDS.

Each of the eight high-speed serial video signals output from the high-speed serial interface is obtained by serializing pixel values (for example, 14 bits) in the read divided area in a predetermined format. Let's call it 8 channels.
Eight high-speed serial video signals output from each of the R sensor 1, B sensor 2, and G sensor 3 are taken into the processing unit 7 via the respective input side connectors 4, 5, 6 and input. Is supplied to the high-speed crosspoint switch 8 having the N × M (N = 8) configuration.
At this time, the high-speed crosspoint switch 8 is a switch having a 3N × 6M (M = 6) configuration, and is crossed by control signals from the respective FPGAs of the signal processing circuits 10 to 15 including FPGA1 to FPGAAn (n = 6). Point selection is controlled.

That is, the high-speed crosspoint switch 8 can arbitrarily connect all of the high-speed serial video signals (24 channels) constituting one screen to 36 output signals, and rebinds them every 6 channels. The high-speed serial video signal thus input is input to signal processing circuits 10 to 15 including FPGA1 to FPGAAn. However, in this example, only 5 channels are used out of the output signals for every 6 channels, so in FIG.
At this time, each of the FPGA1 to FPGAAn generates a control signal to be given to the high-speed crosspoint switch 8 in addition to the logical information and wiring information necessary for signal processing as the monochrome image processing unit required at this time. Information is loaded.

Here, first, 5 channels from 1 channel to 5 channels of the R channel video signal are input to the signal processing circuit 10, and 4 channels to 5 channels of the R channel video signal are input to the signal processing circuit 11. Up to 5 channels are input. Various correction processes are performed on the R video signal by both the FPGA 1 of the signal processing circuit 10 and the FPGA 2 of the signal processing circuit 11.
Next, 5 channels from 1 channel to 5 channels of the G channel video signal are input to the signal processing circuit 12, and 1 channel to 5 channels of the G channel video signal are input to the signal processing circuit 13. 5 channels are input. Various correction processes for the G video signal are performed by both the FPGA 3 of the signal processing circuit 12 and the FPGA 4 of the signal processing circuit 13.

The signal processing circuit 14 receives 5 channels from the 1st channel to the 5th channel of the B channel video signal, and the signal processing circuit 15 receives 5 signals from the 1st channel to the 5th channel of the B channel video signal. Channels are input, and various correction processes for the B video signal are performed by both the FPGA 5 of the signal processing circuit 14 and the FPGA An of the signal processing circuit 15.
For each color, 5 channels are input redundantly and used as a margin, so processing (shading correction or the like) that crosses between the region divisions becomes possible.
In this way, the signal processing circuits 10 to 15 perform correction processing separately (independently) for each of the RGB color video signals, and as a result, five signal processing circuits 10 to 15 each. Thus, a video signal corrected by the high-speed serial I / F is output.

The corrected video signals output from the signal processing circuits 10 to 15 are supplied to the high-speed crosspoint switch 16 on the output side.
Here, this high-speed crosspoint switch 16 is a switch of 6M × 3N configuration, and, like the high-speed crosspoint switch 8 on the input side, this is also a crosspoint according to the control signal given from each FPGA of the signal processing circuits 10-15. Selection is controlled. In this example, since the signal is returned to the same type as that on the input side, the connection state in the high-speed crosspoint switch 16 on the output side is exactly the reverse of that on the high-speed crosspoint switch 8 on the input side.

That is, the corrected R channel video signals of 5 channels each output from the signal processing circuit 10 and the signal processing circuit 11 are converted into 8-channel R channel video signals by the high speed serial interface I / F by the high speed crosspoint switch 16. It is selected and taken out to the output side connector 17.
Next, the 5-channel corrected G-channel video signals output from the signal processing circuit 12 and the signal processing circuit 13 are converted into 8-channel G-channel video signals by the high-speed serial interface I / F as high-speed crosspoint switches 16. And is taken out to the output side connector 18.

The five-channel corrected B-channel video signals output from the signal processing circuit 14 and the signal processing circuit 15 are converted into 8-channel B-channel video signals by the high-speed serial interface I / F by the high-speed crosspoint switch 16. It is selected and taken out to the output side connector 19.
Therefore, as shown in FIG. 1, the processing unit 7 can function as a monochromatic image processing unit by selecting the cross point of the high speed cross point switch 8 and the high speed cross point switch 16.

Next, FIG. 2 shows an example in which the processing unit 7 according to the embodiment of the present invention is switched to the RGB image processing unit. In this case, the crossing by the high-speed crosspoint switch 8 and the high-speed crosspoint switch 16 is performed. The configuration of the processing unit 7 itself is the same as in FIG. 1 except that the point selection position is different from that in FIG. 1, and other configurations are the same as those in FIG.
Then, the RGB 8-channel video signals are selected as 4 sets of 6-channel video signals under the control of the high-speed crosspoint switch 8 and input to the signal processing circuits 10 to 13 including FPGA 1 to FPGA 4. .

At this time, the FPGA 1 to FPGA 4 are loaded with necessary logic information and wiring information corresponding to the signal processing as the RGB image processing unit required at this time.
Therefore, first, the signal processing circuit 10 is inputted with two channels of one channel and two channels of video signals of R channel, G channel, and B channel. Then, various correction processes necessary for the RGB image are performed by the FPGA 1 of the signal processing circuit 10 for two channels of the R video signal, the G video signal, and the B video signal.
Next, the signal processing circuit 11 receives 2 channels of 3 channels and 4 channels of video signals of R channel, G channel, and B channel.

Then, various correction processes necessary for the RGB image are performed by the FPGA 2 of the signal processing circuit 11 for 2 channels of 3 channels and 4 channels of the R video signal, the G video signal, and the B video signal.
Further, the signal processing circuit 12 is inputted with 2 channels of 5 channels and 6 channels of video signals of R channel, G channel, and B channel.
Then, various correction processes necessary for the RGB image are performed by the FPGA 3 of the signal processing circuit 12 on the 5 channels and 6 channels of the R video signal, the G video signal, and the B video signal.

Further, the signal processing circuit 13 receives two channels of 7 channels and 8 channels of video signals of R channel, G channel, and B channel. Then, various correction processes necessary for the RGB image are performed by the FPGA 4 of the signal processing circuit 13 on the two channels of the R video signal, the G video signal, and the B video signal for two channels.
Therefore, various correction processes necessary for RGB image processing are performed on all channels by the signal processing circuits 10 to 13. As a result, 6 channel correction processes are performed from the signal processing circuits 10 to 13, respectively. A finished video signal is output.

The corrected video signal output from each of the signal processing circuits 10 to 13 is supplied to the high-speed crosspoint switch 16 on the output side.
Then, the corrected RGB video signals for 2 channels each output from these signal processing circuits 10 to 13 are selected by the high-speed crosspoint switch 16 as 8-channel RGB video signals by the high-speed serial I / F, and R The video signal is supplied to the output side connector 17, the G video signal is supplied to the output side connector 18, and the B video signal is supplied to the output side connector 19.

  Therefore, as shown in FIG. 2, the processing unit 7 can function as an RGB image processing unit by selecting the cross point of the high speed cross point switch 8 and the high speed cross point switch 16, such as color masking. Color processing can be performed in parallel efficiently. Further, processing that does not require continuity between divided areas, such as fixed pattern correction, is also possible.

  At this time, the monochromatic image processing unit of FIG. 1 and the RGB image processing unit of FIG. 2 are both obtained by switching the same processing unit 7, and this switching is performed by loading logical information to the FPGA. No need to wake up another circuit board.

Here, the high-speed crosspoint switches 8 and 16 in the above embodiment will be described more specifically.
First, FIG. 3 shows a crosspoint selection state when a crosspoint switch based on, for example, the product name AD8152 of Analog Devices Inc. of the United States is used as the high-speed crosspoint switch 8 on the input side. The upper side of FIG. 4 represents the signal at the input terminal of the high-speed crosspoint switch 8, and the right side represents the signal at the output terminal (connected to the FPGA of each signal processing circuit).
The X symbol at the intersection of the signal lines indicates a portion that needs to be connected in either monochromatic image processing or RGB image processing, and the fact that the R1-R2 channels and FPGA1-I1-I2 are fixed, Even if the number of signal lines is 36, it can be realized by a 34 × 34 crosspoint switch.

  Next, FIG. 4 to FIG. 6 use six 8 × 8 crosspoint switches FS1 to FS6 and six 8 × 8 crosspoint switches SS1 to SS6, and crosspoint switches FS1 to FS6 and crosspoints. FIG. 2 shows a selection state of cross points when switches SS1 to SS6 are vertically connected to form a high-speed cross point switch 8 on the input side in the monochromatic processing unit of FIG. 1, where an input terminal and an output terminal are shown here for convenience. The cross points are selected by connecting them with a line to represent the selected state.

  FIGS. 7 to 9 show that six 8 × 8 crosspoint switches FS1 to FS6 and six 8 × 8 crosspoint switches SS1 to SS6 are used, and the crosspoint switches FS1 to FS6 and the crosspoint switches are used. FIG. 3 shows a cross point selection state when SS1 to SS6 are vertically connected to form a high-speed cross point switch 8 on the input side in the RGB image processing unit of FIG.

FIG. 11 is a configuration diagram of a high-definition video signal processing apparatus according to another embodiment of the present invention.
In FIG. 11, a frame with rounded corners corresponds to the processing unit 7 described above, and a high-definition video signal processing apparatus is configured using 30 processing units.
Also in this example, a three-plate camera is assumed, and color separation interpolation and the like associated with the Bayer array are not necessary, and therefore, monochromatic image processing is fundamental. However, most processing units are different from the previous embodiment in principle in that only one color is processed.

  Specifically, a unit that corrects FPN (Fixed Pattern Noise) and white defects that occur for each pixel of the image sensor, and flare (a phenomenon in which strong incident light is scattered without an imaging system and becomes cloudy without image formation) ) And shading (unevenness of sensitivity), digital gain processing, test pattern switching processing, registration color aberration correction unit that corrects image misregistration between color channels (mainly due to chromatic aberration), and lens Aperture correction unsharp unit that performs high-frequency emphasis (sharpening) according to the resolution of the image, and R, G, B, C (cyan), M (magenta), and Y (yellow) separately for each of the six colors 6-color independent color masking unit with adjustable saturation and hue, detail correction mainly for contour enhancement, knee correction mainly improving high-tone gradation reproducibility, and mainly halftone brightness To adjust And a unit for performing gamma correction, are realized correction process is serially connected (cascaded).

  Color masking, which requires computation between color channels, uses 4 units and inputs and processes all RGB channels for each of the 4 divided areas. Upstream of these, 3 units are arranged in parallel for each RGB. Provide and process. Further, since it is desirable to perform gamma processing (tone curve image correction) having strong nonlinearity last, it is performed after color masking processing.

1 R sensor (red sensor part of image sensor)
2 G sensor (blue sensor part of image sensor)
3 B sensor (green sensor part of image sensor)
4-6 Input side connector 7 Processing unit 8 Input side high-speed crosspoint switch 10-15 Using signal processing circuits (FPGA1-FPGAn (n = 6))
(Signal processing circuit)
16 High-speed crosspoint switch on output side 17-19 Connector on output side

Claims (2)

In a high-definition image signal processing apparatus of a method for processing an image signal output from an image sensor with a signal processing circuit made of FPGA
A crosspoint switch is provided on each of the input side and output side of the signal processing circuit,
A high-definition configuration in which the connection change between the input side and the output side of the signal processing circuit necessary for switching the image processing system as a whole unit can be obtained by selecting a cross point of the cross point switch. Image signal processing device. The high-definition image signal processing device according to claim 1,
The high-definition image signal processing apparatus is configured so that selection of a crosspoint of the crosspoint switch is given by loading logical information to an FPGA of the signal processing circuit.

Images