CN2698267Y - Real-time system for correcting standard video image distortion of medical electronic endoscope - Google Patents

Abstract

The utility model relates to a real-time system for correcting standard video image distortion of medical electronic endoscope. The composite video signal which is output from the camera CCD is converted to a digital image by a decoder, and then, stored separately to the corresponding incoming frame memory grouped by odd field and even field. At the same time, the image of the even field or the odd field is adjusted, the grey scale information of the image of the even field or the odd field is read from the even field or the odd field input frame memory, and then written into the even field or the odd field output frame memory. Besides, the adjusted image of the last field is read from the even field or the odd field output frame memory, and then sent to the code device which encodes and outputs the adjusted video signal. The utility model realizes a real-time performance, and adopts a PAL / NTSC standard. The field frequency is 50 / 60 Hz, and the output video signal can enter to a monitor to display directly. The time-delay of the image from being input to being output is the time of two fields, which is equivalent to 40 / 33 ms. Thus, when observing the adjusted image, the existence of time-delay can not be felt by human eyes, and the design requirement of real-time performance is reached. The sampling precision of the image is high, the distortion correction to the color image is realized, and the cost of the system is reduced greatly by adopting alternate-line scanning.

Description

Medical electronic endoscope normal video pattern distortion System with Real-Time

[technical field]: this utility model relates to a kind of Medical Instruments, particularly the medical electronic endoscope image is proofreaied and correct the system and device of processing.

[background technology]: in order to improve the range of observation of medical electronic endoscope, endoscope needs bigger visual field (being generally 120 °), and the diameter of endoscope head is subjected to the restriction of human body body cavity, the overall dimensions of imaging system can not be very big, its imaging optical system also can not be very complicated, therefore the more serious optical distortion of the general existence of endoscope's optical imaging system influences the doctor and correctly judges diseased region.

Reducing pattern distortion by complicated lens group is a kind of ideal way, but because the self structure of medical electronic endoscope and the restriction of usage space, there are very big difficulty in the design of optical texture and manufacturing process, even may not realize.For example Fu Za optical system all is made up of a plurality of lens, and each lens all can be introduced two reflectings surface, and a part of imaging luminous energy is lost because of reflection, in order to improve its transmitance, just need carry out processing such as surface coating.This has improved the manufacturing cost of optical system undoubtedly.

In medical electronic endoscope, object image-forming is exported after being converted into video electrical signal on the photosurface of CCD, gives standard display device, shows as monitor etc., also can enter image capture device (as image pick-up card) and carry out digitized processing.Therefore, utilizing computer to finish the geometric distortion correction algorithm is easy to realize.However, because the restriction of the operational speed of a computer, software can only be realized the distortion correction to still image, can not carry out distored real-time correction to dynamic image.

In order to solve endoscopic images Real-Time Distortion calibration problem, following solution is disclosed in the prior art:

1, Asari, people such as V.K. adopt the minimum area estimation technique that distortion is proofreaied and correct, and the corrected image size is 256 * 192.Asari, V.K. wait the people to adopt monolithic VLSI (ultra-large integrated chip) to realize the function of whole system, reached very high integrated level, but because the disposal ability of monolithic VLSI is limited, make that distortion correction system image resolution ratio is too low, do not solve the distored real-time correction problem of high-definition picture.Referring to: [Non-linear spatial warping of endoscopic images:an architecturalperspective for real time applications/Asari, V.K. // Microprocessors andMicrosystems,-2002,26 (4) .-161～171]

2, Olympus Optical Co., Ltd. (Chiba; Masahiro) 3-D visual endoscope system adopts real time imaging distortion correction technology, but its image source is the image of lining by line scan; The video image of lining by line scan is handled in real time, and the data volume that will read and write and handle in the unit interval is the twice of the interleaving video of equal picture quality.So under the occasion of high-resolution normal video, this system is all very high to the capacity and the read or write speed of the disposal ability of processor, memorizer, whole system cost costliness, bulky.Referring to: [Stereoscopic-vision endoscope system provided with function ofelectrically correcting distortion of image or the like with respect toleft-and right-hand image signals having parallax, independently of eachother US5,860,912/0lympus Optical Co., Ltd. (Chiba; Masahiro) .-1999.01.19]

3, Interactive Pictures Corporation (Martin; H.Lee) the method about scanogram adopts nonlinear scanning to obtain the analog image pixel, can't carry out interpolation processing to image, and picture quality is lower.Referring to: [Method for directly scanning a rectilinear imaging element usinga non-linear scan US6,243,131/Interactive Pictures Corporation (Martin; H.Lee) .-2001.06.05]

4, Ngo HT, people such as Asari VK adopt VLSI that the image of the 8-bit 2056 * 2056 of wide-angle imaging machine output is proofreaied and correct in real time.Be subjected to the restriction of pixel data width, its image detail is not obvious, and is lower as value.Referring to: [A fully pipelined architecture for barrel-distortion correctionbased on back mapping and linear interpolation, International Conferenceon Embedded Systems and Applications/Ngo HT, Asari VK.-2003-268-274]

[summary of the invention]: the purpose of this utility model is to solve the above-mentioned problems in the prior art, providing a kind of can carry out the distortion in real time correction to high-resolution endoscope color dynamic image, and can reduce cost and reduce the medical electronic endoscope normal video pattern distortion System with Real-Time of system dimension.

This utility model medical electronic endoscope normal video pattern distortion System with Real-Time, comprise Video Decoder, memorizer, video processor and video encoder are formed, wherein, memorizer comprises strange and an idol input frame memory, the corrected data memory of strange field and an idol output frame memorizer and standard jig fault image, from the composite video signal of CCD photographic head output after Video Decoder is converted to digital picture, being divided into strange and even stores into respectively in the corresponding input frame memory, promptly when decoder is exported strange or even field picture, it is write in the strange or even input frame memory; In the strange or even field picture of storage, proofread and correct occasionally strange field picture, it is video processor FPGA reads occasionally strange field picture from the corrected data memory of standard jig fault image correction address, and from an occasionally strange input frame memory, read the half-tone information of occasionally strange field picture according to this address, be written to an occasionally strange output frame memorizer; When proofreading and correct occasionally strange field picture, from strange or an idol output frame memorizer, read last one correcting image, send into encoder, coding output calibration video signal.

Advantage of the present utility model and good effect:

(1) realized real-time.Adopt the PAL/NTSC standard, field frequency 50/60Hz, outputting video signal can directly send into monitor and show.Image is two field times from the time delays that is input to output, is equivalent to 40/33ms, and therefore, when observing correcting image, human eye is the existence of imperceptible time-delay at all, has reached the designing requirement of real-time.

(2) image sampling precision height.Sampling, correction and coding to fault image are all undertaken by 702 * 576 pixels (PAL-system) or the individual point of 795 * 596 (TSC-systems) size, and the precision of outputting video signal can satisfy the resolution requirement of general monitor fully, and correcting image is clear.

(3) realized distortion correction to coloured image.Because trimming process is carried out under the YCbCr color space, to two 1 groups of corrections of colour difference signal, does not change the colouring information of input skew image, and does not have mosaic on the output image, does not have isolated bright spot or stain.

(4) adopt interlacing scan normal video image, the correction parameter data volume is half of lining by line scan, and greatly reduces system cost.

[description of drawings]:

Fig. 1 is this novel dot matrix template drawing;

Fig. 2 is that this novel dot matrix model optical center is determined method figure;

Fig. 3 is this novel real-time correction principle sketch map;

Fig. 4 is this novel Real-Time Distortion corrective system catenation principle figure in actual use;

Fig. 5 is this novel corrective system circuit structure theory diagram;

Fig. 6 is that driving is connected and the sketch map of working this novel SRAM with line;

Fig. 7 is this novel correcting circuit block diagram;

Fig. 8 is the gastroscope fault image;

Fig. 9 is the image after Fig. 8 proofreaies and correct.

[specific embodiment]:

Embodiment 1:

Operation principle: this novel employing standard jig standardizition distortion correction principle, utilize special standard jig exactly, make it pass through optical system imaging to be corrected, because optical system itself exists distortion, thereby picture is deformed,, calculate the ideal undistorted picture of model according to the amplification of model ideal design parameter and optical system, relatively the difference of it and actual picture obtains the functional relationship between them.Gauged process is exactly to utilize this functional relationship to finish the conversion that distored reality looks like distortionless desirable picture.

By hexagonal array, every bit all is identical with the distance of its six adjoint points to this novel dot matrix model, as shown in Figure 1 by some round dots.Simultaneously round dot is arranged in a straight line on three directions (0 °, 60 °, 120 °), and the point on every straight line, and its spacing is identical.Utilize the dot matrix of this arrangement, can determine the optical center of optical system by template image easily.For distored optical system, the straight line in the object space no longer is a straight line generally in the image space, and has only the straight line by optical center to make an exception.According to this character, on the fault image of dot matrix model, have only rectilinear range of points still to be arranged as straight line, and all the other straight lines that depart from optical center will have bending in various degree by optical center.Because the dot matrix model is (0 ° of three specific direction, 60 °, 120 °) on the some arrangement that is in line, on the model fault image, find three point ranges that approach linear array most, with three corresponding point ranges of fitting a straight line, these three straight-line intersections just can be similar to the position of thinking the system optics center.As shown in Figure 2.

On desirable template image, select some pixels, and find the position of their correspondences on actual template image.Pixel selected on ideal image is gathered, and is called the sampling point set.The set of corresponding pixel on real image is called the corresponding point set.Utilize the dot matrix model, can determine the coordinate of pixel in these two set.

On fault image and template image, select corresponding point, the functional relationship between match fault image and template image.Like this, corresponding point set also can be determined, thereby also just obtain the one group corresponding relation of required fault image to ideal image.

Gray correction.Select bilinear interpolation method.Because the medical electronic endoscope output image is the coloured image of rgb format, it is transformed under the YCbCr carries out, gray correction does not change colour difference signal, only to the luminance signal interpolation.

At first, simulate distortion function with software, obtain the positional alignment one-tenth table of each some corresponding point in fault image on the image to be corrected, and convert thereof into the file layout of being convenient in circuit, use, be cured among the hardware memory EPROM as hardware corrected look-up table; Then, hardware circuit obtains the position at the fault image memory space to be corrected according to the correction data of reading from look-up table, again the gray scale of this position is write memory location corresponding on the correcting image and has just finished correction.

So-called correction in real time will be finished exactly to the correction every the input skew image, and the image after proofreading and correct is pressed input sequence export, in other words, be exactly " simultaneously " imported continuously at fault image, must guarantee that correcting image exports continuously, the phenomenon of interruption can not be arranged.The method that realizes is to allow the relative input delay 2 field times output of correcting image, i.e. during the 3rd field distortion image input, the 2nd field picture is proofreaied and correct, and meanwhile exports the 1st correcting image, and three field picture parallel processings have so just realized seriality output.Real-time gauged sketch map is seen shown in Figure 3.

Particular circuit configurations

Circuit structure theory diagram as shown in Figure 5, from the composite video signal of CCD photographic head output after Video Decoder is converted to digital picture, being divided into strange and even stores into respectively in the corresponding input frame memory, promptly when decoder is exported strange or even field picture, it is write in the strange or even input frame memory; In the strange or even field picture of storage, proofread and correct occasionally strange field picture, it is video processor FPGA reads occasionally strange field picture from the corrected data memory (being look-up table) of standard jig fault image correction address, and from an occasionally strange input frame memory, read the half-tone information of occasionally strange field picture according to this address, be written to an occasionally strange output frame memorizer; When proofreading and correct occasionally strange field picture, from strange or an idol output frame memorizer, read last one correcting image, send into encoder, coding output calibration video signal.

The crystal oscillator of this novel employing 24.6MHz is as being input in the middle of the decoder the inner sample frequency 13.5MHz of decoder, the clock of output 13.5MHz, two frequencies of 27MHz; The encoder input clock is the 13.5MHz clock of decoder output, realizes that coding and decoding is synchronous.

Concrete correcting circuit is seen Fig. 7, behind the composite video signal input video decoder of CCD photographic head output, divide four the tunnel to import the line driving X1 that control data writes respectively successively, X3, X5, X7, and then respectively successively through input frame memory SRAM1, SRAM2, SRAM3, behind the SRAM4, line through reading drives X2 again, X4, X6, common input video processor FPGA1 behind the X8, FPGA2, this video processor is deposited the corrected data memory L1～L8 of standard jig fault image by the two-way connection of data wire, and the image information after will proofreading and correct divides four the tunnel to import the line driving X10 that control data writes respectively successively, X12, X14, X16, and then respectively successively through input frame memory SRAM5, SRAM6, SRAM7, behind the SRAM8, line through reading drives X9 again, X11, X13, the common input video encoder of X15, output image shows after D/A switch is composite video signal.

Shown in solid arrow among the figure, viewdata signal passes through in circuit by the direction of input, processing, output; Long dotted line is represented address signal among the figure; Pecked line is represented configuration signal; FPGA sees Fig. 6 to the control signal of SRAM and line driving.EPG1 and EPC2 are respectively the specialized configuration memorizeies of two FPGA, and this memorizer can be provided by FPGA manufacturer.L1～L8 is a lookup table memories of depositing correction data.

Concrete work process:

---the encoding and decoding of normal video image:

The handled dynamic image of native system is the television system of standard: NTSC (National TelevisionSystem Committee) system and PAL (Phase Alternation Line) system.The size of correcting image is 702 * 576 pixels (PAL-system) or 795 * 596 (TSC-systems).Two kinds of video formats are interlace mode.

Video Decoder adopt SAA7111, its function be will input composite video signal through mould/number conversion output digital image, be about to analog image and be converted to digital picture, be convenient to the inter-process of circuit.In hardware circuit, make encoder be operated under the PAL/NTSC standard, interlacing scan, field frequency 50/60Hz, the data format of output image are 16-bit 4:2:2 YUV (CCIR-601), wherein Y and UV are 8bit.For the ease of Flame Image Process, decoder output image size is identical with encoder input picture size, stipulates every field picture sampling 702 * 576 pixels (PAL-system) or the individual point of 795 * 596 (TSC-systems).

Video encoder adopts BT864, its function is that a digital picture of input is composite video signal output through D/A switch, in hardware circuit, encoder also is operated under the PAL/NTSC standard, the input data format is 16-bit 4:2:2 YCbCr, an effective brightness resolution 702 of row and an effective resolution 576 (PAL-system), an effective brightness resolution 795 of row and an effective resolution 596 (TSC-system).

---frame memory and line drive:

Selecting memory capacity for use is that the SRAM of 256K * 8bit does frame memory.Because the data of every field picture are divided into 8bit luminance signal and 8bit colour difference signal again, so storage input and output image altogether need be with 8 sram chips, it is as follows that it stores data allocations:

SRAM1 (S1): the colour difference signal UV7～UV0 of the strange field picture of storage input;

SRAM2 (S2): the colour difference signal UV7～UV0 of the even field picture of storage input;

SRAM3 (S3): the brightness signal Y 7～Y0 of the strange field picture of storage input;

SRAM4 (S4): the brightness signal Y 7～Y0 of the even field picture of storage input;

SRAM5 (S5): the brightness signal Y c7～Yc0 of the even field picture of storage output;

SRAM6 (S6): the brightness signal Y c7～Yc0 of the strange field picture of storage output;

SRAM7 (S7): the colour difference signal CbCr7～CbCr0 of the even field picture of storage output;

SRAM8 (S8): the colour difference signal CbCr7～CbCr0 of the strange field picture of storage output.

The data isolation device adopts 8 bit bufferings and line to drive, and the I/O data are 8bit, input.The principle of line drive control data output is seen shown in Figure 6: when writing idol field SRAM, make X3OE=0, make YUV drive 3 entry-line a by line, with seasonal X4OE=1, make YUV not drive d by line, do not have data to pass through among the circuit d; When reading strange SRAM, make X1OE=1, make YUV not drive 1 by line, do not have even field data among the circuit b, with seasonal X2OE=0, make the data of reading from strange SRAM drive 2 by line, via line c enters FGPA.Because when the enable signal that line drives was high level, outfan was a high-impedance state, can not import data on other data wire that be connected with outfan this moment, draw resistance so also want the outfan of online driving to connect.Because every SRAM needs 2 lines to drive the flow direction of control data, need 16 lines to drive in the entire circuit altogether.

Enable signal X1OE, X2OE, X3OE and X4OE that line drives are controlled by the logic of FPGA, and YUV is the video standard signal of the YUN form of decoder output.The present invention uses 8 SRAM and 16 lines to drive.The line that is connected with SRAM drives the title such as the following table of sequence number and associated enable signal.

	The line that control data writes drives title	Write enable signal	The line that control data is read drives title	Read enable signal
					SRAM1	X1	X1OE	X2	X2OE
SRAM2	X3	X3OE	X4	X4OE
					SRAM3	X5	X1OE	X6	X2OE
SRAM4	X7	X3OE	X8	X4OE
					SRAM5	X10	X10OE	X9	X9OE
SRAM6	X12	X12OE	X11	X11OE
					SRAM7	X14	X10OE	X13	X9OE
SRAM8	X16	X12OE	X15	X11OE

For strange or even field picture, its brightness signal Y and colour difference signal UV are I/O simultaneously, SRAM1 and SRAM3 store the colour difference signal and the luminance signal of strange field picture respectively, so the line driving X1 of SRAM1 and SRAM3 is write in control and the enable signal of X5 is identical, making it all is X1OE; The line driving X2 of SRAM1 and SRAM3 is read in control in like manner and the enable signal of X6 also is identical, and making it is X2OE.By table 4-1 as can be known, for the input and output data flow of control frame memorizer, need X1OE, X2OE, X3OE, X4OE, X9OE, X10OE, X11OE and X12OE totally 8 enable signals.

---video processor FPGA:

FPGA (field programmable gate array) mainly finishes the work of six aspects as the video processor of system:

1, the I of decoder and encoder ²The C initialization of register is provided with, as shown in Figure 7;

2, the Synchronization Control of encoder;

3, the generation of the addressing of frame memory SRAM and control signal;

4, the control logic of line driving;

5, the conversion of YUV and YCbCr;

6, the generation of the addressing of look-up table EPROM and control signal.

---look-up table:

So-called look-up table is exactly that its content is come by dot matrix model algorithm computation by software the table that positional alignment became of each some corresponding point on fault image on the correcting image.In hardware circuit, look-up table stores be to be corrected corresponding to the storage fault image frame memory addresses.Look-up table is realized with EPROM on hardware, because the correction address of storage brightness signal Y and the correction address of colour difference signal UV all are 18bit, needs 8 EPROM altogether in circuit.

Strange field picture and even field picture can separately be proofreaied and correct, and all use the data of proofreading and correct strange field picture.

The whole software program is stored in the middle of two special-purpose EPROM on the circuit board usually, starts working in the middle of can being written into FPGA rapidly after powering on; In system works, can carry out online adjustment and change to software program and do not influence the operate as normal of system.

As shown in Figure 4, this corrective system practical application is in the signal of the position of endoscope's prototyping system.Object 1 is through endoscope optical system 2 imagings, gather by CCD3, the fault image of output is proofreaied and correct in the middle of sending into correcting circuit 5 of the present invention with the form of video signal 4, correcting image is exported with the form of video signal 6 equally, and send into monitor 7 and show in real time, or send into image pick-up card and be stored in the computer by the A/D conversion.

Fig. 8,9 has provided the image sketch map after gastroscope fault image and the correction respectively.

Claims (2)

1, a kind of medical electronic endoscope normal video pattern distortion System with Real-Time, comprise Video Decoder, memorizer, video processor and video encoder are formed, it is characterized in that described memorizer comprises strange and an idol input frame memory, the corrected data memory of strange field and an idol output frame memorizer and standard jig fault image, from the composite video signal of CCD photographic head output after Video Decoder is converted to digital picture, being divided into strange and even stores into respectively in the corresponding input frame memory, promptly when decoder is exported strange or even field picture, it is write in the strange or even input frame memory; In the strange or even field picture of storage, proofread and correct occasionally strange field picture, it is video processor FPGA reads occasionally strange field picture from the corrected data memory of standard jig fault image correction address, and from an occasionally strange input frame memory, read the half-tone information of occasionally strange field picture according to this address, be written to an occasionally strange output frame memorizer; When proofreading and correct occasionally strange field picture, from strange or an idol output frame memorizer, read last one correcting image, send into encoder, coding output calibration video signal.

2, medical electronic endoscope normal video pattern distortion System with Real-Time according to claim 1, it is characterized in that behind the composite video signal input video decoder of CCD photographic head output, divide four the tunnel to import the line driving X1 that control data writes respectively successively, X3, X5, X7, and then respectively successively through input frame memory SRAM1, SRAM2, SRAM3, behind the SRAM4, line through reading drives X2 again, X4, X6, common input video processor FPGA1 behind the X8, FPGA2, this video processor is deposited the corrected data memory L1～L8 of standard jig fault image by the two-way connection of data wire, and the image information after will proofreading and correct divides four the tunnel to import the line driving X10 that control data writes respectively successively, X12, X14, X16, and then respectively successively through input frame memory SRAM5, SRAM6, SRAM7, behind the SRAM8, line through reading drives X9 again, X11, X13, the common input video encoder of X15, output image shows after D/A switch is composite video signal.

Images