CN202406224U

CN202406224U - Multipath image acquisition front-end device

Info

Publication number: CN202406224U
Application number: CN2011205303248U
Authority: CN
Inventors: 栾志超
Original assignee: Individual
Current assignee: Individual
Priority date: 2011-12-16
Filing date: 2011-12-16
Publication date: 2012-08-29
Anticipated expiration: 2021-12-16

Abstract

The utility model relates to multipath image acquiring and processing equipment. Aiming at avoiding the defects of complex wire arrangement and complex equipment in the prior art, the utility model discloses an FPGA (Field Programmable Gate Array)-based multipath image acquisition front-end device. The technical scheme adopted by the utility model is that the multipath image acquisition front-end device comprises image sensors, an image processing device, a control interface and an image output interface; n image sensors are provided; the image processing device comprises the FPGA, a cache unit and a nonvolatile memory; the cache unit and the nonvolatile memory are connected with the FPGA; the FPGA is at least provided with n groups of parallel interfaces which are respectively connected with the n image sensors; the control interface is connected with the FPGA; and the image output interface is connected with the FPGA, wherein n is a positive integer. The complexity of wire arrangement and the cost of system hardware are reduced; each image sensor cannot generate transmission delay; and the multipath image acquisition front-end device is very suitable for the field of industrial measurement and video monitoring.

Description

The multiplex image acquisition fore device

Technical field

The utility model relates to multiplex image acquisition and treatment facility, particularly a kind of based on FPGA (FPGA) as core, the multiplex image acquisition fore device of each imageing sensor output image of integration processing.

Background technology

At present in video monitoring and industrial detection field; A plurality of imageing sensors (or are called camera, video camera, rig camera, ccd video camera etc.; Usually also comprise signaling conversion circuit and necessary auxiliary equipment that it is built-in; Like The Cloud Terrace, photoflash lamp etc.) demand of working simultaneously is increasing, because only in this way could realize more wide monitoring visual field and meticulousr detectability.Along with increasing of imageing sensor quantity, the requirement of the reception of the wiring complexity of whole system and rear end, demonstration, treatment facility has also been improved accordingly.

Prior art is to adopt an imageing sensor to take a passage alone, and then is linked into Ethernet or Multiplexing Image Grab Card or multichannel input DVR etc.Such as in field of video monitoring, the multiplex image acquisition fore device is as shown in Figure 4, and first rig camera is all monopolized a passage separately to n rig camera, and the vision cable through special use is linked into multichannel input DVR.Along with the increase of number of cameras, the also corresponding raising of the requirement of DVR has also increased difficulty simultaneously in wiring.The video camera of the supervisory control system that also has is to adopt web camera; Be that each video camera all is an independently network equipment; Be linked into then in the Ethernet, exist so too wiring complicated, need more the network switch of multichannel number and the network equipment of managing more different IP.

In the industrial detection field, if shown in Figure 5.4 industrial CCD video cameras detect a testee, and the view data after the collection returns to the detection computations machine through local area network (LAN).The detection computations machine is controlled the action of testboard through PLC simultaneously.Each ccd video camera takies a LAN IP separately, and is connected on the local area network (LAN) through the network switch.

The common issue with that above-mentioned prior art exists is exactly that each imageing sensor all takies a passage alone, and then converges on the back-end processing equipment.Can cause wiring complicated, higher to the back-end processing equipment requirements simultaneously like this, need have the ability of multiway images input.For video monitoring, in picture bank this place of business hall, to place a plurality of rig cameras at a regional diverse location and could realize monitoring whole zone, its wiring complexity is well imagined.For industrial detection, some problem below existing method also has: 1) synchronism problem.The synchronism problem be divided into again between the video camera synchronously and video camera and control desk synchronous.Synchronous for video camera, because many ccd video cameras are connected to the detection computations machine through the LAN-sharing network, because the existence of network delay, so many ccd video cameras can't reach good synchronous images collection.If make ccd video camera be operated in external sync mode, the extra outer synchronizer that then need increase.Even and the outer synchronizer of employing also must make the control of outer synchronizer and testboard get up just meaningful synchronously.2) detection efficiency problem.Because the existence of network delay and image processing are not carried out before the image transmission, therefore the judgement to final detection result has bigger delay.

The content of utility model

The utility model technical problem to be solved is exactly complicated to the multiplex image acquisition system wiring of prior art, and the back-end processing problem is many, the shortcoming of complex equipments, and a kind of multiplex image acquisition fore device based on FPGA is provided.

The utility model solve the technical problem, and the technical solution of employing is the multiplex image acquisition fore device; Comprise imageing sensor, image processing apparatus, control interface and image output interface, it is characterized in that said imageing sensor quantity is n; Said image processing apparatus comprises FPGA, buffer unit and nonvolatile storage, and said buffer unit is connected with said FPGA with nonvolatile storage, and said FPGA has n group parallel interface at least; Be connected with n imageing sensor respectively, said control interface is connected with said FPGA, and said image output interface is connected with said FPGA; Wherein, n is a positive integer.

The utility model utilizes FPGA abundant parallel interface resource and programmable characteristics; Simultaneously the view data of a plurality of imageing sensor outputs is carried out integration processing before transmission; After the view data processing of the mode of setting, export to back-end processing equipment with the mode that takies an image output channel with all images transducer or certain several imageing sensor.

Imageing sensor is imported the parallel interface of FPGA with the viewdata signal of gathering, and receives the Equipment Control instruction that existing FPGA sends over, and control chart image-position sensor and auxiliary device thereof are like the work of The Cloud Terrace, photoflash lamp etc.The image output interface is used for the image of FPGA output is exported with VGA mode or other modes, so that display and back-end processing equipment can receive view data.The view data output interface includes necessary Signal Matching circuit, can carry out level conversion, number-Mo conversion etc., and other host-host protocol processing modules.FPGA is responsible for dispatching and control functional unit and the module that all link to each other, and carries out the reception and the buffer memory of view data, and the view data of buffer memory is exported through the image output interface with the integration form of user's appointment.FPGA also will receive the user's control command that passes over through control interface simultaneously, and operates accordingly according to command content.Control interface is used to receive user's control command, also can give the user by return parameters.Control interface can be universal serial port, network interface etc.Nonvolatile storage is used to preserve the current control command that passes over through control interface, so that when power on next time, enters into the mode of operation before closed electricity last time automatically.Buffer unit is used for the view data of cache image sensor acquisition.According to user-defined mode of operation, can buffer memory the view data of all or parts of images transducer, the execution of its buffer memory action is controlled by FPGA.

Further, inner at said FPGA, an every all corresponding FIFO (first in first out) buffer cell of imageing sensor.

After the view data input parallel interface, be introduced into its corresponding FIFO buffer cell.The inner control logic of FPGA is carried out the read-write operation of corresponding FIFO according to the FIFO current state, and the view data that will from FIFO, read is written in the outside buffer unit.

Further, said imageing sensor built-in signal translation circuit, the picture signal that said imageing sensor is gathered is connected with the parallel interface of FPGA through said signal conversion circuit.

Signal conversion circuit converts the view data of imageing sensor output into the signal of FPGA coupling on the one hand, and the control signal that on the other hand FPGA is transmitted is transformed to its auxiliary device, the signal compatible like The Cloud Terrace, photoflash lamp etc.

Concrete, said buffer unit is divided into m independently memory space, and m is a positive integer.

The size of buffer unit memory space and configuring condition, relevant with the image data rates of the resolution of the quantity of imageing sensor and image and transducer output.Under the lower situation of image rate, possibly use 1 memory, it is just passable to open up a memory space in memory inside, perhaps in a slice memory, opens up two or more independent memory space; If data rate is higher, possibly need 2 or the memory more than 2 so.No matter be multi-disc memory or at 1 memory, can open up a plurality of storage space, these memory spaces can be separate.Each memory space can be the part of a slice memory, also can comprise one or more pieces memories.

Special, m=2.

Buffer unit is divided into two parts, uses different memory spaces respectively, can realize the table tennis read-write of two memory spaces.

More specifically, said n=4.

Adopt 4 imageing sensors, can realize omnibearing monitoring detection.

Concrete, said 4 imageing sensors point to all directions four direction respectively.

The all directions four direction is pointed in 4 back-to-back configurations of imageing sensor respectively, is fit to the monitoring than the occasions such as hall of large space.

Concrete, a said n imageing sensor points to same measured object.

This is a kind of situation of commercial Application.Require 4 imageing sensor strict synchronism, and each image is transferred to the detection computations machine and postpones and should equate.Adopt the technical scheme of the utility model; The picture signal that is used for each imageing sensor is integrated before transmission; Adopt same image output interface and same transmission line to carry out the image transmission, each imageing sensor can not produce delay error, uses very much the applied environment of industrial detection.

The beneficial effect of the utility model is: (1) is because the interface resource that has of FPGA is abundant and characteristics that can parallel processing; The view data that therefore can receive the output of the multiway images transducer simultaneously row cache of going forward side by side; Or else lose then, only take an image output channel output after the distortionless view data integration processing, reduce wiring complexity and system hardware cost the output of a plurality of imageing sensors; (2) in industrial detection is used because the promptness that the control of imageing sensor and view data are handled, and the simultaneity of image data transmission, so have detection rapidly and efficiently, characteristics accurately; (3) because FPGA has programmable characteristics, therefore can under the prerequisite that does not change hardware circuit, can adapt to the imageing sensor of more kinds of types and changeable user's request, thereby have autgmentability flexibly.

Description of drawings

Fig. 1 is that embodiment forms sketch map;

Fig. 2 is 4 camera plane configuration schematic diagram;

Fig. 3 is a kind of industrial detection camera configuration schematic diagram;

Fig. 4 prior art video monitoring system structural representation;

Fig. 5 is existing industrial video detection system sketch map.

Embodiment

Below in conjunction with accompanying drawing and embodiment, the technical scheme of the utility model is described in detail.Obviously the description below only is used for the explanation to the utility model, and cited embodiment only is a part of embodiment of the utility model, but not whole embodiment.Those skilled in the art are under the prerequisite of not making creative work, and according to the description of the utility model, the routine of being done is replaced or is equal to replacement, all within the protection range of the utility model.

The operation principle of the utility model is described below:

Because different sensor or same imageing sensor have the output of pictures different form under the different working pattern, therefore at first to carry out the conversion of data format, change rgb format etc. such as yuv format.The imageing sensor that has itself has the signal format translation function, necessarily needs built-in special signaling conversion circuit.The view data of imageing sensor output makes the picture signal of its output and the parallel interface compatibility of FPGA through signal transformation.FPGA has rich interface and characteristics that can concurrent working, and its parallel interface can receive from the picture signal of each imageing sensor output simultaneously.In FPGA inside to an all corresponding first in first out (FIFO) buffer cell of each imageing sensor.After the data format conversion is accomplished; View data just gets into its corresponding FIFO buffer cell; The inner control logic of FPGA is carried out the read-write operation of corresponding FIFO according to the FIFO current state, and the view data that will from FIFO, read is written in the outside buffer unit.View data is written to which position of where organizing memory and memory of buffer unit, is all controlled by FPGA.

Read and have two kinds of situation for data cached from the output of view data output interface: the one, requirement is only exported and is triggered image constantly, and the 2nd, the real-time smooth output of requirement view data.

For first kind of situation, can buffer unit be divided into 2 independently memory spaces, realize the table tennis read-write of two memory spaces then, each memory space is controlled by FPGA respectively.

For second kind of situation, though one or two independently memory space can realize that also under view data output interface data rate and the unequal situation of imageing sensor output data rate, then control logic is can be very complicated.Want to guarantee the smooth in real time output of not losing of view data, the data rate that just necessarily requires the view data output interface to export must be equal to, or greater than the summation of all images transducer output data rate.Therefore independently do not cover the memory space of not read in the memory space writing each, and reading each independently in the memory space, guarantee can not read the memory space that not have renewal.

In sum; The inner control logic of FPGA is monitored the Data Update state of buffer unit simultaneously; After certain memory space in the buffer unit is write the whole frame view data, will get into the state of reading this memory space, the data of reading at first get into the FIFO buffer cell of dateout; And then, the image output interface is read and exported to the data in the FIFO buffer cell of dateout according to the time sequence status of image output interface.And when writing buffer unit,, confirm the data of which memory space of this renewal according to the state that buffer unit has been read, the data that will import then in the FIFO buffer cell of data write this memory space.

Parameter configuration and control for imageing sensor and auxiliary device thereof are then passed through the control interface sending controling instruction by the user, in FPGA, resolve control command and send imageing sensor and the discernible control command of auxiliary device thereof according to command content.Control to imageing sensor comprises mode of operation, trigger action, gain controlling, white balance control etc., and the imageing sensor auxiliary device can be camera lens, photoflash lamp, The Cloud Terrace etc.

The setting of whole system operation pattern is passed through the control interface sending controling instruction to FPGA by the user.In FPGA, resolve control command.The whole system operation pattern can be the finally arrangement mode in display frame, the processing mode of view data etc. of each imageing sensor view data.

For the processing of view data, such as the statistics of image pixel gray value, the parameter that needs is also sent by control interface, also is through control interface for returning of result simultaneously.

Mode of operation parameter, the image processing parameter of all imageing sensors and the Control Parameter of auxiliary device thereof, system all are kept in the nonvolatile storage.After system powered on, at first FPGA read parameter from nonvolatile storage, disposed the mode of operation of each transducer and auxiliary device thereof and the mode of operation of system.In the whole system course of work,, all can store in the nonvolatile storage if there is any parameter to change.Nonvolatile storage is also controlled by FPGA.

Embodiment 1

Referring to Fig. 1, the multiplex image acquisition fore device that this is routine comprises imageing sensor, image processing apparatus, control interface and image output interface.Wherein, imageing sensor quantity is 4, and image processing apparatus comprises FPGA, buffer unit and nonvolatile storage.This routine FPGA adopts the XC3S1000-4FT256C commodity device of Xilinx company, and buffer unit can adopt SDRAM (synchronous DRAM) chip, and nonvolatile storage then can be selected EEPROM (a kind of electricallyerasable ROM (EEROM)) chip for use.Buffer unit is connected with FPGA with nonvolatile storage, accepts its control.Corresponding FPGA has 4 groups of parallel interfaces at least, is connected with 4 imageing sensors respectively, the picture signal of its collection is handled, through image output interface output image data.This routine control interface adopts universal serial port (UART), is mainly used in the control command that transmits the user, and the operational factor of FPGA output etc.

Realize a rectangular area is monitored, need place 4 cameras at diverse location.Using conventional methods major part is respectively to place a camera at this regional diagonal, though can realize the monitoring in whole zone like this, wiring can be complicated, and requires high to the image receiving apparatus of rear end.This example only need be placed a multiplex image acquisition fore device and got final product in the regional center position, 4 cameras of installing back-to-back are housed on this device, and as shown in Figure 2,4 directions in all directions are pointed in 4 back-to-back configurations of camera respectively.And the output of final image is 4 images that camera image combines; Therefore on a picture, just can see these 4 images that camera head monitor arrives simultaneously; And 1 tunnel video channel that only need connect up is to Surveillance center; Thereby make original 4 road wiring channels into 1 road wiring channel, and the monitoring effect that reaches is identical.The structured flowchart of present embodiment is as shown in Figure 1, and the acp chip of 4 imageing sensors (camera) is the 0V7670 of Omnivision, and each camera links to each other with FPGA through FFC (Flexible Flat Cable).FPGA is inner for each camera has disposed relevant FIFO buffer cell, is convenient to and the buffer unit swap data.This routine buffer unit is made up of 4 K4S561632J, and the data/address bus of every K4S561632J, address bus, control bus all are independently, and all links to each other with FPGA, constitutes independently memory space.This routine nonvolatile storage model is 24LC08, is used for the running parameter of saved system.This routine control interface model is MAX3232, and it is the video output interface of ADV7123 that this illustration adopts model as output interface, can connect the DVR that the VGA display perhaps has the VGA input interface.The image of setting the output of 4 cameras is arranged with upper and lower, left and right 1/4 the mode that occupies whole image respectively when integrating.Because the resolution of 0V7670 is 640x480, so the video resolution of ADV7123 output is 1280x960.

Embodiment 2

In industrial detection, need carry out high-resolution IMAQ to measured object, carry out the graphical analysis of Pixel-level then.Therefore, require IMAQ and treatment system to have high-resolution, high real-time, good synchronism.4 MT9P401 that imageing sensor is Omnivision that in this application example, adopt, its highest resolution is 2592x1944, and the auxiliary device of installing simultaneously has lighting source, and lighting source has multistage light to be regulated.This illustration image-position sensor configuration structure is as shown in Figure 3, and 4 cameras are installed in respectively on 4 ribs of four-prism, points to same measured object and over against measured object.Image processing apparatus places between 4 imageing sensors.Imageing sensor and lighting source thereof link to each other with FPGA through FFC.This routine FPGA adopts the XC6SLX45T of Xilinx company, and data buffer storage unit is made up of 2 K4S561632J, and the data/address bus of every K4S561632J, address bus, control bus all are independently, and all link to each other with FPGA.24LC08 is as the running parameter of nonvolatile storage saved system, and universal serial port MAX3232 links to each other as control interface and with XC6SLX45T.FPGA links to each other with the image input interface of the PL-B777 plate of pixeLINK company, and the image after this image output interface can be integrated FPGA returns to the detection computations machine through USB2.0, Firewire A or GigE.In image integration, get the continuous 486 row pixels of each imageing sensor, after integration, be exactly the image of a frame 2592x1944 resolution like this.The image of each imageing sensor is to arrange from top to bottom at the arrangement mode of whole image.For an imageing sensor, specifically get any subregion, send parameter to FPGA by control end, realize the storage and the image integration of effective image-region view data by FPGA.In practical application, not high to the image real-time display requirement, but need and can specify photographic images constantly; Therefore; In FPGA, to realize receiving the photographic images instruction that control end sends, and the image of storage shooting effective coverage constantly, carry out image integration simultaneously.And the control of illumination light source is also passed through the control interface sending controling instruction to FPGA, realizes the control of illumination light source.Have need application to graphical analysis in, to realize that at FPGA each pixel is carried out particular analysis to be handled.Such as the gray value of each pixel being judged and added up, and statistics is returned to control end through control port, for the usefulness of decision-making.

Claims

1. the multiplex image acquisition fore device comprises imageing sensor, image processing apparatus, control interface and image output interface, it is characterized in that; Said imageing sensor quantity is n, and said image processing apparatus comprises FPGA, buffer unit and nonvolatile storage, and said buffer unit is connected with said FPGA with nonvolatile storage; Said FPGA has n group parallel interface at least; Be connected with n imageing sensor respectively, said control interface is connected with said FPGA, and said image output interface is connected with said FPGA; Wherein, n is a positive integer.

2. multiplex image acquisition fore device according to claim 1 is characterized in that, and is inner at said FPGA, every all corresponding FIFO buffer cell of imageing sensor.

3. multiplex image acquisition fore device according to claim 1 is characterized in that, said imageing sensor built-in signal translation circuit, and the picture signal that said imageing sensor is gathered is connected with the parallel interface of FPGA through said signal conversion circuit.

4. multiplex image acquisition fore device according to claim 1 is characterized in that, said buffer unit is divided into m independently memory space, and m is a positive integer.

5. multiplex image acquisition fore device according to claim 4 is characterized in that m=2.

6. multiplex image acquisition fore device according to claim 1 is characterized in that, said n=4.

7. multiplex image acquisition fore device according to claim 6 is characterized in that, said 4 imageing sensors point to all directions four direction respectively.

8. multiplex image acquisition fore device according to claim 1 is characterized in that, a said n imageing sensor points to same measured object.