JP2005025599A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that it is impossible for an LSI exclusive for image processing whose function is fixed to facilitate countermeasures to the addition or change of a new function in a conventional image processor using the LSI exclusive for image processing, and there is no choice but to facilitate countermeasures with software processing, and that it is difficult to construct any image processing system to individually facilitate countermeasures to the request of each user. <P>SOLUTION: This image processor is provided with an FPGA which performs image processing associated with adding/changing functions instead of a processor, and performs exclusive hardware-like processing to be operated in response to an instruction from the processor. This image processor is also provided with a non-volatile memory including an FROM in which an image processing program associated with the adding/changing functions to be transferred to the FPGA is stored, and the rewriting of records is made arbitrarily available. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus.
[0002]
[Prior art]
The image processing system is used in various fields, and even if it is called image processing, there are various configurations and processing contents depending on the application. For example, in an apparatus for inspecting the appearance of a wafer in a semiconductor manufacturing process. The demand for processing performance (responding to miniaturization and speeding up of processing time) is increasing due to miniaturization of semiconductor processes.
[0003]
An example of an image processing apparatus used in a semiconductor inspection apparatus is shown in FIGS.
[0004]
As shown in FIG. 6, the image processing apparatus includes a function for controlling input / output of image data to / from an image processing system, a memory for storing a processing target image or processed image data, and a processor for performing image processing. The
[0005]
In the configuration of FIG. 6, image processing is performed only using a processor. That is, an example in which image processing is performed as software processing, and FIG. 7 is a configuration example when a plurality of processors are used to parallelize image processing.
[0006]
FIG. 8 shows an example of an image processing apparatus equipped with an image processing dedicated LSI (image processing accelerator) for the purpose of speeding up image processing using a processor. In the configuration of FIG. 8, the image processing contents performed using the processor in FIG. 6 are partially hardware-processed using an image processing dedicated LSI to achieve high speed.
[0007]
In general, the configuration using an LSI dedicated to image processing is aimed at speeding up compared to image processing by a processor, but image processing by a processor can easily cope with changes in image processing contents. When an image processing dedicated LSI is used, the processing content is fixed, and there is a problem that it is not possible to cope with a change in the image processing content.
[0008]
For example, there is a device that processes half of the contents of the installed image processing with a dedicated LSI and performs image processing with the other half with a processor. When a new image processing function is added to such an image processing apparatus, the function of the LSI dedicated to image processing is fixed, and a new function cannot be realized. As a result, an image processing function is added.
[0009]
In this way, in an image processing apparatus using an image processing dedicated LSI, in order to add a new image processing function, all of them are added as software processing, and the new image processing function is implemented at high speed with the image processing dedicated LSI. To do so, it is necessary to redesign the image processing dedicated LSI.
[0010]
In addition, if there is a change in the implementation method or specifications even with the same image processing function, the image processing function realized with the LSI dedicated to image processing cannot be changed. There must be.
[0011]
Also, the image processing apparatus may have different functions used by the user and the content of the image processing desired to be performed at high speed. As in the example described above, when the image processing function to be mounted is separated by the processor and the image processing function to be processed by using the image processing dedicated LSI, the image processing apparatus determines which image processing function is to be realized. Will be judged by the developers.
[0012]
This is determined in consideration of the user's request of the image processing device to some extent, but not all users use the specification that matches the provided image processing device. Depending on the user, in the provided image processing device, if it is desired to perform image processing of the contents being processed by the LSI dedicated to image processing to increase the speed, or the contents of the image processing apparatus implemented by the LSI dedicated to image processing The demand for the image processing apparatus varies depending on the user, such as when it is not necessary. It is impossible to customize the image processing apparatus so as to match all users.
[0013]
There is also a technique for realizing image processing using an FPGA instead of an image processing dedicated LSI whose processing is fixed. The feature of the FPGA is that the processing function can be changed. However, since the processing function of the FPGA needs to be written in a memory for storing the processing function data in advance, when using it as an image processing apparatus, The function will be decided.
[0014]
Japanese Patent Laid-Open No. 2000-90237 (Patent Document 1) describes a method of rewriting processing functions mounted on an FPGA. However, processing functions to be rewritten are prepared in advance, and these are used for a certain image. It is rewritten according to the processing sequence.
[0015]
[Patent Document 1]
JP 2000-90237 A [0016]
[Problems to be solved by the invention]
As described above, the image processing apparatus is configured to realize software image processing using a processor, and image processing is performed using both the processor and the image processing dedicated LSI using the image processing dedicated LSI. There is something that takes a configuration to do.
[0017]
In general, a configuration using an LSI dedicated to image processing is more advantageous for speeding up the processing. However, since the contents of the image processing LSI are fixed, a new image processing function cannot be realized by the image processing LSI and must be added as software processing. When a change is made to the function implemented in the LSI dedicated to image processing, the changed part must be handled by software processing, and a customized image processing apparatus that meets the needs of each user can be realized. It cannot be done.
[0018]
Even when image processing is performed using an FPGA, the FPGA itself is a rewritable device, but when used as an apparatus, it is necessary to prepare a predetermined processing function. It cannot cope with the addition or change of processing functions that have occurred since the operation.
[0019]
SUMMARY OF THE INVENTION It is an object of the present invention to address the above-described problems, to perform image processing that has been speeded up by hardware processing, and to cope with the addition of new processing functions and processing changes. .
[0020]
[Means for Solving the Problems]
The present invention has a memory for storing image data and a program, and a processor for processing the image data with the downloaded software of the program and controlling overall operations and the like, and an addition / change function An FPGA that performs dedicated hardware processing that takes charge of image processing on behalf of the processor and that operates according to the instructions of the processor is provided, an image processing program relating to the function of addition / change to be passed to the FPGA is stored, and recording A nonvolatile memory including a FROM which can be rewritten arbitrarily is provided.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments according to embodiments of the present invention will be described with reference to the drawings.
[0022]
First, a description will be given along the block diagram of the image processing apparatus shown in FIG.
[0023]
The processor 1 performs software image processing. The memory 2 stores image data and programs. The processor 1 performs image processing of image data in software using software of a program downloaded from the memory 2. The processor 1 receives and outputs input image and output image signals via the image data input / output control unit 3.
[0024]
An FPGA 4 (Field Programmable Gate Array) dedicated to image processing takes charge of image processing related to the function of addition / change on behalf of the processor 1. The FPGA 4 performs image processing in a dedicated hardware manner. The operation of the FPGA 4 is performed according to an instruction from the processor 1. That is, an image processing system related to image processing is formed by the processor 1, the memory 2, and the FPGA 4.
[0025]
The FROM 5 (non-volatile memory) stores an image processing program relating to an addition / change function to be transferred to the FPGA. The controller 6 provides an image processing program related to the addition / change function to the FROM 5 (nonvolatile memory).
[0026]
The controller 6, the FROM 5, the image data input / output control unit 3, the processor 1, the memory 2, and the FPGA 4 are connected via a data bus.
[0027]
An FPGA 4 (Field Programmable Gate Array) dedicated to image processing is a highly integrated device of several million gates, and performs hardware processing in the same way as an LSI, so it can process image processing with a large amount of data very quickly. Moreover, the FPGA can arbitrarily rewrite internal logic, and can easily cope with rewriting of an image processing program in response to a request for addition / change of image processing.
[0028]
It is not necessary to prepare a dedicated LSI for image processing that satisfies the addition / change of specifications, and it can be handled simply by creating an image processing program related to addition / change, so the time spent adding / changing specifications can be greatly reduced. It is possible to respond flexibly and quickly to various requests regarding user specification changes.
[0029]
In addition, it is no longer necessary to replace the LSI dedicated to image processing with a printed circuit board on which various electronic components including a processor are mounted. Of course, the image processing program related to addition / change can be replaced (rewritten) on the user side, and since it is compatible with software change, it is possible to customize individually for various requests different for each user. .
[0030]
Rewriting of the image processing program relating to addition / change will be described along the processing flow of FIG.
[0031]
In step {circle around (1)}, a write instruction to the FROM is issued from the controller, and in step {circle around (2)}, new function data is written to the FROM. When the writing is completed, the operation of the image processing system is stopped in step (3). While the operation of the image processing system is stopped, a configuration start instruction is issued from the controller to the FPGA in step (4). Accordingly, in step (5), new function data is downloaded from the FROM to the FPGA. At the same time, the old function data is deleted. When writing (storing) by downloading new function data is completed, the operation of the image processing system is started in step (6).
[0032]
In this manner, downloading (writing) of new function data to the FPGA cannot be executed unless the operation of the image processing system is temporarily stopped. By preliminarily preparing for storing new function data in the FROM, the operation interruption time of the image processing system can be minimized.
[0033]
In addition, since writing of new function data is fast, it can be done during input / output of image data exchanged between the image data input / output control unit and the image processing system, or during an image processing execution waiting time.
[0034]
The FPGA is configured by an SRAM system. When the holding power is cut off due to a power failure or the like, the stored contents stored in the FPGA are lost. Therefore, the same data as the contents stored in the FPGA is stored in a nonvolatile memory such as FROM or PROM, and can be immediately rewritten from the nonvolatile memory to the FPGA.
[0035]
In the event of a power failure or the like, when the image processing apparatus is turned on again and restarted, supplementary provision is automatically provided to rewrite the data (image processing program) stored in the non-volatile memory at the same time as the operation. Done.
[0036]
This rewriting to the FPGA is also performed when the image processing apparatus is normally started. A mechanism is constructed in which these rewrites are performed in accordance with an instruction of a control signal issued from the control controller.
[0037]
In the embodiment of FIG. 1 described above, an example is shown in which a control controller for controlling the entire system is mounted in addition to a processor for performing image processing. However, this control controller can also be used as an image processing processor. is there.
[0038]
FIG. 2 shows a configuration example of a register as an on-board function identification function of an FPGA dedicated to image processing.
[0039]
As described above, the fact that the internal functions of the FPGA dedicated to image processing can be changed at an arbitrary timing means that there are a plurality of functions installed in the same FPGA dedicated to image processing, not unique. For this reason, a function for identifying which function is installed in the FPGA dedicated to image processing is required.
[0040]
This is also effective when checking the installed functions when different functions are provided for each user or when a plurality of processing functions are combined and mounted in an image processing dedicated FPGA. Although FIG. 3 shows an example of identification by bit arrangement for each function, it is also possible to manage the installed function as a version number and use the value as the identification function.
[0041]
FIG. 3 shows an example in which an arithmetic part for performing image processing itself and a data control unit for controlling input / output of image data to / from the outside of the FPGA are logically separated inside the FPGA dedicated to image processing.
[0042]
In order for an FPGA dedicated to image processing to perform processing, a function of capturing a processing target image in some form and outputting a processing result is indispensable.
[0043]
For example, a dedicated image memory is attached to an image processing dedicated FPGA, and an input / output image of the image processing dedicated FPGA is input / output from the image memory. A data control function for controlling the exchange of image data between them is required.
[0044]
There are various types of memories in the world, but what plays a role as an image memory is often a relatively large capacity such as SRAM or SDRAM. The data control function with these external memories is uniquely determined by the peripheral system configuration to which the FPGA dedicated to image processing is applied, and is not changed in the same system as the image processing function described above. .
[0045]
Therefore, in the FPGA dedicated to image processing, if the functional part for performing image processing and the functional part for controlling input / output of image data are separated, the image memory in the FPGA dedicated to image processing that implements different functions is also different. If they are the same, a common data control unit can be used. Further, when changing the image processing function, the same data control unit can be used and only the image processing function can be changed.
[0046]
In addition, the data control unit prepares several operation variations so as to be compatible with image memory types that are generally widely used, and can be selected as necessary. Therefore, it is possible to use the same FPGA dedicated to image processing even in different systems.
[0047]
FIG. 4 relates to another embodiment. A configuration example in the case where a plurality of FPGAs dedicated to image processing having the configuration of the embodiment described so far is used will be described.
[0048]
Some image processing apparatuses that require high speed implement high-speed image processing by mounting a plurality of dedicated processors for image processing and dividing the image processing into individual processors for parallel processing.
[0049]
When the FPGA dedicated to image processing described so far is a high-speed image processing by conventional software processing, the image processing apparatus which has been realized at high speed by a plurality of processors can also perform image processing on each processor. If a dedicated FPGA is provided, the processing speed can be further increased. In such a configuration, an image to be processed is divided and the same processing is assigned to each processor.
[0050]
Furthermore, as described above, each image processing dedicated FPGA can have different functions depending on the application, so that each processor and the image processing dedicated FPGA can execute different processes simultaneously or sequentially. Is possible.
[0051]
【The invention's effect】
According to the present invention, the image processing function that has been realized and fixed by the LSI until now can be changed and added even after the system is constructed.
[0052]
In addition, since different functions can be implemented in the same FPGA dedicated to image processing, it is possible to program the necessary functions in the dedicated FPGA for image processing in the same system when necessary.
[0053]
Furthermore, an image processing apparatus customized for each user can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image processing apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration of a register according to an embodiment of the present invention.
FIG. 3 is a diagram showing an internal functional configuration of an FPGA according to an embodiment of the present invention.
FIG. 4 is a block diagram of an image processing apparatus according to another embodiment of the present invention.
FIG. 5 is a diagram showing a processing flow based on the block diagram of FIG. 1;
FIG. 6 is a block diagram showing a conventional image processing apparatus.
FIG. 7 is a block diagram showing a conventional image processing apparatus.
FIG. 8 is a block diagram showing a conventional image processing apparatus.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Processor, 2 ... Memory, 3 ... Image data input / output control part, 4 ... FPGA (Field Programmable Gate Array), 5 ... FROM (nonvolatile memory), 6 ... Control controller.

Claims (7)

A memory for storing image data and a program, and a processor for processing the image data with the downloaded software of the program and controlling overall operations and the like,
An FPGA for performing image processing related to the function of addition / change on behalf of the processor and performing dedicated hardware processing that operates according to the instruction of the processor is provided.
An image processing apparatus comprising: a non-volatile memory including a FROM that stores an image processing program related to the addition / change function to be transferred to the FPGA and that can arbitrarily rewrite the recording. The image processing apparatus according to claim 1.
An image processing apparatus comprising: a controller that provides an image processing program related to the addition / change function to the nonvolatile memory. The image processing apparatus according to claim 1 or 2,
An image processing apparatus comprising a function identification register for identifying a plurality of functions existing in the image processing program in the FPGA. The image processing apparatus according to claim 1 or 2,
An image processing apparatus, comprising: an image processing unit that performs image processing provided in the FPGA; and a data control unit that controls input and output of the image data processed by the image processing unit. The image processing apparatus according to any one of claims 1 to 4,
An image processing apparatus comprising at least two FPGAs. The image processing apparatus according to any one of claims 1 to 5,
The image processing program of the FPGA that disappeared due to a power failure or the like is supplemented and provided from a nonvolatile memory simultaneously with the operation of the image processing device. A memory for storing image data and a program, and a processor for processing the image data with software of the downloaded program,
An FPGA for performing image processing related to the function of addition / change on behalf of the processor and performing dedicated hardware processing that operates according to the instruction of the processor is provided.
A non-volatile memory including a FROM that stores an image processing program related to the function of addition / change to be passed to the FPGA and can arbitrarily rewrite the recording is provided,
An image processing apparatus, comprising: a control controller that provides an image processing program related to the addition / change function to the nonvolatile memory and controls overall operations and the like.

Images