JP2004054647A - Image processing device and method for the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device and a method for the same capable of simplifying a processing unit, enhancing the efficiency of two-dimensional processing, reducing the size of a crossbar circuit, and increasing the speed of processing. <P>SOLUTION: A plurality of POPs (pixel operation processors) 0-3 as a function unit for performing highly parallel operation processing are included. Each of the POPs includes computing elements POPEs (pixel operation processing elements) 0-3 arranged in parallel. Each of the computing elements OPEs 0-3 conducts predetermined operation and outputs an operation result to a next stage POPE, followed by an reference image read from two caches RO$, RW$, element data of an object image, and operation parameter by a filter function unit FFU. The next stage POPE adds the operation result in a former stage to the operation result of itself, outputs to a next stage POPE, and sums the operation results of all of the POPEs 0-3 in a final stage POPE 3. Each of the POPs selects only the operation result of the POPE 3 from the operation outputs of the plurality of the POPEs, and outputs to the crossbar circuit 13125. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing apparatus and a method for performing parallel processing by sharing a plurality of processing data.
[0002]
[Prior art]
In conjunction with the recent increase in computational speed and the enhancement of drawing functions in computer systems, "computer graphics (CG)" technology for creating and processing figures and images using computer resources has been actively researched and developed. , Has been put to practical use.
[0003]
For example, in three-dimensional graphics, an optical phenomenon when a three-dimensional object is illuminated by a predetermined light source is expressed by a mathematical model, and based on this model, the surface of the object is shaded, shaded, or a pattern is pasted. For example, a more realistic three-dimensional two-dimensional high-definition image is generated.
Such computer graphics have been increasingly used in CAD / CAM and other various application fields in development fields such as science, engineering, and manufacturing.
[0004]
The three-dimensional graphics generally includes a “geometry subsystem” positioned as a front end and a “raster subsystem” positioned as a back end.
[0005]
The geometry subsystem is a process of performing a geometric operation such as a position and a posture of a three-dimensional object displayed on a display screen.
In the geometry subsystem, an object is generally treated as an aggregate of a large number of polygons, and geometric calculation processing such as “coordinate conversion”, “clipping”, and “light source calculation” is performed in units of polygons.
[0006]
On the other hand, the raster subsystem is a process of painting each pixel constituting an object.
The rasterizing process is realized by, for example, interpolating the image parameters of all the pixels included in the polygon based on the image parameters obtained for each vertex of the polygon.
The image parameters referred to here include color (rendering color) data expressed in a so-called RGB format and the like, z values indicating a distance in the depth direction, and the like.
In recent high-definition three-dimensional graphics processing, image parameters such as f (fog: fog) for creating a sense of perspective and texture (texture) that expresses a material feeling or a pattern on the surface of an object to provide reality by expressing the image parameter. Is included as one.
[0007]
Here, the process of generating pixels inside the polygon from the vertex information of the polygon is often performed using a linear interpolation method called DDA (Digital @ Differential @ Analyzer).
In the DDA process, the inclination of data in the direction of the side of the polygon is obtained from the vertex information, the data on the side is calculated using this inclination, and then the inclination in the raster scanning direction (X direction) is calculated. The internal pixel is generated by adding the parameter change obtained from the above to the parameter value of the scanning start point.
[0008]
Incidentally, in order to improve the performance of the graphics LSI, it is effective not only to increase the operating frequency of the LSI but also to use a parallel processing method. The method of parallel processing is roughly classified as follows.
The first is a parallel processing method based on region division, the second is a parallel processing method at a primitive level, and the third is a parallel processing method at a pixel level.
[0009]
The above classification is based on the granularity of the parallel processing. The granularity of the region division parallel processing is the highest, and the granularity of the pixel level parallel processing is the finest. The outline of each method is described below.
[0010]
Parallel processing by region division
This is a method in which a screen is divided into a plurality of rectangular areas and parallel processing is performed while allocating areas in charge of each of the plurality of processing units.
[0011]
Parallel processing at the primitive level
This is a method in which different primitives (for example, triangles) are given to a plurality of processing units to operate in parallel.
[0012]
Pixel-level parallel processing
This is the most granular parallel processing method.
FIG. 1 is a diagram conceptually illustrating a parallel processing at a primitive level based on a technique of a parallel processing at a pixel level.
As shown in FIG. 1, in the parallel processing method at the pixel level, when rasterizing a triangle, pixels are formed in a rectangular area unit called a pixel stamp (Pixel @ Stamp) PS composed of pixels arranged in a 2 × 8 matrix. Generated.
In the example of FIG. 1, a total of eight pixel stamps from the pixel stamp PS0 to the pixel stamp PS7 are generated. Up to 16 pixels included in these pixel stamps PS0 to PS7 are processed simultaneously.
This method is more efficient in parallel processing because the granularity is smaller than other methods.
[0013]
[Problems to be solved by the invention]
However, in the case of the above-described parallel processing by region division, in order to efficiently operate the processing units in parallel, it is necessary to classify objects to be drawn in each region in advance, and the load of scene data analysis is heavy.
In addition, drawing is not started after all scene data for one frame is completed, but drawing is performed in a so-called immediate mode in which drawing is started immediately when object data is provided. I can't.
[0014]
Further, in the case of parallel processing at the primitive level, there is a difference in the time for processing one primitive for each processing unit because the size of the primitives constituting the object varies. When the difference is large, the area where the processing unit draws is also greatly different, and the locality of data is lost. Therefore, for example, page misses of the DRAM constituting the memory module frequently occur, and the performance is reduced.
Further, in the case of this method, there is a problem that the wiring cost is high. Generally, in hardware that performs graphics processing, memory interleaving is performed using a plurality of memory modules in order to increase the memory bandwidth.
At that time, it is necessary to connect all the processing units and each built-in memory module.
[0015]
On the other hand, in the case of the parallel processing at the pixel level, as described above, there is an advantage that the efficiency of the parallel processing is high due to the finer granularity, and the processing including the actual filtering is performed according to the procedure shown in FIG. ing.
[0016]
That is, DDA parameters, for example, DDA parameters such as inclination of various data (Z, texture coordinates, color, etc.) necessary for rasterization are calculated (ST1).
Next, the texture data is read from the memory (ST2), the sub-word rearrangement processing is performed by the first functional unit including the plurality of arithmetic units (ST3), and the second bar including the plurality of arithmetic units is operated by the crossbar circuit. It is integrated into functional units (ST4).
Next, texture filtering (Texture Filtering) is performed (ST5). In this case, the second functional unit performs filtering processing such as 4-neighbor interpolation using the read texture data and the decimal part obtained when the (u, v) address is calculated.
Next, pixel-level processing (Per-Pixel @ Operation), specifically, a pixel-by-pixel operation is performed using filtered texture data and various types of rasterized data (ST5).
Then, the pixel data that has passed various tests in the pixel level processing is drawn on the frame buffers and the Z buffers on the plurality of memory modules (ST6).
[0017]
By the way, the memory access of the texture read system is different from the memory access of the drawing system, so that it is necessary to read from the memory belonging to another module.
Therefore, as for the memory access of the texture read system, wiring such as a crossbar circuit is required as described above.
[0018]
However, in a conventional image processing apparatus, the first functional unit performs a large amount of arithmetic processing on data read from a memory by a plurality of arithmetic units, so that the configuration is complicated.
Further, for example, in order to perform two-dimensional processing such as template matching, it is necessary to perform vertical addition on element data over a plurality of columns and then perform horizontal addition. However, high-parallel arithmetic processing is required, but simply using multiple arithmetic units makes it difficult to distribute the data of all the arithmetic units. Is difficult.
Further, since the obtained data is output from each arithmetic unit to the crossbar circuit, there is a disadvantage that the crossbar circuit as a global bus becomes large, which hinders high-speed processing from the viewpoint of wiring delay.
[0019]
SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has as its object to simplify a functional unit, improve the efficiency of processing, reduce the size of a crossbar circuit, and increase the processing speed. An object of the present invention is to provide a processing apparatus and a method thereof.
[0020]
[Means for Solving the Problems]
In order to achieve the above object, a first aspect of the present invention is an image processing apparatus that performs a template matching process by comparing a reference image and a changing target image, wherein the reference image is stored, and a plurality of ports are connected. A first memory, a second memory storing the target image and having a plurality of ports, and a plurality of ports of the first and second memories. A functional unit including a plurality of arithmetic units for performing parallel arithmetic processing based on the element data of the reference image and the element data of the target image read from the first memory and the second memory to generate continuous stream data; The operation units of the functional unit are cascade-connected from the first stage to the last stage, and each operation unit is connected to each port of the first and second memories, respectively. A predetermined operation is performed on the element data read from the element, and the result is output to the next-stage operation unit. The next-stage operation unit adds the operation result of the previous stage to the operation result of its own stage, and The result is output to the next-stage operation unit, the sum of all the operation units is obtained by the last-stage operation unit, and the operation result of the last stage is output as stream data.
[0021]
A second aspect of the present invention is an image processing apparatus that performs template matching processing by comparing a reference image and a changing target image, wherein the first memory stores the reference image and has a plurality of ports; A second memory that stores the target image and has a plurality of ports, and is provided corresponding to the plurality of ports of the first and second memories, and includes calculation parameters, the first memory and the second memory; A plurality of first functional units including a plurality of arithmetic units for performing parallel arithmetic processing based on the element data of the reference image and the element data of the target image read from the memory to generate continuous stream data; A second functional unit including a plurality of arithmetic units for performing intensive arithmetic processing on stream data generated by each of the first functional units; and the plurality of first functions. And a crossbar circuit for interconnecting the plurality of arithmetic units of the second functional unit with each other. The arithmetic units of each first functional unit are cascaded from a first stage to a last stage. Connected to each other, performs predetermined arithmetic processing on the element data read from each port of the first and second memories, and outputs the result to the next-stage arithmetic unit. The arithmetic unit adds the operation result of the previous stage to the operation result of its own stage, outputs the addition result to the next-stage arithmetic unit, calculates the sum of all the arithmetic units in the final-stage arithmetic unit, and calculates the final-stage operation result. Output as stream data.
[0022]
In the present invention, the reading of the element data of the reference image and the element data of the target image from the first and second memories to the respective arithmetic units of the functional unit is performed by sequentially inputting the data from the first stage, An address generator that generates an address so that the input of the operation result of the unit becomes a timing at which the operation of the stage is completed and the operation result of the previous stage can be added, and supplies the address to the first and second memories. Have.
[0023]
In the present invention, a memory module that stores at least one of the target image and the reference image and has a plurality of ports is provided, and the first and second memories are read from at least each port of the memory module. A plurality of first and second caches for storing image data and supplying the stored data to each of the operation units of the functional unit according to a cache address; When the element data of the reference image and the element data of the target image are read out to the arithmetic unit, the data is sequentially input from the first stage, and the input of the operation result of the previous stage arithmetic unit is completed when the operation of the own stage is completed and the operation of the previous stage is completed. Address generator for generating an address so as to be able to add the operation results of the units and supplying the address to the first and second caches Further comprising.
[0024]
In the present invention, preferably, the second functional unit is reconfigurable according to a control signal, and connects the arithmetic unit with an electrical connection network according to the control signal, and The electrical connection of the circuit is established to form an arithmetic circuit including a plurality of arithmetic units.
[0025]
In the present invention, the parallel processing is parallel processing at a pixel level.
[0026]
According to a third aspect of the present invention, there is provided an image processing apparatus in which a plurality of modules share processing data and perform parallel processing, the image processing apparatus including a global module and a plurality of local modules, wherein the global module includes the plurality of local modules. When the modules are connected in parallel and receive a request from the local module, processing data is output to the local module that issued the request in accordance with the request,
The plurality of local modules are modules for performing a template matching process by comparing a reference image and a changing target image, wherein the first memory storing the reference image and having a plurality of ports includes: A second memory that is stored and has a plurality of ports, and is provided corresponding to the plurality of ports of the first and second memories, and is read from the first memory and the second memory for operation parameters. And a functional unit including a plurality of arithmetic units for performing parallel arithmetic processing based on the element data of the reference image and the element data of the target image, and generating continuous stream data. The arithmetic units are cascaded from the first stage to the last stage, and each arithmetic unit is read from each port of the first and second memories. The specified element data is subjected to predetermined arithmetic processing and output to the next-stage arithmetic unit, and the next-stage arithmetic unit adds the previous-stage arithmetic result to the own-stage arithmetic result, and adds the addition result to the next-stage arithmetic unit. The output is output to the arithmetic unit, the sum of all the arithmetic units is obtained by the final stage arithmetic unit, and the operation result of the final stage is output as stream data.
[0027]
A fourth aspect of the present invention is an image processing apparatus in which a plurality of modules share processing data and perform parallel processing, the image processing apparatus including a global module and a plurality of local modules, wherein the global module includes the plurality of local modules. When the modules are connected in parallel and receive a request from the local module, the processing data is output to the local module that issued the request according to the request, and the plurality of local modules compare the reference image with the changing target image. A module that stores the reference image and has a plurality of ports; a second memory that stores the target image and has a plurality of ports; Provided in correspondence with the plurality of ports of the first and second memories; And a plurality of arithmetic units that perform parallel arithmetic processing based on the element data of the reference image and the element data of the target image read from the first memory and the second memory and generate continuous stream data. A plurality of first functional units, a second functional unit including a plurality of arithmetic units that perform intensive arithmetic processing on stream data generated by each of the first functional units, and a plurality of first functional units. A crossbar circuit interconnecting the functional units and the plurality of arithmetic units of the second functional unit, wherein the arithmetic units of the first functional units are arranged from the first stage to the last stage The arithmetic units are connected in cascade, perform predetermined arithmetic processing on the element data read from each port of the first and second memories, and output the data to the next arithmetic unit. The next-stage operation unit adds the operation result of the previous stage to the operation result of its own stage, outputs the addition result to the next-stage operation unit, calculates the sum of all the operation units in the last-stage operation unit, Is output as stream data.
[0028]
A fifth aspect of the present invention is an image processing method for performing a template matching process by comparing a reference image and a changing target image, wherein the first and second memories are respectively provided in a plurality of cascaded operation stages. A predetermined calculation process is performed on the element data of the reference image and the target image read from each port, and the next calculation stage adds the calculation result of the previous stage to the calculation result of its own stage, and The arithmetic stage calculates the sum of all the arithmetic stages, and outputs the arithmetic result of the final stage as stream data.
[0029]
A sixth aspect of the present invention is an image processing method for performing a template matching process by comparing a reference image and a changing target image, wherein each of the plurality of operation stages connected in cascade of a plurality of functional units has a first And performs predetermined arithmetic processing on the element data of the reference image and the target image read from each port of the second memory, and adds the arithmetic result of the preceding stage to the arithmetic result of its own stage in the next arithmetic stage The final operation stage calculates the sum of all the operation stages, outputs the operation result of the final stage to the crossbar circuit as stream data, and collects the crossbar circuit for a plurality of transferred stream data. Perform various arithmetic processing.
[0030]
According to the present invention, for example, the element data of the reference image and the target image stored in the first and second memories are read out from each port of the first and second memories, and the correspondence of each of the first functional units is determined. Is supplied to each computing unit.
In each computing unit of each first functional unit, predetermined computation processing (subtraction, multiplication, and total addition) is performed on the element data supplied from the cache.
At this time, in a plurality of cascade-connected arithmetic units, a predetermined arithmetic process is performed on the element data read from each port of the first and second memories, respectively. The operation result of the previous stage is added to the operation result of the stage, and the sum of all the operation stages is obtained in the final operation stage.
Then, the operation result of the final stage is output to the crossbar circuit as stream data.
The plurality of stream data by each first functional unit transferred to the crossbar circuit is supplied to the second functional unit.
In the second functional unit, an arithmetic circuit suitable for arithmetic operation is reconfigured by a control signal, and the arithmetic circuit performs intensive arithmetic processing on a plurality of stream data by each first functional unit.
[0031]
Further, according to the present invention, for example, in the case of texture-based processing, calculation parameters are generated in the controller, and the generated parameters are broadcast to the local module via, for example, a global module.
In each local module, for example, the following processing is performed.
That is, upon receiving the broadcasted parameter, it is determined whether or not the triangle belongs to an area in which it is responsible, for example, an area interleaved in units of a rectangular area of 4 × 4 pill cells. As a result, if they belong, various data (Z, texture coordinates, colors, etc.) are rasterized.
Next, a mipmap (MipMap) level is calculated by LOD (Level @ Detail) calculation, and a (u, v) address calculation for texture access is performed.
Then, the texture is read from the memory to the first functional unit.
Next, in the first functional unit of the local module, filtering processing such as 4-neighbor interpolation is performed using the read texture data and the decimal part obtained when the (u, v) address is calculated.
Next, pixel processing is performed in the second functional unit using the texture data after filtering and various data after rasterization.
Then, the pixel data that has passed various tests in the pixel-level processing is written to a memory module, for example, a frame buffer and a Z buffer on a built-in DRAM memory.
[0032]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 3 is a block diagram showing an embodiment of the image processing apparatus according to the present invention.
[0033]
As shown in FIG. 3, the image processing device 10 according to the present embodiment includes a stream data controller (SDC) 11, a global module 12, and a plurality of local modules 13-0 to 13-3.
[0034]
In the present image processing apparatus 10, the SDC 11 and the global module 12 exchange data, and a plurality m of one global module 12, in this embodiment, four local modules 13-0 to 13-3 are connected in parallel. The processing data is shared by the plurality of local modules 13-0 to 13-3 and processed in parallel. For the texture read system, memory access to another local module is required. Instead of taking the form of a global access bus, access is performed through one global module 12 having a function as a router.
The global module 12 has a global cache, and each of the local modules 13-0 to 13-3 has a local cache.
That is, the image processing apparatus 10 has two levels of cache, for example, a global cache shared by four local modules 13-0 to 13-3 and a local cache locally owned by each local module.
[0035]
The configuration and function of each component will be described below in order with reference to the drawings.
[0036]
The SDC 11 is responsible for sending and receiving data to and from the CPU and the external memory, and sending and receiving data to and from the global module 12, performs operations on vertex data, and performs rasterization in the processing units of the local modules 13-0 to 13-3. , Such as generation of parameters required for.
[0037]
The specific processing contents in the SDC 11 are as follows. FIG. 4 shows the processing procedure of the SDC 11.
[0038]
First, when data is input (ST1), the SDC 11 performs a Per-Vertex operation (ST2).
In this process, when the respective vertex data of the three-dimensional coordinates, the normal vector, and the texture coordinates are input, an operation is performed on the vertex data. As typical calculations, there are calculation processing of coordinate transformation for performing deformation of an object and projection on a screen, calculation processing of Lighting, and calculation processing of Clipping.
The processing performed here corresponds to the execution of so-called Vertex @ Shader.
[0039]
Next, a DDA (Digital @ Differential @ Analyzer) parameter is calculated (ST3).
In this process, DDA parameters such as the inclination of various data (Z, texture coordinates, color, etc.) necessary for rasterization are calculated.
[0040]
Next, the calculated DDA parameters are broadcast to all the local modules 13-0 to 13-3 via the global module 12 (ST4).
In this process, the broadcasted parameters are passed to the local modules 13-0 to 13-3 via the global module 12 using a channel different from the cache fill. However, it does not affect the contents of the global cache.
[0041]
The global module 12 has a router function and a global cache 121 shared by all local modules.
The global module 12 broadcasts the DDA parameters by the SDC 11 to all the local modules 13-0 to 13-3 connected in parallel.
[0042]
Further, upon receiving a request for a local cache fill (Local \ Cache \ Fill) LCF from a certain local module, for example, as shown in FIG. 5, the global module 12 checks an entry of the global cache (ST11). (ST12), reads the requested block data (ST13), sends the read data to the local module that sent the request (ST14), and if there is no entry (ST12), holds the block data. A request for a global cache fill (Global Cache Fill) GCF is sent to the target local module to be executed (ST15), and the global cache is sent using the block data sent thereafter. New to together (ST16, ST17), reads out the block data (ST13), the read data is sent to the local module that sent the request for the local cache fill LDF the (ST14).
[0043]
The local module 13-0 is a processing unit 131-0, for example, a memory module 132-0 including a DRAM, a local cache 133-0 unique to the module, and a global interface (Global Access Interface: GAIF) for controlling an interface with the global module 12. ) 134-0.
[0044]
Similarly, the local module 13-1 includes a processing unit 131-1, for example, a memory module 132-1 including a DRAM, a module-specific local cache 133-1, and a global interface (GAIF) 134 that manages an interface with the global module 12. -1.
The local module 13-2 includes a processing unit 131-2, for example, a memory module 132-2 including a DRAM, a module-specific local cache 133-2, and a global interface (GAIF) 134-2 for controlling an interface with the global module 12. Have.
The local module 13-3 includes a processing unit 131-3, for example, a memory module 132-3 composed of a DRAM, a module-specific local cache 133-3, and a global interface (GAIF) 134-3 for controlling an interface with the global module 12. Have.
[0045]
In each of the local modules 13-0 to 13-3, the memory modules 132-0 to 132-3 are interleaved in units of a predetermined size, for example, a rectangular area of 4 × 4. 0, the memory module 132-1 and the processing unit 131-1, the memory module 132-2 and the processing unit 131-2, and the memory module 132-3 and the processing unit 131-3 have one-to-one correspondence. In the drawing system, no memory access to other local modules occurs.
On the other hand, each of the local modules 13-0 to 13-3 requires memory access to other local modules for the texture read system. In this case, the local modules 13-0 to 13-3 access via the global module 12.
[0046]
Each of the processing units 131-0 to 131-3 of each of the local modules 13-0 to 13-3 is a streaming processor that executes so-called streaming data processing with high throughput, which is characteristic of image processing and graphics processing.
[0047]
The processing units 131-0 to 131-3 of the local modules 13-0 to 13-3 respectively perform, for example, the following graphics processing and image processing.
[0048]
First, the outline of the graphics processing of the processing units 131-0 to 131-3 will be described with reference to the flowcharts of FIGS.
[0049]
When the broadcasted parameter data is input (ST21), the processing unit 131 (-0 to -3) determines whether or not the triangle is an area in which the processing unit is in charge (ST22). , Rasterization is performed (ST23).
That is, upon receiving the broadcasted parameters, it is determined whether or not the triangle belongs to its own area, for example, an area interleaved in units of a rectangular area of 4 × 4 pixels. Various data (Z, texture coordinates, colors, etc.) are rasterized. In this case, the generation unit is 2 × 2 pixels in one cycle per local module.
[0050]
Next, a perspective collection of texture coordinates (Perspective @ Correction) is performed (ST24). In addition, this processing stage includes calculation of a mipmap (MipMap) level by LOD (Level @ of @ Detail) calculation, and calculation of (u, v) address for texture access.
[0051]
Next, the texture is read (ST25).
In this case, as shown in FIG. 7, the processing units 131-0 to 131-3 of each of the local modules 13-0 to 13-3 first execute the local cache 133-0 to 133-3 at the time of texture reading. The entry is checked (ST31). If there is an entry (ST32), necessary texture data is read (ST33).
When the required texture data is not in the local caches 133-0 to 133-3, the processing units 131-0 to 131-3 send the processing data to the global module 12 through the global interfaces 134-0 to 134-3. A request for a local cache fill is sent to it (ST34).
Then, the global module 12 returns the requested block to the local module that sent the request. If there is no requested block, as described above (described with reference to FIG. 5), the global module 12 sends the requested block to the local module that holds the block. Send a global cache fill request. After that, the block data is filled into the global cache, and the data is sent to the local module that sent the request.
When the requested block data is sent from the global module 12, the corresponding local module updates the local cache (ST35, ST36), and the processing unit reads the block data (ST33).
Here, it is assumed that a maximum of four textures are simultaneously processed, and the number of texture data to be read is 16 texels per pixel.
[0052]
Next, texture filtering (Texture Filtering) is performed (ST26).
In this case, the processing units 133-0 to 133-3 perform filtering processing such as 4-neighbor interpolation using the read texture data and the decimal part obtained at the time of calculating the (u, v) address.
[0053]
Next, pixel-level processing (Per-Pixel @ Operation) is performed (ST27).
In this process, a pixel-by-pixel operation is performed using the texture data after filtering and various data after rasterization. The processing performed here corresponds to so-called Pixel @ Shader such as pixel-level lighting (Per-Pixel @ Lighting). In addition, the following processing is included.
That is, the processing includes alpha test, scissoring, Z buffer test, stencil test, alpha blending, logical operation, and dithering.
[0054]
Then, the pixel data that has passed various tests in the pixel level processing is written to the memory modules 132-0 to 132-3, for example, the frame buffer and the Z buffer on the built-in DRAM memory (ST28: Memory).
Write).
[0055]
Next, an outline of the image processing of the processing units 131-0 to 131-3 will be described with reference to the flowchart of FIG.
[0056]
Before executing the image processing, the image data is loaded into the memory module 132 (-0 to -3).
Then, in the processing unit 131 (-0 to -3), commands and data necessary for generating a read (source) address and a write (destination) address required for image processing are input (ST41). .
Then, in the processing unit 131 (-0 to -3), a source address and a destination address are generated (ST42).
Next, the source image is read from the memory module 132 (-0 to -3) or supplied from the global module 12 (ST43), and predetermined image processing such as template matching is performed (ST44).
Then, predetermined arithmetic processing is performed as necessary (ST45), and the result is written to the area specified by the destination address of the memory module 132 (-0 to -3) (ST46).
[0057]
The local caches 133-0 to 133-3 of the local modules 13-0 to 13-3 store drawing data and texture data necessary for the processing of the processing units 131-0 to 131-3, respectively. To send and receive data (write and read) to and from the memory modules 132-0 to 132-3.
[0058]
FIG. 9 is a block diagram illustrating a configuration example of the local caches 133-0 to 133-3 of the local modules 13-0 to 13-3.
[0059]
As shown in FIG. 9, the local cache 133 includes a read-only cache (RO #) 1331, a read / write cache (RW #) 1332, a reorder buffer (Reorder \ Buffer: RB) 1333, and a memory controller (MC) 1334.
[0060]
The read-only cache 1331 is a read-only cache for reading a source image or the like of an arithmetic process, and is used for storing, for example, texture data.
The read / write cache 1332 is a cache for executing an operation that requires both reading and writing typified by, for example, read-modify-write (Read-Modify-Write) in graphics processing. Used.
[0061]
The reorder buffer 1333 is a so-called queuing buffer. If there is no necessary data in the local cache, the order of data sent to the global module 12 may be different when a local cache fill request is issued. The order of the data is adjusted so as to comply with this order and return to the processing units 131-0 to 131-3 in the order of request.
[0062]
FIG. 10 is a block diagram showing a configuration example of a texture system of the memory controller 1334.
As shown in FIG. 10, the memory controller 1334 arbitrates the cache controllers 13340 to 13343 corresponding to the four caches CSH0 to CSH3 and the local cache fill requests output from each of the cache controllers 13340 to 13343, and performs global arbitration. An arbiter 13344 for outputting data to -0 to 3 # and a memory interface 13345 for controlling data transfer in response to a global cache fill request input via the global interface 134 # -0 to 3 #.
[0063]
In addition, the cache controllers 13340 to 13343 provide the two-dimensional addresses COuv00 to COuv03, COuv10 to COuv13, COuv20 to COuv23 of each data necessary when performing the 4-neighbor interpolation on the data corresponding to the four pixels PX0 to PX3, respectively. The conflict checker CC10 that checks and distributes address conflicts in response to COuv30 to COuv33 and checks the address distributed by the conflict checker CC10 to determine whether or not the data indicated by the address exists in the read-only cache 1331. It has a tag circuit TAG10 and a queue register QR10.
The tag circuit TAG10 has four tag memories BX10 to BX13 corresponding to addressing related to interleaving of banks, which will be described later, and is stored in the read-only cache 1331.
The address distributed by the conflict checker CC10 holding the address tag of the block data is compared with the address tag, and a flag indicating whether or not the address is matched and the address are set in the queue register QR10. The address is sent to the arbiter 13344.
The arbiter 13344 performs an arbitration operation in response to the address transmitted from the cache controllers 13340 to 13343, selects an address according to the number of requests that can be transmitted simultaneously via the global interface (GAIF) 134, and sets the address as a local cache fill request. Output to the global interface (GAIF) 134.
When data is sent from the global cache 12 in response to the local cache fill request sent via the global interface (GAIF) 134, the data is set in the reorder buffer 1333.
The cache controllers 13340 to 13343 check the flag at the head of the queue register QRL0, and if a flag indicating that they match is set, based on the address at the head of the queue register QRL0, the read-only cache 1331 Is read and given to the processing unit 131. On the other hand, if the flag indicating the match has not been set, the corresponding data is read from the reorder buffer 1333 when the corresponding data is set in the reorder buffer 1333, and read with the block data based on the address of the queue register QRL0. It updates the only cache 1331 and outputs it to the processing unit 131.
[0064]
Next, the memory capacity of the DRAM as a memory module, the local cache, and the global cache will be described.
The relationship between the memory capacities is naturally DRAM> global cache> local cache, but the ratio depends on the application.
The cache block size corresponds to the data size read from the lower-level memory when the cache is filled.
A characteristic of the DRAM is that the performance is reduced at the time of random access, but the continuous access of data belonging to the same row (ROW) is fast.
[0065]
The performance of the global cache is preferably higher in terms of performance in terms of reading data from the DRAM.
Therefore, the size of the cache block is set large.
For example, the size of the cache block of the global cache can be set to a block size of one row of the DRAM macro.
[0066]
On the other hand, in the case of the local cache, if the block size is increased, the ratio of unused data increases even if it is cached, and the lower layer is a global cache, which is not DRAM and does not need continuous access. , The block size is set small.
As the block size of the local cache, a value close to the size of the rectangular area of the memory interleave is appropriate. In the present embodiment, the block size is 4 × 4 pixels, that is, 512 bits.
[0067]
Next, texture compression will be described.
Since a plurality of pieces of texture data are required to process one pixel, the texture readout bandwidth often becomes a bottleneck, but a method of compressing the texture is often adopted to reduce this.
There are various compression methods. In the case of a method capable of compressing / decompressing data in units of a small rectangular area such as 4 × 4 pixels, the compressed data is stored in the global cache, and the decompressed data is stored in the local cache. It is preferable to put later data.
[0068]
Next, a specific configuration example of the processing units 131-0 to 131-3 of the local modules 13-0 to 13-3 will be described.
[0069]
FIG. 11 is a block diagram illustrating a specific configuration example of the processing unit of the local module according to the present embodiment.
[0070]
As shown in FIG. 11, the processing units 131 (-0 to -3) of the local module 13 (-0 to -3) have a rasterizer (RSTR) 1311 and a core (Core) 1312.
Among these components, an arithmetic processing unit that implements the present architecture is a core 1312, and the core 1312 is supplied with various data for graphics processing such as addresses and coordinates and image processing by a rasterizer 1311.
[0071]
In the case of graphics processing, the rasterizer 1311 receives the parameter data broadcast from the global module 12 and determines, for example, whether or not the triangle is in its own area. Rasterization is performed based on the input triangle vertex data, and the generated pixel data is supplied to the core 1312.
The pixel data generated by the rasterizer 1311 includes window coordinates (X, Y, Z), primary colors (Primary @ Color: PC) (Rp, Gp, Bp, Ap), and secondary colors (Secondary @ Color: SC) (Rs, Gs, Bs, As), Fog coefficient (f), texture coordinates, normal vector, line-of-sight vector, light vector ((V1x, V1y, V1z), (V2x, V2y, V2z)).
The data supply line from the rasterizer 1311 to the core 1312 includes, for example, a supply line for window coordinates (X, Y, Z), another primary color (Rp, Gp, Bp, Ap), and a secondary color (Rs, Gs). , Bs, As), the Fog coefficient (f), the texture coordinates (V1x, V1y, V1z), and the supply line of (V2x, V2y, V2z) are formed by different wirings.
[0072]
In the case of image processing, the rasterizer 1311 includes, for example, a source address for reading image data from the memory module 132 (−0 to −3) and an image processing result output from a higher-level device (not illustrated) via the global module 12. Command and data necessary for generating a destination address for writing the data, such as the width and height data (Ws, Hs) and block size data (Wbk, Hbk) of the search rectangular area, and based on the input data, The source address (X1s, Y1s) and / or (X2s, Y2s) are generated, and the destination address (Xd, Yd) is generated and supplied to the core 1312.
As a supply line of data from the rasterizer 1311 to the core 1312 at the time of image processing, for example, a supply line of window coordinates (X, Y, Z) at the time of graphics processing is shared with respect to a destination address (Xd, Yd). For the addresses (X1s, Y1s) and (X2s, Y2s), supply lines such as the texture coordinates (V1x, V1y, V1z) and (V2x, V2y, V2z) are shared.
[0073]
The core 1312 is an arithmetic processing unit for realizing the present architecture, and the core 1312 is supplied with various data by a rasterizer 1311.
The core 1312 has the following functional units that perform arithmetic processing on stream data.
That is, the core 1312 includes a graphics unit (Graphics {Unit}: GRU) 13121 as a first functional unit, a pixel engine (Pixel Engineer: PXE) 13122 as a third functional unit, and a pixel as a second functional unit. An arithmetic processor (Pixel 0 operation Processor: POP) group 13123 is provided.
The core 1312 supports various algorithms by switching the connection between these functional units according to, for example, a data flow graph (Data {Flow} Graph: $ DFG). {Furthermore, the core 1312 includes a register unit (Register Unit: RGU) 13124 and a crossbar circuit (Interconnection X-Bar}: IXB) 13125.
[0074]
The graphics unit (GRU) 13121 is a functional unit in which the addition of dedicated hardware, which is clearly advantageous in terms of cost performance when executing graphics processing, is implemented by hard-wired logic.
The graphics unit 13121 implements functions such as perspective collection (Perspective @ Correction) and MIPMAP level calculation, which are related to graphics processing.
[0075]
The graphics unit 13121 is supplied with the texture coordinates (V1x, V1y, V1z) supplied by the rasterizer 1311 via the crossbar circuit 13125 and the register unit (RGU) 13124, and / or supplied by the rasterizer 1311 or the pixel engine (PXE) 13122. Based on the input data, input the texture coordinate (V2x, V2y, V2z) data, perform perspective correction, calculate a mipmap (MIPMAP) level by LOD (LevelofDetail) calculation, select a surface of a cubic map (Cube @ Map), A calculation process of the normalized texel coordinates (s, t) is performed, and for example, graphics data (s1, t1, l) including the normalized texel coordinates (s, t) and LOD data (lod) d1) and / or (s2, t2, lod2) is output to the pixel operation processor (POP) group 13123.
The output graphics data (s1, t1, lod1) and (s2, t2, lod2) of the graphics unit 13121 are passed through the crossbar circuit 13125 and the register unit (RGU) 13124, or as indicated by broken lines in FIG. Is supplied directly to a pixel operation processor (POP) group 13123 via another wiring.
[0076]
A pixel engine (PXE) 13122 as a third functional unit is a functional unit that performs stream data processing, and has a plurality of arithmetic units inside. The pixel engine 13122 has a higher degree of freedom in connection between arithmetic units as compared to the pixel operation processor (POP) group 13123, and has a rich function of the arithmetic units.
[0077]
The pixel engine (PXE) 13122 sets the information on the drawing target and the operation result in the pixel operation processor (POP) group 13123 in a desired FIFO register of the register unit (RGU) 13124 by the crossbar circuit 13125, for example, The signal is directly supplied via a register unit (RGU) 13124 without passing through the bar circuit 13125.
The data input to the pixel engine (PXE) 13122 include, for example, information on the surface to be drawn (surface direction, color, reflectance, pattern (texture), etc.), and information on the light hitting the surface (incident direction, intensity). And the like, and past calculation results (intermediate values of the calculations).
[0078]
The pixel engine (PXE) 13122 has a plurality of arithmetic units, and is, for example, an arithmetic unit capable of reconfiguring an arithmetic path by external control, and is provided between internal arithmetic units so as to realize a desired arithmetic operation. An electrical connection is established, and the data input through the register unit (RGU) 13124 is input to a data path of a series of arithmetic units formed from the arithmetic unit and an electrical connection network (interconnect), thereby performing an arithmetic operation. And outputs the operation result.
[0079]
That is, the pixel engine 13122 has, for example, a plurality of reconfigurable data paths, and connects arithmetic units (adders, multipliers, multiply-adders, and the like) by an electrical connection network. An arithmetic circuit is constructed.
Then, the pixel engine 13122 can continuously input data and perform an operation to the operation circuit reconfigured in this way. For example, the pixel engine 13122 is a binary tree-shaped DFG (data flow graph). The arithmetic circuit can be configured using a connection network that can efficiently realize the expressed arithmetic with a small circuit scale.
[0080]
FIG. 12 is a block diagram illustrating a configuration example of the pixel engine (PXE) 13122.
The present pixel engine (PXE) 13122 has a computing unit pool 200 as shown in FIG.
The computing unit pool 200 includes at least one (four in the example of FIG. 12) computing units 201 to 204 that exchange data with the stream register unit 13124 via the data bus BS.
Each of the arithmetic units 201 to 204 includes an electrical connection between a plurality of (eight in the example of FIG. 12) arithmetic units (adders, multipliers, multipliers, etc.) OP1 to OP8 and the arithmetic units OP1 to OP8. Includes a connection network CCN whose connection can be changed.
[0081]
That is, in the pixel engine (PXE) 13122, the connection network operation unit CCN exists between the operation units OP.
As described above, by independently providing the register file, the circuit amount can be reduced.
[0082]
In the example of FIG. 12, each of the arithmetic units 201 to 204 is configured to individually transmit and receive data to and from the register unit (RGU) 13124 via the data bus BS and the crossbar circuit 13125. For example, a mode is also possible in which each of the arithmetic units 201 to 204 is connected by another signal line, and an arithmetic operation is performed using the arithmetic result of one arithmetic unit in another arithmetic unit.
[0083]
FIG. 13 is a diagram showing a configuration example of the connection network CCN according to the present invention.
A feature of the configuration of the connection network CCN is that, for example, when there are 2n input buses, n arithmetic units select a pair of inputs from a register unit (RGU) 13124 and an output of a preceding (left) arithmetic unit. And input it to the arithmetic unit at its own stage. The remaining arithmetic units select all inputs from the register unit (RGU) 13124 and the output of the preceding stage (left side) and input them to the arithmetic units.
[0084]
The connection network CCN of FIG. 13 is a configuration example having four pairs of eight input buses L11, L12, L21, L22, L31, L32, and L41, L42.
In FIG. 13, a black circle shown at a predetermined intersection of the input bus indicates a selector. FIG. 14 shows a configuration example of each selector.
Note that the lines in the drawing indicate bundled wires (a group of two or more signal lines).
[0085]
FIG. 13 shows an example of a configuration having seven operation units OP1 to OP7. The stages from the input to the output of the operation units arranged in parallel to the stages STG1 to STG7 (the output of the operation unit OP7 of the final stage 7 is The data is sent to the stream register file) as follows.
[0086]
That is, in the first stage STG1, a pair of input buses L11 and L12 are connected to the input of the arithmetic unit OP1, and the output of the arithmetic unit OP1 is output from the next stage and thereafter (in the example of FIG. 13, the input side of the third stage STG3). It is connected to the.
In the second stage STG2, a pair of input buses L21 and L22 are connected to the input of the operation unit OP2, and the output of the operation unit OP2 is connected to the input side of the third stage STG3.
[0087]
In the third stage STG3, a pair of input buses L31 and L32 are connected to the input of the operation unit OP3, and the output of the operation unit OP3 is connected to the input side of the third stage STG4. The output lines of the operation unit OP1 of the first stage STG1 and the output line of the operation unit OP2 of the second stage STG2 intersect with the input buses L31 and L32, respectively. The selector SLC shown in FIG.
[0088]
In the fourth stage STG4, a pair of input buses L41 and L42 are connected to the input of the operation unit OP4, and the output of the operation unit OP4 is connected to the input side of the fifth stage STG5. The output lines of the operation unit OP1 of the first stage STG1, the output line of the operation unit OP2 of the second stage STG2, and the output line of the operation unit OP3 of the third stage STG3 cross the input buses L41 and L42. The selector SLC shown in FIG. 13 is arranged at each of these six intersections.
[0089]
In the fifth stage STG5, the input bus L42 and the input bus L42 are selected so that all the inputs from the register unit (RGU) 13124 and the outputs of the first to fourth stages STG1 to STG4 are selectively input to the arithmetic unit OP5. Eight intersections with the buses L21, L22, L31, L32, L41, the output line of the operation unit OP1 of the first stage STG1, the output line of the operation unit OP2 of the second stage STG2, and the operation unit OP3 of the third stage STG3 The selector SLC shown in FIG. 14 is disposed at each of the eight intersections where the output line of the fourth stage STG4 intersects with the output line of the operator OP4.
[0090]
In the sixth stage STG6, the input bus L42 and the input bus L42 are input to the operation unit OP6 such that all the inputs from the register unit (RGU) 13124 and the outputs of the first to fifth stages STG1 to STG5 are selected and input. Eight intersections with the buses L21, L22, L31, L32, L41, the output line of the operation unit OP1 of the first stage STG1, the output line of the operation unit OP2 of the second stage STG2, and the operation unit OP3 of the third stage STG3 The selector SLC shown in FIG. 14 is arranged at each of ten intersections where the output line of the fourth stage STG4, the output line of the operation unit OP4 of the fourth stage STG4, and the output line of the operation unit OP5 of the fifth stage STG5 intersect. I have.
[0091]
In the seventh stage STG7, the input bus L42 and the input bus L42 are input to the operation unit OP7 such that all the inputs from the register unit (RGU) 13124 and the outputs of the first to sixth stages STG1 to STG6 are selected and input. Eight intersections with the buses L21, L22, L31, L32, L41, the output line of the operation unit OP1 of the first stage STG1, the output line of the operation unit OP2 of the second stage STG2, and the operation unit OP3 of the third stage STG3 , The output line of the operator OP4 of the fourth stage STG4, the output line of the operator OP5 of the fifth stage STG5, and the twelve intersections where the output lines of the operator OP6 of the sixth stage STG6 intersect. The selector SLC shown in FIG.
[0092]
Here, the outline of the calculation execution of the pixel engine (PXE) 13122 according to the present invention will be described with reference to FIGS.
[0093]
For example, an operation of reading data A, B, C, and D from the register unit (RGU) 13124 and writing a value corresponding to Y in the following equation to the register unit (RGU) 13124 is performed a plurality of times.
[0094]
(Equation 1)
Y [i] = (A [i] + B [i]) × (C [i] + D [i])
[0095]
FIG. 15 shows a DFG (data flow graph) of the operation shown in Expression (1).
[0096]
The operation unit OP1 having the function of executing the operation 1 is connected to the output of the register file 2011 to which the value corresponding to the data A is output by the connection 0 which is an electrical connection path corresponding to the branch 0.
Similarly, through connections 1 to 3 corresponding to branches 1 to 3, the output corresponding to data B of the register unit (RGU) 13124 is connected to the operating unit OP1 and the outputs corresponding to data C and D are connected to the operating unit OP2. .
The output of the operation unit OP1 is connected to a connection 4 corresponding to the branch 4 and the output of the operation unit OP2 is connected to an input of a calculation unit OP3 having a function of executing the operation 3 through a connection 5 corresponding to the branch 5.
The output of the arithmetic unit OP3 is input to the register file 2011 via the connection 6 corresponding to the branch 6, and a path for writing a value corresponding to Y is established.
[0097]
In this way, the electrical connection between the register file and the arithmetic unit is realized, and data corresponding to A [i], B [i], C [i], and D [i] are sequentially read from the register file, and the arithmetic operation is performed. , The operations for the plurality of A to D are efficiently realized.
The pixel engine (PXE) 13122 realized in this manner can easily cope with different calculations by changing the electrical connection between the calculators.
In the case of the configuration of FIG. 13, arithmetic units OP1 to OP3 are used, and connection 0 and connection 1 correspond to input buses L11 and L12, and connection 2 and connection 3 correspond to input buses L21 and L22.
The selector 4 shown in FIG. 13 forms a connection 4 for inputting the operation result of the operation unit OP1 to the operation unit OP3, and the selector SLC2 forms a connection 5 for inputting the operation result of the operation unit OP2 to the operation unit OP3.
Then, a connection 6 for outputting the operation result of the operation unit OP3 to the register unit (RGU) 13124 as it is is formed.
[0098]
The above operation is executed by pipeline processing as shown in FIG.
17A shows a clock, RR in FIG. 17B shows a process of reading data A to D from the register unit (RGU) 13124, and FIGS. 17C, 17E, and 17G. IC indicates data transfer processing via the connection network CCN, Add1 / 0 in FIG. 17D indicates arithmetic processing by the operating units OP1 and OP2, and mul in FIG. 17F indicates arithmetic processing by the operating unit OP3. WB in FIG. 17H indicates a process of writing the operation result to the register unit (RGU) 13124.
[0099]
The pixel engine (PXE) 13122 according to the present embodiment can dynamically reconstruct the data path as described above.
Accordingly, the pixel engine (PXE) 13122 can change the electrical connection between the operation units when executing the operation in a pipeline using the operation circuit.
Also, by dynamically changing the configuration between the computing units as described above, different computations can be performed without delay.
[0100]
In addition, the pixel engine (PXE) 13122 automatically stores information on the next operation to be performed by the control circuit added to each of the arithmetic units, and detects the end of a series of operations. It has a function to switch to control for the next calculation.
Then, in the pixel engine (PXE) 13122, when the control circuit assigned to each connection point of the connection network CCN holds information on the next connection configuration to be taken and detects the end of a series of data transfer. Each of the control circuits has a function of automatically switching the control of the connection point.
[0101]
Next, a method for implementing dynamic reconfiguration will be described.
[0102]
Realization method 1 of dynamic restructuring
First, a first method of realizing dynamic reconfiguration will be described with reference to FIGS.
In this case, as shown in FIG. 18, the control circuit 301 for each operation unit OP and the connection network control circuit CCN includes current control information (current control information) CIFM and information on control to be performed next (next control information). Keep two NIFMs.
Then, the operation data OPDT is transmitted in synchronization with the control signal CTL that can identify that it is the final data used for the operation.
When it is determined that the data is the last data, the control circuit 301 rewrites the current control information CIFM with the next control information NIFM simultaneously with the completion of the currently executed operation.
Thus, the control of the arithmetic circuit can be changed, and different arithmetic can be performed.
[0103]
The same applies to the connection network CCN. As shown in FIG. 19, when the control signal CTL identifies that the data is the last data, the control circuit 301 simultaneously completes the data transfer currently being executed with the current control signal. The information CIFM is rewritten with the next control information NIFM.
As a result, the control of the connection network can be changed, and different electrical connections can be realized.
[0104]
Next, a second method of realizing dynamic reconfiguration will be described with reference to FIGS.
[0105]
Realization method 2 of dynamic restructuring
As described above, when different arithmetic operations are continuously performed using an arithmetic device including an arithmetic circuit and a connection network, as shown in FIG. (Overlap section of operation 1 and operation 2) occurs.
During this time, data for different operations exist simultaneously on the operation circuit and the connection network.
In this section, control corresponding to operation 2 is performed on some circuits while the final data of operation 1 exists on the operation circuit.
Therefore, when the final data of the operation 1 arrives at the operation unit performing the operation 2 or the connection network control circuit under the control corresponding to the operation 2, they are recognized as the end of the operation, and There is a possibility that the control is switched from the control to the control for the calculation 3, and the calculation for the remaining calculation 2 is not normally performed.
An example of realizing dynamic reconfiguration corresponding to this will be described below.
[0106]
In this case, as shown in FIG. 21, the control circuit 301 for each operation unit OP and the connection network control circuit CCN transmits the current control information (current control information) CIFM and the information (current Identification information) CDSC, information on next control to be performed (next control information) NIFM, and information (next identification information) NDSC for identifying an operation to be executed next.
The operation data OPDT is synchronized with the control signal CTL indicating information that can be identified as the final data used for the operation and information that can identify whether the data is for the operation 1 or the operation 2. Will be sent.
When the control circuit 301 determines that the transmitted data is the final data and is for the operation indicated by the current identification information CDSC, the control circuit 301 simultaneously executes the completion of the operation currently being executed and the current control information. The CIFM and the current identification information CDSC are rewritten with the next control information NIFM and the next identification information NDSC, respectively.
As a result, it is possible to switch to a different operation at an appropriate timing with respect to stream data continuously input.
[0107]
The same applies to the connection network CCN. As shown in FIG. 22, the current control information CIFM and the current identification information IDSC are rewritten with the next control information NIFM and the next identification information NDSC, respectively, at the same time as the completion of the currently executed data transfer. .
This makes it possible to switch to a different electrical connection at appropriate timing for data that is continuously input.
[0108]
FIG. 23 is a diagram illustrating a preferred configuration example of the pixel engine (PXE) 13122 and a connection example with the register unit (RGU) 13124 and the crossbar circuit 13125.
[0109]
As shown in FIG. 23, the pixel engine (PXE) 13122 has a plurality of (16 in the example of FIG. 23) arithmetic units OP1 to OP8 and OP11 to OP18 based on a two- or three-input MAC (Multiply and Accumulator). And one or more (four in the example of FIG. 23) look-up tables LUT1, LUT2, LUT11, and LUT12.
[0110]
As shown in FIG. 23, the two inputs of each of the arithmetic units OP1 to OP8 and OP11 to OP18 in the pixel engine (PXE) 13122 are connected to the FIFO (First-IN {First-Out)} register FREG of the register unit (RGU) 13124. Is directly connected to.
Similarly, one input of the look-up tables LUT1, LUT2, LUT11, and LUT12 is directly connected to the FIFO register FREG of the register unit (RGU) 13124.
The outputs of the operation units OP1 to OP8, OP11 to OP18 and the lookup tables LUT1, LUT2, LUT11, LUT12 are connected to a crossbar circuit 13125.
[0111]
Further, in the example of FIG. 23, the output of the operation unit OP1 is connected to two inputs of the operation units OP3 and OP4 and one input of the three-input operation unit OP2, respectively. Similarly, the output of the operation unit OP2 is connected to two inputs of the operation unit OP4 and one input of the three-input operation unit OP3, respectively. The output of the operation unit OP3 is connected to one input of a three-input operation unit OP4.
The output of the operation unit OP5 is connected to two inputs of the operation units OP7 and OP8 and one input of the three-input operation unit OP6, respectively. Similarly, the output of the operation unit OP6 is connected to the two inputs of the operation unit OP8 and the one input of the three-input operation unit OP7, respectively. The output of the operation unit OP7 is connected to one input of the three-input operation unit OP8.
Further, the output of the operation unit OP11 is connected to two inputs of the operation units OP13 and OP14 and one input of the three-input operation unit OP12, respectively. Similarly, the output of the operation unit OP12 is connected to two inputs of the operation unit OP14 and one input of the three-input operation unit OP13, respectively. The output of the operation unit OP13 is connected to one input of the three-input operation unit OP14.
The output of the operation unit OP15 is connected to two inputs of the operation units OP17 and OP18 and one input of the three-input operation unit OP16, respectively. Similarly, the output of the operation unit OP16 is connected to two inputs of the operation unit OP18 and one input of the three-input operation unit OP17, respectively. The output of the operation unit OP17 is connected to one input of a three-input operation unit OP18.
[0112]
As described above, in the pixel engine (PXE) 13122 of FIG. 23, the output of the arithmetic unit OP1 is connected to the arithmetic units OP2, OP3, and OP4 via the forwarding path, and the arithmetic units OP2, OP3, and OP4 The output of OP1 can be referred to as a source operand.
The output of the operation unit OP2 is connected to the operation units OP3 and OP4 via a forwarding path, and the operation units OP3 and OP4 can refer to the output of the operation unit OP2 as a source operand.
The output of the operation unit OP3 is connected to the operation unit OP4 via a forwarding path, and the operation unit OP4 can refer to the output of the operation unit OP3 as a source operand.
The output of the computing element OP5 is connected to the computing elements OP6, OP7, OP8 via a forwarding path, and the outputs of the computing elements OP6, OP7, OP8 and OP5 can be referred to as source operands.
The output of the operation unit OP6 is connected to the operation units OP7 and OP8 via a forwarding path, and the operation units OP7 and OP8 can refer to the output of the operation unit OP6 as a source operand.
The output of the operation unit OP7 is connected to the operation unit OP8 via a forwarding path, and the operation unit OP8 can refer to the output of the operation unit OP7 as a source operand.
Similarly, the output of the operation unit OP11 is connected to the operation units OP12, OP13, and OP14 via a forwarding path, and the operation units OP12, OP13, and OP14 can refer to the output of the operation unit OP11 as a source operand.
The output of the operation unit OP12 is connected to the operation units OP13 and OP14 via a forwarding path, and the operation units OP13 and OP14 can refer to the output of the operation unit OP12 as a source operand.
The output of the operation unit OP13 is connected to the operation unit OP14 via a forwarding path, and the operation unit OP14 can refer to the output of the operation unit OP13 as a source operand.
The output of the operation unit OP15 is connected to the operation units OP16, OP17, and OP18 via a forwarding path, and the outputs of the operation units OP16, OP17, OP18, and the operation unit OP15 can be referred to as source operands.
The output of the operation unit OP16 is connected to the operation units OP17 and OP18 via a forwarding path, and the operation units OP17 and OP18 can refer to the output of the operation unit OP16 as a source operand.
The output of the operation unit OP17 is connected to the operation unit OP18 via a forwarding path, and the operation unit OP18 can refer to the output of the operation unit OP17 as a source operand.
The look-up tables LUT1, LUT2, LUT11, and LUT12 are, for example, arbitrarily definable RAM-LUTs, and can refer to a maximum of L (L: the number of tables that can be referred to simultaneously) in one context. The lookup tables LUT1, LUT2, LUT11, and LUT12 hold elementary functions such as sin / cos, for example.
[0113]
In the above configuration, as for the number of connections between the pixel engine (PXE) 13122 and the register unit (RGU) 13124, the number CN1 of connections from the pixel engine (PXE) 13122 to the crossbar circuit (IBX) 13125 is as follows. .
[0114]
(Equation 2)
CN1 = (the number of arithmetic units + the number of LUTs that can be referenced simultaneously) × 1
[0115]
The number of connections CN2 from the register unit (RGU) 13124 to the pixel engine (PXE) 13122 is as follows.
[0116]
(Equation 3)
CN2 = number of arithmetic units × 2 + number of LUTs that can be referred to simultaneously × 1
[0117]
The pixel engine (PXE) 13122 having the above configuration is set to a desired FIFO register of the register unit (RGU) 13124 via the crossbar circuit 13125 during, for example, graphics processing, and is directly input from the FIFO register. Operation result data (TR1, TG1, TB1, TA1) and (TR2, TG2, TB2, TA2) in the pixel operation processor (POP) group 13123, and set to a desired FIFO register of the register unit (RGU) 13124 by the rasterizer 1311 Then, based on the primary color (PC), secondary color (SC), and Fog coefficient (F) directly input from the FIFO register, an operation such as a pixel shader (Pixel @ Shader) is performed. , Color data (FR1, FG1, FB1) and mixing value (a blend value: FA1) Request.
The pixel engine (PXE) 13122 converts the data (FR1, FG1, FB1, FA1) into a predetermined POP of the pixel operation processor (POP) group 13123 or through a crossbar circuit 13125 and a register unit (RGU) 13124. The data is transferred to a separately provided light unit WU.
[0118]
The pixel operation processor (POP) group 13123 includes a plurality of POPs, which are functional units that perform highly parallel arithmetic processing utilizing the memory bandwidth, and in this embodiment, for example, as shown in FIG. 24, four POPs POP0 to POP3. Have.
Each POP includes a plurality of arithmetic units called POPE (Pixel \ Operation \ Processing \ Element) arranged in parallel. It also has an address generation function for the memory.
Since the pixel operation processor (POP) group 13123 and the cache are connected with a wide bandwidth and have a built-in address generation function for memory access, stream data that maximizes the operation capability of the operation unit Can be supplied.
[0119]
The pixel operation processor (POP) group 13123 performs, for example, the following processing during graphics processing.
For example, based on the values of (s1, t1, lod1) and (s2, t2, lod2) directly supplied from the graphics unit (GRU) 13121, the (u, v) address calculation for texture access is performed. , (U, v) coordinates of four neighbors for performing four neighbor filtering based on address data (ui, vi, lodi), that is, (u0, v0), (u1, v1), (u2, v2), (U3, v3) is calculated and supplied to the memory controller MC to read desired texel data from the memory module 132 to each POPE through, for example, a read-only cache RO #.
Further, the pixel operation processor (POP) group 13123 calculates a texture filter coefficient K based on data (uf, vf, lodf) for coefficient generation and supplies it to each POP.
Then, in each POP of the pixel operation processor (POP) group 13123, color data (TR, TG, TB) and a mixture value (blend value: TA) are obtained, and (TR, TG, TB, TA) is converted to a crossbar circuit 13125. , Via a register unit (RGU) 13124 to a pixel engine (PXE) 13122.
[0120]
On the other hand, the pixel operation processor (POP) group 13123 performs, for example, the following processing during image processing.
The pixel operation processor (POP) group 13123 is generated by, for example, the rasterizer 1311 and set in the register unit (RGU) 13124, and is directly supplied to the graphics unit (GRU) 13121 without passing through the crossbar circuit 13125. Based on the obtained source addresses (X1s, Y1s) and (X2s, Y2s), image data stored in the memory module 132 is read out, for example, via a read-only cache RO # and / or a read-write cache RW #, A predetermined calculation process is performed on the read data, and the calculation result is transferred to the write unit WU via the crossbar circuit 13125 and the register unit (RGU) 13124.
[0121]
A more specific configuration of the POP having the above-described functions will be described later in detail.
[0122]
The register unit (RGU) 13124 is a register file having a FIFO structure for storing stream data processed by each functional unit in the core 1312.
In addition, when a DFG must be divided into a plurality of sub-DFGs (Sub-DFGs) and executed due to hardware resources, it also functions as an intermediate value storage buffer between the sub-DFGs.
As shown in FIG. 23, the output of the FIFO register FREG in the register unit (RGU) 13124 and the input ports of the arithmetic units of the pixel engine (PXE) 13122 and the pixel operation processor (POP) group 13123 which are functional units are as follows. Corresponds one-to-one.
[0123]
The crossbar circuit 13125 realizes this connection switching so that the core 1312 can support various algorithms by changing the connection between the functional units according to the DFG.
As described above, the output of the FIFO register FREG in the register unit (RGU) 13124 and the input port of the functional unit are fixed and correspond one-to-one, but the output port of the functional unit and the FIFO in the register unit (RGU) 13124 The input of the register FREG is switched by the crossbar circuit 13125.
[0124]
FIG. 25 is a diagram illustrating a connection form between a POP (pixel operation processor) and a memory and a configuration example of the POP.
In the example of FIG. 25, each POP (0 to 3) has four arithmetic units POPE0 to POP3 arranged in parallel.
[0125]
In the present embodiment, the image data is stored in the memory module 132 (-0 to -3) of the local module 13 (-0 to -3). , POP (0 to 3) and the memory module 132, respectively.
In such a configuration, when performing pixel-level parallel operation processing in POP0 to POP3, there are the following two methods for accessing image data.
The first is a method of directly reading out image data stored in the memory module 132 and performing an operation.
A second method is to store a part of the image data stored in the memory module 132 required for the operation in the local cache 133 and read the data in the local cache 133 to perform the operation.
[0126]
In the present embodiment, the above-described second method is adopted.
In the local cache 133, read-only caches RO # 0 to RO # 3 and read / write caches RW # 0 to RW # 3 are arranged corresponding to POP0 to POP3 of POP (0 to 3), respectively.
[0127]
Further, the local cache 133 has selectors SEL1 to SEL12 as shown in FIG.
The selectors SEL1 to SEL4 select either read data of a 32-bit width from the corresponding read line port p (0) to p (3) of the memory module 132 or read data from another port to perform read / write. Output to caches RW # 0-RW # 3 and selectors SEL9-SEL12.
The selector SEL5 selects either the operation result of POP0 of POP or the processing result of the write unit WU and supplies it to the read / write cache RW # 0.
The selector SEL6 selects either the operation result of POP1 of POP or the processing result of the write unit WU and supplies it to the read / write cache RW # 1.
The selector SEL7 selects either the operation result of POP2 of POP or the processing result of the write unit WU and supplies it to the read / write cache RW # 2.
The selector SEL8 selects either the operation result of POP3 of POP or the processing result of the write unit WU and supplies it to the read / write cache RW # 3.
The selector SEL9 selects either the data from the selector SEL1 or the data transferred from the global module 12, and supplies the selected data to the read-only cache RO # 0.
The selector SEL10 selects either the data from the selector SEL2 or the data transferred from the global module 12, and supplies the selected data to the read-only cache RO # 1.
The selector SEL11 selects either the data from the selector SEL3 or the data transferred from the global module 12, and supplies the selected data to the read-only cache RO # 2.
The selector SEL12 selects either the data from the selector SEL4 or the data transferred from the global module 12, and supplies the selected data to the read-only cache RO # 3.
[0128]
Each of the POPs (0 to 3) includes a write unit WU as a fourth functional unit, a filter functional unit FFU, an output selection circuit OSLC, and an address generator in addition to the four arithmetic units POPE0 to POPE3 arranged in parallel. Has AG.
[0129]
In the case of graphics processing, the write unit WU reads source data from the register unit (RGU) 13124, specifically, color data (RGB) and mixed value data (A), depth data (Z), and read data. Calculations required for pixel writing of graphics processing such as α blending, various tests, and logical operations based on destination color data (RGB), mixed value data (A), and depth data (Z) from write cache RW # And the operation result is written back to the read / write cache RW #.
Further, in the case of image processing, the write unit WU receives the data of the operation result by the pixel operation processor (POP) group 13123, for example, the destination address directly input from a specific FIFO register of the register unit (RGU) 13124. (Xd, Yd) is stored in the memory module 132 via the read / write cache RW #.
[0130]
Although the example of FIG. 25 shows an example in which the write unit WU is provided in each POP, it is provided in only one POP and supplied to a plurality of divided local caches D133, or for two POPs. It can be configured in various modes, such as providing one and supplying it to the corresponding divided local cache D133, or providing it separately from the POP.
[0131]
The filter function unit FFU is provided in each of the POPE0 to POPE3 with an operation parameter set in the FIFO register of the register unit register (RGU) 13124, specifically, via the register unit (RGU) 13124 or the graphics unit (GRU). ) Based on the value of (s, t, lod) directly supplied from 13121, calculate (u, v) address and output address data (si, ti, lodi) to address generator AG, A texture filter coefficient K is calculated based on data (sf, tf, lodf) for coefficient generation, and the calculated filter coefficient is supplied to each of the corresponding POPE0 to POPE3.
[0132]
The address generator AG is a (u, v) coordinate of four neighbors for performing four neighbor filtering based on the address data (si, ti, lodi) supplied by the filter function unit FFU, that is, (u0, v0). , (U1, v1), (u2, v2), (u3, v3) and supply them to the memory controller MC.
[0133]
When using the read-only cache RO # as a local cache for data sent from the global bus, the memory controller MC calculates a physical address based on the (u, v) coordinates, and performs a cache hit and a transfer to the global bus. A request is sent, a read-only cache RO # is filled, and data is sent from the read-only cache RO # to a corresponding POP.
When using the read / write cache RW # as a write cache to the memory module 132, the memory controller MC calculates a physical address based on the destination address (Xd, Yd), and writes back to the cache and the memory module 132. Perform control.
[0134]
POPE0 receives a 32-bit width data read from read-only cache RO # 0 or read / write cache RW # 0 and an operation parameter (eg, filter coefficient) by filter function unit FFU and performs a predetermined operation (eg, addition). Then, the operation result is output to the next-stage POPE1. Further, POPE0 has an 8-bit × 4 output line OTL0 for outputting the predetermined operation result to the output selection circuit OSLC.
Further, the POPE0 is transferred to the crossbar circuit 13125, receives data set in the register unit (RGU) 13124, performs a predetermined operation, and divides the operation result via the selector SEL5 of the divided local cache D133 (0). Output to read / write cache RW # 0.
[0135]
POPE1 receives a 32-bit width data read from read-only cache RO # 1 or read / write cache RW # 1 and a calculation parameter by filter function unit FFU, and performs a predetermined calculation (for example, addition). And POPE0 to add the operation result and output it to the next-stage POPE2. Further, POPE1 has an 8-bit × 4 output line OTL1 for outputting the predetermined operation result to the output selection circuit OSLC.
Further, the POPE 1 receives the data set in the register unit (RGU) 13124, is transferred to the crossbar circuit 13125, performs a predetermined operation, and divides the operation result via the selector SEL6 of the divided local cache D133 (0). Output to read / write cache RW # 1.
[0136]
POPE2 receives a 32-bit width data read from read-only cache RO # 2 or read / write cache RW # 2 and a calculation parameter by filter function unit FFU, and performs a predetermined calculation (for example, addition). And POPE1 to add the operation result and output it to the next-stage POPE3. Further, POPE2 has an 8-bit × 4 output line OTL2 for outputting the predetermined operation result to the output selection circuit OSLC.
Further, the POPE 2 receives the data set in the register unit (RGU) 13124, is transferred to the crossbar circuit 13125, performs a predetermined operation, and divides the operation result via the selector SEL7 of the divided local cache D133 (0). Output to read / write cache RW # 2.
[0137]
POPE3 receives a 32-bit width data read from read-only cache RO # 3 or read / write cache RW # 3 and a calculation parameter by filter function unit FFU, and performs a predetermined calculation (for example, addition). And the operation result is added by POP2, and the operation result (total in one POP) is output to the output selection circuit OSLC through the 8-bit × 4 output line OTL3.
Further, the POPE 3 receives the data set in the register unit (RGU) 13124 which is transferred to the crossbar circuit 13125, performs a predetermined operation, and divides the operation result via the selector SEL8 of the divided local cache D133 (0). Output to read / write cache RW # 3.
[0138]
FIG. 26 is a circuit diagram illustrating a specific configuration example of POPE (0 to 3) according to the present embodiment.
As shown in FIG. 26, the present POPE includes multiplexers (MUX) 401 to 405, an adder / subtractor (addsub) 406, a multiplier (mul) 407, an adder / subtractor (addsub) 408, and an accumulation register 409.
[0139]
The multiplexer 401 stores the data from the register unit (RGU) 13124, the operation parameters from the filter function unit FFU, the data read from the read-only cache RO # (0-3), or the data read from the read / write cache RW # (0-3). One of them is selected and supplied to the adder / subtractor 406.
[0140]
The multiplexer 402 selects one of data read by the register unit (RGU) 13124, read-only cache RO # (0-3), or read-write cache RW # (0-3). , And to the adder / subtractor 406.
[0141]
The multiplexer 403 stores the data read by the register unit (RGU) 13124, the operation parameters by the filter function unit FFU, the data read from the read-only cache RO # (0-3), or the data read from the read / write cache RW # (0-3). One of them is selected and supplied to the multiplier 407.
[0142]
The multiplexer 404 selects one of the operation result of the preceding POPE (0 to 2) or the output data of the accumulation register 409 and supplies the selected result to the adder / subtractor 408.
[0143]
The multiplexer 405 stores the data read by the register unit (RGU) 13124, the operation parameter by the filter function unit FFU, the data read from the read-only cache RO # (0-3), or the data read from the read / write cache RW # (0-3). One of them is selected and supplied to the adder / subtractor 408.
[0144]
The adder / subtractor 406 adds (subtracts) the selection data of the multiplexer 401 and the selection data of the multiplexer 402 and outputs the result to the multiplier 407.
The multiplier 407 multiplies the output data of the adder / subtractor 406 and the data selected by the multiplexer 403 and outputs the result to the adder / subtractor 408.
The adder / subtracter 408 adds (subtracts) the multiplier 407 and the output data, the selection data of the multiplexer 404, and the selection data of the multiplexer 405, and outputs the result to the accumulation register 409.
Then, the data held in the integration register 409 is output to the output selection circuit OSLC and the next-stage POPE (1 to 3) as the operation result of each POPE.
[0145]
The output selection circuit OSLC has a function of selecting any of the operation data transferred from the output lines OTL0 to OTL3 of the respective POPE0 to P0PE3 and outputting the selected operation data to the crossbar circuit 13125.
In this embodiment, the output selection circuit OSLC is configured to select the operation data transferred on the output line OTL3 of POP3 that outputs the total in one POP, and output the selected operation data to the crossbar circuit 13125.
The operation data output to the crossbar circuit 13125 is set in the register unit 13124, and the set data is directly supplied to a predetermined operation unit of the pixel engine 13122 without passing through the crossbar circuit 13125.
[0146]
As shown in FIG. 27, the address generator AG transfers data from the memory module 132 simultaneously in one column (for four POPs), and reads each of the divided local caches D133 (0) to D133 (3). Since the access to the only caches RO # 0 to RO # 3 or the read / write caches RW # 0 to RW # 3 is performed independently, each of the read only caches RO # 0 to RO # 3 or the read / write cache RW # The cache addresses CADR0 to CADR3 for reading the element data read in parallel from the ports p (0) to p (3) of the memory module 132 to the corresponding POPE0 to POPE3 are respectively assigned to 0 to RW # 3. Generate and supply.
The address generator AG supplies, for example, the operation result OPR0 of POPE0 to POPE1 at the timing when the operation of POPE1 ends, the operation result of POPE1 (the result of adding the operation result OPR0 of POP0) OPR1 ends the operation of POPE2 Read-only caches RO # 0 to ROPE0 so that the operation result of POPE2 (the result obtained by adding the operation result OPR1 of POPE1) OPR2 is supplied to POPE3 at the timing when the operation of POPE3 ends. Cache addresses CADR0 to CADR3 are supplied to RO # 3 or read / write caches RW # 0 to RW # 3 at a predetermined timing. For example, when the number of element data supplied to each of POPE0 to POPE3 is the same, and element data is sequentially added at each of POPE0 to POPE3, address supply is performed by shifting the address supply timing by one address in order.
As a result, arithmetic operations without errors can be efficiently performed. That is, in the core 1312 according to the present embodiment, an improvement in operation efficiency is achieved.
[0147]
Next, template matching processing, which is one of two-dimensional image processing using the pixel operation processor group 13123 and the local cache 133 having the above configuration, will be described.
[0148]
FIG. 28 is a diagram for describing template matching.
The image shown in FIG. 28A is a target image OBIM, in which a frame indicated by MP is a previous frame matching position, and a frame indicated by SRC is a search range. The target image OBIM changes during the matching process until it matches within the search range SRC. The target image OBIM is stored in the memory module 132, for example.
The image shown in FIG. 28B is a reference image RFIM, which does not change at the time of matching and is stored in, for example, the memory module 132 or supplied from the global module 12 to the read-only cache RO # via the global bus. . In the present embodiment, the size of the reference image RFIM is 16 pixels × 16 pixels.
FIG. 28C shows a mask.
[0149]
When performing template matching, it is necessary to obtain the absolute value difference between the reference image RFIM and the target image OBIM by one arithmetic unit POPE. Therefore, two caches per one POPE, that is, as shown in FIG. Two caches, read-only cache RO # (0-3) and read / write cache RW # (0-3), provided corresponding to POPE0-POPE3 are used.
For example, as shown in FIG. 29, a reference image RFIM of 16 × 16 pixels that does not change and does not need to be refilled is stored in the read-only cache RO # (0 to 3). On the other hand, the target image OBIM that has changed and needs to be refilled as needed (for example, stored in the eDRAM) is stored in the read / write cache RW # (0 to 3).
[0150]
In the case of this template matching, in the POPE shown in FIG. 26, for example, the target image OBIM read from the read / write cache RW # (0-3) is selected by the multiplexer 401, and the read only cache RO # (0 3), the reference image RFIM read out is selected, and the difference between the two images is obtained by the adder / subtractor 406.
The multiplexer 403 selects an operation parameter (coefficient) by the filter function unit FFU, and the multiplier 407 multiplies the operation result of the adder / subtractor 406 by a coefficient.
Then, the adder / subtracter 408 multiplies and sums the operation results of the own stage selected by the multiplexer 404, and outputs the total result of the own stage from the first stage POPE0 to the next stage POPE1.
In addition, in POPE1 to POPE3 except for the first stage POPE0, the sum result of the previous stage POPE0 to 2 selected by the multiplexer 404 is added to the sum result of the own stage and output to the next stage POPE2 to POPE3.
[0151]
Next, an operation of template matching in which the pixel processing processor group 13123 performs calculation processing based on data in the memory and further performs calculation in the pixel engine 13122 will be described with reference to FIGS.
[0152]
Step ST51
First, in step ST51, the reference image OBIM is simultaneously transferred from the memory module (eDRAM) to the read / write caches RW # 0 to RO # 3 of the local cache 133 in one row (for four POPs). Further, the reference image RFIM is transferred to each of the read-only caches RO # 0 to RO # 3 via the global bus.
Next, as shown in FIGS. 32 (A), (C), (E), and (G), the address generator AG shifts each address to POPE0 to POP3 within one POP independently by each cache. Thus, cache addresses CADR0 to CADR3 are supplied.
Thus, the 16 element data of the reference image RFIM and the 16 element data of the target image OBIM are sequentially read out to each of POP0 to POP3 of POP0 to POP3, respectively.
[0153]
For example, cache addresses CADR00 to CADR0F are sequentially applied to read only cache RO # 0 and read / write cache RW # 0 of divided local cache 133 (0), and data 00 to 0F for one column is stored in POP0 of POP0 accordingly. Each is read.
Similarly, the cache addresses CADR10 to CADR1F are sequentially given to the read only cache RO # 1 and the read / write cache RW # 1 of the divided local cache 133 (0), and accordingly, one column of data 10 to POP1 of POP0. 1F is read out.
The cache addresses CADR20 to CADR2F are sequentially given to the read only cache RO # 2 and the read / write cache RW # 2 of the divided local cache 133 (0), and the data 20 to 2F for one column are respectively stored in POP2 of POP0. Is read.
The cache addresses CADR30 to CADR3F are sequentially given to the read only cache RO # 3 and the read / write cache RW # 3 of the divided local cache 133 (0), and the data 30 to 3F for one column are respectively stored in the POP3 of POP0. Is read.
[0154]
The cache addresses CADR40 to CADR4F are sequentially given to the read-only cache RO # 0 and the read / write cache RW # 0 of the divided local cache 133 (1), and one column of data 40 to 4F is respectively assigned to POP0 of POP1. Is read.
Similarly, cache addresses CADR50 to CADR5F are sequentially given to read-only cache RO # 1 and read / write cache RW # 1 of divided local cache 133 (1), and data 50 to CADR1 of POP1 are stored in POP1 of POP1 accordingly. 5F are read out.
The cache addresses CADR60 to CADR6F are sequentially given to the read-only cache RO # 2 and the read / write cache RW # 2 of the divided local cache 133 (1), and the data 60 to 6F for one column are respectively stored in POP2 of POP1. Is read.
The cache addresses CADR70 to CADR7F are sequentially given to the read only cache RO # 3 and the read / write cache RW # 3 of the divided local cache 133 (1), and the data 70 to 7F for one column are respectively stored in POP3 of POP1. Is read.
[0155]
The cache addresses CADR80 to CADR8F are sequentially given to the read only cache RO # 0 and the read / write cache RW # 0 of the divided local cache 133 (2), and the data 80 to 8F for one column are respectively stored in POP0 of POP2. Is read.
Similarly, cache addresses CADR90 to CADR9F are sequentially given to read-only cache RO # 1 and read / write cache RW # 1 of divided local cache 133 (2), and accordingly, one row of data 90 to CADR9F is assigned to POP1 of POP2. 9F is read out.
The cache addresses CADRA0 to CADRAF are sequentially given to the read-only cache RO # 2 and the read / write cache RW # 2 of the divided local cache 133 (2), and one column of data A0 to AF is respectively stored in POP2 of POP2. Is read.
The cache addresses CADRB0 to CADRBF are sequentially given to the read only cache RO # 3 and the read / write cache RW # 3 of the divided local cache 133 (2), and one column of data B0 to BF is respectively assigned to POP3 of POP2. Is read.
[0156]
The cache addresses CADRC0 to CADRCF are sequentially given to the read-only cache RO # 0 and the read / write cache RW # 0 of the divided local cache 133 (3), and one column of data C0 to CF is respectively stored in POP0 of POP3. Is read.
Similarly, cache addresses CADRD0 to CADRDF are sequentially given to read only cache RO # 1 and read / write cache RW # 1 of divided local cache 133 (3), and one column of data D0 to POP1 of POP3 in response thereto. The DF is read out.
The cache addresses CADRE0 to CADREF are sequentially given to the read-only cache RO # 2 and the read / write cache RW # 2 of the divided local cache 133 (3), and the data E0 to EF for one column are respectively stored in POP2 of POP3. Is read.
The cache addresses CADRF0 to CADRFF are sequentially given to the read only cache RO # 3 and the read / write cache RW # 3 of the divided local cache 133 (3), and data F0 to FF for one column are respectively stored in POP3 of POP3. Is read.
[0157]
Step ST52
In step ST52, as described above, data obtained by multiplying the difference between the reference image RFIM and the target image OBIM by a coefficient in each of POP0 to POP3 of each POP (0 to 3) is added for one column of one element (16). Is done.
Specifically, in POP0 of POP0, data 00 to 0F are sequentially added as shown in FIG. 32B, and the operation result OPR0 is output to POP1.
In POPE1 of POP0, as shown in FIG. 32D, data 10 to 1F are sequentially added.
In POPE2 of POP0, data 20 to 2F are sequentially added as shown in FIG.
In POPE3 of POP0, data 30 to 3F are sequentially added as shown in FIG.
The same applies to the other POP1 to POP3.
[0158]
Step ST53
In step ST53, the operation results of each of POP0 to POP3 of each of POPs (0 to 3) are added to obtain an addition result of 16 × 4 elements.
Specifically, as shown in FIGS. 32 (B), (D) and FIG. 36, the operation result OPR0 of POP0 of POP0 is output to POP1.
32 (D), (F) and FIG. 34, the operation result OPR0 of POP0 of POP0 is added to POPE1 of POP0, and the operation result OPR1 is output to POPE2.
32 (F), (H), and FIG. 34, the operation result OPR1 of the POP0 POP1 is added to the operation result OPR1 of the POP0, and the operation result OPR2 is output to the POPE3.
Then, in POPE3 of POP0, as shown in FIG. 32H, the operation result OPR2 of POP2 of POP0 is added to its own operation result, and the operation result OPR3 is output to the output selection circuit OSLC.
The same applies to the other POP1 to POP3.
[0159]
Step ST54
In step ST54, the total operation result OPR3 is transferred from the output selection circuit OSLC of each of POP0 to POP3 to the register unit (RGU) 13124 via the crossbar circuit 13125.
For example, as shown in FIG. 33, the total operation result OPR3 of POP3 of POP0 is stored in the FIFO register FREG1 of the register unit (RGU) 13124 via the crossbar circuit 13125.
The total operation result OPR3 of POPE3 of POP1 is stored in the FIFO register FREG2 of the register unit (RGU) 13124 via the crossbar circuit 13125.
The total operation result OPR3 of POPE3 of POP2 is stored in the FIFO register FREG3 of the register unit (RGU) 13124 via the crossbar circuit 13125.
The total operation result OPR3 of POPE3 of POP3 is stored in the FIFO register FREG4 of the register unit (RGU) 13124 via the crossbar circuit 13125.
[0160]
Step ST55
In step ST55, the total operation result of POP0 and POP1 set in the FIFO registers FREG1 and FREG2 of the register unit (RGU) 13124 is added by the first adder ADD1 of the pixel engine (PXE) 13122. Is stored in the FIFO register FREG5 of the register unit (RGU) 13124 via the crossbar circuit 13125.
Further, the total operation result of POP2 and POP3 set in the FIFO registers FREG3 and FREG4 of the register unit (RGU) 13124 is added by the second adder ADD2 of the pixel engine (PXE) 13122, and the operation result is added to the crossbar. The data is stored in the FIFO register FREG6 of the register unit (RGU) 13124 via the circuit 13125.
Then, the operation results of the first and second adders ADD1 and ADD2 set in the FIFO registers FREG5 and FREG6 of the register unit (RGU) 13124 are added by the third adder ADD3 of the stream processing engine (SPE) 13143. Is done.
[0161]
Step ST56
In step ST56, as shown in FIG. 32 (P), the addition result of the third adder ADD3 of the pixel engine (PXE) 13122 is output as a series of calculation results.
[0162]
FIG. 35 is a diagram showing an outline of operation including a pixel engine (PXE) 13122 of a core, a pixel operation processor (POP) group 13123, a register unit (RGU) 13124, and a memory part in the processing unit according to the present embodiment.
[0163]
In FIG. 35, the broken line indicates the flow of address data, the dashed line indicates the flow of read data, and the solid line indicates the flow of write data.
In the register unit (RGU) 13124, FREGA1 and FREGA2 denote FIFO registers used for an address system, FREGR denotes a FIFO register used for read data, and FREGW denotes a FIFO register used for write data.
[0164]
In the example of FIG. 35, for example, source (reading) address data generated by the rasterizer 1311 is set in the FIFO registers FREGA1 and FREGA2 of the register unit (RGU) 13124 via the crossbar circuit 13125.
The address data set in the FIFO register FREGA1 is directly supplied to the address generator AG1 of the pixel operation processor (POP) 13123 without passing through the crossbar circuit 13125, for example. The address of the data to be read is generated by the address generator AG1, and the desired data read from the memory module 132 to the read-only cache 1331 is supplied to each processor (POPE) of the pixel processor (POP) 13123 based on the generated address. Is done.
[0165]
The operation result of each operation unit (POPE) of the pixel operation processor (POP) 13123 is set in the FIFO register FREGR of the register unit (RGU) 13124 via the crossbar circuit 13125.
The data set in the FIFO register FREGR is directly supplied to each operation unit OP of the pixel engine (PXE) 13122 without passing through the crossbar circuit 13125.
Then, the operation result of each operation unit OP of the pixel engine (PXE) 13122 is set in the FIFO register FREGW of the register unit (RGU) 13124 via the crossbar circuit 13125.
The data set in the FIFO register FREGW is supplied to each operation unit (POPE) of the pixel operation processor (POP) 13123.
[0166]
Further, destination (write) address data generated by the rasterizer 1311 is set in the FIFO register FREGA2 of the register unit (RGU) 13124 via the crossbar circuit 13125.
Then, the address data set in the FIFO register FREGA2 is supplied directly to the address generator AG2 of the pixel operation processor (POP) 13123 without passing through the crossbar circuit 13125. The address of the data to be written is generated by the address generator AG2, and based on this, the operation result of each operation unit (POPE) of the pixel operation processor (POP) 13123 is written to the read / write cache 1332 and further written to the memory module 132. .
[0167]
Note that, in the example of FIG. 35, the read / write cache 1332 is described to perform only writing, but reading is also performed by the same operation as in the case of the above-described read-only cache 1331.
[0168]
Next, an operation example in the case of graphics processing and image processing in the processing unit 131 (-0 to -3) having the above configuration will be described with reference to the drawings.
[0169]
First, the graphics processing when there is no dependent texture will be described with reference to FIGS. 36 and 37.
[0170]
In this case, the rasterizer 1311 receives the parameter data broadcast from the global module 12 and determines, for example, whether or not the triangle is the area in charge of itself. , Each pixel data is generated and supplied to the core 1312.
Specifically, in the rasterizer 1311, window coordinates (X, Y, Z), primary colors (PC; Rp, Gp, Bp, Ap), secondary colors (SC; Rs, Gs, Bs, As), and Fog coefficients ( f), various kinds of pixel data of texture coordinates and various vectors (V1x, V1y, V1z) and (V2x, V2y, V2z) are generated.
[0171]
Then, the generated window coordinates (X, Y, Z) are passed through a specific FIFO register of the register unit (RGU) 13124, directly into the pixel operation processor (POP) group 13123, or a light provided separately. It is supplied to the unit WU.
Further, the generated two sets of texture coordinate data and various vectors (V1x, V1y, V1z) and (V2x, V2y, V2z) are passed through the FIFO unit of the crossbar circuit 13125 and the register unit (RGU) 13124 to the graphics unit ( GRU) 12121.
Further, the generated primary color (PC), secondary color (SC), and Fog coefficient (F) are supplied to a pixel engine (PXE) 13122 through a crossbar circuit 13125 and a FIFO register of a register unit (RGU) 13124.
[0172]
The graphics unit (GRU) 13121 performs perspective collection, LOD (Level) based on the supplied texture coordinate data, various vectors (V1x, V1y, V1z), and (V2x, V2y, V2z).
The calculation of the mipmap (MIPMAP) level by the calculation of of @ Detail, the selection of the surface of the cube map (CubeMap), and the calculation of the normalized texel coordinates (s, t) are performed.
Then, two sets of data (s1, t1, lod1), (s2, t2, lod2) generated by the graphics unit (GRU) 13121 including, for example, normalized texel coordinates (s, t) and LOD data (lod). ) Are directly supplied to the pixel operation processor (POP) group 13123 via individual wirings without passing through the crossbar circuit 13125, for example.
[0173]
In the pixel operation processor (POP) group 13123, as shown in FIG. 37, (s1, t1, lod1), (s2, t2, lod2) directly supplied from the graphics unit (GRU) 13121 in the filter function unit FFU. ), The (u, v) address calculation for texture access is performed, the address data (ui, vi, lodi) is supplied to the address generator AG, and the data (uf, vf, lodf) are supplied to the coefficient generator COF.
[0174]
The address generator AG receives the address data (ui, vi, lodi) and performs (N, V) coordinates of four neighbors for performing four-neighbor filtering, that is, (u0, v0), (u1, v1). , (U2, v2), (u3, v3) are calculated and supplied to the memory controller MC.
Thereby, desired texel data is read out from memory module 132 to each POP of pixel operation processor (POP) group 13123 through, for example, read-only cache RO #.
Further, the coefficient generator COF receives the data (uf, vf, lodf), calculates a texture filter coefficient K (0 to 3), and supplies the texture filter coefficient K to each corresponding POPE of the pixel operation processor (POP) group 13123. .
Then, in each POP of the pixel operation processor (POP) group 13123, color data (TR, TG, TB) and a mixed value (blend value: TA) are obtained, and two sets of data (TR1, TG1, TB1, TA1) are obtained. And (TR2, TG2, TB2, TA2) are transferred through the crossbar circuit 13125 and set in a predetermined FIFO register of the register unit (RGU) 13124, and this setting data is directly transmitted without passing through the crossbar circuit 13125. It is supplied to a pixel engine (PXE) 13122.
[0175]
In the pixel engine (PXE) 13122, the data (TR1, TG1, TB1, TA1) and (TR2, TG2, TB2, TA2) by the pixel operation processor (POP) group 13123, the primary color (PC) by the rasterizer 1311 and the secondary Based on the color (SC) and the Fog coefficient (F), for example, an operation of Pixel @ Shader is performed to obtain color data (FR1, FG1, FB1) and a mixed value (blend value: FA1). FG1, FB1, FA1) are transferred to the crossbar circuit 13125 and set in a predetermined FIFO register of the register unit (RGU) 13124, and the setting data is directly sent to the pixel operation processor ( POP) group 1 123 is supplied to a predetermined POP within or separately provided light unit WU of.
[0176]
In the write unit WU, based on the window coordinates (X, Y, Z) by the rasterizer 1311, for example, the destination color data (RGB), the mixed value data (A), and the depth data (A) from the memory module 132 through the read / write cache RW #. Z) is read.
In the write unit WU, the data (FR1, FG1, FB1, FA1) by the pixel engine (PXE) 13122, the destination color data (RGB) read from the memory module 132 through the read / write cache RW #, and the mixed value data ( Based on A) and the depth data (Z), calculations required for pixel writing of graphics processing such as α blending, various tests, and logical operations are performed, and the calculation results are written back to the read / write cache RW #.
[0177]
Next, the graphics processing when there is a dependent texture will be described with reference to FIGS.
[0178]
In this case, in the rasterizer 1311, window coordinates (X, Y, Z), primary colors (PC; Rp, Gp, Bp, Ap), secondary colors (SC; Rs, Gs, Bs, As), and Fog coefficients (f) , Various pixel data of the texture coordinates (V1x, V1y, V1z) are generated.
[0179]
Then, the generated window coordinates (X, Y, Z) are directly supplied to the pixel operation processor (POP) group 13124 through a specific FIFO register of the register unit (RGU) 13124.
Further, the generated texture coordinates (V1x, V1y, V1z) are supplied to the graphics unit (GRU) 12121 through the crossbar circuit 13125 and the FIFO register of the register unit (RGU) 13124.
Further, the generated primary color (PC), secondary color (SC), and Fog coefficient (F) are supplied to a pixel engine (PXE) 13122 through a crossbar circuit 13125 and a FIFO register of a register unit (RGU) 13124.
[0180]
In the graphics unit (GRU) 13121, based on the supplied texture coordinate (V1x, V1y, V1z) data, perspective collection, calculation of a mipmap (MIPMAP) level by LOD calculation, surface selection of a cube map (CubeMap), Calculation processing of the normalized texel coordinates (s, t) is performed.
Then, a set of data (s1, t1, lod1) including, for example, normalized texel coordinates (s, t) and LOD data (lod) generated by the graphics unit (GRU) 13121 is output to the crossbar circuit 13125, for example. And is directly supplied to a pixel operation processor (POP) group 13123 without passing through the POP.
[0181]
In the pixel operation processor (POP) group 13123, as shown in FIG. 37, the texture is based on the value of (s1, t1, lod1) directly supplied from the graphics unit (GRU) 13121 in the filter function unit FFU. The (u, v) address calculation for access is performed, the address data (ui, vi, lodi) is supplied to the address generator AG, and the data (uf, vf, lodf) is calculated for the coefficient by the coefficient generator. Supplied to COF.
[0182]
The address generator AG receives the address data (ui, vi, lodi) and performs (N, V) coordinates of four neighbors for performing four-neighbor filtering, that is, (u0, v0), (u1, v1). , (U2, v2), (u3, v3) are calculated and supplied to the memory controller MC.
Thereby, desired texel data is read out from memory module 132 to each POP of pixel operation processor (POP) group 13123 through, for example, read-only cache RO #.
Further, the coefficient generator COF receives the data (uf, vf, lodf), calculates a texture filter coefficient K (0 to 3), and supplies it to each POPE of the pixel operation processor (POP) group 13123.
Then, in each POP of the pixel operation processor (POP) group 13123, color data (TR, TG, TB) and a mixture value (blend value: TA) are obtained, and the data (TR1, TG1, TB1, TA1) are crossed. The bar circuit 13125 is transferred and set in a predetermined FIFO register of a register unit (RGU) 13124, and the setting data is directly supplied to the pixel engine (PXE) 13122 without passing through the crossbar circuit 13125.
[0183]
In the pixel engine (PXE) 13122, the data (TR1, TG1, TB1, TA1) by the pixel operation processor (POP) group 13123, and the primary color (PC), secondary color (SC), and Fog coefficient (F) by the rasterizer 1311 For example, Pixel @ Shader is calculated based on the above, texture coordinates (V2x, V2y, V2z) are generated and supplied to the graphics unit (GRU) 13121 via the crossbar circuit 13125 and the register unit (RGU) 13124. You.
[0184]
In the graphics unit (GRU) 13121, based on the supplied texture coordinate (V2x, V2y, V2z) data, perspective collection, calculation of a mipmap (MIPMAP) level by LOD calculation, surface selection of a cube map (CubeMap), Calculation processing of the normalized texel coordinates (s, t) is performed.
Then, the data (s2, t2, lod2) including, for example, the normalized texel coordinates (s, t) and the LOD data (lod) generated by the graphics unit (GRU) 13121 does not pass through the crossbar circuit 13125, for example. It is directly supplied to a pixel operation processor (POP) group 13123.
[0185]
In the pixel operation processor (POP) group 13123, as shown in FIG. 37, the texture is based on the value of (s2, t2, lod2) directly supplied from the graphics unit (GRU) 13121 in the filter function unit FFU. The (u, v) address calculation for access is performed, the address data (ui, vi, lodi) is supplied to the address generator AG, and the data (uf, vf, lodf) is calculated for the coefficient by the coefficient generator. Supplied to COF.
[0186]
The address generator AG receives the address data (ui, vi, lodi) and performs (N, V) coordinates of four neighbors for performing four-neighbor filtering, that is, (u0, v0), (u1, v1). , (U2, v2), (u3, v3) are calculated and supplied to the memory controller MC.
Thereby, desired texel data is read out from memory module 132 to each POP of pixel operation processor (POP) group 13123 through, for example, read-only cache RO #.
Further, the coefficient generator COF receives the data (uf, vf, lodf), calculates a texture filter coefficient K (0 to 3), and supplies it to each POPE of the pixel operation processor (POP) group 13123.
Then, in each POP of the pixel operation processor (POP) group 13123, color data (TR, TG, TB) and a mixture value (blend value: TA) are obtained, and the data (TR2, TG2, TB2, TA2) are crossed. The bar circuit 13125 is transferred and set in a predetermined FIFO register of a register unit (RGU) 13124, and the setting data is directly supplied to the pixel engine (PXE) 13122 without passing through the crossbar circuit 13125.
[0187]
In the pixel engine (PXE) 13122, data (TR2, TG2, TB2, TA2) by the pixel operation processor (POP) group 13123, and a primary color (PC), a secondary color (SC), and a Fog coefficient (F) by the rasterizer 1311 , A predetermined filtering operation such as 4-neighbor interpolation is performed, color data (FR1, FG1, FB1) and a mixture value (blend value: FA1) are obtained, and the data (FR1, FG1, FB1, FA1) are obtained. ) Is transferred to the crossbar circuit 13125 and set in a predetermined FIFO register of the register unit (RGU) 13124, and this set data is directly transmitted to the pixel operation processor (POP) group 13123 without passing through the crossbar circuit 13125. Within a given POP or separately It is provided by supplying to the light unit WU.
[0188]
In the write unit WU, based on the window coordinates (X, Y, Z) by the rasterizer 1311, for example, the destination color data (RGB), the mixed value data (A), and the depth data (A) from the memory module 132 through the read / write cache RW #. Z) is read.
Then, in the write unit WU, data (FR1, FG1, FB1, FA1) by the pixel engine (PXE) 13122, and destination color data (RGB) and mixed value data (A) read from the memory module 132 through the read / write cache RW #. ) And the depth data (Z), calculations required for pixel writing of graphics processing such as α blending, various tests, and logical operations are performed, and the calculation results are written back to the read / write cache RW #.
[0189]
Next, image processing will be described.
[0190]
First, the operation in the case of performing the SAD (Summed Absolute \ Difference) processing as shown in FIG. 39 will be described with reference to FIG.
[0191]
In the SAD processing, for one block (X1s, Y1s) of the original image ORIM as shown in FIG. 39A, the search rectangular area SRGN of the reference image RFIM as shown in FIG. While shifting, the SAD (absolute value difference) in the corresponding block BLK is obtained.
Among them, the position (X2s, y2s) and the SAD value of the block having the minimum SAD are stored in (Xd, Yd) as shown in FIG.
(X1s, Y1s) is set as a context in a register in the POP from an upper position (not shown).
[0192]
In this case, the source address and the image processing result for reading the reference image data from the memory module 132 (−0 to −3) output from, for example, a higher-level device (not shown) via the global module 12 are transmitted to the rasterizer 1311. Commands and data necessary for generating a destination address for writing, for example, width and height (Ws, Hs) data and block size (Wbk, Hbk) data of the search rectangular area SRGN are input.
The rasterizer 1311 generates a source address (X2s, Y2s) of the reference image RFIM stored in the memory module 132 based on the input data, and generates a destination address (X2s, Y2s) for storing the processing result in the memory module 132. Xd, Yd) are generated.
[0193]
In the generated destination address (Xd, Yd), the supply line of the window coordinates (X, Y, Z) at the time of graphics processing is shared, and directly through a specific FIFO register of the register unit (RGU) 13124. It is supplied to the light unit WU of the pixel operation processor (POP) group 13124.
The source address (X2s, Y2s) of the generated reference image RFIM is supplied to the graphics unit (GRU) 12121 through the crossbar circuit 13125 and the FIFO register of the register unit (RGU) 13124. The source address (X2s, Y2s) is supplied directly to the pixel operation processor (POP) group 13123 without passing through the crossbar circuit 13125, for example, without passing through the graphics unit (GRU) 12121.
[0194]
In the pixel operation processor (POP) group 13123, based on the supplied source addresses (X1s, Y1s) and (X2s, Y2s), the memory module 132 receives the read-only cache RO # and the read / write cache RW #, for example. The stored data of the original image ORIM and the stored data of the reference image RFIM are read.
Here, the coordinates of the original image ORIM are set in a register as a context. The coordinates of the reference image RFIM are given, for example, as the coordinates of a sub-block that is assigned to each of the four POPs.
Then, the pixel operation processor (POP) group 13123 shifts the SAD in the corresponding sub-block BLK with respect to one block (X1s, Y1s) of the original image ORIM by shifting the search rectangular area SRGN of the reference image RFIM by one pixel. (Absolute value difference) is obtained from time to time.
Then, the position (X2s, y2s) of each sub-block and each SAD value are transferred to the crossbar circuit 13125 and set in a predetermined FIFO register of the register unit (RGU) 13124, and the setting data is stored in the crossbar circuit 13125. Is directly transferred to the pixel engine (PXE) 13122 without going through.
[0195]
In the pixel engine (PXE) 3122, the SAD of the entire block is totaled, and the block position (X2s, y2s) and the SAD value are transferred to the crossbar circuit 13125 and set in a predetermined FIFO register of the register unit (RGU) 13124. The setting data is directly transferred to the write unit WU without passing through the crossbar circuit 13125.
[0196]
In the write unit WU, the pixel engine (PXE) 13122 stores the block position (X2s, y2s) and the SAD value in the destination address (Xd, Yd) by the rasterizer 1311.
In this case, the SAD value read from the memory module 132 to the read / write cache RW # and the SAD value from the pixel engine (PXE) 13122 are used, for example, by using a function (Z comparison) for performing hidden surface removal (Hidden \ Surface \ Removal). Are compared.
As a result of the comparison, when the SAD value of the pixel engine (PXE) 13122 is smaller than the stored value, the block position (X2s, y2s) and the SAD value of the pixel engine (PXE) 13122 are converted to the destination address (Xd). , Yd) are written (updated) through the read / write cache RW #.
[0197]
Next, an operation in the case of performing a convolution filter (Convolution @ Filter) process as shown in FIG. 41 will be described with reference to FIG.
[0198]
In the convolution filter processing, for each pixel (X1s, Y1s) of the target image OBIM as shown in FIG. 41 (A), peripheral pixels of a filter kernel size are read, and a result obtained by multiplying by a filter coefficient is added. The result is stored in the destination address (Xd, Yd) as shown in FIG.
The storage address of the filter kernel coefficient is set as a context in a register in the POP.
[0199]
In this case, a source address and an image for reading image data (pixel data) from the memory module 132 (-0 to -3) output from, for example, a higher-level device (not shown) to the rasterizer 1311 via the global module 12 are output. Commands and data necessary for generating a destination address for writing a processing result, for example, filter kernel size data (Wk, Hk) are input.
The rasterizer 1311 generates a source address (X1s, Y1s) of the target image OBIM stored in the memory module 132 based on the input data, and generates a destination address (X1s, Y1s) for storing the processing result in the memory module 132. Xd, Yd) are generated.
[0200]
In the generated destination address (Xd, Yd), the supply line of the window coordinates (X, Y, Z) at the time of graphics processing is shared, and directly through a specific FIFO register of the register unit (RGU) 13124. It is supplied to the light unit WU of the pixel operation processor (POP) group 13124.
The source address (X1s, Y1s) of the generated target image OBIM is supplied to the graphics unit (GRU) 12121 through the crossbar circuit 13125 and the FIFO register of the register unit (RGU) 13124. The source address (X1s, Y1s) is supplied directly to the pixel operation processor (POP) group 13123 without passing through the crossbar circuit 13125, for example, without passing through the graphics unit (GRU) 12121.
[0201]
In the pixel operation processor (POP) group 13123, peripheral pixels having a kernel size enabled in the memory module 132 are read out, for example, via the read-only cache RO # based on the supplied source addresses (X1s, Y1s). .
In a pixel operation processor (POP) group 13123, a predetermined filter coefficient is multiplied by the read data, and these are added together. The result is color data (R, G, B) and mixed value data (A ) Is transferred to the write unit WU via the crossbar circuit 13125 and the register unit (RGU) 13124.
[0202]
In the write unit WU, data from the pixel operation processor (POP) group 13123 is stored in the destination address (Xd, Yd) via the read / write cache RW #.
[0203]
Finally, the operation according to the system configuration of FIG. 3 will be described.
Here, the processing of the texture system will be described.
[0204]
First, in the SDC 11, when each vertex data of three-dimensional coordinates, normal vectors, and texture coordinates is input, an operation is performed on the vertex data.
Next, various parameters required for rasterization are calculated.
Then, in the SDC 11, the calculated parameters are broadcast to all the local modules 13-0 to 13-3 via the global module 12.
In this process, the broadcasted parameter is passed to each of the local modules 13-0 to 13-3 via the global module 12 using a channel different from a cache fill described later. However, it does not affect the contents of the global cache.
[0205]
In each of the local modules 13-0 to 13-3, the following processing is performed in the processing units 131-0 to 131-3.
That is, in the processing units 131 (-0 to 3), upon receiving the broadcasted parameter, it is determined whether or not the triangle belongs to its own area, for example, an area interleaved in units of a rectangular area of 4 × 4 pixels. Is determined. As a result, if they belong, various data (Z, texture coordinates, colors, etc.) are rasterized.
Next, calculation of a mipmap (MIPMAP) level by LOD (Level @ Detail) calculation and (u, v) address calculation for texture access are performed.
[0206]
Then, the texture is read out.
In this case, the processing units 131-0 to 131-3 of the local modules 13-0 to 13-3 first check the entries of the local caches 133-0 to 133-3 at the time of texture reading.
As a result, when there is an entry, necessary texture data is read.
When the required texture data is not in the local caches 133-0 to 133-3, each of the processing units 131-0 to 131-3 sends a request to the global module 12 through the global interface 134-0 to 134-3. Request for a local cache fill.
[0207]
In the global module 12, when it is determined that the requested block data is in any of the global caches 121-0 to 121-3, the block data is read from any of the corresponding global caches 121-0 to 121-3. The request is sent back to the local module that sent the request through a predetermined channel.
[0208]
On the other hand, if it is determined that the requested block data is not in any of the global caches 121-0 to 121-3, the global cache fill of the local module holding the block from any of the desired channels is performed. A request is sent.
In the local module receiving the request for the global cache fill, the corresponding block data is read from the memory and sent to the global module 12 through the global interface.
Thereafter, in the global module 12, the block data is filled in a desired global cache, and the data is transmitted from a desired channel to the local module which has transmitted the request.
[0209]
When the requested block data is sent from the global module 12, the local cache is updated in the corresponding local module, and the block data is read by the processing unit.
[0210]
Next, in the local modules 13-0 to 13-3, filtering processing such as 4-neighbor interpolation is performed on the read texture data and the (u, v) address using the decimal part obtained at the time of calculation.
Next, a pixel-by-pixel operation is performed using the texture data after filtering and various data after rasterization.
Then, the pixel data that has passed various tests in the pixel level processing is written to the memory modules 132-0 to 132-3, for example, the frame buffer and the Z buffer on the built-in DRAM memory.
[0211]
As described above, according to the present embodiment, there are a plurality of POP0 to POP3 which are functional units for performing highly parallel arithmetic processing utilizing a memory bandwidth, and each POP is a computing unit arranged in parallel. Each of POPE0 to POPE3 receives a reference image RFIM and target image OBIM read from two caches, and receives predetermined operation (for example, subtraction, multiplication and Total addition) to output the operation result to the next-stage POPE. The next-stage POPE adds its own operation result to the previous-stage operation result, outputs the operation result to the next-stage POPE, and outputs the operation result to the final-stage POPE. In POP3, the sum of the operation results of all POP0 to POP3 is obtained. Only the calculation result of POPE3 from providing the pixel operation processor (POP) group 13124 to be output to the crossbar circuit 13146 selects the template matching process with a simple configuration, can be executed with high efficiency.
Further, the size of the crossbar circuit can be reduced, and the processing speed can be increased.
[0212]
In the present embodiment, the pixel operation processor (POP) group 13123 and the cache are connected with a wide bandwidth, and have an address generation function for memory access. It is possible to supply the maximum amount of stream data.
[0213]
Further, in the present embodiment, since the arithmetic units are arranged at high density in the form of matching the output data width near the memory and the regularity of the processing data is used, a large amount of arithmetic is performed by the minimum arithmetic unit. In addition, there is an advantage that it can be realized with a simple configuration, and the cost can be reduced.
[0214]
According to the present embodiment, at the time of graphics processing, parameter data broadcast from the global module 12 is received, and window coordinates, a primary color (PC), a secondary color (SC), a Fog coefficient (f), and a texture coordinate are received. And a rasterizer 1311 that generates a source address and a destination address based on input data at the time of image processing, a register unit 13124 having a plurality of FIFO registers, Graphics data (s, t, l) including texel coordinates (s, t) and LOD data is generated based on the texture coordinates set in the FIFO register 13124, and output without passing the source address. When the graphics unit 13121 performs graphics processing, it performs predetermined arithmetic processing based on the graphics data (s, t, l), transfers the calculated data to the crossbar circuit 13125, and sets the calculated data in a predetermined register of the register unit 13124. At the time of image processing, the pixel data processor 13123 reads out image data corresponding to the source address and performs a predetermined image processing operation, and transfers the operation data to the crossbar circuit 13125 to set the operation data in a predetermined register of the register unit 13124. A predetermined calculation process is performed on the calculation data of the pixel calculation processor 13123 set in the register based on the color data, and the calculation data is transferred to the crossbar circuit 13125 and set in the predetermined register of the register unit 13124. The pixel engine 13122 performs processing necessary for pixel writing based on window coordinates set in a register and operation data of the pixel engine 13122 at the time of graphics processing. At the time of processing, since the write unit WU for writing the operation data of the pixel operation processor 13123 set in the register to the destination address of the memory is provided, the following effects can be obtained.
[0215]
That is, according to the present embodiment, a large amount of arithmetic units can be efficiently used, the degree of freedom of the algorithm is high, the flexibility is high, and the complexity of the circuit is not increased and the cost is not increased. Processing can be performed with high throughput.
[0216]
Further, the processing unit 131 (-0 to -3) executes an algorithm represented by a data flow graph (Data {Flow} Graph: DFG) without a branch, and notes and edges of the DFG are calculated by an arithmetic unit or an arithmetic unit and its Can be seen as a connection relationship. Therefore, the processing unit 131 (-0 to -3) is a so-called dynamically reconfigurable hardware that dynamically switches the connection between the operation resources according to the DFG to be executed. And the connection relationship between them corresponds to the microprogram of the processing unit, and the DFG applied to each element of the stream data is the same, so that the bandwidth for issuing instructions can be reduced.
[0219]
In the processing unit 131 (-0 to -3), the designation of the arithmetic function and the switching control of the connection between the arithmetic units are data driven, and can be said to be distributed independent control.
By employing such dynamic scheduling, when the DFG is switched, the epilogue / prologue can be overlapped, and the overhead of DFG switching can be reduced.
[0218]
Further, when the scale of the DFG increases, it becomes impossible to map the algorithm to the internal calculation resources at once. In such a case, it is necessary to divide the data into a plurality of sub-DFGs (sub-DFG #).
As a method of dividing and executing a plurality of sub-DFGs, there is a multi-pass method of storing an intermediate value between the sub-DFGs in a memory. In this method, when the number of passes increases, the memory bandwidth is consumed and performance is reduced.
As described above, the processing units 131 (-0 to -3) transfer the stream data between the arithmetic units and the arithmetic units via the FIFO type register unit (RGU). Intermediate values can be passed through a file and the number of multi-passes can be reduced.
Although the division of the DFG itself is performed statically by the compiler, the execution control of the divided DFG is performed by hardware, so that there is an advantage that the load on software is light.
[0219]
Furthermore, in the present embodiment, the stream data set in the FIFO register of the register unit 13124 by transferring the crossbar circuit 13125 is directly passed through the graphics unit (GRU) 13121 and the pixel engine (PXE) without passing through the crossbar circuit. ) 13122, a pixel operation processor (POP) group 13123, and a light unit WU, and the graphics operation data obtained by the graphics unit 13121 is directly passed through a specific wiring without passing through a crossbar circuit. Since it is supplied to the pixel operation processor (POP) group 13123, the crossbar circuit can be further simplified and downsized, the number of multi-passes can be reduced, and the processing speed can be further increased. it can.
[0220]
Further, in the present embodiment, a configuration in which only one core 1312 is provided as an arithmetic processing unit for realizing the present architecture has been described as an example. However, for example, as shown in FIG. It is also possible to adopt a configuration in which the cores 1312-1 to 1312-n are provided in parallel.
Even in this case, the DFG executed in each core is the same.
In addition, the unit of parallelization of the configuration in which a plurality of cores are provided is, for example, a small rectangular area (stamp) unit in graphics processing and a block unit in image processing. In this case, there is an advantage that parallel processing with a fine granularity can be realized.
[0221]
Further, according to the present embodiment, the SDC 11 and the global module 12 exchange data, and a plurality of (four in the present embodiment) local modules 13-0 to 13-3 are provided for one global module 12. Are connected in parallel, the processing data is shared by a plurality of local modules 13-0 to 13-3 and processed in parallel, the global module 12 has a global cache, and each of the local modules 13-0 to 13-3 is Since each of the local modules has a local cache, and has two levels of cache layers, a global cache shared by the four local modules 13-0 to 13-3 and a local cache locally owned by each local module, a plurality of processes are performed. Reduces duplicate access when devices share processing data and perform parallel processing Yellow, cross-bar is not required a lot of number of wires. As a result, there is an advantage that an image processing apparatus that can be easily designed and that can reduce wiring cost and wiring delay can be realized.
[0222]
Further, according to the present embodiment, as shown in FIG. 3, the local module 13-0 to the local module 13-0 to the local module 13-0 By arranging 13-3 near its periphery, the distance between each corresponding channel block and the local module can be kept uniform, the wiring areas can be arranged neatly, and the average wiring length can be shortened. Therefore, there is an advantage that the wiring delay and the wiring cost can be reduced, and the processing speed can be improved.
[0223]
In this embodiment, the case where the texture data is stored in the built-in DRAM is described as an example. However, as another case, only the color data and the z data are placed in the built-in DRAM, and the texture data is stored in the external memory. It is also possible to put in. In this case, when a miss occurs in the global cache, a cache fill request is issued to the external DRAM.
[0224]
In the above description, the configuration of FIG. 3, that is, an image processing apparatus in which a plurality of (four in the present embodiment) local modules 13-0 to 13-3 are connected in parallel to one global module 12 10 is an example of a case where parallel processing is performed, but the configuration of FIG. 3 is arranged as one cluster CLST, and four clusters CLST0 to CLST3 are arranged in a matrix as shown in FIG. Thus, data can be exchanged between the global modules 12-0 to 12-3 of each of the clusters CLST0 to CLST3.
In the example of FIG. 44, the global module 12-0 of the cluster CLST0 is connected to the global module 12-1 of the cluster CLST1, the global module 12-1 of the cluster CLST1 is connected to the global module 12-3 of the cluster CLST3, The global module 12-3 of the cluster CLST3 is connected to the global module 12-2 of the cluster CLST2, and the global module 12-2 of the cluster CLST2 is connected to the global module 12-0 of the cluster CLST0.
That is, the global modules 12-0 to 12-3 of the plurality of clusters CLST0 to CLST3 are connected in a ring.
In the case of the configuration in FIG. 44, it is possible to configure so that parameters are broadcast from one SDC to global modules 12-0 to 12-3 of CLST0 to CLST3.
[0225]
By adopting such a configuration, more accurate image processing can be realized, and the wiring between the clusters is simply connected in one direction as a two-way system, so that the load between the clusters is kept uniform. The wiring regions can be arranged neatly, and the average wiring length can be shortened. Therefore, wiring delay and wiring cost can be reduced, and the processing speed can be improved.
[0226]
【The invention's effect】
As described above, according to the present invention, a template matching process can be executed efficiently with a simple configuration.
Further, when a plurality of processing devices share processing data and perform parallel processing, redundant access can be reduced, and a crossbar circuit having a large number of wirings can be reduced in size. As a result, there are advantages that the design is easy, the wiring cost and the wiring delay can be reduced, and the image processing can be speeded up.
[Brief description of the drawings]
FIG. 1 is a diagram conceptually illustrating a parallel processing at a primitive level based on a technique of parallel processing at a pixel level.
FIG. 2 is a diagram illustrating a processing procedure including texture filtering in a general image processing apparatus.
FIG. 3 is a block diagram showing an embodiment of an image processing apparatus according to the present invention.
FIG. 4 is a flowchart for explaining main processing of a stream data controller (SDC) according to the embodiment.
FIG. 5 is a flowchart for explaining functions of a global module according to the embodiment.
FIG. 6 is a diagram illustrating graphics processing of a processing unit in a local module according to the embodiment.
FIG. 7 is a flowchart for explaining the operation of the local module at the time of texture reading according to the embodiment;
FIG. 8 is a diagram for describing image processing of a processing unit in the local module according to the embodiment.
FIG. 9 is a block diagram illustrating a configuration example of a local cache in the local module according to the embodiment;
FIG. 10 is a block diagram illustrating a configuration example of a memory controller of a local cache according to the embodiment;
FIG. 11 is a block diagram illustrating a specific configuration example of a processing unit of a local module according to the embodiment.
FIG. 12 is a block diagram illustrating a configuration example of a pixel engine according to the embodiment.
FIG. 13 is a diagram illustrating a configuration example of a connection network CCN according to the present embodiment.
FIG. 14 is a diagram illustrating a configuration example of a selector according to the embodiment.
FIG. 15 is an explanatory diagram illustrating an outline of calculation execution of the pixel engine according to the embodiment, and is a diagram illustrating a data flow graph of the calculation.
FIG. 16 is an explanatory diagram illustrating an outline of calculation execution of the pixel engine according to the embodiment.
FIG. 17 is an explanatory diagram of pipeline processing of the pixel engine according to the embodiment.
FIG. 18 is a diagram illustrating a first method of realizing dynamic reconfiguration of the pixel engine according to the embodiment.
FIG. 19 is a diagram illustrating a first method of realizing dynamic reconfiguration of the pixel engine according to the embodiment.
FIG. 20 is a diagram illustrating a second method of realizing dynamic reconfiguration of the pixel engine according to the embodiment.
FIG. 21 is a diagram illustrating a second method of realizing dynamic reconfiguration of the pixel engine according to the embodiment.
FIG. 22 is a diagram illustrating a second method of realizing dynamic reconfiguration of the pixel engine according to the embodiment.
FIG. 23 is a diagram illustrating a configuration example of a pixel engine according to the present embodiment, and a connection example with a register unit (RGU) and a crossbar circuit.
FIG. 24 is a diagram illustrating a configuration example of a pixel operation processor (POP) group according to the present embodiment.
FIG. 25 is a diagram illustrating a connection mode between a POP (pixel operation processor) and a memory according to the present embodiment and a configuration example of the POP.
FIG. 26 is a circuit diagram showing a specific configuration example of POPE according to the present embodiment.
FIG. 27 is a diagram showing a form of reading data from a memory to a cache and a form of reading data from a cache to each POPE according to the present embodiment.
FIG. 28 is a diagram for explaining template matching according to the embodiment.
FIG. 29 is a diagram for explaining template matching according to the embodiment.
FIG. 30 is a flowchart for explaining an operation in a case where a pixel processing processor group performs calculation processing based on data in a memory according to the present embodiment and further performs calculation with a pixel engine.
FIG. 31 is a diagram for explaining an operation in a case where arithmetic processing is performed by a group of pixel arithmetic processors based on data in a memory according to the present embodiment, and further, arithmetic is performed by a pixel engine.
FIG. 32 is a timing chart for explaining an operation in a case where arithmetic processing is performed by a group of pixel arithmetic processors based on data in a memory according to the present embodiment, and further, arithmetic is performed by a pixel engine.
FIG. 33 is a block diagram for explaining an operation in a case where arithmetic processing is performed by a group of pixel arithmetic processors based on data in a memory according to the present embodiment, and further, arithmetic is performed by a pixel engine.
FIG. 34 is a diagram for explaining an operation in a case where a pixel operation processor group performs calculation processing based on data in a memory according to the present embodiment and further performs calculation with a pixel engine.
FIG. 35 is a diagram showing an operation outline including a pixel engine (PXE) of a core, a pixel operation processor (POP), a register unit (RGU), and a memory part in the processing unit according to the embodiment.
FIG. 36 is a diagram for describing graphics processing in the case where there is no dependent texture in the processing unit according to the present embodiment.
FIG. 37 is a diagram illustrating a specific operation of a pixel operation processor (POP) group for graphics processing in the processing unit according to the embodiment.
FIG. 38 is a diagram for describing graphics processing in the case where there is a dependent texture in the processing unit according to the present embodiment.
FIG. 39 is a diagram for describing SAD (Summed Absolute \ Difference) processing.
FIG. 40 is a diagram for describing SAD processing in the processing unit according to the present embodiment.
FIG. 41 is a diagram for describing a convolution filter (Convolution @ Filter) process;
FIG. 42 is a diagram for explaining the convolution filter processing in the processing unit according to the embodiment.
FIG. 43 is a diagram illustrating another configuration example (an example in which a plurality of cores are provided) in the processing unit according to the embodiment;
FIG. 44 is a block diagram showing another embodiment of the image processing apparatus according to the present invention.
[Explanation of symbols]
10, 10A image processing apparatus, 11 stream data controller (SDC), 12-0 to 12-3 global module, 121-0 to 121-3 global cache, 13-0 to 13-3 local module 131-0 to 131-3 processing unit, 132-0 to 132-3 memory module, 133-0 to 133-3 local cache, 134-0 to 134-3 global interface (GAIF), CLST0 to CLST ... Cluster, 1311 ... Rasterizer, 1312, 132-1 to 1312-n ... Core, 13121 ... Graphics unit (GRU), 13122 ... Pixel engine (PXE), 13123 ... Pixel operation processor (POP) group, 13124 ... Register unit (RGU), 1 125 ... crossbar circuitry (IXB), POPE0~3 ... calculator, OSLC ... output selection circuit.

Claims (16)

An image processing apparatus that performs a template matching process by comparing a reference image and a changing target image,
A first memory storing the reference image and having a plurality of ports;
A second memory storing the target image and having a plurality of ports;
Arithmetic parameters and element data of the reference image and element data of the target image, which are provided corresponding to the plurality of ports of the first and second memories and are read from the first and second memories. A functional unit including a plurality of arithmetic units that perform parallel arithmetic processing based on and generate continuous stream data,
The computing units of the functional unit are cascaded from the first stage to the last stage, and each computing unit performs a predetermined operation on the element data read from each port of the first and second memories. Performs the processing and outputs the result to the next-stage operation unit. The next-stage operation unit adds the previous-stage operation result to the own-stage operation result, and outputs the addition result to the next-stage operation unit. An image processing device for calculating the sum of all the arithmetic units by an arithmetic unit and outputting the operation result of the final stage as stream data. The reading of the element data of the reference image and the element data of the target image from the first and second memories to the respective operation units of the functional unit is performed by sequentially inputting the data from the first stage and calculating the operation result of the preceding operation unit. 2. An address generator, which generates an address so that the input of the first stage has a timing at which the operation of its own stage is completed and the operation result of the previous stage arithmetic unit can be added, and supplies the address to the first and second memories. An image processing apparatus as described in the above. At least one of the target image or the reference image is stored, and has a memory module having a plurality of ports,
The first and second memories store at least image data read from each port of the memory module, and supply the first and second memories to each of the functional units of the functional unit according to a cache address. And a second cache,
The reading of the element data of the reference image and the element data of the target image from the first and second caches to the respective operation units of the functional unit is performed by sequentially inputting the data from the first stage and calculating the operation result of the preceding operation unit. Further comprising: an address generator for generating an address so that the input of the first stage is at a timing at which the operation of the own stage is completed and the operation result of the preceding stage arithmetic unit can be added, and supplied to the first and second caches. Item 2. The image processing apparatus according to Item 1. A second functional unit including a plurality of arithmetic units that perform predetermined arithmetic processing on the stream data generated by the functional unit;
The image processing apparatus according to claim 1, further comprising a crossbar circuit that interconnects the plurality of arithmetic units of the first functional unit and the plurality of arithmetic units of the second functional unit. The image processing apparatus according to claim 1, wherein the parallel processing is a parallel processing at a pixel level. An image processing apparatus that performs a template matching process by comparing a reference image and a changing target image,
A first memory storing the reference image and having a plurality of ports;
A second memory storing the target image and having a plurality of ports;
Arithmetic parameters and element data of the reference image and element data of the target image, which are provided corresponding to the plurality of ports of the first and second memories and are read from the first and second memories. A plurality of first functional units including a plurality of arithmetic units for performing parallel arithmetic processing based on and generating continuous stream data;
A second functional unit including a plurality of arithmetic units that perform intensive arithmetic processing on the stream data generated by each of the first functional units;
A crossbar circuit interconnecting the plurality of first functional units and the plurality of arithmetic units of the second functional unit,
The arithmetic units of each of the first functional units are cascaded from the first stage to the last stage, and each of the arithmetic units operates on element data read from each port of the first and second memories. And performs a predetermined arithmetic process and outputs the result to the next-stage arithmetic unit. The next-stage arithmetic unit adds the previous-stage arithmetic result to the own-stage arithmetic result, and outputs the addition result to the next-stage arithmetic unit. An image processing apparatus for calculating the sum of all the arithmetic units in a final stage arithmetic unit and outputting the arithmetic result of the final stage as stream data. The reading of the element data of the reference image and the element data of the target image from the first and second memories to the respective operation units of the functional unit is performed by sequentially inputting the data from the first stage and calculating the operation result of the previous operation unit. 7. An address generator having an input for generating an address so as to have a timing at which the operation of its own stage is completed and the operation result of the previous stage arithmetic unit can be added, and supplies the address to the first and second memories. Image processing device. At least one of the target image or the reference image is stored, and has a memory module having a plurality of ports,
The first and second memories store at least image data read from each port of the memory module, and supply the stored data to each computing unit of the first functional unit according to a cache address. Including first and second caches of
The reading of the element data of the reference image and the element data of the target image from the first and second caches to the operation units of the first functional unit is performed by sequentially inputting the data from the first stage and executing the operation at the preceding stage. And an address generator for generating an address so that the input of the operation result of (1) is at a timing at which the operation of the own stage is completed and the operation result of the previous stage operation unit can be added, and supplies the address to the first and second caches. The image processing apparatus according to claim 6, further comprising: The second functional unit is reconfigurable in accordance with a control signal, and connects the arithmetic unit with an electrical connection network according to the control signal to establish an electrical connection of a plurality of arithmetic unit circuits. 7. The image processing apparatus according to claim 6, wherein an arithmetic circuit including a plurality of arithmetic units is configured. The image processing apparatus according to claim 6, wherein the parallel processing is a parallel processing at a pixel level. An image processing apparatus in which a plurality of modules share processing data and perform parallel processing,
Including a global module and multiple local modules,
The above global module is
When the plurality of local modules are connected in parallel and receive a request from the local module, processing data is output to the local module that issued the request in accordance with the request,
The plurality of local modules are modules that perform a template matching process by comparing a reference image and a changing target image,
A first memory storing the reference image and having a plurality of ports;
A second memory storing the target image and having a plurality of ports;
Arithmetic parameters and element data of the reference image and element data of the target image, which are provided corresponding to the plurality of ports of the first and second memories and are read from the first and second memories. A functional unit including a plurality of arithmetic units that perform parallel arithmetic processing based on and generate continuous stream data,
The computing units of the functional unit are cascaded from the first stage to the last stage, and each computing unit performs a predetermined operation on the element data read from each port of the first and second memories. Performs the processing and outputs the result to the next-stage operation unit. The next-stage operation unit adds the previous-stage operation result to the own-stage operation result, and outputs the addition result to the next-stage operation unit. An image processing device for calculating the sum of all the arithmetic units by an arithmetic unit and outputting the operation result of the final stage as stream data. An image processing apparatus in which a plurality of modules share processing data and perform parallel processing,
Including a global module and multiple local modules,
The above global module is
When the plurality of local modules are connected in parallel and receive a request from the local module, processing data is output to the local module that issued the request in accordance with the request,
The plurality of local modules are modules that perform a template matching process by comparing a reference image and a changing target image,
A first memory storing the reference image and having a plurality of ports;
A second memory storing the target image and having a plurality of ports;
Arithmetic parameters and element data of the reference image and element data of the target image, which are provided corresponding to the plurality of ports of the first and second memories and are read from the first and second memories. A plurality of first functional units including a plurality of arithmetic units for performing parallel arithmetic processing based on and generating continuous stream data;
A second functional unit including a plurality of arithmetic units that perform intensive arithmetic processing on the stream data generated by each of the first functional units;
A crossbar circuit interconnecting the plurality of first functional units and the plurality of arithmetic units of the second functional unit,
The arithmetic units of each of the first functional units are cascaded from the first stage to the last stage, and each of the arithmetic units operates on element data read from each port of the first and second memories. And performs a predetermined arithmetic process and outputs the result to the next-stage arithmetic unit. The next-stage arithmetic unit adds the previous-stage arithmetic result to the own-stage arithmetic result, and outputs the addition result to the next-stage arithmetic unit. An image processing apparatus for calculating the sum of all the arithmetic units in a final stage arithmetic unit and outputting the arithmetic result of the final stage as stream data. An image processing method of performing a template matching process by comparing a reference image and a changing target image,
In a plurality of cascade-connected operation stages, predetermined operation processing is performed on the element data of the reference image and the target image read from each port of the first and second memories, respectively.
In the next operation stage, the operation result of the previous stage is added to the operation result of the own stage,
An image processing method in which a sum of all operation stages is obtained in a final operation stage, and the operation result of the final stage is output as stream data. When reading data from the above-mentioned memory to each operation stage, data is input in order from the first stage, and the input of the operation result of the previous operation stage adds the operation result of the previous operation stage when the operation of the own stage is completed. 14. The image processing method according to claim 13, wherein the method is performed so as to be able to perform the timing. An image processing method of performing a template matching process by comparing a reference image and a changing target image,
In a plurality of operation stages connected in cascade of a plurality of functional units, a predetermined operation is performed on the element data of the reference image and the target image read from each port of the first and second memories, respectively.
In the next operation stage, the operation result of the previous stage is added to the operation result of the own stage,
The final operation stage calculates the sum of all the operation stages, outputs the operation result of the final stage to the crossbar circuit as stream data,
An image processing method for performing intensive arithmetic processing on a plurality of stream data transferred through the crossbar circuit. When reading data from the above-mentioned memory to each operation stage, data is input in order from the first stage, and the input of the operation result of the previous operation stage adds the operation result of the previous operation stage when the operation of the own stage is completed. 16. The image processing method according to claim 15, wherein the method is performed so as to be able to perform the timing.

Images