JP2006072658A - Image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To issue access requests at a frequency depending on a throughput requested by a client without complicating hardware (client) that issues access requests. <P>SOLUTION: An image processing device has a clock thinning control block 10 for optimizing a bandwidth request for access requests to a memory by distributing the frequency of access requests within a specified period according to the size of image data to be processed by the access requests and processing time allowable to complete the processing. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus such as an image processor that performs access request processing for a memory that stores image data.

  Conventionally, when processing an access request to a memory, the memory controller prioritizes each access request and waits for the access request, and arbitrates between the access requests. The access request is accepted before the processing of the individual client issuing the access request fails, and the timing and capacity are controlled.

  Further, as disclosed in Patent Document 1, when data is supplied in response to a request that occurs irregularly from a client requesting memory access that exists in parallel, there is a problem in processing on the memory controller side.

  For example, in Patent Document 2, as a solution for avoiding temporary concentration of access, “a plurality of accesses to a memory are changed according to the status of each access, and the changed priority is changed. Accordingly, arbitration and scheduling of memory access is performed to prevent concentration on a specific memory access, occurrence of an invalid period, and the like. "

  That is, in the above-mentioned Patent Document 2, an attempt is made to suppress the increase in bandwidth by the function of the arbitration side possessed on the memory controller side, and “concentration on a specific memory access is also made on the client side. I was n’t doing something like that. ”

Further, in Patent Document 3, the clock frequency input to the image processor is controlled according to processing, but the generation method is based on the original frequency of a plurality of types of clocks supplied and the plurality of clock inputs. The clock frequency supplied to the image processor can be controlled by controlling a circuit that selects the image processor.
JP-A-8-50651 JP-A-8-314793 Japanese Patent Laid-Open No. 7-5864

  As described above, in the conventional technique, the client side is not configured to have a function for generating requests at an average time interval so as to avoid temporary concentration of access.

  Furthermore, when the clock frequency is changed by the clock supply source, that is, the oscillator or PLL (Phase Locked Loop), it is generally necessary to have a transition period until the clock output after the change becomes stable. There is a possibility that a desired clock frequency cannot be obtained.

  Therefore, in the present invention, the original clock is always set so that the maximum processing capability can be exhibited, and thinning control is given to the clock enabler in the subsequent stage, so that the optimum frequency is supplied to the image processor as a result. Like that. As an advantage of this, there is no need to change the source clock first, so consider the transient period that occurs when changing the clock frequency of the oscillator or PLL until the clock output after the frequency change is stabilized. No need to do. That is, even if the frequency of the clock supplied continuously to the image processor (decimation rate with respect to the original clock) is changed for each image to be processed, the image can be processed without any problem.

The present invention is a measure for realizing the optimization of the bandwidth according to the operation state in a system in which a plurality of blocks are connected to a memory bus via a memory interface and each has an individual memory access request. Hardware configuration as an image processing apparatus capable of issuing an access request at a frequency according to the processing capability required of the client without complicating the hardware (client) on the side of issuing the access request Is to provide.
Hereinafter, each hardware module that issues an access request to the memory controller is defined as a “client”.

  In order to solve the above problems and achieve the object of the present invention, the image processing apparatus of the present invention determines the frequency of access requests according to the size of image data to be subjected to access request processing and the processing time allowed for completion of the processing. By distributing within a specified period, an optimization means for optimizing the bandwidth request in the access request to the memory is provided.

  By distributing the frequency of access requests, for example, although the access rate when viewed at a time interval of 30 ms is low, access requests are concentrated there when viewed at a certain time, for example, 1 ms. The bandwidth that needs to be secured as a system is increased. ”Itself can be avoided.

  In realizing the above functions, the configuration of the image processing apparatus itself, which is an image processor, is designed to satisfy a state where the maximum processing capability can be exhibited (the amount of processing data is the largest, that is, the state where the bandwidth requirement is the highest). The clock frequency supplied from the source is also a frequency that can meet this requirement.

  As described above, the image processor itself and the supplied clock frequency are designed for peak times. The optimization of the bandwidth requirement under the peak state, that is, the averaging of the access request is to control the clock frequency supplied to the image processor in accordance with “image size” and “allowable processing time”. Realize with.

  Control of the clock frequency does not fluctuate the source oscillation that supplies the clock, for example, the input clock from the outside, the oscillator, and the output clock of the PLL, but the source oscillation frequency from these can satisfy the maximum processing capacity The frequency is supplied.

  The clock frequency is controlled, for example, by inserting a clock enabler (gated clock) between the supplied clocks. That is, the clock is thinned out from the supplied clock. The clock thinning control is performed by writing a set value in a register that can be controlled by software. The hardware generates a control pulse for performing clock thinning according to a set value (thinning rate).

  By reducing the access frequency to the memory by controlling the clock frequency to be reduced, the arbitration on the memory controller side (allocation of access to the memory and arbitration between the source modules (clients) that have issued access requests) can be simplified. .

  Here, the setting of the clock decimation rate is determined by a calculation formula based on the access data amount per predetermined unit time (for example, 1/60 seconds) and the clock frequency before decimation.

  According to the present invention, by averaging the frequency of access to the memory by controlling to reduce the clock frequency, the arbitration on the memory controller side (allocation of access to the memory and source modules (clients) that have issued access requests) Mediation) can be simplified.

  In addition, the memory controller provides a buffer that prevents data underflow or overflow for a single request, so that a handshake with the client that is required when underflow or overflow occurs is possible. Can be eliminated. This simplifies the I / F specification between the memory controller and the client, but the capacity must be large enough to withstand the data I / F required by each client.

  In addition, a plurality of access requests issued from the image processing apparatus which is each image processor to the memory controller exist in the system by being distributed and averaged within the processing time allowed for the image processor. Since it is possible to reduce the waiting period when an access request is issued to another client, the buffer memory for guaranteeing the data I / F during the waiting period can be reduced.

  Further, the access frequency is averaged by clock thinning control, but complicated setting for this control is not required, and it is only necessary to set the parameter “N” that determines the thinning rate. “N” is a value that can be calculated from the frequency of the supplied clock (CLKIN) and the number of data that must be processed in unit time (for example, 1/60 seconds). The control side only implements this calculation formula. It's okay.

Embodiments of the present invention will be described below with reference to the drawings as appropriate.
FIG. 2 is a system configuration diagram. As a specific configuration diagram, FIG. 2 is an example of the entire system including the image processor 13 in which the clock thinning control block 10 according to the present embodiment is incorporated. In addition to the image processor 3, the system includes functional blocks 14 to 18, all of which have a possibility of issuing an access request for data I / F with the large capacity memory 11. Yes.

  Each of the functional blocks 13 to 18 (client) issues a read or write access request irregularly. The memory controller 12 receives these irregular access requests and sends data to each client (when the read request is from the memory 11) and receives data (when it is a write request to the memory). In parallel with answering each access request, access (read / write) to the memory 11 is sequentially performed in response to the incoming access request.

  Here, the details of the three image processors that are objects of the present embodiment will be described. This block is required to handle images of various sizes. For example, it is the image size included in MPEG2 MP @ HL. In addition, it is indispensable to complete the processing of the data amount determined in units of 1/60 second or 1/30. Typical image sizes that must be processed within 1/60 second by the image processor in this case are 1280 × 720, 1920 × 540, 720 × 480, 720 × 240, and the like. (These are examples and other than the above.)

Continuing the explanation based on this example, the image processor in this case handles the image data of the above size as data of 8 bits each for Y / C, so the data access amount per 1/60 is as follows:
・ 1280 x 720 x 16 = 14745600 [bit]
・ 1920 x 540 x 16 = 16588800 [bit]
・ 720 x 480 x 16 = 5529600 [bit]
・ 720 x 240 x 16 = 2764800 [bit]
It becomes.

In the above example, the image processor itself can handle 16588800 [bit] data in 1/60 second as the maximum processing capacity. The read / write request is issued.
The image processor must be supplied with a clock having a frequency that can satisfy the highest processing capability and access.

FIG. 1 is a diagram showing a clock thinning control block according to the present invention.
In FIG. 1, the clock decimation control block 10 combines a counter 1 that counts the clock frequency supplied to the terminal 7 and a count output from the counter 1 in accordance with a decimation rate preset in the terminal 6. A combinational circuit 2 that generates a signal and a clock enabler 3 that enables a clock frequency supplied by a clock enable signal from the combinational circuit 2 and outputs a clock to the terminal 8 are configured. Here, the clock enabler 3 includes a latch 4 and an AND circuit 5.

The operation of the clock thinning control block according to the present invention will be described below.
3A and 3B are diagrams showing the access request frequency. FIG. 3A is an access request image at the time of 1920 × 540 processing, and FIG. 3B is an access request image at the time of 720 × 240 processing.
FIG. 3 shows the issuance frequency of write requests to the memory controller 12, but the image when the maximum processing capacity is exhibited (hereinafter referred to as “peak time”) is shown in the figure. A. Here, when a size of 1920 × 540 is handled, 36 access requests are shown in 1/60 second from T1 to T36 when the process is started and T1 is terminated.

  Next, when handling a size of 720 × 240, only the image size condition given to the image processor 13 is simply changed, that is, an image when processing is performed with the supplied clock frequency at the peak specification is shown in the figure. B. Here, six access requests are shown in 1/360 seconds from T1 to T6 when processing is started and T1 is terminated. As can be seen from FIG. 3B, the amount of data to be processed by the image processor 13 is 1/6, so that the processing is completed in 1/6 as compared with FIG. 3A.

  However, in the case of the process of FIG. 3B, the data transfer rate during the process (within a period of 1/360 seconds in the figure) is the same as that of FIG. 3A. From the perspective of the memory controller that is doing this, the bandwidth for 1/360 seconds has not changed at all from the peak state.

  On the other hand, the other functional blocks 14 to 18 also have access requests to the memory 11 in the same manner as the image processor 13. Here, if the requests of the other functional blocks 14 to 18 do not have real time property, the data I / F can be kept waiting until the processing of the image processor 13 is finished, but the real time property is guaranteed. If it is a functional block that must be present, it is necessary to perform data I / F in parallel with the image processor 13. Here, for example, if the data rate that can be processed systematically by the memory controller 12 is equivalent to the peak time of the image processor 13, the function can be performed while waiting for access requests from the other functional blocks 14 to 18. A buffer memory that absorbs the data I / F with the block is required.

4A and 4B are diagrams illustrating access request averaging. FIG. 4A is an access request image at the time of 1920 × 540 processing, and FIG. 4B is an access request image at the time of 720 × 240 processing.
Therefore, when the access requests from T1 to T6 in FIG. 3B are distributed to T1, T8, T15, T22, T29, and T36 as shown in FIG. 4B, the access is more effective than when the peak processing capacity is exhibited. As the request interval (solid arrow) increases, the bandwidth that must be reserved for processing by the image processor 13 is 1/6 on average. As a result, it is possible to allocate a portion that can accept an access request with a margin to requests from other functional blocks 14 to 18. (Dotted arrow)

  As the other functional blocks 14 to 18, compared with the case of FIG. 3B, since the time for which the request is kept waiting is reduced, the buffer memory for guaranteeing the data I / F can be reduced.

  That is, by generating the access request occurrence frequency of the image processor 13 in accordance with the amount of data processed by the image processor 13, it can be allocated to requests of other functional blocks 14 to 18. As a result, the buffer memory can be reduced.

  Various means can be considered as means for averaging the frequency of occurrence of access requests of the image processor 13, but in the present invention, the frequency of access requests can be controlled by controlling the clock supplied to the image processor 13. I have to.

FIG. 5 is a diagram showing access request generation timing and the number of clocks, FIG. 5A is an access request, and FIG. 5B is a supply clock.
As shown in FIG. 5A, the image processor 13 generally determines how many clocks are supplied by the number of clocks supplied to the image processor 13 shown in FIG. 5B when an access request is generated during processing. . The example of FIG. 5 is an example at the peak time, and the clock is supplied in a state where it is not thinned out. In this example, an access request is generated at the time of T1, T7, T13, and T19 every 6 clocks. ing.

FIG. 6 is a diagram showing access request generation timing and clock control. FIG. 6A shows an access request, and FIG. 6B shows a supply clock.
On the other hand, in FIG. 6B, image processing is performed by adding thinning-out control to the clocks supplied at times T2 to T4, T4 to T6, T9 to T11, T11 to T13, T13 to T15, T16 to T18, and T18 to T19. 13 is a state in which the processing capacity is reduced. The access request is generated every 6 clocks at the time of T1 and T11, which is the same as the peak time in FIG. 5, but the frequency of access requests is reduced because the clock is thinned out. I understand that.

  The above-described configuration of FIG. 1 shows a specific example in which clock thinning, setting of a thinning rate, and control of a thinning rate are performed with respect to a clock supply having a specification capable of exhibiting the maximum processing capability. .

  FIG. 7 shows an example of a timing chart of clock thinning control in the clock thinning control block 10. 7A is an input clock, FIG. 7B is a clock decimation counter, FIG. 7C is a clock decimation rate setting value, FIG. 7D is a combinational circuit output, and FIG. 7E is an output clock.

  The clock CLKIN shown in FIG. 7A input to the terminal 7 of the clock thinning control block 10 shown in FIG. 1 is a frequency satisfying the maximum processing capability required for the image processor 13 in the example of the present embodiment. Have been supplied.

  The clock CLKIN shown in FIG. 7A is controlled to be thinned out by the clock enabler 3 shown in FIG. 1, and output to the subsequent stage as the output clock CLKOUT shown in FIG. 7E. In the example of the present embodiment, a circuit that is the main body of the image processor 13 is configured earlier.

  Clock decimation by the clock enabler 3 is controlled by a clock enable control signal DDA_CKEN which is an output of the combinational circuit shown in FIG. 7D. This signal is generated by a block called combinational circuit 2 in FIG. The combinational circuit 2 is a module that generates a pulse having an average N / M ratio by inputting a clock decimation rate setting value M, N shown in FIG. 7C and a counter value for comparing these two numerical values. It is. In the example of the present embodiment, the upper limit of M and N is 255 (8-bit width), M is fixed at 255, and N is a value that can be set from the outside as a register setting.

  1 is a counter that generates a counter value used for comparison, and the counter value output from the clock thinning counter shown in FIG. 7B is input to the combinational circuit 2. Since the pulse output from the combinational circuit 2 shown in FIG. 7D is a thinning control pulse for the clock CLKIN clock shown in FIG. 7A, the counter 1 in FIG. 1 is driven by the clock CLKIN shown in FIG. 7A.

  With the circuit configuration of the clock thinning control block 10 described above, the image processor 13 is supplied with a “thinned clock” corresponding to the amount of data to be processed. As a result, it is issued to the memory controller 12. Can be issued at regular intervals according to the processing state.

Further, the thinning-out control is possible only by setting the set value “N” from the register.
For example, when N of N / 256 is set, as an example for generating an enable pulse so that a clock can be generated on the average according to the value of N, as shown in FIG. An enable signal generation table of the counter value “82” for generating the enable pulse is created, and this is used for the combinational circuit 2 to output N clocks equally among 0 to 255 of the clock CLKIN shown in FIG. 7A. What is necessary is just to use it in order to make the enable signal shown to FIG. 7D.

  When N is a value between 129 and 255, the above table with N between 0 and 128 is used. For example, when N = 2, when the counter value is 0 and 128, the enable pulse is “ However, when N = 254, the generated enable signal may be inverted so that the enable pulse is set to “L” when the counter value is 0 or 128.

  This is simply because the “circuit that outputs“ H ”according to the table” as in this example significantly increases the gate scale.

  Needless to say, the configuration is not limited to the above-described embodiment of the present invention, and the configuration may be appropriately changed within the scope of the claims of the present invention.

It is a figure which shows the clock thinning-out control block of this invention. It is a system configuration diagram. FIGS. 3A and 3B show access request frequencies. FIG. 3A shows an access request image during 1920 × 540 processing, and FIG. 3B shows an access request image during 720 × 240 processing. FIG. 4A is a diagram showing access request averaging, FIG. 4A is an access request image at the time of 1920 × 540 processing, and FIG. 4B is an access request image at the time of 720 × 240 processing. FIG. 5A is a diagram showing access request generation timing and the number of clocks, FIG. 5A is an access request, and FIG. 5B is a supply clock. FIG. 6A shows access request generation timing and clock control. FIG. 6A shows an access request, and FIG. 6B shows a supply clock. FIG. 7A is an input clock, FIG. 7B is a clock decimation counter, FIG. 7C is a clock decimation rate setting value, FIG. 7D is a combinational circuit output, and FIG. 7E is an output clock. It is an enable signal generation table.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Counter, 2 ... Combination circuit, 3 ... Clock enabler, 10 ... Clock thinning control block, 11 ... Memory, 12 ... Memory controller, 13 ... Image processor, 14-18 ... Other functional blocks

Claims (8)

In an image processing apparatus that performs access request processing for a memory that stores image data,
By distributing the frequency of the access requests within a specified period according to the size of the image data to be processed for the access request and the processing time allowed for the completion of the processing, the bandwidth request in the access request to the memory can be reduced. An image processing apparatus comprising an optimization means for optimizing. The image processing apparatus according to claim 1.
The image processing apparatus according to claim 1, wherein the optimization means is supplied with a clock frequency that satisfies a state in which the maximum processing capacity can be exhibited with the largest amount of processing data and the highest bandwidth requirement. The image processing apparatus according to claim 2.
The image processing apparatus according to claim 1, wherein the optimization means optimizes a bandwidth requirement in a state below a state where the maximum processing capability can be exhibited, and controls a supplied clock frequency. The image processing apparatus according to claim 3.
An image processing apparatus characterized in that the control of the clock frequency of the optimization means performs thinning-out control without changing the supplied clock frequency. The image processing apparatus according to claim 4.
The image processing apparatus according to claim 1, wherein the clock frequency of the optimization unit is controlled according to a preset thinning rate. The image processing apparatus according to claim 5.
The image processing apparatus according to claim 1, wherein the optimization unit performs the thinning control by generating a control pulse for thinning the clock frequency in accordance with a thinning rate set in advance with respect to the clock frequency. The image processing apparatus according to claim 1.
The optimization means includes a counter that counts a supplied clock frequency, a combinational circuit that generates a clock enable signal by combining the count output from the counter according to a preset thinning rate, and the combinational circuit An image processing apparatus comprising: a clock enabler that enables the clock frequency supplied by the clock enable signal. The image processing apparatus according to claim 7.
The thinning rate is set based on a predetermined access data amount per unit time and a clock frequency before thinning.

Images