JP2013161471A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow for selection of reconfiguration processing corresponding to an image region for each image region in an image upon achieving a plurality of image processing functions by a reconfigurable circuit.SOLUTION: An image division part 104 divides image data into a plurality of image regions. An image analysis part 106 analyzes, for each image region, a mixture state of a plurality of pieces of pixel data corresponding to image processing A and a plurality of pieces of pixel data corresponding to image processing B to obtain an analysis result. A processing time estimation part 108 estimates, based on an analysis result for each region, a processing time by an entire reconfiguration and a processing time by a partial reconfiguration. A reconfiguration determination part 110 obtains, for each image region, a determination result to the effect that a smaller processing time of the entire configuration or the partial reconfiguration is reconfiguration processing corresponding to the image region. In execution of the image processing A and the image processing B, the entire reconfiguration or the partial reconfiguration is selected for each image region in accordance the determination result of the reconfiguration determination part 110.

Description

  The present invention relates to an image processing apparatus.

  For example, when performing image processing such as color conversion processing or filter processing on an image, a technique for realizing a function of image processing with a reconfigurable circuit is known.

  For example, a reconfigurable circuit (also referred to as a programmable logic circuit) such as a PLD (Programmable Logic Device) or an FPGA (Field Programmable Gate Array) that can reconfigure (change) the internal logic circuit configuration is used.

  PLDs and FPGAs generally set an internal logic circuit configuration at the time of circuit startup, but those capable of changing the logic circuit configuration while the circuit is operating have been developed. In addition, the use of a dynamically reconfigurable processor (DRP: Dynamic Reconfigurable Processor) capable of dynamically reconfiguring an internal logic circuit configuration is also progressing.

  By using a reconfigurable circuit, for example, a reconfigurable circuit is configured (reconstruction processing) so as to switch an image processing unit corresponding to a plurality of image processing, and a plurality of images are generated by the reconfigurable circuit. Processing can be realized.

  An object of the present invention is to enable selection of a reconstruction process corresponding to an image area for each image area in an image when a plurality of image processing functions are realized by a reconfigurable circuit.

  The invention according to claim 1 is an image processing apparatus that processes an image composed of a plurality of pixel data, and includes a control unit that configures a plurality of image processing units in a reconfigurable circuit, and a plurality of image regions. An analysis unit that analyzes the arrangement state of a plurality of pixel data in the image area for each image area of the divided image, and an image of the entire reconstruction process for each image area based on the result of the analysis A prediction unit for predicting the processing performance and the image processing performance of the partial reconstruction processing; and for each image region, the full reconstruction processing or the partial reconstruction processing selected based on the comparison of the image processing performance for the image region A determination unit that obtains a determination result as a reconstruction process corresponding to the image, and the control unit performs at least in the reconfigurable circuit according to the determination result for each image area in the image processing for the image. one A total reconstruction process that constitutes at least one image processing unit so as to image-process a plurality of pixel data over the entire area of each image region by the maintained image processing unit. Alternatively, a plurality of image processing units are configured to perform image processing on a plurality of pixel data in each image region by each image processing unit corresponding to each pixel data while switching the plurality of image processing units in the reconfigurable circuit. The image processing apparatus is characterized by selecting and executing a partial reconstruction process that constitutes.

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, each pixel data constituting the image requires image processing by each image processing unit corresponding to the pixel data, and the analysis unit includes: , For each image region, based on the correspondence between each pixel data and each image processing unit, analyze the mixed state of a plurality of pixel data corresponding to a plurality of image processing unit, the prediction unit, for each image region, Based on the result of the analysis, a processing time by the full reconstruction process and a processing time by the partial reconstruction process are predicted, and the determination unit performs the full reconstruction process or the partial reconstruction process for each image area. The control unit obtains a determination result that the reconstruction processing time corresponding to the image region is smaller, and the control unit performs the entire reconstruction within the entire image region in one image processing unit. After image processing A plurality of image processing units are configured so that other image processing units perform image processing on the entire image area before image processing, and in the partial reconstruction processing, image processing is sequentially performed in each image area. A plurality of image processing units are configured so that each pixel data is processed by each image processing unit corresponding to the pixel data.

  According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect, each pixel data constituting the image is a normal power consumption image processing unit or a low power consumption that is equal to or lower than the normal power consumption. The image processing unit needs to perform processing by a plurality of pixel data that can be processed by a low power consumption image processing unit and a normal power consumption image processing unit for each image region. Analyzing the mixed state of a plurality of pixel data, the prediction unit predicts the power consumption by the full reconstruction process and the power consumption by the partial reconstruction process based on the result of the analysis for each image region, The determination unit obtains a determination result for each image region as a reconstruction process corresponding to the image region with a smaller power consumption of the full reconstruction process or the partial reconstruction process, and the control unit includes: The entire surface In the reconfiguration processing, normal power consumption image processing is performed so that a plurality of pixel data is image-processed over the entire image area while maintaining the normal power consumption image processing unit in the reconfigurable circuit. In the partial reconstruction process, each pixel data to be image-processed one after another in each image area is converted into a normal power consumption image processing unit or a low power consumption image processing unit according to the pixel data. An image processing apparatus characterized by switching between the image processing units so as to perform image processing.

  According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, prior to image processing on the image by the plurality of image processing units, the reconfigurable circuit includes: A preceding image processing unit, the analysis unit, the prediction unit, and the determination unit are configured, and in parallel with the image processing on the image by the preceding image processing unit, the analysis unit performs the analysis, and the prediction unit An image processing apparatus, wherein the prediction is executed, and the determination unit obtains the determination result.

  According to the first aspect of the present invention, when a plurality of image processing functions are realized by a reconfigurable circuit, a reconstruction process corresponding to the image area can be selected for each image area in the image.

  According to the invention which concerns on Claim 2, based on the estimated processing time, the reconstruction process corresponding to the image area can be selected for every image area in an image.

  According to the third aspect of the present invention, it is possible to select a reconstruction process corresponding to an image area for each image area in the image based on the predicted power consumption.

  According to the fourth aspect of the invention, in parallel with the image processing executed using the reconfigurable circuit, the reconfigurable circuit obtains the determination result of the reconstruction processing corresponding to each image area. Can do.

It is a figure for demonstrating a suitable image processing apparatus in implementation of this invention. It is a figure for demonstrating the specific example 1 of the circuit comprised in a reconfigurable circuit. It is a figure for demonstrating the switching of the image processing by partial reconstruction. It is a figure for demonstrating the determination process of a reconstruction. It is a figure for demonstrating the specific example 2 of the circuit comprised in a reconfigurable circuit. It is a figure for demonstrating switching of normal power consumption and low power consumption. It is a figure for demonstrating the determination process of the reconstruction in the image processing of normal power consumption and low power consumption.

  FIG. 1 is a diagram for explaining an image processing apparatus suitable for implementing the present invention. FIG. 1 shows an image processing processor 100 included in the image processing apparatus (the present image processing apparatus).

  Image data to be processed by the image processing apparatus is provided to the image processing apparatus from an external device such as a computer, and is sent to the image processing processor 100 via a data bus (not shown). Further, the image processing apparatus may include an image reading function (scanning function) (not shown), and the image processing processor 100 may process image data obtained from a medium such as paper via the function. Further, the image processing apparatus may be provided with a printing function (printing function) (not shown), for example, to print an image corresponding to the processed image data on paper or the like, or the processed image data may be printed on an external device. You may make it provide.

  The image processor 100 can be realized by, for example, a DRP (Dynamic Reconfigurable Processor), performs image processing on input image data, and outputs the image data after image processing. . Of course, the image processor 100 may be realized by other existing processors or other processors that will be developed in the future.

  An image processor 100 illustrated in FIG. 1 includes a reconfigurable circuit 10, a control unit 20, and a reconfiguration data storage unit 30. The reconfigurable circuit 10 is a circuit that can reconfigure its internal logic circuit configuration dynamically, that is, while the processor is in operation, and includes, for example, an array of a number of circuit elements (PE: processor elements). ing. The connection configuration between these circuit elements can be rearranged at a relatively high speed in accordance with the reconfiguration data (configuration data).

  The circuits in the reconfigurable circuit 10 are reconfigured (reconfigured) based on the reconfiguration data stored in the reconfiguration data storage unit 30 under the control of the control unit 20. In FIG. 1, as an example of reconstruction data stored in the reconstruction data storage unit 30, reconstruction data 1 and image processing A circuit for realizing the image processing A circuit and the image processing B circuit by full reconstruction processing. And reconstruction data 2 for realizing the image processing B circuit by partial reconstruction processing, reconstruction data 3 for realizing a circuit for determining whether full reconstruction or partial reconstruction is appropriate, and preceding image processing circuit Reconstruction data 4 for realizing the above is shown. Of course, reconstruction data other than these may be stored in the reconstruction data storage unit 30.

  The image processing apparatus is configured in the reconfigurable circuit 10 so as to switch between the image processing A circuit and the image processing B circuit. Thereby, for example, image data in which a plurality of pixel data subjected to different image processing is mixed can be processed. Specifically, when color conversion processing is performed on a plurality of pixel data constituting image data, different color conversion processing (image processing A and image processing B) is used depending on the brightness of each pixel data. Can be processed. Although image processing A and image processing B perform image processing with the same contents, image processing A can be processed relatively quickly when the same pixel data continues, and image processing B is performed when pixel data changes frequently. In other words, the image processing A and the image processing B may be different from each other in terms of processing capability. In addition, the image processing C and the like may be added to the image processing A and the image processing B, and a configuration in which three or more image processing circuits are switched may be employed.

  FIG. 2 is a diagram for explaining a specific example 1 of a circuit configured in the reconfigurable circuit 10. When the image processing apparatus performs image processing A and image processing B on image data, FIG. A circuit configured in the configurable circuit 10 is shown.

  In FIG. 2, a circuit configuration 1 shows a circuit configuration in the reconfigurable circuit 10 before the execution of the image processing A and the image processing B. The circuit configuration 2 and the circuit configuration 3 are the image processing A and the image processing B. The circuit configuration in the reconfigurable circuit 10 at the time of execution is shown. When executing the image processing A and the image processing B, the full reconstruction (circuit configuration 2) or the partial reconstruction (circuit configuration 3) is selected when switching between the image processing A and the image processing B circuits. The selection of full reconstruction or partial reconstruction is performed based on the state of pixel data in image data to be processed by image processing A and image processing B. Therefore, prior to the image processing A and the image processing B, the state in the image data is analyzed in the circuit configuration 1, and it is determined whether to select full reconstruction or partial reconstruction.

  Specifically, as the circuit configuration 1, a reconfigurable circuit 10 includes a data input unit 102, an image division unit 104, an image analysis unit 106, a processing time prediction unit 108, a reconstruction determination unit 110, and a data output unit 112. Then, the state in the image data to be processed by the image processing A and the image processing B is analyzed, and the reconstruction is determined. These circuits (circuits related to the reconstruction determination) from the data input unit 102 to the data output unit 112 are configured based on the reconstruction data 3 stored in the reconstruction data storage unit 30 in FIG. The processing by the circuit relating to the reconstruction determination will be described in detail later using FIG.

  Further, processing in the circuit relating to reconstruction determination is executed prior to image processing A and image processing B. The execution is preferably performed in parallel with some other process, for example, a preceding image process different from the image processes A and B.

  Therefore, as shown in FIG. 2, as the circuit configuration 1, the reconfigurable circuit 10 includes a data input unit 12p, a preceding image processing circuit 14p, and a data output unit 16p. These circuits related to the preceding image processing are configured based on the reconstruction data 4 stored in the reconstruction data storage unit 30 of FIG. Then, when the image data to be processed is input to the data input unit 12p, a plurality of pixel data in the image data are sequentially sent to the preceding image processing circuit 14p to perform the preceding image processing, and after the processing A plurality of pixel data is sent to the data output unit 16p. Thus, when all the pixel data in the image data is processed and sent to the data output unit 16p, the preceding image processing for the image data is completed.

  When the processing related to the reconstruction determination by the circuit configuration 1 and the preceding image processing are completed, the reconfigurable circuit 10 is reconfigured into the circuit configuration 2 or the circuit configuration 3 according to the result of the reconstruction determination, and the image processing A And image processing B is executed. In the image processing A and the image processing B, the circuit configuration 2 or the circuit configuration 3 is selected for each image area for the image data divided into a plurality of image areas.

  The circuit configuration 2 shows a circuit configuration in the reconfigurable circuit 10 in the case of full-surface reconfiguration, and is configured based on the reconfiguration data 1 stored in the reconfiguration data storage unit 30 in FIG.

  In full-surface reconstruction, first, a data input unit 12a, an image processing A circuit 14a, and a data output unit 16a, which are circuit configurations related to image processing A, are configured in the reconfigurable circuit 10. When the image data of the image area to be processed is input to the data input unit 12a, a plurality of pixel data in the image data are sequentially sent to the image processing A circuit 14a and subjected to image processing A, The processed pixel data is sent to the data output unit 16a. In this way, when all the pixel data in the image area is processed and sent to the data output unit 16a, the image processing A relating to the image area ends.

  In the entire surface reconstruction, image processing B is further performed on the image region before image processing A is performed after image processing A relating to a certain image region is completed. Therefore, instead of the circuit configuration related to the image processing A, a data input unit 12b, an image processing B circuit 14b, and a data output unit 16b, which are circuit configurations related to the image processing B, are configured in the reconfigurable circuit 10. When the image data of the image area to be processed is input to the data input unit 12b, a plurality of pixel data in the image data are sequentially sent to the image processing B circuit 14b and subjected to image processing B. The processed pixel data is sent to the data output unit 16b. In this way, when all the pixel data in the image area is processed and sent to the data output unit 16b, the image processing B relating to the image area ends.

  As described above, in the entire surface reconstruction, the image processing A is performed over the entire area of a certain image area, and then the image processing B is performed over the entire area of the image area before the image processing A is performed. The reconfigurable circuit 10 is reconfigured.

  On the other hand, the circuit configuration 3 shows a circuit configuration in the reconfigurable circuit 10 in the case of partial reconfiguration, and is configured based on the reconfiguration data 2 stored in the reconfiguration data storage unit 30 in FIG. Is done.

  In the partial reconstruction, a data input unit 12, a data determination unit 13, and a data output unit 16 are configured in the reconfigurable circuit 10, and the image processing A circuit 14a and the image processing A circuit 14a are arranged in accordance with the pixel data to be processed. The image processing B circuit 14b is configured to be selectively switched.

  That is, when image data of an image area to be processed is input to the data input unit 12, a plurality of pixel data in the image data is sequentially sent to the data determination unit 13. The data determination unit 13 determines for each pixel data whether the pixel data corresponds to the image processing A or the image processing B. For example, it is determined which of different color conversion processes (image processing A and image processing B) is applied according to the brightness of each pixel data.

  When it is determined that certain pixel data corresponds to the image processing A, the image processing A circuit 14a is formed in the reconfigurable circuit 10, and the pixel data is processed in the image processing A circuit 14a. On the other hand, if it is determined that certain pixel data corresponds to the image processing B, an image processing B circuit 14b is formed in the reconfigurable circuit 10 in place of the image processing A circuit 14a, and the image processing B circuit 14b Pixel data is processed. Thereafter, the image processing A circuit 14a and the image processing B circuit 14b are switched in accordance with the pixel data. When the image processing is color conversion processing, for example, only the color conversion lookup table may be switched.

  Thus, when all the pixel data in the image area are processed and sent to the data output unit 16, the image processing A and the image processing B relating to the image area are completed.

  As described above, in the partial reconstruction, the reconfigurable circuit 10 is configured so that the image processing A or the image processing B corresponding to the pixel data is performed for each pixel data subjected to image processing one after another in a certain image region. Is reconstructed.

  FIG. 3 is a diagram for explaining switching of image processing by partial reconstruction. FIG. 3 shows a specific example when the image processing A circuit 14a is switched to the image processing B circuit 14b in the circuit configuration 3 (FIG. 2).

  First, when image processing is started (S1), image data of an image area to be processed is input to the data input unit 12. In FIG. 3, broken-line rectangles with numbers inside indicate pixel data. In this example, pixel data 5 to 8 corresponding to image processing B following pixel data 1 to 4 corresponding to image processing A is shown. Is input to the data input unit 12.

  The plurality of pixel data input to the data input unit 12 are sent to the data determination unit 13 one after another, for example, every clock. The data determination unit 13 determines for each pixel data whether the pixel data corresponds to the image processing A or the image processing B.

  In the example of FIG. 3, the image processing A circuit 14 a is configured at the start of the image processing, and the pixel data 1 to 4 correspond to the image processing A. Therefore, after the determination in the data determination unit 13, In the image processing A circuit 14a, the pixel data 1 to 4 are subjected to image processing one after another.

  When the pixel data 5 is input to the data determination unit 13 subsequent to the pixel data 4, it is determined that switching from the image processing A to the image processing B is necessary because the pixel data 5 corresponds to the image processing B. (S2). However, since the pixel data 1 to 4 are image-processed in the image processing A circuit 14a at the time of this determination, it cannot be switched to the image processing B circuit 14b.

  Therefore, it waits for the pixel data 4 immediately before the pixel data 5 determined to be switched to be processed in the image processing A circuit 14a and output (saved) to the data output unit 16 (S3). After the processed pixel data 4 is output to the data output unit 16, the image processing A circuit 14a is partially reconfigured to the image processing B circuit 14b (S4).

  As described above, in switching of image processing by partial reconstruction, for each switching, a time for saving data (S3) (for example, about 100 clocks) and a time for partial reconstruction (S4) (for example, 19 clocks). Degree) is required. Therefore, partial reconstruction is preferable in terms of processing time when the number of times of switching is small, whereas it is not preferable in terms of processing time when the number of times of switching is large. The number of times of switching depends on the state of pixel data in the image data.

  Therefore, in this image processing apparatus, as outlined with reference to FIG. 2, prior to the image processing A and the image processing B, the state in the image data is analyzed in the circuit configuration 1, and the entire reconstruction or partial reconstruction is performed. Which one should be selected is determined.

  FIG. 4 is a diagram for explaining the reconstruction determination process. The reconfiguration determination process is executed by a circuit related to the reconfiguration determination from the data input unit 102 to the data output unit 112, which is configured in the reconfigurable circuit 10 in the circuit configuration 1 of FIG. The processing executed by the circuit relating to the reconstruction determination in FIG. 2 will be described in detail with reference to FIG. In addition, about the structure (part) shown in FIG. 2, the code | symbol of FIG. 2 is utilized also in the following description.

  First, image data to be determined is input to the data input unit 102. The image data to be determined is image data subjected to image processing A and image processing B. Therefore, for example, the image data subjected to the preceding image processing obtained from the data output unit 16p is input to the data input unit 102. If the preceding image processing does not affect the result of the reconstruction determination, for example, when the color conversion processing (image processing A and image processing B) is used separately according to the brightness of each pixel data, the preceding image is processed. If the process is a process that does not change the brightness of each pixel data, the image data before the preceding image process input to the data input unit 12p may be input to the data input unit 102. Image data input to the data input unit 102 is processed in the following units in the following manner.

<1> Image Division The image data input to the data input unit 102 is sent to the image division unit 104. The image dividing unit 104 divides the image data into a plurality of image areas. The image data is composed of a plurality of pixel data. As shown in <1> in FIG. 4, each line is formed in order from the line on the upper side for each line along the horizontal direction (the dashed line arrow). A plurality of pixel data to be transmitted is sequentially sent from the data input unit 102 to the image dividing unit 104.

  For example, the image dividing unit 104 sets a bundle of the number of lines determined according to the size (number of pixels) of the image data as one image region. As a result, the image data is divided into a plurality of image areas. Note that, although it is desirable that the sizes of the plurality of image regions are equal, the sizes are not limited to being equal.

<2> Image Analysis Pixel data sequentially output from the data input unit 102 is further sent from the image dividing unit 104 to the image analysis unit 106. The image analysis unit 106 analyzes a mixed state of a plurality of pixel data corresponding to the image processing A and a plurality of pixel data corresponding to the image processing B for each image region to obtain an analysis result.

  In <2> of FIG. 4, a broken-line rectangle with A or B inside indicates pixel data, and pixel data A with A corresponds to image processing A, and B is attached. The pixel data B corresponds to the image processing B. The image analysis unit 106 counts the number of pixel data A, the number of pixel data B, and the number of times of switching from pixel data A to pixel data B or from pixel data B to pixel data A, and counts them for each image area. Get a number. In this way, as shown by <2> in FIG. 4, the number of data dan of the image processing A, the number of data dbn of the image processing B, and the number of switching times Nn of the image processing B are obtained for each image region n.

<3> Processing Time Prediction The processing time prediction unit 108 uses processing time when using full reconstruction and partial reconstruction for each image region n based on the analysis result obtained by the image analysis unit 106. Predict the processing time. Specifically, the entire reconstruction prediction processing time Twn and the partial reconstruction prediction processing time Tpn are calculated for each image region n by the following equations.

(1) Twn = (Pa x dalln) + (Pb x dalln) + Trw
(2) Tpn = (Pa x dan) + (Pb x dbn) + (Nn x Trp)

  Note that Pa is one stage of processing time (for example, one clock) in the image processing A, Pb is one stage of processing time (for example, one clock) in the image processing B, and dalln is in the image area n. This is the number of all pixel data. Trw is the time required for one full reconstruction, and Trp is the time required for one partial reconstruction. Trp is obtained by adding the time for data saving (S3) described with reference to FIG. 3 and the time for partial reconstruction (S4).

<4> Reconfiguration Determination The reconstruction determination unit 110 has a short prediction processing time of full reconstruction or partial reconstruction for each image region n based on the prediction processing time obtained in the processing time prediction unit 108. Is obtained as a reconstruction process in the image region n. That is, for each image region n, the entire reconstruction prediction processing time Twn and the partial reconstruction prediction processing time Tpn are compared. If Twn is equal to or less than Tpn, the entire reconstruction is selected, and if Tpn is smaller than Twn, the partial reconstruction is performed. A configuration is selected. Note that partial reconstruction may be selected when Twn and Tpn are equal.

  Reconstruction determination results for all image areas n obtained by the reconstruction determination unit 110 are sent to the data output unit 112, and the entire image area n is reconstructed for each image area n in image processing A and image processing B to be executed later. Referenced when selecting configuration or partial reconfiguration. For example, according to the determination result obtained in the reconstruction determination unit 110, the control unit 20 illustrated in FIG. 1 selects the reconstruction data 1 or the reconstruction data 2 stored in the reconstruction data storage unit 30, and the reconstruction is performed. The circuit configuration in the reconfigurable circuit 10 at the time of image processing A and B shown in FIG.

  When the smaller one of Twn or Tpn is selected one after another for each image region n, the processing time Td of all image data composed of all the image regions n is calculated as follows. In the following equation, min (Twn, Tpn) means the smaller of Twn and Tpn.

(3) Td = min (Tw1, Tp1) + min (Tw2, Tp2) + ... + min (Twn, Tpn) + ...

  Incidentally, the total processing time Tw when full reconstruction is selected for all image areas n and the total processing time Tp when partial reconstruction is selected for all image areas n are as follows.

(4) Tw = Tw1 + Tw2 + ... + Twn + ...
(5) Tp = Tp1 + Tp2 ++ ... + Tpn + ...

  Td in equation (3) is less than or equal to Tw in equation (4), and Td in equation (3) is also less than or equal to Tp in equation (5). That is, the present image processing apparatus that selectively uses full reconstruction or partial reconstruction according to the image region n, as compared with the case where the entire reconstruction or partial reconstruction is used consistently in all image regions n. This is advantageous in terms of processing time.

  In the circuit configuration 2 of FIG. 2, the data input unit 12a, the image processing A circuit 14a, and the data output unit 16a that are circuit configurations related to the image processing A are combined with the data input unit 12b that is a circuit configuration related to the image processing B. An example in which the image processing B circuit 14b and the data output unit 16b are entirely reconfigured is shown. Instead of the example of the entire reconstruction, only the image processing A circuit 14a and the image processing B circuit 14b may be partially reconfigured. That is, first, the data input unit 12a, the image processing A circuit 14a, and the data output unit 16a, which are circuit configurations related to the image processing A, are configured in the reconfigurable circuit 10, and all the image data of the image area to be processed is stored. On the other hand, image processing A is performed. Then, after the image processing A related to the image area is completed, only the image processing A circuit 14a is partially reconfigured into an image processing B circuit 14b, and the image processing B is applied to all image data in the image area to be processed. You may make it give.

  Further, when executing the image processing A and the image processing B, a division determination as to whether to divide the image data may be added. If the image processing A is applied to the entire area of the image data without dividing the image data and then the image processing B is reconfigured and the image processing B is applied, the processing time is as follows: .

(6) Tw´ = (Pa x dall) + (Pb x dall) + Trw

  Pa is the processing time of one stage in the image processing A (for example, one clock), Pb is the processing time of one stage in the image processing B (for example, one clock), and dall is all the pixels constituting the image data. This is the number of data. Trw is the time required for one full reconstruction. That is, in this case, only one reconstruction process is required to process the entire image data. Therefore, when Tw ′ in the expression (6) is compared with Td in the expression (3), Tw ′ may be smaller than Td depending on the state of the pixel data in the image data. Therefore, division determination is performed by comparing Tw ′ and Td. If Tw ′ is equal to or less than Td, image processing A is applied to the entire area of the image data without dividing the image data, and image processing is performed. The non-divided reconstruction in which the image processing B is applied by reconfiguring the circuit according to the above may be used.

  In the specific example 1 described with reference to FIGS. 2 to 4, the processing time is considered when selecting the full reconstruction or the partial reconstruction, but the power consumption is considered instead of the processing time or together with the processing time. Thus, full reconstruction or partial reconstruction may be selected. Therefore, specific example 2 in consideration of power consumption will be described below.

  FIG. 5 is a diagram for explaining a specific example 2 of the circuit configured in the reconfigurable circuit 10. This image processing apparatus performs normal power consumption image processing N or low power consumption image processing N ′. A circuit configured in the reconfigurable circuit 10 when applied to image data is shown.

  In FIG. 5, a circuit configuration 1 ′ indicates a circuit configuration in the reconfigurable circuit 10 before execution of the image processing N and N ′, and the circuit configuration 2 ′ and the circuit configuration 3 ′ are image processing N and N The circuit configuration in the reconfigurable circuit 10 at the time of execution of 'is shown. When the image processing N, N ′ is executed, full reconstruction (circuit configuration 2 ′) or partial reconstruction (circuit configuration 3 ′) is selected. The selection of full reconstruction or partial reconstruction is performed based on the state of pixel data in the image data to be processed by the image processing N and N ′. Therefore, prior to image processing N and N ′, the state in the image data is analyzed in the circuit configuration 1 ′ to determine whether to select full reconstruction or partial reconstruction.

  Specifically, as the circuit configuration 1 ′, the data input unit 102, the image division unit 104, the image analysis unit 106, the power consumption prediction unit 109, the reconstruction determination unit 110, and the data output unit 112 are included in the reconfigurable circuit 10. Is constructed, the state in the image data to be processed by the image processing N or the image processing N ′ is analyzed, and the reconstruction is determined. These circuits (circuits related to the reconstruction determination) from the data input unit 102 to the data output unit 112 are configured based on the reconstruction data stored in the reconstruction data storage unit 30 in FIG. The processing by the circuit relating to the reconstruction determination will be described in detail later using FIG.

  Further, processing in the circuit relating to reconstruction determination is executed prior to image processing N and N ′. The execution is desirably performed in parallel with some other process, for example, the preceding image process executed before the image processes N and N ′.

  Therefore, as shown in FIG. 5, as the circuit configuration 1 ′, a data input unit 12p, a preceding image processing circuit 14p, and a data output unit 16p are also configured in the reconfigurable circuit 10. These circuits related to the preceding image processing are configured based on the reconstruction data stored in the reconstruction data storage unit 30 in FIG. Then, when the image data to be processed is input to the data input unit 12p, a plurality of pixel data in the image data are sequentially sent to the preceding image processing circuit 14p to perform the preceding image processing, and after the processing A plurality of pixel data is sent to the data output unit 16p. Thus, when all the pixel data in the image data is processed and sent to the data output unit 16p, the preceding image processing for the image data is completed.

  When the processing related to the reconstruction determination by the circuit configuration 1 ′ and the preceding image processing are completed, the reconfigurable circuit 10 is reconfigured into the circuit configuration 2 ′ or the circuit configuration 3 ′ according to the result of the reconstruction determination. Image processing N, N ′ is executed. In the execution, the circuit configuration 2 ′ or the circuit configuration 3 ′ is selected for each image area for the image data divided into a plurality of image areas.

  The circuit configuration 2 ′ indicates a circuit configuration in the reconfigurable circuit 10 in the case of full-surface reconfiguration, and is configured based on the reconfiguration data stored in the reconfiguration data storage unit 30 in FIG.

  In the entire reconfiguration, first, the reconfigurable circuit 10 includes a data input unit 12, a normal power consumption image processing circuit 14n, and a data output unit 16 that are circuit configurations related to the normal power consumption image processing N. . When the image data of the image area to be processed is input to the data input unit 12a, a plurality of pixel data in the image data are sequentially sent to the normal power consumption image processing circuit 14n to perform image processing N. The processed pixel data is sent to the data output unit 16. In this way, when all the pixel data in the image area is processed and sent to the data output unit 16, the image processing N relating to the image area ends.

  As described above, in the entire surface reconstruction, the inside of the reconfigurable circuit 10 is reconfigured so that the image processing N is performed over the entire image area while maintaining the normal power consumption image processing circuit 14n.

  On the other hand, the circuit configuration 3 ′ shows the circuit configuration in the reconfigurable circuit 10 in the case of partial reconfiguration, and is configured based on the reconfiguration data stored in the reconfiguration data storage unit 30 in FIG. Is done.

  In the partial reconstruction, a data input unit 12, a data determination unit 13, and a data output unit 16 are configured in the reconfigurable circuit 10, and a normal power consumption image processing circuit is further used according to pixel data to be processed. 14n and the low power consumption image processing circuit 14n ′ are configured to be selectively switched.

  That is, when image data of an image area to be processed is input to the data input unit 12, a plurality of pixel data in the image data are sequentially sent to the data determination unit 13. For each pixel data, the data determination unit 13 determines whether the pixel data requires processing by the normal power consumption image processing N or can be processed by the low power consumption image processing N ′. To do.

  When it is determined that certain pixel data needs to be processed by the image processing N, a normal power consumption image processing circuit 14n is formed in the reconfigurable circuit 10, and the normal power consumption image processing circuit 14n The pixel data is processed. On the other hand, when it is determined that certain pixel data can be processed by the image processing N ′, a low power consumption image processing circuit 14n ′ is formed in the reconfigurable circuit 10 and the low power consumption image processing circuit 14n. The pixel data is processed at '. Thereafter, the normal power consumption image processing circuit 14n and the low power consumption image processing circuit 14n ′ are switched in accordance with the pixel data.

  In this way, when all the pixel data in the image area is processed and sent to the data output unit 16, the image processing N and N ′ relating to the image area is completed.

  As described above, in the partial reconstruction, a reconfigurable circuit is used so as to use the image processing N or the image processing N ′ corresponding to the pixel data for each piece of pixel data subjected to image processing one after another in a certain image region. 10 is reconfigured.

  FIG. 6 is a diagram for explaining switching between normal power consumption and low power consumption. FIG. 6 shows a run compression process (run length compression process) as a specific example of the normal power consumption image processing circuit 14n and the low power consumption image processing circuit 14n ′ in the circuit configuration 3 ′ (FIG. 5). .

  In the run compression process, for multiple pixel data that make up image data, when multiple pixel data are continuous with the same value, the number of continuous pixel data (run length) corresponds to the value (data value) of the pixel data. By obtaining the attached processing result, the data amount of the entire image data is compressed.

  The run counter shown in FIG. 6 counts the run length, which is the number of consecutive pixel data with the same data value, for a plurality of pixel data obtained one after another via the data determination unit 13 (FIG. 5). Then, the compressed data generation unit shown in FIG. 6 generates a processing result in which the run lengths of the continuous pixel data with the same data value are associated with the data value, and outputs the processing result to the data output unit 16 (FIG. 5).

  As described above, the compressed data generation unit may use the run length counting result in the run counter and the data value of the pixel data to be counted. Therefore, a circuit configuration that does not use the compressed data generation unit during the run length counting by the run counter is possible.

  Therefore, this image processing apparatus selectively selects a normal power consumption image processing circuit 14n having a run counter and a compressed data generation unit, and a low power consumption image processing circuit 14n 'having a run counter but no compression data generation unit. To use.

  That is, when a plurality of pixel data in the image data is sequentially sent to the data determination unit 13, the data determination unit 13 requires that the pixel data be processed by the normal power consumption image processing circuit 14n for each pixel data. It is determined whether it is possible to perform processing by the low power consumption image processing circuit 14n ′. For example, when pixel data having the same data value is continuous, the low power consumption image processing circuit 14n ′ is selected, and the run length is counted by the run counter in the low power consumption image processing circuit 14n ′.

  When a plurality of pixel data consecutive with the same data value are sequentially processed, and subsequently pixel data having different data values are sent to the data determination unit 13, processing by the normal power consumption image processing circuit 14n is necessary. It is determined that there is. The compressed data generation unit is reconfigured at the subsequent stage of the run counter while maintaining the run length counting result in the run counter. That is, the normal power consumption image processing circuit 14n is reconfigured. Further, the compressed data generation unit formed by the reconstruction generates a processing result in which the run length counting result in the run counter is associated with the counted data value of the pixel data, and outputs the processing result to the data output unit 16. .

  Thereafter, when pixel data having the same data value continues, the low power consumption image processing circuit 14n ′ is reconfigured, and the run length is counted by the run counter in the low power consumption image processing circuit 14n ′. The Of course, the processing by the normal power consumption image processing circuit 14n may be continued according to, for example, the run length.

  In this way, by using the low power consumption image processing circuit 14n ′, the power consumption of the entire circuit is reduced by the amount of power consumption in the compressed data generation unit compared to the case of using only the normal power consumption image processing circuit 14n. can do. However, power is consumed when the normal power consumption image processing circuit 14n and the low power consumption image processing circuit 14n ′ are switched, that is, in partial reconfiguration for switching between them. Therefore, partial reconstruction using the low power consumption image processing circuit 14n ′ is preferable in terms of power consumption when the number of times of switching is small, whereas it is not preferable in terms of power consumption when the number of times of switching is large. The number of times of switching depends on the state of pixel data in the image data.

  Therefore, in this image processing apparatus, as outlined with reference to FIG. 5, the state in the image data is analyzed in the circuit configuration 1 ′ prior to the image processing N and N ′, and the entire reconstruction or partial reconstruction is performed. Which one should be selected is determined.

  FIG. 7 is a diagram for explaining a reconstruction determination process in normal power consumption and low power consumption image processing. This reconfiguration determination process is executed by a circuit related to reconfiguration determination from the data input unit 102 to the data output unit 112, which is configured in the reconfigurable circuit 10 in the circuit configuration 1 ′ of FIG. The processing executed by the circuit relating to the reconstruction determination in FIG. 5 will be described in detail with reference to FIG. In addition, about the structure (part) shown in FIG. 5, the code | symbol of FIG. 5 is utilized also in the following description.

  First, image data to be determined is input to the data input unit 102. The image data to be determined is image data subjected to image processing N, N ′ such as run compression processing. For example, image data subjected to the preceding image processing obtained from the data output unit 16p is input to the data input unit 102. If the preceding image processing does not affect the result of the reconstruction determination, the image data before the preceding image processing input to the data input unit 12p may be input to the data input unit 102. Image data input to the data input unit 102 is processed in the following units in the following manner.

<1> Image Division The image data input to the data input unit 102 is sent to the image division unit 104. The processing by the image dividing unit 104 in FIG. 5 is the same as the processing by the image dividing unit 104 in FIG. That is, the image dividing unit 104 divides the image data into a plurality of image areas. As a result, the image data is divided into a plurality of image areas.

<2> Image Analysis Pixel data sequentially output from the data input unit 102 is further sent from the image dividing unit 104 to the image analysis unit 106. The image analysis unit 106 analyzes, for each image area, a mixed state of a plurality of pixel data that requires the image processing N with normal power consumption and a plurality of pixel data that can be processed by the image processing N ′ with low power consumption. To obtain the analysis results.

  In <2> of FIG. 7, a broken-line rectangle with an alphabet inside indicates pixel data, and a plurality of pixel data with the same alphabet corresponds to the same data value. For example, a plurality of pixel data with A attached therein corresponds to the same data value A, and a plurality of pixel data with B added therein correspond to the same data value B.

  The image analysis unit 106 counts the number of data whose run length is equal to or greater than the threshold and the number of times that the image processing circuit needs to be switched, and obtains the count number for each image region. In this way, as shown in <2> in FIG. 7, the number of data hitn whose run length is equal to or greater than the threshold Nth and the number of times Nn of switching of the image processing circuit are obtained as analysis results for each image region n. Note that the threshold Nth regarding the run length is a threshold for setting how much low power consumption is required, and is set by the user, for example. Incidentally, in the partial specific example relating to the plurality of pixel data shown in <2> of FIG. 7, if the threshold value Nth = 3, the run length is 3 or more in this part is given an A inside. 5 pixel data and 3 pixel data with C added inside, and the number of data hitn = 8.

  In addition, a plurality of pixel data counted as the number of data hitn is a processing target of the low power consumption image processing N ′, and a plurality of pixel data not counted as the number of data hitn is processed in the normal power consumption image processing N ′. Be targeted. Therefore, in the specific example of the plurality of pixel data shown in <2> of FIG. 7, the number of switching of the image processing circuit in this part is Nn = 3.

<3> Power Consumption Prediction The power consumption prediction unit 109 uses the power consumption when using full reconstruction and partial reconstruction for each image region n based on the analysis result obtained by the image analysis unit 106. Predict the power consumption when you do. Specifically, for each image region n, the total reconstruction predicted power consumption Twn and the partial reconstruction predicted power consumption Tpn are calculated by the following equations.

(7) Twn = (Wa × dalln) + Trw
(8) Tpn = (Wa x (dalln-hitn)) + (Wb x hitn) + (Nn x Trp)

  Wa is power consumption necessary for processing one data of image processing N (for example, run compression processing) of normal power consumption, and Wb is one data of image processing N ′ (for example, run compression processing) of low power consumption. This is the power consumption required for processing. Further, dalln is the number of all pixel data in the image region n, Trw is the power consumption required for one full reconstruction, and Trp is the power consumption required for one partial reconstruction.

<4> Reconstruction Determination The reconstruction determination unit 110 has a small predicted power consumption of full reconstruction or partial reconstruction for each image region n based on the predicted power consumption obtained by the power consumption prediction unit 109. Is obtained as a reconstruction process in the image region n. That is, for each image region n, the total reconstruction predicted power consumption Twn and the partial reconstruction predicted power consumption Tpn are compared. If Twn is equal to or less than Tpn, the full reconstruction is selected, and if Tpn is smaller than Twn, partial reconstruction is performed. A configuration is selected. Note that partial reconstruction may be selected when Twn and Tpn are equal.

  Reconstruction determination results for all image regions n obtained by the reconstruction determination unit 110 are sent to the data output unit 112, and the entire surface reconstruction is performed for each image region n in image processing N and N ′ to be executed later. Or it is referred when selecting partial reconstruction. For example, in accordance with the determination result obtained in the reconfiguration determination unit 110, the control unit 20 illustrated in FIG. 1 is based on the reconfiguration data stored in the reconfiguration data storage unit 30. By reconfiguring the circuit, the circuit configuration in the reconfigurable circuit 10 at the time of image processing N and N ′ shown in FIG. 5 is realized.

  When the smaller one of Twn or Tpn is selected one after another for each image region n, the power consumption Td of all the image data composed of all the image regions n is calculated as follows. In the following equation, min (Twn, Tpn) means the smaller of Twn and Tpn.

(9) Td = min (Tw1, Tp1) + min (Tw2, Tp2) + ... + min (Twn, Tpn) + ...

  Incidentally, the total power consumption Tw when full reconstruction is selected for all image areas n and the total power consumption Tp when partial reconstruction is selected for all image areas n are as follows.

(10) Tw = Tw1 + Tw2 ++ ... Twn + ...
(11) Tp = Tp1 + Tp2 ++ ... + Tpn + ...

  Td in equation (9) is less than or equal to Tw in equation (10), and Td in equation (9) is also less than or equal to Tp in equation (11). That is, the present image processing apparatus that selectively uses full reconstruction or partial reconstruction according to the image region n, as compared with the case where the entire reconstruction or partial reconstruction is used consistently in all image regions n. This is more advantageous in terms of power consumption.

  In executing the image processing N or the image processing N ′, a division determination as to whether to divide the image data may be added. When the image processing N with normal power consumption is applied to the entire image data without dividing the image data, the power consumption is expressed by the following equation.

(12) Tw´ = (Wa × dall) + Trw

  Wa is power consumption required for processing one data of image processing N (for example, run compression processing) of normal power consumption, and dall is the number of all pixel data constituting the image data. Trw is the time required for one full reconstruction. That is, in this case, only one reconstruction process is required to process the entire image data. Therefore, when Tw ′ in the expression (12) is compared with Td in the expression (9), Tw ′ may be smaller than Td depending on the state of the pixel data in the image data. Therefore, division determination is performed by comparing Tw ′ and Td. When Tw ′ is equal to or less than Td, image processing N with normal power consumption is applied to the entire area of the image data without dividing the image data. Non-division reconstruction may be used.

  As mentioned above, although preferred embodiment of this invention was described, embodiment mentioned above is only a mere illustration in all the points, and does not limit the scope of the present invention. The present invention includes various modifications without departing from the essence thereof. For example, full reconstruction or partial reconstruction may be determined based on a condition that combines the processing time described as specific example 1 and the power consumption described as specific example 2. Furthermore, a plurality of full reconstruction patterns and a plurality of partial reconstruction patterns may be prepared, and a reconstruction pattern corresponding to image data may be selected from the plurality of patterns.

  DESCRIPTION OF SYMBOLS 10 Reconfigurable circuit, 14a Image processing A circuit, 14b Image processing B circuit, 20 Control part, 30 Reconstruction data memory | storage part, 104 Image division part, 106 Image analysis part, 108 Processing time prediction part, 109 Power consumption prediction part 110 Reconfiguration determination unit.

Claims (4)

An image processing apparatus for processing an image composed of a plurality of pixel data,
A control unit constituting a plurality of image processing units in the reconfigurable circuit;
For each image region of the image divided into a plurality of image regions, an analysis unit that analyzes an array state of a plurality of pixel data in the image region;
For each image region, based on the result of the analysis, a prediction unit that predicts the image processing performance of the full reconstruction process and the image processing performance of the partial reconstruction process;
For each image area, a determination unit that obtains a determination result that the full reconstruction process or the partial reconstruction process selected based on the comparison of the image processing performance is a reconstruction process corresponding to the image area;
Have
The controller is
In the image processing for the image, depending on the determination result for each image region,
At least one image processing unit is configured to perform image processing on a plurality of pixel data over the entire area of each image region while maintaining at least one image processing unit in the reconfigurable circuit. Complete reconfiguration process,
Or
A plurality of image processing units are configured to perform image processing on a plurality of pixel data in each image region by each image processing unit corresponding to each pixel data while switching between the plurality of image processing units in the reconfigurable circuit. Partial reconstruction process,
Select and execute,
An image processing apparatus. The image processing apparatus according to claim 1.
Each pixel data constituting the image requires image processing by each image processing unit corresponding to the pixel data,
The analysis unit analyzes a mixed state of a plurality of pixel data corresponding to a plurality of image processing units based on a correspondence relationship between each pixel data and each image processing unit for each image region,
For each image region, the prediction unit predicts the processing time by the full reconstruction process and the processing time by the partial reconstruction process based on the result of the analysis,
For each image area, the determination unit obtains a determination result as a reconstruction process corresponding to the image area, which has a smaller processing time of the full reconstruction process or the partial reconstruction process,
The controller is
In the entire reconstruction process, the entire image area in each image region is processed by one image processing unit, and then the entire image region in the image region before image processing is processed by another image processing unit. Configure multiple image processing units,
In the partial reconstruction processing, a plurality of image processing units are configured so that each pixel data subjected to image processing in each image region is subjected to image processing by each image processing unit corresponding to the pixel data.
An image processing apparatus. The image processing apparatus according to claim 1 or 2,
Each pixel data constituting the image is subjected to image processing by a normal power consumption image processing unit or a low power consumption image processing unit that is equal to or lower than normal power consumption,
The analysis unit analyzes, for each image area, a mixed state of a plurality of pixel data that can be processed by a low power consumption image processing unit and a plurality of pixel data that needs to be processed by a normal power consumption image processing unit. And
The prediction unit predicts the power consumption by the full reconstruction process and the power consumption by the partial reconstruction process based on the result of the analysis for each image region,
For each image area, the determination unit obtains a determination result as a reconstruction process corresponding to the image area with the smaller power consumption of the full reconstruction process or the partial reconstruction process,
The controller is
In the entire reconstruction process, the normal power consumption is performed so that a plurality of pixel data is image-processed over the entire image area while maintaining the normal power consumption image processing unit in the reconfigurable circuit. The image processing unit of
In the partial reconstruction processing, each pixel data that is successively image-processed in each image region is subjected to image processing by a normal power consumption image processing unit or a low power consumption image processing unit according to the pixel data. , Configure by switching those image processing units,
An image processing apparatus. In the image processing device according to any one of claims 1 to 3,
Prior to image processing on the image by the plurality of image processing units, a preceding image processing unit, the analysis unit, the prediction unit, and the determination unit are configured in the reconfigurable circuit,
In parallel with the image processing on the image by the preceding image processing unit, the analysis unit performs the analysis, the prediction unit performs the prediction, and the determination unit obtains the determination result.
An image processing apparatus.

Images