JP4814679B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus capable of forming a selection signal for selecting a function processing circuit for performing image processing by learning.

  In recent years, an image input means such as a TV camera or a CCD camera is used to capture an image of a subject or landscape, and image processing is performed on the obtained moving image, and a specific target, for example, an object that moves within an environment, from the image. Research on image processing apparatuses for extracting the movements and the like has been actively conducted.

As image processing for still images, an image processing technique (ACTIT, see Non-Patent Document 1) that develops image processing on an input image based on a processing program in which various image filters are combined in a tree structure as shown in FIG. Has been.
Shinya Aoki, 1 other person, “Automatic construction method of tree-structured image conversion ACTIT”, The Journal of the Institute of Image Information and Television Engineers, The Institute of Image Information and Television Engineers, 1999, Vol. 53, No. 6, p. 888-894

  However, the image processing apparatus as described above can perform image processing only within the range of combinations of image filters used for image processing, and may not always perform desired image processing. Moreover, the optimization of image processing limited by either providing learning time in advance what kind of image the filter by any number respectively.

  On the other hand, when such an image processing apparatus is mounted on a vehicle, a robot, or the like, for example, if the image processing apparatus is configured using a general-purpose computer and the processing program is configured to perform software processing, the image processing takes time. . For this reason, it is not always easy to process image data in real time, that is, within a practical time.

  The present invention has been made in view of the above circumstances, and provides an image processing apparatus capable of learning an image processing procedure that makes it possible to construct a desired image processing circuit without being limited to an image filter. For the purpose. Furthermore, it aims at providing the image processing apparatus which can improve the processing speed of image processing.

In order to solve the above problem, the first invention provides:
An image processing apparatus for learning image processing procedures,
A learning unit comprising an arithmetic unit in which function processing circuits are arranged in parallel and a selector for selecting data to be input to each function processing circuit;
Learning means for optimizing the order of operations by the learning unit;
A plurality of the learning units are connected in series,
Each selector of the plurality of learning units selects data to be input to each function processing circuit according to a selection signal indicating an image processing procedure,
The learning means optimizes the selection signal to be assigned to the selector of each learning unit by learning, uses the sequence of the selection signal to be assigned to the selector of each learning unit as the learning as a gene, and a target to be obtained by image processing Learning based on the method of genetic algorithm with images as teacher images,
The function processing circuit includes a delay circuit having a function of delaying input data by one clock,
In the learning, the learning unit calculates an evaluation value based on the number of learning units that perform only the delay circuit .

The second invention is an image processing apparatus of the first aspect of the invention, the learning means, a signal parts corresponding to the calculated optimized the selected signal to the arithmetic processing by the learning unit performs only operation of the delay processing A selection signal that has been deleted and newly optimized is reconstructed and output.

According to a third aspect of the present invention, there is provided a learning unit comprising an arithmetic unit in which function processing circuits are arranged in parallel and a selector for selecting data to be input to each function processing circuit
Learning means for optimizing the order of operations by the learning unit;
A plurality of the learning units are connected in series,
Each selector of the plurality of learning units selects data to be input to each function processing circuit according to a selection signal indicating an image processing procedure,
The learning means is an image processing apparatus that optimizes the selection signal to be assigned to a selector of each learning unit by learning ,
If there is a learning unit that performs only delay processing when image processing is performed according to the selection signal optimized by the learning means of the image processing apparatus, the arithmetic processing by the learning unit is deleted, and the learning unit If there is another learning unit that performs operations other than delay processing on the downstream side of the process, the operation processing of the other learning unit is transferred to the upstream learning unit, and the most downstream learning that performs operations other than delay processing The processing result output from the unit is output.

According to a fourth aspect of the present invention, there is provided a learning unit comprising an arithmetic unit in which function processing circuits are arranged in parallel and a selector for selecting data to be input to each function processing circuit
Learning means for optimizing the order of operations by the learning unit;
A plurality of the learning units are connected in series,
Each selector of the plurality of learning units selects data to be input to each function processing circuit according to a selection signal indicating an image processing procedure,
The learning means optimizes the selection signal assigned to the selector of each learning unit by learning ,
An image processing apparatus that performs image processing based on the optimized selection signal,
When there is an arithmetic process corresponding to a learning unit that performs only a delay process when the image processing apparatus performs an image process according to the selection signal, the signal portion corresponding to the arithmetic process is deleted from the selection signal. The selection signal is reconstructed, and an image processing circuit by the function processing circuit is constructed based on the reconstructed selection signal.

  According to the first invention, it is possible to provide an image processing procedure for an image processing circuit constructed by a combination of simple function processing circuits such as an adder instead of a combination of image filters as in the conventional image processing apparatus. it can. Therefore, according to the procedure, it is possible to construct an image processing circuit capable of performing desired image processing without being limited to the type of image processing or the combination thereof.

  Also, an image processing procedure for constructing an image processing circuit capable of performing desired image processing with higher accuracy by performing learning based on a plurality of teacher images to learn desired image processing is provided. It becomes possible.

  Further, according to the optimized selection signal output from the image processing apparatus according to the first aspect of the invention, image processing equivalent to software processing by the processing program is performed by hardware processing and arithmetic processing is performed in units of clocks. As a result, the image processing speed of the image processing apparatus can be dramatically improved, and the image data can be processed in real time.

According to the first invention, heritage to obtain an optimized selection signal by learning based on techniques of heat transfer algorithm, as compared with the case of configuring the artificially optimized selection signals efficiently In addition, it is possible to acquire a selection signal that is more accurately optimized.

According to a further first aspect of the invention, in the optimization of the selection No.択信, calculates an evaluation value based on the number of learning units only delay circuit does not perform an operation for selecting the signal containing a large amount of such a learning unit By giving a better evaluation, the optimized selection signal includes many arithmetic processes by a learning unit that performs only a delay circuit.

  Therefore, at the stage of implementation in the device, for example, by deleting the arithmetic processing by the learning unit that performs only the delay circuit, the processing time can be further reduced by the deleted learning unit, and the processing speed can be reduced. Further improvement is possible. In addition, it is possible to configure an image processing circuit with fewer resources by the number of deleted learning units.

According to the second invention, a signal portion corresponding to such a learning unit is deleted from the optimized selection signal including a large number of arithmetic processes by a learning unit that performs only a delay circuit and newly optimized. By reconfiguring the selection signal, it is possible to further shorten the processing time by the deleted learning unit and further improve the processing speed in the apparatus in which the reconfigured selection signal is mounted. Become. In addition, it is possible to configure an image processing circuit with fewer resources by the number of deleted learning units. Thus, in the second invention, the effects of the first invention are more accurately exhibited.

According to the third invention, in the image processing device, the arithmetic processing by the learning unit that performs only the arithmetic processing of the delay processing is deleted, and the arithmetic processing other than the delay processing performed on the processing downstream side of the learning unit is moved to the upstream side. Thus, the effects of the above-described inventions can be exhibited accurately, and the processing time can be further shortened and the processing speed can be improved by the amount that the arithmetic processing of the learning unit that performs only the arithmetic processing of the delay processing is deleted. . In addition, it is possible to configure an image processing circuit with fewer resources by the number of deleted learning units.

According to the fourth invention, reconfigure the selection signal by removing the signal portion of the calculation processing corresponding to the learning unit performs only operation of the delay processing present in the optimized selectivity signals, reconstruction By performing image processing using the function processing circuit assembled based on the selected signal, the image processing circuit can be configured with fewer resources than the deleted signal portion. In addition, the processing time can be further shortened accordingly, and the processing speed can be improved.

  Embodiments of an image processing apparatus according to the present invention will be described below with reference to the drawings.

[First Embodiment]
In the first embodiment, a case where the image processing apparatus is built on an FPGA (Field Programmable Gate Array) evaluation board (manufactured by Altera) is described, but the present invention is not limited to this.

  The image processing apparatus 1 according to the present embodiment is an apparatus that autonomously learns image processing procedures. As shown in FIG. 1, the image processing apparatus 1 includes a processor 2, a shift register 3, a plurality of learning units 4, a multiplexer 5, and learning means 6.

  In this embodiment, the image data to be subjected to image processing is data of luminance values corrected to 256 gradations of 0 to 255 for each pixel constituting the image. The luminance value data is referred to as pixel data. In the present embodiment, a case where the image size is 256 × 256 pixels and image processing is performed on 3 × 3 pixels in the image will be described.

  In the present embodiment, the image data is 1 on the line of 1 pixel width extending in the horizontal direction of the original image in the order from the leftmost pixel to the right, and sequentially from the uppermost line to the lower line. A case will be described in which pixel-by-pixel input and image processing are performed.

  The processor 2 transmits a clock signal CLK to the shift register 3, the learning unit 4, and the multiplexer 5, and operates them in synchronization with the clock. Further, a selection signal SS transmitted from learning means 6 described later is transmitted to each learning unit 4 and multiplexer 5.

  The shift register 3 has a line buffer function and can temporarily store at least 514 pixel data. As shown in FIGS. 2A to 2C, pixel data is sequentially input from the learning means 6 to the shift register 3 every clock, and the shift register 3 sequentially inputs these pixel data. It is designed to shift each time. 2A to 2C, 00, 01, and 02 represent pixel data of the 0th row, the 0th column, the 0th row, the 1st column, and the 0th row, the 2nd column of the image, respectively. ing.

  The shift register 3 has 0, 1, 255, 256, 257, 511, 512, and 513 centering on the pixel data at address 256 at the clock timing when the leading pixel data is shifted to address 513 as shown in FIG. Nine pixel data of the address pixel data and the pixel data input at this clock timing is transmitted to the learning unit 4.

  The shift register 3 shifts the pixel data at the next clock timing, and outputs the nine pixel data of the pixel data input at the clock timing and the pixel data input at the clock timing. The information is transmitted to the learning unit 4. Thereafter, this operation is repeated.

  When image processing is performed on n × n pixels in a 256 × 256 pixel image, the shift register 3 temporarily stores at least 257 × (n−1) pixel data as shown in FIG. It is desirable to have a capacity that can be stored.

  As shown in FIG. 5, the learning unit 4 includes an arithmetic unit 41 in which function processing circuits F1 to F9 are arranged in parallel, and a selector that selects image data to be input to the function processing circuits F1 to F9 of the arithmetic unit 41. 42. A clock signal CLK from the processor 2 is input to each of the function processing circuits F1 to F9 and the selector 42 of the arithmetic unit 41.

  The function processing circuit F of the arithmetic unit 41 includes a function processing circuit that performs simple operations such as a 2-input adder ADDR, a subtractor SUB, a 1-input multiplier MULT, an absolute value ABS, a delay circuit DFF, and an implication element IF_THEN. Is used.

  Here, in the present embodiment, the multiplier MULT multiplies the input data by a constant, and the constant is optimized by learning in the learning means 6. Moreover, in this embodiment, although the constant is made into the same value in each learning unit 4, it is good also as a different value. In addition, a delay flip-flop is used for the delay circuit DFF, and a delay of one clock is performed. Furthermore, in this embodiment, the implication element IF_THEN is set to 255 when the value of the input data is greater than 255.

  Hereinafter, as a configuration example of the function processing circuit F of the arithmetic unit 41, as shown in FIG. 6, four adders ADDR, a subtractor SUB, a multiplier MULT, an absolute value ABS, a delay circuit DFF, and an implication element IF_THEN, respectively. A case will be described in which a total of nine function processing circuits are combined one by one. However, the configuration of the function processing circuit F is not limited to this, and the type and number of function processing circuits F to be used are set as appropriate. The number of learning units 4 is also set as appropriate.

  A selection signal SS transmitted from the learning means 6 via the processor 2 is input to the selector 42. In the present embodiment, the selector 42 receives nine pieces of data from the shift register 3 and the learning unit 4 on the upstream side of the process, and each of the two inputs has four adders ADDR and subtracter SUB. A total of 14 pieces of data are output in order to input data one by one to the multiplier MULT, the absolute value ABS, the delay circuit DFF, and the implication element IF_THEN.

  The selector 42 has a structure in which one multiplexer-like element is associated with each output, as shown in the enlarged view in the circle of FIG. One data is selected and output from among the data, and is input to the corresponding function processing circuit F. In the present embodiment, the selector 42 includes 14 multiplexer-like elements.

  Further, each element of the selector 42 selects data to be input to the corresponding function processing circuit according to the selection signal SS transmitted from the learning means 6, or does not input data to the function processing circuit without selection. It has become.

  The selection signal SS is a signal for indicating the image processing procedure to each learning unit 4. In this embodiment, for example, as shown in FIG. 7, each element of the selector 42 and the multiplexer 5 of each learning unit 4 This is a selection signal column in which input data numbers to be selected are listed. The selector 42 of each learning unit 4 reads out a numerical value portion designated by itself from the selection signal SS, and assigns a selection signal to each element in advance. In the selection signal SS of FIG. 7, the numerical value part indicated by − indicates that the selection signal is not assigned, and no data is output from the corresponding element.

  In the example of the selection signal SS shown in FIG. 7, when the selection signal SS is input, the selector 42a of the learning unit 4a is 0 for the first element, 1 for the second element,. 4 is assigned to each of the 8th and 11th elements, and the selector 42b of the learning unit 4b sets up by assigning 0 to the first element,..., 3 to the fourth element, and 6 to the 13th element.

  As shown in FIG. 8, each element selects the input data of each number in accordance with the selection signal assigned in advance in this way, and transmits the data to the corresponding function processing circuit F. As described above, the selection signal is assigned to each element of the selector of each learning unit 4, whereby the image processing procedure by each learning unit 4 is determined.

  Further, the selector 42 outputs data from each element in synchronization with the rising edge of the clock signal CLK transmitted from the processor 2 and inputs the data to each corresponding function processing circuit F in synchronization with each other. The function processing circuit F starts the calculation at the same time when the next clock signal CLK rises, and outputs the data after the calculation.

  Note that the plurality of learning units 4a to 4n shown in FIG. 1 all have the same configuration as the learning unit 4 shown in FIGS. Then, the function processing circuit F to be processed is selected by the selection signal SS, and different operations are performed.

  The multiplexer 5 receives output data from the most downstream learning unit 4n. The multiplexer 5 is input in accordance with a selection signal SS transmitted from the learning means 6 via the processor 2. The data is selected from the selected data and output.

  As shown in FIG. 1, the learning means 6 is configured to optimize the selection signal SS assigned to each selector of the learning units 4 a to 4 n by learning in advance, and output the optimized selection signal SSsuit.

  In the present embodiment, learning is performed based on a genetic algorithm technique, and the learning means 6 includes an original image D used for learning and image processing as a teacher image as shown in FIG. A target image T as shown in FIG. 10 to be obtained is input. It is also possible to input a weighted image W as shown in FIG. 11 for weighting when calculating the evaluation value.

  Note that the target image T and the weight image W are created in advance based on the original image D. The target image T shown in FIG. 10 is created so that the luminance value of the pixel belonging to the extraction area EX is 255 and the luminance value of the pixel belonging to the non-extraction area NE is 0. Further, the weight image W shown in FIG. 11 is created so that the weight w corresponds to the reciprocal of the area ratio in each region corresponding to the extraction region EX and the non-extraction region NE of the target image T. It is like that.

  The original image D, the target image T, and the weight image W are stored in a storage unit (not shown) of the learning unit 6.

  On the other hand, in the learning means 6 according to the present embodiment, first, learning is performed on a predetermined number of pixels such as 10 pixels in the target image T that is a teacher image, and then learning is performed on all the pixels of the target image T. Is configured to do.

  In addition, in addition to the difference in the luminance value of each pixel between the output image Dout and the target image T, the individual evaluation is learned by calculating only the number of function processing circuits used for obtaining the output image Dout and the delay circuit. The determination is made on the basis of the number of learning units that perform substantial calculation excluding the unit.

  FIG. 12 is a block diagram showing the configuration of the learning means according to this embodiment. The learning unit 6 includes an initial individual generation unit 61, an evaluation value calculation unit 62, an end determination unit 63, a parent selection unit 64, a crossover unit 65, and a mutation unit 66.

  In response to the learning start instruction, the initial individual generating means 61 randomly selects an individual having, as a gene, a sequence of selection signals SS assigned to the selectors and multiplexers 5 of the learning units 4a to 4n as shown in FIG. A constant number such as 100 individuals is generated. The initial individual generating means 61 is configured to randomly generate a constant used by the multiplier MULT for each individual and write it in the header of the selection signal SS. When the constants are individually set in the learning units 4a to 4n, the constants are randomly generated for each of the learning units 4a to 4n and written in the header.

  When an individual is generated or crossed to be mutated, it is preferably set in advance so as not to generate, cross over, or mutate so as to generate a meaningless individual. Here, the meaningless individual means that, for example, data calculated and output by the function processing circuit F of the learning unit 4 is used for calculation in the next learning unit 4 according to the selection signal SS included in the individual. The individual is not input or the data is not output from the function processing circuit F of the learning unit 4 but the output from the function processing circuit F is input to the function processing circuit F of the next learning unit 4. In addition, individuals whose output data from the final learning unit 4n is not selected and output by the multiplexer 5 are also meaningless individuals.

  As shown in FIG. 13, the evaluation value calculation means 62 randomly selects, for example, ten pixels P from the image of the target image T that is the input teacher image. Further, as shown in FIG. 14, for each of ten pixels P selected from the target image T, a pixel at a corresponding position is extracted from the original image D, and the total of the pixels and the surrounding eight pixels is extracted. The pixel data of 9 pixels are extracted as a set of pixel data COM. In FIG. 13 and FIG. 14, the pixels are expressed larger than actual. Ten pixels P are randomly selected for each generation.

  Subsequently, the evaluation value calculation unit 62 transmits a selection signal SS included in one individual among all the individuals transmitted from the initial individual generation unit 61 to each of the learning units 4a to 4n via the processor 2. Thus, each element of each selector 42 is set up. Each learning unit 4a to 4n reads the constant of the multiplier MULT from the header and sets up the multiplier MULT of its own arithmetic unit 41.

  Then, the evaluation value calculation means 62 directly inputs the nine pixel data of the pixel data set COM one by one to the learning unit 4a without going through the shift register 3. In the learning units 4 a to 4 n, calculation is performed on the input of nine pieces of pixel data, and pixel data of one pixel image Dout is output from the multiplexer 5 and input to the evaluation value calculation means 62.

  The evaluation value calculation means 62 performs this simulation operation for all of the sets COM, and for each individual, the evaluation value Q1 based on the difference in luminance value of each pixel between the target image T and the output image Dout is obtained according to the following equation (1). It comes to calculate.

  Here, in the present embodiment, m in Equation (1) is 10, Tk is the luminance value of the kth pixel P, and Doutk is the output image Dout calculated based on the corresponding kth set COM. Represents the luminance value of a pixel.

  Note that the evaluation value Q1 can also be weighted and calculated according to the following equation (2) using the above-described weighted image W.

  Here, Wk represents the luminance value of the pixel at the position on the weighted image W corresponding to the kth pixel P of the target image T. m, Tk, and Doutk are the same as in the above equation (1).

  The evaluation value calculation means 62 simultaneously calculates the number M of function processing circuits used to obtain the output image Dout from the selection signal SS included in the individual, and the number N of learning units that perform only the delay circuit DFF. The evaluation values Q2 and Q3 are calculated according to the following formulas (3) and (4). Note that FN in the following equation (3) represents the total number of function processing circuits F in the present embodiment, and UN in the following equation (4) represents the total number of learning units 4 in the present embodiment.

  Note that the learning unit that performs only the operation of the delay circuit DFF is, for example, a learning unit as shown in FIG. 15, and performs the operation of the delay circuit DFF as in the learning unit 4b shown in FIG. The circuit F does not include a learning unit that performs an operation. In addition, a learning unit in which a plurality of delay circuits DFF performs an operation in a learning unit including a plurality of delay circuits DFF and an operation is not performed in another function processing circuit F is included in a learning unit that performs an operation only on the delay circuit DFF. It is. Further, in FIG. 8, for example, in the case where the learning unit 4a only calculates the delay circuit DFF and the output data is subjected to the calculation by the adder ADDR of the learning unit 4b, the learning unit 4a is connected to the delay circuit DFF. It is included in the learning unit that only performs computations.

  The evaluation value calculation means 62 associates the evaluation values Q2 and Q3 with each individual together with the evaluation value Q1. The evaluation value calculation means 62 calculates the evaluation values Q1, Q2, and Q3 for all 100 individuals as described above.

  The end determination means 63 is an evaluation value Q1 that is the target evaluation value among all the individuals transmitted from the evaluation value calculation means 62 and the individuals stored in the memory 67 including any elite. If there is no individual that has reached, each individual is transmitted to the parent selection means 64.

  The parent selection means 64 elite-stores the individual having the highest evaluation value Q1 among the individuals and isolates it in the memory 67, and selects a tournament from the copy of the individual and the remaining individuals based on the evaluation value Q1. Thus, selection of 100 individual processing programs to be left in the next generation and multiplication of those processing programs are performed. It is also possible to perform parent selection by other methods such as roulette selection, expected value selection, ranking selection and the like.

  The crossover means 65 makes two pairs of parent individuals selected and proliferated by the parent selection means 64 and crosses each other at a predetermined ratio at a crossover portion randomly selected for each individual pair. Is supposed to be generated. Further, in the mutation means 66, the numerical value of the selection signal SS is changed at a predetermined ratio for each individual, the constant of the multiplier MULT written in the header portion of the selection signal SS is changed, and the like. .

  In this way, the simulation operation and the generation change are repeated until an individual having an evaluation value Q1 of 1 appears.

  When one individual whose evaluation value Q1 is the target evaluation value appears, the end determination unit 63 increases the number of pixels P selected at random from the target image T to 100 with respect to the evaluation value calculation unit 62. So that instructions are issued.

  Therefore, in the evaluation value calculation means 62, from the next generation, for each of 100 pixels P selected from the target image T, a pixel at a corresponding position is extracted from the original image D, and the pixel and its surroundings are extracted. A total of 9 pixel data of 8 pixels is extracted, and 100 sets of pixel data sets COM are extracted. Then, a simulation calculation is performed on the 100 sets, and an evaluation value Q1 is newly calculated for each individual with n in the formula (1) or (2) set to 100.

  As described above, the end determination unit 63 instructs the evaluation value calculation unit 62 to increase the number of pixels to be randomly selected from the target image T when an individual that has reached the target evaluation value for which the evaluation value Q1 is set appears. Is supposed to send. Thereafter, the number of pixels is increased to, for example, 1000 pixels and 10,000 pixels, and finally increased to 256 × 256 = 65536 pixels, which is the total number of pixels of the original image D. The target evaluation value is gradually set to a small value, for example, 1 for 10 pixels, 0.95 for 100 pixels, 0.9 for 1000 pixels, and 0.85 for 10000 pixels.

  Note that the reference for increasing the number of pixels to be randomly selected from the target image T can be set for each set number of generations, for example. In addition, when the number of pixels is increased to the total number of pixels of the original image D, the evaluation value calculating unit 62 extends rightward from the leftmost pixel on the line of one pixel width extending in the horizontal direction of the original image D, and Pixel data is input to the shift register 3 pixel by pixel in order from the uppermost line to the lower line. Nine pixel data are sequentially input from the shift register 3 to the learning unit 4a.

  On the other hand, the learning means 6 expands the simulation calculation range in the evaluation value calculation means 62 to all the pixels of the original image D by the end determination means 63, and the evaluation value Q1 is equal to or higher than the target evaluation value set to 0.8 or the like When an individual appears, the criteria for elite preservation and the criteria for termination are changed.

  Specifically, in this embodiment, the parent selection means 64 has an evaluation value Q1, which is equal to or higher than any of the evaluation values Q1, Q2, Q3 of the elite individuals stored in the memory 67. The elite is not updated unless an individual having Q2 and Q3 appears. That is, even if an individual whose evaluation value Q1 is equal to or higher than the elite appears, if the evaluation value Q2 or the evaluation value Q3 of the individual is smaller than the evaluation value Q2 or the evaluation value Q3 of the elite, the individual is not regarded as elite.

  Further, the end determination means 63 changes the reference for end determination. In the present embodiment, the end determination means 63, when an individual having an evaluation value Q1 of 0.8 or more appears in the simulation calculation for all the pixels of the original image D, the final target evaluation values Qa and Qb set in advance. , The appearance of an individual having an evaluation value Q1, Q2, Q3 equal to or higher than Qc is set as a reference for final termination determination. In addition to this, for example, it is also possible to configure so that the end when the number of generations set in advance is reached is set as a final end determination criterion.

  When the end determination means 63 determines that the final end determination criterion is satisfied, the above routine is ended. Then, the end determination unit 63 extracts the selection signal SS included in the individual satisfying the final end determination criterion, and the signal portion corresponding to the calculation process by the learning unit that performs only the delay process calculation on the selection signal SS. If there is, the signal portion is deleted from the selection signal SS.

  The end determination means 63 outputs the selection signal SS reconstructed in this way as an optimized selection signal SSsuit to be assigned to each learning unit 4a to 4n.

  Next, the operation of the image processing apparatus 1 according to this embodiment will be described.

  Since the operations of the processor 2, the shift register 3, the learning unit 4, and the multiplexer 5 of the image processing apparatus 1 of the present embodiment are as described in the above configuration, the description thereof is omitted.

  First, the learning means 6 outputs the target image T, which is an individual having a low evaluation value Q1 calculated according to the expression (1) or (2), that is, a teacher image, from the individuals generated by the initial individual generation means 61. The evolution process is advanced for a limited number of pixels on the target image T in order to deceive an individual having a large difference in luminance value of each pixel from the image Dout.

  By evolving an individual limited to a small number of pixels in this way, only an output image Dout having a large difference in luminance value from the target image T can be output even for a limited number of pixels on the target image T. An individual having a correct selection signal SS is quickly deceived.

  Therefore, when the end determination unit 63 expands the simulation calculation range in the evaluation value calculation unit 62 to all the pixels of the original image D, the selection signal SS that can output the output image Dout that is already close to the target image T. As a result, a relatively large number of individuals including, and the speed of convergence to the optimal solution is improved.

  On the other hand, the learning means 6 switches the end determination condition at this point, and leaves the evaluation based only on the evaluation value Q1 when an individual whose evaluation value Q1 is equal to or higher than the target evaluation value set to, for example, 0.8 appears. The optimum solution is searched in consideration of the evaluation value Q2 and the evaluation value Q3.

  The evaluation value Q2 takes a larger value as the number M of function processing circuits used for obtaining the output image Dout is smaller, as shown in the equation (3). Therefore, a high evaluation value Q2 means that the image processing circuit can be configured with fewer resources, that is, a function processing circuit, according to the selection signal SS included in the individual.

  Further, as shown in the equation (4), the evaluation value Q3 takes a larger value as the number N of learning units that perform only the delay circuit DFF increases. Therefore, when the evaluation value Q3 is a high value, the selection signal SS included in the individual means that an image processing circuit including a learning unit that performs an operation only for more delay circuits DFF is configured. Means.

  Arithmetic processing by the learning unit that performs computation only on the delay circuit DFF is an arithmetic processing that may be omitted in the implementation stage. Further, as the number of learning units increases, the number of learning units that perform only the delay circuit DFF increases, and the number of arithmetic processing that can be omitted increases.

  Therefore, as in this embodiment, a selection signal newly optimized by deleting a signal portion corresponding to a learning unit that only performs an operation on the delay circuit DFF from the selection signal SS that satisfies the final termination determination criterion. By reconfiguring SSsuit, the processing time is reduced by the deleted learning units, and the processing speed is improved. In addition, it is possible to configure an image processing circuit with fewer resources by the number of deleted learning units.

  As described above, in this embodiment, two clocks of the selector 42 and the arithmetic unit 41 are required for processing of one learning unit, so that the processing time corresponding to the number of clocks twice the number of deleted learning units is the result. Can be shortened and the processing speed can be improved. Further, in the present embodiment, nine function processing circuits are used per learning unit, so that resources can be reduced accordingly.

  The evaluation values Q2 and Q3 have the above-described meanings. In this embodiment, the reduction in resources and the processing speed are assumed on the assumption that the difference in luminance value between the output image Dout and the target image T is reduced. In order to achieve both improvement, the criteria for final final judgment as described above were set. However, an individual with a high evaluation value Q2 is regarded as a new elite with an emphasis on reducing the number of resources, or an individual with a high evaluation value Q3 is updated as an elite in order to give priority to shortening the processing time and improving the processing speed. It is also possible to configure.

  As described above, according to the image processing apparatus 1 according to the present embodiment, first, not a combination of image filters as in the conventional image processing apparatus but a combination of a simple function processing circuit F such as an adder ADDR. Since an image processing procedure for the constructed image processing circuit can be provided, the image processing type is not limited to an image filter or a combination thereof according to the procedure, and an image that can be processed as desired A processing circuit can be constructed.

  In this embodiment, the case where learning is performed by giving only one type of target image T as a teacher image has been described. However, by learning based on a plurality of teacher images in order to learn desired image processing. Therefore, it is possible to provide an image processing procedure for constructing an image processing circuit capable of performing desired image processing with higher accuracy.

  Furthermore, according to the optimized selection signal SSsuit output by the image processing apparatus 1 according to the present embodiment, image processing equivalent to software processing by the processing program is performed by hardware processing, and arithmetic processing is performed in units of clocks. Therefore, the image processing speed can be dramatically improved, and the image data can be processed in real time.

  Further, in the optimization of the selection signal SS, it is possible to reduce the number of resources of the optimized selection signal SSsuit to be output by giving a better evaluation to the selection signal having a small number of function processing circuits F to be used. Can be output as a simple selection signal.

  In addition, since the learning unit that performs only the delay circuit DFF is an arithmetic process that can be omitted in the implementation stage, in the optimization of the selection signal SS, a better evaluation is performed on the selection signal that includes many learning units that perform only the delay circuit DFF. As a result, the selection signal SS satisfying the final termination criterion includes a large number of arithmetic processes by a learning unit that performs only the delay circuit DFF.

  For this reason, by deleting the signal portion corresponding to the learning unit that performs only the delay circuit DFF from such a selection signal SS and reconfiguring the newly optimized selection signal SSsuit, Therefore, the processing time can be further shortened, and the processing speed can be improved. In addition, it is possible to configure an image processing circuit with fewer resources by the number of deleted learning units.

  In the learning using the genetic algorithm in the learning means 6, the learning unit is not used in the case where the above-mentioned restriction is not made so that the meaningless individual is not generated at the time of generation, crossover, and mutation. Even if pixel data is input to 4a to 4n, the data is not output from the multiplexer 5, and as a result, data having a luminance value of 0 may be output.

  In such a case, if, for example, 10 pixels are arbitrarily selected from the target image T, which is a teacher image, and all of them are pixels having a luminance value of 0, the evaluation value Q1 of the meaningless individual is the highest of 1 Since the evaluation is obtained, it is not possible to achieve the purpose of selecting a small number of pixels from the target image T and advancing the evolution process to survive a large number of individuals with meaningful and high evaluation values and improving the speed of convergence to the optimal solution. .

  Therefore, when selecting a small number of pixels from the target image T and proceeding with the evolution process without providing the above-mentioned restrictions, a small number of pixels are arbitrarily selected from pixels whose luminance value of the target image T is not 0. Thus, it is preferable that the evaluation value Q1 is calculated.

  In addition, in the end determination unit 63 of the learning unit 6, in addition to the deletion of the signal portion corresponding to the calculation process by the learning unit that performs only the delay process calculation from the selection signal SS, the extra function processing circuit F is further deleted. With this configuration, it is possible to configure an image processing circuit with fewer resources.

  Specifically, for example, four adders ADDR are provided for each learning unit as in the present embodiment, but the optimized selection signal SSsuit is analyzed to obtain a maximum of 3 for each learning unit. When only the adder ADDR is used, the selection signal SSsuit is modified to use three adders ADDR to obtain a new optimized selection signal SSsuit, thereby using the adder ADDR to be used. Can be reduced.

  Further, in the present embodiment, the target image T is shown as an image in which the luminance value of the pixel in the extraction area EX is 255 and the luminance value of the pixel in the non-extraction area NE is 0. There is no need to create a binary image, and each pixel can be created as an image having a luminance value of 0 to 255.

  In the present embodiment, the case where the original image D, that is, the image input to the arithmetic processing of the learning units 4a to 4n is one type of image has been described, but in addition to this, for example, two types of images are input. It is also possible to configure such that addition or subtraction is performed on the two types of images.

  For example, when configured to input two types of images, two shift registers 3 of this embodiment are provided, and 18 pixel data output from the two shift registers are input to the learning unit 4a. Configure. Further, a total of 18 function processing circuits F of the arithmetic unit 41 of the learning unit 4 are provided, and one data is finally selected from the 18 data output from the learning unit 4n by the multiplexer 5 and output. Configure as follows.

[Second Embodiment]
Next, an image processing apparatus obtained by improving the image processing apparatus 1 according to the first embodiment will be described. Members having the same functions as those of the first embodiment will be described with the same reference numerals.

  As shown in FIG. 16, the image processing apparatus 10 according to the second embodiment is basically the same as the configuration of the image processing apparatus 1 shown in the first embodiment, but learning by the learning unit 6 is performed. The optimized selection signal SSsuit obtained by the above is output to the processor 2 and is sent from the processor 2 to the learning units 4a to 4n to set up the learning units 4a to 4n. ing.

  In addition, image data Din is input to the shift register 3 from the outside, and an output from the multiplexer 5 is output to the outside as output data Dout. Furthermore, the outputs of the learning units 4 a to 4 n are wired so as to be input to the multiplexer 5.

  Further, the processor 2 checks the optimized selection signal SSsuit transmitted from the learning means 6, and the signal portion corresponding to the arithmetic processing by the learning unit that performs only the arithmetic processing of the delay processing is included in the selection signal SSsuit. If there is, the signal portion is deleted from the selection signal SS, and the optimized selection signal SSsuit is reconstructed.

  The learning by the learning means 6 is performed before the operation of the image processing apparatus 10 is started, and the operation of the learning means 6 is stopped during the actual operation of the image processing apparatus 10.

  By reconfiguring the optimized selection signal SSsuit in this way, the arithmetic processing by the learning unit that performs only the arithmetic processing of the delay processing is deleted, and the arithmetic processing other than the delay processing is performed downstream of the processing of the learning unit When there is a learning unit, the computation processing of the other learning unit can be transferred to the upstream learning unit.

  Then, the processing result output from the most downstream learning unit that performs operations other than the delay processing is transmitted to the multiplexer 5, and the multiplexer 5 selects the output from the learning unit to output the output data Dout.

  As described above, according to the image processing apparatus 10 according to the present embodiment, the same effect as the effect of the image processing apparatus 1 described in the first embodiment is effectively exhibited, and the image data Din is imaged in real time. It becomes possible to process.

  In particular, the reconstructed optimized selection signal SSsuit deletes the arithmetic processing of the learning unit that only performs the arithmetic processing of the delay processing, and the arithmetic processing other than the delay processing performed on the downstream side of the processing is performed on the upstream side. This can be done with the learning unit. Further, it is possible to delete the arithmetic processing of the learning unit that performs only the arithmetic processing of the delay processing and to directly input the output to the multiplexer 5.

  Therefore, the processing time can be further shortened by the amount that the arithmetic processing of the learning unit that performs only the arithmetic processing of the delay processing is deleted, and the processing speed can be improved. In addition, it is possible to configure an image processing circuit with fewer resources by the number of deleted learning units.

  Further, in the image processing apparatus 10 according to the present embodiment, the configuration of the learning unit 6 remains as it is, so that an initial individual is newly generated by the learning unit 6 and learning is performed based on another teacher image. By obtaining the selected selection signal SSsuit, it is possible to perform a different type of image processing from the already learned image processing, and to acquire a new image processing function.

  When the image processing apparatus 10 according to the present embodiment is an apparatus dedicated to the first learned image processing, only the output from the final learning unit that performs arithmetic processing other than the delay processing is supplied to the multiplexer 5. Can be wired to input.

[Third Embodiment]
Next, as an improved version of the image processing apparatus 1 according to the first embodiment, an image processing apparatus that performs image processing using the optimized selection signal SSsuit will be described.

  In the image processing apparatus according to the third embodiment, each function processing circuit F may not be configured as the learning unit shown in FIG. However, even in this case, if there is a signal portion corresponding to the arithmetic processing by the learning unit that performs only the delay processing in the optimized selection signal SSsuit, the signal portion is deleted from the selection signal SSsuit and optimized. After reconfiguring the selected selection signal SSsuit, the adder ADDR, subtractor SUB, multiplier MULT, absolute value ABS, delay circuit DFF, implication element IF_THEN, etc. are connected based on the reconfigured selection signal SSsuit. An image processing circuit using a function processing circuit is constructed.

  By constructing an image processing circuit based on the optimized selection signal SSsuit reconstructed in this way, the image processing apparatus according to the present embodiment has the image processing apparatus 1 described in the first embodiment. The effect similar to the effect is effectively exhibited, and the image data can be processed in real time.

  In addition, the image processing circuit can be constructed by deleting the signal portion corresponding to the arithmetic processing by the learning unit that performs only the arithmetic processing of the delay processing in the optimized selection signal SSsuit, so that the processing time can be further reduced accordingly. Thus, the processing speed can be improved. In addition, an image processing circuit can be configured with fewer resources corresponding to the deleted learning unit.

  Furthermore, since it is possible to construct an image processing circuit based on the optimized selection signal SSsuit without a function processing circuit that is not involved in data processing, it is possible to configure the image processing circuit with fewer resources. Become.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment. It is a figure explaining the shift of the pixel data in a shift register. It is a figure which shows the address in the shift register of the pixel data of 3x3 pixel used as the object of arithmetic processing. It is a figure which shows the preservation | save position in the shift register of the pixel data of nxn pixel. It is a block diagram which shows the structure of a learning unit. It is a block diagram which shows the structural example of the learning unit in this embodiment. It is a figure which shows the structure of a selection signal. It is a figure explaining the flow of the data processing in the learning unit to which the selection signal of FIG. 7 was allocated. It is a figure which shows the original image used by this embodiment. It is a figure which shows the target image used by this embodiment. It is a figure which shows a weight image. It is a block diagram which shows the structure of a learning means. It is a figure showing the pixel selected from the target image. It is a figure showing the group of the pixel extracted by the pixel position in the original image corresponding to the pixel of FIG. It is a figure which shows the example of the learning unit which calculates only a delay circuit. It is a block diagram which shows the structure of the image processing apparatus which concerns on 2nd Embodiment. It is a figure showing the processing program which combined the image filter in the tree structure form.

Explanation of symbols

1, 10 Image processing device 4 Learning unit 41 Arithmetic unit 42 Selector 6 Learning means F Function processing circuit
DFF delay circuit SS selection signal SSsuit optimized selection signal CLK clock signal T target image Dout output images Q1, Q2, and Q3 evaluation value M number of function processing circuits N number of learning units that perform only the delay circuit P teacher image Pixel selected from

Claims (4)

A learning unit comprising an arithmetic unit in which function processing circuits are arranged in parallel and a selector for selecting data to be input to each function processing circuit;
Learning means for optimizing the order of operations by the learning unit;
A plurality of the learning units are connected in series,
Each selector of the plurality of learning units selects data to be input to each function processing circuit according to a selection signal indicating an image processing procedure,
The learning means optimizes the selection signal to be assigned to the selector of each learning unit by learning, uses the sequence of the selection signal to be assigned to the selector of each learning unit as the learning as a gene, and a target to be obtained by image processing Learning based on the method of genetic algorithm with images as teacher images,
The function processing circuit includes a delay circuit having a function of delaying input data by one clock,
The image processing apparatus , wherein the learning unit calculates an evaluation value based on the number of learning units that perform only the delay circuit in the learning . The learning means reconfigures a newly optimized selection signal by deleting a signal portion corresponding to the calculation processing by the learning unit that performs only the calculation of the delay processing from the calculated optimized selection signal. The image processing apparatus according to claim 1, wherein the image processing apparatus outputs the image. A learning unit comprising an arithmetic unit in which function processing circuits are arranged in parallel and a selector for selecting data to be input to each function processing circuit;
Learning means for optimizing the order of operations by the learning unit;
A plurality of the learning units are connected in series,
Each selector of the plurality of learning units selects data to be input to each function processing circuit according to a selection signal indicating an image processing procedure,
The learning means is an image processing apparatus that optimizes the selection signal to be assigned to a selector of each learning unit by learning,
If there is a learning unit that performs only delay processing when image processing is performed according to the selection signal optimized by the learning means of the image processing apparatus, the arithmetic processing by the learning unit is deleted, and the learning unit If there is another learning unit that performs operations other than delay processing on the downstream side of the process, the operation processing of the other learning unit is transferred to the upstream learning unit, and the most downstream learning that performs operations other than delay processing An image processing apparatus that outputs a processing result output from a unit. A learning unit comprising an arithmetic unit in which function processing circuits are arranged in parallel and a selector for selecting data to be input to each function processing circuit;
Learning means for optimizing the order of operations by the learning unit;
A plurality of the learning units are connected in series,
Each selector of the plurality of learning units selects data to be input to each function processing circuit according to a selection signal indicating an image processing procedure,
The learning means optimizes the selection signal assigned to the selector of each learning unit by learning ,
An image processing apparatus that performs image processing based on the optimized selection signal,
When there is an arithmetic process corresponding to a learning unit that performs only a delay process when the image processing apparatus performs an image process according to the selection signal, the signal portion corresponding to the arithmetic process is deleted from the selection signal. An image processing apparatus comprising: reconfiguring a selection signal; and constructing an image processing circuit based on the function processing circuit based on the reconfigured selection signal.

Images