JP2009054001A - Image processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To equalize processing loads of image processing even when the processing loads of an image processing part are concentrated on a certain image processing module. <P>SOLUTION: A processing construction part connects a buffer module at least before or after each of one or more image processing modules, constructs an image processing part (refer to (A)) wherein respective modules are connected in a form of a pipeline or a directed acyclic graph, and then divides the image processing part into a plurality of groups so that the processing loads of respective groups are equalized as much as possible (refer to (B)). The processing construction part determines whether a group having the maximum processing load can be further divided or not, and when the group can be divided, divides the group having the maximum processing load into a plurality of partial image processing parts each of which includes an image segmentation module 39 for segmenting data of areas different from each other from the image data and an image processing module, forms an image composition module 41 after of the partial image processing parts (refer to (C)), and threads for each program of partial image processing parts are generated. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a program, and in particular, individual modules are connected in a pipeline form or a directed acyclic graph form so that a buffer module is connected to at least one of the front and rear stages of the image processing module. And an image processing program for causing a computer to function as the image processing apparatus.

  In an image processing device that performs image processing on input image data, a DTP (desktop publishing) system that can handle images, a print system that records an image represented by input image data on a recording material, etc. Various types of image processing such as enlargement / reduction, rotation, affine transformation, color conversion, filter processing, and image synthesis are performed on the image data. In these apparatuses and systems, for example, when various image data having different color spaces and the number of bits per pixel are input, or when image processing contents, procedures, parameters, and the like are variously changed, an image to be executed A configuration that can change processing flexibly is required. In order to satisfy such a requirement, Patent Document 1 discloses that each module is connected in a pipeline form or a DAG (Directed Acyclic Graph: Existence) so that a buffer module is connected to at least one of the front and rear stages of the image processing module. There has been proposed a technique for constructing an image processing unit connected in the form of a non-circular graph) and performing image processing.

Further, in relation to the above, Patent Document 2 discloses that the unit processing of the next processing set is based on the execution result of each unit processing (for example, halftone processing for each color) included in the processing set executed immediately before. The estimated load is estimated, and based on the estimated load, a predetermined type of unit process is classified into a first process group and a second process group having a smaller estimated load than the first process group. After allocating the unit processes included in the processing group to the processing unit and completing the allocation of the unit processes included in the first processing group, the unit processing included in the second processing group is allocated to the processing unit. When allocating a unit process of a process group, the minimum load processing part with the smallest total estimated load of unit processes already assigned to the unit process that has not yet been assigned to the processing part By assigning one technique for reducing the variation of load among the processing units when executing image processing by parallel processing it has been proposed.
JP 2006-338498 A JP 2005-250565 A

  When image processing is performed by constructing an image processing unit using the technique described in Patent Document 1, depending on the content of image processing to be realized, the size of input image data, and the like, individual image processing constituting the image processing unit The processing load (also referred to as processing cost) for each module may vary greatly. This variation in the processing load for each image processing module is particularly significant when the image processing unit is operated in a parallel processing system in which image processing in individual image processing modules is performed in parallel. There is a problem of inviting an increase in

  For the above problem, it is conceivable to apply the technique described in Patent Document 2 for the purpose of reducing the variation in the load between the processing units. This technique, however, executes image processing for a certain color. In order to estimate the processing load in the same image processing for different colors from the results, it is necessary to shift the execution timing of the image processing for the image data by each color unit (sequentially execute the image processing for each color). There is a problem that it takes time to complete the processing results for. When image processing is performed by connecting modules in a pipeline form or a directed acyclic graph form as in the technique described in Patent Document 1, the image data for all colors may be a part of the image, but the previous stage In many cases, image processing cannot be performed on the subsequent stage unless it is output from the side (for example, when color conversion is performed on the subsequent stage), and the processing load of individual image processing modules in the image processing unit is biased Even when the technique described in Patent Document 2 is applied, a situation in which improvement in processing efficiency and reduction in processing time cannot be expected can frequently occur.

  Note that the bias in processing load in the image processing unit adversely affects processing efficiency and processing time is limited to the case where the image processing unit is operated in a parallel processing system in which image processing in individual image processing modules is performed in parallel. Image processing with a large processing load even when the image processing unit is operated by a sequential processing method in which image processing is always performed by a single image processing module and the image processing module for performing image processing is sequentially switched. By increasing the image processing time in the module, similarly, uneven processing load adversely affects the processing time.

  The present invention has been made in consideration of the above facts, and an image processing apparatus and an image processing capable of leveling the processing load even when the processing load in the image processing unit is biased to some image processing modules. The purpose is to get a program.

  In order to achieve the above object, an image processing apparatus according to the first aspect of the present invention performs image processing different from each other on image data acquired from a previous stage of its own module, and the image data that has undergone the image processing or the image processing The image processing module outputs the results of the above to the subsequent stage of the own module, the image data output from the previous module is written to the buffer, and the image data stored in the buffer is read by the subsequent module The image data output from the buffer module and the previous module is divided into a plurality of data corresponding to different areas on the image represented by the image data, and the divided individual data are different from each other in the subsequent modules. Each of the distribution supply unit supplying the module and the plurality of modules in the previous stage A storage unit that stores each program of an image composition unit that combines the acquired data as image data of a single image, and a program stored in the storage unit, selected from the plurality of types of image processing modules The buffer module is connected to at least one of the preceding stage and the subsequent stage of each of the one or more image processing modules, and the modules are connected in a pipeline form or a directed acyclic graph form, and at least a part of them is provided in multiple. An image processing unit is constructed in which individual processing units provided in multiplex are connected to the preceding stage of the multiplexed part via the distribution supply unit and connected to the subsequent stage of the multiplexed part via the image composition unit, respectively. And a control unit that causes the image processing unit constructed by the construction unit to perform image processing. To have.

  In the first aspect of the present invention, a plurality of image processings are performed on the image data acquired from the previous stage of the own module, and the image data subjected to the image processing or the processing result of the image processing is output to the subsequent stage of the own module. Various image processing modules, a buffer module that causes image data output from the preceding module to be written to the buffer and that causes the image data stored in the buffer to be read by the subsequent module, and image data that is output from the preceding module Is divided into a plurality of data corresponding to different regions on the image represented by the image data, and the divided supply unit supplies each divided data to different modules among the plurality of modules in the subsequent stage, and Image data of a single image obtained from each of multiple modules Image synthesizing unit programs are respectively stored in the storage means for then synthesized.

  The construction means according to the first aspect of the invention provides at least a front stage and a rear stage of each of one or more image processing modules selected from a plurality of types of image processing modules by a program stored in the storage means. A buffer module is connected to one side, and each module is connected in a pipeline form or a directed acyclic graph form, and at least a part of them is provided in multiple, and each of the multiple processing units provided via a distribution supply unit Thus, an image processing unit connected to the preceding stage of the multiplexed portion and connected to the subsequent stage of the multiplexed portion via the image synthesis unit is constructed. The control unit according to the first aspect of the invention causes the image processing unit constructed by the construction unit to perform image processing.

  As described above, according to the first aspect of the present invention, the image processing unit constructed by the construction unit includes at least a part of the image processing unit in a multiplex, and the individual processing units provided in the multiplex include the distribution supply unit. Are connected to the preceding stage of the multiplexed part through the image synthesizing unit and are connected to the subsequent stage of the multiplexed part via the image composition unit. Different distribution data of a plurality of data corresponding to different areas on the image obtained by dividing the image data by the distribution supply unit provided in each preceding stage is supplied to the supplied data. On the other hand, after the same image processing is performed in each processing unit, data output from each processing unit through image processing in each processing unit is converted into image data of a single image by the image composition unit. It is made.

  Thus, according to the first aspect of the present invention, when the processing load of each processing unit corresponding to the multiplexed portion is N, the number of the processing units corresponding to the multiplexed portion is N (processing is not performed). 1 / N of the case where multiple parts are not provided, that is, N = 1. In the technique described in Patent Document 2, since image processing is performed for each color, the value corresponding to the above N is constant and cannot be changed. However, in the invention described in Claim 1, different areas on the image are used. Since the image data is divided into data corresponding to, the number N of processing units can be arbitrarily set. Therefore, according to the first aspect of the present invention, even when the processing load in the image processing unit is biased toward some image processing modules, for example, among the image processing units, the processing load is higher than that of other image processing modules. For example, the processing load can be leveled by setting a portion including a large specific image processing module as a multiplexing target.

  The invention described in claim 1 divides image data into a plurality of data corresponding to different regions on the image, and supplies the divided plurality of data to different processing units. Since it is not necessary to shift the execution timing of the image processing for each divided piece of data as in the technique described in (5), for example, as described in (6), at least multiple of the image processing units constructed by the construction unit By configuring the control means so that the programs of the individual processing units corresponding to the multiplex portion are executed in parallel by the computer, it is also possible to cause the individual processing units corresponding to the multiplexed portion to perform image processing in parallel Thus, it is possible to improve processing efficiency and shorten processing time by executing image processing in individual processing units in parallel.

  According to the first aspect of the invention, multiplexing may be difficult depending on the contents of the image processing in the image processing module (for example, geometric transformation processing such as affine transformation and image rotation is performed by synthesis processing in the image synthesis unit. Considering that it is very complicated and is not suitable for multiplexing), the construction means should be a multiplexing target among the image processing modules constituting the image processing unit as described in claim 2, for example. It is preferable that a part including an image processing module registered as a multiplexing target in a predetermined table is selected as a part to be multiplexed. As a result, among a plurality of types of image processing modules whose programs are stored in the storage means, an image processing module suitable for multiplexing (for example, an image processing module having a relatively large processing load and performing image processing suitable for multiplexing) ) Is added to the information and registered in a predetermined table so that the construction means selects the part to be multiplexed from the image processing unit. Based on the presence / absence of addition and the presence / absence of registration in a predetermined table, it can be realized by highly accurate and simple processing.

  Further, in consideration of the fact that the processing load in the image processing module greatly depends on the size of the image to be subjected to image processing (data amount of image data), in the invention according to claim 1 or 2, For example, as described in claim 3, at least a part of the image processing unit may be multiplexed only when the size of the image to be processed by the image processing unit is equal to or larger than a threshold value. . When the size of the image to be subjected to image processing is less than the threshold, the processing load of the image processing unit is relatively small, so that the processing efficiency is reduced by providing the distribution supply unit and the image composition unit. In the present invention, since multiplexing is not performed in the above case, whether to perform multiplexing is switched according to the processing load of the image processing unit, and the processing load of the image processing unit is relatively small It is possible to prevent the processing efficiency from being lowered.

  In the invention according to claim 3, the process of changing the number of processing units corresponding to the multiplexed portion (the number of multiplexing) stepwise with respect to the change in the size of the image processing in which the image processing is performed. You may make it perform. In this case, the number of processing units corresponding to the multiplexed portion (the number of multiplexing) can be optimized according to the processing load of the image processing unit.

  Further, in the invention described in claim 1, the construction means, as described in claim 4, for example, includes a provisional image processing unit in which each module is connected in a pipeline form or a directed acyclic graph form in a plurality of parts. A processing load may be calculated for each individual part when divided, and a part to be multiplexed may be selected from a plurality of parts based on the calculated processing load for each part. As described above, by calculating the processing load for each part and selecting the part to be multiplexed, the part to be multiplexed can be selected with high accuracy.

  In the invention according to claim 4, the construction means, as described in claim 5, for example, multiplexes a plurality of parts based on the processing load calculated for each part of the provisional image processing part. The presence / absence of multiplexing and the number of multiplexing for each part so that the processing load of each individual processing unit corresponding to each part and each non-multiplexed part of the plurality of parts approaches a uniform state It is preferable to provide a configuration in which the same number of processing units as the set number of multiplexing units are provided in the part of the plurality of parts set to have multiplexing. Thereby, each processing load corresponding to the multiplexed portion and each processing load of each non-multiplexed portion can be brought close to a uniform state with high accuracy.

  Further, in the invention according to any one of claims 1 to 5, the control means corresponds to at least a multiplexed part among the image processing parts constructed by the construction means as described in claim 6, for example. The programs of the individual processing units may be configured to be executed in parallel by a computer. Further, in the invention according to claim 5 or claim 6, the control means, as described in claim 7, for example, the program of each processing unit corresponding to the multiplexed part of the plurality of parts and the plurality of parts The programs of the individual non-multiplexed portions may be configured to be executed in parallel by a computer.

  By the way, the distribution supply unit divides the image data into a plurality of data, and supplies the divided individual data to different modules among the plurality of modules at the subsequent stage. For example, the image data to be supplied is stored in a buffer. If the distribution supply unit is configured to supply data to the subsequent module by reading the data from the buffer by the subsequent module, there is a possibility that access to the buffer by multiple modules in the subsequent stage may compete. There is. The image composition unit also composes data acquired from a plurality of modules in the previous stage as image data of a single image, so that the image is obtained by writing data to the buffer by the module in the previous stage. When the combining unit is configured, there is a possibility that access to the buffer by a plurality of modules in the previous stage conflicts.

  In consideration of the above, in the invention according to claim 6 or claim 7, as described in claim 8, for example, as a buffer module program, to the first buffer module and the buffer that perform exclusive control for access to the buffer A buffer module and an image composition unit that function as at least a part of a distribution supply unit in constructing the image processing unit, respectively, in the storage unit. The first buffer module may be selected as the buffer module that functions as a part of the first buffer module, and the second buffer module may be selected as the other buffer module. As a result, it is possible to prevent inconveniences such as the processing being locked due to the above-described contention of access to the buffer.

  The exclusive control in the first buffer module is added as an overhead (excess processing load) to the computer executing the program of the buffer module. In the invention according to claim 8, the buffer module in which the first buffer module is selected. Since the second buffer module that does not perform exclusive control on access to the buffer is selected as the buffer module other than the number of the buffer modules, the number of buffer modules from which the first buffer module is selected among the buffer modules constituting the image processing unit And the overhead due to the exclusive control in the first buffer module is accumulated and can be prevented from being added to the computer as a large processing load. In addition, as in the above-described claim 7, in the case where the programs of the individual processing units corresponding to the multiplexed parts and the programs of the individual non-multiplexed parts are each executed in parallel by the computer, Also for the buffer module located at the position corresponding to the boundary, there is a possibility that contention for access to the buffer may occur, so it is desirable to use the first buffer module.

  The image processing program according to the invention described in claim 9 performs different image processing on the image data acquired from the previous stage of the own module, and the image data subjected to the image processing or the processing result of the image processing is stored in the own module. From a plurality of types of image processing modules to be output to the subsequent stage, a buffer module for causing the image data output from the previous stage module to be written to the buffer and reading out the image data stored in the buffer by the subsequent stage module, from the previous stage module A distribution supply unit that divides the output image data into a plurality of data corresponding to different regions on the image represented by the image data, and supplies the divided data to different modules among the plurality of modules at the subsequent stage , And the data obtained from each module in the previous stage A computer connected to storage means for storing each program of an image composition unit for compositing as image data of one image, based on the programs of the various modules stored in the storage means, The buffer module is connected to at least one of a front stage and a rear stage of each of one or more image processing modules selected from among the modules, and each module is connected in a pipeline form or a directed acyclic graph form, and at least Construction means for constructing an image processing section in which a part of the modules are multiplexed and the distribution module is inserted in the preceding stage of the multiplexed part, and the synthesis module is inserted in the subsequent stage, and the image processing section constructed by the construction means To function as control means for performing image processing.

  The image processing program according to the ninth aspect of the invention is a program for causing a computer connected to the storage means to function as the construction means and the control means. By executing the image processing program, the computer functions as the image processing apparatus according to claim 1, and the processing load in the image processing unit is a part of the image processing as in the invention according to claim 1. Even when the modules are biased, the processing load can be leveled.

  As described above, in the present invention, a buffer module is connected to at least one of the preceding stage and the succeeding stage of each of one or more image processing modules selected from a plurality of types of image processing modules, and each module is configured in a pipeline form or Connected in the form of a directed acyclic graph, and at least a part of them are provided in multiple, and each of the multiple processing units is connected to a plurality of image data from the previous module corresponding to different regions on the image. Each of the divided data is connected to the preceding stage of the multiplexed portion via a distribution supply unit that supplies the divided individual data to different modules in the subsequent stage, and the data acquired from the plurality of modules in the previous stage are each converted into a single image An image processing unit that is connected to the subsequent stage of the multiplexed part via the image composition unit that composes as image data In the case where the processing load in the image processing section is biased in a part of the image processing modules, it is possible to level the processing load, has an excellent effect that.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 shows a computer 10 that can function as an image processing apparatus according to the present invention. The computer 10 is incorporated in any image handling apparatus that needs to perform image processing internally, such as a copying machine, a printer, a facsimile machine, a multifunction machine having these functions, a scanner, a photographic printer, or the like. Alternatively, it may be an independent computer such as a personal computer (PC), or may be a computer incorporated in a portable device such as a PDA (Personal Digital Assistant) or a cellular phone.

  The computer 10 includes a CPU 12, a memory 14 including a DRAM or SRAM, a display unit 16, an operation unit 18, a storage unit 20, an image data supply unit 22, and an image output unit 24, which are connected to each other via a bus 26. Has been. When the computer 10 is incorporated in an image handling device as described above, a display panel, a numeric keypad, or the like composed of an LCD or the like provided in the image handling device can be applied as the display unit 16 and the operation unit 18. When the computer 10 is an independent computer, a display, a keyboard, a mouse, or the like connected to the computer can be applied as the display unit 16 or the operation unit 18. The storage unit 20 is preferably an HDD (Hard Disk Drive), but other nonvolatile storage means such as a flash memory can be used instead.

  The image data supply unit 22 only needs to be able to supply image data to be processed. For example, an image reading unit that reads an image recorded on a recording material such as paper or photographic film, and outputs the image data. A receiving unit that receives image data from the outside via a line, an image storage unit (memory 14 or storage unit 20) that stores image data, and the like can be applied. The image output unit 24 only needs to output image data that has undergone image processing or an image represented by the image data. For example, the image recording unit records an image represented by the image data on a recording material such as paper or a photosensitive material. A display unit that displays an image represented by image data on a display, a writing device that writes image data to a recording medium, and a transmission unit that transmits image data via a communication line can be applied. The image output unit 24 may be an image storage unit (memory 14 or storage unit 20) that simply stores image data that has undergone image processing.

  As shown in FIG. 1, the storage unit 20 includes various programs executed by the CPU 12, such as management of resources such as the memory 14, management of execution of programs by the CPU 12, and communication between the computer 10 and the outside. A program of the system 30, an image processing program group 34 for causing the computer 10 to function as an image processing apparatus according to the present invention, and a desired image for the image processing apparatus realized by the CPU 12 executing the image processing program group. Various programs of applications 32 (indicated as application program group 32 in FIG. 1) to be processed are stored.

  The image processing program group 34 reduces the development load when developing the above-described various image handling devices and portable devices, and reduces the development load when developing an image processing program usable on a PC or the like. The purpose is a program developed so that it can be commonly used in various devices (platforms) such as various image handling devices, portable devices, and PCs, and corresponds to the image processing program according to the present invention. The image processing apparatus realized by the image processing program group 34 constructs an image processing unit that performs image processing instructed by the application 32 in accordance with a construction instruction from the application 32, and executes the image processing unit in accordance with an execution instruction from the application 32. The image processing program group 34 instructs the construction of an image processing unit (an image processing unit having a desired configuration) for performing desired image processing, or the constructed image processing unit. The application 32 is provided with an interface for instructing execution of image processing according to the above. For this reason, even when newly developing an arbitrary device that needs to perform image processing internally, regarding the development of a program for performing the image processing, the above-described interface is used to perform the image processing required for the device. It is only necessary to develop the application 32 to be used and executed by the image processing program group 34, and it is not necessary to newly develop a program for actually performing image processing, so that the development load can be reduced.

  Further, as described above, the image processing apparatus realized by the image processing program group 34 constructs an image processing unit that performs image processing instructed by the application 32 according to the construction instruction from the application 32, and constructs the constructed image processing unit. Therefore, for example, when the color space of the image data to be processed and the number of bits per pixel are indefinite or the contents of the image processing to be executed, the procedure / parameter, etc. are indefinite, When the application 32 instructs to reconstruct the image processing unit, the image processing executed by the image processing apparatus (image processing unit) can be flexibly changed according to the image data to be processed.

  Hereinafter, the image processing program group 34 will be described. As shown in FIG. 1, the image processing program group 34 is roughly divided into a module library 36, a process construction unit 42 program, and a process management unit 46 program. As will be described in detail later, the processing construction unit 42 according to the present embodiment, according to an instruction from the application, as illustrated in FIG. 2 as an example, one or more image processing modules 38 that perform predetermined image processing, A buffer module 40 that is arranged in at least one of the preceding stage and the succeeding stage of each image processing module 38 and has a buffer for storing image data, and a pipeline form or a DAG (Directed Acyclic Graph) form An image processing unit 50 formed by linking is constructed. The entity of each image processing module constituting the image processing unit 50 is a first program executed by the CPU 12 and causing the CPU 12 to perform predetermined image processing, or an external unit not shown in FIG. 2 is a second program for instructing execution of processing to the image processing apparatus (for example, a dedicated image processing board), and the module library 36 described above stores predetermined different image processing (for example, input processing and filter processing). , Color conversion processing, enlargement / reduction processing, skew angle detection processing, image rotation processing, image composition processing, output processing, and the like) are registered. Hereinafter, in order to simplify the description, it is assumed that each image processing module constituting the image processing unit 50 is the first program.

  As shown in FIG. 3A as an example, each image processing module 38 includes an image processing engine 38A that performs image processing on image data by a predetermined unit processing data amount, and a front stage and a rear stage of the image processing module 38. The control unit 38B performs input / output of image data with the module and controls the image processing engine 38A. The unit processing data amount in each image processing module 38 is the image processing engine 38A from an arbitrary number of bytes including one line of the image, a plurality of lines of the image, one pixel of the image, one surface of the image, and the like. Is selected and set in advance according to the type of image processing to be performed. For example, in the image processing module 38 that performs color conversion processing and filter processing, the unit processing data amount is one pixel, and the image to be enlarged / reduced is processed. In the processing module 38, the unit processing data amount is equivalent to one line of the image or a plurality of lines of the image, and in the image processing module 38 that performs image rotation processing, the unit processing data amount is equivalent to one image, and the image compression / decompression processing is performed. In the image processing module 38 to be performed, the unit processing data amount is set to N bytes depending on the execution environment.

  The module library 36 also registers image processing modules 38 having the same type of image processing executed by the image processing engine 38A and different contents of the image processing executed (in FIG. 1, this type of image processing is performed). Modules are indicated as “module 1” and “module 2”). For example, with respect to the image processing module 38 that performs enlargement / reduction processing, the image processing module 38 that performs reduction processing to reduce the input image data to 50% by thinning out every other pixel is designated for the input image data. A plurality of image processing modules 38 such as an image processing module 38 that performs enlargement / reduction processing at the enlarged / reduced rate are prepared. For example, the image processing module 38 that performs color conversion processing includes an image processing module 38 that converts the RGB color space to the CMY color space, an image processing module 38 that converts the color space to the opposite, and an L * a * b * color space. Image processing modules 38 for performing other color space conversions are prepared.

  In addition, the control unit 38B of the image processing module 38 receives an image from a module (for example, the buffer module 40) in the previous stage of its own module in order to input image data necessary for the image processing engine 38A to process each unit processing data amount. Data is acquired for each unit read data amount, and image data output from the image processing engine 38A is output for each unit write data amount to a subsequent module (for example, the buffer module 40) (data amount such as compression by the image processing engine 38A). If the image processing with the increase / decrease is not performed, the unit writing data amount = the unit processing data amount) or the result of the image processing by the image processing engine 38A is output to the outside of the module (for example, the image processing engine 38A When performing image analysis processing such as skew angle detection processing, a scan is performed instead of image data. The image analysis processing result such as the angle detection result may be output). However, the module library 36 has the same type and content of the image processing executed by the image processing engine 38A, and the unit processing described above. Image processing modules 38 having different data amounts, unit read data amounts, and unit write data amounts are also registered. For example, the unit processing data amount in the image processing module 38 that performs image rotation processing is not limited to the one image plane described above, and the same image rotation processing is performed and the unit processing data amounts are different from each other (for example, one image). A plurality of image processing modules 38 (for a line, a plurality of lines, etc.) may be included in the module library 36.

  The program of each image processing module 38 registered in the module library 36 is composed of a program corresponding to the image processing engine 38A and a program corresponding to the control unit 38B, but a program corresponding to the control unit 38B. The image processing module 38 having the same unit read data amount and unit write data amount among the individual image processing modules 38 is concerned with the type and content of image processing executed by the image processing engine 38A. However, the program corresponding to the control unit 38B is shared (the same program is used as the program corresponding to the control unit 38B). Thereby, the development load in developing the program of the image processing module 38 is reduced.

  In the image processing module 38, the unit read data amount and the unit write data amount are not fixed when the input image attribute is unknown, and the input image data attribute is acquired and acquired. There is a module in which the unit read data amount and the unit write data amount are determined by substituting the attribute into a predetermined arithmetic expression, and this type of image processing module 38 has a unit read data amount and For the image processing module 38 in which the unit write data amount is derived using the same arithmetic expression, a program corresponding to the control unit 38B may be shared. The image processing program group 34 according to the present embodiment can be mounted on various devices as described above, but the number, type, and the like of the image processing modules 38 registered in the module library 36 in the image processing program group 34. Needless to say, addition, deletion, replacement, and the like can be appropriately performed in accordance with image processing required by various devices that implement the image processing program group 34.

  Further, as shown in FIG. 3B as an example, the individual buffer modules 40 constituting the image processing unit 50 are configured to input / output image data between the buffer 40A and modules before and after the buffer module 40. The buffer control unit 40B manages the buffer 40A. The buffer 40A is composed of a memory area secured from the memory 14 provided in the computer 10 through the operating system 30 and the resource management unit 46B. The buffer control unit 40B of each buffer module 40 is also a program executed by the CPU 12, and the program of the buffer control unit 40B is also registered in the module library 36 (in FIG. 1, the program of the buffer control unit 40B is registered). "Buffer module")

  In the present embodiment, in consideration of the possibility that a plurality of image processing modules 38 may access the buffer 40A of the buffer module 40 at the same time, an exclusive control function that performs exclusive control on access to the buffer 40A. Provided buffer module 40 (corresponding to the first buffer module according to claim 8) and buffer module 40 without exclusive control function (corresponding to the second buffer module according to claim 8), respectively, are prepared, In the module library 36, the programs of the respective buffer modules 40 are registered.

  Further, in the present embodiment, the image data stored in the module in the previous stage is prepared for the case of further dividing (multiplexing) a group (hot spot) having the maximum processing load in the image processing unit 50 (details will be described later). An image cutout module that cuts out and outputs data of a preset image area from the image, and an image composition module that obtains data from a plurality of modules in the previous stage and combines the obtained data as image data of a single image Each program is also stored.

  In addition, the processing construction unit 42 that constructs the image processing unit 50 in accordance with an instruction from the application 32 includes a plurality of types of module generation units 44 as shown in FIG. The plural types of module generation units 44 support different image processing, and are activated by the application 32 to generate a module group including an image processing module 38 and a buffer module 40 for realizing the corresponding image processing. Perform the process. In FIG. 1, the module generation unit 44 corresponding to the type of image processing executed by each image processing module 38 registered in the module library 36 is shown as an example of the module generation unit 44. The image processing corresponding to the generation unit 44 may be image processing realized by a plurality of types of image processing modules 38 (for example, skew correction processing including skew angle detection processing and image rotation processing). When the required image processing is processing combining a plurality of types of image processing, the application 32 sequentially activates the module generation unit 44 corresponding to any of the plurality of types of image processing. As a result, an image processing unit 50 that performs necessary image processing is constructed by the module generation unit 44 that is sequentially activated by the application 32.

  As shown in FIG. 1, the process management unit 46 includes a workflow management unit 46 </ b> A that controls execution of image processing in the image processing unit 50, a memory 14 by each module of the image processing unit 50, and various files of the computer 10. It includes a resource management unit 46B that manages the use of resources, and an error management unit 46C that manages errors that have occurred in the image processing unit 50. When an error occurs while the image processing unit 50 is executing image processing, the error management unit 46C acquires error information such as the type and location of the error that has occurred, and the image processing program group 34 is installed. The device environment information representing the type or configuration of the device in which the computer 10 is incorporated is acquired from the storage unit 20 or the like, and an error notification method according to the device environment represented by the acquired device environment information is determined and determined. Performs processing to notify the occurrence of an error using the error notification method.

  Next, the operation of this embodiment will be described. In a device in which the image processing program group 34 is installed, when a situation where it is necessary to perform some kind of image processing, this situation is detected by a specific application 32. In addition, as a situation where it is necessary to perform image processing, for example, an image is read by an image reading unit as the image data supply unit 22, and is recorded as an image on a recording material by an image recording unit as the image output unit 24, or image output Whether the image is displayed on the display unit as the unit 24, the image data is written to the recording medium by the writing device as the image output unit 24, the image data is transmitted by the transmission unit as the image output unit 24, or the image output When the execution of a job to be stored in the image storage unit as the unit 24 is instructed by the user, or received by the reception unit as the image data supply unit 22 or stored in the image storage unit as the image data supply unit 22 For the image data being recorded, recording on the recording material, display on the display unit, writing to the recording medium, transmission, Any execution of a job for the storage of the image storage unit include when instructed by the user. In addition, the situation where image processing needs to be performed is not limited to the above. For example, in a state where the names of processes that can be executed by the application 32 in accordance with an instruction from the user are displayed on the display unit 16 as a list, For example, the execution target process may be selected by the user.

  As described above, when detecting that it is necessary to perform some kind of image processing, the application 32 first recognizes the type of the image data supply unit 22 that supplies the image data to be processed, and recognizes the recognized type. Is a buffer area (partial area of the memory 14), a parameter for causing the buffer control unit 40B to recognize the buffer area designated as the image data supply unit 22 as an already secured buffer 40A is set, and the buffer A buffer module 40 (image data supply unit 22) including a designated buffer area is generated by generating a thread (which may be a process or an object; the same shall apply hereinafter) that executes a program of the control unit 40B (generates the buffer control unit 40B). As a buffer module 40).

  Subsequently, in the same manner as described above, the application 32 recognizes the type of the image output unit 24 as the output destination of the image data subjected to image processing, and the recognized type is the buffer area (partial area of the memory 14). In this case, the buffer module 40 including the buffer area designated as the image output unit 24 is generated in the same manner as described above. The buffer module 40 generated here functions as the image output unit 24. Further, the application 32 recognizes the contents of the image processing to be executed, decomposes the image processing to be executed into a combination of image processing at a level corresponding to each module generation unit 44, and realizes the image processing to be executed. Therefore, the type of image processing necessary for the purpose and the execution order of the individual image processing are determined. In this determination, for example, the type of image processing and the execution order of the individual image processing are registered in advance as information in association with the type of job that can be instructed by the user. Can be realized by reading out information corresponding to the type of job instructed.

  Then, the application 32 activates the module generation unit 44 corresponding to the specific image processing (generates a thread for executing the program of the module generation unit 44) based on the type and execution order of the image processing determined above. Input module identification information for identifying an input module for inputting image data to the module group as information necessary for generating the module group by the module generation unit 44 for the activated module generation unit 44, and the module group Output module identification information for identifying an output module that outputs image data, input image attribute information that represents an attribute of input image data input to the module group, and a corresponding module that notifies parameters of image processing to be executed Instruct the generation of a group. In addition, when the required image processing is a combination of a plurality of types of image processing, the application 32 responds to individual image processing when notified of completion of generation of the module group from the instructed module generation unit 44. The process of starting the other module generation unit 44 and notifying the information necessary for generating the module group is repeated in ascending order of the execution order of the individual image processes.

  In the above input module, the image data supply unit 22 is the input module for the module group with the first execution order, and the last module (usually the buffer) for the module group with the second execution order or later. Module 40) is the input module. As for the above output modules, the image output unit 24 is designated as the output module in the module group whose execution order is the last, so that the image output unit 24 is designated as the output module. Specification by the application 32 is not performed for confirmation, and the module generation unit 44 generates and sets it when necessary. The input image attributes and image processing parameters are registered as information in advance in association with, for example, the type of job that can be instructed by the user, and information corresponding to the type of job instructed to be executed is read out. As a result, the application 32 may recognize it, or the user may specify it.

  On the other hand, the module generation unit 44 performs module generation processing when activated by the application 32. In the module generation process, first, input image attribute information representing an attribute of input image data input to the generation target image processing module 38 is acquired. Note that the processing for acquiring the attributes of the input image data is performed in the case where the buffer module 40 exists in the previous stage of the generation target image processing module 38 and the image processing module in the previous stage that writes image data in the buffer module 40. This can be realized by acquiring the attributes of the output image data from the image data 38.

  Then, based on the attribute of the input image data represented by the acquired information, it is determined whether the generation of the image processing module 38 to be generated is necessary. For example, the module generation unit 44 is a module generation unit that generates a group of modules that perform color conversion processing. When the CMY color space is designated as the color space of the output image data by the image processing parameter from the application 32, the acquired input image When the input image data is found to be RGB color space data based on the attribute information, it is necessary to generate an image processing module 38 that performs RGB → CMY color space conversion as an image processing module 38 that performs color space processing. However, when the input image data is data in the CMY color space, the attribute of the input image data and the attribute of the output image data match with respect to the color space, so that the image processing module 38 that performs color space conversion processing generates Judge as unnecessary.

  If it is determined that the generation target image processing module 38 needs to be generated, it is determined whether the buffer module 40 is required after the generation target image processing module 38. This determination is made when the subsequent stage of the image processing module is the output module (image output unit 24) (see, for example, the last stage image processing module 38 in the image processing unit 50 shown in FIGS. 2A to 2C). As an example, like the image processing module 38 that performs skew angle detection processing in the image processing unit 50 shown in FIG. In the case other than the above, the determination is affirmed and the buffer control unit 40B is activated (a thread for executing the program of the buffer control unit 40B is generated). Thus, the buffer module 40 connected to the subsequent stage of the image processing module 38 is generated.

  Subsequently, information on the preceding module (for example, the buffer module 40), information on the succeeding buffer module 40 (only the image processing module 38 that generated the buffer module 40 in the succeeding stage), and input image data input to the image processing module 38 Attributes and processing parameters are given and registered in the module library 36. Among a plurality of candidate modules that can be used as the image processing module 38, the attributes of the input image data acquired earlier and the image processing module 38 The image processing module 38 that matches the processing parameter to be executed is selected and generated (a thread for executing the program of the image processing engine 38A and the control unit 38B is generated).

  For example, the module generation unit 44 is a module generation unit that generates a group of modules that perform color conversion processing. The CMY color space is specified as the color space of the output image data by the processing parameter, and the input image data is data in the RGB color space. If there is, an image processing module 38 that performs RGB → CMY color space conversion is selected and generated from a plurality of types of image processing modules 38 that perform various color space processing registered in the module library 36. The The image processing module 38 is an image processing module 38 that performs enlargement / reduction processing. If the designated enlargement / reduction ratio is other than 50%, the input image data is enlarged at the designated enlargement / reduction ratio. If the image processing module 38 that performs the reduction process is selected and generated and the designated enlargement / reduction ratio is 50%, the enlargement / reduction process specialized for the enlargement / reduction ratio of 50%, that is, the input image data is one pixel. An image processing module 38 that performs a reduction process of reducing to 50% by thinning out at intervals is selected and generated.

  The selection of the image processing module 38 is not limited to the above. For example, a plurality of image processing modules 38 having different unit processing data amounts in image processing by the image processing engine 38A are registered in the module library 36, and the image processing unit The image processing module 38 having an appropriate unit processing data amount is selected according to the operating environment such as the size of the memory area that can be allocated to 50 (for example, the image processing module 38 having a smaller unit processing data amount as the size decreases). Or the application 32 or the user may select it.

  When the generation of the image processing module 38 is completed, the workflow management unit 46A is notified of the set of the ID of the subsequent buffer module 40 and the ID of the generated image processing module 38. The ID may be information that can uniquely identify each module, and may be, for example, a number given in the order of generation of each module, an address of an object of the buffer module 40 or the image processing module 38 on a memory, or the like. Further, the module generation unit 44 performs image processing realized by a plurality of types of image processing modules 38 (for example, skew correction processing realized by the image processing module 38 that performs skew angle detection processing and the image processing module 38 that performs image rotation processing). When generating a module group for performing the above, the above process is repeated to generate a module group including two or more image processing modules 38. The module generation processing described above is sequentially performed by the individual module generation units 44 that are sequentially activated by the application 32, so that the necessary image processing can be performed as shown in FIGS. An image processing unit 50 to perform is constructed. The image processing unit 50 constructed by the above processing corresponds to the “temporary image processing unit” described in claims 4 and 5.

  The processing construction unit 42 corresponds to the construction unit according to the present invention (more specifically, the construction unit described in claims 2 to 4). When the image processing unit 50 is constructed as described above, the constructed image is obtained. The image processing unit division processing shown in FIG. 4 is performed on the processing unit 50. In this image processing section dividing process, first, in step 100, the number N of groups representing how many groups the image processing section 50 is divided into is determined. As the number of groups N, for example, the number of computation resources provided in the computer 10 (for example, resources that can execute computations in parallel, such as a CPU core or a DSP (Digital Signal Processor)) can be applied. The number of computing resources can be obtained using a function provided by the OS. Further, instead of simply using the number of computing resources provided in the computer 10 as the number of groups N, after obtaining the number of computing resources, the usage rate in each computing resource is further obtained. You may make it use the number of the calculation resources below a threshold value as the number N of groups. If the number of groups is N = 1, it is desirable to skip the processing after the next step 102 and end the image processing section division processing.

  In the next step 102, based on the number N of groups determined in step 100, the previously constructed image processing unit 50 is divided into N groups. There are several methods for dividing the image processing unit 50. The first division method uses the total number of image processing modules 38 in the image processing unit 50 as the number M of image processing modules in the image processing unit 50, divides the number M of the above image processing modules by the number N of groups, and generates a remainder. Is generated, the fraction processing such as truncation is performed on the quotient to obtain the number L of image processing modules 38 per group, the image processing unit 50 is scanned in order from the top, and L image processing modules 38 are obtained. Each time it appears, up to the L-th image processing module 38 is grouped as one group, and the count value of the number of image processing modules 38 is reset to 0, so that the image processing unit 50 is divided into N groups. It is a method of dividing. Of the image processing modules 38 constituting the image processing unit 50, the image processing module 38 (image processing module 38 that performs image processing such as image rotation) cannot start image processing unless image data for one surface is prepared. As the number M of image processing modules, a value obtained by subtracting the number of recognized image processing modules 38 from the total number of image processing modules 38 in the image processing unit 50 may be used. The first division method has an advantage that the processing becomes simple.

  Further, the second division method is a method of dividing so that the processing load of each group (here, CPU processing time is used as an example of the processing load) is equal. That is, first, the processing load (CPU processing time) of each image processing module 38 constituting the image processing unit 50 is predicted. Here, among the various image processing modules 38, for the image processing module 38 that performs image processing in which the CPU processing time does not change greatly regardless of changes in the image size and processing parameters, the CPU processing time set in advance is fixed. By acquiring, the CPU processing time can be predicted. In addition, for the image processing module 38 that performs image processing in which the CPU processing time changes greatly with respect to changes in the image size and processing parameters, the relationship between the change in the image size and processing parameters and the CPU processing time is in a form such as a prediction formula The CPU processing time can be predicted by obtaining the prediction formula in advance and calculating the CPU processing time according to the image size and processing parameters.

  Next, the sum of the CPU processing times of the individual image processing modules 38 constituting the image processing unit 50 is calculated, the calculated sum of the CPU processing times is divided by the number of groups N, and the calculation result (CPU processing per group) The average value of time) is set to the threshold Th. Then, while integrating the CPU processing time of the image processing module 38 in order from the top of the image processing unit 50, the integrated value of the CPU processing time is compared with the threshold value Th, and each time the integrated value of the CPU processing time becomes equal to or greater than the threshold value Th, The image processing module 38 whose integrated value is equal to or greater than the threshold Th is grouped as one group, and the integrated value of the CPU processing time is returned to 0, thereby dividing the image processing unit 50 into N groups. . The second division method is more preferable because the processing load of each group can be made as uniform as possible, although the processing is slightly more complicated than the first division method.

  Note that the division method when dividing the image processing unit 50 into N groups is not limited to the above method, and other division methods (for example, the CPU of the j-th image processing module in the second division method). If the integrated value becomes equal to or greater than the threshold Th at the stage where the processing time is added to the integrated value, the difference A between the integrated value and the threshold Th is integrated before adding the CPU processing time of the jth image processing module. When the deviation A is small, the subsequent stage of the j-th image processing module is used as the group boundary. When the deviation B is small, the previous stage of the j-th image processing module is used as the group boundary. A dividing method) may be used.

  When the image processing unit 50 is divided into N groups as described above, in the next step 104, the buffer module 40 located at the boundary of each group is replaced with the buffer module 40 with the exclusive control function. As a result, the image processing unit 50 originally constructed has a configuration in which the image processing modules M1 to M4 and the buffer modules B1 to B3 are alternately connected as shown in FIG. 5A as an example. When the processing unit 50 is divided into the group A composed of the image processing modules M1 to M3 and the group B composed of the image processing module M4 by the division of the image processing unit 50 described above, it is shown in FIG. Thus, the buffer module B3 located at the boundary between the group A and the group B is replaced with a buffer module with an exclusive control function.

  In the next step 106, a value obtained by integrating the CPU processing time of the image processing module 38 for each group is acquired as the processing load of each group of the image processing unit 50. Further, in step 108, based on the processing load of each group acquired in step 106, a group having the maximum processing load (in the present embodiment, this is referred to as “hot spot”) is determined. In step 110, it is determined whether it is appropriate to divide (multiplex) the hot spot determined in step 108 into a plurality of partial image processing units.

  FIG. 5B shows an example of the CPU processing time in each image processing module 38 of the image processing unit 50. In this example, the CPU processing time is largely biased toward the image processing module M4 (the CPU processing time of the image processing module M4 is significantly larger than the integrated value of the CPU processing times of the other image processing modules M1 to M3). Even if the processing unit 50 is divided into the group A composed of the image processing modules M1 to M3 and the group B composed of the image processing module M4, the processing load (CPU processing time) for each group is greatly different. In such a case, in this embodiment, for a group (hot spot) with a large processing load, data of different regions in the image are cut out from the image data, and a plurality of portions that perform the same image processing on the cut out data A process of dividing (multiplexing) into the image processing unit is performed.

  However, when hot spots are divided (multiplexed) into a plurality of partial image processing units, the data processed by the individual partial image processing units is synthesized as a single image by the image synthesis module. However, depending on the content of the image processing performed at the hot spot, the image composition processing in the image composition module becomes very complicated, and therefore it may not be desirable to divide (multiplex) the image into a plurality of partial image processing units ( For example, when the hot spot image processing module 38 performs geometric transformation processing such as affine transformation or image rotation). When the image size is relatively small, even if there is a bias in the processing load for each group, the processing load of the hot spot itself is small (the CPU processing time is not so long), and a plurality of partial image processing units are transferred. By dividing (multiplexing), there is a possibility that the processing efficiency is lowered. Therefore, in the present embodiment, the determination in step 110 (determination as to whether or not it is appropriate to divide (multiplex) the hot spot into a plurality of partial image processing units) is realized by sequentially performing the following determinations. Yes.

  In other words, in the present embodiment, a division availability flag indicating whether division (multiplexing) is possible is added to the program of each image processing module 38 registered in the module library 36 of the image processing program group 34. This division possibility flag is set to 0 for the image processing module 38 that is not suitable for division into a plurality of partial image processing units, and set to 1 for the other image processing modules 38. In step 110, first, it is determined whether or not the division possibility flag added to the hot spot image processing module 38 is 1 (first determination). If the first determination is negative, it can be determined that it is not appropriate to divide the hot spot, so the determination in step 110 is negative. Accordingly, it is possible to prevent the image processing module 38 that performs image processing that is not suitable for division from being divided into a plurality of partial image processing units.

  Instead of adding the above-described division possibility flag, information on the image processing module 38 that is not suitable for division into a plurality of partial image processing units is registered in a predetermined table, and the first This determination may be performed by determining whether the information of the image processing module 38 of the hot spot is not registered in a predetermined table. Further, the division availability flag and a predetermined table are used in combination, and the division availability flag is fixedly set, while the predetermined table is made updatable by the user, and the division added to the image processing module 38 of the hot spot. It is determined whether the availability flag is 1, and if this determination is affirmative, it may be determined whether the information of the image processing module 38 of the hot spot is not registered in a predetermined table. In this aspect, the user can arbitrarily set the image processing module 38 that is prohibited from being divided into a plurality of partial image processing units. The first determination described above corresponds to the invention described in claim 2.

  If the first determination described above is affirmed, it is determined whether or not the size of the image (data) input to the hot spot is equal to or larger than a threshold (second determination). Even if the second determination is negative, it can be determined that it is not appropriate to divide the hot spot, so the determination in step 110 is negative. Thus, by dividing the hot spot image processing module 38 into a plurality of partial image processing units, it is possible to prevent the processing efficiency from being lowered. This second determination corresponds to the invention described in claim 3.

  If the second determination is also affirmed, it is determined whether or not the hot spot processing load (CPU processing time) is equal to or greater than a threshold value (third determination). Even when the third determination is negative, there is a high possibility that the processing efficiency is lowered by dividing the hot spot image processing module 38 into a plurality of partial image processing units. Since it can be determined that it is not appropriate, the determination in step 110 is denied. This third determination is related to the invention of claim 4.

  If any of the first to third determinations described above is denied and the determination of step 110 is denied, the process proceeds to step 116 without performing hot spot division (multiplexing). Each group is assigned as a thread (a thread for executing a program of each group is generated), and the image processing unit dividing process is terminated.

  On the other hand, if the above-described first determination to third determination are affirmed, it can be determined that it is appropriate to divide (multiplex) the hot spot into a plurality of partial image processing units. Thus, the hot spot division number n is determined. The division number n can be determined so that the processing load (CPU processing time) of each of the partial image processing units after the division is as equal as possible to the processing load (CPU processing time) of a group other than the hot spot. Specifically, for example, the CPU processing time of a hot spot is divided by the CPU processing time of a group other than the hot spot (the average value when there are a plurality of corresponding groups), and if a remainder occurs, the quotient is calculated. On the other hand, the result of rounding such as rounding off can be used as the division number n. In the next step 114, it is determined whether or not the division number n is larger than one. If this determination is negative (when the number of divisions n = 1), hot spot division is not necessary, and the process proceeds to step 116 described above.

  If the determination in step 114 is affirmative, the hot spot is divided into the same number of partial image processing units as the division number n in the next step 118 and thereafter. That is, in step 118, the width of the image represented by the image data input to the hot spot (the number of pixels in the x direction (the number of pixels along the input order (raster order) of the image data to the hot spot)) is the division number n. When division occurs and a remainder is generated, the result of rounding the quotient, such as rounding off, is set as the x-direction cutout amount dx. Also, 0 is input to the hot spot as the x-direction extraction start position xs, the x-direction extraction end position xe as the x-direction extraction amount dx, the y-direction extraction start position ys as 0, and the y-direction extraction end position ye. The division number n is set with the height (number of pixels in the y direction (direction orthogonal to the x direction)) of the image represented by the image data as a variable i.

  In the next step 120, it is determined whether or not the variable i is larger than zero. If the determination is affirmative, the process proceeds to step 122 to generate the image cutting module 39, and the generated image cutting module 39 is replaced with the buffer module 40 (buffer with exclusive control function) existing immediately before the hot spot. The module 40) is connected to the generated image cutout module 39. From the image data acquired from the previous buffer module 40, the coordinates (xs, ys) and the coordinates (xe, ye) are opposite to each other. The processing parameters are set so that the data of the rectangular area (see also FIG. 6) to be cut out. In step 124, the buffer module 40 without the exclusive control function is generated, and the generated buffer module 40 is connected to the subsequent stage of the image cutting module 39 generated in step 122. In the next step 126, the hot spot image processing module 38 is generated, and the generated image processing module 38 is connected to the subsequent stage of the buffer module 40 generated in step 124. In step 128, the buffer module 40 with the exclusive control function is generated, and the generated buffer module 40 is connected to the subsequent stage of the image processing module generated in step 126.

  In step 130, the module group generated in steps 122 to 128 (image cutting module 39, buffer module 40 without exclusive control function, image processing module 38, and buffer module 40 with exclusive control function) is a single unit. The partial image processing unit is allocated as a single thread (a single thread for executing the program of each module is generated). In the next step 132, the current x-direction cutout end position xe is set as the new x-direction cutout start position xs, and the x-direction cutout amount dx is set as the new x-direction cutout end position xe. Each of the added values is set, and the variable i is decremented by 1, and the process returns to step 120.

  As a result, step 122 to step 132 are repeated n times until the determination in step 120 is affirmed. As shown in FIG. 5C as an example, each of the image cut-out modules 39 and the exclusive control function is not provided. N partial image processing units formed by serially connecting the buffer module 40, the image processing module 38, and the buffer module 40 with the exclusive control function are generated as separate threads. Further, the x-direction cutout start position xs and the x-direction cutout end position xe set in the image cutout module 39 of each partial image processing unit are shifted by the x-direction cutout amount dx. From the image data held in the buffer module 40 with the exclusive control function in the previous stage by the image cutout module 39 of each partial image processing unit, as shown in FIG. The data of the image area shifted in the range is cut out, and the image processing module 38 of each partial image processing unit performs image processing only on the data of the image area cut out by the image cutting module 39 of the same partial image processing unit. Will be done.

  When n partial image processing units are generated, the determination in step 120 is negative and the process proceeds to step 134. In step 134, the image composition module 41 is generated, and the generated image composition module 41 is connected to the subsequent stage of the buffer module 40 with the exclusive control function, which is located at the last stage of the n partial image processing units. In step 136, each group other than the hot spot is assigned as a thread (a thread for executing a program of each group other than the hot spot is generated), and the image processing section dividing process is terminated.

  It should be noted that the image cutout module 39 of each partial image processing unit is a buffer module 40 with an exclusive control function located in the preceding stage of the hot spot (a buffer module denoted as “B3” in the example of FIG. 5C). 40), and the buffer module 40 with the exclusive control function corresponds to the “buffer module functioning as a part of the distribution supply unit” according to claim 8. Yes. In addition, the buffer module 40 with the exclusive control function of each partial image processing unit (the buffer module 40 represented as “B5” in the example of FIG. 5C) is an image synthesis module located at the subsequent stage of the hot spot. The buffer module 40 with the exclusive control function of each partial image processing unit is configured as “a buffer module that functions as a part of the image combining unit” according to claim 8. It corresponds to.

  When the construction of the image processing unit 50 is completed by the above processing, the workflow management unit 46A of the processing management unit 46 assigns a plurality of threads generated by the processing construction unit 42 to execute the program of the image processing unit 50 to the CPU 12 (a The processing request is input to the image processing module 38 located at the last stage of the image processing unit 50. Each time a data request is input from an arbitrary buffer module 40, a processing request is input to the image processing module 38 in the preceding stage of the requesting buffer module 40, and a processing completion notification is input from the arbitrary image processing module. Each time, a processing request is input to the image processing module as a notification source until the end of the entire processing (end of image processing for image data for one surface) is notified from the last image processing module. As a result, the image processing unit 50 operates in a parallel processing system in which image processing in the individual image processing modules 38 is performed in parallel. The workflow management unit 46A of the process management unit 46 corresponds to the control means according to the present invention (more specifically, the control means described in claims 6 and 7).

  When the processing load (CPU processing time) of the module unit of the image processing unit 50 originally constructed by the processing construction unit 42 is greatly biased to a specific image processing module 38, the module is divided into a plurality of groups as a unit. Even if it is performed, for example, as shown in FIG. 5B, it is difficult to equalize the processing load of each group, and the processing efficiency when the programs of each group are executed in parallel is greatly reduced. . On the other hand, in the present embodiment, in the image processing unit division processing described above, among the groups of the image processing unit 50, a group (hot spot) having the largest processing load is divided (multiplexed) into a plurality of partial image processing units. If the division (multiplexing) into a plurality of partial image processing units is appropriate, the processing load (CPU processing time) of the group other than the hot spot is as equal as possible. Thus, the number n of hot spot divisions is determined and divided into n partial image processing units, and the programs of the n partial image processing units are executed in parallel as separate threads. Even when the processing load of each module is greatly biased to a specific image processing module 38, the processing load for each execution unit (thread) of the program of the image processing unit 50 is uniform or uniform. Can be brought close to, enhancing performance when executing each thread in parallel, it is possible to realize the shortening of the turn processing time.

  In addition, by executing a plurality of partial image processing units in parallel as separate threads, the buffer module 40 located in the preceding stage of the hot spot can be subjected to the image cutting module 39 of each partial image processing unit. As a result, the buffer 40A is accessed asynchronously with each other. However, in the present embodiment, the buffer module with the exclusive control function is used as the buffer module 40 described above. It is possible to prevent inconveniences such as processing locking due to access conflict. In addition, since the individual partial image processing units and the image synthesis module 41 also operate in parallel, the buffer module 40 provided by the image processing module 38 in the preceding stage is not used for the buffer module 40 provided in the last stage of each partial image processing unit. And the access to the buffer 40A by the subsequent image composition module 41 may conflict with each other. In this embodiment, since the buffer module with the exclusive control function is also used as the buffer module 40, the previous image It is also possible to prevent inconveniences such as processing being locked due to contention between accesses to the buffer 40A by the processing module 38 and the subsequent image composition module 41.

  In addition, the buffer module with the exclusive control function increases the overhead of computation resources such as the CPU 12 by the amount of exclusive control. In the present embodiment, the buffer module with the exclusive control function is replaced with a necessary part (hot spot). Since the buffer modules without exclusive control function are used as buffer modules provided only at the boundary of each group including the previous stage and the last stage of each partial image processing unit), the buffer module is exclusive. It is possible to suppress an increase in overhead of computation resources such as the CPU 12 accompanying the control to the minimum necessary.

  In the above description, a mode in which a group (hot spot) composed of a single image processing module 38 is divided (multiplexed) into a plurality of partial image processing units has been described. However, the present invention is not limited to this. The group to be divided (multiplexed) into the image processing unit may be a group including a plurality of image processing modules 38.

  In the above description, a mode has been described in which division (multiplexing) into a plurality of partial image processing units is performed only for a single group (hot spot) having the maximum processing load among a plurality of groups of the image processing unit 50. However, the present invention is not limited to this, and a predetermined number (a plurality of) groups are selected in descending order of processing load, and division (multiplexing) into a plurality of partial image processing units is performed using each of the selected groups as a hot spot. Alternatively, division (multiplexing) into a plurality of partial image processing units may be performed for an infinite number of groups without fixing the number of groups to be divided (multiplexed). Specifically, for example, in parallel with the execution of the program of the image processing unit 50, other application programs are also executed by the computer 10, and the CPU processing time of individual threads corresponding to the other application programs is When the CPU processing time of each thread corresponding to the image processing unit 50 is greatly different from the CPU processing time of each thread corresponding to the image processing unit 50, the individual thread corresponding to another application program For the purpose of approaching the CPU processing time, the presence / absence of division (multiplexing) and the number of divisions n (number of multiplexing) are set for all groups of the image processing unit 50, and multiplexing is performed among all groups. May be divided into the same number of partial image processing units as the division number n set for the group. . This aspect corresponds to the invention described in claim 5.

  Also, in the above, after a specific group to be divided (multiplexed) is divided (multiplexed) into a plurality of partial image processing units, different image regions of the image represented by the input image data in each partial image processing unit However, the present invention is not limited to this. For example, when the number of divisions n is larger, a plurality of image regions are processed for the image data. In addition to the division, the division for each color is also performed, and the divided data may be input to individual partial image processing units to perform image processing. Specifically, for example, when the division number n is 9 and the input image data is composed of R, G, B color data, the image represented by the input image data is divided into three image areas and divided. By further dividing the data of the individual image areas for each color of R, G and B, the input image data is divided into nine data, and each data is input to different partial image processing units for image processing. May be performed. The present invention includes the above-described embodiments within the scope of rights.

  Furthermore, in the above, after dividing the hot spot of the image processing unit 50 into a plurality of partial image processing units, a plurality of threads for executing the program of the image processing unit 50 are executed in parallel by the CPU 12 (and other computing resources). In the above, the mode in which the image processing unit 50 is operated in the parallel processing method has been described. However, in the present invention, image processing is always performed by the single image processing module 38 and the image processing module 38 that performs image processing is sequentially performed. The present invention can also be applied to the case where the image processing unit 50 is operated by the sequential processing method for switching. That is, even when the image processing unit 50 is operated by the sequential processing method, if the processing load of the image processing unit 50 is biased toward a specific image processing module, only the specific image processing module performs image processing. The length of the current period becomes a bottleneck and the processing time increases. Here, when there are a plurality of calculation resources, if a specific image processing module is divided into a plurality of partial image processing units and a plurality of partial image processing units are operated in parallel by the plurality of calculation resources, Processing time can be greatly reduced.

  In the above description, the image processing program group 34 corresponding to the image processing program according to the present invention is stored (installed) in the storage unit 20 in advance. However, the image processing program according to the present invention is a CD-ROM. It is also possible to provide it in a form recorded on a recording medium such as a DVD-ROM.

It is a block diagram which shows schematic structure of the computer (image processing apparatus) which concerns on this embodiment. It is a block diagram which shows the structural example of an image process part. (A) is an image processing module, and (B) is a block diagram showing a schematic configuration of a buffer module and processing to be executed. It is a flowchart which shows the content of the image process part division | segmentation process performed by the process construction part. It is a block diagram which shows an example of the image process part for demonstrating the division | segmentation of the image process part by an image process part division | segmentation process. It is an image figure for demonstrating the image area | region which each partial image process part makes a process target when a hot spot is divided | segmented into the some partial image process part.

Explanation of symbols

10 Computer 12 CPU
20 Storage Unit 38 Image Processing Module 39 Image Extraction Module 40 Buffer Module 41 Image Composition Module 42 Processing Construction Unit 46 Processing Management Unit 50 Image Processing Unit

Claims (9)

A plurality of types of image processing modules that perform different image processing on the image data acquired from the previous stage of the own module, and output the image data that has undergone the image processing or the processing result of the image processing to the subsequent stage of the own module, A buffer module that causes image data output from the module to be written to the buffer and that the image data stored in the buffer is read out by a subsequent module, and an image data that represents the image data output from the previous module. The distribution supply unit that divides and divides the divided data into a plurality of data corresponding to different areas of the plurality of modules, and supplies data to different modules among the plurality of modules in the subsequent stage, and data acquired from the modules in the previous stage, respectively Image composition unit that composes image data as single image data Storage means for each storing a program,
The buffer module is connected to at least one of the preceding stage and the succeeding stage of each of the one or more image processing modules selected from the plurality of types of image processing modules by a program stored in the storage means, They are connected in a pipeline form or a directed acyclic graph form, and at least a part of them is provided in multiples, and the individual processing units provided in multiples are respectively connected to the preceding stage of the multiplexed part via the distribution supply unit. And a constructing means for constructing an image processing unit connected to the subsequent stage of the multiplexed part via the image synthesizing unit,
Control means for performing image processing by the image processing section constructed by the construction means;
An image processing apparatus.   The construction means is an image processing module to which information indicating that it is a multiplexing target is added from among the image processing modules constituting the image processing unit, or is registered as a multiplexing target in a predetermined table The image processing apparatus according to claim 1, wherein a part including the image is selected as a part to be multiplexed.   The construction means performs multiplexing of at least a part of the image processing unit only when the size of the image on which the image processing is performed by the image processing unit is equal to or larger than a threshold value. Item 3. The image processing apparatus according to Item 2.   The construction means calculates the processing load for each part when the provisional image processing unit in which each module is connected in a pipeline form or a directed acyclic graph form is divided into a plurality of parts, The image processing apparatus according to claim 1, wherein a part to be multiplexed is selected from the plurality of parts based on a processing load for each part.   The constructing unit is configured based on a processing load calculated for each part of the temporary image processing unit, and each of the plurality of parts corresponding to the multiplexed part and the part of the plurality of parts. Set the presence / absence of multiplexing and the number of multiplexing for each individual part so that the processing load of each individual non-multiplexed part approaches a uniform state, and set the presence of multiplexing among the plurality of parts 5. The image processing apparatus according to claim 4, wherein the same number of processing units as the set number of multiplexing units are provided in the set part.   The control unit causes a computer to execute in parallel each of the programs of at least the individual processing units corresponding to the multiplexed portion among the image processing units constructed by the construction unit. Item 6. The image processing device according to any one of Items 5 to 6.   The control means causes a computer to execute a program of an individual processing unit corresponding to the multiplexed part of the plurality of parts and a program of an individual non-multiplexed part of the plurality of parts by a computer in parallel. The image processing apparatus according to claim 5, wherein: The storage means stores, as the buffer module program, a first buffer module that performs exclusive control for access to the buffer and a second buffer module program that does not perform exclusive control for access to the buffer. Has been
In constructing the image processing unit, the constructing unit selects the first buffer module as a buffer module that functions as at least a part of the distribution supply unit and a buffer module that functions as a part of the image composition unit, respectively. 8. The image processing apparatus according to claim 6, wherein the second buffer module is selected as another buffer module. A plurality of types of image processing modules that perform different image processing on the image data acquired from the previous stage of the own module, and output the image data that has undergone the image processing or the processing result of the image processing to the subsequent stage of the own module, A buffer module that causes image data output from the module to be written to the buffer and that the image data stored in the buffer is read out by a subsequent module, and an image data that represents the image data output from the previous module. The distribution supply unit that divides and divides the divided data into a plurality of data corresponding to different areas of the plurality of modules, and supplies data to different modules among the plurality of modules in the subsequent stage, and data acquired from the modules in the previous stage, respectively Image composition unit that composes image data as single image data Each computer that is connected to the storage means for storing a program,
The buffer module is connected to at least one of the preceding stage and the succeeding stage of each of the one or more image processing modules selected from the plurality of types of image processing modules based on the programs of the various modules stored in the storage means. , Each module is connected in the form of a pipeline or a directed acyclic graph, and at least a part of the modules is multiplexed, and the distribution module is inserted before the multiplexed part, and the synthesis module is inserted after the multiplexed part. A construction means for constructing the processed image processing unit,
And an image processing program for causing the image processing unit constructed by the construction unit to function as a control unit that performs image processing.