JP2008139968A - Image processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To switch priority given to a decrease in the use amount of a storage resource or the processing efficiency of image processing according to an operating environment. <P>SOLUTION: An image processing part is configured to connect buffer modules each carrying out processing for writing image data outputted from a prior-stage module into a buffer in at least one of a preceding stage and a succeeding stage of each of a plurality of image processing modules which acquire image data from the preceding stages to perform image processing and output the image data to the succeeding stages, and for reading the image data by a succeeding-stage module. In constituting the image processing part, a memory capacity allottable to each buffer module is referred to when initializing the individual buffer modules (78). When the allottable memory capacity is smaller than a capacity for one image region as a memory region allotment request size used as the buffer, the allottable memory capacity is set (82), and when the allottable memory capacity is a capacity for the one image region or more, a capacity for the one image region is set (84). The allotment of the memory region of the set size is then requested (86). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a program, and in particular, each module is in a pipeline form or a directed acyclic graph form so that a buffer module is connected to at least one of a front stage and a rear stage of a plurality of image processing modules. And an image processing program for causing a computer to function as the image processing apparatus.

  In an image processing device that performs image processing on input image data, a DTP (desktop publishing) system that can handle images, a print system that records an image represented by input image data on a recording material, etc. Various types of image processing such as enlargement / reduction, rotation, affine transformation, color conversion, filter processing, and image synthesis are performed on the image data. In these devices and systems, if the attributes of the input image data and the contents, procedures, parameters, etc. of the image processing for the image data are fixed, the image processing may be performed by specially designed hardware. However, for example, when various image data with different color space and the number of bits per pixel are input, or when the contents, procedures, and parameters of the image processing are changed variously, the image processing to be executed is more flexible. A changeable configuration is required.

  In order to satisfy such a requirement, for example, in Patent Document 1, a programmable processing module is connected to a pipeline form or a DAG (Directed Acyclic Graph) form to perform desired image processing. Technologies that enable this have been proposed. The technology described in Patent Literature 1 is configured so that the contents of the arithmetic processing in each of the plurality of programmable arithmetic processing units and the connection form of each programmable arithmetic processing unit by the network unit can be freely set from the outside through the host control means. As a result, a digital video signal processing apparatus capable of high-speed and advanced arithmetic processing and having a high degree of freedom for function change and system change is realized.

On the other hand, in Patent Document 2, conversion processing classification settings representing pixel areas for performing conversion processing collectively are stored in association with resolutions, the resolution set for the image data is acquired, and the acquired resolution is acquired. Technology that performs image conversion processing using the error diffusion method in units of the same conversion processing section, with neighboring pixels set as conversion processing sections in the conversion processing section setting corresponding to, and all the pixels in the conversion processing section as a conversion target Is disclosed.
JP-A-5-260373 JP 2005-39388 A

  When configuring an image processing apparatus that performs desired image processing by arbitrarily combining a plurality of types of image processing modules as in the technique described in Patent Document 1, there are the following problems. That is, each image processing module has a unit (for example, a pixel unit, a line unit, a plurality of line units, a surface unit, etc.) that can be easily processed according to the type and content of image processing to be executed. However, in order to be able to connect the image processing modules in an arbitrary order and process them in a coordinated manner, the output units of all the image processing modules are aligned, or each image processing module is set to an arbitrary input unit. It is necessary to be able to cope with it, and the configuration of each image processing module becomes complicated. In addition, since each image processing module operates in cooperation with other image processing modules, each image processing module is connected to its own module other than the part that actually performs image processing on the input image data. In addition, a part for controlling processing for transferring image data to and from other image processing modules is required, and the configuration of each image processing module is further complicated.

  The above problem is that a buffer is provided before and after the image processing module, and various image processing is performed by each image processing module. For example, the conversion target in the image conversion processing using the error diffusion method described in Patent Document 2 Image processing modules (number of pixels to be converted) according to the resolution, each image processing module can be processed from the previous buffer in units that are easy to process according to the type and content of the image processing to be executed. It can be solved by configuring to acquire

  However, in this configuration, considering the case where the environment in which the image processing apparatus operates is an environment in which the capacity of storage resources such as a memory that can be used as a buffer is limited, the individual buffers provided before and after the image processing module Each time a buffer is required, it requests allocation of a storage resource having a capacity corresponding to the processing unit in the preceding and succeeding image processing modules, and uses the allocated storage resource as a buffer. It is desirable to configure as a buffer module having a function of releasing the storage resources used as the. However, in the above-described configuration, the allocation of storage resources is frequently requested by individual buffer modules, which increases the overhead (load due to processing other than image processing) and reduces the processing efficiency of image processing. .

  The present invention has been made in consideration of the above facts, and is an image processing apparatus and an image processing program that can switch according to the operating environment whether priority is given to reducing the use amount of storage resources or processing efficiency of image processing. Is the purpose.

  In order to achieve the above object, an image processing apparatus according to the first aspect of the present invention acquires image data from the previous stage of its own module, performs predetermined image processing on the acquired image data, and receives the image processing. Alternatively, at least one of the front and rear stages of the plurality of image processing modules having a function of outputting the processing result of the image processing to the subsequent stage of the module requests the allocation of the storage resource every time the storage resource used as a buffer is insufficient. Using the allocated storage resource as a buffer, the image data output from the module in the previous stage of the own module is written into the buffer, and the image data stored in the buffer is read out by the module in the subsequent stage of the own module. The individual modules are connected so that the buffer modules that perform the processing Each time a storage resource allocation is requested from the image processing unit configured to be connected in a pipeline form or a directed acyclic graph form, and the buffer module requests storage resource allocation, the requested storage module is transferred to the requesting buffer module. Information indicating the storage resource management means to be allocated and the capacity of the storage resource that can be allocated to each buffer module, or set to each buffer module that is set according to the capacity of the storage resource that can be allocated to each buffer module Holding means for holding information representing a storage resource allocation rule, and each of the buffer modules requests storage resource allocation to the storage resource management means based on the information stored in the holding means For switching the capacity of the storage resource requested to the storage resource management means It is characterized in that it comprises a stage.

  The image processing apparatus according to the first aspect of the present invention acquires image data from the previous stage of its own module, performs predetermined image processing on the acquired image data, and obtains the image data subjected to the image processing or the processing result of the image processing. At least one of the front and rear stages of a plurality of image processing modules having a function of outputting to the subsequent stage of the own module requests the allocation of the storage resource whenever the storage resource used as a buffer is insufficient, and the allocated storage resource is buffered. The buffer modules are connected to each other so that the image data output from the module preceding the module is written to the buffer and the image data stored in the buffer is read by the module subsequent to the module. Each module can be pipelined or directed acyclic graph In is constructed are connected.

  The storage resource management means allocates the requested storage resource to the requesting buffer module each time a storage resource is requested from the buffer module, and the holding means can be assigned to each buffer module. The information indicating the storage resource capacity or the storage resource allocation rule to each buffer module set according to the storage resource capacity that can be allocated to each buffer module is stored. A memory is suitable as the storage resource, but an HDD (Hard Disk Drive), a flash memory, or the like may be used. In addition, as a rule for allocating storage resources to individual buffer modules, specifically, “For each buffer module, for example, storage resources for one surface of image data or for one line (corresponding capacity) An allocation rule such as “allocation” can be applied. Such an allocation rule is input by a user through an input unit as described in claim 5, for example, and a holding unit is input through the input unit. It is possible to configure so as to hold information representing the allocation rule input by the user. According to the first aspect of the present invention, the capacity of the storage resource requested to the storage resource management means in the storage resource allocation request to the storage resource management means based on the information stored in the holding means in each buffer module Control means for switching is provided.

  Thus, for example, in an operating environment in which the storage resources that can be used as buffers are limited, the storage means holds information that limits the capacity of storage resources that can be allocated to individual buffer modules to a relatively small capacity. Thus, each buffer module requests storage resource management means to allocate storage resources in units of a relatively small capacity, and the use of storage resources is saved. Also, for example, in an operating environment where there are plenty of storage resources that can be used as buffers, the holding means holds information that makes the storage resource capacity that can be allocated to each buffer module relatively large. Thus, each buffer module requests storage resource management means to allocate storage resources in units of relatively large capacity, and the frequency with which storage resource management means is requested for storage resource management means. By reducing the overhead (reducing the number of times) and reducing the overhead, the processing efficiency of the image processing is improved. Therefore, according to the first aspect of the present invention, it is possible to switch according to the operating environment whether to give priority to reducing the use amount of storage resources or to give priority to the processing efficiency of image processing.

  When there are plenty of storage resources that can be used as buffers, the larger the storage resource capacity that the buffer module requests from the storage resource management means in the storage resource allocation request, the more the storage resource management means Although the frequency at which storage resource allocation is requested decreases (the number of times decreases), if a storage resource with a capacity larger than necessary is allocated to each buffer module, part of the allocated storage resource is used effectively. In other words, the effective utilization rate of the storage resource is reduced.

  In consideration of the above, in the first aspect of the invention, the control means of each buffer module can be assigned to each buffer module based on the information stored in the holding means, for example, as described in claim 2 The storage resource management means is requested to allocate the smaller one of the recognized capacity and the predetermined upper limit capacity (for example, the capacity capable of storing an image for one surface). In this case, it is preferable to set the storage resource capacity required for the storage resource management means. As a result, even when there are plenty of storage resources that can be used as buffers, the storage resource capacity requested by the buffer module to the storage resource management means in the storage resource allocation request is limited to the upper limit capacity or less. The storage resource can be used effectively.

  An image processing apparatus according to a third aspect of the present invention provides an image processing engine that performs image processing on image data for each unit processing data amount set in advance, and image data that the image processing engine performs image processing on At least one of the front and rear stages of a plurality of image processing modules provided with control means for obtaining from the previous stage and outputting image data processed by the image processing engine or the processing result of the image processing to the subsequent stage of the own module Using the allocated storage resource as a buffer, the image data output from the module in the previous stage of the own module is written into the buffer, and the image data stored in the buffer is read out by the module in the subsequent stage of the own module. Each time a storage resource used as a buffer is insufficient Storage of a capacity corresponding to the amount of image data written to the buffer by one writing by the module preceding the module and the amount of image data read from the buffer by one reading by the module subsequent to the module An image processing unit configured such that individual modules are connected in a pipelined form or a directed acyclic graph form so that buffer modules that perform processing for requesting module allocation are connected to each other, and stored from the buffer module Each time resource allocation is requested, storage resource management means for allocating the requested storage resource to the requesting buffer module and information indicating the storage resource capacity that can be allocated to each buffer module, or individual Storage resources that can be assigned to different buffer modules Holding means for holding information indicating a rule for allocating storage resources to individual buffer modules, which is set according to the capacity of the storage module, and the control means of each of the image processing modules stores the information in the holding means The number of repetitions N of image processing is set based on the stored information, and image data is acquired from the previous stage of the own module and is sent to the subsequent stage of the own module in units of the amount of image data obtained by multiplying the unit processing data amount by N. In addition to outputting image data, the image processing for each unit processing data amount by the image processing engine is repeated N times to perform image processing on the image data from the previous stage of the own module.

  An image processing unit according to a third aspect of the invention includes an image processing engine that performs image processing on image data by a predetermined unit processing data amount, and image data that the image processing engine performs image processing on the front stage of its own module. Assigned to at least one of the preceding and succeeding stages of a plurality of image processing modules having control means for outputting image data processed by the image processing engine or image processing results to the succeeding stage of the own module. The stored memory resource is used as a buffer, image data output from the module in the previous stage of its own module is written into the buffer, and the image data stored in the buffer is read out by the module in the subsequent stage of its own module, Each time there is a shortage of storage resources to use, Processing for requesting allocation of a storage module having a capacity corresponding to the amount of image data written to the buffer by one writing by the module and the amount of image data read from the buffer by one reading by the module subsequent to the module. The individual modules are connected in a pipeline form or a directed acyclic graph form so that the buffer modules to be performed are connected to each other.

  The invention described in claim 3 is provided with storage resource management means and holding means similar to those of the invention described in claim 1, and the control means of each image processing module is information stored in the holding means. The number N of image processing repetitions is set based on the above, and the image data amount obtained by multiplying the unit processing data amount by N is used as a unit to acquire image data from the previous stage of the own module and output the image data to the subsequent stage of the own module. In addition, the image processing for each unit processing data amount by the image processing engine is repeated N times to perform image processing on the image data from the previous stage of the own module.

  As described above, the buffer module according to the third aspect of the present invention has the amount of image data written to the buffer by one writing by the module preceding the module each time the storage resource used as the buffer is insufficient, and the buffer module itself. Storage resource requested by the buffer module to the storage resource management means in the storage resource allocation request because the allocation of the storage module having a capacity corresponding to the amount of image data read from the buffer by one reading by the module subsequent to the module is requested. The capacity of the image data changes in accordance with the amount of image data written to the buffer at a time by the image processing module in the previous stage of the buffer module and the amount of image data read out from the buffer at the same time by the image processing module in the subsequent stage.

  For this reason, for example, in an operating environment where the storage resources that can be used as buffers are limited, the storage means holds information that limits the storage resource capacity that can be allocated to each buffer module to a relatively small capacity. Thus, the control unit of the image processing module connected to the buffer module sets a relatively small value as the number of repetitions N of the image processing, and the image from the previous stage of the own module is set in units of a relatively small amount of image data. Data is acquired and image data is output to the subsequent stage of the module. Along with this, each buffer module requests storage resource management means to allocate storage resources in units of relatively small capacity, thereby saving the use of storage resources. Also, for example, in an operating environment where there are plenty of storage resources that can be used as buffers, the holding means holds information that makes the storage resource capacity that can be allocated to each buffer module relatively large. Thus, the control means of the image processing module connected to the buffer module sets a relatively large value as the number of repetitions N of the image processing, and the image from the previous stage of the own module in units of a relatively large amount of image data. Data is acquired and image data is output to the subsequent stage of the module. Along with this, each buffer module requests storage resource management means to allocate storage resources in units of relatively large capacity, and storage resource management means are requested to allocate storage resources. The frequency of image processing decreases (the number of times decreases) and overhead is reduced, so that the processing efficiency of image processing is improved. Therefore, in the invention as set forth in claim 3, it is possible to switch according to the operating environment whether priority is given to reducing the use amount of storage resources or priority is given to the processing efficiency of image processing.

  In addition, in the invention described in claim 3, the control means of each image processing module performs unit processing on the image processing engine and image data on which the image processing engine performs image processing as described in claim 4, for example. Based on the information stored in the holding means in the image processing module provided with the first control means for obtaining the data amount and outputting the image data processed by the image processing engine or the processing result of the image processing, The number N of repetitions of image processing is set, and image data from the previous stage of the module is acquired in units of the amount of image data obtained by multiplying the unit processing data amount by N. The acquired image data is incremented by the unit processing data amount. The image data or the image output from the first control means is delivered to the one control means and the image processing engine performs image processing. Processing result of the processing a second control unit for outputting to the subsequent module itself is added after N times accumulation, the second control unit and the first control unit may also be realized by cooperating.

  Thus, when the image processing apparatus according to the third aspect of the present invention is realized by causing a computer to execute a program, the image processing apparatus performs the individual processing regardless of the type of image processing performed by the image processing engine of each image processing module. Since the program that functions as the second control means of the image processing module can be shared by all the image processing modules, the development load of the image processing apparatus can be reduced.

  According to a sixth aspect of the present invention, there is provided an image processing program that obtains image data from a preceding stage of its own module, performs predetermined image processing on the acquired image data, and the image data that has undergone the image processing or the image processing When the storage resource used as a buffer is insufficient, at least one of the front and rear stages of the plurality of image processing modules having the function of outputting the processing result of the module to the subsequent stage of the module is requested and allocated. Using the storage resource as a buffer, the image data output from the module preceding the own module is written to the buffer, and the image data stored in the buffer is read by the module subsequent to the module. Individual modules are pied so that each buffer module is connected. An image processing unit configured to be connected in a line form or a directed acyclic graph form, and whenever a storage resource is requested to be allocated from the buffer module, a storage resource having a requested capacity is allocated to the requesting buffer module. Information indicating the storage resource management means and the storage resource capacity that can be allocated to each buffer module, or set to each buffer module that is set according to the storage resource capacity that can be allocated to each buffer module Functioning as a holding means for holding information representing an allocation rule of storage resources, and each of the buffer modules in a storage resource allocation request to the storage resource management means based on the information stored in the holding means Switching the storage resource capacity requested to the storage resource management means It is characterized in that it comprises a control means.

  According to a sixth aspect of the present invention, there is provided an image processing program that causes a computer to function as the image processing unit, the storage resource management unit, and the holding unit, and that the individual buffer modules constituting the image processing unit are controlled by the control unit. Since the computer is a program for providing the configuration, the computer functions as the image processing apparatus according to claim 1 by executing the image processing program according to the invention according to claim 6, Similarly to the first aspect of the present invention, it is possible to switch according to the operating environment whether priority is given to reducing the use amount of storage resources or processing efficiency of image processing.

  According to a seventh aspect of the present invention, there is provided an image processing program for causing a computer to execute an image processing engine that performs image processing on image data by a predetermined unit processing data amount, and image data that the image processing engine performs image processing. Obtained from the previous stage of the own module and provided with control means for outputting the image data subjected to the image processing by the image processing engine or the processing result of the image processing to the subsequent stage of the own module. At least one of the allocated storage resources is used as a buffer, the image data output from the module in the previous stage of the own module is written into the buffer, and the image data stored in the buffer is transferred to the module in the subsequent stage of the own module. And read as a buffer. The amount of image data written to the buffer by one write by the module preceding the module each time the resource is insufficient, and the amount of image data read from the buffer by one read by the module subsequent to the module An image processing unit configured such that individual modules are connected in a pipeline form or a directed acyclic graph form so that buffer modules that perform processing for requesting allocation of storage modules according to Each time storage resource allocation is requested from the buffer module, storage resource management means for allocating the requested storage resource to the requesting buffer module, and the storage resource capacity that can be allocated to each buffer module Information or individual buffer modules Functioning as a holding unit that holds information representing a storage resource allocation rule for each buffer module, which is set according to the capacity of the storage resource that can be allocated to each of the image processing modules. Based on the information stored in the holding means, the number N of image processing repetitions is set, and the image data amount obtained from the previous stage of the own module and the own module are set in units of the image data amount obtained by multiplying the unit processing data amount by N. The image data is output to the subsequent stage, and the image processing is performed on the image data from the previous stage of the own module by repeating the image processing for each unit processing data amount by the image processing engine N times. Yes.

  An image processing program according to a seventh aspect of the present invention causes a computer to function as the image processing unit, the storage resource management unit, and the holding unit, and controls the control unit of each image processing module constituting the image processing unit. Since this is a program for configuring, when the computer executes the image processing program according to the invention described in claim 7, the computer functions as the image processing apparatus described in claim 3. Similarly to the invention described in 3, it is possible to switch whether to give priority to reduction of the use amount of storage resources or to give priority to the processing efficiency of image processing according to the operating environment.

  As described above, the present invention is based on information indicating the capacity of storage resources that can be allocated to individual buffer modules or information indicating the rules for allocating storage resources to individual buffer modules. Since the storage resource capacity requested to the storage resource management means is switched in the storage resource allocation request to the storage resource management means, it is determined whether priority is given to reducing the use amount of the storage resource or processing efficiency of the image processing. It has an excellent effect that it can be switched according to the operating environment.

  According to the present invention, each image processing module repeats image processing based on information indicating the capacity of storage resources that can be allocated to individual buffer modules or information indicating rules for allocating storage resources to individual buffer modules. The number of times N is set, and the image data amount obtained by multiplying the unit processing data amount by N is used as a unit to acquire the image data from the previous stage of the own module and output the image data to the subsequent stage of the own module. By repeating the image processing for each unit processing data amount by N times, the image processing is performed on the image data from the previous stage of its own module. It has an excellent effect that it is possible to switch whether to give priority to efficiency according to the operating environment.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 shows a computer 10 that can function as an image processing apparatus according to the present invention. The computer 10 is incorporated in any image handling apparatus that needs to perform image processing internally, such as a copying machine, a printer, a facsimile machine, a multifunction machine having these functions, a scanner, a photographic printer, or the like. Alternatively, it may be an independent computer such as a personal computer (PC), or may be a computer incorporated in a portable device such as a PDA (Personal Digital Assistant) or a cellular phone.

  The computer 10 includes a CPU 12, a memory 14 including a DRAM or SRAM, a display unit 16, an operation unit 18, a storage unit 20, an image data supply unit 22, and an image output unit 24, which are connected to each other via a bus 26. Has been. When the computer 10 is incorporated in the image handling device as described above, a display panel, a numeric keypad, or the like composed of an LCD or the like provided in the image handling device can be applied as the display unit 16 and the operation unit 18. When the computer 10 is an independent computer, a display, a keyboard, a mouse, or the like connected to the computer can be applied as the display unit 16 or the operation unit 18. The storage unit 20 is preferably an HDD (Hard Disk Drive), but other nonvolatile storage means such as a flash memory can be used instead.

  The image data supply unit 22 only needs to be able to supply image data to be processed. For example, an image reading unit that reads an image recorded on a recording material such as paper or photographic film, and outputs the image data. A receiving unit that receives image data from the outside via a line, an image storage unit (memory 14 or storage unit 20) that stores image data, and the like can be applied. The image output unit 24 may be any unit that outputs image data that has undergone image processing or an image represented by the image data. For example, the image recording unit records an image represented by the image data on a recording material such as paper or a photosensitive material. A display unit that displays an image represented by image data on a display, a writing device that writes image data to a recording medium, and a transmission unit that transmits image data via a communication line can be applied. The image output unit 24 may be an image storage unit (memory 14 or storage unit 20) that simply stores image data that has undergone image processing.

  As shown in FIG. 1, the storage unit 20 includes various programs executed by the CPU 12, such as management of resources such as the memory 14, management of execution of programs by the CPU 12, and communication between the computer 10 and the outside. A program of the system 30, an image processing program group 34 for causing the computer 10 to function as an image processing apparatus according to the present invention, and a desired image for the image processing apparatus realized by the CPU 12 executing the image processing program group. Programs of various applications 32 to be processed (represented as application program group 32 in FIG. 1) are stored.

  The image processing program group 34 reduces the development load when developing the above-described various image handling devices and portable devices, and reduces the development load when developing an image processing program usable on a PC or the like. The purpose is a program developed so that it can be commonly used in various devices (platforms) such as various image handling devices, portable devices, and PCs, and corresponds to the image processing program according to the present invention. The image processing apparatus realized by the image processing program group 34 constructs an image processing unit that performs image processing instructed by the application 32 in accordance with a construction instruction from the application 32, and executes the image processing unit in accordance with an execution instruction from the application 32. The image processing program group 34 instructs the construction of an image processing unit (an image processing unit having a desired configuration) for performing desired image processing, or the constructed image processing unit. The application 32 is provided with an interface for instructing execution of image processing according to the above. For this reason, even when newly developing an arbitrary device that needs to perform image processing internally, regarding the development of a program for performing the image processing, the above-described interface is used to perform the image processing required for the device. It is only necessary to develop the application 32 to be used and executed by the image processing program group 34, and it is not necessary to newly develop a program for actually performing image processing, so that the development load can be reduced.

  Further, as described above, the image processing apparatus realized by the image processing program group 34 constructs an image processing unit that performs image processing instructed by the application 32 according to the construction instruction from the application 32, and constructs the constructed image processing unit. Therefore, for example, when the color space of the image data to be processed and the number of bits per pixel are indefinite or the contents of the image processing to be executed, the procedure / parameter, etc. are indefinite, When the application 32 instructs to reconstruct the image processing unit, the image processing executed by the image processing apparatus (image processing unit) can be flexibly changed according to the image data to be processed.

  Hereinafter, the image processing program group 34 will be described. As shown in FIG. 1, the image processing program group 34 is roughly divided into a module library 36, a process construction unit 42 program, and a process management unit 46 program. As will be described in detail later, the processing construction unit 42 according to the present embodiment, according to an instruction from the application, as illustrated in FIG. 2 as an example, one or more image processing modules 38 that perform predetermined image processing, A buffer module 40 that is arranged in at least one of the preceding stage and the succeeding stage of each image processing module 38 and has a buffer for storing image data, and a pipeline form or a DAG (Directed Acyclic Graph) form An image processing unit 50 formed by linking is constructed. The entity of each image processing module constituting the image processing unit 50 is a first program executed by the CPU 12 and causing the CPU 12 to perform predetermined image processing, or an external unit that is executed by the CPU 12 and not shown in FIG. 2 is a second program for instructing execution of processing to the image processing apparatus (for example, a dedicated image processing board), and the module library 36 described above stores predetermined different image processing (for example, input processing and filter processing). , Color conversion processing, enlargement / reduction processing, skew angle detection processing, image rotation processing, image composition processing, output processing, and the like) are registered. Hereinafter, in order to simplify the description, it is assumed that the individual image processing modules constituting the image processing unit 50 are the first program.

  As shown in FIG. 3A as an example, each image processing module 38 includes an image processing engine 38A that performs image processing on image data by a predetermined unit processing data amount, and a front stage and a rear stage of the image processing module 38. The control unit 38B is configured to input / output image data to / from the module and to control the image processing engine 38A. The unit processing data amount in each image processing module 38 is the image processing engine 38A from an arbitrary number of bytes including one line of the image, a plurality of lines of the image, one pixel of the image, one surface of the image, and the like. Is selected and set in advance according to the type of image processing to be performed. For example, in the image processing module 38 that performs color conversion processing and filter processing, the unit processing data amount is one pixel, and the image to be enlarged / reduced is processed. In the processing module 38, the unit processing data amount is equivalent to one line of the image or a plurality of lines of the image, and in the image processing module 38 that performs image rotation processing, the unit processing data amount is equivalent to one image, and the image compression / decompression processing is performed. In the image processing module 38 to be performed, the unit processing data amount is N bytes depending on the execution environment.

  The module library 36 also registers image processing modules 38 having the same type of image processing executed by the image processing engine 38A and different contents of the image processing executed (in FIG. 1, this type of image processing is performed). Modules are indicated as “module 1” and “module 2”). For example, with respect to the image processing module 38 that performs enlargement / reduction processing, the image processing module 38 that performs reduction processing to reduce the input image data to 50% by thinning out every other pixel is designated for the input image data. A plurality of image processing modules 38 such as an image processing module 38 that performs enlargement / reduction processing at the enlarged / reduced rate are prepared. For example, the image processing module 38 that performs color conversion processing includes an image processing module 38 that converts the RGB color space to the CMY color space, an image processing module 38 that converts the color space to the opposite, and an L * a * b * color space. Image processing modules 38 for performing other color space conversions are prepared.

  In addition, the control unit 38B of the image processing module 38 receives an image from a module (for example, the buffer module 40) in the previous stage of its own module in order to input image data necessary for the image processing engine 38A to process each unit processing data amount. Data is acquired for each unit read data amount, and image data output from the image processing engine 38A is output for each unit write data amount to a subsequent module (for example, the buffer module 40) (data amount such as compression by the image processing engine 38A). If the image processing with the increase / decrease is not performed, the unit writing data amount = the unit processing data amount) or the result of the image processing by the image processing engine 38A is output to the outside of the module (for example, the image processing engine 38A When performing image analysis processing such as skew angle detection processing, a scan is performed instead of image data. The image analysis processing result such as the angle detection result may be output). However, the module library 36 has the same type and content of the image processing executed by the image processing engine 38A, and the unit processing described above. Image processing modules 38 having different data amounts, unit read data amounts, and unit write data amounts are also registered. For example, the unit processing data amount in the image processing module 38 that performs image rotation processing is not limited to the one image plane described above, and the same image rotation processing is performed and the unit processing data amounts are different from each other (for example, one image). A plurality of image processing modules 38 (for a line, a plurality of lines, etc.) may be included in the module library 36.

  The program of each image processing module 38 registered in the module library 36 is composed of a program corresponding to the image processing engine 38A and a program corresponding to the control unit 38B, but a program corresponding to the control unit 38B. The image processing module 38 having the same unit read data amount and unit write data amount among the individual image processing modules 38 is concerned with the type and content of image processing executed by the image processing engine 38A. However, the program corresponding to the control unit 38B is shared (the same program is used as the program corresponding to the control unit 38B). Thereby, the development load in developing the program of the image processing module 38 is reduced.

  In the image processing module 38, the unit read data amount and the unit write data amount are not fixed when the input image attribute is unknown, and the input image data attribute is acquired and acquired. There is a module in which the unit read data amount and the unit write data amount are determined by substituting the attribute into a predetermined arithmetic expression, and this type of image processing module 38 has a unit read data amount and For the image processing module 38 in which the unit write data amount is derived using the same arithmetic expression, a program corresponding to the control unit 38B may be shared. The image processing program group 34 according to the present embodiment can be mounted on various devices as described above, but the number and types of image processing modules 38 registered in the module library 36 in the image processing program group 34 are described. Needless to say, addition, deletion, replacement, and the like can be appropriately performed in accordance with image processing required by various devices that implement the image processing program group 34.

  In addition, as shown in FIG. 3B as an example, the individual buffer modules 40 constituting the image processing unit 50 include input / output of image data between the buffer 40A and modules upstream and downstream of the buffer module 40. The buffer control unit 40B manages the buffer 40A. The buffer 40A is composed of a memory area secured from the memory 14 provided in the computer 10 through the operating system 30 and the resource management unit 46B. The buffer control unit 40B of each buffer module 40 is also a program executed by the CPU 12, and the program of the buffer control unit 40B is also registered in the module library 36 (in FIG. 1, the program of the buffer control unit 40B is registered). "Buffer module")

  Further, the processing construction unit 42 that constructs the image processing unit 50 in accordance with an instruction from the application 32 includes a plurality of types of module generation units 44 as shown in FIG. The plural types of module generation units 44 support different image processing, and are activated by the application 32 to generate a module group including an image processing module 38 and a buffer module 40 for realizing the corresponding image processing. Perform the process. In FIG. 1, as an example of the module generation unit 44, the module generation unit 44 corresponding to the type of image processing executed by each image processing module 38 registered in the module library 36 is shown. The image processing corresponding to the generation unit 44 may be image processing realized by a plurality of types of image processing modules 38 (for example, skew correction processing including skew angle detection processing and image rotation processing). When the required image processing is processing combining a plurality of types of image processing, the application 32 sequentially activates the module generation unit 44 corresponding to any of the plurality of types of image processing. As a result, an image processing unit 50 that performs necessary image processing is constructed by the module generation unit 44 that is sequentially activated by the application 32.

  As shown in FIG. 1, the process management unit 46 includes a workflow management unit 46 </ b> A that controls execution of image processing in the image processing unit 50, a memory 14 by each module of the image processing unit 50, and various files of the computer 10. It includes a resource management unit 46B that manages the use of resources, and an error management unit 46C that manages errors that have occurred in the image processing unit 50. When an error occurs while the image processing unit 50 is executing image processing, the error management unit 46C acquires error information such as the type and location of the error that has occurred, and the image processing program group 34 is installed. The device environment information representing the type or configuration of the device in which the computer 10 is incorporated is acquired from the storage unit 20 or the like, and an error notification method according to the device environment represented by the acquired device environment information is determined and determined. Performs processing to notify the occurrence of an error using the error notification method.

  As a memory management method by the resource management unit 46B, for example, a first memory area allocated from the memory 14 to the request source module is secured through the operating system 30 for each request from each module of the image processing unit 50. Management method, a memory area of a certain size is secured in advance from the memory 14 through the operating system 30 (for example, when the computer 10 is turned on), and if there is a request from an individual module, one of the memory areas secured in advance A second management method for allocating a partial area to a requesting module, and a memory area of a certain size is secured in advance from the memory 14 through the operating system 30, and when there is a request from an individual module, the requested memory area Secure in advance if size is below threshold In the third management method, a part of the allocated memory area is allocated to the request source module, and if the size of the requested memory area is equal to or larger than the threshold, the memory area allocated to the request source module is secured through the operating system 30. Either can be adopted. Further, it may be possible to select and set which management method is used for memory management. The resource manager 46B corresponds to resource management means according to the present invention.

  Next, the operation of this embodiment will be described. In a device in which the image processing program group 34 is installed, when a situation where it is necessary to perform some kind of image processing, this situation is detected by a specific application 32. In addition, as a situation where it is necessary to perform image processing, for example, an image is read by an image reading unit as the image data supply unit 22, and is recorded as an image on a recording material by an image recording unit as the image output unit 24, or image output Whether the image is displayed on the display unit as the unit 24, the image data is written on the recording medium by the writing device as the image output unit 24, the image data is transmitted by the transmission unit as the image output unit 24, or the image output When the execution of a job to be stored in the image storage unit as the unit 24 is instructed by the user, or received by the reception unit as the image data supply unit 22 or stored in the image storage unit as the image data supply unit 22 For the image data being recorded, recording on the recording material, display on the display unit, writing to the recording medium, transmission, Any execution of a job for the storage of the image storage unit include when instructed by the user. In addition, the situation where image processing needs to be performed is not limited to the above. For example, in a state where the names of processes that can be executed by the application 32 in accordance with an instruction from the user are displayed in a list on the display unit 16, For example, the execution target process may be selected by the user.

  As described above, when detecting that it is necessary to perform some kind of image processing, the application 32 first recognizes the type of the image data supply unit 22 that supplies the image data to be processed, and recognizes the recognized type. Is a buffer area (a partial area of the memory 14 or the storage unit 20), a parameter for causing the buffer control unit 40B to recognize the buffer area designated as the image data supply unit 22 as the already secured buffer 40A. The buffer module 40 including the designated buffer area (the buffer module 40 functioning as the image data supply unit 22) is generated by generating a thread for executing the program of the buffer control unit 40B (generating the buffer control unit 40B). ) Is generated. Further, the buffer module 40 may be generated as a process or an object instead of a thread.

  Subsequently, in the same manner as described above, the application 32 recognizes the type of the image output unit 24 as an output destination of the image data subjected to image processing, and the recognized type is a buffer area (a partial area of the memory 14 or the storage unit 20). ), The buffer module 40 including the buffer area designated as the image output unit 24 is generated in the same manner as described above. The buffer module 40 generated here functions as the image output unit 24. Further, the application 32 recognizes the contents of the image processing to be executed, decomposes the image processing to be executed into a combination of image processing at a level corresponding to each module generation unit 44, and realizes the image processing to be executed. Therefore, the type of image processing necessary for the purpose and the execution order of the individual image processing are determined. In this determination, for example, the type of image processing and the execution order of the individual image processing are registered in advance in association with the type of job that can be instructed by the user, and the application 32 executes Can be realized by performing a process of reading information corresponding to the type of job instructed.

  Then, the application 32 activates the module generation unit 44 corresponding to the specific image processing based on the image processing type and execution order determined above (generates a process, a thread, or an object for executing the program of the module generation unit 44). ), The input module identification information for identifying the input module that inputs the image data to the module group as information necessary for generating the module group by the module generation unit 44. Output module identification information for identifying an output module from which the module group outputs image data, input image attribute information representing an attribute of input image data input to the module group, and parameters of image processing to be executed To generate a corresponding module group. In addition, when the required image processing is a combination of a plurality of types of image processing, the application 32 responds to individual image processing when notified of completion of generation of the module group from the instructed module generation unit 44. The process of starting the other module generation unit 44 and notifying the information necessary for generating the module group is repeated in ascending order of the execution order of the individual image processes.

  In the above input module, the image data supply unit 22 is the input module for the module group with the first execution order, and the last module (usually the buffer) for the module group with the second execution order or later. Module 40) is the input module. For the above output modules, the image output unit 24 is designated as the output module in the module group whose execution order is the last, so that the image output unit 24 is designated as the output module. Specification by the application 32 is not performed for confirmation, and it is generated and set by the module generation unit 44 when necessary. The input image attributes and image processing parameters are registered as information in advance in association with, for example, the type of job that the user can instruct execution, and information corresponding to the type of job instructed to execute is read out. Thus, the application 32 may be recognized, or the user may be designated.

  On the other hand, the module generation unit 44 performs module generation processing when activated by the application 32. In the module generation process, first, input image attribute information representing an attribute of input image data input to the generation target image processing module 38 is acquired. Note that the processing for acquiring the attributes of the input image data is performed in the case where the buffer module 40 exists in the previous stage of the generation target image processing module 38 and the image processing module in the previous stage that writes image data in the buffer module 40. This can be realized by acquiring the attributes of the output image data from the image data 38.

  Then, based on the attribute of the input image data represented by the acquired information, it is determined whether the generation of the image processing module 38 to be generated is necessary. For example, the module generation unit 44 is a module generation unit that generates a group of modules that perform color conversion processing. When the CMY color space is designated as the color space of the output image data by the image processing parameter from the application 32, the acquired input image When it is determined that the input image data is RGB color space data based on the attribute information, it is necessary to generate an image processing module 38 that performs RGB → CMY color space conversion as an image processing module 38 that performs color space processing. However, when the input image data is data in the CMY color space, the attribute of the input image data and the attribute of the output image data match with respect to the color space, so that the image processing module 38 that performs color space conversion processing generates Judge as unnecessary.

  When it is determined that the generation of the generation target image processing module 38 is necessary, it is determined whether the buffer module 40 is necessary after the generation target image processing module 38. This determination is made when the subsequent stage of the image processing module is the output module (image output unit 24) (see, for example, the last stage image processing module 38 in the image processing unit 50 shown in FIGS. 2A to 2C). As an example, like the image processing module 38 that performs skew angle detection processing in the image processing unit 50 shown in FIG. In the other cases, the determination is affirmed and the buffer control unit 40B is activated (a thread for executing the program of the buffer control unit 40B is generated). Thus, the buffer module 40 connected to the subsequent stage of the image processing module 38 is generated. Note that the buffer module 40 may be generated as a process or an object instead of a thread.

  Subsequently, information on the preceding module (for example, the buffer module 40), information on the succeeding buffer module 40 (only the image processing module 38 that generated the buffer module 40 in the succeeding stage), and input image data input to the image processing module 38 Attributes and processing parameters are given and registered in the module library 36. Among a plurality of candidate modules that can be used as the image processing module 38, the attributes of the input image data acquired earlier and the image processing module 38 The image processing module 38 that matches the processing parameter to be executed is selected and generated (a thread for executing the program of the image processing engine 38A and the control unit 38B is generated). Note that the image processing module 38 may be generated as a process or an object instead of a thread.

  For example, the module generation unit 44 is a module generation unit that generates a group of modules that perform color conversion processing. The CMY color space is designated as the color space of the output image data by the processing parameters, and the input image data is data in the RGB color space. If there is, an image processing module 38 that performs RGB → CMY color space conversion is selected and generated from a plurality of types of image processing modules 38 that perform various color space processing registered in the module library 36. The The image processing module 38 is an image processing module 38 that performs enlargement / reduction processing. If the designated enlargement / reduction ratio is other than 50%, the input image data is enlarged at the designated enlargement / reduction ratio. If the image processing module 38 that performs the reduction process is selected and generated and the designated enlargement / reduction ratio is 50%, the enlargement / reduction process specialized for the enlargement / reduction ratio of 50%, that is, the input image data is one pixel. An image processing module 38 that performs a reduction process to reduce to 50% by thinning out at intervals is selected and generated.

  The selection of the image processing module 38 is not limited to the above. For example, a plurality of image processing modules 38 having different unit processing data amounts in image processing by the image processing engine 38A are registered in the module library 36, and the image processing unit The image processing module 38 having an appropriate unit processing data amount is selected according to the operating environment such as the size of the memory area that can be allocated to 50 (for example, the image processing module 38 having a smaller unit processing data amount as the size decreases). Or the application 32 or the user may select it.

  When the generation of the image processing module 38 is completed, the workflow management unit 46A is notified of the set of the ID of the subsequent buffer module 40 and the ID of the generated image processing module 38. The ID may be information that can uniquely identify each module, and may be, for example, a number given in the order of generation of each module, an address of an object of the buffer module 40 or the image processing module 38 on a memory, or the like. Further, the module generation unit 44 performs image processing realized by a plurality of types of image processing modules 38 (for example, skew correction processing realized by the image processing module 38 that performs skew angle detection processing and the image processing module 38 that performs image rotation processing). When generating a module group for performing the above, the above process is repeated to generate a module group including two or more image processing modules 38. The module generation processing described above is sequentially performed by the individual module generation units 44 that are sequentially activated by the application 32, so that the necessary image processing can be performed as shown in FIGS. An image processing unit 50 to perform is constructed.

  In the present embodiment, when the execution frequency of specific image processing is high, the application 32 generates a plurality of types of module generation units 44 for generating the image processing unit 50 that performs specific image processing. Even after the image processing unit 50 that performs specific image processing is generated, it is left as a thread (may be a process or an object) by not instructing the end of processing, and each time it is necessary to perform specific image processing, It is also possible to regenerate the image processing unit 50 that performs specific image processing by sequentially instructing the remaining module generation units 44 to generate module groups. This eliminates the need to start each of the corresponding module generation units 44 every time specific image processing needs to be performed, thereby reducing the time required to regenerate the image processing unit 50 that performs specific image processing. can do.

  Incidentally, the control unit 38B of the image processing module 38 initializes the image processing module 38 when activated by the module generation unit 44. In this initialization, first, the information of the modules before and after the own module given from the module generation unit 44 is stored. Subsequently, the module preceding the own module is determined. No processing is performed when no module exists in the previous stage of the own module, but if the previous module is other than the buffer module 40, for example, the image data supply unit 22, a specific file, or the like, as necessary. The initialization process is performed. When the buffer module 40 exists in the previous stage of the own module, the data amount (unit read data amount) of the image data acquired by reading the image data from the previous buffer module 40 once is recognized. . This unit read data amount is only one if the number of buffer modules 40 in the previous stage of the own module is one, but for example, image processing for performing image composition processing in the image processing unit 50 shown in FIG. As in the case of the module 38, when there are a plurality of buffer modules 40 in the previous stage and the image processing engine 38A performs image processing using image data acquired from the plurality of buffer modules 40, the individual buffer modules in the previous stage The unit read data amount corresponding to 40 is determined according to the type and content of image processing performed by the image processing engine 38A of the own module, the number of buffer modules 40 in the previous stage, and the like. Then, the unit read data amount is set in all the buffer modules 40 existing in the preceding stage by notifying the recognized unit read data amount to all the buffer modules 40 existing in the preceding stage (FIG. 3 ( (See also A) (1)).

  Next, the module following the own module is determined. If the module subsequent to the own module is other than the buffer module 40, such as the image output unit 24 or a specific file, the initialization process is performed as necessary (for example, if the subsequent module is the image output unit 24, the unit For example, processing for notifying output of image data by the amount of data corresponding to the amount of write data). If the subsequent module is the buffer module 40, the amount of image data (unit write data amount) in one image data write is recognized, and the unit write data amount is set in the subsequent buffer module. (See also (2) in FIG. 3A). Then, the completion of initialization of the image processing module 38 is notified to the module generation unit 44, and the process is terminated.

  On the other hand, when the buffer control unit 40B of each buffer module 40 constituting the image processing unit 50 is activated by the module generation unit 44 or the application 32, the buffer module initialization process shown in FIG. 4 is performed. In this buffer module initialization process, first, in step 70, the unit write data amount is notified from the image processing module 38 in the previous stage of the own module, or the unit read data amount is notified from the image processing module 38 in the subsequent stage of the own module. Each time it is done, the notified unit write data amount or unit read data amount is stored (see also (1) and (2) in FIG. 3B). In step 72, it is determined whether the unit write data amount or the unit read data amount is notified from all the image processing modules 38 connected to the own module. If the determination is negative, the process returns to step 70, and steps 70 and 72 are repeated until the determination of step 72 is affirmed.

  When the unit write data amount or the unit read data amount is notified from all the image processing modules 38 connected to the own module and the determination in step 72 is affirmed, the process proceeds to step 74 and connected to the own module. Based on the unit write data amount and the unit read data amount set by each image processing module 38, the size of the unit buffer area, which is the management unit of the buffer 40A of the own module, is determined, and the determined unit buffer area Remember the size. As the size of the unit buffer area, the maximum value of the unit write data amount and the unit read data amount set in the own module is suitable. However, the unit write data amount may be set or the unit read data amount may be set. The amount of data (the maximum value of the unit read data amount set by each image processing module 38 when a plurality of image processing modules 38 are connected to the subsequent stage of the own module) may be set. The least common multiple of the write data amount and the unit read data amount (the maximum value thereof) may be set. If the least common multiple is less than the predetermined value, the least common multiple is set. A value (for example, any one of the above-mentioned unit write data amount and unit read data amount, the unit write data amount, and the unit read data amount (the maximum value)) may be set.

  In the next step 76, the amount of unit write data and the amount of unit read data are notified to the resource manager 46B as the size of the memory region to be secured, and the memory region used for writing (write buffer region: FIG. )) And a memory area used as a write buffer area and a read buffer by requesting the resource manager 46B to secure a memory area used for reading (read buffer area: see also FIG. 8B). Each memory area used as an area is secured by the resource manager 46B.

  Also, as shown in FIG. 3B, the workflow management unit 46A of the process management unit 46 according to the present embodiment has a memory area capacity (which can be allocated to each buffer module 40 for the purpose of use as the buffer 40A). (Allocable memory capacity) is held in a predetermined area on the memory 14. This allocatable memory capacity is, for example, when a thread or a process or object that executes a program of the workflow management unit 46A is generated and the workflow management unit 46A starts operating by executing the thread or process or object. The workflow management unit 46A obtains the total capacity of the memory 14 provided in the computer 10, calculates the allocatable memory capacity by substituting the obtained total capacity into a predetermined arithmetic expression, and calculates the calculated allocatable memory capacity. By performing the process of storing in this area, it is held in the predetermined area. The predetermined area holding the allocatable memory capacity corresponds to the holding means according to the present invention.

  In step 78, the allocatable memory capacity held in a predetermined area by the process management unit 46 (the workflow management unit 46A) is referred to, and in the next step 80, the allocatable memory capacity contains image data for one image. It is determined whether or not it is smaller than the memory capacity necessary for storage (corresponding to the upper limit capacity described in claim 2). If this determination is affirmative, it is determined that the capacity of the memory area that can be used as the buffer 40A is limited in the environment in which the image processing unit 50 operates (the memory 14 provided in the computer 10 is relatively small capacity). Therefore, the process proceeds to step 82, the allocatable memory capacity is set as the allocation request size, and the process proceeds to step 86. On the other hand, if the determination in step 80 is negative, there are plenty of memory areas that can be used as the buffer 40A in the environment where the image processing unit 50 operates (the memory 14 provided in the computer 10 is relatively small). Therefore, the process proceeds to step 84. In consideration of effective use of the memory 14, the memory capacity corresponding to the image data for one image is set as the allocation request size, and then the process proceeds to step 86. In steps 82 and 84, the allocation request size is equal to or smaller than an allocatable memory capacity or a memory capacity corresponding to image data for one image, and satisfies an integral multiple of the unit buffer area determined in the previous step 74. The maximum size may be set.

  In step 86, the allocation request size set in step 82 or 84 is notified to the resource manager 46B as the size of the memory area to be secured, and the resource manager 46B is requested to secure the memory area used as the buffer 40A. The memory area used as 40A is secured by the resource management unit 46B. In step 88, the memory area reserved for the buffer 40A is divided for each unit buffer area size determined in the previous step 74 (hereinafter, each divided memory area is referred to as a storage unit buffer area). At the same time, the valid data pointer corresponding to each image processing module 38 in the subsequent stage of its own module is initialized. The valid data pointer is image data (valid data) that has not been read by the corresponding succeeding image processing module 38 among the image data written in the buffer 40A of the own module by the preceding image processing module of the own module. Are pointers that respectively indicate the start position (next read start position) and the end position of the, and specific information that means that no valid data exists at the time of initialization is set.

  When the own module is generated by the application 32 and is the buffer module 40 functioning as the image data supply unit 22 or the image output unit 24, when the memory area used as the buffer 40A of the own module already exists. Of the above-described processes, the process of securing the memory area used as the buffer 40A is omitted. In addition, image data to be image-processed may already be written in the memory area that already exists. In this case, each image processing module in which the head position and the end position of the image data are in the subsequent stage Each is set to a valid data pointer corresponding to 38. With the above processing, the initialization of the buffer module 40 is completed, and the buffer control unit 40B notifies the completion of the buffer module initialization processing to the workflow management unit 46A.

  On the other hand, when the construction of the image processing unit 50 that performs the required image processing is completed by the above-described module generation processing being sequentially performed by the module generation unit 44 that is sequentially activated, the application 32 starts the program of the workflow management unit 46A. By starting a thread (or process or object) that executes the process, the image management unit 46A is instructed to execute image processing by the image processing unit 50.

  The workflow management unit 46A of the process management unit 46 performs the parallel control process shown in FIG. 10 when the program is started. In the parallel control processing, the workflow management unit 46A inputs processing requests to the individual image processing modules 38 constituting the image processing unit 50, so that the image processing by the image processing unit 50 is performed in parallel for each individual image processing module 38. However, in the following, prior to the description of the operation of the entire image processing unit 50, the processing after the completion of the initialization process performed by the buffer control unit 40B of each buffer module 40, the control unit 38B of each image processing module 38, The image processing module control process will be described in order.

  In this embodiment, when the image processing module 38 writes image data to the subsequent buffer module 40, a write request is input from the image processing module 38 to the buffer module 40, and the image processing module 38 receives the previous buffer module 40. When reading image data from the image processing module 38, a read request is input from the image processing module 38 to the buffer module 40. When some information including a write request from the previous image processing module 38 is input to the buffer module 40 (and a timer described later times out), the data writing process shown in FIG. 5 is performed by the buffer control unit 40B. Executed by. Note that the data writing process described below may be started by calling a function or method.

  In the data writing process, first, in step 100, it is determined whether or not the activation factor of the current data writing process is the activation due to the time-out of the timer. If the determination is negative, the process proceeds to step 104. If the determination is affirmative, the process proceeds to step 102, and the write request information input in the past and stored in the work memory or the like is read from the work memory or the like. Take out. In step 106, it is determined whether or not the buffer 40A of the own module is being accessed. Since data is also read from the buffer 40A, if the determination is affirmative, the routine proceeds to step 108, where the write request information to be processed at this time is stored in the work memory or the like, and in the next step 110. The timer is started and the data writing process is temporarily terminated.

  On the other hand, if the determination in step 106 is negative, the process proceeds to step 114, where a storage unit having a free area larger than the unit write data amount in the storage unit buffer area constituting the buffer 40A of the own module. It is determined whether or not there is a buffer area (a storage unit buffer area into which image data of the unit write data amount can be written). The image data is written sequentially to the storage unit buffer area constituting the buffer 40A, but the storage unit buffer area has a free area larger than the unit write data amount as the writing of the image data proceeds. When there is no more, the above determination is denied.

  If the determination in step 114 is negative, the process proceeds to step 115, the allocation request size previously set as the size of the memory area to be secured is notified to the resource management unit 46B, and the reservation of the memory area to be used as the buffer 40A is a resource. A request is made to the management unit 46B, and a memory area used as the buffer 40A is secured by the resource management unit 46B. In step 116, the memory area reserved for the buffer 40A is divided for each unit buffer area size determined previously, and the valid data pointers corresponding to the individual image processing modules 38 subsequent to the own module are updated. The process proceeds to step 118. If the determination in step 114 is affirmed, steps 115 and 116 are skipped and the process proceeds to step 118. In step 118, the write buffer area previously secured is used as the write area, and the start address of the write area is notified to the image processing module 38 that is the write request source, and the image data to be written is also notified. Request to write in order from the first address. As a result, the image processing module 38 of the write request source writes the image data in the write area (write buffer area (or storage unit buffer area: described later)) to which the start address is notified (FIG. 6B See also)).

  For example, when the size of the storage unit buffer area is not an integral multiple of the unit write data amount, the image data of the unit write data amount is repeatedly written to the storage unit buffer area, as an example, as shown in FIG. ), The size of the empty area in the storage unit buffer area with the empty area is smaller than the unit write data amount. In this case, the area in which the image data of the unit write data amount is written extends over a plurality of storage unit buffer areas. The plurality of storage unit buffer areas may be memory areas secured at different timings. In this case, it is not guaranteed that the plurality of storage unit buffer areas are continuous areas on the actual memory 14. On the other hand, in the present embodiment, the image processing module 38 writes the image data to the writing buffer area secured separately from the storage unit buffer area, and as shown in FIG. Since the image data once written in the write buffer area is copied to a single or a plurality of storage unit buffer areas, regardless of whether the area in which the image data is written spans a plurality of storage unit buffer areas As described above, the notification of the write area to the image processing module 38 that is the write request source only needs to be notified of the start address, and the interface with the image processing module 38 is simplified.

  If the own module is the buffer module 40 generated by the application 32, that is, if the memory area used as the buffer 40A has already been secured, step 114 to step 116 described above are skipped and step 118 is skipped. , The address of the memory area already secured is notified to the image processing module 38 as the address of the writing area, and the image data is written to the writing area.

  When the writing of the image data to the writing area by the preceding image processing module 38 is completed, the process proceeds to step 120, and the image data written in the writing buffer area is written in the storage unit buffer area. When the size of the empty area in the storage unit buffer area with the empty area is smaller than the unit write data amount, the image data written in the write buffer area is as shown in FIG. The data is written separately into a plurality of storage unit buffer areas. In the next step 122, the pointer indicating the end position of the effective data among the effective data pointers corresponding to the individual image processing modules 38 subsequent to the own module is used, and the end position of the effective data indicated by the pointer is the unit write data amount. The data is updated so as to move backward (see FIG. 6C), and the data writing process is terminated.

  Subsequently, a data read process executed by the buffer control unit 40B of the buffer module 40 when a read request is input from the subsequent image processing module 38 to the buffer module 40 (and a timer described later times out). This will be described with reference to FIG. Note that the data reading process described below may be started by calling a function or method.

  In the data reading process, first, in step 170, it is determined whether or not the activation factor of the current data reading process is an activation due to reception of a reading request from the subsequent image processing module. If the determination is negative, the process proceeds to step 174. If the above determination is affirmative, the process proceeds to step 172, and the read request information received this time from the subsequent image processing module is registered at the end of the read queue. To do. In step 174, it is determined whether or not the buffer 40A of the own module is being accessed. Since data is also written to the buffer 40A, if the determination is affirmative, the process proceeds to step 208, and it is determined whether or not the read request information is registered in the read queue. If the determination is negative, the data reading process is terminated. If the determination is affirmative, a timer is started at step 210 to end the data reading process once. When the timer is started, when the timer times out, the data read process is started again, and an unprocessed read request (information) registered in the read queue is taken out again. Processing is performed.

  On the other hand, if the determination in step 174 is negative, the process proceeds to step 176, and the read request information registered at the head is taken out from the read queue. In the next step 178, the image processing module 38 of the read request source is recognized based on the request source identification information included in the read request information extracted from the read queue, and is set by the image processing module 38 of the read request source. In addition to recognizing the unit read data amount, based on the valid data pointer corresponding to the image processing module 38 of the read request source, the head position of the valid data corresponding to the image processing module 38 of the read request source on the buffer 40A and Recognize the end position. In the next step 180, based on the start position and the end position of the valid data recognized in step 178, the valid data corresponding to the image processing module 38 of the read request source (the image that can be read by the image processing module 38 of the read request source). It is determined whether or not (data) exceeds the unit read data amount.

  If the determination in step 180 is negative, the process proceeds to step 182 and whether or not the end of valid data stored in the buffer 40A and readable by the image processing module 38 of the read request source is the end of the image data to be processed. judge. Valid data corresponding to the image processing module 38 of the read request source is stored in the buffer 40A at a unit read data amount or more, or valid data corresponding to the image processing module 38 of the read request source stored in the buffer 40A is stored. If it is less than the unit read data amount but the end of the valid data is the end of the image data to be processed, the determination in step 180 or step 182 is affirmed and the routine proceeds to step 186.

  In step 186, the effective data to be read is read from the buffer 40A by the amount of unit read data, and the read effective data is written in the read buffer area. In the next step 188, the start address of the read buffer area is notified to the read request source image processing module 38 as the start address of the read area, and the image data is requested to be read sequentially from the notified start address. As a result, the image processing module 38 of the read request source reads the image data from the read area (read buffer area (or storage unit buffer area: described later)) to which the head address is notified. When the effective data to be read is data corresponding to the end of the image data to be processed, the size of the image data to be read is requested at the end of the image data to be processed when the image data is requested to be read. This is also notified to the image processing module 38 of the reading request source. When the own module is the buffer module 40 generated by the application 32, the memory area (collection of storage unit buffer areas) used as the buffer 40A is a continuous area. The writing of image data to be read may be omitted, and the subsequent image processing module 38 may read the image data directly from the storage unit buffer area.

  As an example, as shown in FIG. 8A, the amount of effective data stored in the storage unit buffer area storing the image data of the head portion of the effective data is less than the unit read data amount, and the read When the target valid data extends over a plurality of storage unit buffer areas, the valid data to be read this time is not necessarily stored in a continuous area on the actual memory 14, but In the data reading process, as shown in FIGS. 8B and 8C, even in such a case, the image data to be read is once written in the reading buffer area, and then the image data is read from the reading buffer area. Therefore, regardless of whether image data to be read is stored across a plurality of storage unit buffer areas, the read area notification to the image processing module 38 of the read request source is as described above. It is only necessary to notify the head address to the image processing module 38, and the interface with the image processing module 38 is simplified.

  In the next step 190, it is determined whether reading of image data from the reading area by the image processing module 38 that is the reading request source is completed, and step 190 is repeated until the determination is affirmed. When the reading completion is notified from the image processing module 38 of the read request source, the determination in step 190 is affirmed and the process proceeds to step 194, and the valid data pointer of the valid data pointer corresponding to the image processing module 38 of the read request source is displayed. The pointer representing the head position is updated by moving the head position of the valid data pointed to by the pointer backward by the unit read data amount (see also FIG. 8C).

  In step 196, the valid data pointers corresponding to the individual image processing modules 38 in the subsequent stage are referred to, and the image data stored in the storage unit buffer area constituting the buffer 40A is updated by the pointer update in step 194. It is determined whether or not a storage unit buffer area that has been completely read by each image processing module 38 in the subsequent stage, that is, a storage unit buffer area that does not store valid data, has appeared. If the determination is negative, the process proceeds to step 208, and the data reading process is terminated through the above-described steps 208 and 210. If the determination is affirmative, the process proceeds to step 198, and the valid data is not stored. After the unit buffer area is released by the resource management unit 46B, the process proceeds to step 208, and the data read process is terminated through steps 208 and 210.

  On the other hand, the amount of valid data stored in the buffer 40A and readable by the image processing module 38 that is the read request source is less than the unit read data amount, and the end of the readable valid data is the image data to be processed. If it is not the end (when no valid data that can be read is detected in (4) of FIG. 3B), the determinations in steps 180 and 182 are respectively denied, and the process proceeds to step 200, where new image data is obtained. Is output to the workflow management unit 46A (see also (5) in FIG. 3B). In this case, the workflow management unit 46A inputs a processing request to the image processing module 38 in the previous stage of the own module. In step 202, the read request information taken out from the read queue is registered again in the original queue (at the head or tail), and the data read process is terminated through steps 208 and 210. Thus, until it is detected that the amount of valid data that can be read exceeds the unit read data amount or that the end of the valid data that can be read is the end of the image data to be processed (step 180 or step 182). The corresponding read request information is stored in the read queue and is periodically retrieved to repeatedly execute the requested process (until the determination is affirmed).

  Although details will be described later, when a data request is input from the buffer module 40, the workflow management unit 46A inputs the processing request to the image processing module 38 in the preceding stage of the buffer module 40 that is the data request source (see FIG. 3B). (See also (6)). When the processing performed by the control unit 38B of the preceding image processing module 38 using the input of the processing request as a trigger causes the preceding image processing module 38 to be able to write image data into the buffer module 40, the preceding image processing is performed. When the writing request is input from the module 38, the above-described data writing process (FIG. 4) is performed, and image data is written from the preceding image processing module 38 to the buffer 40A of the buffer module 40 (FIG. 3B). (See also (7) and (8)). As a result, the image data is read from the buffer 40A by the image processing module 38 in the subsequent stage (see also (9) in FIG. 3B).

  In the above-described data writing process and data reading process, exclusive control is performed so that while one is accessing the buffer 40A of its own module, the other stops accessing the buffer 40A. Thereby, even if the CPU 12 of the computer 10 executes processes or threads corresponding to the individual modules constituting the image processing unit 50 in parallel, a plurality of requests are input to the single buffer module 40 simultaneously or substantially simultaneously. Therefore, the CPU 12 of the computer 10 can execute processes or threads corresponding to individual modules in parallel. Of course, the buffer module may be realized as a normal program or object.

  Subsequently, each time a processing request is input from the workflow management unit 46A to the individual image processing modules 38 constituting the image processing unit 50, the image processing modules respectively performed by the control unit 38B of the individual image processing module 38. The control process (FIG. 9) will be described.

  In the image processing module control process, first, in step 220, when a module (the buffer module 40, the image data supply unit 22, the image processing module 38, etc.) exists in the previous stage of the own module, the module in the previous stage is processed. Data (image data or processing result of image processing such as analysis) is requested. In the next step 222, it is determined whether data can be acquired from the previous module. If the determination in step 222 is negative, it is determined in step 224 whether the end of the entire process has been notified. If the determination in step 224 is negative, the process returns to step 222, and steps 222 and 224 are repeated until data can be acquired from the preceding module. If the determination in step 222 is affirmed, in step 226, data acquisition processing for acquiring data from the previous module is performed.

  Here, if the previous module of the own module is the buffer module 40, when data is requested in the previous step 220 (read request), the readable valid data is transferred to the buffer 40A of the buffer module 40 in the unit read data amount. As long as the end of the valid data that has been stored or readable matches the end of the image data to be processed, the image processing module 38 in the previous stage of the buffer module 40 immediately returns to the state of the image data to be processed. After the image data is written in the buffer 40A of the buffer module 40, the state is changed to the above state, and then the start address of the reading area is notified from the buffer module 40 and the reading of the image data is requested. As a result, the determination in step 222 is affirmed, and the process proceeds to step 226. Data for reading image data of a unit read data amount (or a data amount less than that) from the read area in which the head address is notified from the previous buffer module 40 An acquisition process is performed (see also (3) in FIG. 3A).

  If the previous module of the own module is the image data supply unit 22, when the data request is output in the previous step 220, it is immediately notified from the previous image data supply unit 22 that the image data can be acquired. As a result, the determination at step 222 is affirmed and the routine proceeds to step 226, where image data acquisition processing for acquiring image data of a unit read data amount from the preceding image data supply unit 22 is performed. If the previous module of the own module is the image processing module 38, a data request (processing request) is output in the previous step 220. If the previous image processing module 38 is ready to perform image processing, the data request is written. Since it is notified that the data (image processing result) can be acquired by inputting the load request, the determination in step 222 is affirmed and the process proceeds to step 226. The data acquisition process for writing the data output from the image processing module 38 in the previous stage to the temporary storage memory area is performed by notifying the address of the temporary storage memory area where the data is written and requesting the writing.

  In the next step 228, it is determined whether or not a plurality of modules are connected to the previous stage of the own module. If the determination is negative, the process proceeds to step 232 without performing any processing. If the determination is affirmative, the process proceeds to step 230, and whether or not data has been acquired from all modules connected in the previous stage. To determine. If the determination in step 230 is negative, the process returns to step 220, and steps 220 to 230 are repeated until the determination in step 230 is positive. When all the data to be acquired from the previous module is obtained, the determination in step 228 is denied or the determination in step 230 is affirmed, and the process proceeds to step 232.

  In the next step 232, an area for data output is requested to the module subsequent to the module, and the determination is repeated until the data output area can be acquired in step 232 (until the start address of the data output area is notified). . If the subsequent module is the buffer module 40, the request for the data output area is made by outputting a write request to the buffer module 40. Once the data output area (the write area in which the start address is notified from the buffer module 40 if the subsequent module is the buffer module 40) can be obtained (see also (4) in FIG. 3A), the next step In 236, the data acquired in the previous data acquisition process and the data output area (first address) acquired from the subsequent module are input to the image processing engine 38A, and predetermined image processing is performed on the input data ( At the same time, the processed data is written in the data output area (see also (6) of FIG. 3A). When the input of the unit read data amount to the image processing engine 38A is completed and all the data output from the image processing engine 38A is written in the data output area, the subsequent step 238 indicates that the output has been completed. Notify the module.

  The processing (unit processing) for the data of the unit processing data amount in the image processing module 38 is completed by the above steps 220 to 238. However, in the processing request input from the workflow management unit 46A to the image processing module 38, the workflow management unit The number of executions of unit processing may be specified by 46A. Therefore, in step 240, it is determined whether the number of executions of the unit process has reached the number of executions instructed by the input processing request. If the number of executions of the instructed unit process is 1, this determination is unconditionally affirmed. However, if the number of executions of the instructed unit process is 2 or more, the process returns to Step 220, and the determination in Step 240 is performed. Steps 220 to 240 are repeated until affirmative. If the determination in step 240 is affirmed, the process proceeds to step 242 to output a processing completion notification to the workflow management unit 46A, thereby notifying the workflow management unit 46A that the processing corresponding to the input processing request has been completed. Then, the image processing module control process ends.

  Further, when the processing image data is processed to the end by repeating the above-described processing every time a processing request is input from the workflow management unit 46A, the end of the processing target image data is notified from the preceding module. Thus, the determination in step 224 is affirmed, and the process proceeds to step 244. The image data to be processed (note that the image data to be processed is often image data for one page, but is image data for a plurality of pages. An overall process end notification that means that the process for (which may be present) has ended is output to the workflow management unit 46A and the subsequent module. In the next step 246, a process for requesting the release of all acquired resources and deleting the own module is performed, and the image processing module control process is terminated.

  On the other hand, when the application 32 is instructed to execute image processing by the application 32, the workflow management unit 46A performs the parallel control processing 1 shown in FIG. As described above, in the input of processing requests to the individual image processing modules 38 of the image processing unit 50 by the workflow management unit 46A, the number of executions of unit processing can be specified. In step 500, the number of executions of unit processing specified by one processing request is determined for each image processing module 38. The number of executions of unit processing per processing request can be determined so that, for example, the number of processing requests input to individual image processing modules 38 during the processing of the entire image data to be processed is averaged. However, it may be determined according to other criteria. In the next step 504, a processing request is input to the last image processing module 38 in the image processing unit 50 (see also (1) in FIG. 11), and the parallel control processing 1 is terminated.

Here, in the image processing unit 50 shown in FIG. 10, when the process from the workflow management unit 46A to the image processing module 38 ₄ of the last stage request is input, the control unit 38B of the image processing module 38 ₄ preceding buffer module 40 _A read request is input to ₃ (see (2) in FIG. 11). At this time, since the image processing module 38 ₄ readable valid data in the buffer 40A of the buffer module 40 ₃ (image data) is not stored, the buffer controller 40B of the buffer module 40 ₃ data in the workflow management unit 46A A request is input (see (3) in FIG. 11).

The workflow management unit 46A performs the parallel control process 2 shown in FIG. 10B every time a data request is input from the buffer module 40. In the parallel control processing 2, in step 510, the image processing module 38 preceding the data request input source of the buffer module 40 (wherein the buffer module 40 ₃₎ (in this case the image processing module 38 ₃₎ recognizes, recognized front The processing request is input to the image processing module 38 (see (4) in FIG. 11), and the processing ends.

Control unit 38B of the image processing module 38 _3, the processing request is input to the input read request preceding the buffer module 40 ₂ (see (5) in FIG. 11), read in the buffer 40A of the buffer module 40 ₂ since possible image data is not stored, the buffer controller 40B of the buffer module 40 ₂ inputs the data request to the workflow management unit 46A (see (6) in FIG. 11). Workflow management unit 46A, even if the data request is input from the buffer module 40 _2, by performing the parallel control processing 2 described above again, enter the processing request to the image processing module 38 ₂ of the preceding stage (Figure 11 (7)), the control unit 38B of the image processing module 38 ₃ inputs the read request to the buffer module 40 ₁ of the front reference ((8 in FIG. 11)). Further, since the readable image data to the buffer 40A of the buffer module 40 ₁ is not stored, the buffer controller 40B of the buffer module 40 ₁ also inputs the data request to the workflow management unit 46A (in FIG. 11 (9) reference). Even when a data request is input from the buffer module 40 ₁ , the workflow management unit 46A performs the above-described parallel control processing 2 again to input a processing request to the preceding image processing module 38 ₁ (FIG. 11). (See (10)).

Here, since the previous module of the image processing module 38 ₁ is the image data supply unit 22, the control unit 38 B of the image processing module 38 ₁ inputs a data request to the image data supply unit 22, and thereby the image data supply unit 22. The image data of the unit read data amount is acquired from the image data 22 (see (11) in FIG. 11), and the image data obtained by the image processing engine 38A performing image processing on the acquired image data is stored in the subsequent buffer. write module 40 _first buffer 40A (see (12) in FIG. 11).

Further, the buffer control unit 40B of the buffer module 40 ₁ requests the image processing module 38 ₂ to read when valid data exceeding the unit read data amount that can be read by the subsequent image processing module 38 ₂ is written. the image processing module 38 _{and second} control unit 38B with the reads image data of the unit read data amount from the buffer module 40 ₁ of buffer 40A (see (13) in FIG. 11), the image processing engine on the acquired image data 38A is an image data obtained by performing image processing, written in the following buffer module 40 _second buffer 40A (see (14) in FIG. 11). The buffer control unit 40B of the buffer module 40 ₂ requests the image processing module 38 ₃ to read when valid data exceeding the unit read data amount that can be read by the subsequent image processing module 38 ₃ is written, and the image processing module 38 _3. the control unit 38B of the (see (15) in FIG. 11) reads the image data of the unit read data amount from the buffer module 40 ₂ of the buffer 40A, the image processing engine 38A on the acquired image data to perform image processing the image data obtained by, written to the subsequent buffer module 40 _third buffer 40A (see (16) in FIG. 11).

Furthermore, the buffer controller 40B of the buffer module 40 ₃ may request a read to the image processing module 38 ₄ When the subsequent image processing module 38 ₄ readable unit read data amount or more valid data is written, which control unit 38B of the image processing module 38 ₄ with the reads the image data of the unit read data amount from the buffer module 40 _third buffer 40A (see (17) in FIG. 11), the image processing engine on the acquired image data The image data obtained by the image processing performed by 38A is output to the image output unit 24, which is a subsequent module (see (18) in FIG. 11).

  In addition, when the control unit 38B of each image processing module 38 completes the writing of the image data to the buffer 40A of the subsequent buffer module 40, the control unit 38B inputs a processing completion notification to the workflow management unit 46A. The workflow management unit 46A performs the parallel control process 3 shown in FIG. 10C every time a processing completion notification is input from the image processing module 38. In this parallel control processing 3, in step 520, the processing request is input again to the image processing module 38 that is the processing completion notification source, and the processing ends.

  In this way, in the parallel control processing by the workflow management unit 46A, every time processing completion is notified from any image processing module 38, the processing request is input again to the image processing module 38 that is the processing completion notification source, thereby Parallel processing in which target image data is sequentially transferred from a module on the front side to a module on the back side in units of a size (block) smaller than one image, and individual image processing modules 38 perform image processing in parallel with each other. Image processing is performed on the image data to be processed according to the method. When the image data supplied from the image data supply unit 22 reaches the end of the image data to be processed, the input of the overall processing end notification from the individual image processing module 38 to the workflow management unit 46A is sent to the preceding image. The processing modules 38 are sequentially performed.

  The workflow management unit 46A performs the parallel control process 4 shown in FIG. 10D every time an overall process end notification is input from the image processing module 38. In this parallel control process 4, in step 540, it is determined whether the image processing module 38 that is the input source of the overall process end notification is the last-stage image processing module 38. If the determination is negative, the process ends without performing any process, but all the image data that has undergone the necessary image processing on the image data to be processed is output to the image output unit 24. When an overall process end notification is input from the last-stage image processing module 38, the determination in step 540 is affirmed and the process proceeds to step 542 to notify the application 32 of the completion of the image processing and perform parallel control processing. 4 is finished. The application 32 notified of the completion of the image processing notifies the user of the completion of the image processing.

  While the image processing unit 50 is performing image processing as described above, each buffer module 40 (corresponding to the thread) constituting the image processing unit 50 is a memory used as the buffer 40A as described above. The acquisition (allocation request) and release of the area are repeated for the resource management unit 46B. Here, when the memory 14 has a relatively small capacity and the capacity of the memory area usable as the buffer 40A is limited, a relatively small value is set as the allocatable memory capacity held in the process management unit 46, Since the allocation request size of each buffer module 40 is also set to a relatively small value (= allocable memory capacity), each buffer module 40 allocates a memory area to be used as the buffer 40A in units of a relatively small capacity. Request. Therefore, in this case, the allocation of memory areas is frequently requested from the individual buffer modules 40 to the resource management unit 46B, which increases the overhead and decreases the processing efficiency. Since the average capacity of the memory area secured and held as 40A is reduced, the usage amount of the memory 14 is reduced and the memory 14 can be used effectively.

  On the other hand, when the memory 14 has a relatively large capacity and a sufficient amount of memory area can be used as the buffer 40A, a relatively large value is set as the allocatable memory capacity held in the process management unit 46. The allocation request size of the buffer module 40 is also set to a relatively large value (= memory capacity corresponding to image data for one image), so that each buffer module 40 has a buffer with a relatively large capacity as a unit. Requests allocation of a memory area used as 40A. Therefore, in this case, although the amount of use of the memory 14 increases because the average capacity of the memory area secured and held by the individual buffer module 40 as the buffer 40A increases, the individual buffer module 40B uses the individual buffer. By reducing the frequency at which the memory area is requested from the module 40, the overhead can be reduced and the processing efficiency can be improved. Accordingly, it is possible to switch whether priority is given to reducing the usage amount of the memory 14 or improvement of the processing efficiency of image processing according to the operating environment (capacity of the memory 14 or the like).

  In the first embodiment, it has been described that the memory area used as the buffer 40A is secured (allocation request) in units of allocation request size, and the memory area used as the buffer 40A is released in units of unit buffer areas. The memory area may also be released in units of allocation request size. In this case, although there is a possibility that the average capacity of the memory area secured and held by each buffer module 40 as the buffer 40A may increase, the overhead can be further reduced.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. Since the second embodiment has the same configuration as that of the first embodiment, the same reference numerals are given to the respective parts and description of the configuration is omitted, and the operation of the second embodiment will be described. Only different parts will be described.

  When activated by the module generation unit 44, the control unit 38B of the image processing module 38 according to the second embodiment performs an image processing module initialization process illustrated in FIG. In this image processing module initialization process, as in the first embodiment, first, information on the modules before and after the own module is stored (step 600), the modules before and after the own module are determined, and unit reading is performed. The data amount and the unit write data amount are recognized (step 602).

  In step 604, the process management unit 46 (the workflow management unit 46A) refers to and recognizes the allocatable memory capacity held in a predetermined area. In the next step 606, the recognized allocatable memory capacity is determined in step 602. The number of repetitions N is obtained by dividing by the recognized unit write data amount. When a remainder (fraction) is generated by the above division, rounding, rounding down, or rounding up is performed to make the number of repetitions N an integer. In the above division, the unit read data amount may be used instead of the unit write data amount, or an average value of the unit write data amount and the unit read data amount may be used. In step 608, a value obtained by multiplying the unit write data amount recognized in step 602 by the number of repetitions N obtained in step 606 is set as the write data amount. In step 610, the write data amount (data amount N times the unit write data amount) obtained in step 608 is set as the unit write data amount for the buffer module 40 existing in the subsequent stage.

  In step 612, it is determined whether or not the value obtained by multiplying the unit read data amount recognized in step 602 by the number of repetitions N obtained in step 606 is smaller than the memory capacity corresponding to the image data for one image. . If the determination is affirmative, the routine proceeds to step 614, where a value obtained by multiplying the unit read data amount by the number of repetitions N is set as the read data amount. If the determination in step 612 is negative, the process proceeds to step 616, and a memory capacity corresponding to image data for one image is set as the read data amount. In step 618, the read data amount obtained in step 614 or 616 is set as the unit read data amount for the buffer module 40 existing in the preceding stage, and the completion of the image processing module initialization processing is set as the module generation unit 44. To terminate the process.

  In the first embodiment described above, the buffer module 40 uses the smaller value of the allocatable memory capacity and the memory capacity corresponding to the image data for one image as the allocation request size. The buffer module 40 according to the embodiment sets the size of the unit buffer area determined based on the unit write data amount and the unit read data amount as the allocation request size. However, the unit write data amount notified to the buffer module 40 in the second embodiment is actually the original unit write data amount (the image output from the image processing engine 38A at a time as described above). The unit read data amount notified to the buffer module 40 is actually processed at a time by the original unit read data amount (by the image processing engine 38A). Since the image data amount is multiplied by the number of repetitions N, a capacity approximately N times the original unit write data amount and unit read data amount is set as the allocation request size. The number of repetitions N is obtained by dividing the allocatable memory capacity by the unit write data amount or the like.

  Thereby, also in the second embodiment, the memory 14 has a relatively small capacity and the capacity of the memory area that can be used as the buffer 40A is limited. When the relatively small value is set, the number N of repetitions becomes 1 or a small value close to 1, so that the allocation request size of each buffer module 40 is also set to a relatively small value. Requests allocation of a memory area to be used as the buffer 40A with a relatively small capacity as a unit. Therefore, in this case, the allocation of memory areas is frequently requested from the individual buffer modules 40 to the resource management unit 46B, which increases the overhead and decreases the processing efficiency. Since the average capacity of the memory area secured and held as 40A is reduced, the usage amount of the memory 14 is reduced and the memory 14 can be used effectively.

  On the other hand, since the memory 14 has a relatively large capacity and plenty of memory areas that can be used as the buffer 40A, a relatively large value is set as the allocatable memory capacity held in the process management unit 46. In this case, since the number of repetitions N is a relatively large value, the allocation request size of each buffer module 40 is also set to a relatively large value, and each buffer module 40 uses the buffer 40A as a unit with a relatively large capacity. Requests the allocation of the memory area used as Therefore, in this case, although the amount of use of the memory 14 increases because the average capacity of the memory area secured and held by the individual buffer module 40 as the buffer 40A increases, the individual buffer module 40B uses the individual buffer. By reducing the frequency at which the memory area is requested from the module 40, the overhead can be reduced and the processing efficiency can be improved. Therefore, also in the second embodiment, it is possible to switch between giving priority to reducing the usage amount of the memory 14 and giving priority to improving the processing efficiency of image processing according to the operating environment (capacity of the memory 14 and the like).

  Next, the image processing module control processing according to the second embodiment will be described with reference to FIG. 13, and the difference from the image processing module control processing (FIG. 9) described in the first embodiment. In the image processing module control process according to the second embodiment, data is acquired from the previous module in step 226 as in the first embodiment. If the previous module is the buffer module 40, the process proceeds to the buffer module 40. Since the unit read data amount notified in advance is a value N times the original unit read data amount, image data having a data amount N times the original unit read data amount is also acquired in step 226. In step 232, as in the first embodiment, a data output area is requested to the succeeding module. If the succeeding module is the buffer module 40, the unit write data notified to the buffer module 40 in advance. Since the amount is N times the original unit write data amount, a data output area having a capacity capable of writing image data having a data amount N times the original unit write data amount based on the request. Is secured.

  In the image processing module control process according to the second embodiment, the image processing engine 38A causes the image processing engine 38A to perform image processing in the same manner as in the first embodiment in step 236, and the image processing engine 38A performs the processing of the image data acquired in step 226. A predetermined image processing is performed on the image data corresponding to the unit read data amount, and the image data corresponding to the unit write data amount obtained by the image processing is stored in the data output area secured by the processing in steps 232 and 234. In the next step 237, it is determined whether or not the image processing (step 236) by the image processing engine 38A has been repeated N times. If the determination is negative, the process returns to step 236, and steps 236 and 237 are repeated until the determination in step 237 is affirmed. If the determination in step 237 is affirmed, the process proceeds to step 238 to notify the subsequent module that the output of the image data has been completed.

  As a result, the unit read data amount or unit write data amount (original unit read data amount or unit write) notified from the image processing module 38 is transferred between the image processing module 38 and the preceding and subsequent buffer modules 40. On the other hand, the image data is delivered in units of image data having a data amount N times the data amount). On the other hand, in the image processing module 38, the image processing engine 38A is used in units of the original unit read data amount and unit write data amount. The image processing by is sequentially performed.

In the second embodiment, the mode in which the function as the control unit of the image processing module 38 is realized by the single control unit 38B has been described. However, the present invention is not limited to this and is illustrated as an example. as shown in 14, the control unit 38B separates the first control unit 38B ₁ to the second controller 38B _2, the first control unit 38B _1, similarly to the control unit 38B described in the first embodiment, front While the image data is read from the module in units of the unit read data amount and the image data is output to the subsequent module in units of the unit write data amount, the second control unit 38B ₂ calculates the number of repetitions N, the read data of N times the unit read data amount received from the first control unit 38B ₁ set to the front stage of the buffer module 40, image de from the preceding buffer module 40 Reads the data by reading the data volume, the read image data with passes to the first control unit 38B ₁ by the unit read data amount via the virtual buffer, the unit write data amount received via virtual buffer from the first control unit 38B ₁ the write data amount of N times set to the subsequent buffer module 40, and outputs the image data outputted from the first control unit 38B ₁ by the unit write data amount to the subsequent buffer module 40 after N times accumulation You may comprise as follows. In this embodiment, although will be first a control unit 38B ₁ functions as the control means of the image processing module 38 by the second controller 38B ₂ cooperate is achieved, the control means first controller 38B ₁ and by separating the second controller 38B _2, the programs corresponding to the first control unit 38B _1, for example, the common unit read data amount and the unit write data amount the same image processing module 38 as a unit while reduction may be about the program corresponding to the second control unit 38B ₂ can be common for all of the image processing module 38, to further reduce the development load program. The above aspect corresponds to the invention described in claim 4.

  In the above description, the mode in which the allocatable memory capacity held in the workflow management unit 46A of the process management unit 46 is set according to the total capacity of the memory 14 has been described. However, the present invention is not limited to this. The allocatable memory capacity may be set in consideration of the processing capacity (clock frequency, etc.) of the CPU 12 executing the program in addition to the total capacity of 14, or other parameters.

  Also, the information held in the workflow management unit 46A is not limited to the above-mentioned allocatable memory capacity, and the rules for allocating memory to individual buffer modules 40 input by the user via the operation unit 18 or the like. (For example, information for selecting the capacity of the memory area allocated to each buffer module 40 from options such as one line / n lines / one image). In this case, the actual allocatable memory capacity may change according to the total capacity of the memory 14 and the like, and when the actual allocatable memory capacity is indeterminate when referring to the held memory allocation rule, The allocatable memory capacity may be calculated based on the total capacity of the memory 14 or the like. The above aspect corresponds to the invention described in claim 5.

  Further, in the above description, the image processing program group 34 corresponding to the image processing program according to the present invention has been stored (installed) in the computer 10 in advance. However, the image processing program according to the present invention may be a CD-ROM, It is also possible to provide in a form recorded on a recording medium such as a DVD-ROM.

It is a block diagram which shows schematic structure of the computer (image processing apparatus) which concerns on this embodiment. It is a block diagram which shows the structural example of an image process part. (A) is an image processing module, and (B) is a block diagram showing a schematic configuration of a buffer module and processing to be executed. It is a flowchart which shows the content of the buffer module initialization process performed by the buffer control part of a buffer module. It is a flowchart which shows the content of the data writing process performed by the buffer control part of a buffer module. It is the schematic explaining the process in case the image data of writing object straddles several storage unit buffer area | regions. It is a flowchart which shows the content of the data reading process performed by the buffer control part of a buffer module. It is the schematic explaining the process in case the image data of read object straddles several storage unit buffer area | regions. It is a flowchart which shows the content of the image processing module control process performed by the control part of an image processing module. It is a flowchart which shows the content of the parallel control processes 1-4 performed by a process management part. It is the schematic explaining the flow of the image processing in an image process part. It is a flowchart which shows the content of the image processing module initialization process performed by the control part of the image processing module which concerns on 2nd Embodiment. It is a flowchart which shows the content of the image processing module control process performed by the control part of the image processing module which concerns on 2nd Embodiment. It is a block diagram which shows the other structure of an image processing module.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Computer 14 Memory 18 Operation part 38 Image processing module 38A Image processing engine 38B Control part 40 Buffer module 40A Buffer 40B Buffer control part 46 Process management part 46A Workflow management part 50 Image processing part

Claims (7)

A plurality of functions having the function of acquiring image data from the previous stage of the own module, performing predetermined image processing on the acquired image data, and outputting the image data that has undergone the image processing or the processing result of the image processing to the subsequent stage of the own module Requests at least one of the preceding and following stages of the image processing module to allocate storage resources each time the storage resources used as a buffer are insufficient, and outputs the output from the module before the own module using the allocated storage resources as a buffer. Each module is connected to the pipeline so that the image data stored in the buffer is written into the buffer, and the buffer modules that perform the process of reading the image data stored in the buffer by the module subsequent to the module are connected. Images connected in a form or directed acyclic graph form And the processing section,
Storage resource management means for allocating a storage resource of the requested capacity to the requesting buffer module each time storage resource allocation is requested from the buffer module;
Information indicating the capacity of storage resources that can be allocated to individual buffer modules, or the rules for allocating storage resources to individual buffer modules that are set according to the capacity of storage resources that can be allocated to individual buffer modules Holding means for holding information;
Including
Control means for switching the capacity of the storage resource requested by the storage resource management means in response to the storage resource allocation request to the storage resource management means based on the information stored in the storage means by each of the buffer modules An image processing apparatus comprising:   The control means of each of the buffer modules recognizes the capacity of the storage resource that can be allocated to each buffer module based on the information stored in the holding means, and among the recognized capacity and a predetermined upper limit capacity 2. The image processing apparatus according to claim 1, wherein the smaller one is set as a storage resource capacity requested to the storage resource management unit in a storage resource allocation request to the storage resource management unit. An image processing engine that performs image processing on image data for each unit processing data amount that is set in advance, and image data that the image processing engine performs image processing from the previous stage of the own module, and image processing by the image processing engine The allocated storage resource is used as a buffer in at least one of the preceding stage and the succeeding stage of the plurality of image processing modules provided with control means for outputting the processed image data or the processing result of the image processing to the succeeding stage of the own module. Each time image data output from the module preceding the module is written to the buffer, the image data stored in the buffer is read by the module subsequent to the module, and the storage resource used as the buffer is insufficient Depends on the module before the module Processing for requesting allocation of a storage module having a capacity corresponding to the amount of image data written to the buffer by one writing and the amount of image data read from the buffer by one reading by a module subsequent to the own module An image processing unit configured by connecting individual modules in a pipeline form or a directed acyclic graph form so that the buffer modules are connected to each other;
Storage resource management means for allocating a storage resource of the requested capacity to the requesting buffer module each time storage resource allocation is requested from the buffer module;
Information indicating the capacity of storage resources that can be allocated to individual buffer modules, or the rules for allocating storage resources to individual buffer modules that are set according to the capacity of storage resources that can be allocated to individual buffer modules Holding means for holding information;
Including
The control means of each of the image processing modules sets the number N of image processing repetitions based on the information stored in the holding means, and automatically sets the unit image data amount by multiplying N by the unit data amount. Acquire image data from the previous stage of the module and output the image data to the subsequent stage of the module, and repeat the image processing for each unit processing data amount by the image processing engine N times, so that An image processing apparatus that performs image processing on image data.   The control means of each of the image processing modules acquires the image processing engine and image data that the image processing engine performs image processing for each unit processing data amount, and image data that has undergone image processing by the image processing engine Alternatively, in the image processing module having the first control means for outputting the processing result of the image processing, the number N of image processing repetitions is set based on the information stored in the holding means, and the unit processing data The image data amount obtained by multiplying the amount by N as a unit is acquired as the unit, and the acquired image data is transferred to the first control means in increments of the unit processing data amount, and the image processing engine Image data output from the first control means or image processing results are stored N times by performing image processing. 4. The image processing apparatus according to claim 3, wherein a second control unit that outputs to a subsequent stage of the module after the second control unit is added and the first control unit and the second control unit cooperate with each other. .   The allocation rule of the storage resource to the individual buffer module is input by the user through the input unit, and the holding unit holds information representing the allocation rule input by the user through the input unit. The image processing apparatus according to claim 1 or claim 3, wherein Computer
A plurality of functions having the function of acquiring image data from the previous stage of the own module, performing predetermined image processing on the acquired image data, and outputting the image data that has undergone the image processing or the processing result of the image processing to the subsequent stage of the own module Requests at least one of the preceding and following stages of the image processing module to allocate storage resources each time the storage resources used as a buffer are insufficient, and outputs the output from the module before the own module using the allocated storage resources as a buffer. Each module is connected to the pipeline so that the image data stored in the buffer is written into the buffer, and the buffer modules that perform the process of reading the image data stored in the buffer by the module subsequent to the module are connected. Images connected in a form or directed acyclic graph form Processing section,
A storage resource management means for allocating a storage resource having a requested capacity to a requesting buffer module each time storage resource allocation is requested from the buffer module;
Information indicating the capacity of storage resources that can be allocated to individual buffer modules, or rules for allocating storage resources to individual buffer modules that are set according to the capacity of storage resources that can be allocated to individual buffer modules Function as a holding means to hold information representing
Control means for switching the capacity of the storage resource requested by the storage resource management means in response to the storage resource allocation request to the storage resource management means based on the information stored in the storage means by each of the buffer modules An image processing program comprising: Computer
An image processing engine that performs image processing on image data for each unit processing data amount that is set in advance, and image data that the image processing engine performs image processing from the previous stage of the own module, and image processing by the image processing engine The allocated storage resource is used as a buffer in at least one of the preceding stage and the succeeding stage of the plurality of image processing modules provided with control means for outputting the processed image data or the processing result of the image processing to the succeeding stage of the own module. Each time image data output from the module preceding the module is written to the buffer, the image data stored in the buffer is read by the module subsequent to the module, and the storage resource used as the buffer is insufficient Depends on the module before the module Processing for requesting allocation of a storage module having a capacity corresponding to the amount of image data written to the buffer by one writing and the amount of image data read from the buffer by one reading by a module subsequent to the own module An image processing unit configured such that individual modules are connected in a pipeline form or a directed acyclic graph form so that the buffer modules are connected to each other,
A storage resource management means for allocating a storage resource having a requested capacity to a requesting buffer module each time storage resource allocation is requested from the buffer module;
Information indicating the capacity of storage resources that can be allocated to individual buffer modules, or rules for allocating storage resources to individual buffer modules that are set according to the capacity of storage resources that can be allocated to individual buffer modules Function as a holding means to hold information representing
The control means of each of the image processing modules sets the number N of image processing repetitions based on the information stored in the holding means, and automatically sets the unit image data amount by multiplying N by the unit data amount. Acquire image data from the previous stage of the module and output the image data to the subsequent stage of the module, and repeat the image processing for each unit processing data amount by the image processing engine N times, so that An image processing program for performing image processing on image data.

Images