CN110572527B - Image processing apparatus, control method thereof, and computer-readable storage medium - Google Patents

Image processing apparatus, control method thereof, and computer-readable storage medium Download PDF

Info

Publication number
CN110572527B
CN110572527B CN201910405589.6A CN201910405589A CN110572527B CN 110572527 B CN110572527 B CN 110572527B CN 201910405589 A CN201910405589 A CN 201910405589A CN 110572527 B CN110572527 B CN 110572527B
Authority
CN
China
Prior art keywords
image processing
application
memory
memory usage
plug
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910405589.6A
Other languages
Chinese (zh)
Other versions
CN110572527A (en
Inventor
佐佐木祐人
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Publication of CN110572527A publication Critical patent/CN110572527A/en
Application granted granted Critical
Publication of CN110572527B publication Critical patent/CN110572527B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00962—Input arrangements for operating instructions or parameters, e.g. updating internal software
    • H04N1/00965—Input arrangements for operating instructions or parameters, e.g. updating internal software using a plug-in memory module, e.g. memory card, memory stick
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/12—Digital output to print unit, e.g. line printer, chain printer
    • G06F3/1201—Dedicated interfaces to print systems
    • G06F3/1202—Dedicated interfaces to print systems specifically adapted to achieve a particular effect
    • G06F3/121—Facilitating exception or error detection and recovery, e.g. fault, media or consumables depleted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/12—Digital output to print unit, e.g. line printer, chain printer
    • G06F3/1201—Dedicated interfaces to print systems
    • G06F3/1202—Dedicated interfaces to print systems specifically adapted to achieve a particular effect
    • G06F3/1218—Reducing or saving of used resources, e.g. avoiding waste of consumables or improving usage of hardware resources
    • G06F3/122—Reducing or saving of used resources, e.g. avoiding waste of consumables or improving usage of hardware resources with regard to computing resources, e.g. memory, CPU
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/12—Digital output to print unit, e.g. line printer, chain printer
    • G06F3/1201—Dedicated interfaces to print systems
    • G06F3/1223—Dedicated interfaces to print systems specifically adapted to use a particular technique
    • G06F3/1224—Client or server resources management
    • G06F3/1225—Software update, e.g. print driver, modules, plug-ins, fonts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
    • G06F9/44526—Plug-ins; Add-ons
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00002—Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for
    • H04N1/00071—Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for characterised by the action taken
    • H04N1/00074—Indicating or reporting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00912—Arrangements for controlling a still picture apparatus or components thereof not otherwise provided for
    • H04N1/00938—Software related arrangements, e.g. loading applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00962—Input arrangements for operating instructions or parameters, e.g. updating internal software
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Facsimiles In General (AREA)
  • Stored Programmes (AREA)

Abstract

The invention provides an image processing apparatus, a control method thereof and a computer-readable storage medium. The image processing apparatus compares a total memory usage amount obtained from the obtained memory usage amounts of the respective image processes and the memory usage amount when the requested new image process is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in executing the plug-in application; and in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, new image processing is performed, and in a case where the total memory usage exceeds the allowable memory capacity, an error indicating that the new image processing cannot be performed due to a shortage of memory is notified to the request source without performing the new image processing.

Description

Image processing apparatus, control method thereof, and computer-readable storage medium
Technical Field
The invention relates to an image processing apparatus, a control method thereof, and a computer-readable storage medium.
Background
Conventionally, it is known that when various jobs including image processing not provided by an image processing apparatus are executed, a job execution application including new image processing is created as a plug-in and installed in the image processing apparatus to implement the new image processing. The various jobs are, for example, device control processing including image processing executable on the image processing apparatus, such as copying, printing, scanning and saving, and scanning and transmitting. The new image processing is processing that cannot be implemented by an existing job in the image processing apparatus, such as skew correction, format conversion, or translation after OCR (optical feature recognition) for input data.
The image processing apparatus to which the plurality of new image processes are added can execute the added new image processes in parallel. However, when a plurality of image processes using many memories are simultaneously performed, the memory usage amount may exceed the upper limit of the memory of the image processing apparatus.
When the memory usage exceeds the upper limit of the memory, the OS performs a swap operation to temporarily save data in a memory area with a low frequency of use in the hard disk, release the area, and write back the data to the memory if necessary. At this time, the write process is performed on the HDD, and the performance of various processes may be degraded in the process. For example, when a UI module exists in the process, a phenomenon may occur in which the UI screen does not respond quickly to the user operation. The following phenomena may also occur: the time until the completion of the plurality of image processes performed in the same process becomes longer than the time taken when the exchanging operation is not performed. If the contiguous switching area is exhausted, the OS forces termination of the process using the large amount of memory resources. In this case, the process for executing the image processing ends, so the image processing cannot be restarted unless the image processing apparatus is restarted.
In order to solve this problem, Japanese patent application laid-open No. 2013-120103 proposes the following technique: the memory usage of each card is measured in real time and the card is not started when it exceeds a predetermined maximum memory capacity.
However, the above-described prior art has problems to be described below. In this related art, for example, at the time of starting up a plug-in, the sum of the declared memory usage amount defined for each plug-in and the usage amount of the memory already used in the image processing apparatus is calculated. If the total memory usage exceeds a predetermined maximum memory capacity, the memory starvation as described above is avoided by not activating the plug-in itself. In the above-described control, when the calculated memory usage exceeds the maximum memory capacity, a new card cannot be started unless the card that has been started is stopped. Even a plug-in that can actually be executed cannot be started for processing unless executed in parallel. This impairs user convenience.
Disclosure of Invention
The present invention can implement the following mechanism: the total memory usage amount is dynamically calculated during the execution of the process (runtime), and a plurality of image processes are executed in parallel while appropriately avoiding memory shortage, without restricting the startup of the plug-in itself.
An aspect of the present invention provides an image processing apparatus capable of executing a plurality of image processes by one or more plug-in applications, the image processing apparatus comprising: a memory device storing a set of instructions; and at least one processor that executes the set of instructions to: in a case where new image processing by a plug-in application is requested during execution of image processing by the plug-in application, searching for a plug-in application capable of executing the new image processing; obtaining a memory usage amount at the time of executing new image processing of the plug-in application, which is held together with the searched plug-in application, and a memory usage amount of each image processing of the plug-in application that has been on the fly; comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in executing the plug-in application; and performing new image processing in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory without performing the new image processing in a case where the total memory usage exceeds the allowable memory capacity.
Another aspect of the present invention provides an image processing apparatus capable of executing a plurality of image processes in a first execution environment for executing a program described in a first programming language and a second execution environment for executing a program described in a second programming language, the image processing apparatus comprising: a memory device storing a set of instructions; and at least one processor that executes the set of instructions to: searching for a first application capable of performing new image processing in a case where the execution of the new image processing by a first application written in a first programming language is requested during the execution of the image processing in the first execution environment; obtaining a memory usage amount at the time of executing new image processing of the first application, which is held together with the searched first application, and memory usage amounts of respective image processing that has been in operation in the first execution environment; comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in the first execution environment; and performing new image processing in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory without performing the new image processing in a case where the total memory usage exceeds the allowable memory capacity.
Still another aspect of the present invention provides a control method of an image processing apparatus capable of executing a plurality of image processes by one or more plug-in applications, the control method comprising: in a case where a new image processing by the plug-in application is requested during the execution of the image processing by the plug-in application, searching for a plug-in application capable of executing the new image processing; obtaining a memory usage amount at the time of executing new image processing of the plug-in application, which is held together with the searched plug-in application, and a memory usage amount of each image processing of the plug-in application that has been on the fly; comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in executing the plug-in application; and performing new image processing in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory without performing the new image processing in a case where the total memory usage exceeds the allowable memory capacity.
Still another aspect of the present invention provides a control method of an image processing apparatus capable of executing a plurality of image processes in a first execution environment for executing a program described in a first programming language and a second execution environment for executing a program described in a second programming language, the control method comprising: in a case where a first application written in a first programming language is requested to execute new image processing during execution of image processing in a first execution environment, searching for a first application capable of executing new image processing; obtaining a memory usage amount at the time of executing new image processing of the first application, which is held together with the searched first application, and memory usage amounts of respective image processing that has been in operation in the first execution environment; comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in the first execution environment; and performing new image processing in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory without performing the new image processing in a case where the total memory usage exceeds the allowable memory capacity.
Yet another aspect of the present invention provides a non-transitory computer-readable storage medium storing a computer program for causing a computer to execute each step in a control method of an image processing apparatus capable of executing a plurality of image processes by one or more plug-in applications, the control method comprising: in a case where new image processing by a plug-in application is requested during execution of image processing by the plug-in application, searching for a plug-in application capable of executing the new image processing; obtaining a memory usage amount at the time of executing new image processing of the plug-in application, which is held together with the searched plug-in application, and a memory usage amount of each image processing of the plug-in application that has been on the fly; comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in executing the plug-in application; and performing new image processing in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory without performing the new image processing in a case where the total memory usage exceeds the allowable memory capacity.
Yet another aspect of the present invention provides a non-transitory computer-readable storage medium storing a computer program for causing a computer to execute steps in a control method of an image processing apparatus capable of executing a plurality of image processes in a first execution environment for executing a program described in a first programming language and a second execution environment for executing a program described in a second programming language, the control method comprising: in a case where new image processing by a plug-in application is requested during execution of image processing by the plug-in application, searching for a plug-in application capable of executing the new image processing; causing an obtaining unit to obtain a memory usage amount at the time of executing new image processing of the plug-in application, which is held together with the searched plug-in application, and a memory usage amount of each image processing of the plug-in application that has been in operation; comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in executing the plug-in application; and performing new image processing in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory without performing the new image processing in a case where the total memory usage exceeds the allowable memory capacity.
Further features of the invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Drawings
Fig. 1 is a diagram showing an example of the arrangement of an image processing forming system according to an embodiment;
fig. 2 is a block diagram showing an example of a hardware arrangement of an image processing apparatus according to the embodiment;
fig. 3 is a block diagram showing an example of a software arrangement of an image processing apparatus according to the embodiment;
fig. 4 is a flowchart showing a processing procedure of the image processing apparatus according to the embodiment;
fig. 5 is a flowchart showing a processing procedure of a local (native) image processing server according to an embodiment;
fig. 6 is a block diagram showing an example of a software arrangement of an image processing apparatus according to the embodiment;
fig. 7 is a flowchart showing a processing procedure of the image processing apparatus according to the embodiment;
fig. 8A and 8B are flowcharts showing a processing procedure of a local image processing module according to the embodiment;
fig. 9 is a flowchart showing a processing procedure of the local image processing server according to the embodiment;
fig. 10 is a flowchart showing a processing procedure of the local image processing server according to the embodiment;
fig. 11 is a flowchart showing a processing procedure of the local image processing module according to the embodiment.
Detailed Description
Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement of the components, the numerical expressions, and numerical values given in these embodiments do not limit the scope of the present invention unless specifically noted otherwise.
< first embodiment >
< System arrangement >
A first embodiment of the present invention will be described below with reference to the accompanying drawings. First, an example of the arrangement of the image forming system will be described with reference to fig. 1. The image forming system according to this embodiment includes image processing apparatuses 101 and 102, information processing terminals 103 and 104, and a server 105. The image processing apparatuses 101 and 102, the information processing terminals 103 and 104, and the server 105 are connected to each other via a network 106 so that they can communicate with each other.
Fig. 1 illustrates an example in which two image processing apparatuses 101 and 102 are arranged, but the number of image processing apparatuses is arbitrary (one or more). The respective image processing apparatuses 101 and 102 can be realized by the same arrangement. Thus, the arrangement of the image processing apparatus 101, which is representative of the image processing apparatuses 101 and 102, will be described, and a detailed description of the image processing apparatus 102 will be omitted. Note that the network 106 is a network such as a LAN (local area network) or the internet, through which devices in the image forming system can communicate with each other. The number and functions of the information processing terminals and servers are not intended to be limited. The server 105 can be implemented by a plurality of devices by distributing its functions. Note that the image processing apparatus to which the present invention is applied does not always have to include an image forming function, and an apparatus capable of executing a plurality of image processes simultaneously (in parallel) is applicable.
The image processing apparatus 101 can receive a print request (print data) of image data from the information processing terminals 103 and 104 and print, and read the image data by a scanner of the image processing apparatus 101 or print the image data read by the scanner. The image processing apparatus 101 can also save print data received from the information processing terminals 103 and 104, or transmit an image read by a scanner of the image processing apparatus 101 to the information processing terminals 103 and 104. The image processing apparatus 101 can perform image processing using the server 105 or print a document stored in the server 105. In addition, the image processing apparatus 101 can realize the functions of a known image processing apparatus such as an MFP (multi function peripheral).
< hardware arrangement of image processing apparatus >
Next, an example of the hardware arrangement of the image processing apparatus 101 will be described with reference to fig. 2. The image processing apparatus 101 includes a controller 201, a printer 202, a scanner 203, and an operation unit 204. The controller 201 includes a CPU 211, a RAM 212, an HDD 213, a network I/F214, a printer I/F215, a scanner I/F216, an operation unit I/F217, and an extension I/F218. The CPU 211 is capable of exchanging data with a RAM 212, HDD 213, network I/F214, printer I/F215, scanner I/F216, operation unit I/F217, and extension I/F218.
The CPU 211 develops instructions read out from the HDD 213 into the RAM 212, executes the instructions developed in the RAM 212, and executes various processes to be described later. The HDD 213 can store instructions executable by the CPU 211, setting values used in the image processing apparatus 101, data associated with processing requested from a user, and the like. The RAM 212 is an area for temporarily storing instructions read out from the HDD 213 by the CPU 211. The RAM 212 is also capable of storing various data required to execute instructions. For example, image processing can be performed by expanding input data in the RAM 212.
The network I/F214 is an interface for performing network communication with devices in the image forming system. The network I/F214 can notify the CPU 211 of the reception of data or transmit data in the RAM 212 to the network 106. The printer I/F215 can transmit print data received from the CPU 211 to the printer 202, or transfer the status of the printer 202 received from the printer 202 to the CPU 211. The scanner I/F216 can transmit an image reading instruction received from the CPU 211 to the scanner 203 and transfer image data received from the scanner 203 to the CPU 211. Further, the scanner I/F216 can also transmit the status received from the scanner 203 to the CPU 211.
The operation unit I/F217 can transfer an instruction input by the user on the operation unit 204 to the CPU 211, or transfer screen information for the operation of the user to the operation unit 204. The extension I/F218 is an interface capable of connecting an external device to the image processing apparatus 101. The expansion I/F218 includes, for example, a USB (universal serial bus) interface. When an external storage device such as a USB memory or the like is connected to the expansion I/F218, the image processing apparatus 101 can read out data stored in the external storage device and write data into the external storage device.
The printer 202 is capable of printing image data received from the printer I/F215 on a printing medium such as paper or sheet, or transferring the status of the printer 202 to the printer I/F215. The scanner 203 can read information written on a sheet set on the scanner 203, digitize the information, and transfer it to the scanner I/F216 according to an image reading instruction received from the scanner I/F216. The scanner 203 can communicate its status to the scanner I/F216. The operation unit 204 is an interface for prompting the user to perform operations for inputting various instructions to the image processing apparatus 101. The operation unit 204 includes, for example, a liquid crystal screen having a touch panel, provides a user with an operation screen, and accepts an operation from the user.
< software arrangement of image processing apparatus >
Next, an example of the structure of software processed by the CPU 211 according to the present embodiment will be described with reference to fig. 3. Software having the structure shown in fig. 3 is constructed using a program stored in the HDD 213 of the image processing apparatus 101. In fig. 3, the upper layer uses the services provided by the lower layer, with few exceptions.
In fig. 3, the lowermost layer is a layer including an operating system 317, and program execution management, memory management, and the like are performed in this layer. A printer control driver 318, a scanner control driver 319, and a network I/F control driver 320 are embedded in the operating system 317. The printer control driver 318, the scanner control driver 319, and the network I/F control driver 320 can function in cooperation with each other. The printer control driver 318 is software for controlling the printer 202 via the printer I/F215. The scanner control driver 319 is software for controlling the scanner 203 via the scanner I/F216. The network I/F control driver 320 is software for controlling the network I/F214.
The second layer from the lowermost layer includes a language device control library 309 and an image processing controller 340 serving as examples of the second execution environment. In the present embodiment, the device control library 309 and the image processing controller 340 are described in, for example, a compiler language (second programming language) such as C language or C + + language. Thus, the device control library 309 and the image processing controller 340 are stored in the HDD 213 in the form of an object file directly executable by the CPU 211. The device control library 309 is statically or dynamically linked to a single-function plug-in application 302 or a device control application 304, which will be described later. The device control library 309 uses the operating system 317 based on instructions from each application program. In addition, the device control library 309 can request image processing of the local image processing connection library 314.
Next, an example of the arrangement of the device control library 309 will be described. The print library 310 is a library that provides an API (application program interface) for controlling a print job using the functions of the printer control driver 318. The print job represents a series of processes of printing print data saved in the HDD 213 of the image processing apparatus 101 or printing print data received from an external device via the network I/F214. The external device is, for example, the information processing terminal 103 or 104. The copy library 311 is a library that provides an API for controlling a copy job using the functions of the scanner control driver 319 and the printer control driver 318. The copy job represents a series of processes of printing image data scanned by the scanner 203 by the printer 202.
The scan save library 312 is a library that provides an API for controlling a scan save job using the function of the scanner control driver 319. The scan save job represents a series of processes of converting image data scanned by the scanner 203 into print data or data in a common format and saving the data in the HDD 213 or an external storage device such as a USB memory connected to the expansion I/F218. Note that the common format is a data format such as PDF (portable file format) or JPEG (joint photographic experts group).
The scan transmission library 313 is a library that provides an API for controlling a scan transmission job using functions of the scanner control driver 319 and the network I/F control driver 320. The scan-and-send job represents a series of processes of converting image data scanned by the scanner 203 into data of a common format and sending the data to a file server via the network I/F214, or attaching the data to an email and sending to an external apparatus. The file server is, for example, a server 105, and the external device is, for example, an information processing terminal 103 or 104.
The image processing controller 340 includes a local image processing connection library 314, a local image processing server 315, and a local image processing module (local application) 316. The native application is an application that is installed in advance in the image processing apparatus 101 and provides various functions such as image processing. In contrast, a plug-in application to be described later is an application that can be added to the image processing apparatus 101 after product shipment. When the local image is atWhen the image processing execution request is received from the device control library 309, the image processing connection library 314 transfers the request content to the image processing connection library 332. Upon receiving a message from a later-described receiver
Figure BDA0002061099100000101
The native image processing server 315 provides the function of executing the native image processing module when requested by language-written software. The local image processing module 316 is software capable of performing various image processing. Note that the local image processing server 315 and the local image processing module 316 are executed on a local image processing control process 350, which local image processing control process 350 is a program execution unit having a logical memory space separate from the other software programs shown in fig. 3. As a specific method of dividing the memory space, it is conceivable to use a processing mechanism provided by a general-purpose OS (operating system), but other methods are also available. Even in the case of performing complicated calculations and additional memory operations, the configuration of dividing the logical memory space when performing image processing can suppress the influence of processing errors on the processing request side.
The Java language execution environment 330 serving as the uppermost layer is an example of the first execution environment, and is an application layer including the plug-in application 301 and the device control application 304. In this embodiment, the plug-in application 301 and the device control application 304 are written in the Java language (first programming language) and stored in the HDD 213 in a Java bytecode format interpreted by a Java virtual machine. The CPU 211 executes a program of the Java virtual machine, and the Java virtual machine reads out and executes Java byte codes to perform processing. One reason for using such programming languages is to facilitate program description. In Java, the memory area does not need to be managed by a developer, but is automatically managed. The labor for describing the program is reduced and high development efficiency is expected.
The applications run and provide various functions using the APIs of the device control library 309 and the image processing connection library 332. The function of the device control application 304 can be extended by updating the firmware. The print application 305, the copy application 306, the scan save application 307, and the scan transmission application 308 included in the device control application 304 have screen information 321, screen information 322, screen information 323, and screen information 324, respectively. The CPU 211 can display screen information 321, screen information 322, screen information 323, and screen information 324 on the operation unit 204 via the operation unit I/F217. When the user operates the operation unit 204 and the CPU 211 detects a setting change of the device control application 304, the CPU 211 writes the change content in the HDD 213.
When the CPU 211 (device control application 304) detects a job execution request via the operation unit 204, the CPU 211 calls an API of the device control library 309 and starts a job. The device control application 304 can request image processing of the image processing connection library 332.
The image processing connection library 332 holds the permission memory definition section 325. The allowable memory defining unit 325 defines the maximum allowable memory capacity of the image processing connection library 332 when executing the simultaneous processing. For example, the permitted memory defined in the permitted memory definition section 325 may be directly written in the program and obtained from the HDD 213. Alternatively, the files defining the allowed amount may be pre-installed and loaded at execution time. Note that the method of obtaining the permission memory may be other than the above-described method. The image processing connection library 332 also holds the running memory management section 326. The running memory management unit 326 manages the total memory capacity of the running image processing via the image processing connection library 332.
Next, an example of the device control application 304 will be described. The print application 305 calls an API of the print library 310 and executes a print job. The copy application 306 calls the API of the copy library 311 and executes a copy job. The scan save application 307 calls an API of the scan save library 312 and executes a scan save job. The scan transmission application 308 calls an API of the scan transmission library 313 and executes a scan transmission job.
Next, an example of the plug-in application 301 will be described. Unlike the device control application 304 which is a resident application (persistent application), the plug-in application 301 is an application that can be installed and uninstalled as a plug-in. The plug-in application 301 has a single-function plug-in application 302 and an image processing plug-in application 303. In the plug-in application 301, programs necessary for respective operations are packaged. The plug-in application 301 is installed in the image processing apparatus 101 by using a remote UI (user interface) or the like. Note that the remote UI is one mechanism as follows: the image processing apparatus 101 can be accessed from a Web browser of an external device via the network I/F214, and confirmation of the status of the image processing apparatus 101, operation of a print job, various settings, and the like are performed. The external device is, for example, the information processing terminal 103 or 104. The plug-in application 301 (the single-function plug-in application 302 and the image processing plug-in application 303) can be started and stopped independently.
An example of a series of operations of starting, stopping, and uninstalling from the installation of the plug-in application 301 will be described below. When the CPU 211 detects the installation of the plug-in application 301, it saves the information of the plug-in application 301 in the HDD 213. When the CPU 211 detects an instruction to start the plug-in application 301, it gives an instruction to start the plug-in application 301. The plug-in application 301 can execute the contents of each program while it is started. When the CPU 211 detects an instruction to stop the plug-in application 301, it gives an instruction to stop the plug-in application 301. When the CPU 211 detects an instruction to uninstall the plug-in application 301, it deletes the information of the plug-in application 301 from the HDD 213. Note that these instructions can be input from the remote UI or the operation unit 204, but these instructions may be input by methods other than the above-described methods.
Next, an example of the single-function plug-in application 302 will be described. The single-function plug-in application 302 has screen information 321. By calling the API of the device control library 309, the single function plug-in application 302 can provide the user with functions and screens different from those of the resident device control application 304. For example, the single-function plug-in application 302 can provide a plurality of functions combined, such as a function of copying and scanning given image data and a function of transmitting a document to a specific destination in a transmission destination database held by the single-function plug-in application 302. Note that the single-function plug-in application 302 need not have a function of performing image processing. In this case, no setting is made for image processing. When the device control library 309 receives print data or image data converted into a common format, the device control library 309 instructs an appropriate operating system 317 to perform processing control and causes it to execute a job.
Next, an example of the image processing plug-in application 303 will be described. The image processing plug-in application 303 is an application that provides specific image processing. The image processing plug-in application 303 accepts processing target image data and processing parameters from the single-function plug-in application 302, the device control application 304, and the like via the image processing connection library 332, and executes the requested image processing. The image processing plug-in application 303 is able to receive image processing requests from even the device control library 309 via the local image processing connection library 314 and the image processing connection library 332. Note that a plurality of image processing plug-in applications 303 may exist in the HDD 213. For example, there may be respective image processing plug-in applications 303 capable of image format conversion, skew correction, form recognition, OCR processing, and the like for an input image.
The image processing plug-in application 303 may not have an image processing function by itself. For example, the image processing plug-in application 303 can use the native image processing client 331 and use the image processing functions of the native image processing module 316. One of the reasons that the image processing plug-in application 303 uses the local image processing module 316 is a high processing speed when performing image processing. When image processing is performed, a large number of complicated numerical calculations are sometimes performed, and a large amount of memory capacity is required during the processing. In this case, instead of using a programming language processing system that performs processing via a virtual machine such as Java, processing speed can be increased by using a compiler language that can generate an object file that can be directly executed by the CPU.
The image processing plug-in application 303 holds an execution memory definition section 327. The execution memory definition unit 327 defines the maximum memory capacity used for image processing executed by each plug-in. Note that the above memory capacity may be maintained for each plug-in or for each function of image processing.
< treatment Process >
Next, an example of a processing procedure of the image processing apparatus 101 according to the present embodiment will be described with reference to fig. 4. The following is assumed here: the image processing connection library 332 accepts image processing, performs memory control, and then requests processing of the image processing plug-in application 303. This flowchart shows an example of image processing using the image processing plug-in application 303. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
First, in step S401, the image processing connection library 332 receives an image processing execution request from the device control application 304, the local image processing connection library 314, or the single-function plug-in application 302. At this time, it is assumed that the received execution request includes parameters necessary for specifying the image processing plug-in application 303 whose processing is requested.
Then, in step S402, the image processing connection library 332 searches for the image processing plug-in application 303 capable of executing the image processing corresponding to the obtained execution request. In step S403, the image processing connection repository 332 determines whether there is an object image processing plug-in application based on the search result in step S402.
If the image processing connection library 332 determines in step S403 that there is no application capable of executing the received image processing request, the process proceeds to step S404, and in step S404, the image processing connection library 332 sends back an error indicating that there is no application capable of executing the received image processing request to the module as the request source. After that, the process ends. For example, when the local image processing connection library 314 is a request source, the above-described error content is delivered to the device control application 304 or the single-function plug-in application 302 via the device control library 309. The device control application 304 or the single function plug-in application 302 displays the received error content on the operation screen of each application. The error can inform the user that there are no modules required for processing. The service person or IT administrator installs the required image processing plug-in application 303 by the above installation method and can continue the processing.
If the image processing connection library 332 determines in step S403 that the subject image processing plug-in application 303 exists, the process proceeds to step S405, and in step S405, the image processing connection library 332 obtains the execution memory capacity from the execution memory defining section 327 of the subject image processing plug-in application 303. As described above, the execution memory capacity may be defined for each plug-in unit or held by each functional unit of image processing in each plug-in. Subsequently, in step S406, the image processing connection library 332 obtains the allowable memory capacity from the allowable memory defining section 325 of the image processing connection library 332. In step S407, the image processing connection library 332 obtains the total in-operation memory capacity from the in-operation memory management section 326 of the image processing connection library 332.
In step S408, the image processing link library 332 determines whether the sum of the obtained in-operation memory capacity and the total in-operation memory capacity managed by the image processing link library 332 exceeds the allowable memory capacity (whether the memory becomes insufficient). If the image processing connection library 332 determines that the sum does not exceed the allowable memory capacity, the execution memory capacity defined by the execution memory defining section 327 is added to the in-operation memory capacity managed by the in-operation memory managing section 326 in step S409. In step S410, the image processing connection library 332 requests the processing of the subject image processing plug-in application 303. The image processing plug-in application 303 receives an image processing request together with processing parameters from the image processing connection library 332, and performs image processing according to the processing parameters. There are a method of image processing within the image processing plug-in application 303 and a method performed by the local image processing module 316, the details of which will be described later. In executing image processing, the CPU 211 expands input data in the RAM 212 in the image processing plug-in application 303 or the local image processing module 316. At this time, it is necessary to assign a memory having a capacity corresponding to the image processing content and the input data size. Thereafter, in step S411, the image processing connection library 332 receives the processing result from the image processing plug-in application 303, and subtracts the execution memory capacity from the total in-flight memory capacity in step S412. In step S413, the image processing connection library 332 sends back the image processing result to each module as the request source and ends the processing.
If the image processing link library 332 determines in step S408 that the sum exceeds the allowable memory capacity, the process proceeds to step S414, and in step S414, the image processing link library 332 sends back an error indicating that image processing cannot be performed because the memory resources required for image processing are currently insufficient (because the memory is insufficient), to the module as the request source, and ends the process. When the local image processing connection library 314 is a request source, the above-described error content is delivered to the device control application 304 or the single-function plug-in application 302 via the device control library 309. The device control application 304 or the single function plug-in application 302 performs exception processing based on the received error content described above.
An example of exception handling is retry handling. When each application receives the above-described error, it can determine that a plurality of image processes are simultaneously performed on the image processing connection library 332. Each application waits until the currently running image processing is temporarily finished by retry, and transmits an image processing request again when the running image processing is finished. Processing can continue without interruption.
Another exception processing is control for displaying an error on the operation screen of each application, which prompts a job to be input again after a certain time (after a predetermined time elapses) since a plurality of image processes are simultaneously executed. This error can inform the user that the job can be executed after a certain time.
As described above, according to the present embodiment, before requesting processing for a module (plug-in application) that performs image processing, it is determined whether the memory capacity exceeds the allowable memory capacity due to the image processing to be performed. If the memory capacity exceeds the allowable memory capacity, image processing is not requested first. In the present embodiment, it is possible to avoid assigning a memory exceeding the allowable memory capacity. Memory assignment can be controlled before system operation becomes unstable due to assigning memory that exceeds the allowed memory capacity of the system.
< image processing of image processing plug-in application >
Image processing by the image processing plug-in application will be described below. When image processing is requested to the image processing plug-in application 303, the image processing plug-in application 303 performs image processing based on the requested processing content. The image processing plug-in application 303 is capable of performing image processing by at least two methods. The first method is to perform image processing within the image processing plug-in application 303. In this case, the image processing plug-in application performs image processing based on the processing request and the processing parameters received from the image processing connection library 332. In this case, the image processing plug-in application needs to be described in the Java language. The second method is to request image processing by the local image processing module 316. The image processing plug-in application 303 requests processing of the local image processing client 331, and is capable of performing image processing in a process dedicated to image processing different from that of the image processing plug-in application 303.
A processing procedure of performing image processing by the image processing plug-in application 303 will be described with reference to fig. 5. This flowchart shows a process of requesting the local image processing server 315 to perform image processing. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
The image processing plug-in application 303 sends an image processing request to the local image processing client 331 along with the processing parameters. The local image processing client 331 transmits the received processing parameters to the local image processing server 315 and requests processing of the local image processing server 315.
In step S501, the local image processing server 315 receives the image processing execution request, and searches the object local image processing module 316 based on the received processing parameter in step S502. In step S503, the local image processing server 315 determines whether the object local image processing module 316 exists. If there is no object local image processing module 316, the process proceeds to step S504, and in step S504, the local image processing server 315 sends back an error indicating that there is no object local image processing module 316 to the local image processing client 331, and ends the process.
If the object local image processing module 316 exists, the processing proceeds to step S505, and in step S505, the local image processing server 315 causes the object local image processing module 316 to perform image processing based on the received processing parameters. Then, in step S506, the local image processing server 315 sends back the result of the image processing in step S505 to the local image processing client 331, and ends the processing. When the local image processing client 331 receives the image processing result or the error content, it sends the content back to the image processing plug-in application 303.
As described above, the image processing apparatus according to the present embodiment is an image processing apparatus capable of executing a plurality of image processes by one or more plug-in applications. When the image processing apparatus is requested to perform new image processing by the plug-in application during execution of the image processing by the plug-in application, the image processing apparatus searches for the plug-in application capable of performing the new image processing. Further, the image processing apparatus obtains the memory usage amount at the time of executing new image processing of the plug-in application, which is held together with the searched plug-in application, and the memory usage amount of each image processing of the plug-in application which has been in operation. The image processing apparatus compares a total memory usage amount obtained from the obtained memory usage amount of each image processing and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating the memory usage amount allowed to be used in executing the plug-in application. When the comparison result is that the total memory usage does not exceed the allowable memory capacity, the image processing apparatus performs new image processing. When the total memory usage exceeds the allowable memory capacity, the image processing apparatus notifies the request source of an error indicating that new image processing cannot be performed due to a shortage of memory without performing the new image processing. Therefore, memory control of a plurality of image processes can be performed at runtime by calculating the total memory of image processes in execution at the time of executing the image processes without restricting the startup (installation) of the image processing plug-in application 303. According to the present embodiment, a plurality of image processes can be performed while preferably avoiding memory shortage. This can improve the convenience of the user.
The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the execution memory defined in the execution memory defining section 327 of the image processing plug-in application 303 can describe an execution memory used when the local image processing module 316 executes image processing.
< second embodiment >
A second embodiment of the present invention will be described below. The first embodiment describes an example in which the image processing connection library 332 performs memory control of a plurality of image processings at runtime. The second embodiment will describe the following example: when the plug-in application requests image processing, the local image processing server 315 performs memory control of a plurality of image processing in the local image processing control process 350.
< software arrangement of image processing apparatus >
First, an example of a software structure processed by the CPU 211 according to the present embodiment will be described with reference to fig. 6. Note that the basic arrangement is the same as that described with reference to fig. 3 in the first embodiment, so the difference will be mainly explained. The same reference numerals denote the same parts, and the description thereof will not be repeated.
The local image processing server 315 holds the permitted memory definition section 601. The allowable memory definition section 601 defines the maximum allowable memory capacity that is feasible when executed simultaneously on the local image processing server 315. The allowable memory capacity defined in the allowable memory defining section 601 may be directly written in the program and obtained from the HDD 213. Alternatively, files defining the allowable memory capacity may be pre-installed and loaded at execution time. Note that the method of obtaining the permission memory may be a method other than the above-described method. The local image processing server 315 also holds an on-the-fly memory management section 602. The running memory management section 602 manages the total memory capacity of the running image processing via the local image processing server 315.
< treatment Process >
Next, an example of a processing procedure of the image processing apparatus 101 according to the present embodiment will be described with reference to fig. 7. The following is assumed here: the local image processing server 315 manages memory of image processing in operation and performs image processing. The source of this image processing request is the image processing plug-in application 303. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
In step S701, the local image processing server 315 receives an image processing execution request together with processing parameters from the local image processing client 331, and searches for the local image processing module 316 in step S702. In step S703, the local image processing server 315 determines whether the object local image processing module 316 exists based on the search result in step S702.
If the object local image processing module 316 does not exist, the process proceeds to step S704, and in step S704, the local image processing server 315 sends back an error indicating that the object local image processing module 316 does not exist, and ends the process. When receiving an error, the local image processing client 331 delivers the content of the error to the image processing plug-in application 303 serving as a request source. In this case, the required local image processing module 316 is not installed in the HDD 213 of the image processing apparatus 101 and is also not generated under normal operation. Therefore, the developer needs to modify the source code.
If the object local image processing module 316 exists, the processing proceeds to step S705, and in step S705, the local image processing server 315 obtains the execution memory capacity from the execution memory defining section 603 of the object local image processing module 316. Subsequently in step S706, the local image processing server 315 obtains the allowable memory capacity from the allowable memory defining section 601 of the local image processing server 315. In step S707, the local image processing server 315 obtains the total on-the-fly memory capacity from the on-the-fly memory management section 602 of the local image processing server 315.
In step S708, the local image processing server 315 determines whether the sum of the obtained in-flight memory capacity and the total in-flight memory capacity exceeds the allowable memory capacity. If the local image processing server 315 determines that the sum does not exceed the allowable memory capacity, the process proceeds to step S709, where the local image processing server 315 adds the execution memory capacity to the total in-flight memory capacity in step S709. In step S710, the local image processing server 315 requests the local image processing module 316 to perform image processing. The local image processing module 316 receives the image processing request along with the processing parameters and performs image processing according to the processing parameters. In performing image processing, the CPU 211 develops input data in the RAM 212, and performs processing in the local image processing module 316. At this time, it is necessary to assign a memory according to the image processing content and the input data size.
In step S711, the local image processing server 315 receives the image processing result from the local image processing module 316, and subtracts the execution memory capacity from the total in-flight memory capacity in step S712. In step S713, the local image processing server 315 sends back the image processing result to the local image processing client 331 and ends the processing.
If the local image processing server 315 determines in step S708 that the sum exceeds the allowable memory capacity, the process proceeds to step S714, and in step S714, the local image processing server 315 sends back an error indicating that image processing cannot be performed because the required memory resources are currently insufficient, to the local image processing client 331. After that, the process ends.
The local image processing client 331 passes the above-described image processing result and error content to the image processing plug-in application 303. The image processing plug-in application receives the image processing result and passes it to the module that is the source of the request via the image processing connection repository 332. When an error indicating that the memory resources are insufficient is received, exception processing is performed. An example of exception handling is retry handling. When receiving the above-mentioned error, the image processing plug-in application 303 can determine that the image processing cannot be performed due to the image processing in operation. Thereby, the image processing client 331 is requested to perform processing after a predetermined time and the processing can be executed. The retry process performed within the image processing plug-in application has an advantage that the retry process is not performed by the module as the request source. As another exception handling, the above error is even passed to the single function plug-in application 302 and the device control application 304 at the source of the request. When both a case where retry is desired and a case where interrupt retry is desired are conceivable depending on the application, switching of retry and interrupt can be performed within the application by transferring the above-described error to the application side.
As described above, control is performed by determining in advance whether the memory capacity exceeds the allowable memory capacity due to the image processing to be performed before requesting the processing to the local image processing module 316. If the memory capacity exceeds the allowable memory capacity, image processing is not requested first and memory is not assigned wastefully. Memory assignment can be controlled in advance before system operation becomes unstable due to assigning memory that exceeds the allowed memory capacity of the system (device).
In the present embodiment, the same memory control can be performed for even another image processing control process implemented by the plug-in, in parallel with the above-described memory control within the local image processing control process 350. That is, the allowable memory capacity may be defined for each process. For example, the arrangement according to the second embodiment may be applied in combination with the arrangement according to the first embodiment.
According to the present embodiment, by calculating the total memory of the image processing in execution at the time of executing the image processing, the memory control of a plurality of image processing can be performed in the execution time without limiting the startup of the image processing plug-in application 303. This can improve user convenience. Unlike the first embodiment, memory management can be performed restrictively within a process (local image processing control process 350) dedicated to performing image processing. The image processing plug-in application 303 need not be concerned with memory control of image processing on the local image processing control process 350. The development efficiency of the image processing plug-in application 303 is significantly improved.
< third embodiment >
A third embodiment of the present invention will be described below. This embodiment will explain an example in which the local image processing module 316 mainly performs memory control. In the third embodiment, when memory control is performed and parallel processing cannot be performed, processing is performed without quickly sending back an error, unlike the first embodiment and the second embodiment. When image processing of multiple pages can be processed in parallel, parallel processing is used to speed up the process.
< treatment Process >
First, an example of a processing procedure of the image processing apparatus 101 according to the present embodiment will be described with reference to fig. 8A and 8B. The following examples will be explained: the local image processing module 316 registers execution memory in the local image processing server 315 and performs memory control. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
First, in step S801, the local image processing module 316 receives an image processing execution request from the local image processing server 315. Then, in step S802, the local image processing module 316 obtains the memory size (memory usage amount) necessary for the received image processing from the execution memory defining section 603. In step S803, the local image processing module 316 determines whether parallel processing is possible for the received image processing. The local image processing module 316 may have information indicating whether parallel processing is possible and use this information for the determination. Note that, as described in the above first and second embodiments, based on the required memory usage amount and the memory usage amount of each image processing that has been run by the native application, it is possible to determine whether parallel processing is possible using information indicating whether parallel processing is possible. For example, a threshold memory capacity corresponding to the total memory usage may be determined such that the total memory usage does not exceed the allowable memory capacity when parallel processing is performed with two total memory usage, and the determination is made based on a result of comparison between the threshold memory capacity and the allowable memory capacity. Alternatively, information (e.g., threshold memory capacity) indicating whether parallel processing is possible may be added to the execution memory defining section 603 as meta information in advance and read out for determination.
If, in step S804, the local image processing module 316 determines that the received image processing cannot be parallel-processed based on the determination result in step S803, the process proceeds to step S805. If the local image processing module 316 determines that parallel processing is possible, the process proceeds to step S811.
Step S805 indicates a process of repeating the processes of steps S806 to S810 in accordance with the number of pages of the synchronously received image processing when parallel processing is not possible (respective image processing is sequentially executed). In step S806, the local image processing module 316 registers the memory information obtained from the execution memory defining section 603 of the local image processing module 316 in the local image processing server 315. Details of registering the memory information by the local image processing server 315 will be described later with reference to fig. 9. In step S807, the local image processing module 316 determines whether the registration of the execution memory in step S806 is successful. If the registration fails, the process proceeds to step S808, and in step S808, the local image processing module 316 sends back an error indicating that image processing cannot be performed due to insufficient memory resources required for image processing to the local image processing server 315, and ends the process. The local image processing server 315 delivers the error to the application on the calling side via the local image processing client 331.
If the registration of the execution memory in the local image processing server 315 is successful, the processing proceeds to step S809, and in step S809, the local image processing module 316 executes the object image processing. When the image processing is completed, the local image processing module 316 requests the local image processing server 315 to delete the execution memory and the memory information to be deleted in step S810. After the repeated loop processing ends, the processing ends.
If the local image processing module 316 determines in step S804 that parallel processing is possible, the processing proceeds to step S811, and in step S811, when parallel processing is possible, the processing in steps S812 to S814 is repeated in accordance with the number of pages of synchronously received image processing. In step S812, the local image processing module 316 registers the execution memory obtained from the execution memory defining section 603 of the local image processing module 316 in the local image processing server 315. Note that a sequence registered by the memory of the local image processing server 315 will be described later with reference to fig. 9. In step S813, the local image processing module 316 determines whether the registration of the execution memory is successful. If the registration is successful, the process proceeds to step S814, and in step S814, the local image processing module 316 increments the parallel execution count. The parallel execution count represents the number of threads (threads) to be executed in parallel and is a variable managed in the RAM 212, HDD 213, and the like. If the registration of the execution memory fails, the processing exits from the loop processing and proceeds to step S815. If the repeated loop processing ends, the processing proceeds to step S815. At the stage where the processing exits from the loop processing, a positive integer ranging from 0 to the set page count is input in the parallel execution count.
In step S815, the local image processing module 316 determines whether the calculated parallel execution count is greater than 0. If the parallel execution count is greater than 0, the process proceeds to step S816; otherwise, the process proceeds to step S808. In step S808, the local image processing module 316 sends back an error indicating that image processing cannot be performed due to insufficient memory resources required for image processing to the local image processing server 315, and ends the processing. The local image processing server 315 delivers the error to the application on the calling side via the local image processing client 331.
If the parallel execution count is an integer of 1 or more, the process proceeds to step S816, and in step S816, the local image processing module 316 starts a thread in accordance with the parallel execution count. Although the thread is started in the present embodiment, the processing may be started. Subsequently, the process proceeds to step S817, where the process in step S818 is repeated in accordance with the set page count. In step S818, the local image processing module 316 assigns image processing of each page to an idle thread (free thread) and executes it. When the processing exits from this loop processing, the execution of the image processing of all the threads is completed, and the image processing of all the received pages is completed. Finally, in step S819, the local image processing module 316 requests the local image processing server 315 to delete the execution memory in accordance with the started thread count, and ends the processing.
< registration of execution memory >
A processing procedure when the local image processing server 315 receives the execution memory registration request from the local image processing module 316 in steps S806 and S812 will be described with reference to fig. 9. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
In step S901, the local image processing server 315 receives an execution memory registration request together with execution memory information from the local image processing module 316. In step S902, the local image processing server 315 determines whether the sum of the received memory size and the memory size of the image processing in operation exceeds the allowable memory size. If the sum exceeds the allowable memory size, the processing proceeds to step S903, and in step S903, the local image processing server 315 sends a message indicating registration failure back to the local image processing module 316. Then, the process ends. If the sum does not exceed the allowable memory size, the process proceeds to step S904, and in step S904, the local image processing server 315 adds the received memory size to the running memory, and sends a message indicating that the registration is successful back to the local image processing module 316 in step S905. Then, the process ends.
< performing deletion of memory >
A processing procedure when the local image processing server 315 receives the execution memory deletion request from the local image processing module 316 in step S810 will be described with reference to fig. 10. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
In step S1001, the local image processing server 315 receives an execution memory deletion request from the local image processing module 316. In step S1002, the local image processing server 315 subtracts the received memory size from the memory in operation, and ends the processing.
As described above, according to the present embodiment, the local image processing module 316 can mainly perform memory control. Thread control may be performed appropriately depending on whether parallel processing is possible and is not dependent on the characteristics of the local image processing module 316. The speed for multiple pages can be increased while performing memory control.
< fourth embodiment >
A fourth embodiment of the present invention will be described below. The third embodiment has described the following example: the parallel execution count is determined before the image processing of the plurality of pages is executed, and then parallel processing is performed in an image processing module capable of parallel execution. The fourth embodiment will describe an example in which the parallel execution count is dynamically increased during the execution of image processing, as a modification of the third embodiment.
< treatment Process >
First, a processing procedure of dynamically increasing the parallel execution count during image processing according to the present embodiment will be described with reference to fig. 11. The flowchart is a modified example of the processing in steps S816 to S819 in the flowchart of fig. 8B. The processing performed before step S816 is the same as in the flowchart of fig. 8B. Note that the processing to be described below is realized when, for example, the CPU 211 reads out a program saved in the HDD 213 to the RAM 212 and executes the program.
In step S1101, the local image processing module 316 starts a thread in accordance with the parallel execution count. The process proceeds to step S1102, and in step S1102, the loop processing of steps S1103 to S1107 is repeatedly executed in accordance with the set page count. More specifically, in step S1103, the local image processing module 316 assigns image processing of each page to an idle thread and executes it. The process proceeds to step S1104, and in step S1104, the loop processing of steps S1105 to S1107 is repeatedly executed in accordance with the number of pages waiting for image processing.
In step S1105, the local image processing module 316 registers the execution memory in the local image processing server 315. In step S1106, the local image processing module 316 determines whether the registration of the execution memory is successful. If the registration of the execution memory fails, the processing exits from the loop from step S1104, and returns to the loop of step S1102. If the local image processing module 316 determines in step S1106 that the registration of the execution memory is successful, an execution thread is generated and the process returns to step S1104. These loops are repeatedly executed in accordance with the number of pages corresponding to the set image processing (step S1102). As a result, the image processing for all pages is completed, and the process proceeds to step S1108.
In step S1108, the local image processing module 316 requests the local image processing server 315 to delete the execution memory in accordance with the started thread count, and ends the processing. Note that the processing sequence of the local image processing server 315 for performing memory registration (step S1105) and performing memory deletion (step S1108) is the same as that described with reference to fig. 9 and 10.
As described above, according to the present embodiment, execution threads can be dynamically added by making an inquiry to the local image processing server 315 between image processing of respective pages in image processing of a plurality of pages. Therefore, image processing of a plurality of pages can be performed more efficiently than in the third embodiment.
< other examples >
Embodiments of the present invention may also be implemented by a computer of a system or apparatus that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (also more fully referred to as "non-transitory computer-readable storage medium") to perform the functions of one or more of the above-described embodiments, and/or includes one or more circuits (e.g., Application Specific Integrated Circuits (ASICs)) for performing the functions of one or more of the above-described embodiments, and methods may be utilized by which a computer of the system or apparatus, for example, reads and executes the computer-executable instructions from the storage medium to perform the functions of one or more of the above-described embodiments, and/or controls the one or more circuits to perform the functions of one or more of the above-described embodiments, to implement embodiments of the present invention. The computer may include one or more processors (e.g., a Central Processing Unit (CPU), a Micro Processing Unit (MPU)) and may include a separate computer or a network of separate processors to read out and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), a memory of a distributed computing system, an optical disc such as a Compact Disc (CD), a Digital Versatile Disc (DVD), or a blu-ray disc (BD) TM, a flash memory device, and a memory card, among others.
The embodiments of the present invention can also be realized by a method in which software (programs) that perform the functions of the above-described embodiments are supplied to a system or an apparatus through a network or various storage media, and a computer or a Central Processing Unit (CPU), a Micro Processing Unit (MPU) of the system or the apparatus reads out and executes the methods of the programs.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims (16)

1. An image processing apparatus capable of performing image processing by a plug-in application and image processing by a native application, respectively, comprising: a storage device storing a series of instructions; and at least one processor that executes the series of instructions to:
installing and activating the plug-in application;
executing image processing by the plug-in application in the process of operating the plug-in application when the plug-in application is requested to perform the image processing;
in the case where new image processing of the native application is requested during execution of image processing by the native application installed in advance in an image processing apparatus, after the new image processing is requested,
obtaining a memory usage amount at the time of executing new image processing of a native application, which is held together with the native application, and a memory usage amount of each image processing of the native application that has been on the fly;
comparing the total memory usage obtained from the memory usage of the obtained respective image processings and the memory usage when the requested new image processing is performed, with an allowable memory capacity representing a memory usage that is allowed to be used in a process other than a process in which the plug-in application operates, in which the local application operates; and
performing new image processing during operation of the native application in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a memory shortage in a case where the total memory usage exceeds the allowable memory capacity without performing the new image processing;
wherein, in a case where the plug-in application executes image processing during operation of the plug-in application, the image processing is not controlled based on an allowable memory capacity indicating a memory usage amount allowed to be used during operation of the native application.
2. The image processing apparatus of claim 1, wherein the at least one processor executes instructions in the memory device to:
in a case where new image processing of the native application is requested during execution of image processing by the native application installed in advance in an image processing apparatus, it is determined whether parallel processing can be executed according to a memory usage amount when new image processing of the native application is executed and memory usage amounts of respective image processing of the native application that has been in operation; and
in the case where the determination result is that the parallel processing cannot be performed, the new image processing and the image processing that has been in operation are sequentially performed, whereas in the case where the parallel processing can be performed, the new image processing and the image processing that has been in operation are performed in parallel.
3. The image processing apparatus of claim 2, wherein at least one processor executes instructions in the memory device to dynamically determine whether parallel processing is possible for each page of image processing.
4. The image processing apparatus of claim 2, wherein the at least one processor executes instructions in the memory device to:
in the case where an execution memory for executing image processing cannot be assigned, an error indicating that image processing cannot be executed is transmitted.
5. The image processing apparatus according to claim 1, wherein in a case where the request source receives an error, the request source requests new image processing again after a predetermined time elapses.
6. The image processing apparatus according to claim 1, wherein a request source of new image processing of the native application operates in a process different from a process of performing new image processing.
7. The image processing apparatus according to claim 1, wherein a memory usage amount when performing the image processing is held for each function included in the image processing.
8. An image processing apparatus capable of executing a plurality of image processes in a first execution environment for executing a program described in a first programming language and in a second execution environment for executing a program described in a second programming language, the image processing apparatus comprising: a memory device storing a series of instructions; and at least one processor that executes the series of instructions to:
executing, by a first application, image processing in a first execution environment in a case where image processing of the first application described in a first programming language is requested;
in a case where new image processing of a second application written in a second programming language is requested during execution of image processing in a second execution environment, after the new image processing is requested, obtaining a memory usage amount at the time of execution of the new image processing of the second application, which is held together with the second application, and memory usage amounts of the respective image processing already in operation in the second execution environment;
comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in the second execution environment; and
performing new image processing in the second execution environment in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory in a case where the total memory usage exceeds the allowable memory capacity without performing the new image processing;
wherein, in a case where the image processing of the first application is executed in the first execution environment, the image processing is not controlled based on the permitted memory capacity indicating the memory usage amount permitted to be used in the second execution environment.
9. The image processing apparatus of claim 8, wherein the at least one processor executes instructions in the memory device to:
in a case where new image processing of a second application written in a second programming language is requested during execution of image processing in a second execution environment, it is determined whether parallel processing can be executed according to a memory usage amount when new image processing of the second application is executed and memory usage amounts of respective image processing that has been in operation in the second execution environment; and
in the case where the determination result is that the parallel processing cannot be performed, the new image processing and the image processing that has been in operation are sequentially performed, whereas in the case where the parallel processing can be performed, the new image processing and the image processing that has been in operation are performed in parallel.
10. The image processing apparatus of claim 9, wherein at least one processor executes instructions in the memory device to dynamically determine whether parallel processing is possible for each page of image processing.
11. The image processing apparatus of claim 9, wherein the at least one processor executes instructions in the memory device to:
in the case where an execution memory for executing image processing cannot be assigned, an error indicating that image processing cannot be executed is transmitted.
12. The image processing apparatus according to claim 8, wherein in a case where the request source receives an error, the request source requests new image processing again after a predetermined time elapses.
13. A control method of an image processing apparatus capable of performing image processing by a plug-in application and image processing by a native application, respectively, the image processing apparatus comprising: a storage device storing a series of instructions; and at least one processor executing the series of instructions, the control method comprising:
installing and activating the plug-in application;
executing image processing by the plug-in application in the process of operating the plug-in application when the plug-in application is requested to perform the image processing;
in a case where new image processing of a native application is requested during execution of image processing by the native application installed in advance in an image processing apparatus, after the new image processing is requested, a memory usage amount at the time of execution of the new image processing of the native application, which is held together with the native application, and memory usage amounts of the respective image processing of the native application that have been in operation are obtained;
comparing the total memory usage obtained from the memory usage of the obtained respective image processings and the memory usage when the requested new image processing is performed, with an allowable memory capacity representing a memory usage that is allowed to be used in a process other than a process in which the plug-in application operates, in which the local application operates; and
performing new image processing during operation of the native application in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a memory shortage in a case where the total memory usage exceeds the allowable memory capacity without performing the new image processing;
wherein, in a case where the plug-in application executes image processing during operation of the plug-in application, the image processing is not controlled based on an allowable memory capacity indicating a memory usage amount allowed to be used during operation of the native application.
14. A control method of an image processing apparatus capable of executing a plurality of image processes in a first execution environment for executing a program described in a first programming language and in a second execution environment for executing a program described in a second programming language, the image processing apparatus comprising: a memory device storing a series of instructions; and at least one processor executing the series of instructions, the control method comprising:
executing, by a first application, image processing in a first execution environment in a case where image processing of the first application described in a first programming language is requested;
in a case where new image processing of a second application written in a second programming language is requested during execution of image processing in a second execution environment, after the new image processing is requested, obtaining a memory usage amount at the time of execution of the new image processing of the second application, which is held together with the second application, and memory usage amounts of the respective image processing already in operation in the second execution environment;
comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in the second execution environment; and
performing new image processing in the second execution environment in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory in a case where the total memory usage exceeds the allowable memory capacity without performing the new image processing;
wherein, in a case where the image processing of the first application is executed in the first execution environment, the image processing is not controlled based on the permitted memory capacity indicating the memory usage amount permitted to be used in the second execution environment.
15. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute steps in a control method of an image processing apparatus capable of performing image processing by a plug-in application and image processing by a native application, respectively, the image processing apparatus comprising: a storage device storing a series of instructions; and at least one processor executing the series of instructions, the control method comprising:
installing and activating the plug-in application;
executing image processing by the plug-in application in the process of operating the plug-in application when the plug-in application is requested to perform the image processing;
in a case where new image processing of a native application is requested during execution of image processing by the native application installed in advance in an image processing apparatus, after the new image processing is requested, a memory usage amount at the time of execution of the new image processing of the native application, which is held together with the native application, and memory usage amounts of the respective image processing of the native application that have been in operation are obtained;
comparing the total memory usage obtained from the memory usage of the obtained respective image processings and the memory usage when the requested new image processing is performed, with an allowable memory capacity representing a memory usage that is allowed to be used in a process other than a process in which the plug-in application operates, in which the local application operates; and
performing new image processing during operation of the native application in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a memory shortage in a case where the total memory usage exceeds the allowable memory capacity without performing the new image processing;
wherein, in a case where the plug-in application executes image processing during operation of the plug-in application, the image processing is not controlled based on an allowable memory capacity indicating a memory usage amount allowed to be used during operation of the native application.
16. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute steps in a control method of an image processing apparatus capable of executing a plurality of image processes in a first execution environment for executing a program described in a first programming language and a second execution environment for executing a program described in a second programming language, the image processing apparatus comprising: a memory device storing a series of instructions; and at least one processor executing the series of instructions, the control method comprising:
executing, by a first application, image processing in a first execution environment in a case where image processing of the first application described in a first programming language is requested;
in a case where new image processing of a second application written in a second programming language is requested during execution of image processing in a second execution environment, after the new image processing is requested, obtaining a memory usage amount at the time of execution of the new image processing of the second application, which is held together with the second application, and memory usage amounts of the respective image processing already in operation in the second execution environment;
comparing a total memory usage amount obtained from the obtained memory usage amounts of the respective image processings and the memory usage amount when the requested new image processing is executed, with an allowable memory capacity indicating a memory usage amount allowed to be used in the second execution environment; and
performing new image processing in the second execution environment in a case where the comparison result is that the total memory usage does not exceed the allowable memory capacity, and notifying the request source of an error indicating that the new image processing cannot be performed due to a shortage of memory in a case where the total memory usage exceeds the allowable memory capacity without performing the new image processing;
wherein, in a case where the image processing of the first application is executed in the first execution environment, the image processing is not controlled based on the permitted memory capacity indicating the memory usage amount permitted to be used in the second execution environment.
CN201910405589.6A 2018-05-17 2019-05-16 Image processing apparatus, control method thereof, and computer-readable storage medium Active CN110572527B (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2018095632 2018-05-17
JP2018-095632 2018-05-17
JP2018150641A JP7186541B2 (en) 2018-05-17 2018-08-09 IMAGE PROCESSING DEVICE, CONTROL METHOD THEREOF, AND PROGRAM
JP2018-150641 2018-08-09

Publications (2)

Publication Number Publication Date
CN110572527A CN110572527A (en) 2019-12-13
CN110572527B true CN110572527B (en) 2021-09-10

Family

ID=68727125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910405589.6A Active CN110572527B (en) 2018-05-17 2019-05-16 Image processing apparatus, control method thereof, and computer-readable storage medium

Country Status (3)

Country Link
JP (1) JP7186541B2 (en)
KR (1) KR102543344B1 (en)
CN (1) CN110572527B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101017477A (en) * 2006-02-10 2007-08-15 富士施乐株式会社 Image formation processing simulation apparatus and image formation processing simulation method
CN101610337A (en) * 2008-06-19 2009-12-23 佳能株式会社 The EMS memory management process of image processing apparatus and image processing apparatus
CN104243764A (en) * 2013-06-11 2014-12-24 株式会社理光 Image processing system, information processing device and information processing method
CN104461655A (en) * 2014-12-29 2015-03-25 北京网秦天下科技有限公司 Method for automatically selecting application installation position and mobile terminal
CN106648747A (en) * 2016-11-14 2017-05-10 腾讯科技（深圳）有限公司 Method and device for processing pre-loading of process
CN107659746A (en) * 2016-07-26 2018-02-02 佳能株式会社 The control method and recording medium of image forming apparatus, image processing equipment
CN107977242A (en) * 2016-10-21 2018-05-01 联发科技股份有限公司 Application program launching method and electronic device

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4003679B2 (en) * 2003-03-31 2007-11-07 ブラザー工業株式会社 Image copying machine
JP2006186777A (en) * 2004-12-28 2006-07-13 Kyocera Mita Corp Apparatus and program for image formation
JP4696748B2 (en) * 2005-07-25 2011-06-08 富士ゼロックス株式会社 Image forming system, interrupt processing method, and printing apparatus
JP5167897B2 (en) * 2007-04-27 2013-03-21 株式会社リコー Electronic device, information processing method, and information processing program
JP4986144B2 (en) * 2007-05-25 2012-07-25 富士ゼロックス株式会社 Printing control apparatus and image forming system
JP4561883B2 (en) * 2008-06-19 2010-10-13 コニカミノルタビジネステクノロジーズ株式会社 Image forming apparatus, program, and image forming processing method
JP2011242890A (en) * 2010-05-14 2011-12-01 Hitachi Ltd Request processing method, request processing program and request processing apparatus
JP5644269B2 (en) * 2010-08-30 2014-12-24 株式会社リコー Image processing apparatus, image processing system, image processing method, and image processing program
JP2014149606A (en) * 2013-01-31 2014-08-21 Fujitsu Ltd Resource usage totaling program, resource usage totaling method and resource usage totaling device
JP6418006B2 (en) * 2015-02-27 2018-11-07 ブラザー工業株式会社 Facsimile apparatus and printing method of facsimile apparatus
JP2017027467A (en) * 2015-07-24 2017-02-02 富士ゼロックス株式会社 Management device, information processing system and program

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101017477A (en) * 2006-02-10 2007-08-15 富士施乐株式会社 Image formation processing simulation apparatus and image formation processing simulation method
CN101610337A (en) * 2008-06-19 2009-12-23 佳能株式会社 The EMS memory management process of image processing apparatus and image processing apparatus
CN104243764A (en) * 2013-06-11 2014-12-24 株式会社理光 Image processing system, information processing device and information processing method
CN104461655A (en) * 2014-12-29 2015-03-25 北京网秦天下科技有限公司 Method for automatically selecting application installation position and mobile terminal
CN107659746A (en) * 2016-07-26 2018-02-02 佳能株式会社 The control method and recording medium of image forming apparatus, image processing equipment
CN107977242A (en) * 2016-10-21 2018-05-01 联发科技股份有限公司 Application program launching method and electronic device
CN106648747A (en) * 2016-11-14 2017-05-10 腾讯科技（深圳）有限公司 Method and device for processing pre-loading of process

Also Published As

Publication number Publication date
KR102543344B1 (en) 2023-06-14
KR20190132218A (en) 2019-11-27
JP2019204477A (en) 2019-11-28
CN110572527A (en) 2019-12-13
JP7186541B2 (en) 2022-12-09

Similar Documents

Publication Publication Date Title
KR102348317B1 (en) Image processing apparatus, method for controlling the same, and non-transitory computer-readable storage medium
US11140291B2 (en) Information processing apparatus, control method thereof, and storage medium
US7895373B2 (en) Electronic device for data access management
CN109660688B (en) Information processing apparatus and control method thereof
JP2018015947A (en) Image formation apparatus, image formation method and program
US10922039B2 (en) Image processing apparatus, information processing method, and storage medium
JP7402643B2 (en) Information processing device, its control method, and program
JP6961412B2 (en) Image forming device, control method of image forming device, and program
CN110572527B (en) Image processing apparatus, control method thereof, and computer-readable storage medium
US9691010B2 (en) Information processing apparatus, distributed printing system, and method of controlling printing
US10992834B2 (en) Image processing apparatus, method for controlling the same, and computer-readable storage medium
JP7282507B2 (en) Information processing device, server device, control method, control program
JP7080061B2 (en) Image processing equipment
JP7102248B2 (en) Image processing equipment, information processing methods and programs
US20170214813A1 (en) Image forming apparatus that executes received jobs, control method therefor, and storage medium
JP2005173888A (en) Information processor, method and program for restoring program, and recording medium
JP2010097366A (en) Electronic apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant