JP5709417B2 - Image processing apparatus, control method therefor, and program - Google Patents

Description

  The present invention relates to an image processing apparatus having an image processing means by hardware processing and an image processing means by software processing, and a control method thereof.

  Conventionally, when processing image data, an image processing unit using hardware processing and an image processing unit using software processing are properly used (see Patent Document 1).

JP 2003-78769 A

  The one described in Patent Document 1 uses hardware processing and software processing in a fixed manner depending on the application. When processing image data, depending on the situation, there are cases in which the processing can be completed earlier as a result of performing software processing than waiting until hardware processing is possible. However, conventionally, since hardware processing and software processing are fixedly used, the processing speed is not necessarily efficient.

  The present invention has been made in view of the above points, and provides an image processing apparatus capable of efficiently using an image processing means by hardware processing and an image processing means by software processing, a control method thereof, and a program To do.

In order to solve the above problems, an image processing apparatus of the present invention includes a first image processing unit that performs image processing on image data by hardware processing, a second image processing unit that performs image processing on image data by software processing, When the first image data is determined as a printing target, a determination unit that determines to process the first image data using the first image processing unit, and a printing process of the first image data If the reservation instruction print instruction before exiting from the start is accepted, the first image display means for displaying data, said display means subject to the image data of the first image displayed by and from said data first image data and detecting means for detecting a reading instruction for changing to a different second image data, when the reading instruction has been detected, the second image And determining means for determining whether or not the data includes a thumbnail image, and when the determining means determines that the second image data includes a thumbnail image, the determining means includes: The thumbnail image is determined to be processed by the second image processing unit, and the display unit determines the second image processing unit as a processing result of the second image processing unit before printing of the first image data is completed. Image data is displayed.

  According to the present invention, since hardware processing or software processing is determined based on the processing content of the image data and the content of the image data to be processed, each can be used efficiently. It becomes.

It is a block diagram which shows the structure of the image processing apparatus of embodiment. It is a figure which shows the structure of the program of an image processing apparatus. It is a flowchart which shows the flow of a process by an image processing apparatus. It is a flowchart which shows the flow of a process by an image processing apparatus. It is a flowchart which shows the flow of a process by an image processing apparatus. It is a flowchart which shows the flow of a process by an image processing apparatus. It is a flowchart which shows the flow of a process by an image processing apparatus. It is a figure which shows the processing speed with the case where it does not perform when the process of this embodiment is performed. It is a figure which shows the processing speed with the case where it does not perform when the process of this embodiment is performed. It is a flowchart which shows the flow of a process by an image processing apparatus. It is a flowchart which shows the flow of a process by an image processing apparatus.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to the present embodiment.

  In FIG. 1, an operation unit 101 includes hard keys and soft keys and accepts various operations by a user. The card interface 102 can set a memory card, and can input image data stored in the set memory card in accordance with an instruction from the operation unit 101. The reading unit 103 includes a CCD, which reads an original image and outputs analog luminance data of red (R), green (G), and blue (B) colors. The printing unit 104 prints an image by applying a recording agent such as ink or toner on the recording sheet according to the input image data.

  The CPU 200 loads a control command program, an image processing program, and the like stored in the ROM 201 into the RAM 202 and executes them to control the operation of the image processing apparatus. The JPEG decompression hardware 203 is a hardware module that can decompress JPEG-compressed image data. The JPEG decompression hardware 203 can search for JPEG-compressed data, collect information necessary for decompression and rotation, etc., and can also decompress the image data in units of 90 degrees. The display unit 204 can display various information to be notified to the user, such as the operation state of the operation unit 101 and various states of the image processing apparatus.

  FIG. 2 is a diagram schematically showing the relationship between each configuration shown in FIG. 1 and processing executed by a program stored in the ROM 201.

  The application control unit 301 is a control program that issues a job in response to a user instruction input via the operation unit 101. The print job control unit 302 is a control program that controls processing for causing the printing unit 104 to print the generated image data. The display job control unit 303 is a control program that controls processing for causing the display unit 204 to display generated image data. The quantization processing unit 304 is an image processing program that performs halftone processing on image data in a form suitable for the form to be printed by the printing unit 104. The JPEG processing unit 305 is an image processing control program that controls the JPEG decompression hardware 203 and the JPEG decompression software 308. The color conversion processing unit 306 is an image processing program that performs color conversion of image data. The scaling processing unit 307 is an image processing program that performs scaling of image data. The JPEG decompression software 308 is an image processing program having a function of decompressing compressed data in a JPEG image. The JPEG decompression software 308 also has a JPEG analysis / decompression function that retrieves all compressed data in a JPEG image and collects information necessary for rotational decompression. The JPEG decompression software 308 can decompress the image data by rotating it 90 degrees after the JPEG decompression software 308 has executed the JPEG analysis / decompression function. The JPEG analysis / decompression function is a function that searches all compressed data in a JPEG image and collects information necessary for rotational expansion. The JPEG processing unit 305, the color conversion processing unit 306, and the scaling processing unit 307 are shared so that they can be called from both the print job control unit and the display job control unit. Although not shown, the application control unit 301, the print job control unit 302, the display job control unit 303, and the JPEG processing unit 305 are configured to be able to read an image file via the card interface 102.

  When the CPU 200 controls the image processing apparatus, all or part of the program group shown in FIG. 2 is loaded into the RAM 202 and executed. The flowcharts shown below show the processing flow based on this program.

  FIG. 3 shows a flow of processing by the application control unit 301 when controlling a printing application from the memory card.

  First, in step S401, the memory card is accessed via the card interface 102, a stored JPEG image file is searched, an image ID is generated for each image file, and a file list with an image file name added is generated. In the present embodiment, when an image file is searched, an image ID (1 to N) is assigned to each image file, and a file list in which the image ID is associated with the image file is generated on the RAM 202. Remember. Note that the image file handled in the present embodiment is an Exif format JPEG file including various types of information captured by a digital still camera.

  In step S <b> 402, a display job for the image with the image ID = 1 is issued to the display job control unit 303. Thereafter, the determinations in S403 and S404 are repeated in order to receive a user instruction via the operation unit 101. In S403, it is confirmed whether or not the user has instructed browsing of the image next to the image data currently being displayed or being displayed. If the next image browsing instruction is detected by the user, in S402, a display job for the image with the image ID = 2, which is the next image on the file list, is issued to the display job control unit 303, and again. Wait for instructions in S403 and S404. If an instruction to browse the next image is not detected in S403, it is detected in S404 whether or not a print image is designated by the user. When the print image is designated by the user, the image with the image ID = 2 currently being displayed or being displayed is determined as a print target, and S404 is executed.

  In step S <b> 405, a print setting instruction is received from the user. In this embodiment, it is assumed that the number of print copies can be set. Hereinafter, a process when the number of copies is set to 3 in S405 will be described. In step S <b> 406, the print job is issued to the print job control unit so that three copies of the image with the image ID = 2 are printed. Furthermore, the user reserves a print instruction for another image, checks whether or not there is a reservation instruction, and repeats S407 and S408 in order to detect the end of a print job being executed or reserved. If it is determined in S407 that there is no reservation instruction, S408 is executed. In S408, if there is a print job currently being executed or reserved, the process S407 is executed. Next, when a reservation instruction is detected in process S407, process S403 is executed. At this stage, the display unit 204 displays a screen indicating that the image data of image ID = 2 that last issued the display job or the image ID = 2 is being displayed. Next, when an instruction to browse the next image data is detected in step S403, a display job control unit displays a display job for the image with the image ID = 3 that is the next image on the file list in S402. The process waits for instructions in S403 and S404 again. Thereafter, when the user designates an image with an image ID = 3 as a print target in S405 and is set to print two copies in S405, the image ID == to the print job control unit in S406. A print job is issued so that two copies of the three images are printed. Thereafter, if there is no reservation instruction from the user, S407 and S408 are repeated. Three copies of image ID = 2 for which print jobs have been issued so far are printed, and when two copies of image ID = 3 have been printed, the end of all print jobs is detected in S408, and S403 is executed. At this stage, the display unit 204 displays a screen indicating that the image data with the image ID = 3 that lastly issued the display job or the image data with the image ID = 3 is being displayed. When the control of the application control unit 301 is configured as described above, the user can make a reservation instruction to print a different image even when the image is being printed.

  FIG. 4 is a flowchart showing the flow of processing of the display job control unit 303 that has received the display job issued in S402 in FIG. Here, an example is shown in which after a display job for image ID = 1 is received first, an image display job for image ID = 2 is received, and finally a display job for image ID = 3 is received.

  In step S601, it is determined whether there is a display job currently being executed. Since there is no other display job being executed during the execution of image ID = 1, which is the first request, the processes after S603 are sequentially executed. In step S603, the display unit 204 displays a screen indicating that image display processing is being performed. In this embodiment, the image file name of image ID = 1 is displayed on this screen. Next, the processing after S604 is executed. When the next image display job with the image ID = 2 is received during the subsequent processing execution, it is determined that there is a display job being executed in S601, and S602 is executed. Run. In step S602, control is performed to immediately end the image display job with the image ID = 1. In the following description, it is assumed that a job for image ID = 2 has been received.

  In step S603, the image file name with the image ID = 2 is displayed, and the process S604 is executed. In S604, the header portion of the image data corresponding to the image ID = 2 is read from the card interface 102 to a predetermined area of the RAM 202. In S605, the header portion of the image data corresponding to the image ID = 2 is analyzed, and information such as the image size and quantization table necessary for JPEG expansion is collected. Also, many of the JPEG images taken with a digital still camera have thumbnail image data compressed with JPEG inside the header of the image file. Therefore, in S605, it is also detected whether or not a thumbnail image is added to the image file. If a thumbnail is added, information necessary for thumbnail expansion is also collected. Here, it is assumed that a thumbnail image is added to the image file with the image ID = 2.

  In S606, the next process is switched using the presence / absence of the thumbnail detected in S605 as a determination material. If it is an image file to which no thumbnail is added, S607 is executed, but since it is determined that there is a thumbnail for image ID = 2, S608 is executed. In step S608, a JPEG decompression input buffer is secured in the RAM 202, and the thumbnail image in the file with the image ID = 2 is read from the card interface 102 and stored in the JPEG decompression input buffer. Further, the fact that the thumbnail image has been read is stored. It is assumed that the JPEG decompression input buffer secures an area where at least all compressed thumbnail images can be stored. In step S609, it is determined whether JPEG decompression is to be executed by software or hardware.

  FIG. 5 is a flowchart showing a flow of processing for determining whether to decompress an input JPEG image by software processing or hardware processing.

  In S701, information indicating whether or not the thumbnail image is stored in the JPEG decompression input buffer in S608 in FIG. 4 is referred to. If the thumbnail is stored, S703 is executed. In S703, the soft decompression process is selected. That is, since it is known that the image data to be processed is a thumbnail image and its size is small (not large), it is determined to perform processing by software. On the other hand, if the main image is stored in S701, S702 is executed. In S702, the hard decompression process is selected. That is, when the image data to be processed is not a thumbnail image, the image data is likely to be large, and the processing is performed by hardware. When the processing content of the image data is display as described above, the content of the image data to be processed is determined, and whether to perform hardware processing or software processing is determined according to the determination result. This time, in step S608, since the thumbnail image in the file with the image ID = 2 is taken in, software execution is selected in step S609.

  In step S610, since execution by software is selected in step S609, control is performed so that step S611 is executed to execute processing by software. In step S611, the JPEG processing unit 305 is requested to perform JPEG decompression using the JPEG decompression software 308 and waits for the request to be completed. Note that, depending on the image size after decompression, there may be a case where it is difficult to secure an area enough to store the entire image data after decompression. Therefore, in the present embodiment, the number of lines that can store the decompressed image data is N units. A processing request shall be made.

  In step S613, the image data is enlarged or reduced to a size suitable for the size and resolution of the image display area in the display unit 204. In step S <b> 614, the color conversion processing unit 306 performs color conversion into a format suitable for the display unit 204. The color conversion processing unit 306 performs conversion into the device color space of the display unit using, for example, matrix calculation or table reference.

  Subsequently, in S615, it is determined whether the entire image data (thumbnail image with image ID = 2) selected this time has been processed. If unprocessed data remains, S610, S611, S613, S614 are determined. , S615 are sequentially executed.

  As a result of repeatedly executing the image processing as described above, if it is determined in S615 that the entire image data has been processed, S616 is executed. In step S <b> 616, the image processed image data is transferred to the display unit 204, and the image processed image data is displayed on the display unit 204.

  In step S617, it is determined whether the currently displayed image is a thumbnail image. Since the image data displayed this time is a thumbnail image with the image ID = 2, control is performed so as to execute S607 next. In S607, a JPEG decompression input buffer is secured in the RAM 202, and main image data in the file with the image ID = 2 is read from the card interface 102 and stored in the JPEG decompression input buffer. When the size of the main image data is larger than the size that can be secured by the JPEG decompression input buffer, it is assumed that the main image data is read from the beginning to the size of the JPEG decompression input buffer.

  Next, in S604, since it is determined that the main image is to be expanded in S701 in FIG. 5, execution by hardware is selected. Subsequently, in S610, since execution by hardware is selected in S609, control is performed so that S612 is executed in order to execute processing by hardware. In step S612, the JPEG processing unit 305 is requested to perform JPEG decompression of the number N of lines that can be accommodated in the decompressed image data storage area using the JPEG decompression hardware 203, and waits for the request to be completed. Thereafter, as in the case of displaying thumbnail images, S613, S614, and S615 are executed, and a series of image processing is repeatedly executed on the main image data until the entire image processing is completed.

  In step S <b> 616, the image processed image data is transferred to the display unit 204, and the image processed image data is displayed on the display unit 204. In step S617, it is determined whether the currently displayed image is a thumbnail image. Since the image data displayed this time is the main image with the image ID = 2, the display job for the image ID = 2 received this time is ended.

  In the above example, a display job for the image ID = 3 is received next to the image ID = 2, but the subsequent processing is the same as that described above using the image ID = 2. By configuring the display job control unit 303 as described above, when a thumbnail is included in the image file to be displayed, a series of image processing is finished and the thumbnail image is temporarily displayed on the display unit 204. . Thereafter, a series of image processing is executed on the main image, and the main image is switched to be displayed. Even when processing of the main image data takes time and it takes time until display, if the thumbnail image is displayed at high speed, the user can browse the image to be printed. In addition, since switching between processing by hardware and processing by software does not occur in one image, color unevenness does not occur in a display image even if there is a calculation error between hardware and software.

  FIG. 6 is a flowchart showing a processing flow of the print job control unit 302 that has received the print job issued in S406 in FIG. In this example, it is assumed that after a print job for image ID = 2 is first received, a print job for image ID = 3 is received.

  First, in step S801, the accepted print job is temporarily registered in the print job queue. In step S802, it is determined whether there is a print job currently being executed. When a print job for the first received image ID = 2 is requested, since there is no print job currently being executed, control is performed to execute S803. On the other hand, if a print job execution request for image ID = 3 is received during execution of a print job for image ID = 2, the print job for image ID = 2 is continuously executed after registration in the print job queue in S801. Return control to

  In the following description, it is assumed that a print job for image ID = 2 is being executed. In step S803, the print job corresponding to the image ID = 2, which is the oldest print job, is acquired from the print job queue, and the print setting (here, the number of copies = 3) is acquired. In step S804, the header portion of the image data corresponding to the image ID = 2 is read from the card interface 102 into a predetermined area of the RAM 202. Further, the header portion of the image data corresponding to S805 image ID = 2 is analyzed, and information such as the image size and quantization table necessary for JPEG decompression is collected. Here, in the header analysis of the print job, only the information necessary for decompressing the main image is collected without determining whether or not the thumbnail exists.

  In step S806, a JPEG decompression input buffer is secured in the RAM 202, and the main image in the file with the image ID = 2 is read from the card interface 102 and stored in the JPEG decompression input buffer. When the size of the main image data is larger than the size that can be secured by the JPEG decompression input buffer, it is assumed that the main image data is read from the beginning to the size of the JPEG decompression input buffer.

  Next, the process proceeds to JPEG image expansion processing. Most of the image data captured by the digital still camera is stored as landscape image data as a landscape image. On the other hand, in the present embodiment, the printing unit 104 feeds paper in the longitudinal direction of printing paper during printing. Therefore, it is necessary to rotate and expand the image data to be printed by 90 degrees. In step S807, the JPEG processing unit 305 is requested to perform JPEG analysis / decompression with the JPEG decompression hardware 203 and waits for the processing to end. Next, in step S808, a request is made to output data after the 90-degree rotation and expansion have been performed using the JPEG expansion hardware 203, and the request is waited for completion.

  In step S <b> 809, the scaling processing unit 307 enlarges or reduces the image data to a size suitable for the size of the printing paper 902 and the printing resolution of the printing unit 104. In the present embodiment, the printing unit 104 performs printing processing in units of 16 lines. Therefore, in S808, it is assumed that a processing request is made in units of M lines, which is the number of lines necessary and sufficient for the scaling processing unit 307 to output 16 lines, and the scaling processing unit 307 outputs 16 lines of data in S809. It shall be. In step S <b> 810, the color conversion processing unit 306 performs color conversion so that the color space is suitable for the printing unit 104. For example, conversion to the device color space of the printing unit 104 is performed using matrix calculation or table reference.

  Next, in S811, a quantization process for converting the image data into a dot pattern suitable for the printing unit 104 is performed. In step S812, data is transmitted to the printing unit 104 so as to print the dot data for 16 lines generated in step S811. In step S813, it is detected whether or not printing of all image data has been completed. If unprinted data remains, it is determined to execute a series of processes from step S808 to step S812. When printing of all the image data is completed, it is checked in S814 whether or not printing of all the numbers has been completed. If printing of all the numbers has been completed, control is performed so that the processing from S804 is executed for the remaining number of copies. I do. Since the image ID = 2 is requested to be printed in three copies, all the printing is completed by performing the processing from S804 onward three times, and S815 is executed.

  In step S815, the print job queue is referenced to check whether there is a print job. If a print job remains, control is performed so that the processing in step S803 and subsequent steps is continued until there is no print job. When the print job with the image ID = 2 is received before the print job with the image ID = 2 arrives at S815, the print job exists in the print job queue, and thus S803 is executed. In step S803, the print job with the image ID = 3 is taken out of the print job queue, and the subsequent processing is executed until the requested two copies are completed.

  By configuring the print job control unit 302 as described above, the application control unit 301 can issue a print job regardless of whether the print job control unit 302 is executing. The print job control unit 302 can take out a print job stored in the print job queue and start printing as soon as the printing process being executed is completed. In addition, since switching between processing by hardware and processing by software does not occur in one image, color unevenness does not occur in a printed image even if there is a calculation error between hardware and software.

  FIG. 7 is a flowchart showing the flow of processing by the JPEG processing unit 305.

  When receiving the processing request, the JPEG processing unit 305 first determines in S1001 whether the request is instructed to be executed by hardware or software. If an instruction is given to execute the processing by hardware, control is performed so that the processing after S1002 is executed in order to execute the processing by hardware. When instructed to be executed by software, control is performed to execute the processing from S1002 onward in order to execute the processing by software.

  First, the processing when instructed to be executed by software will be described from S1002. In the processing of the display job control unit 303 described above, when it is determined that there is a thumbnail in S606, the case where soft JPEG expansion is requested in S611 corresponds.

  When JPEG processing is executed by software, it is determined in step S1002 whether an analysis / decompression request is received or an expansion request is received. If an analysis / decompression request is received, an analysis / decompression request is made to the JPEG decompression software 308 in S1004, and the process waits until a response is returned. On the other hand, if a normal decompression request is made, a decompression request is made to the JPEG decompression software 308 in S1003 and waits until a response is returned. The JPEG decompression software 308 sequentially performs analysis decompression processing or decompression processing on the data in the JPEG decompression input buffer, and when the data on the JPEG decompression input buffer is exhausted, requests input data necessary for continued processing from the caller. On the other hand, when the process is completed without running out of data on the JPEG decompression input buffer, the caller is notified of the completion.

  In the present embodiment, the JPEG decompression software 308 is used only for the thumbnail image decompression process described above. Since the compressed data of the thumbnail image is requested in a state where it has already been read into the JPEG decompression input buffer, the input data is never requested from the caller.

  If there is a response from the JPEG decompression software 308, S1005 is executed. In S1005, it is determined whether the response from the JPEG decompression software 308 is an input data request or the processing is completed. If there is an input data request, S1006 is executed. In step S1006, input data necessary for the continuation process designated from the JPEG decompression software 308 is read from the card interface 102 and stored in the JPEG decompression input buffer. On the other hand, if the response from the JPEG decompression software 308 is processing completion in S1005, the processing is terminated and the calling source is notified of the termination.

  Subsequently, the processing in the case where an instruction to execute hardware is given will be described from S1007. In S1007, it is determined whether or not hardware resources are released. If hardware resources have not been released, continue to wait until hardware resources are released. On the other hand, if the hardware resource is released, the hardware resource is acquired in S1008. Even after another hardware expansion request is received after the acquisition of hardware resources in S1008 until the release, the release is kept waiting in S1007. In step S1009, it is determined whether an analysis expansion request has been received or an expansion request has been received.

  If an analysis decompression request is received, an analysis decompression request is made to the JPEG decompression hardware 203 in S1011 and waits until a response is returned. On the other hand, if a normal decompression request has been made, in S1010, a decompression request is made to the JPEG decompression hardware 203 and waits until a response is returned. The JPEG decompression hardware 203 sequentially analyzes and decompresses the data in the JPEG decompression input buffer, and when the data on the JPEG decompression input buffer is exhausted, requests the caller for input data necessary for continued processing. On the other hand, when the process is completed without running out of data on the JPEG decompression input buffer, the caller is notified of the completion.

  If there is a response from the JPEG decompression hardware 203, S1012 is executed. In S1012, it is determined whether the response from the JPEG decompression hardware 203 is an input data request or the processing is complete. If there is an input data request, S1013 is executed. In step S1013, input data necessary for the continuation process designated from the JPEG decompression hardware 203 is read from the card interface 102 and stored in the JPEG decompression input buffer. On the other hand, if the response from the JPEG decompression hardware 203 is completed in S1012, the processing is terminated after releasing the hardware resources in S1014 and the termination is notified to the caller.

  By configuring the JPEG processing unit 305 as described above, it is possible to appropriately divide and execute processing regardless of whether it is a processing request by hardware or a processing request by software. Further, since hardware resource management is appropriately performed, the JPEG decompression hardware 203 need only have a function that always satisfies one processing request.

  Next, the processing efficiency by the processing of this embodiment will be described.

  It is assumed that the processing capability (processing speed) of the decompression processing or analysis decompression processing of the JPEG decompression hardware 203 takes 0.01 seconds for a compressed image data size of 50 Kbytes. The processing capability of the JPEG decompression software 308 is about 1/10 that of JPEG decompression hardware, and the compressed image data size takes 0.1 second per 50 Kbytes.

  FIG. 8 is a timing chart showing the JPEG decompression hardware, the JPEG decompression software, and the time required for processing each job, and is a diagram showing the control by this embodiment and the others. FIG. 8A shows the behavior when control is performed so that the JPEG decompression hardware is always used. FIG. 8B shows the behavior by the control of this embodiment described above. Also, these are the situations when the display job control unit 303 issues a JPEG expansion request for a thumbnail image for image ID = 3 immediately after the print job control unit 302 makes a JPEG analysis / decompression request for image ID = 2. Show.

  The size of the main image data with the image ID = 2 is 5 Mbytes, the size of the thumbnail image data with the image ID = 3 is 50 Kbytes, and the size of the main image data is 5 Mbytes. Each of T1101 to T1105 represents the time when an event described later occurs. Also, in FIGS. 8 (a) and 8 (b), the same reference numerals are assigned to the same times.

  T1101 represents the time at which a JPEG analysis / decompression request is made for the main image with the image ID = 2 in the print job. T1102 represents the time when a JPEG decompression request is made for a thumbnail image with an image ID = 3 in a display job. T1103 represents the time when the processing for the JPEG analysis / decompression request for the main image with the image ID = 2 is completed.

  In FIG. 8A, since the hardware is not released at the time of receiving the JPEG decompression request from the display job control unit, the process waits until the hardware is released, that is, until the time of T1103. Become. Since the main image size of the image ID = 2 is 5 Mbytes and the processing capacity of the image expansion hardware is 0.01 seconds per 50 Kbytes, in the worst case, JPEG expansion of the display job is waited for about 1 second. Thereafter, the release of JPEG decompression hardware is detected, image decompression is performed immediately, and the decompression process ends at time T1104. The time taken from T1103 to T1104 is as fast as about 0.01 seconds, but due to the release waiting time that is the preceding stage, the processing for the display request takes more than 1 second.

  On the other hand, in FIG. 8B, when a JPEG decompression request is received from the display job control unit, control is performed so that a thumbnail image software decompression request is immediately made. Therefore, the display process is performed regardless of the release status of the JPEG decompression hardware, and the decompression process ends at the time indicated by T1105. Since the thumbnail size of the image ID = 3 is 50 Kbytes and the processing capacity of the image expansion software is 0.1 second per 50 Kbytes, the time taken from T1102 to T1105 is about 0.1 seconds. Although the processing itself is slower than when hardware is used, the thumbnail image data size is small, so that it is possible to expand the image for display without causing stress to the user.

  FIG. 9 is a timing chart showing the time required for processing JPEG decompression hardware, JPEG decompression software, and a print job, and is a diagram showing the control by this embodiment and the others. In FIG. 9A, without performing the control as in the present embodiment, the hardware processing and the software processing are fixedly used according to the usage, or the hardware release status is confirmed and the software processing is executed. Shows the behavior of the case. FIG. 9B shows the behavior by the control of this embodiment described above. FIG. 9 differs from FIG. 8 in that the display job control unit 303 issues a JPEG decompression request for the main image for image ID = 3 immediately after the print job control unit 302 makes a JPEG analysis / decompression request for image ID = 2. If you are in a situation. Each of T1201 to T1205 represents the time when an event described later occurs. This represents the time when the JPEG analysis / decompression request for the main image with the image ID = 2 is made. T1202 represents the time at which a JPEG decompression request is made for the main image with the image ID = 3 in the display job. T1203 represents the time when the processing for the JPEG analysis / decompression request for the main image with the image ID = 2 is completed.

  In FIG. 9A, since the hardware has not been released when the JPEG decompression request is received from the display job control unit, control is performed so as to be executed by software. Alternatively, it is predetermined to use software for display purposes. In any case, when a JPEG decompression request is received from the display job control unit, control is performed so that a software decompression request for the main image is immediately made. Then, the software process ends at time T1204. The time from T1202 to T1204 takes approximately 10 seconds from the processing capability. If no thumbnail is added to the image ID = 3, the user cannot view the selected image, which is very stressful.

  On the other hand, in FIG. 9B, after waiting for hardware to be released, processing by hardware is executed. For this reason, even in the worst case, the expansion process is started after waiting for 1202 to T1203, that is, about 1 second. Thereafter, the process ends at time T1205 when the decompression process ends. Since the time from T1203 to T1205 is about 1 second, the expansion process for display is completed in about 2 seconds including the hardware release waiting time. Compared to 10 seconds in FIG. 9A, the user's stress is significantly reduced.

  Next, an example in which JPEG decompression is performed by software processing or hardware processing is determined by a method different from the above-described example (FIG. 5 and the like). FIG. 10 is a flowchart showing the flow of processing in this case. The process of this flowchart is called and executed from S609 of FIG. In FIG. 10, in S1301, the size of the image data to be processed is compared with a predetermined threshold value, and if it is equal to or smaller than the threshold value, software processing is selected, and if larger than the threshold value, hardware processing is selected. Thereafter, the process selected as described above is executed.

  This threshold value may be determined based on, for example, the time during which the user does not feel stress and the processing capability of the software processing. For example, even if the processing is performed by software, the speed at which the user cannot feel stress is 0.5 seconds, and the processing capability of the JPEG decompression software 308 is 0.1 seconds per 50 KB of compressed image data. In such a case, the threshold value is 250 KBytes.

  In the processing of FIG. 10, if the size of the image data to be subjected to JPEG expansion processing is 250 KBytes or less, it is immediately executed by the JPEG expansion software 308. Therefore, regardless of the hardware occupation status, the decompression process is completed within 0.5 seconds, which is known not to give stress to the user.

  If the size is larger than 250 KBytes, it is executed by the JPEG decompression hardware 203. Therefore, although the time for releasing the JPEG expansion hardware 203 is added, it is possible to prevent the JPEG expansion software 308 from executing huge data.

  That is, in the example of FIG. 5, the processing method is switched depending on whether or not the image to be processed is a thumbnail image. However, in the example of FIG. To do. Therefore, the processing efficiency can be improved with respect to the decompression process as compared with the example of FIG.

  In the above calculation of the threshold value, it is determined that the processing capability of the JPEG decompression software 308 is always 100%, but a predetermined value may be added, subtracted, multiplied, or divided. For example, a method can be considered that takes into account the competition of the CPU 200 executing JPEG decompression software. Since the print job and the display job are executed at the same time, if the time that the CPU 200 allocates to the display job is halved, a method of setting 125 Kbytes obtained by adding 1/2 to 250 Kbytes as a threshold value can be considered.

  Next, an example will be described in which JPEG decompression is executed by software processing or hardware processing to determine whether to execute the processing by a different method. FIG. 11 is a flowchart showing the flow of processing in this case. The process of this flowchart is called and executed from S609 of FIG.

  In the example of FIG. 11, first, in S1401, a predicted value of the remaining occupation time of the occupied hardware is acquired. Here, it is assumed that the predicted value of the occupation time is obtained from the unprocessed data size of the hardware occupied by the print job. Note that the processing capabilities of the JPEG decompression software 308 and the JPEG decompression hardware 203 are the same as those described above, for example. For example, if the JPEG decompression hardware 203 has finished processing the 3 MByte data out of the 5 MByte data, the time required to process the remaining 2 MByte data is 0.4 seconds. When hardware is not occupied, the time is 0 second.

  Next, in S1402, the threshold used in S1403 is obtained. For example, the threshold is calculated from the sum of the predicted value of the decompression processing time by the JPEG decompression hardware 203 obtained from the size of the image data to be processed and the predicted value of the time required to release the JPEG decompression hardware 203 acquired in S1401. Find it. For example, when decompressing 100 Kbytes of data, it takes 0.02 seconds due to the processing capability of the JPEG decompression hardware 203. That is, 0.42 seconds obtained by adding the hardware release waiting time of 0.4 seconds and the hardware execution time of 0.02 seconds is the time required for processing when the JPEG decompression hardware 203 is used. The JPEG decompression software 308 can process 210 Kbytes in 0.42 seconds because of its processing capability, so 210 Kbytes is set as a threshold value.

  In step S1403, the size of the input data is compared with the threshold value obtained in step S1402. If the input data size is equal to or smaller than the threshold value, software processing is selected, and if larger than the threshold value, hardware processing is selected. Thereafter, the process selected as described above is executed.

  As described above, in the example of FIG. 11, it is determined whether to perform software processing or hardware processing in consideration of the time when the JPEG decompression hardware 203 is released. Therefore, the processing delay time is shortened, and software and hardware can be used more efficiently depending on the situation. Here, only the remaining occupation time of the occupied hardware is considered in obtaining the threshold value, but the time required for reading the image data may be added. As shown in FIG. 10, the image data is read in S1013 while the hardware resource is acquired, and the image data read time is added to the hardware occupation time. If a job that is occupied by hardware does not read all the image data, the time required for reading the unread data can be obtained from the unread data size and the file reading capability, and used for threshold calculation. In this case, the time required for reading the unread data may be added to the remaining occupied time of the occupied hardware obtained in S1401. For example, when the processing time of the hardware is sufficiently high and only the time required for reading the image data becomes a problem, only the unread data size of the job occupied by the hardware may be used. Furthermore, in order to achieve a simpler configuration, switching between hardware and software may be performed only by determining whether there is unread data. For example, when a job occupied by hardware has unread data, a method of executing software processing can sufficiently achieve the effect.

  As described above, the examples of FIGS. 5, 10, and 11 are given as the method for determining whether JPEG decompression is software processing or hardware processing. However, these may be combined as appropriate. That is, for example, by combining FIG. 5 and FIG. 10, it is first determined whether or not the image data to be processed is a thumbnail image (S701). If it is a thumbnail image, the process proceeds to S702, and if it is not a thumbnail image, the process proceeds to S1301. In step S1301, the size of the image data is determined, and the process proceeds to step S1302 or S1303 according to the result. 5 and FIG. 11 are combined, and if it is determined in S701 that the image is not a thumbnail image, the process proceeds to S1401, and the same processing is performed. Further, FIGS. 5, 10, and 11 may be combined. That is, if it is determined in S701 that the image is not a thumbnail image, the process proceeds to S1301, and the size of the image data to be processed is compared with the first threshold value. Similar processing is performed in comparison with the second threshold. In addition to these, various conditions may be added or changed, and hardware processing and software processing may be used properly. Also, JPEG decompressed data is not limited to image display and printing, but can be applied to various processes.

  In the above description, an example in which software processing and hardware processing are selectively used for JPEG decompression processing has been described. However, the present invention can be effectively applied to other image processing. For example, as long as the configuration includes hardware that executes scaling processing and color conversion processing, the present invention may be applied to a configuration that switches between execution of software and hardware of these image processing.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the function of the above-described embodiment (processing for determining whether data to be processed is hardware processing or software processing) is systemized via a network or various storage media. Or it supplies to an apparatus. And it is realizable when the computer (CPU, MPU, etc.) of the system or apparatus reads and runs a program. Further, this program may be executed by a single computer or may be executed in conjunction with a plurality of computers. Moreover, it is not necessary to implement all of the above-described processing by software, and part or all of the processing may be implemented by hardware.

Claims (5)

First image processing means for image processing image data by hardware processing;
Second image processing means for image processing image data by software processing;
A determining unit that determines that the first image data is to be processed using the first image processing unit when the first image data is determined to be printed;
Display means for displaying the first image data when a print reservation instruction is received before the print processing of the first image data starts and ends;
Detecting means for detecting a browsing instruction for changing the image data to be displayed by the display means from the first image data to second image data different from the first image data;
A determination means for determining whether the second image data includes a thumbnail image when the browsing instruction is detected;
If the determination means determines that the second image data includes a thumbnail image, the determination means determines that the thumbnail image of the second image data is to be processed by the second image processing means,
The image processing apparatus, wherein the display means displays the second image data as a processing result of the second image processing means before the printing of the first image data is finished. In the case where the determination unit determines that the second image data does not include a thumbnail image, the image processing unit further includes a size determination unit that determines whether the size of the image data is smaller than a predetermined size,
Based on the determination result of the previous SL size determining unit, the second image data is image processing according to claim 1, characterized in that is processed by the first image processing means or the second image processing means apparatus. A time determination unit for determining a time until the first image processing unit is released when the determination unit determines that the second image data does not include a thumbnail image ;
Based on the determination result of the previous SL time determining means, said second image data, according to be processed before Symbol first image processing means or the second image processing means to claim 1, wherein Image processing device. A control method for an image processing apparatus, comprising: first image processing means for image processing image data by hardware processing; and second image processing means for image processing image data by software processing,
When the first image data is determined as a printing target, the first image data is determined to be processed using the first image processing unit;
If the reservation printing instruction is received before the printing process of the first image data is completed from the start to display the first image data,
Detecting a browsing instruction for changing the image data to be displayed by the display means from the first image data to second image data different from the first image data;
When the browsing instruction is detected, it is determined whether the second image data includes a thumbnail image;
When it is determined that the second image data includes a thumbnail image, it is determined that the thumbnail image of the second image data is to be processed by the second image processing unit, and the printing of the first image data is performed. A control method comprising: displaying the second image data as a processing result of the second image processing means before the end. A program for realizing the control method of the image processing apparatus according to claim 4 by a computer.

Images