JP2015050564A - Image processing apparatus, image forming apparatus, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily switch between display and non-display of a predetermined area of image data.SOLUTION: An image processing apparatus (12) includes a selected area recognition unit that detects a selected area of image data, a mask image creation unit that creates mask image data, and a file creation unit that creates a file in which display and non-display of the mask image data is switched according to the arrangement of the mask image data associated with the selected area, addition of a selection input unit, and input to the selection input unit.

Description

  The present invention relates to an image processing apparatus, an image forming apparatus, a program, and a recording medium that generate a file in which a mask image layer is added to an original image layer.

  Conventionally, when a paper document is digitized, an image forming apparatus such as a multifunction peripheral generates a PDF (Portable Document Format) file having a plurality of layers in which an additional information layer is embedded in addition to the original image layer. The one having the function to perform is known. As for the display of the PDF file, a display desired by the user can be designated.

  For example, in Patent Document 1, a searchable PDF is once created, and information acquired from all of a plurality of additional information sources is added to the searchable PDF. As a result, a file having an additional information layer that designates a predetermined insertion form as a default setting is generated. Then, additional editing for designating an insertion form for each additional information source is performed so as to change the default setting. Specifically, a searchable PDF is created by performing additional editing such as selecting the level of the translation result to be added, changing the size of characters used to insert additional information, or extracting only the original text.

JP 2011-150599 A (released on August 4, 2011) JP 2009-111611 (released May 21, 2009)

  By the way, for example, when a document including personal information or the like is digitized, a file can be generated by embedding a layer of a mask image for displaying personal information in a filled state in addition to a layer of the document image itself. For example, the display / non-display of the personal information can be switched by switching the display / non-display of the mask image. However, in the conventional configuration described above, a separate editing operation is required when changing the display / non-display of the mask image. For this reason, there is a problem that the user cannot easily switch between displaying and hiding the mask image (displaying / hiding personal information) in response to a request on the spot when the file is displayed. doing.

  Therefore, the present invention provides an image processing apparatus, an image forming apparatus, a program, and a recording medium that can easily switch between displaying and hiding a mask image in a file in which a mask image layer is added to an original image layer. It is aimed.

  In order to solve the above problems, an image processing apparatus according to an aspect of the present invention detects a selection area specified for input image data and generates selection area information indicating the selection area. A mask image generation unit that generates mask image data corresponding to the selection region with reference to the selection region information, and the mask image data is arranged in association with the selection region of the input image data, A file generation unit for adding a selection input unit for selecting display or non-display of the mask image data and generating a file for switching between display and non-display of the mask image data according to the selection input to the selection input unit It is characterized by having.

  According to one aspect of the present invention, since the generated file is switched between display and non-display of the mask image data by the user operating the selection input unit, the display state and non-display state of the predetermined area in the file Can be easily switched.

1 is a block diagram illustrating a configuration of an image forming apparatus according to an embodiment of the present invention. It is a block diagram which shows the structure of the area | region selection part shown in FIG. It is explanatory drawing which shows the recognition operation | movement of the marker part of the image data by the marker recognition part shown in FIG. It is a block diagram which shows the structure of the file generation part shown in FIG. It is a flowchart which shows the operation | movement which recognizes the file format of the file preserve | saved at the memory | storage part by the control part shown in FIG. FIG. 6A is a flowchart showing the generation operation of the display switching PDF file in the image forming apparatus, and FIG. 6B is a flowchart showing the contents of the operation of S26 in FIG. FIG. 2 is a schematic diagram illustrating a configuration of a display switching PDF file generated by a file generation unit illustrated in FIG. 1 and a configuration for controlling display / non-display of a marker area mask image in the display switching PDF file. 8A is an explanatory diagram showing an example of a document catalog set in the display switching PDF file generated by the file generation unit shown in FIG. 1, and FIG. 8B is the optional content. FIG. 8C is an explanatory diagram showing an example of a group dictionary, and FIG. 8C is an explanatory diagram showing an example of range specification of the optional content. It is explanatory drawing which shows the example of the drawing command for a display set to the display switching PDF file produced | generated in the file production | generation part shown in FIG. FIG. 10A illustrates an example of a page object set in the display switching PDF file with respect to a button for instructing display / non-display switching of the marker area mask image generated by the file generation unit. FIG. 10B is an explanatory diagram showing the Widget annotation, and FIG. 10C is an explanatory diagram showing the form XObject (appearance of the button). 11A is an explanatory diagram showing the display state (displayed state) of the marker area mask image data in the display switching PDF file, and FIG. 11B is the marker area mask image data in the display switching PDF file. It is explanatory drawing which shows the non-display state (state which is not displayed). 12A is another example of a button set in the display switching PDF file shown in FIG. 11A, and is an explanatory diagram showing an initial state of the button, and FIG. FIG. 13 is an explanatory diagram showing a state where the button shown in FIG. 12A is operated. FIG. 13A is an explanatory diagram showing an example of a Widget annotation set in the display switching PDF file generated by the file generation unit. FIG. 13B shows the form XObject (appearance of the button). ) Is an explanatory diagram, and FIG. 13C is an explanatory diagram illustrating an example of the graphic state parameter dictionary (translucent drawing state). It is explanatory drawing which shows the example of switching of printing / non-printing of a button set to the display switching PDF file produced | generated in the said file production | generation part. It is explanatory drawing which shows the example of switching of display / non-display of marker area mask image data by default set in the display switching PDF file generated by the file generation unit. It is a schematic diagram which shows the structure which controls display / non-display of a marker area mask image for every page in the said display switching PDF file of several pages. FIG. 5 is a schematic diagram showing a configuration for collectively controlling display / non-display of marker area mask images for all pages in accordance with button operations in the display switching PDF file for a plurality of pages. 18A is an explanatory diagram showing a document image in which marker lines are written by the user, and FIG. 18B is an explanatory diagram showing a state where the marker area is masked in the display switching PDF file. (C) of FIG. 18 is an explanatory diagram showing a state where the outside of the marker area is masked and the document image in the selected area is displayed in the display switching PDF file. It is explanatory drawing which shows the example of the drawing command for a display in the case of displaying and non-displaying marker area mask image data set to the display switching PDF file produced | generated in the said file production | generation part. It is a block diagram which shows the structure of the image reading apparatus in other embodiment of this invention. It is a block diagram which shows the structure of the image forming apparatus in further embodiment of this invention. 22A is a flowchart showing the display switching PDF file generation operation in the image forming apparatus shown in FIG. 21, and FIG. 22B shows the contents of the operation of S47 in FIG. It is a flowchart. It is a block diagram which shows the structure of the portable terminal device used in the image forming apparatus of further another embodiment of this invention. It is a block diagram which shows the structure of the image process part with which the portable terminal device shown in FIG. 23 is provided. It is a block diagram which shows the structure of the image forming apparatus in other embodiment of this invention. FIG. 26 is a block diagram illustrating a configuration of an image processing unit included in the image forming apparatus illustrated in FIG. 25. FIG. 27A is a flowchart showing an outline of processing performed by the mobile terminal device shown in FIG. 23 and the image forming apparatus shown in FIG. 25, and FIG. 27B is a flowchart of processing in the mobile terminal device. 27 is a flowchart showing the contents, FIG. 27C is a flowchart showing the contents of the processing in the image forming apparatus, and FIG. 27D is the contents of the formatting process shown in FIG. It is a flowchart to show. It is a flowchart which shows an example of the super-resolution correction | amendment by the super-resolution correction | amendment part shown in FIG. It is explanatory drawing which shows the reference | standard pixel and interpolation pixel of the super-resolution image data produced | generated by the said super-resolution correction | amendment. It is explanatory drawing which shows the example of the primary differential filter used for the said super-resolution correction | amendment. It is a figure which shows the determination method of the pixel value of the interpolation pixel in the said super-resolution correction | amendment.

[Embodiment 1]
Embodiments of the present invention will be described below with reference to the drawings.

(Outline of image forming apparatus 1)
FIG. 1 is a block diagram showing a configuration of an image forming apparatus 1 according to an embodiment of the present invention.
The image forming apparatus 1 is a digital printer such as a color printer, a color copying machine, or a color multifunction machine. When the image forming apparatus 1 is a digital color multifunction peripheral, the image forming apparatus 1 includes, for example, a copy function, a printer function, a facsimile transmission function, a scan to e-mail function, and the like.

  As shown in FIG. 1, the image forming apparatus 1 includes an image input device 11, an image processing device 12, an image output device 13, a control unit 14, a transmission / reception unit 15, a storage unit 16, an encoding / combining unit 17, and an operation panel. 18 is provided.

(Image input device 11)
The image input device 11 has a function as a scanner and is composed of a CCD (Charge Coupled Device) line sensor, and R, G, B (R: red, G: green, B: blue) reflecting light reflected from the document. The color signal is converted into an electrical signal.

(Image processing apparatus 12)
The image processing device 12 processes the image signal (RGB signal) input from the image input device 11 and outputs it to the image output device 13. The image processing apparatus 12 includes an A / D (analog / digital) conversion unit 31, a shading correction unit 32, an input processing unit 33, an area selection unit 34, a color correction unit 35, a black generation / under color removal unit 36, and a spatial filter unit. 37, a halftone generation unit 39, a region separation unit 41, and a file generation unit 40.

(Image output device 13)
The image output device 13 is a device that reproduces an image, such as an electrophotographic printer or an inkjet printer.

(Control unit 14)
The control unit 14 is a general control device including a CPU, and controls the operation of each unit of the image forming apparatus 1. For example, data read / write control with respect to the storage unit 16, data flow control between the units of the image forming apparatus 1, and control based on information specified by a UI (user interface) are performed.

  The control unit 14 also analyzes the header of the file stored in the storage unit 16 and recognizes the file format. If the file is a TIFF file, it is determined whether the file is binary or multi-valued by examining the tag. If the file is binary, the region selection unit 34 is controlled not to perform marker recognition processing.

(Transceiver 15)
The transmission / reception unit 15 exchanges data with an external device via a network or a communication line.

(Storage unit 16)
The storage unit 16 is configured by a general data storage device such as a hard disk, and appropriately stores image data processed by the area selection unit 34 and the file generation unit 40 of the image processing device 12 and image data being processed. Remember.

(Encoding / Decoding Unit 17)
The encoding / decoding unit 17 encodes image data when the image data is input to the storage unit 16 from the input processing unit 33 and the region selection unit 34 of the image processing apparatus 12, and the input processing unit 33 from the storage unit 16. The image data is decoded when the image data is output to the area selection unit 34 or the like. When image data is input to the storage unit 16, whether or not to encode the image data may be selected on the operation panel 18.

  The encoding / decoding unit 17 encodes the input image data in the storage unit 16 when the image data output from the storage unit 16 is input to the input processing unit 33 and the region selection unit 34. Depending on the presence or absence, the image data is appropriately decoded so that the image data is input in a decoded state.

(Operation panel 18)
The operation panel 18 includes a display unit, and a touch panel on the display unit. Further, various input keys operated by the user may be provided.

(Area selection unit 34)
When the marker designation mode is selected, the area selection unit 34 is a user-selected area (marker area) of document image data (hereinafter simply referred to as image data) indicating a document image read by the image input device 11. ) And user selection area information indicating the user selection area is created.

  FIG. 2 is a block diagram showing the configuration of the area selection unit 34. As shown in FIG. 2, the region selection unit 34 includes a resolution conversion unit 51 and a marker recognition unit (selection region recognition unit) 52.

  The resolution converter 51 reduces the resolution of the input image data. For example, image data read at 1200 dpi or 600 dpi is converted to 75 dpi. As a resolution conversion method, a known nearest neighbor method, bilinear method, bicubic method, or the like can be used.

  The marker recognition unit 52 compares the image data output from the resolution conversion unit 51 with a predetermined color, recognizes a marker portion (marker region) of the image data, and obtains a marker recognition result. The predetermined color is designated by the user. The marker recognition unit 52 generates user selection area information from the marker recognition result.

  The marker recognition result includes marker detection data and marker area information. The marker detection data is a designated color pixel map extracted from input image data by a marker recognition method described later. In the designated color pixel map, 1 is set for the designated color pixel, and 0 is set for the non-designated color pixel. The marker area information indicates the start coordinates of the marker area and the vertical and horizontal sizes of the marker area. The selection area information includes marker area information (starting coordinates and vertical and horizontal sizes of the marker area) at the input image resolution (see FIG. 3).

(Marker part recognition method by marker recognition unit 52)
Here, a method of recognizing the marker portion (marker line 72: marker region) of the image data by the marker recognizing unit 52 will be described. FIG. 3 is an explanatory diagram showing the recognition operation of the marker portion (marker line 72: marker region) of the image data by the marker recognition unit 52.

  The marker recognizing unit 52 converts RGB image data to HSL image data, extracts pixels of a specific marker color, and extracts a marker region based thereon, as in the method described in Patent Document 2, for example.

  Specifically, first, the following processing is performed on all data of the input image data (image data whose resolution has been reduced by the resolution conversion unit 51) to generate marker detection data. The marker detection data takes values of “0” and “1”, and “1” is set for marker pixels and “0” is set for non-marker pixels.

  Next, 8-bit H, S, L (hue, saturation, brightness) data is generated from the R, G, B data of the input image data by the conversion formula shown below.

MAX = max (R, G, B)
MIN = min (R, G, B)
L = (MAX + MIN) / 2
(1) When MAX = MIN S = 0, H = 0
(2) When MAX ≠ MIN When L <= 128 S = 255 × (MAX−MIN) / (MAX + MIN)
When L> 128 S = 255 × (MAX-MIN) / (2-MAX-MIN)
When MAX.noteq.MIN, the values of Cr, Cg, and Cb are calculated according to the following formula to obtain the hue.

Cr = (MAX-R) / (MAX-MIN)
Cg = (MAX-G) / (MAX-MIN)
Cb = (MAX-B) / (MAX-MIN)
When MAX = R, H = Cb-Cg
When MAX = G H = 2 + Cr-Cb
When MAX = B H = 4 + Cg-Cr
H = 60 x H
When H <0, H = H + 360
H = H × 255/360
Since H is calculated in the range of 0 to 359, when H takes a negative value, 360 is added so as to be within that range. In this example, since H is 8 bits, rounding to 8 bits is performed at the end.

  Next, with respect to the H, S, and L data, it is determined whether each data is within the range of the upper limit threshold and the lower limit threshold for detecting the designated marker. If the result of this determination is within the range, the marker detection data of the corresponding pixel is set to “1” indicating that it is valid, and if it is outside the range, it is set to “0” indicating that it is invalid. .

  For example, when it is desired to extract a green marker, the upper and lower thresholds for marker detection for H, S, and L data are set as follows.

Green marker extraction upper limit H data: 110
Green marker extraction lower limit H data: 50
Green marker extraction upper limit S data: 255
Green marker extraction lower limit S data: 128
Green marker extraction upper limit L data: 215
Lower limit L data for green marker extraction: 95
This set value is prepared for each corresponding marker color.

  Next, as shown in FIG. 3, the circumscribed rectangle of the effective pixel is obtained for the generated marker detection data, and the start coordinates of the circumscribed rectangle and the vertical and horizontal size information for the input image data are generated.

(File generation unit 40)
The file generation unit 40 generates marker region mask image data for masking a predetermined region of the image data based on the selection region information input from the region selection unit 34 and user settings. Further, a PDF file including a layer of image data and a layer of marker area mask image data and having a format in which display or non-display of the marker area mask image can be selected is created.

  FIG. 4 is a block diagram illustrating a configuration of the file generation unit 40. As shown in FIG. 4, the file generation unit 40 includes a mask image generation unit 61, a drawing command generation unit (file generation unit) 62, and a formatting processing unit (file generation unit) 63.

  The mask image generation unit 61 generates marker region mask image data based on the selection region information input from the region selection unit 34 and user settings.

  The user setting is performed in the user setting mode. In the user setting mode, for example, selection information indicating whether the marker area or the area outside the marker area is to be filled is generated by a user input on the operation panel 18, and the fill color and the transmittance of the fill color are designated. As a result, a mask image (marker region mask image data) that masks the selected area or the outside of the selected area with the specified color is generated.

  The drawing command generation unit 62 generates each layer of the image data and the marker region mask image data based on the user selection region information generated by the region selection unit 34 and the user setting. Further, a command for arranging the user selection area mask image data in association with the marker portion (marker area) recognized by the marker recognition unit 52 of the image data is generated.

  The formatting processing unit 63 changes each layer of the image data and the marker area mask image data into a PDF file in a format in which display or non-display of the marker area mask image can be selected.

  In the above configuration, the operation of the image forming apparatus 1 of the present embodiment will be described below.

(Printing operation)
In the printing operation (printing mode), analog RGB signals (color image signals) input from the image input device 11 to the image processing device 12 are converted into digital RGB signals by the A / D conversion unit 31, and shading correction is performed. Various distortions that occur in the illumination system, the imaging system, and the imaging system of the image input apparatus 11 are removed by the unit 32. Thereafter, the input processing unit 33 performs processing such as γ correction on each of the RGB signals.

  The RGB signal output from the input processing unit 33 is compressed into, for example, a JPEG code by the encoding / decoding unit 17 and stored in the storage unit 16. During the printing operation, no particular processing is performed on the marker area of the image data.

  When transmitting the JPEG code to an external device (image transmission), the JPEG code is read from the storage unit 16, and the JPEG code is externally transmitted by the transmission / reception unit 15 via a network or a communication line. Transmitted to the device. In this case, the control of the filing data management of the storage unit 16 and the data delivery operation are performed by the control unit 14.

  The RGB signal output from the input processing unit 33 is input to the color correction unit 35, the region separation unit 41, and the file generation unit 40 via the region selection unit 34.

  In the color correction unit 35, a CMY (C: cyan, M: magenta, Y: yellow) signal that is a complementary color of the RGB signal is generated from the RGB signal, and processing for further improving color reproducibility is performed.

  The CMY signal output from the color correction unit 35 is converted into a CMYK (K: black) 4-color signal by the black generation / under color removal unit 36. The CMYK signals output from the black generation / undercolor removal unit 36 are sequentially processed by the spatial filter unit 37, the output tone correction unit 38, and the halftone generation unit 39.

  The spatial filter unit 37 performs spatial filter processing (enhancement processing and smoothing processing) using a digital filter on the CMYK signal based on the region separation signal. The output tone correction unit 38 performs output γ correction processing. The halftone generation unit 39 performs gradation reproduction processing for outputting an image with respect to the CMYK signal.

  In the region separation processing unit 39, it is determined which pixel of the input image data (RGB signal) belongs to which region among black characters, color characters, halftone dots, photographic paper photograph (continuous tone region), etc. Region separation data indicating the determination result is generated. The region separation data is output from the region separation unit 41 to the black generation / undercolor removal unit 36, the spatial filter unit 37, and the halftone generation unit 39. Thereby, in each of these units, processing is switched according to the region separation data, and appropriate processing according to each region indicated by the region separation data is performed on the image data.

  The CMYK signal output from the halftone generator 39 is input to the image output device 13. The image output device 13 forms an output image based on the CMYK signal.

(Image transmission operation)
The processing contents of the image input device 11, the A / D conversion unit 31, the shading correction unit 32, the input processing unit 33, and the region separation unit 41 in the image transmission operation (image transmission mode) are the same as those in the printing operation. Note that the region separation unit 41 outputs a region separation signal to the spatial filter unit 37.

  Here, when the marker designation mode is selected, the area selection unit 34 detects a user selection area (marker area) in the document (image data).

  The color correction unit 35 converts the image data into R′G′B ′ image data (for example, sRGB data) suitable for display characteristics of a display device in which the image data is widely used. Further, the spatial filter unit 37 performs spatial filter processing (enhancement processing, smoothing processing) on the image data using a digital filter based on the region separation signal.

  The mask image generation unit 61 of the file generation unit 40 generates user selection region mask image data from the user selection region information and image data generated by the region selection unit 34, and the drawing command generation unit 62 converts the image data into image data. On the other hand, a command for arranging the user selection area mask image data is generated.

  Next, the formatting processing unit 63 embeds a command for enabling switching between display and non-display and a command for switching between display and non-display by a button operation in the user selection area mask image data. User-selectable area mask image data capable of switching display is created. Further, the formatting processing unit 63 converts the user selection area mask image data into a file such as a PDF file (display switching PDF file) together with the image data of R′G′B ′. Thereafter, the image file is output to a file storage destination designated by the network or the user.

(File format recognition operation of control unit 14)
Next, the file format recognition operation of the control unit 14 will be described. FIG. 5 is a flowchart illustrating an operation performed by the control unit 14 to recognize the file format of a file stored in the storage unit 16.

  Various file formats often have a characteristic byte string at the beginning of the file. Therefore, the control unit 14 can easily recognize the file type by checking the first byte string of the file.

  As shown in FIG. 5, the control unit 14 first obtains the first byte string of the file stored in the storage unit 16 (S11).

  Next, it is determined whether or not the first byte string of the acquired file starts with the hexadecimal numbers 0x49, 0x49, 0x2A, 0x00 (S12). If the result of this determination is Yes, the file is determined to be a TIFF image file (S17).

  If the result of determination in S12 is No, it is determined whether or not the first byte sequence of the acquired file begins with the hexadecimal numbers 0x4D, 0x4D, 0x00, and 0x2A (S13). If the result of this determination is Yes, the file is determined to be a TIFF image file (S17).

  If the result of determination in S13 is No, it is determined whether or not the first byte sequence of the acquired file starts with hexadecimal numbers 0xFF and 0xD8 (S14). If the result of this determination is Yes, the file is determined to be a JPEG image file (S18).

  If the result of determination in S14 is No, it is determined whether or not the first byte sequence of the acquired file begins with the hexadecimal numbers 0x25, 0x50, 0x44, 0x46 (S15). If the result of this determination is Yes, the file is determined as a PDF file (S19).

  If the result of determination in S15 is No (when the first byte string of the acquired file does not correspond to any of the above), it is determined that the file cannot be processed (S16).

  Next, an operation of generating a PDF file (hereinafter simply referred to as a display switching PDF file) that can switch between display and non-display of a predetermined area in the image forming apparatus 1 will be described. FIG. 6A is a flowchart showing the generation operation of the display switching PDF file in the image forming apparatus 1, and FIG. 6B is a flowchart showing the contents of the operation in S26 of FIG.

  In the generation of the display switching PDF file, as shown in FIG. 6A, the following processing setting is performed by the user on the operation panel 18 of the image forming apparatus 1 (S21). The setting contents here are stored in the storage unit 16 as setting information. The marker area is designated by the user surrounding the target portion of the document with a highlighter pen in advance.

  (1) When the marker designation mode is selected by the user, the user setting mode is set. Further, the user selects, for example, within the marker area (the area designated by the marker according to the guidance screen display of the image display device on the operation panel 18). (Inside) or outside the marker area, and setting of the fill color and the transmissivity of the fill color are performed.

  (2) When the marker designation mode is selected, a screen prompting the user to select whether or not to perform display / non-display control of the marker area in the display switching PDF file is displayed on the image display device of the operation panel 18.

  (3) When the display / non-display control is selected, when the created file (display switching PDF file) is opened, a screen prompting selection of whether or not to display the marker area is displayed. On this screen, it is also required to select whether the control object is inside or outside the marker area.

  (4) When transmitting the created file (display switching PDF file), the destination address is set by the user. When the created file (display switching PDF file) is saved, the save destination is designated by the user. For example, when data is read from a USB memory and processed data (display switching PDF file) is stored in the same USB memory, the user designates the USB memory as a storage destination and sets a file name to be stored after processing.

  When the above setting is completed and a start button (not shown) on the operation panel 18 is operated by the user, a display switching PDF file generation process is started. In this process, first, the image input device 11 reads a document to obtain image data (S22). The original can be read by placing the original on a contact glass (not shown) of the image input apparatus 11 even if the original is set on an ADF (automatic original conveyance apparatus) provided in the image input apparatus 11 and automatically read. It may be set and read sheet by sheet.

  Next, with respect to the image data acquired by reading the document, the resolution conversion unit 51 of the area selection unit 34 reduces the resolution of the image data, and the marker recognition unit 52 performs a marker recognition process (S23). In the marker recognition process, the marker portion of the image data is recognized, and user selection area information indicating the marker portion is generated.

  Next, the mask image generation unit 61 of the file generation unit 40 uses the marker region mask image data (based on the user selection region information generated by the region selection unit 34 and the setting information obtained by the processing setting of S21. A marker area (an image that can make the area specified by the marker invisible) is generated (S24).

Next, the drawing command generation unit 62 generates each layer of image data and marker region mask image data, and further, the marker portion (marker region) in which the user selection region mask image data is recognized by the marker recognition unit 52 of the image data. ) To generate an instruction (drawing command) to be arranged. (S25)
Next, the formatting processing unit 63 changes each layer of the image data and the marker area mask image data to a PDF file in a format in which display or non-display of the marker area mask image can be selected as the formatting process (S26). .

  In the formatting process of S26, as shown in FIG. 6B, a drawing command for embedding the display / non-display of the marker area mask image indicated by the marker area mask image data is embedded (S31). Further, a button for instructing switching between display and non-display of the marker area mask image is embedded (S32). Thereby, marker area mask image data is generated.

  FIG. 7 is a schematic diagram illustrating a configuration of a display switching PDF file generated by the file generation unit 40 and a configuration for controlling display / non-display of a marker area mask image in the display switching PDF file.

  8A is an explanatory diagram showing an example of a document catalog set in a display switching PDF file generated by the file generation unit 40, and FIG. 8B is an example of the optional content group dictionary. FIG. 8C is an explanatory diagram showing an example of specifying the range of the optional content.

  The document catalog shown in (a) of FIG. 8 represents information of the entire document, and is set for each page and each object for the object to be displayed / hidden. In the example of FIG. 8, the display / non-display of one object “390” (reference numeral 76 in FIG. 8A) is switched (displayed).

  The optional content group dictionary shown in (b) of FIG. 8 defines labels for organizing mutual relations when performing operations (see FIGS. 7, 16, and 17). In the example of FIG. 8B, the object “39 0” is used as the switching label, and the name and type of the object “39 0” are defined.

  The optional content range designation shown in FIG. 8C indicates content information for each page. In the example of FIG. 8C, a range (marker region mask layer) of an object (marker region mask image) to be switched between display and non-display is represented. Since the mask image is generated with the area to be filled 1 and the area not to be filled 0, the same command can be used to fill the area outside the area. However, the size and information in the stream change.

  FIG. 9 is an explanatory diagram illustrating an example of a display drawing command set in the display switching PDF file generated by the file generation unit 40.

  The drawing command for display defines a drawing command for display. In the example of FIG. 9, image data is drawn, and marker area mask image data is also drawn at the same time.

  FIG. 10A is an explanatory diagram showing an example of a page object set in the display switching PDF file regarding the button for instructing display / non-display switching of the marker area mask image, and FIG. 10B. FIG. 10 is an explanatory diagram showing the Widget annotation, and FIG. 10C is an explanatory diagram showing the form XObject (appearance of the button).

  A page object shown in FIG. 10A indicates reference information when an action (display or non-display, movement to a link destination, etc.) is caused, and is linked to a Widget annotation.

  The Widget annotation shown in (b) of FIG. 10 describes an object that causes an action. When the button is operated, display / non-display of the object “390” is switched. In the example of FIG. 10B, the button is set not to be printed (default setting).

  A form XObject (appearance of a button) shown in (c) of FIG. 10 defines a drawing image of the button.

  An example of the display switching PDF file generated as described above is as shown in FIGS. 11A is an explanatory diagram showing a display state (displayed state) of marker area mask image data in the display switching PDF file 73, and FIG. 11B is a marker area mask in the display switching PDF file 73. FIG. It is explanatory drawing which shows the non-display state (state which is not displayed) of image data.

  As shown in FIGS. 11A and 11B, a button (selection input unit) 71 is displayed in the display switching PDF file 73. Note that reference numeral 72 shown in FIG. 11B is a marker line written on the document by the user.

  In the display switching PDF file 73, every time the button 71 is clicked by the user, a marker area mask image display state (a document image area surrounded by the marker line 72 is not displayed) shown in FIG. 11B, the marker area mask image non-display state (the document image area surrounded by the marker line 72 is displayed) can be switched.

  Further, when printing is performed in the display state of the marker area mask image shown in FIG. 11A, a print image of the original image in which the area (marker area) surrounded by the marker line 72 is masked is obtained. It is done. On the other hand, when printing is performed in a state where the marker area mask image shown in FIG. 11B is not displayed, a print image of only the original image is obtained. In the latter case, the marker line 72 written on the document image is printed as it is.

  As described above, in the display switching PDF file 73 created by the image processing apparatus 12, the button 71 added (embedded) to the display switching PDF file 73 in a state where the display switching PDF file 73 is displayed. Can be switched between display and non-display of the marker area mask image provided for the image data. Thus, in the state where the display switching PDF file 73 is displayed, the display / non-display state of the predetermined area of the image data is changed by operating the button 71 to switch the display / non-display of the marker area mask image. Can be easily switched.

  Further, by embedding the button 71 in the display switching PDF file 73 in a state where it is not printed, it is possible to prevent the button 71 from being printed when the display switching PDF file 73 is printed.

[Embodiment 2]
Another embodiment of the present invention will be described below with reference to the drawings.
In the present embodiment, the display state of the button 71 in the display switching PDF file 73 is different from the display state of the above embodiment. 12A is an explanatory diagram showing an initial state of the button 71, and FIG. 12B is an explanatory diagram showing a state in which the button 71 is operated.

  As shown in FIG. 12A, the button 71 is displayed in a translucent state in an initial state where the button 71 is not operated. On the other hand, as shown in FIG. 12 (b), the button 71 is displayed in an opaque state (a semi-transparent state is canceled and a normal density state is released) when operated. Reference numeral 74 denotes a cursor 74. Further, when the button 71 is operated, the operation of the button 71 displays that the display / non-display of the marker area mask image is switched (operation content) as a balloon 75. The button 71 is embedded in the display switching PDF file 73 so as to display as described above.

  FIG. 13A is an explanatory diagram showing an example of a Widget annotation set in the display switching PDF file 73 generated by the file generation unit 40, and FIG. 13B shows the form XObject (button FIG. 13C is an explanatory diagram showing an example of the graphic state parameter dictionary (translucent drawing state).

  The Widget annotation shown in FIG. 13A describes an object that causes an action when the button 71 is operated. In the example of FIG. 13A, the button 71 is operated to switch display / non-display of the object “390”.

  In this example, the button 71 is set not to be printed (default setting). The Widget annotation is linked to a form XObject (“45 0”) which is a semi-transparent drawing image of the button 71 and a form XObject (“44 0”) which is an opaque drawing image. Further, a character string surrounded by “()” following “/ TU” is displayed as a balloon explaining the operation content.

  A form XObject (appearance of the button) shown in FIG. 13B defines a semi-transparent drawing image of the button 71. In this example, the part designating the translucent drawing state is different from the button drawing image (form XObject (button appearance)) shown in FIG.

  The graphic state parameter dictionary (translucent drawing state) shown in FIG. 13C defines the drawing state. In this example, the button 71 is set to a 30% transparent state (translucent state).

  According to the above configuration, the button 71 is displayed in a semi-transparent state in the initial state ((a) in FIG. 12), and when the cursor 74 is placed on the button 71 for clicking, the button 71 is in an opaque state. Is displayed. The display when the cursor 74 does not exist on the button 71 is called a normal appearance, and the display when the cursor 74 exists on the button 71 is called a rollover appearance. The operation when the button 71 is clicked (switching the display of the display switching PDF file 73) is the same as that described above (in the case of FIG. 10).

[Embodiment 3]
Still another embodiment of the present invention will be described below with reference to the drawings.
In the present embodiment, the button 71 is embedded in the display switching PDF file 73 so as not to be printed. FIG. 14 is an explanatory diagram showing an example of switching between printing and non-printing of the button 71 set in the display switching PDF file 73 generated by the file generation unit 40.

  In order to print the button 71 when the display switching PDF file 73 is printed, as shown in FIG. 14, the command “/ F4” is inserted into the Widget annotation shown in FIG. Good. In this way, if the Widget annotation shown in FIG. 10B is changed, the display switching PDF file 73 can be brought into a state in which the button 71 is printed. On the other hand, in order to prevent the button 71 from being printed when the display switching PDF file 73 is printed, the above-mentioned command “/ F4” is deleted in the printing / non-printing switching of the button 71 shown in FIG. Good.

[Embodiment 4]
Still another embodiment of the present invention will be described below with reference to the drawings.
In the present embodiment, in the display switching PDF file 73, the marker area mask image data is embedded in a non-display state. Thus, when the display switching PDF file 73 is opened, the marker area mask image is not displayed. Note that the marker region mask image in the non-display state can be switched to the display state by operating the button 71 as described above.

  FIG. 15 is an explanatory diagram showing an example of switching between displaying and hiding the layer of the marker area mask image data by default, which is set in the display switching PDF file 73 generated by the file generation unit 40.

  Embedding the marker area mask image (for example, see FIG. 7) in the display switching PDF file 73 in the non-display state is performed by using the command “/ OFF” in the document catalog shown in FIG. This can be done by inserting [39 0 R] ".

  In FIG. 15, when the user opens the display switching PDF file 73, the object “390” is controlled to be hidden. Whether or not to hide the marker area mask image when the PDF file 73 is opened is set by a user input from the operation panel 18 before starting the processing.

[Embodiment 5]
Still another embodiment of the present invention will be described below with reference to the drawings.
In the present embodiment, in the display switching PDF file 73 for a plurality of pages, display / non-display of the marker area mask image for all pages is controlled collectively according to the operation of the button 71.

  FIG. 16 is a schematic diagram showing a configuration for controlling the display / non-display of the marker area mask image for each page in the multi-page display switching PDF file 73. FIG. 17 is a schematic diagram showing a configuration in which the display / non-display of the marker area mask image of all pages is collectively controlled in accordance with the operation of the button 71 in the multi-page display switching PDF file 73.

  In the multi-page display switching PDF file 73, when the display / non-display of the marker area mask image is controlled for each page, as shown in FIG. 16, different labels are used for each page of the multi-page display switching PDF file 73. And associate with the layer to be controlled. At this time, the display form of the button 71 of each display switching PDF file 73 may be the same or different.

  On the other hand, in the multi-page display switching PDF file 73, when the display / non-display of the marker area mask image of all pages is controlled collectively in response to the operation of the button 71 of any display switching PDF file 73. As shown in FIG. 17, the marker area mask images of the display switching PDF file 73 for each page are defined as different objects, and are associated with the same label (in this example, “Marker View”). In this case, the display form of the button 71 is the same in the display switching PDF file 73 of each page.

[Embodiment 6]
Still another embodiment of the present invention will be described below with reference to the drawings.
In the present embodiment, display / non-display of the marker area mask image data is switched in the display switching PDF file 73. Switching between display and non-display of the marker region mask image can be performed by operating the button 71 as described above.

  18A is an explanatory diagram showing a document image in which the marker line 72 is written by the user, and FIG. 18B shows an area (marker) surrounded by the marker line 72 in the display switching PDF file 73. FIG. 18C shows an area (marker area) surrounded by the marker line 72 in the display switching PDF file 73. FIG. 18C is an explanatory diagram showing a state where the area (selected area) is masked (a state where the marker area is filled). : Selection area) is masked outside and is an explanatory view showing a state in which the inside of the selection area is displayed. FIG. 19 is an explanatory diagram illustrating an example of a display drawing command that is set in the display switching PDF file 73 generated by the file generation unit 40 when displaying or hiding the marker area mask image data.

  As shown in FIG. 18C, in the display switching PDF file 73, when specifying and embedding the transmittance of the marker area mask image, the display drawing command shown in FIG. It is possible to display the mask image with the set transmittance at the time of display. Specifically, as shown in FIG. 19, the transmittance of the marker area mask image may be set in the display drawing command shown in FIG. In the example of FIG. 19, the transmittance of the marker area mask image is set to 50%.

  According to the above configuration, by setting the marker area mask image outside the marker area, the display switching PDF file 73 can be displayed with the inside of the marker area highlighted.

[Embodiment 7]
Still another embodiment of the present invention will be described below with reference to the drawings.
FIG. 20 is a block diagram illustrating a configuration of an image reading apparatus 81 that reads a color image and includes the image processing apparatus 82 according to the present embodiment. This embodiment shows an image reading apparatus 81 to which an image processing apparatus 82 including an input processing unit 33 and a file generation unit 40 is applied.

  As shown in FIG. 20, the image reading device 81 includes an image input device 11, an image processing device 82, a control unit 14, a transmission / reception unit 15, a storage unit 16, an encoding / decoding unit 17, and an operation panel 18. . The image processing apparatus 82 includes an A / D conversion unit 31, a shading correction unit 32, an input processing unit 33, a region selection unit 34, and a file generation unit 40.

  The configuration and operation of each unit are the same as those of the image forming apparatus 1 shown in FIG. In the image processing apparatus 82, the display switching PDF file 73 after the processing is output to a computer, a hard disk, a network, or the like.

  Also in the image processing apparatus 82 according to the present embodiment, in a state where the display switching PDF file 73 is being displayed, the button 71 is operated to switch between display and non-display of the marker area mask image. The display state and non-display state of the area can be easily switched. Further, by embedding the button 71 in the display switching PDF file 73 in a state where it is not printed, it is possible to prevent the button 71 from being printed when the display switching PDF file 73 is printed.

[Embodiment 8]
Still another embodiment of the present invention will be described below with reference to the drawings.
FIG. 21 is a block diagram showing a configuration of the image forming apparatus 91 in the present embodiment. The image forming apparatus 91 includes an image processing apparatus 92 instead of the image processing apparatus 12, and further includes an image display apparatus 93.

  In the image forming apparatus 1 described above, the user enters the marker line 72 on the document image, and reads the document image with the image input device 11 as a scanner, whereby image data having the marker line 72 (marker region). Was getting. On the other hand, in the image forming apparatus 91 according to the present embodiment, a preview image of image data is displayed on the image display device 93, and the user displays the marker line 72 (marker on the screen of the image display device 93 on which the preview image is displayed. Area) can be set.

  In the present embodiment, the display operation of the image display device 93 that displays the preview image is controlled by the display control unit 94 of the image processing device 92. The image display device 93 is provided on the operation panel 18. The operation panel 18 includes an operation unit 18a including various input keys and a coordinate storage unit 18b.

(Image display device 93)
The image display device 93 is configured by a liquid crystal display provided with a touch panel, for example. The image display device 93 displays a preview image of the image data, and further displays setting conditions such as copy and printout. The setting is input by the user operating the setting unit on the screen of the image display device 93 with a finger or a pen, and the touch panel reads the operation content.

  On the screen of the image display device 93, coordinate information indicating an operation for inputting the marker line 72 is stored in the coordinate storage unit 18 b and sent to the storage unit 16.

(Display control unit 94)
The display control unit 94 reads image data from the storage unit 16 and performs processing such as thinning out pixels on the image data so that the entire original image is displayed on the image display device 93. Display as a confirmation image.

The following interpolation method can be used as a method for thinning out pixels of image data.
(1) Nearest Neighbor Method The value of an existing pixel closest to the pixel to be interpolated or having a predetermined positional relationship with the pixel to be interpolated is set as the value of the interpolated pixel.
(2) Bilinear method An average of values weighted in a form proportional to the distance between four existing pixels surrounding a pixel to be interpolated is obtained, and that value is used as the interpolation pixel.
(3) Bicubic method Interpolation is performed using the values of a total of 16 pixels obtained by adding 12 points surrounding these pixels to 4 points surrounding the pixels to be interpolated.

  The display control unit 94 further performs gradation correction (gamma correction) based on the display characteristics of the display device on the generated confirmation image. Other configurations are the same as those of the image forming apparatus 1.

  Next, the generation operation of the display switching PDF file 73 in the image forming apparatus 91 will be described. FIG. 22A is a flowchart showing the generation operation of the display switching PDF file 73 in the image forming apparatus 91, and FIG. 22B is a flowchart showing the contents of the operation of S47 in FIG.

  In the generation of the display switching PDF file 73, as shown in FIG. 22A, the following processing setting is performed by the user on the operation panel 18 of the image forming apparatus 1 (S41). The setting contents are stored in the storage unit 16 as setting information.

(1) Selection of an area designation mode by the user (corresponding to the marker designation mode in the description of FIG. 6A) (2) When the area designation mode is selected by the user, the area designation mode is entered, and the display switching PDF A screen prompting the user to select whether or not to perform display / non-display control of the designated area in the file 73 is displayed on the image display device 93.

  (3) When display / non-display control is selected, when the created display switching PDF file 73 is opened, a screen prompting selection of whether or not to display the designated area is displayed. On this screen, it is also required to select whether to control within the designated area or outside the designated area.

  (4) When the created display switching PDF file 73 is transmitted, the destination address is set by the user. When the created display switching PDF file 73 is stored, the storage destination is designated by the user. For example, when reading from a USB memory and storing processed data in the same USB memory, the user designates the USB memory as a storage destination and sets a file name to be stored after processing.

  When the above setting is completed and the start button (not shown) of the operation panel 18 is operated by the user, the generation process of the display switching PDF file 73 is started. In this process, first, the image input device 11 reads a document and acquires image data (S42). The acquired image data is stored in the storage unit 16.

  Next, the display control unit 94 reads the image data from the storage unit 16 and displays it on the image display device 93 as a preview image (S43).

  Next, on the screen of the image display device 93 displaying the preview image, when the user designates a selection area to be displayed / hidden (S44), the designated content (setting information) is stored. Stored in the unit 16. The selection area is specified by, for example, the user specifying the upper left position and the lower right position of the area to be specified with a finger. At this time, if the preview image is small and it is desired to display an enlarged image, the user may press and hold the center of the area to be enlarged to perform the enlarged display.

  Thereafter, the mask image generation unit 61 of the file generation unit 40 performs the process of S45, the drawing command generation unit 62 performs the process of S46, and the formatting processing unit 63 performs the process of S47. Thereby, the display switching PDF file 73 is generated. The processes in S45 to S47 are the same processes as S24 to S26 in FIG.

  The formatting process in S47 is as shown in FIG. The process of S51-S52 shown to (b) of FIG. 22 is the same process as the process of S31-S32 of (b) of FIG.

  Also in the image processing apparatus 92 according to the present embodiment, when the display switching PDF file 73 is displayed, the button 71 is operated to switch between display and non-display of the marker area mask image, so that predetermined image data is displayed. The display state and non-display state of the area can be easily switched. Further, by embedding the button 71 in the display switching PDF file 73 in a state where it is not printed, it is possible to prevent the button 71 from being printed when the display switching PDF file 73 is printed.

[Embodiment 9]
Still another embodiment of the present invention will be described below with reference to the drawings.
The image forming apparatus according to the present embodiment may process image data acquired by camera shooting. In this case, the image processing apparatus provided in the image forming apparatus is supplied with image data from, for example, a portable terminal device or a tablet terminal device (including the digital camera itself) having a digital camera function instead of the scanner.

  FIG. 23 is a block diagram showing a configuration of the mobile terminal device 100 used in the image forming apparatus 131 according to the embodiment of the present invention. 24 is a block diagram illustrating a configuration of the image processing unit 103 included in the mobile terminal device 100 illustrated in FIG. FIG. 25 is a block diagram showing a configuration of the image forming apparatus 131 according to the embodiment of the present invention. FIG. 26 is a block diagram illustrating a configuration of the image processing unit 142 included in the image forming apparatus 131 illustrated in FIG.

  As illustrated in FIG. 23, the mobile terminal device 100 includes an imaging unit 101, a captured image determination unit 110, an image processing unit 103, a communication unit 104, a display unit 105, an input unit 106, a recording medium access unit 107, a storage unit 108, A control unit 109 and a display control unit 111 are provided.

  The imaging unit 101 captures an imaging target using a CCD sensor or a CMOS sensor, and acquires a captured image (image data). The image processing unit 103 performs processing according to the output destination of the image data on the input captured image (image data). The captured image determination unit 110 determines whether the captured image is appropriate for the image data processed by the image processing unit 103.

  The communication unit 104 has a wireless data communication function such as serial transfer or parallel transfer based on the USB (Universal Serial Bus) 1.1 or USB 2.0 standard. The communication unit 104 transmits the output target image data to the image forming apparatus 131 or the image display apparatus in accordance with a transmission instruction input by the user.

  The display unit 105 is configured by, for example, a touch panel type liquid crystal display. The display control unit 111 displays the captured image (image data) on the display unit 105. In addition, an instruction mark indicating an outer frame of the image detected from the image (image data) and an instruction mark indicating whether the image is appropriately captured (a part of the image is not cut off) are displayed. .

  The input unit 106 has a plurality of buttons, and various instructions are input by the user. The input unit 106 includes, for example, a shutter, an autofocus setting button, an exposure adjustment bar, an image reading button (a button for reading a captured image), a condition setting button, and the like.

  The recording medium access unit 107 reads a program from an external recording medium on which a program for performing each process of the mobile terminal device 100 is recorded. This program is stored in the storage unit 108. The storage unit 108 stores a program for performing each process of the mobile terminal device 100, model information of the mobile terminal device 100, user information, and data necessary for performing the process. The control unit 109 controls the operation of each unit of the mobile terminal device 100 based on the program stored in the storage unit 108.

  As illustrated in FIG. 24, the image processing unit 103 includes an A / D conversion unit 121, an inclination detection unit 122, a geometric arrangement detection unit 123, and an imaging condition detection unit 124.

  The A / D conversion unit 121 performs A / D conversion on the image data acquired by the imaging unit 101, and the tilt detection unit 122 detects the tilt of the image when the imaging unit 101 captures an image.

  The geometrical arrangement detection unit 123 assumes that the imaging target is rectangular and extracts an edge pixel group that serves as a boundary between the imaging target and the background, so that the geometrical arrangement of the imaging target ( Detect geometric distortion). For this purpose, the geometric arrangement detection unit 123 performs edge detection, line segment extraction, and line segment intersection calculation in the image data. The imaging condition detection unit 124 determines the position of an image captured by the imaging unit 101.

  As shown in FIG. 25, the image forming apparatus 131 includes an image reading unit 141, an image processing unit 142, an authentication unit 143, an image forming unit 144, a display unit 145, an input unit 146, a first communication unit 147, and a second communication unit. 148, a recording medium access unit 149, a storage unit 150, and a control unit 151.

  The image reading unit 141 reads an image of a document and acquires image data. The image processing unit 142 performs processing or the like on the input image data according to the output destination of the image data. The authentication unit 143 performs user authentication when outputting the image data processed by the image processing unit 142.

  The image forming unit 144 prints an image corresponding to the image data on a recording material such as paper by, for example, an electrophotographic method or an inkjet method. The display unit 145 is configured by a display device such as a liquid crystal display disposed on the operation panel 18 as the input unit 146, for example, and displays various types of information including image data. The input unit 146 includes, for example, a touch panel disposed on the display unit 145 and input keys, and various instructions are input by the user.

  The first communication unit 147 has a wireless data communication function such as serial transfer and parallel transfer based on the USB (Universal Serial Bus) 1.1 or USB 2.0 standard.

  The second communication unit 148 performs data communication using a wireless technology based on any one of IEEE802.11a, IEEE802.11b, and IEEE802.11g, which are wireless LAN standards. Alternatively, it has a function of a communication interface using Ethernet (registered trademark), and performs data communication with a network via a LAN cable. Alternatively, data communication is performed using a wireless technology based on one of infrared communication standards such as IEEE802.15.1 (so-called Bluetooth) and IrSimple which are wireless communication standards.

  The recording medium access unit 149 reads a program from a recording medium on which the program is recorded outside the image forming apparatus 131. This program is stored in the storage unit 150.

  The storage unit 150 stores a program used for the control of the control unit 151, the received model information of the mobile terminal device, user information, and data necessary for processing.

  The control unit 151 controls the operation of each unit based on a program stored in the storage unit 150.

  As shown in FIG. 26, the image processing unit 142 includes an image quality adjustment unit 161, a geometric correction unit 162, a region selection unit 163, a lens distortion correction unit 164, a super-resolution correction unit 165, a file generation unit 166, and an output process. Part 167 is provided.

  The image quality adjustment unit 161 corrects color balance, contrast, and brightness for the input image data.

  The geometric correction unit 162 forms a document image on a rectangular imaging object such as a poster or manuscript paper when the image data is acquired by imaging with the imaging unit 101. Distortion of an imaging object (that is, distortion of a rectangular plane on which a document image is formed) due to imaging from a direction different from the normal direction of the plane is corrected. Furthermore, the inclination of the imaging target in the image data is corrected.

  The area selection unit 163 performs the same process as the area selection unit 34.

  The lens distortion correction unit 164 corrects the lens distortion when the image data is acquired by imaging with the imaging unit 101. For example, the edge pixels of the imaging object in the image data are sequentially detected by raster scanning, the detected edge pixels are approximated by a curve, and lens distortion correction is performed from the equation of the curve.

  The super-resolution correction unit 165 performs super-resolution correction on the output target image data. In the present embodiment, the super-resolution correction unit 165 performs super-resolution correction on the output image data based on one output target image data.

  Regarding a method for creating a super-resolution image from one image data, several methods are introduced in the Journal of the Institute of Image Information and Television Engineers Vol.62, No.2, pp.181-189 (2008).

  In general, the edge directionality of the image pattern is detected, interpolation is performed according to the orientation, and noise removal processing is performed for the purpose of removing distortion caused by interpolation and the influence of noise components existing in the input image. Thus, super-resolution correction can be executed. This will be specifically described below.

  FIG. 28 is a flowchart showing the flow of super-resolution correction processing in the present embodiment. Here, an example of performing double resolution conversion in each of the horizontal direction and the vertical direction will be described. In the case of performing double resolution conversion, when the number of pixels of output target image data to be corrected is n × m, the number of pixels of super-resolution image data after correction is 2n × 2 m. Such super-resolution correction (double resolution conversion) uses each pixel in the output target image data as a reference pixel, generates a new pixel as an interpolation pixel in the center between the reference pixels, This is executed by generating image data including both of the interpolation pixels as super-resolution image data. FIG. 29 shows the relationship between the reference pixel and the interpolation pixel, where the pixel a indicates the reference pixel and the pixel b indicates the interpolation pixel.

  First, the super-resolution correction unit 165 performs edge extraction on the output target image data. For example, the super-resolution correction unit 165 performs edge extraction using a primary differential filter as shown in FIG. 30, performs binarization processing, and generates binarized image data (S101). If the pixel value is 1 in the binarized image data, it indicates that the pixel is highly likely to be an edge.

  Next, the super-resolution correction unit 165 determines whether the pixel of interest in the captured image data is an edge based on the binarized image data generated in S101 (S102). Specifically, if the value of the pixel corresponding to the pixel of interest in the binarized image data is 1, the super-resolution correction unit 165 determines that the pixel of interest is an edge.

  Note that the pixel of interest refers to a pixel of interest when attention is paid to each pixel in the captured image data in an arbitrary order.

  When the target pixel is an edge (Yes in S102), the super-resolution correction unit 165 detects the edge direction using N × N (N> 1) partial images including the target pixel (S103). Specifically, it is determined whether or not all reference pixels included in the N × N partial image are edge pixels. When the upper left reference pixel and the lower right reference pixel of the target pixel are edge pixels, the super-resolution correction unit 165 determines that the edge direction in the partial image is the upper left-lower right direction. Similarly, if the left reference pixel and the right reference pixel of the target pixel are edge pixels, it is determined that the edge direction is the left-right direction, and the reference pixel above and below the target pixel are If the pixel is an edge pixel, the edge direction is determined to be an upper-lower direction. If the upper right reference pixel and the lower left reference pixel of the target pixel are edge images, the edge direction is determined to be an upper right-lower left direction. To do.

  In FIG. 31, the dotted line indicates the detected edge direction. In FIG. 31, pixels (1) to (9) are reference pixels, and pixel (5) is a pixel of interest. Pixels A, B, and C are an interpolation pixel between the reference pixels (1) and (5), an interpolation pixel between the reference pixels (2) and (5), and a reference pixel (4), respectively. Interpolated pixels between (5).

  Next, in accordance with the edge direction detected in S103, the super-resolution correction unit 165 includes the interpolation pixel A at the upper left of the target pixel, the interpolation pixel B above the target pixel, and the interpolation pixel C at the left of the target pixel. The value is obtained by interpolation. At this time, the pixel value of the interpolation pixel is obtained using the reference pixel along the edge direction.

When the edge direction is the upper left-lower right direction, as shown in FIG. 31A, the reference pixels (1), (5), (9) are edge pixels, and a line connecting these pixels is an edge. Become a line. For the pixel value VA of the interpolation pixel A on the edge line (the notation of “V” is omitted in the figure. The same applies hereinafter), the reference pixel (1) (pixel value) on the edge line adjacent to the interpolation pixel A V (1)) and the pixel value of the reference pixel (5) (pixel value V (5)), the following equation VA = (V (1) + V (5)) / 2
Ask for.

On the other hand, for the interpolation pixels B and C that are not on the edge line, the reference pixel that is the closest to the interpolation pixel (the closest reference pixel) among the reference pixels other than the reference pixel on the edge line is on a line parallel to the edge direction. Is interpolated using the reference pixel. For example, in FIG. 31A, the interpolation pixel B includes the reference pixel (2) which is the closest reference pixel, and the line parallel to the edge direction is a line connecting the reference pixels (2) and (6). It is. Then, the point perpendicular to the line from the interpolation pixel B internally divides the line segment connecting the reference pixels (2) and (6). Therefore, the pixel value VB of the interpolation pixel B is expressed by the following formula VB = (9 × V (2) + 4 × V (6)) / 13.
Find using.

Similarly, the pixel value VC of the interpolation pixel C is the pixel value of the reference pixel (4) that is the closest reference pixel and the reference pixel (8) on the line that includes the reference pixel (4) and is parallel to the edge direction. Using the following formula VC = (9 × V (4) + 4 × V (8)) / 13
Ask for.

When the edge direction is the left-right direction, as shown in FIG. 31B, the reference pixels (4), (5), and (6) are edge pixels, and a line connecting these pixels is It becomes an edge line. Then, for the pixel value VC of the interpolation pixel C on the edge line, the reference pixel (4) (pixel value V (4)) and the reference pixel (5) (pixel value V (5) on the edge line adjacent to the interpolation pixel C are used. ), The following formula VC = (V (4) + V (5)) / 2
Ask for.

On the other hand, for the interpolation pixels A and B that are not on the edge line, of the reference pixels excluding the reference pixel on the edge line, the reference pixel closest to the interpolation pixel (nearest reference pixel) is included on the line parallel to the edge direction. Is interpolated using the reference pixel. For example, in FIG. 31B, the interpolation pixel A includes the reference pixel (1) or (2) that is the closest reference pixel, and the lines parallel to the edge direction are the reference pixels (1) and (2). Is a line connecting A point perpendicular to the line from the interpolation pixel A exists at the center of the reference pixels (1) and (2). Therefore, the pixel value VA of the interpolation pixel A is expressed by the following formula VA = (V (1) + V (2)) / 2.
Find using.

  The interpolated pixel B includes the reference pixel (2) that is the closest reference pixel, and the line parallel to the edge direction is a line connecting the reference pixels (1), (2), and (3). A point perpendicular to the line from the interpolation pixel B coincides with the reference pixel (2). Therefore, the pixel value VB of the interpolation pixel B is set to the same value as the pixel value V (2) of the reference pixel (2).

  When the edge direction is the upper right-lower left direction, as shown in FIG. 31C, the reference pixels (3), (5), (7) are edge pixels, and a line connecting these pixels is It becomes an edge line. The interpolation pixels A, B, and C are not all on the edge line.

For the interpolated pixel A, the closest reference pixels are the reference pixels (1), (2), and (4). Here, the reference pixels (2) and (4) are located on the same line parallel to the edge direction, but the reference pixel (1) is not located on the line. Therefore, for the pixel value VA of the interpolated pixel A, using the pixel values of the reference pixels (1), (2), and (4) that are the closest reference pixels, the following equation VA = (V (1) + V (2) ) + V (4)) / 3
Ask for.

On the other hand, for the interpolation pixels B and C, among the reference pixels excluding the reference pixels on the edge line, the reference pixel closest to the interpolation pixel (nearest reference pixel) and the reference pixel on the line parallel to the edge direction is selected. To interpolate. For example, in FIG. 31C, the interpolation pixel B includes the reference pixel (2) that is the closest reference pixel, and the line parallel to the edge direction is a line connecting the reference pixels (2) and (4). It is. Then, the point perpendicular to the line from the interpolation pixel B internally divides the line segment connecting the reference pixels (2) and (4). Therefore, the pixel value VB of the interpolation pixel B is expressed by the following formula VB = (9 × V (2) + 4 × V (4)) / 13.
Find using.

Similarly, the pixel value VC of the interpolation pixel C is the pixel value of the reference pixel (4) that is the closest reference pixel and the reference pixel (2) on the line including the reference pixel (4) and parallel to the edge direction. Using the following formula VC = (4 × V (2) + 9 × V (4)) / 13
Ask for.

When the edge direction is the upward-downward direction, as shown in FIG. 31D, the reference pixels (2), (5), and (8) are edge pixels, and a line connecting these pixels is It becomes an edge line. For the pixel value VB of the interpolation pixel B on the edge line, using the pixel values of the reference pixel (2) and the reference pixel (5) on the edge line adjacent to the interpolation pixel B, the following formula VC = (V ( 2) + V (5)) / 2
Ask for.

On the other hand, for the interpolation pixels A and C that are not on the edge line, the reference pixel that is the closest to the interpolation pixel (the closest reference pixel) among the reference pixels excluding the reference pixel on the edge line, and that is parallel to the edge direction. Is interpolated using the reference pixel. For example, in FIG. 31D, the interpolation pixel A includes the reference pixel (1) or (4) which is the closest reference pixel, and the lines parallel to the edge direction are the reference pixels (1) and (4). Is a line connecting A point perpendicular to the line from the interpolation pixel A exists at the center of the reference pixels (1) and (4). Therefore, the pixel value VA of the interpolation pixel A is expressed by the following formula VA = (V (1) + V (4)) / 2.
Find using.

  The interpolated pixel C includes the reference pixel (4) which is the closest reference pixel, and the line parallel to the edge direction is a line connecting the reference pixels (1), (4) and (7). A point perpendicular to the line from the interpolation pixel C coincides with the reference pixel (4). Therefore, the pixel value VC of the interpolation pixel C is set to the same value as the pixel value V (4) of the reference pixel (4).

  Note that the storage unit 150 stores in advance information that associates the edge direction with an arithmetic expression for obtaining the pixel values of the interpolation pixels A, B, and C. Then, the super-resolution correction unit 165 reads the arithmetic expression corresponding to the edge direction detected in S103 from the storage unit 150, and obtains the pixel values of the interpolation pixels A, B, and C based on the read arithmetic expression. That's fine.

  FIG. 31 shows only when the edge direction is linear. However, the edge may be bent in the N × N partial image. For example, there are a case where the edge is bent as a reference pixel (2)-(5)-(4) and a case where the edge is bent as a reference pixel (1)-(5)-(7). Also in such a case, the information which matched the computing equation for calculating | requiring the pixel value of interpolation pixel A, B, C is stored beforehand. For example, when the edge bends like the reference pixels (2)-(5)-(4), the interpolation pixel A is the same as in FIG. 31 (c), and the interpolation pixel B is the same as in FIG. 31 (b). For the interpolation pixel C, an arithmetic expression similar to that shown in FIG. Further, when the edge bends like the reference pixels (1)-(5)-(7), the interpolation pixel A is the same as in FIG. 31 (a), and the interpolation pixel B is the same as in FIG. 31 (a). For the interpolation pixel C, an arithmetic expression similar to that shown in FIG. The other edge direction patterns are also stored in the same manner.

  In this manner, the super-resolution correction unit 165 obtains the pixel value of the interpolation pixel located around the reference pixel determined as the edge pixel.

  On the other hand, when the target pixel is not an edge (No in S102), the super-resolution correction unit 165 causes the interpolation pixel A adjacent to the upper left of the target pixel, the interpolation pixel B adjacent to the target pixel, and the target pixel. The pixel value of the left interpolation pixel C is obtained by a general interpolation calculation method (bilinear bicubic etc.) (S104).

  The super-resolution correction unit 165 generates the interpolated image data including both the reference pixel and the interpolation pixel by executing the processing of S102 to S104 for all the reference pixels included in one image data. (S105).

  Thereafter, the super-resolution correction unit 165 performs high image quality processing on the generated interpolated image data. For example, the super-resolution correction unit 165 applies a noise removal filter, a sharpening filter, or the like to the interpolated image data to generate super-resolution image data. A conventional unsharp mask or a filter with a center coefficient of 5 in FIG. 30 is a sharpening filter. As a noise removal, a median filter or the like is widely known. As a more advanced method, a Bilateral filter [Proceedings of the 1998 IEEE International Conference on Computer Vision,] or the like may be used as a method having both the above-described edge preservation and high image quality.

  The super-resolution correction unit 165 is not limited to the above-described method, and various methods as described in the Journal of the Institute of Image Information and Television Engineers Vol. 62, No. 2, pp. 181-189 (2008). May be used to generate super-resolution image data from one captured image data.

  The file generation unit 166 performs the same processing as the file generation unit 40.

  The output processing unit 167 includes a processing unit necessary for outputting an image by printing, facsimile transmission, image transmission such as e-mail, and filing. Specifically, it includes the region separation processing unit 34, the color correction unit 35, the black generation and under color removal processing unit 36, the spatial filter processing unit 37, the halftone processing unit 38, and the like shown in FIG.

  When the configuration of the image forming apparatus 131 shown in FIG. 25 is applied to an image display apparatus, the image display apparatus includes an image processing unit 142, an authentication unit 143, a display unit 145, an input unit 146, a first communication unit 147, The second communication unit 148, the storage unit 150, and the control unit (computer) 151 are provided. In this case, the image processing unit 142 includes, for example, the image quality adjustment unit 161, the geometric correction unit 162, the lens distortion correction unit 164, and the super-resolution correction unit 165 shown in FIG. The correction unit 35 and the spatial filter processing unit 37 are provided. Note that the authentication unit 143 and the super-resolution correction unit 165 may be omitted. Other configurations are the same as those shown in the first embodiment.

  In the above configuration, the operations of the mobile terminal device 100 and the image forming apparatus 131 will be described below. FIG. 27A is a flowchart illustrating an outline of processing performed by the mobile terminal device 100 and the image forming apparatus 131. FIG. 27B is a flowchart showing the contents of processing in the mobile terminal device 100. FIG. 27C is a flowchart showing the contents of processing in the image forming apparatus 131 (main apparatus). FIG. 27D is a flowchart showing the contents of the formatting process shown in FIG.

  The generation of the display switching PDF file 73 is performed using a captured image (image data) by the mobile terminal device 100. In this process, as shown in FIG. 27A, the following process setting is performed by the user (S61). This processing setting is performed in the mobile terminal device 100, for example. The setting content is temporarily stored in the storage unit 108 of the portable terminal device 100, then transmitted from the communication unit 104 to the image forming apparatus (MFP) 131, and stored in the storage unit 150 of the image forming apparatus 131.

  (1) When the marker designation mode is selected by the user, the user setting mode is set, and further, for example, in the marker area (marker) according to the guidance screen display displayed on the display unit 105 of the mobile terminal device 100 by the user. The setting of whether to fill in the area specified by (1) or outside the marker area, and the fill color and the transmissivity of the fill color are set.

  (2) When the marker designation mode is selected, a screen that prompts the user to select whether or not to perform display / non-display control of the marker area in the display switching PDF file 73 is displayed on the display unit 105 of the mobile terminal device 100.

  (3) When display / non-display control is selected, when the created display switching PDF file 73 is opened, a screen prompting selection of whether or not to display the marker area is displayed. On this screen, it is also required to select whether the control object is inside or outside the marker area.

  Next, predetermined processing (terminal-side processing) is performed in the mobile terminal device 100 (S62), and information and image data used in each correction unit in the image processing unit 142 of the image forming apparatus 131 are stored in the image forming apparatus 131. Sent.

  The image forming apparatus 131 performs predetermined processing (main body side processing) on the information and image data received from the mobile terminal device 100 (S63).

  Next, the image forming apparatus 131 performs marker recognition processing (S64), mask image layer information generation processing (S65), drawing command generation processing (S66), and formatting processing (S67) on the image data, and displays them. A switching PDF file 73 is generated. The contents of the formatting process (S67) are the processes of S91 and S92 shown in (d) of FIG. Further, the processes of S64 to S67 are the same as the processes of S23 to S26 shown in FIG. 6A, and the processes of S91 and S92 are the same as those of S31 and S32 shown in FIG. Same as processing.

  Further, the terminal side processing in S62 is as shown in FIG. 27B, and the main body side processing in S63 is as shown in FIG. 27C.

  In the terminal-side processing shown in FIG. 27B, in the mobile terminal device 100, the imaging unit 101 sequentially captures images in accordance with the user's operation to acquire captured images (image data) (S71), and the captured image determination The unit 110 determines whether or not these captured images (image data) are appropriate (S72). If the captured image (image data) is not appropriate, the display unit 105 notifies the user of that fact (S73), and the user captures the image again.

  As a result of the determination in S72, an appropriate captured image (image data) is stored in the storage unit 108 (S74), and a file name is created (S75).

  Next, the captured image (image data) stored in the storage unit 108 includes geometric correction information, model information of the mobile terminal device, user information, and setting information attached as attribute information. MFP) 131 (or image display device) (S76). Thereafter, when all the captured images (image data) are transmitted (S77), the process is terminated.

  In the main body side processing shown in FIG. 27C, when the image forming apparatus 131 receives a captured image (image data) and various attribute information from the mobile terminal device 100 (S81), the image quality adjustment of the image processing unit 142 is performed. Color balance / contrast / brightness correction is performed by the unit 161 (S82). In addition, the geometric correction unit 162 performs geometric correction and inclination correction (S83). Further, super-resolution correction is performed by the super-resolution correction unit 165 (S84), and the processed image data is stored in the storage unit 150 (S85).

  Also in the image processing unit 142 of the present embodiment, in a state where the display switching PDF file 73 is displayed, the button 71 is operated to switch between display and non-display of the marker area mask image, thereby causing the predetermined image data to be displayed. The display state and non-display state of the area can be easily switched. Further, by embedding the button 71 in the display switching PDF file 73 in a state where it is not printed, it is possible to prevent the button 71 from being printed when the display switching PDF file 73 is printed.

[Embodiment 10]
(Example of software implementation)
Each block (particularly the region selection unit 34 and the file generation unit 40) of the image processing devices 12, 82, 92 and the image processing unit 142 is realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. Alternatively, it may be realized by software using a CPU (Central Processing Unit).

  In the latter case, the core device includes a CPU that executes instructions of a program that is software that realizes each function, a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by a computer (or CPU), or A storage device (these are referred to as “recording media”), a RAM (Random Access Memory) that expands the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Summary]
An image processing apparatus according to aspect 1 of the present invention detects a selection area specified for input image data, and generates a selection area information (marker recognition section 52) that generates selection area information indicating the selection area; A mask image generating unit 61 that generates mask image data corresponding to the selection area with reference to the selection area information, the mask image data is arranged in association with the selection area of the input image data, and the mask A file (display switching PDF) to which a selection input section (button 71) for selecting display or non-display of image data is added and the mask image data is switched between display and non-display according to the selection input to the selection input section A file generation unit (drawing command generation unit 62, formatting processing unit 63) for generating the file 73).

  According to the above configuration, the selection area recognition unit detects a selection area designated for the input image data, and generates selection area information indicating the selection area. The mask image generation unit 61 refers to the selection area information and generates mask image data corresponding to the selection area. In the file generation unit, mask image data is arranged in association with a selection area of input image data, a selection input unit for selecting display or non-display of mask image data is added, and in response to a selection input to the selection input unit A file for switching between display and non-display of the mask image data is generated.

  Therefore, since the generated file is switched between display and non-display of the mask image data by the user operating the selection input unit, the display state and non-display state of the predetermined area in the file can be easily switched. it can.

  Thus, for example, when viewing a file alone, the mask image data is not displayed and a predetermined area of the file is displayed, while when the file is viewed with another person, the mask image data is displayed. It is possible to display such that a predetermined area of the file is hidden (hidden).

  The image processing apparatus according to aspect 2 of the present invention is the image processing apparatus according to aspect 1, wherein the selection area indicated by the selection area information is specified to surround a predetermined area of the input image data, and the mask image generation The unit 61 generates the mask image data that fills either the inner region of the selection region or the outer region of the selection region, and the file generation unit (drawing command generation unit 62, formatting processing unit 63) It is good also as a structure which sets the transmittance | permeability of the said mask image data when displaying the said file (display switching PDF file 73).

  According to the above configuration, the mask image generation unit 61 generates mask image data that fills either the inner area of the selected area or the outer area of the selected area that is specified to surround the predetermined area of the input image data. To do. The file generation unit sets the transmittance of the mask image data when displaying the image file.

  According to the above configuration, if the user designates the selection area so as to surround the predetermined area of the input image data, for example, depending on the user's selection, either the inner area of the selection area or the outer area of the selection area is selected. Mask image data to be filled can be generated. Further, the transmittance of the mask image data can be set by the file generation unit, for example, according to the user's selection. Therefore, when the generated image file is displayed, the region masked by the mask image data of the input image data is displayed in a semi-transparent state (transmittance 50%, masked region) according to the preset setting. Display in which the presence of an image can be confirmed) or non-transmission state (transmittance is 0%, masked area is not displayed). Note that the transmittance is not set when filling.

  In the image processing apparatus according to aspect 3 of the present invention, in the above aspect 1 or 2, the file generation unit (drawing command generation unit 62, formatting processing unit 63) prints the image file (display switching PDF file 73). In this case, the selection input unit (button 71) may be set not to be printed.

  According to said structure, when printing a file, the situation where the selection input part which is unnecessary on a print image is printed can be prevented.

  In the image processing device according to aspect 4 of the present invention, in any one of the above aspects 1 to 3, the file generation unit (drawing command generation unit 62, formatting processing unit 63) is configured such that the input image data includes a plurality of pages. In this case, the selection input unit (button 71) displayed on each page of a plurality of files (display switching PDF file 73) generated corresponding to each input image data is operated to operate the plurality of files. The mask image data common to these files may be set to be switched between display and non-display.

  According to the above configuration, when the input image data is composed of a plurality of pages, the file generation unit is one of the selection input units displayed on each page of the plurality of files generated corresponding to each input image data. By being operated, a setting is made so that mask image data common to a plurality of files is switched between display and non-display. As a result, the user can collectively switch between displaying and hiding the mask image data of a plurality of files by operating the selection input unit of one of the files.

  An image processing method according to an aspect 5 of the present invention includes a selection area recognition step of detecting a selection area designated for input image data and generating selection area information indicating the selection area, and referring to the selection area information A mask image generation step for generating mask image data corresponding to the selection area; and the mask image data is arranged in association with the selection area of the input image data, and the mask image data is displayed or hidden. And a file generation step of generating a file that switches between display and non-display of the mask image data in accordance with the selection input to the selection input unit.

  According to said structure, there exists an effect similar to the image processing apparatus of aspect 1.

  The image processing apparatus according to each aspect of the present invention may be realized by a computer. In this case, a program for causing the computer to realize the image processing by causing the computer to operate as each unit included in the image processing apparatus, A computer-readable recording medium on which it is recorded also falls within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

  The present invention can be applied to an image processing system that performs data communication between a portable terminal device and an image forming apparatus or an image display apparatus.

DESCRIPTION OF SYMBOLS 1 Image forming apparatus 11 Image input apparatus 12 Image processing apparatus 13 Image output apparatus 14 Control part 18 Operation panel 34 Area selection part 40 File generation part 51 Resolution conversion part 52 Marker recognition part (selection area recognition part)
61 mask image generation unit 62 drawing command generation unit (file generation unit)
63 Formatting processor (file generator)
71 button (selection input section)
72 Marker Line 73 Display Switching PDF File 81 Image Reading Device 82 Image Processing Device 91 Image Forming Device 142 Image Processing Unit

Claims (7)

A selection area recognition unit that detects a selection area designated for input image data and generates selection area information indicating the selection area;
With reference to the selection area information, a mask image generation unit that generates mask image data corresponding to the selection area;
The mask image data is arranged in association with the selection area of the input image data, a selection input unit for selecting display or non-display of the mask image data is added, and in response to a selection input to the selection input unit An image processing apparatus comprising: a file generation unit that generates a file for switching between display and non-display of the mask image data. The selection area indicated by the selection area information is designated to surround a predetermined area of the input image data,
The mask image generation unit generates the mask image data that fills either the inner region of the selection region or the outer region of the selection region,
The image processing apparatus according to claim 1, wherein the file generation unit sets a transmittance of the mask image data when the file is displayed.   The image processing apparatus according to claim 1, wherein the file generation unit is configured not to print the selection input unit when the image file is printed.   When the input image data is composed of a plurality of pages, the file generation unit operates any of the selection input units displayed on each page of a plurality of files generated corresponding to each input image data. 4. The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to switch between display and non-display of the mask image data common to the plurality of files.   An image forming apparatus comprising the image processing apparatus according to claim 1.   A program for causing a computer to function as the image processing apparatus according to claim 1, wherein the program causes the computer to function as each of the above-described units.   A computer-readable recording medium on which the program according to claim 6 is recorded.