JP2015203918A - Image processor, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that the consumption of a holding area of raster image data becomes huge when a system is operated for a long term because a conventionally used source document data retrieval method is configured to hold the whole raster image data during printing.SOLUTION: Print layout information used during printing and a drawing object required to form the layout are divided into partial drawing objects to be held. During source document data retrieval, a print layout is applied to the partial drawing objects to reconstruct drawing contents of a printing time. During retrieving an image coincident with a scan image of a printed matter, retrieval processing which does not need a raster image of a printing time is achieved by making the reconstructed drawing contents a retrieval object.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for searching for an image.

  There is a technique for retrieving document data of a printing source from a printed material. In Patent Document 1, raster image data generated at the time of printing and its feature amount are extracted and stored, and used for image search. When the printed matter to be searched is scanned, the feature amount of the scanned image is extracted, and the similarity with the feature amount of the raster image data held is compared. A document having a high similarity is determined to be original document data, and a search result is output. Even if the layout is different from the original document data, such as when printing after converting to a layout specialized for printing, because the feature quantity for raster image data at the time of printing is used for image search, the original document The data can be searched.

JP 2001-256256 A

  However, since the technique of Patent Document 1 needs to retain all raster image data and its feature values every time a print job is processed, the required raster image data and its features as the number of print jobs increases. The amount holding area also increases. Raster image data and its feature values are retained even when there is almost no change in the drawing content of the printed matter, such as when reprinting one side printed on both sides when reprinting with different paper sizes. Therefore, the consumption of the holding area is wasteful, and this is not a preferable mode in the case where it is assumed that the system is maintained for a long time.

  The image processing apparatus of the present invention that solves the above problems has the following configuration.

  Generation means for generating print layout information, means for generating a partial drawing object by dividing the drawing content to be printed in units of drawing objects, and storing the print layout information and the partial drawing object in association with each other as print information data And holding means for searching for the print information data held by the holding means when searching for an image similar to the input image.

  According to the present invention, it is possible to efficiently search for document data that does not require a storage area for information required for searching.

System configuration diagram Hardware block diagram of image forming apparatus Hardware block diagram of file server Software module diagram of image forming device Software module diagram of file server Original document data search result screen Display screen of document data on PC Print result from PC display Display screen of document data on mobile terminal Data structure of document data Example of print result of document data without page concept Example of print layout for document data without page concept Print information data table Partial drawing object table Example of retention mode for partially drawn objects Flow chart of printing process of image forming apparatus Flow chart of print information data storage processing of file server Flowchart of original document data search processing of printed matter of image forming apparatus Flow chart of original document data search processing of file server

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[Example 1]
<System configuration diagram>
First, an example of a system configuration according to the present invention will be described with reference to FIG. The system according to this embodiment includes a file server 101 and an image forming apparatus (image processing apparatus) 102. The file server 101 is connected to a plurality of image forming apparatuses 102 via a network 103.

  The file server 101 manages document data to be applied to the system of the present embodiment, and associates and holds print information data and document data transmitted from the image forming apparatus 102 during printing. Further, it has a function of searching for an original document data of a printing source by performing an image search from a scanned image of a printed matter transmitted from the image forming apparatus 102. At the time of image search, search processing is performed using print information data received from the image forming apparatus 102 in the past. Details of the image search processing will be described later in detail with reference to FIG.

  An image forming apparatus (image processing apparatus) 102 is a multifunction peripheral having various functions such as scanning, FAX, printing, copying, or a printer having only a printing function. The image forming apparatus 102 transmits print layout information from the print settings used at the time of printing and a partial drawing object included in document data necessary for configuring the layout to the file server 101 as print information data. Details of the print information data will be described later in detail with reference to FIG. Further, the image forming apparatus 102 can scan a printed matter and transmit it to the file server 101 to issue a search request for original document data. A device that prints document data and a device that scans a printed matter and issues a search request for original document data can be executed from different devices. In addition, the function of the file server 101 described in the present embodiment may be provided in the controller of the image forming apparatus 102 and configured only in the image forming apparatus 102. Even in a case where only the image forming apparatus 102 is configured, a plurality of image forming apparatuses 102 may be connected via the network 103 to execute the same function.

  The network 103 is a wireless or wired network composed of a WAN or a LAN.

  The information processing apparatus 104 is a client PC used by a user or a mobile terminal that is becoming popular in recent years. The information processing apparatus 104 can input a print job to the image 102 and transmit document data to the file server 101 via the network 103. A plurality of information processing apparatuses 104 can be connected via the network 103, and document data transmitted to the file server 101 can be shared between the connected information processing apparatuses 104.

<Hardware block diagram of image forming apparatus>
FIG. 2 is a hardware block diagram illustrating the configuration of the image forming apparatus 102 of the present embodiment. FIG. 2 shows only the configuration of the image forming apparatus alone, but the image forming apparatus 102 is connected to devices such as the file server 101 and the information processing apparatus 104 shown in FIG. Is done. A print job is mainly transferred from the information processing apparatus 104 connected to the network 103 to the image forming apparatus 102 via an interface represented by the network 103 or USB.

  In FIG. 2, a controller unit 200 controls input / output of image signals and device information. The central processing unit 201 reads the program stored in the auxiliary storage device 203 to the main storage device 202 and executes it. Further, the central processing unit 201 controls each device connected to the system bus 212 in an integrated manner. The central processing unit 201 may employ a multiprocessor system having a plurality of processors as hardware in order to realize parallel processing. Further, the microprocessor may include a plurality of processor cores. It is also possible to use a technique that can use a technique for making one processor appear as if it is a plurality of processors by effectively utilizing registers and pipeline idle time in the processor such as hyperthreading. A configuration in which many processes are separated and executed on the information processing apparatus 104 connected via the network 103 is also possible. The main storage device 202 functions as the main memory and work memory of the central processing unit 201. The auxiliary storage device 203 is also used for the purpose of holding large-capacity data temporarily or for a long time. The RIP 204 is dedicated hardware for expanding intermediate print data into a raster image. The RIP 204 processes the intermediate print data generated on the main storage device 202 by the central processing unit 201 at high speed and in parallel with the execution of the central processing unit 201. A network 205 is connected to the network 103 and performs input / output of print data and device information to / from the outside of the printer. Alternatively, the program can be installed in the main storage device 203 via the network 205. The scanning apparatus I / F 206 is an interface unit with the scanning apparatus 209, and plays a role of storing the scanned image transmitted / received from the scanning apparatus 209 in the main storage device 202 or the auxiliary storage device 203. An operation unit I / F 207 is an interface unit with the operation unit 210 and outputs image data to be displayed on the operation unit 210 to the operation unit 210. Also, it plays a role of transmitting information input by the user of the apparatus from the operation unit 210 to the central processing unit 201. The operation unit 210 includes a liquid crystal panel and a sound source as an output device, and includes a touch panel, hard keys, and a microphone as input devices. The controller unit 200 is connected to the printer engine 211 via the device I / F 208. The device I / F 208 transmits an image signal, instructs a device operation, and receives device information based on an instruction from the central processing unit 201. The printer engine 211 is an output device that outputs an image signal from the controller unit 200 onto a medium, and may be either an electrophotographic system or an inkjet system. Further, a disk drive for a portable disk recording medium such as a CD or a DVD, a memory reader / writer for a portable non-volatile recording medium such as a flash memory, or the like may be connected to the system bus 212. In addition, a program describing the processing contents of this embodiment can be stored in the auxiliary storage device 203 via the portable storage medium and installed in the image forming apparatus.

<Hardware block diagram of file server>
FIG. 3 is a hardware block diagram illustrating the configuration of the file server 101 of this embodiment. In the file server 101, a program describing the processing contents of this embodiment is stored in the auxiliary storage device 304 and installed in the file server 101. The central processing unit 301 reads the program stored in the auxiliary storage device 304 to the main storage device 302 and executes it. Further, the central processing unit 301 comprehensively controls each device connected to the system bus 305. The main storage device 302 functions as the main memory and work memory of the central processing unit 301. A network 303 is connected to the network 103 and inputs / outputs information such as document data and print information data from / to the outside of the file server 101. A program can also be installed in the auxiliary storage device 304 via the network 303. The auxiliary storage device 304 stores the operating system and the main body of the control program in which the processing contents of this embodiment are described. Document data that is a target of the document data management function in this embodiment and print information data used in the image search function are also held in this area. To the system bus 305, a disk drive for a portable disk recording medium such as a CD or a DVD, a memory reader / writer for a portable nonvolatile recording medium such as a flash memory, or the like may be connected. A program in which the processing content of the embodiment is described can be stored in the auxiliary storage device 304 via the portable storage medium and can be installed in the file server 101.

<Software module diagram of image forming apparatus>
FIG. 4 is a software module configuration diagram of the image forming apparatus 102 according to the present exemplary embodiment. Each software module described in FIG. 4 is stored in the auxiliary storage device 203 as a program, loaded into the main storage device 202, and executed by the central processing unit 201. A transmission unit 401 receives print information data, scan image transmission, print data, and image search results generated by a print information data generation unit 405 (to be described later) from the network 205. The data interpretation unit 402 reads print data received from the transmission / reception unit 401 via the control unit 403, and interprets drawing information in the document. The drawing information extracted as a result of the data interpretation process is sent to the renderer 404 via the control unit 403 and used for the rendering process. The control unit 403 is responsible for overall control of various processes from reception of print data, data interpretation, image rendering, print information data generation, printed matter scan, scan image transmission, and search result reception in this embodiment. The renderer 404 uses the RIP 204 from the drawing information sent from the data interpretation unit 402 to generate a bitmap image. The generated bitmap image is sent to the printer engine via the device I / F 208, and an image forming process is performed on the printing paper. A print information data generation unit 405 generates print information at the time of printing as print information data. The print information data includes print layout information used at the time of printing and a partial drawing object included in document data necessary for configuring the layout. Details of the print information data will be described later in detail with reference to FIG.

<Software module diagram of file server>
FIG. 5 is a software module configuration diagram of the file server 101 in this embodiment. Each software module described in FIG. 5 is stored in the auxiliary storage device 304 as a program in FIG. 3, loaded into the main storage device 302, and executed by the central processing unit 301. The transmission / reception unit 501 receives print information data and scan images transmitted from the image forming apparatus 102 from the network 303, receives document data transmitted from the report processing apparatus 104, and transmits a search result of an image search process described later to the image forming apparatus 101. . The document data management unit 502 uses the auxiliary storage device 304 to hold a plurality of document data to be managed by the system of this embodiment. The control unit 503 is responsible for overall control of various processes such as reception and storage of scanned images, print information data, and document data, image search processing, and search result transmission performed in this embodiment. The print information data management unit 504 holds print information data transmitted from the image forming apparatus 102 using the auxiliary storage device 304. Note that this print information data is held in a form associated with the document data to be printed at the time of printing. The associated document data is the one held in the document data management unit 502. Details will be described later in detail with reference to FIG. The document data search unit 505 searches for document data from the document data managed by the document data management unit 502 via the control unit 503. A scan image is received from the image forming apparatus 102 via the transmission / reception unit 501, the drawing contents at the time of printing are reconstructed from the print information data managed by the print information data management unit 504, and an image search is performed. Document data whose feature amount matches the scanned image is extracted by image search, and the search result is transmitted to the image forming apparatus 102.

<Original document data search result screen>
FIG. 6 is displayed on the operation unit 210 of the image forming apparatus 102. It is a screen displaying the result of searching and executing original document data from a scanned image of a printed matter. When the image forming apparatus 102 receives a search result of document data from the file server 101, this screen is displayed.

  FIG. 6A shows a search result screen when corresponding document data is detected as a search result of the original document data. The pop-up 601 includes a file name form 602, a print layout setting form 603, a print button 604, a detail setting button 605, and a Cancel button 606. The file name form 602 displays the file name of the detected document data. In this embodiment, the document data having the highest similarity between the scanned image and the document data is selected as one file and displayed as a search result. . A print layout setting form 603 displays a print layout setting when printing is performed on a printed matter scanned from the detected document data. Although there are various print settings related to the print layout, in this embodiment, an arbitrary name can be assigned to the print layout information, and the name of the print layout information is displayed. Details of the print layout information described here will be described later in detail with reference to FIG. A print button 604 is a button that is selected when the detected document data is printed again. The detail selection button 605 is a button that is selected when a transition is made to a screen where more detailed print settings can be confirmed and changed. This is selected when it is desired to confirm what print settings the scanned printed material is output, or when only specific print settings are to be changed when the print button 604 is selected and reprinting is performed. The Cancel button 606 is a button that is used to confirm the search result of the original document data and to finish without performing any particular processing.

  FIG. 6B shows a search result screen displayed when the corresponding document data is not detected as the search result of the original document data. The pop-up 701 includes an OK button 702 together with a word that notifies that the corresponding document data is not found as a search result. When the OK button 702 is selected, the pop-up 701 is closed and the normal job standby screen is displayed.

<Display screen of document data on PC>
FIG. 7 shows a display screen when the document data targeted by this embodiment is displayed on the PC. In the present embodiment, document data composed of a document format using HTML and CSS used in many websites is targeted. In this document format, HTML has only the basic structure of document data and information on content elements, and CSS defines how information defined in HTML is laid out and displayed.

  7A and 7B, the content forms 801 and 802 display content elements defined in HTML, and the display contents are changed by the user's scrolling operation. In areas other than the content forms 801 and 802, the display content does not change regardless of the user's scroll operation. When rolling down from the state of FIG. 7A, the state transitions to FIG. When rolling up from the state of FIG. 7B, the state transitions to FIG.

<Print result from PC display>
FIG. 8 shows an example of the print result when the document data targeted by the present embodiment is instructed from the PC. Document data in the document format using HTML and CSS targeted by this embodiment generally has different print results depending on the display contents displayed on the PC at the time of printing. This is because the document format using HTML and CSS does not have the concept of a page, and the rendering result cannot be specified fixedly on each page. How a document without a page concept is laid out in physical coordinates of a printed matter depends on the algorithm of the printer driver of each image forming apparatus 102.

  In the example of FIG. 8, the drawing contents displayed on the PC are expanded on the main memory of the PC, and the layout processing is performed so that the drawing contents expanded on the main memory are printed as they are when a print instruction is issued. Is shown. FIG. 8A shows a print result that is printed when a print instruction is given in the state of FIG. 7A, and the drawing content of the content form 801 of FIG. FIG. 8B shows a print result printed when a print instruction is given in the state of FIG. 7B. The drawing content of the content form 802 of FIG. 7B is also printed in the print area 902. The The print areas 901 and 902 are switched depending on what is drawn on the content forms 801 and 802 on the PC. Even if a drawing object is defined as a content element in HTML, if it is not displayed on the PC, it is excluded from the drawing object and is not printed.

  In recent years, an increasing number of websites are compatible with the display environment of mobile terminals such as tablets and multi-function mobile phones (smartphones) in a document format without these page concepts. In these websites, multiple CSSs defined for PC and mobile terminals are prepared, and the CSS is selectively switched according to the display terminal, thereby realizing the optimal layout display for the display terminal. Yes.

<Display screen of document data on mobile terminal>
FIG. 9 shows a display screen when document data using HTML and CSS corresponding to the display environment of these mobile terminals is displayed on the mobile terminal. Since the display area of the mobile terminal is narrower than that of the PC, the mobile terminal has a vertically long layout, and the entire drawing content is changed by the user's scroll operation. When rolling down from the state of FIG. 9A, the drawing contents transition to FIG. 9B and FIG. 9C. Further, when the roll-up is performed from the state of FIG. 9C, the drawing content transitions to FIGS. 9B and 9A. As a whole, the vertically drawn content as shown in FIG. 9D is scroll-displayed.

<Data structure of document data>
The data structure of document data in document format using HTML and CSS corresponding to the display environment of these PCs and mobile terminals will be described in more detail with reference to FIG.

  FIG. 10A shows an outline of the HTML data structure. In the basic structure area 1001, the basic structure of the Web site is defined as header information. In addition to the page title, the character code to be used, the outline of the page, the display character string at the time of bookmarking, the definition of the embedded advertisement, etc. are written. In addition, the definition of switching which CSS is read according to the display environment of the PC or mobile terminal is also described in this basic structure area 1001. The content elements 1002, 1003, and 1004 define content contents to be displayed on the Web site, respectively. Here, it is possible to define an image such as a character string, a graphic, or a photograph, or JavaScript (registered trademark) that dynamically switches display contents in accordance with a user operation. In this embodiment, it is assumed that the content element 1002 defines Web site title information, the content element 1003 defines Web site menu information, and the content element 1004 defines article text and image information.

  FIG. 10B shows an outline of the CSS data structure for PC. CSS defines how to lay out the basic structure and content element information defined in HTML. In FIG. 10B, content areas 1101, 1102, 1103, and 1104 are defined, and a layout can be individually defined in each area. For example, the width and height of the display area, coordinate values, background color and font size, behavior of display contents according to user operations, and the like can be defined. An HTML content element is assigned to each content area, and a defined layout is applied to each content area. In the example of FIG. 7, a content element 1002 that is website title information is assigned to the content area 1101, and a content element 1103 that is website menu information is assigned to the content area 1102. Content elements 1004 that are article text and image information are assigned to the content area 1103, and embedded advertisement information is assigned to the content area 1104 from the basic structure area 1001.

  FIG. 10C shows an outline of the CSS data structure for the mobile terminal. Similarly, content areas 1201, 1202, 1203, and 1204 are defined, and a layout is individually defined in each area. Also here, CSS defines how to lay out the basic structure and content element information defined in HTML and realize display suitable for the display environment of the mobile terminal. In the example of FIG. 9, a content element 1002 that is Web site title information is assigned to the content area 1201, and a content element 1003 that is Web site menu information is assigned to the content element 1202. Content elements 1004 that are article documents and image information are assigned to the content area 1203, and embedded advertisement information is assigned to the content area 1204 from the basic structure 1001.

  In the document format using HTML and CSS corresponding to the display environment of the PC and mobile terminal, the CSS described in FIGS. 10B and 10C is switched according to the display environment at the stage of reading the document data. Is changing.

<Special print layout for document data without page concept>
As described above, the document format using HTML and CSS used on the website described with reference to FIGS. 7 to 10 does not have a page concept. Therefore, when printing, it is unique for laying out the physical coordinates of the printed matter. The layout algorithm is applied. Various kinds of unique layout algorithms have been studied and developed, and examples thereof are shown in FIGS.

  FIG. 11A shows a printing result when printing is performed by applying the layout algorithm using the mobile terminal layout described in FIGS. 9 and 10C. Here, the vertically drawn content defined by CSS for the display environment of the mobile terminal is divided into strips by the length of the vertical width of the printed matter and arranged horizontally. This layout structure is defined in the print layout information shown in FIG. 12A. In this embodiment, the layout structure is named mobile terminal layout. In the content area 1301, the drawing content of the content element 1002 that is title information of the Web page is printed. In the content area 1302, the drawing content of the content element 1003, which is menu information of the Web page, is printed. In the content area 1303, the drawing content of the content element 1004 which is an article document and image information is printed. In the content area 1304, the contents of the embedded advertisement information defined in the basic structure 1001 are printed. In this layout, the drawing content of the vertical width is printed in a single printed matter, so that the number of print pages is small, the drawing content is the same as that of the mobile terminal, and a print result in which information is collected can be obtained.

  FIG. 11B shows an example of a printing result when a layout algorithm that is a little modified to the PC layout described in FIGS. 8 and 9A and 9B is applied. Here, the layout is rearranged so that all the drawing objects can be printed on one printed matter from the basic structure and content element information defined in HTML. This layout structure is defined in the print layout information shown in FIG. 12B. In this embodiment, the layout structure is named as an aggregate PC layout. In the content area 1401, the drawing content of the content element 1002 that is the title information of the Web site is printed. In the content area 1402, the drawing content of the content element 1003 that is menu information of the Web site is printed. In the content area 1403, the drawing content of the content element 1004, which is an article document and image information, is printed. In the content area 1404, the drawing content of the embedded advertisement information defined in the basic structure 1001 is printed. In this layout, the content document 801 displayed by scrolling the content field 801 in the PC-use CSS shown in FIG. 7 can be printed on a single printed matter by changing the layout. Is. Although the print result is different from the display on the PC, information on all the content elements defined in HTML can be combined into a single printed matter.

  FIG. 11C shows an example of a print result when a layout algorithm that has been greatly modified is applied to the PC layout described with reference to FIGS. 8 and 9A and 9B. Here, the layout is rearranged so as to print only drawing objects with a small amount of toner consumption during printing, based on the basic structure and content element information defined in HTML. This layout structure is defined in the print layout information shown in FIG. 12C, and is named “consumed toner reduction layout” in this embodiment. In the content area 1501, the drawing content of the content element 1002, which is the title information of the website, is printed in a form that consumes less toner. In the content area 1502, the drawing content of the content element 1003 that is the menu information of the website is printed in a form that consumes less toner. In the content area 1503, the drawing content of the content element 1004, which is an article document and image information, is printed in a form with less toner consumption. In this layout, in order to reduce toner consumption at the time of printing, thinning is performed for a drawing object having a large amount of toner usage so that the amount of toner usage is reduced. For example, in the content area 1501 in FIG. 12C, the title information is printed in monochrome without drawing the background in order to reduce the amount of toner used. Also in the content area 1503, the background is not drawn, and an image drawing object such as a photograph that consumes a large amount of toner is not printed, and only a vector graphic is printed as a drawing object. This also results in a printing result different from the display on the PC, but when only the necessary information of the document data is to be printed, a printing result with a reduced toner usage can be obtained.

<Outline of original document data search processing>
The present invention does not hold the raster image data generated at the time of printing and its feature values for all print jobs, but holds the print layout information used at the time of printing and the partial drawing objects necessary for configuring the layout. This reduces the consumption of the holding area. That is, the raster image is not held as it is, but is divided in units of drawing objects used in the print layout and is held as a partial drawing object by partial raster image, vector data, or font information. Various print layouts can be applied to document data without a page concept. However, if the same partial drawing object is used even if the print layout is different, the corresponding partial drawing object is used. Can be turned. Therefore, by using these partial raster images, vector data, and font information as partial drawing objects between different print layouts, the storage area consumption is reduced. Hereinafter, the details of the original document data search processing of the present invention will be described with reference to FIGS.

  Note that the present invention may be applied to document formats of general office documents as well as document data on websites using HTML and CSS as original document data to be searched for printed matter. Is possible. Since a general office document has a page concept and holds a layout structure as a data structure, it is not necessary to define a print layout unlike document data without a page concept. However, as long as print layout information is separately generated and partial drawing object information necessary for configuring the layout is held as in the present embodiment, it can be applied to a document format without a page concept. it can.

<Print information data table>
FIG. 13 shows a print information data table. The print information data table is generated by the print information data generation unit 405 when the image forming apparatus 102 prints a print job transmitted from the information processing apparatus 104, and is transmitted to the file server 101. The file server 101 receives the transmitted print information data table and holds it in the auxiliary storage device 304 by the print information data management unit 504. The print information data table holds a print job ID, document data information and page number used at the time of printing, print layout information, and partial drawing object information included in the document data necessary for configuring the layout. The print job ID is an identifier assigned by the system for uniquely identifying the print job. The document data information is information on the original document data that has been submitted to the print job. The page number is a logical page number in the document data of the page to be printed. The partial drawing object information is partial drawing object information that is a component when the print layout information is applied. The print layout information is print layout information when document data is printed. In this embodiment, it is assumed that the PC layout illustrated in FIG. 8, the mobile terminal layout, the aggregate PC layout, and the consumed toner reduction layout illustrated in FIG. 11 exist.

<Partial drawing object table>
Next, the partial drawing object table will be described with reference to FIG. The partial drawing object table holds information on partial drawing objects included in document data necessary for configuring the layout in the print layout held in the print information data table. In the present invention, a raster image generated by a print job is not held as it is, but is divided into units of drawing objects constituting a print layout and held as a partial drawing object with partial raster images, vector data, and font information. Then, the drawing content at the time of printing is reconstructed by applying the print layout to the partial raster image, vector data, and font information which are these partial drawing objects at the time of image search. Even when different print layout information is used, if the partial drawing objects are the same as necessary to configure the layout, the partial drawing objects are reused to reconstruct the drawing contents. Therefore, even if the same partial drawing object is used between different layouts, it is not redundantly held in the holding area. Accordingly, the partial drawing object table is updated at the timing when the image forming apparatus 102 processes the print job, but the partial drawing object already held in the file server 101 is not added to the partial drawing object table. Only the partial drawing objects newly used in the print job are added.

  The partial drawing object table has information on drawing object IDs, drawing object attributes, and partial drawing object names. The drawing object ID is an identifier that can uniquely identify a partial drawing object allocated by the system. The drawing object attribute has attribute information indicating whether the target partial drawing object is a text sentence, a vector format graphic, or image data such as a photograph. In the present embodiment, a “Text” attribute is assigned to a text sentence, a “Graphic” attribute is assigned to a graphic, and an “Image” attribute is assigned to image data. The partial drawing object name is the name of the partial drawing object to be assigned by the system. In this embodiment, the name of the object is switched according to the retention format of the partial drawing object. A partial drawing object can take a partial raster image format, a vector data format, or a font format as a holding form. Image data such as photos is in a partial raster image format, and if the resolution of the image data itself is lower than the output resolution, the image data remains the same as the resolution of the image data. And hold as a partial raster image. Graphics are stored in vector data format. The text sentence may be held in either a font format or a partial raster image format. However, when the data size is to be reduced, it is desirable to have the text sentence in the font format. When a download font that is not built in the image forming apparatus 102 is used, it is provided in a partial raster image format.

<Retention form of partial drawing object>
FIG. 15 shows partial raster images, vector data, and font information, which are partial drawing objects constituting the layout, in the document data of the Web site exemplified in FIGS. FIG. 15A shows a partial raster image of a partial drawing object whose drawing object attribute is “Image” in the partial drawing object table of FIG. FIG. 15B shows vector data of a partial drawing object whose drawing object attribute is “Graphic” in the object table of FIG. Since it is vector data, the data size is smaller than that of the partial raster image. FIG. 15C shows the object table of FIG. 14 in which the drawing object attribute is “Text” and the holding format of the partial drawing object is the partial raster image format. Bitmap-01_01 and Bitmap-01_02 correspond to the content element 1002 which is the title information of the Web site defined by the HTML data structure shown in FIG. Although the same content element 1002 corresponds, since the applied print layout is different, the generated partial raster images are different. Since Bitmap-01_02 has applied the toner consumption reduction layout, the background is not drawn, and a partial raster image with only a frame and text is provided. FIG. 15D shows an example of a holding form in the object table of FIG. 14 where the drawing object attribute is “Text” and the holding form of the partial drawing object is a font form. In this embodiment, a character string and a font name to be used are designated and held as a font form holding form. FIG. 15D illustrates a case where an article document is held in a font format with content elements defined in the HTML data structure shown in FIG. FIG. 15D shows the “Text” attribute having the same drawing object attribute as FIG. 15C, but FIG. 15D uses the font built in the controller of the image forming apparatus 102, so the font format Is holding in. When reconstructing the drawing contents, the drawing contents can be reproduced by reading the font specified in the font format from the controller built-in font. On the other hand, FIG. 15C shows the title part of the Web site, and a special font is used, and therefore a font not included in the controller built-in font is used. Therefore, it is stored as a partial vector image and used when reconstructing the drawing content.

  When retrieving the original document data of the printed matter, the partial drawing objects shown in FIGS. 15A, 15B, 15C, and 15D are applied to the print layouts shown in FIG. Reconstruct the drawing contents of the time.

<Flow of print processing of image forming apparatus>
Next, a printing process in the image forming apparatus 102 will be described with reference to FIG. When printing the document data, the image forming apparatus 102 generates print information data for the original document data search process of the printed matter. The print information data is composed of print layout information and partial drawing objects necessary for configuring the layout. The partial drawing object can take the form of a partial raster image, vector data, font information, and the like. The generated print information data is transmitted to the file server 101 and stored in the print information data management unit. When a print job is input from the information processing apparatus (not shown) to the image forming apparatus 102 via the network 102 in FIG. 1, this processing is started.

  First, in step S <b> 1001, the central processing unit 201 receives print data from the information processing apparatus 104 via the network 103. In step S <b> 1002, the central processing unit 201 receives print settings from the information processing apparatus 104 via the network 103. The print setting includes designation of a print layout shown in FIG.

  Next, in S1003, the central processing unit 201 executes print processing. In step S <b> 1004, the central processing unit 201 generates print raster information at the time of printing and partial raster images, vector data, and font information as partial drawing objects necessary for configuring the layout. To summarize.

  In step S <b> 1005, the central processing unit 201 transmits the print information data generated in step S <b> 1004 to the file server 101. Through the above processing, the printing processing in the image forming apparatus 102 is performed.

<Flow of print information data storage processing on file server>
Next, print information data storage processing in the file server 101 will be described with reference to FIG. When the print information data is received from the image forming apparatus 102, the file server 101 updates the print information data table and the partial drawing object table shown in FIG. In order to avoid duplication, the print information data table and the partial drawing object table do not add information to each table when the contents are the same. The print layout information and the partial drawing object included in the received print information table are also held in the print information data management unit. However, if duplicate data already exists, it is not held. In S1005 in FIG. 16, when print information data is transmitted from the image forming apparatus 102 to the file server 101 via the network 102 in FIG. 1, this processing is started.

  First, in S2001, the central processing unit 301 receives print information data from the image forming apparatus 102 via the network 103. In step S <b> 2002, the central processing unit 301 refers to the print information data table held in the print information data table management unit 504.

  In step S2003, the central processing unit 301 determines whether the print information data received in step S2001 already exists in the print information data table. In the present invention, when the system is operated for a long period of time, the print information data holding area is efficiently used. Therefore, if there is overlapping data in the holding area, the print information data is not saved. Therefore, here, it is determined whether the received print information data is information that already exists in the holding area of the system, and it is determined whether the print information data should be stored in the system. If the print information data already exists in the print information data table, the process ends. If the print information data does not exist in the print data table, the process proceeds to S2004.

  Next, in S2004, the central processing unit 301 stores the print layout information and the partial drawing object included in the print information data in the auxiliary storage device 304. At the same time, the print information data table and the partial drawing object data table are updated.

  Through the above process, the print information data storage process in the file server 101 is performed.

<Flow of Original Document Data Retrieval Process of Printed Material of Image Forming Apparatus>
Next, an original document data search process for a printed matter in the image forming apparatus 102 will be described with reference to FIG. In this embodiment, in the original document data search process of the image forming apparatus 102, the image search process is not performed on the image forming apparatus 102 but is performed on the file server 101. The image forming apparatus 102 scans the printed matter and transmits the acquired scanned image to the file server 101. The image search processing of the original document data that matches the scanned image is executed in the file server 101, and the search result is transmitted to the image forming apparatus 102. The image forming apparatus 102 receives the search result transmitted from the file server 101 and displays it on the operation unit 210. When the user instructs to scan the printed matter using the scanning device 209 of the image forming apparatus 102 shown in FIG. 2, this processing is started.

  First, in step S <b> 3001, the central processing unit 201 uses the scanning device 209 to obtain a scanned image of a printed matter. In step S <b> 3002, the central processing unit 201 transmits the scanned image to the file server 101. After the scan image is transmitted, the process waits until a search result is received from the file server 101. When it is transmitted from the search result from the file server 101, this processing is resumed.

  In step S <b> 3003, the central processing unit 201 receives a search result transmitted from the file server 101. Next, in S3004, the central processing unit 201 displays the received search result on the operation unit 210.

  Through the above processing, the printed material scan processing in the image forming apparatus 102 is performed.

<Flow of original document data search processing on file server>
Next, the original document data search process in the file server 101 will be described with reference to FIG. In the original document data search process in the file server 101, first, a scan image transmitted from the image forming apparatus 102 is received, and a feature amount is extracted. A print layout set at the time of printing is determined from the extracted feature amount. Each feature amount of the print layout, such as the arrangement position of the drawing object, has a certain tendency. For this reason, the print layout from the scanned image is determined based on the similarity between the print layout feature quantity printable from the image forming apparatus 102 and the scan image feature quantity trend. When the print layout of the scanned image is determined, print information data having the corresponding print layout information is extracted from the print information data table shown in FIG. 13 and narrowed down. Then, by reading the corresponding partial drawing object from the partial drawing object information included in the print information data and applying the print layout to the partial drawing object, the drawing contents at the time of printing are reconstructed. A feature amount is calculated for the reconstructed result of the drawing content and compared with the feature amount of the scanned image. If they match, the corresponding original document data information is transmitted to the image forming apparatus 102 as an inspection result. If they do not match, return to the layout narrowing process again and perform a search again. If the original document data search process is repeated, and all the data in the print information data table is searched, if no match is found with the scan image feature amount, the search result is imaged as no corresponding data. To device 102. When a scan image is transmitted from the image forming apparatus 102 in S3002 of FIG. 18 from the print information data, this processing is started.

  In step S <b> 4001, the central processing unit 301 receives a scanned image from the image forming apparatus 102 via the network 103.

  In step S4002, the central processing unit 301 extracts feature amounts from the scanned image received in step S4001, and determines a print layout applied at the time of printing. When this system is operated for a long time, the amount of information held in the print information data table increases. If search processing is performed on all print information data held in the print information data table, the print information data increases in proportion to the operation period, and the search result calculation time also increases. Therefore, here, the tendency of the feature amount of the scanned image is extracted, and a print layout that can be printed by the image forming apparatus 102 is detected that has a high similarity in feature amount tendency. When searching, only the print information data having the same print layout information as the print layout having a high degree of similarity is set as a search target, so that the search process is made efficient.

  In step S <b> 4003, the central processing unit 301 refers to the print information data table managed by the print information data management unit 504.

  In step S <b> 4004, the central processing unit 301 determines whether print information data including the print layout information extracted in step S <b> 4002 exists in the reference print information data table. If the print information data exists, the corresponding print information data is read, and the process proceeds to S4005. If there is no print information data, the process returns to S4002 to repeat the process. When print information data exists, if there are a plurality of corresponding print information data, all the corresponding print information data is read. In this case, the processing from S4005 to S4008 is repeated by the number of print information data until the feature amount of the drawing content reconstructed in S4008 matches the feature amount of the scanned image.

  Next, in S4005, the central processing unit 301 determines whether there is print information data that is not a search target in the print information data including the print layout information extracted in S4002 read in S4004. If there is print information data that is not a search target, the process advances to step S4006. If there is no print information data that is not a search target, the process advances to step S4010.

  In step S4006, the central processing unit 301 reads a partial drawing object used in the print information data including the print layout information extracted in step S4002. In step S4007, the central processing unit 301 applies the print layout extracted in step S4002 to the partial drawing object read in step S4006 to reconstruct the drawing contents at the time of printing.

  In step S4008, the central processing unit 301 extracts the feature amount of the drawing content at the time of printing reconstructed in step S4009 and the feature amount of the scanned image, and determines whether or not they match. If they match, the process proceeds to S4009. If not, the process returns to S4005 and repeats the process.

  In step S <b> 4009, the central processing unit 301 transmits document data information included in the print information data to the image forming apparatus 102. If the feature amount of the drawing content reconstructed in S4008 matches the feature amount of the scanned image, it can be determined that the document data information included in the print information data is the original document data of the scanned output. Therefore, here, the search result is transmitted to the image forming apparatus 102 on the assumption that the document data information included in the print information data is the original document data of the printed matter.

  In step S4010, the central processing unit 301 determines whether search processing has been executed for all print information data tables included in the print information data table. If the search process has not been executed for all print information data tables, the process returns to S4002 to repeat the process. If the search process has been executed for all print information data tables, the process advances to step S4011.

  In S4011, since the original document data of the scanned image cannot be detected even if the search process is executed for all the print information data included in the print information data table, the central processing unit 301 has no corresponding data. As a result, the search result is transmitted to the image forming apparatus 102.

  With the above processing, the original document data search processing in the file server 101 is performed.

  As described above, according to the present invention described so far, even when the system is operated for a long period of time, it is possible to efficiently retrieve original document data from a scanned image of a printed matter without constraining the search information data holding area.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (6)

Generating means for generating print layout information;
Means for generating a partial drawing object by dividing the drawing content to be printed in units of drawing objects;
Holding means for holding the print layout information and the partial drawing object in association as print information data;
An image processing apparatus comprising: search means for searching for the print information data held by the holding means when searching for an image similar to the input image.   The image processing apparatus according to claim 1, wherein the input image is a scanned image obtained by reading a printed material.   The image processing apparatus according to claim 1, further comprising notification means for notifying a search result of the search means.   The image processing apparatus according to claim 1, wherein the partial drawing object selectively takes a form of a raster image, vector data, or font information as a holding form. A generation step for generating print layout information;
Generating a partial drawing object by dividing the drawing content to be printed in units of drawing objects;
A holding step of associating the print layout information with the partial drawing object and holding the information as print information data;
An image processing method comprising: a search step of searching the print information data held by the holding step when searching for an image similar to the input image.   The program for functioning a computer as an information processing apparatus as described in any one of Claims 1 thru | or 4.

Images