JP5420032B2 - Information processing apparatus, information processing method, information processing system, and computer program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, an information processing system, and a computer program.

  In recent years, copying machines have become multifunctional at a remarkable speed due to the digitization of internal image processing. For example, there are a copy function for copying an original, a PDL print function capable of printing a document created by a host computer, and a SEND function for transmitting a scanned original to the outside of the copier via a network. In addition, the BOX function (storage function) that enables the original image generated during copying or PDL printing to be saved on the hard disk inside the copier and reused, or the original image stored inside the copier by the BOX function is used. There is also an editing function for editing composition and bookbinding. Thus, it has an innumerable function.

  On the other hand, the demand for image quality is high, and the image processing resolution inside the copying machine has increased from 600 dpi to 1200 dpi and 2400 dpi because of high reproducibility of the original. Also, the number of bits of the signal has been increasing from 8 bits to 10 bits and 12 bits. In order to process bitmap data for internal processing with such a large amount of information, it is necessary to add memory and storage, replace with a high-performance CPU, etc., increasing the cost of equipment and development. The situation cannot be ignored.

  Under such circumstances, a technique is conceived in which a read original is stored in an object unit so that it can be easily reused. In Patent Document 1, a document element such as a photograph or chart in a document is registered separately, and the document element can be searched and displayed. Japanese Patent Application Laid-Open No. 2004-228561 describes that each part in a document is converted into vector data and a file having a file format suitable for each part is created.

  On the other hand, it is also considered to display a list of data such as images registered in the database in an easy-to-read manner. For example, in Patent Document 3, a reduced image of a small size and a large size is created for card data registered in a database, and the display is switched or a link is established between cards according to circumstances. It is described. Further, Patent Document 4 describes that the number of images to be displayed can be specified, and the image is enlarged and reduced to a size corresponding to the size of the display area regardless of the size of the original image.

Japanese Patent Laid-Open No. 06-162104 JP 2006-146486 A JP 05-257788 A JP 2000-040142 A

  However, the data corresponding to the input keyword is identified, the other data associated with the identified data is identified, the identified other data is displayed on the display means, and the other data includes Information relating to information for determining whether the other data is to be displayed only for a specific user or to be displayed for all users including the specific user. There was no disclosure.

Keywords based on that is input, a first specifying means for specifying data corresponding to the input keyword,
When the user instructs to associate the data specified by the first specifying means with other data not corresponding to the input keyword, the data specified by the first specifying means Association means for associating with the other data;
Based on that the keyword is the input is entered again, identify the other data in the state associated with the association of the association means prior Symbol data corresponding to the inputted again Keyword A second specifying means to:
Means for causing the display means to display the other data specified by the second specifying means;
Have
In the other data, in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. Information processing apparatus characterized in that the information is associated with each other.

  Specified by the first specifying means for specifying the data corresponding to the input keyword, the second specifying means for specifying other data associated with the specified data, and the second specifying means Means for displaying other data on the display means, and the other data includes all the data including the specific user so that the other data is displayed only for the specific user. It is possible to provide an information processing apparatus characterized in that information for determining whether to be displayed to the user is associated.

1 is a block diagram illustrating a configuration of an image processing system according to a first embodiment. 1 is a block diagram illustrating a detailed configuration of an MFP according to a first embodiment. Flow chart for region separation Scanned document image An image of a scanned document with objects separated Diagram for explaining vectorization processing Diagram for explaining vectorization processing Diagram showing an example of metadata configuration The figure which shows an example displayed on UI screen of MFP UI screen for automatic layout processing UI screen 1 showing object search results UI screen 2 showing object search results UI screen 1 showing object selection UI screen 2 showing object selection UI screen showing automatic layout results Example of metadata configuration that shows object relationships Group search result of Embodiment 1 Group search flowchart

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of an image processing system 10 according to the first embodiment of the present invention. The configuration of FIG. 1 is an example, and the present invention is not limited to this.

  This image processing system is connected to a LAN 107 in the office via a network 104 such as the Internet and a proxy server 103. In addition, a multifunction peripheral (MFP) 100 having a plurality of types of functions (copying function, printing function, transmission function, etc.), a client PC 102 that can communicate with the MFP 100, a database 105, a proxy server 103, and the like are connected to the LAN 107. Yes.

  In addition, various terminals of the client PC 102 are configured by standard components (for example, a CPU, a RAM, a ROM, a hard disk, an external storage device, a display, a keyboard, and a mouse) mounted on a general-purpose computer. .

  FIG. 2 is a block diagram showing a detailed configuration of the MFP 100 according to the first embodiment of the present invention. The image reading unit 201 is a scanner including an auto document feeder (ADF), reads a plurality of document images or a single document image, and obtains it as bitmap data with a predetermined resolution (600 dpi or the like).

  The printing unit 203 has a copying function for printing an image corresponding to the image reading signal on a recording medium. When copying one original image, the data processing unit 206 performs image processing on the image reading signal. A recording signal is generated and printed on a recording medium. On the other hand, when a plurality of copies are made from one original image, a recording signal for one page is temporarily stored in the storage unit 202, and is output to the printing unit 203 a plurality of times to be printed on a recording medium.

  The network I / F 205 is used when bitmap data obtained from the image reading unit 201 is converted into object data by processing described later and transmitted to the database 105. The network I / F 205 receives object data stored in the database 105 for reuse, converts it into an image file in a vector data file format such as XPS or PDF, and transmits it to the client PC 102. Also used for.

  Further, the data processing device 206 receives print data from the client PC 102 via the network I / F 205 and converts the received print data into a recording signal that can be printed by the printing unit 203. Then, the printing unit 203 forms an image on the print medium based on the recording signal.

  The input unit 204 is an interface for inputting an instruction from the operator. Based on the instruction input from the input unit 204, the control unit controls to execute various processes. These operations are controlled by a control unit (a CPU (not shown) or the like) in the data processing unit 206. Further, the display unit 207 displays a state display when a user's operation input is performed, and image data display during an image search or editing process.

  The storage unit 202 includes a hard disk that stores object data obtained by processing to be described later, an image processing buffer memory that is used when various types of image processing are executed in the data processing unit, and the like. There is also an image editing buffer memory used when image editing based on the object data is performed.

<Separation of object (area) of original image>
FIG. 3 is a flowchart showing the entire object separation of a document image.

  First, in step S301, a document is read by a scanner and subjected to image processing during scanning, whereby document image data (multi-valued input image data) is obtained. Note that image processing at the time of scanner refers to color processing and filter processing. In this embodiment, the document image data obtained by scanning the paper document is described as the input of the document image data in step S301. However, the present invention is not limited to this. For example, image data received from a PC via a network may be input as the original image data.

  In step S302, a histogram is generated for examining the color value distribution of the image. Based on the generated histogram, in the next step S303, a binarization threshold value used for binarization processing is determined. In step S304, binarization processing is executed using the binarization threshold determined in step S303, and a binary image is obtained.

  In step S305, a black pixel portion is detected in the binary image obtained by the binarization process in step S304. For example, the black pixel portion is detected by a known technique such as a black pixel contour tracking process or a labeling process.

  In step S306, the inside of the extracted black pixel region is analyzed, and whether or not an inverted character (a white character on a black background) is included in the black pixel region is analyzed. For example, the number of black pixels in the black pixel area is compared with the number of white pixels. If the number of black pixels is larger than the number of white pixels, whether the size of the connected white pixels is within a predetermined character size range. If it is within the range, it is determined that the reverse character is included.

  If it is determined in step S307 that the reverse character is included, the black pixel region portion including the reverse character is reversed in black and white to generate a final binarized image.

  The binarization process in steps S302 to S307 can be a known binarization process. For example, a binarization process is performed on the input image 401 in FIG. 4 to obtain a final binary image 402.

  In step S308, the final binarized image is divided into attribute-specific areas (image area, graphics area, text area, etc.). A known technique can be used for the region division processing for the binarized image. In this way, the position of the area (object) for each attribute included in the binary image is determined.

  In step S309, area information (object information) including area attribute information and position information is generated for each area (object) determined in the area dividing process of S308. Further, a document image ID (identifier) assigned to each input image, an input time of the image (for example, a scanning time), and the like are generated as additional information (index information) used when searching for an object and added to the object information. To do. Therefore, the same document image ID is assigned to the area (object) extracted from the same document image.

  In step S310, the attribute of each area stored in the area information is determined. After performing the processing of steps S311 to S314 according to the determined attribute, information about each area is stored in the BOX of the storage unit 202 in step S315. In the present embodiment, a storage area (storage or database) in which information such as a document image or area object is stored is called a BOX.

  For the area object determined as the image area in step S310, in step S314, the image area portion is extracted from the original document image data based on the position information of the area, and the multivalue of the extracted image area is obtained. Perform JPEG compression on the image. Thereafter, in step S315, the data of the JPEG compressed image area is stored in the BOX in association with area information (object information) related to the area.

  For the area object determined as the graphics area in step S310, in step S313, the graphics area portion is extracted from the binary image based on the information on the area, and the extracted graphics area is extracted. A vectorization process is performed on the binarized image. In the vectorization process, for example, vectorized (passed) data (vector data) is obtained by detecting the outline (outline) of a black pixel of the binary image and approximating the function with a straight line and a curve. At this time, the color distribution of the original document image data corresponding to the position of the object determined to be the graphics (the position of the black pixel in the graphics area on the binary image) is examined, and the graphics object The representative color is determined (that is, color information is given). For example, the representative color can be determined by taking an average color based on the color distribution. At this time, a plurality of colors may be selected as representative colors. In step S315, the vectorized (passed) data and the representative color information obtained regarding the graphics area are stored in the BOX in association with the area information (object information) regarding the area.

  For the object determined to be a text area in step S310, the process proceeds to step S311 to perform a character cutting process (character extraction process) for cutting out characters one by one from the text area. For example, the character cut processing is to cut out a character line by examining the distribution of the histogram in the horizontal direction and the vertical direction for the binary image of the character area, and further cut out the character based on the histogram in the direction perpendicular to the character line. Thus, a character image can be extracted in units of one character. Note that the character cut processing is not limited to this, and other known techniques may be used. Next, in step S312, OCR processing (character recognition processing) is performed on the character image cut out in step S311, and character code information of the OCR result is obtained. In step S313, vectorization processing is executed on the character image cut out from the binary image in step S311 to obtain vectorized (passed) data. In the vectorization process for the character image, for example, vector data can be obtained by executing the same process as the vectorization process for the graphics area described above. However, the vectorization process is not limited to this, and other methods may be used. For example, by identifying the font type of the character image and combining it with the OCR result obtained in S312, vector data composed of the identified font vector data and the character code of the OCR result can be generated. Further, at this time, the color distribution of the original document image data pixel corresponding to the position of the object determined as each character (the position of the black pixel in the character area on the binary image) is examined, and the graphics Determine the representative color of the object. For example, the representative color (color information of each character) can be determined by taking the average color of each character. In step S315, the character code information of the OCR result obtained in S312 and the vectorized (passed) data (vector data) obtained in S313 and the character color information are obtained from region information (object Information) and store it in the BOX. In addition, an index (keyword) extracted from the OCR result may be stored in the area information as metadata, or the entire character string of the OCR result may be used during full text search.

  In step S316, the separated objects are displayed separately for each area. For example, when the object extraction process of S308 to S314 is performed on the input image (original image) 401 and the binarized image 402 of FIG. 4, area information (object information) of each area and its drawing information ( Vector data or JPEG compressed data). Such information is stored in the BOX. In step S316, the object extraction result can be displayed in units of objects (character area, image area, graphics area, etc.) constituting the document image, as shown in FIG.

  Below, the detail of the process mentioned above is described.

<Binary processing (steps S302 to S307)>
First, the input image is binarized into black and white, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. For a black pixel block whose area is larger than the predetermined threshold value, the white pixel block is extracted by tracing the outline of the white pixel inside, and the white pixel block exceeding a certain area If there is a lump, a lump of black pixels is further extracted from the inside of the lump of white pixels. Further, the reversed character area is reversed in black and white to obtain a final binary image. Note that the binarization process is not limited to this, and a known technique can be used.

<Object Extraction Processing (S308 to S315)>
In the region separation processing (object separation processing) in S308, black pixel blocks are extracted from the final binary image, the extracted black pixel blocks are classified by size and shape, and are classified into regions having different attributes. I will do it. At this time, the black pixel block within the predetermined distance is determined to be the same region, and the regions are also integrated. For example, if an area having an aspect ratio close to 1 and having a constant size is a pixel block corresponding to a character image, and pixel blocks corresponding to adjacent character images are arranged in a well-aligned manner, they are grouped (integrated). Judged as a character area. Further, an area where irregular pixel clusters are scattered is defined as an image area such as a photograph. A pixel block having any other shape is used as the graphics area. In the present embodiment, classification is performed in this way, but the present invention is not limited to this, and a known region separation technique can be applied. Further, a table attribute (table attribute) may be further added as an area attribute. In this case, since the character attribute area can be extracted from the table attribute area, the character attribute portion is subjected to OCR processing / vectorization processing. Further, the table attribute area portion other than the character attribute area may be vectorized in the same manner as the graphics area to convert the ruled line into vector data. In this way, it is separated into areas (objects) for each attribute.

  An example of the OCR process (character recognition process) in S312 will be described. For example, character recognition is performed on a character image cut out from a text region in character units by using a pattern matching technique, and a corresponding character code is acquired. This character recognition process compares an observed feature vector obtained by converting a feature obtained from a character image into a numerical sequence of several tens of dimensions with a dictionary feature vector obtained in advance for each character type. Is the recognition result. There are various known methods for extracting a feature vector. For example, there is a method characterized by dividing a character into meshes and using a mesh number-dimensional vector obtained by counting character lines in each mesh block as line elements according to directions. When character recognition processing is performed on a text area, first, as described in S311, horizontal / vertical writing is determined for the text area, and a character string is cut out based on the determined direction. A character image is obtained by cutting out a character from the cut out character string. For example, horizontal / vertical writing is determined by taking a horizontal / vertical projection of the pixel value in each text area, and determining horizontal writing when the horizontal projection variance is large and vertical writing when the vertical projection variance is large. To do. In the case of a horizontally written text area, the character string and character are decomposed by using the horizontal projection to cut out the line, and then cutting out the character from the vertical projection of the cut out line. To do. On the other hand, for vertical text areas, the horizontal and vertical directions may be reversed. In addition, when performing this character recognition process, the size of a character can also be detected.

  The vectorization process in step S313 will be described.

  When the vectorization process is applied to the character area, first, a font recognition process is performed on each character of the character block obtained by the OCR process in step S313. In the font recognition processing, for example, a plurality of dictionary feature vectors corresponding to the number of character types used in the character recognition processing are prepared for each character shape type (that is, font type), and the font type is output together with the character code at the time of matching. Thus, the font of the character can be recognized. Then, using the character code and font information obtained by the character recognition process and the font recognition process, the outline data of the font prepared in advance for each font is used to convert the character part information into vector data. When the document image is a multi-value image (color image), the representative color of each character is determined based on the multi-value image and recorded together with vector data. Note that the character area vectorization processing may be performed in the same manner as the vectorization processing for a graphics area described later to generate vector data. For character images determined to match the character recognition results, vectorization processing using font outline data is performed, while characters that have been determined to have a high possibility of erroneous recognition. For an image, a vectorization process based on an image outline similar to a graphic area described later may be executed.

  When the vectorization process is applied to the graphics area, the contour of the black pixel block extracted in the graphics area is converted into vector data. Specifically, a point sequence of pixels forming an outline is divided by points regarded as corners, and each section is approximated by a partial straight line or curve. The corner is a point where the curvature is larger than a predetermined threshold, and the point where the curvature is large is as shown in FIG. 6 between the points Pi-k and Pi + k which are k left and right apart from the arbitrary point Pi. When a string is drawn, a point where the distance l between the string and PI is larger than a predetermined threshold is regarded as a point having a large curvature. Alternatively, {(string length) / (arc length)} between Pi−k and Pi + k may be R, and a point where the value of R is equal to or less than a threshold may be regarded as a corner. For each section after being divided by the corners, the section determined to be a straight line part is approximated using a calculation formula such as a least square method for the point sequence, and for the section determined to be a curved part, 3 is used. Vector data can be generated by approximation using a function such as a next spline function. Note that vector data can also be generated by performing the vectorization process on the white pixel block in the same manner for the white pixel block inside the black pixel block.

  As described above, by using contour lane marking approximation, vector outlines can be generated by vectorizing outlines of arbitrarily shaped figures. If the original image is a color image, the figure color is extracted from the color image and recorded together with the vector data.

  In the above-described example, the contour vector data of the graphic is described. However, for a linear graphic, the contour may not be used as it is, but may be expressed as vector data having line width information. For example, as shown in FIG. 7, when two contours are close to each other in a certain section of the figure, the two contour lines can be collectively expressed as one line vector data having line width information. it can. Specifically, when a line is drawn from each point Pi of a certain contour to a point Qi that is the shortest distance on another contour, and the distance PQi has a substantially constant length in a certain attention section, the attention section is The midpoint of PQi is approximated by a straight line or a curve as a point sequence, and the line width can be converted into vector data that is the average value of PQi.

<Metadata>
As shown in FIG. 8, the structure of metadata is hierarchical. For example, the basic information 801 describes the date and time when the object was created, the creator, the document ID, and the like. Each additional information includes information on the shape of each object, the position on the document, and the like. Further, a keyword created from the OCR result can be stored in the additional information. As described above, the metadata includes additional information 802-804 added to each of a plurality of objects extracted from one original image and basic information 801 added to the entire original image. Is held in association with each other.

  Further, as will be described later, in the present embodiment, an object can be searched and reused. Furthermore, a new document can be created by combining and laying out objects selected based on a user's instruction from the retrieved object group. At this time, the search keyword used when searching for each object used when creating the new document is also reflected in the metadata of other objects laid out together. With this configuration, when it becomes necessary to create another new document with a combination of similar objects again, it is possible to easily search for the combination, and operability when reusing it. Will improve.

  Further, information about security may be included in the metadata. In this case, the users that can be used for each object can be limited. Note that the hierarchical structure of metadata such as additional information 802 to 804 can change the level of security according to the hierarchy, and distinguishes information that can be seen by anyone or information that can only be seen by a specific user. This is because it is convenient to do.

  As for the metadata format, a description language such as XML is used. However, the metadata format is not limited to this, and it may be managed using a table or the like.

<Display UI screen by object>
In step S316, the result of the process of separating the input image into objects and storing it in the BOX is displayed on the UI screen as shown in FIG. FIG. 9 shows an example in which an object subjected to metadata generation processing, vector data generation processing, and BOX storage processing is displayed on the UI screen 901 of the MFP after the document of FIG.

  There are seven area objects extracted from the original in FIG. 4, and objects 903 to 909 are displayed as shown in the UI screen 901. Further, by selecting each area object, it is possible to refer to the contents of metadata associated with the area (not shown).

  Further, when the SEND button 902 is pressed by the user, the selected object can be transmitted to the PC. Further, when a plurality of objects are selected and automatic layout processing described later is executed, a document in which the selected objects are automatically laid out can be created.

  FIG. 10 shows an automatic layout processing UI screen.

  The objects displayed in the screen 901 in FIG. 10 are objects that are separated into regions by the above-described processing and stored in the BOX. In FIG. 10, various objects extracted from a plurality of documents and stored in a BOX are displayed. A check button (1001 to 1012) is prepared for each object, and this check button is a user interface used when the user selects each object. The object shown here is stored in advance in the BOX of the storage unit 202 described above.

  In addition, a search key input field 1013 for inputting a keyword is provided on the UI so that a desired object can be searched from a plurality of objects. When a keyword is input to the search key input field 1013 via the input unit 204, an object can be searched.

  The search is configured based on the information of the additional information (FIG. 8) described above. That is, the search is performed by determining whether or not the additional information 802 to 804 added to each object matches the keyword input in the search key input field 1013. Then, an object whose additional information matches in the search is displayed in the screen 901.

  When the automatic layout button 1014 is pressed by the user, the object selected by the check buttons 1001 to 1012 is automatically laid out to create a new document.

  Below, it demonstrates using a specific example. FIG. 11 shows a state in which the keyword “digital camera” is input to the key input field 1101 via the input unit 204, the normal search button 1016 is pressed, and the search process is executed. A plurality of objects 1102 to 1107 displayed in the screen 901 of FIG. 11 represent examples of results obtained by keyword search.

  On the other hand, FIG. 12 shows an example of the result of executing the object search process when the keyword “natural” is input to the key input field 1201. Here, an example is shown in which six objects 1202 to 1207 are displayed as a result of keyword search.

  In addition, the screen display of FIG. 11 and FIG. 12 can be switched by inputting a search keyword. Alternatively, each search result may be temporarily stored, and the search result may be switched and displayed by operating a search result screen switching button (not shown).

  Next, the object selection operation will be described with reference to FIGS.

  FIG. 13 shows a state where an object is selected using check buttons 1301 to 1306 on the screen 901 showing the result of the search using the keyword “digital camera” in FIG. In the example of FIG. 13, four objects (1302, 1304, 1305, 1306) are selected.

  FIG. 14 shows a state in which several objects are selected as a result of searching with the keyword “nature” in FIG. 12. In FIG. 14, three objects (1401, 1403, and 1404) are selected.

  As described with reference to FIGS. 13 and 14, a total of seven objects are selected in this embodiment. In this embodiment, creating a new document by combining these objects and laying out is called “re-layout” processing. In other words, the re-layout means that a new document is generated using an object selected by a user operation from among a plurality of objects searched by one or more keywords. Regarding this “re-layout” process, the automatic placement of selected objects and the generation of a new document will be referred to as “automatic layout (automatic layout)”. On the other hand, when a new document is generated by arranging the selected object based on an instruction of the arrangement position by the user, it is referred to as “manual layout (manual re-layout)”.

  When the automatic layout button (AUTO LAYOUT button) 1014 shown in FIGS. 13 and 14 is pressed by the user, the object selected at that time is automatically laid out and a new document is generated. The result of executing this “automatic layout” process is shown in FIG. A new document generated by the automatic layout process is displayed on the preview screen 1501. This new document shows the result of layout combining the objects (1302, 1304, 1305, 1306, 1401, 1403, and 1404) selected in FIG. 13 and FIG. At this time, when the edit button (EDIT button) 1502 is pressed by the user, the position and size of the object in the document displayed on the preview screen can be corrected according to the user's instruction. Further, when the save button (SAVE button) 1503 is pressed by the user, the document of the layout displayed at that time is stored as a new document. Then, information relating to the combination of objects used at the time of the layout of the new document (keyword used when searching for each object, group ID, etc.) is additionally stored as additional information in FIG. 8 as group information. By storing this group information, it is possible to use it as a reference candidate for an object used for the combination when it is desired to create a document with a similar combination later.

  FIG. 16 is an example of additional information added to each object when a new document is created. In FIG. 16, 1601 and 1606 are examples of objects. Reference numerals 1602 and 1607 denote basic information added when each object is extracted and created from the original document. Reference numerals 1603 and 1608 denote keywords added to each object based on the OCR result and key input by the user when each object is extracted and created from the original document. Reference numerals 1603 to 1605 and 1609 to 1611 denote information added when a new document is created by reusing each object. The group ID, the keyword used when creating a new document, and the reused information for each group. Etc. are added. The group ID is a number that is automatically assigned every time a new document is created and saved. Since the objects 1601 and 1606 are used at the same time when the document with the group ID = 1 is created, additional information 1604 and 1609 are added to the objects 1601 and 1606, respectively. Further, since the objects 1601 and 1606 are used when the document with the group ID = 2 is created, additional information 1605 and 1610 are added. Further, when the document with the group ID = 3 is created, the object 1601 is not used, and the object 1606 is used. Therefore, additional information 1611 is added to the object 1606.

  That is, the group used at the time of document creation is described in the additional information section of each object, and the association is performed. Furthermore, since the keywords used at the time of document creation are also stored together, the keywords used for searching for another object used together with the object are also associated. As a result, for example, the keyword “natural” used when creating the group is added as a keyword for the group to an object (for example, 1601) that is not originally associated with the keyword “natural”. Therefore, the object 1601 is not searched for by the keyword “natural” only with the originally added keyword, but can be searched by the keyword “natural” when performing a group search. Therefore, since each object can be searched based on the relevance of the group, the reusability of the object is improved.

  FIG. 17 shows a group search result when the keyword “digital camera” is input in the search key input field 1701 (1013 in FIG. 10) and the group search button 1702 (1017 in FIG. 10) is pressed by the user. It is an example of the screen to perform.

  When the normal search button 1016 is pressed, the search is performed based on the keywords (for example, 1603 and 1608 in FIG. 16) originally added to each object, so that the search result as shown in FIG. Become. On the other hand, when the group search button 1702 is pressed, a search is performed using a group ID and a group keyword (for example, 1604 to 1605 and 1609 to 1611 in FIG. 16) together with the original keyword, as shown in FIG. The search result is as follows. That is, in FIG. 17, the objects used in the combination when the original is created by the past relayout are also displayed together.

  In FIG. 17, the vertical column 1703 displays the objects arranged in the vertical direction in descending order of the search matching degree as a result of the search based on the keyword originally added to each object. Then, another object included in each group when each object in the column 1703 has been reused is displayed in the related object display column 1704 adjacent to the right side thereof. For example, the object 1601 in FIG. 16 is searched, and other objects belonging to the group ID = 1 related to the object 1601 are displayed in the first line of the related object display column in FIG. Further, other objects belonging to the group ID = 2 related to the object 1601 are displayed in the second line of the related object display column in FIG. In this way, each group is displayed by group. Also, a check box for selecting each object is provided for each displayed object. Further, in the group information column 1705 on the right side of the related object display column 1704, a check box for selecting collectively by group, a group ID, and keywords other than 1701 used at the time of the previous search are displayed. .

  When a user clicks on a keyword displayed in the group information column 1705 (that is, a keyword used to search for a grouped object), a normal search is performed using the clicked keyword as a key. And That is, the user can easily search using the keyword used when searching for the object 1704 and the object 1704 used together in the past. Since the same keyword is often added to similar objects, another object similar to the object used together in the past can be easily searched, and the object can be easily used.

  In addition, each group related to the searched object 1703 displays a group having a high use count as a group at the top. For groups with the same number of uses, a group with a large number of related objects to which the same keyword as the search keyword 1701 is originally added is displayed at the top. In FIG. 17, the same color is displayed as the background color (indicated by hatching in FIG. 17) of the related object to which the same keyword as the search keyword 1701 is originally added. That is, the background color of the object 1703 searched by the search keyword and the background color of the object to which the same keyword is originally added among the objects included in the related group are the same. In FIG. 17, the background color of the area in which the object to which the same keyword is originally added is displayed is the same, but is not limited to the color. For example, it may be displayed so as to be identifiable by using a background pattern or symbol.

  FIG. 18 is a flowchart of group search processing that is performed when a search keyword is input and the group search button is pressed.

  In step S1801, the objects are searched based on the input search keyword and the keyword added to each object, and are sorted in descending order of the matching degree of the search results. First, the process proceeds to step S1802 with the first object as a related group search target.

  In step S1802, information related to the related group (group ID, number of times used, etc.) is searched based on the additional information of the target object. In step S1803, another object belonging to each related group is searched based on the group ID.

  In step S1804, groups are sorted based on information on related groups and information on other objects belonging to each related group. Here, based on the number of times the related group is used and the keywords added to other objects belonging to each related group, the groups that are likely to be used are sorted so as to be ranked higher.

  In step S1805, if there is a next object in the search result in step S1801, that object is set as a related group search target, and the process returns to step S1802.

  In step S1806, the search result object based on the search keyword, the group information related to the object, and the other objects belonging to each group are displayed in the sort order. As a result, a display as shown in FIG. 17 is made.

  In the result display screen of FIG. 17, the user can select a desired object by checking a check box of each object or a check box of each group. When the automatic layout button is pressed, a new document can be created using the selected desired object.

  As described above, according to the present embodiment, the group based on the past use history as shown in FIG. 17 is stored by associating and storing the group information at the time of reuse together with the keyword for each object. Search can be performed. Since combinations that have been used in the past are often reused in the same or similar combinations afterwards, the display as shown in FIG. It becomes easy to use.

(Embodiment 2)
The second embodiment is characterized in that the additional object information (for example, 1604 to 1605, 1609 to 1611) also stores the shape of the object processed at the time of reuse in the group. The shape to be processed is, for example, enlargement / reduction, deformation, or rotation.

  And when displaying a group search result like FIG. 17, each object shall be displayed based on the shape processed at the time of the past reuse. Each object may be processed in a shape (for example, enlargement / reduction, deformation, rotation, etc.) each time it is reused, but is displayed with the most processed shape.

  Thus, the user can know the object shape used in the past, and can use it as a reference for the layout when generating a new document in the re-layout process.

  In addition, since the shape that is reused most often is considered to be a recommended shape at the time of reuse, reusability is also improved.

(Other embodiments)
In Embodiments 1 and 2 described above, control such as search and display is performed in the control unit of the MFP (multifunction machine). The control unit of the MFP includes a CPU (computer), and performs the above-described search and display processes by reading and executing a computer program stored in a computer-readable storage medium (for example, a hard disk). . That is, the computer functions as control means for performing each process by executing the program.

  Note that the present invention is not limited to the MFP, and can also be achieved, for example, when the computer connected to the MFP or a computer of a general-purpose personal computer realizes the above-described processing by executing a program. .

  The present invention is realized by a computer executing a computer program. However, part or all of the processing may be realized by hardware such as an electric circuit. In that case, each hardware functions as a control means for performing each process.

  As described above, according to the present specification, by storing information on object groups used together in the past as a group, it is possible to search for a group related to the object and improve the reusability of the object. To do.

  This makes it easy to search for a desired object from among the stored objects, and it becomes even easier to create a new document by reusing the object.

  In addition, there is a high possibility that a useful object that could not be found by a simple keyword search can be easily found.

Claims (20)

Keywords based on that is input, a first specifying means for specifying data corresponding to the input keyword,
When the user instructs to associate the data specified by the first specifying means with other data not corresponding to the input keyword, the data specified by the first specifying means Association means for associating with the other data;
Based on that the keyword is the input is entered again, identify the other data in the state associated with the association of the association means prior Symbol data corresponding to the inputted again Keyword A second specifying means to:
Means for causing the display means to display the other data specified by the second specifying means;
Have
In the other data, in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. Information processing apparatus characterized in that the information is associated with each other. Keywords based on that is input, a first specifying means for specifying data corresponding to the input keyword,
When an association between the data specified by the first specifying unit and other data corresponding to a keyword different from the keyword is instructed by the user, the data specified by the first specifying unit and the data An association means for associating with other data;
Based on that the keyword is the input is entered again, identify the other data in the state associated with the association of the association means prior Symbol data corresponding to the inputted again Keyword A second specifying means to:
Means for causing the display means to display the other data specified by the second specifying means;
Have
In the other data , in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. the information processing apparatus characterized by information of Ru attached communication function. The first specifying means includes:
Means for identifying a plurality of data corresponding to the input keyword;
Means for specifying the selected data as the data corresponding to the inputted keyword by selecting one data from the plurality of specified data based on a user instruction;
The information processing apparatus according to claim 1, further comprising: It said information to any one of claims 1 to 3, characterized in that any data of said other data is information for determining whether to be displayed only for a specific user The information processing apparatus described in 1. It said information claims 1 to 4, characterized in that any data of said other data is information for determining whether to be displayed to all users, including the particular user The information processing apparatus according to any one of the above. The information is such that a part of the other data is displayed only to a specific user, and the other part of the other data includes the specific user. the information processing apparatus according to any one of claims 1 to 3, characterized in that the information to be displayed to the user of the. The display means, the first has been the data identified by the identifying means, the information processing apparatus according to any one of claims 1 to 6, characterized in that to display along with the other data. Wherein the data specified by the first specifying means, and the other data specified by the second specifying means is an image data,
Said first specifying means, in addition to the image data corresponding to the entered the keywords to identify the other image data corresponding to the entered the keywords,
The display means, according to claim 1 which has been the picture image data and the other image data specified by the first specifying means, and wherein the displaying the image data specified by the second specifying means to 7 the information processing apparatus according to any one of. Receiving means for inputting a keyword from a user, and receiving means for inputting the keyword;
Based on said keyword entered by the receiving means, a first specifying means for specifying data corresponding to the input keyword,
When the user instructs to associate the data specified by the first specifying means with other data not corresponding to the input keyword, the data specified by the first specifying means Association means for associating with the other data;
Based on said reception the keywords entered by the means is inputted again by the receiving means, the state associated with the association of the association means prior Symbol data corresponding to the inputted again Keyword A second specifying means for specifying the other data;
Display means for displaying the other data specified by the second specifying means;
Have
In the other data, in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. An information processing system characterized by being associated with the information. Receiving means for inputting a keyword from a user, and receiving means for inputting the keyword;
Based on said keyword entered by the receiving means, a first specifying means for specifying data corresponding to the input keyword,
When an association between the data specified by the first specifying unit and other data corresponding to a keyword different from the keyword is instructed by the user, the data specified by the first specifying unit and the data An association means for associating with other data;
Based on said reception the keywords entered by the means is inputted again by the receiving means, the state associated with the association of the association means prior Symbol data corresponding to the inputted again Keyword A second specifying means for specifying the other data;
Display means for displaying the other data specified by the second specifying means;
Have
In the other data , in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. the information processing system characterized in that information of Ru attached communication function. The information processing system according to claim 9 or 10, wherein the display means is a display. The first specifying means includes:
Means for identifying a plurality of data corresponding to the input keyword;
Means for specifying the selected data as the data corresponding to the inputted keyword by selecting one data from the plurality of specified data based on a user instruction;
The information processing system according to claim 9, wherein the information processing system includes: It said information to any one of claims 9 to 12, characterized in that any data of said other data is information for determining whether to be displayed only for a specific user Information processing system described in 1. It said information claims 9 to 13, characterized in that any data of said other data is information for determining whether to be displayed to all users, including the particular user The information processing system according to any one of the above. The information is such that a part of the other data is displayed only to a specific user, and the other part of the other data includes the specific user. The information processing system according to any one of claims 9 to 12, wherein the information processing system is information for display to a user. The display means, the first information processing system according to any one of claims 9 to 15 has been the data identified by the identifying means, characterized in that to display along with the other data. Wherein the data specified by the first specifying means, and the other data specified by the second specifying means is an image data,
Said first specifying means, in addition to the image data corresponding to the entered the keywords to identify the other image data corresponding to the entered the keywords,
The display means, wherein, characterized in that to display the been the picture image data and the other image data specified by the first specifying means, along with other image data specified by the second specifying means Item 17. The information processing system according to any one of Items 9 to 16 . Keywords based on that is input, a first specifying step of specifying the data corresponding to the input keyword,
When the user instructs to associate the data specified in the first specifying step with other data not corresponding to the input keyword, the data specified in the first specifying step and Associating with the other data;
Based on that the keyword is the input is entered again, identify the other data in the state associated with the association of the associated process before Symbol data corresponding to the inputted again Keyword A second specific step to
Displaying the other data specified in the second specifying step on a display means;
Have
In the other data, in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. An information processing method characterized in that the information is associated with each other. Keywords based on that is input, a first specifying step of specifying the data corresponding to the input keyword,
When an association between the data specified in the first specifying step and other data corresponding to a keyword different from the keyword is instructed by the user, the data specified in the first specifying step and the data An association step for associating with other data;
Based on that the keyword is the input is entered again, identify the other data in the state associated with the association of the associated process before Symbol data corresponding to the inputted again Keyword A second specific step to
Displaying the other data specified in the second specifying step on a display means;
Have
In the other data , in order to determine whether the other data is to be displayed only to a specific user or to be displayed to all users including the specific user. the information processing method characterized in that information of Ru attached communication function.   The computer program for functioning a computer as each means of any one of Claims 1 thru | or 8.

Images