JP2007065763A - Image data retrieval device, image data retrieval method, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document data retrieval system using attribute information found in grid units so as to retrieve document data on the basis of ambiguous layout information stored by a user. <P>SOLUTION: A data registration part 1 performs area separation processing to inputted image data, finds the attribute information in each grid, associates the image data and the attribute information, and stores them into a data storage part 2. A condition input part 3 inputs a layout condition in the grid unit, and a retrieval part 4 retrieves the image data from an image data group by use of the layout condition, and displays a retrieval result on a display part 5 or transmits it to an external display via the Internet or the like. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image data search apparatus and an image data search method, and more particularly to a technique for searching and browsing desired data from previously filed document data and image data groups.

  There is a filing technology that optically reads an enormous amount of paper documents in an office using a scanner or the like, converts it into digital data, and stores it. In order to search for arbitrary data from a large amount of document data that has been filed, there is a method such as searching by character information obtained from character recognition using OCR, but the characters of the optically read document data Recognition is not always accurate. Moreover, it is not easy to determine a keyword for efficiently performing a search from a large amount of document data.

In the above case, it is considered that the search based on the layout information is effective. In Japanese Patent Laid-Open No. 9-237282 (Patent Document 1), document data is searched from the similarity of document layout features by using an exemplary document image input or created by a user. However, it is often thought that there are cases where the user does not have a document image that can be modeled when searching for document data, or there is only an ambiguous memory for the document data that the user desires. It is done.
JP-A-9-237282

  The present invention provides a document data search system using attribute information obtained in units of grids in order to enable searching for document data based on ambiguous layout information stored by a user. Here, attribute information means area information itself such as character area, photo area, halftone dot area, background area, etc. estimated by area separation processing, or feature information indicating the characteristics of the area obtained from the area information. To do.

  According to a first technical means of the present invention, in an image data search device for searching for arbitrary image data from an electronically filed image data group, a condition input unit for inputting layout conditions in units of grids, And a search unit for searching for image data from an image data group and an output unit for outputting a search result.

  According to a second technical means, in the first technical means, the condition input unit takes a unit of a grid area that can be specified to be larger than a pixel unit of an original image.

  According to a third technical means, in the first technical means, the condition input unit has means for designating a shape of a layout condition when inputting the condition.

  According to a fourth technical means, in the first technical means, the condition input unit has means for designating a direction of a layout condition when inputting the condition.

  According to a fifth technical means, in the first technical means, the search unit determines whether the attribute information filed at the time of the search or the designated layout condition is 0 degree, 90 degrees, 180 degrees, or 270 degrees, respectively. It has a means for performing a search by rotating at a rotation angle and comparing with the other.

  According to a sixth technical means, in the fifth technical means, when the output unit presents the image data corresponding to the attribute information meeting the condition as a candidate to the user, the output unit can be matched with the layout designated by the user. And a means for rotating and outputting image data.

  According to a seventh technical means, in the first technical means, the condition input unit includes means capable of changing a granularity of the grid.

  According to an eighth technical means, in the seventh technical means, the condition input unit causes the condition input screen after the granularity change to be reflected so as to inherit any condition already specified before the granularity change. It is characterized by having.

  A ninth technical means is characterized in that, in the first technical means, the condition input unit has means capable of designating conditions such as designating all or a plurality of attributes for the grid when designating layout conditions. It is a thing.

  A tenth technical means is characterized in that, in the first technical means, the condition input unit has means for specifying a condition that specifies a condition not to be searched for a grid when a layout condition is specified. It is a thing.

  The eleventh technical means is characterized in that, in the tenth technical means, the condition input unit designates in advance the conditions of the initial state of all grids when the layout conditions are designated.

  A twelfth technical means uses a condition input unit for inputting a plurality of layout conditions in units of grids and a layout condition in an image data search apparatus for searching for arbitrary image data from an electronically filed image data group. The image processing apparatus includes a search unit that searches image data from an image data group and an output unit that outputs a search result using a document including pages that meet a plurality of designated layout conditions as candidates.

  A thirteenth technical means is a retrieval method for retrieving arbitrary image data from an electronically filed image data group, a step of inputting a layout condition in units of grids, and an image from the image data group using the layout condition. It is characterized by comprising a step of searching for data and a step of outputting a search result.

  A fourteenth technical means is the same as the thirteenth technical means, wherein the grid granularity can be changed.

  According to a fifteenth technical means, in an image data search method for searching for arbitrary image data from an electronically filed image data group, a step of inputting a plurality of layout conditions in units of grids, and image data using the layout conditions The method includes a step of retrieving image data from the group and a step of outputting a retrieval result with candidates including documents including pages that meet a plurality of designated layout conditions.

  A sixteenth technical means is a control program for an image search apparatus for operating any one of the first to twelfth technical means, and is for causing a computer to function as each of the above means.

  The seventeenth technical means is a computer-readable recording medium on which the sixteenth technical means is recorded.

  According to the first technical means, the layout condition to be searched is specified in units of grids, so that the user can specify the layout envisioned in his head in an ambiguous manner.

  According to the second technical means, the unit of the grid area that can be specified is set to be larger than the pixel unit of the original image. And operability can be improved. When inputting from a display device such as a liquid crystal panel other than the touch panel and an input device such as a mouse or a keyboard, visibility and operability can be improved by setting the grid size to be sufficiently identifiable by the display device.

  According to the third technical means, since the shape of the layout condition is designated at the time of inputting the condition, it is possible to switch the image data shape such as letter or A size on the condition input screen, thereby improving user operability. be able to.

  According to the fourth technical means, since the direction of the layout condition is designated at the time of inputting the condition, the operability for the user can be improved by making it possible to switch the image data direction such as portrait and landscape.

  According to the fifth technical means, either the attribute information filed at the time of search or the designated layout condition is rotated at rotation angles of 0 degrees, 90 degrees, 180 degrees, and 270 degrees and compared with the other. As a result, operability and search efficiency can be improved.

  According to the sixth technical means, when the image data corresponding to the attribute information meeting the conditions is output to the user as a candidate, the image data is rotated and output so as to be consistent with the layout designated by the user. As a result, operability and search efficiency can be improved.

  According to the seventh technical means, since the condition input unit can change the granularity of the grid, more detailed condition input by the user is possible.

  According to the 8th technical means, the condition input part is reflected in the condition input screen after the granularity change so that any condition already specified before the granularity change is inherited. Can be achieved.

  According to the ninth technical means, since the condition input unit can specify conditions such as specifying all or a plurality of attributes for the grid when specifying layout conditions, operability can be improved.

  According to the tenth technical means, the condition input unit can specify a condition for specifying a condition not to be searched for the grid when specifying the layout condition, so that a part of the grid can be specified not to be searched. By doing so, the user can perform a search by inputting only the minimum necessary grid.

  According to the eleventh technical means, the condition input unit can preliminarily specify the initial state conditions of all the grids when designating the layout conditions, so that it is possible to improve operability and search efficiency.

  According to the twelfth technical means, since the condition input unit can input a plurality of layout conditions in units of grids, when a plurality of continuous image data groups at the time of filing are associated as one document, the layout of one page If it is not possible to narrow down only with the conditions, a more detailed search can be performed by specifying layout conditions for a plurality of pages.

  According to the thirteenth technical means, the method includes a step of inputting layout conditions in units of grids. By designating layout conditions to be searched in units of grids, the user can specify the layout to be imagined in his / her head. Is possible.

  According to the fourteenth technical means, since the granularity of the grid can be changed, more detailed condition input by the user is possible.

  According to the fifteenth technical means, since there is a step of inputting a plurality of layout conditions in units of grids, when image data groups of a plurality of pages continuous at the time of filing are associated as one document, only the layout conditions for one page If it is impossible to narrow down, a more detailed search can be performed by specifying a layout condition of a plurality of pages.

  FIG. 1 is a diagram showing a system configuration example as an embodiment of an image data search apparatus 10 (or method) according to the present invention. In the figure, 1 indicates a data registration unit, and document data is a scanner or the like at the time of filing. Is optically read from the image or rendered as raster data, and is input to the data registration unit 1 as image data for each page. The data registration unit 1 performs region separation processing on the image data, obtains attribute information for each grid, associates the image data with the attribute information, and stores them in the data storage unit 2. When searching, the user designates attribute information in units of grids, and inputs to the condition input unit 3 as layout conditions, whereby the search for image data is started. The search unit 4 compares the layout condition sent from the condition input unit 3 and the attribute information associated with each image data sequentially referred to from the data storage unit 2 in a grid unit, thereby laying out the layout for each image data. Find the similarity of conditions. If the image data stored in the data storage unit 2 includes image data having a similarity equal to or higher than a predetermined threshold, the search unit 4 sends the corresponding image data to the display unit 5. The display unit 5 displays an image of the image data on the display device.

  The image data is assumed to be raster data (bitmap data) that is read by a scanner or expanded for printing. For example, the object data is stored in an application program file such as Word (registered trademark). It is also possible to extract the position or the like (using the application program or its viewer) and create attribute information (layout information) based on the result.

  In addition, the image displayed here may be a thumbnail image of image data. When a thumbnail image is to be displayed, a thumbnail image is created in advance separately from the document data and attribute information at the time of filing and stored in the data storage unit in association with the thumbnail image. May be.

  FIG. 1 is a diagram of the present invention configured assuming a stand-alone, but it is also possible to perform a search on a server as shown in FIG. In the stand-alone mode, the input is a keyboard or pointing device, and the output is a display. However, in the server, the input is a means for receiving search conditions from another PC or the like. In addition, the output is a display unit in the stand-alone mode, but the server serves as a transmission means (output) of search results to other PCs and the like, and is displayed on a display device (display) on the server by the result output unit.

<Create attribute information>
The attribute information is created from the result of determining what kind of attribute a certain unit area has by area separation processing or the like. The result of the region separation process itself may be used, or a result created separately from the result of the region separation process may be used. Having one attribute for each required minimum grid size can reduce the amount of memory for storing attribute information, but it may be stored as information for each pixel. When obtaining attribute information of a grid composed of a plurality of pixels, there is a method such as calculating the sum of the number of pixels having each attribute for each attribute included in the grid and setting the attribute having the largest value as attribute information. is there. Further, the attribute having the largest value may be obtained as the attribute information of the grid from the result of multiplying the sum of the number of pixels for each attribute by the weighting factor for each attribute. For example, since it is assumed that the number of pixels that should be determined as a character area is larger than the number of pixels that are actually determined as characters, the sum of character attributes has a larger weighting factor than other attributes. It may be possible to apply Also, as area information for one grid, a plurality of attributes such as “character or background” may be provided, and in order to provide more flexibility, it is provided as a parameter indicating the appearance rate of each attribute. Also good.

<Input layout conditions>
When a layout condition is input, a layout condition creation frame 11 as shown in FIG. 3 is displayed on the display device 5 such as a liquid crystal panel. The user inputs search layout conditions by designating attribute information in units of grids. For example, according to the example of FIG. 3, there are buttons 12 to 14 indicating three pieces of attribute information of background, text, and figure, and the user selects one of the buttons and then selects the grid area. Attribute information is applied to the specified grid area. The layout condition specified at the time when the search button 15 is pressed is sent to the search unit 4 as a search condition.

<Search process>
4 is a flowchart for explaining an example of a search processing procedure according to the present invention. In FIG. 4, FIG. 4 (A) is a flowchart of the search processing, and FIG. 4 (B) is a flowchart of FIG. 4 (A). The flow of the similarity calculation process S5 which is a part is shown.

  The image data registered in advance in the data registration unit 1 is previously assigned an ID at the time of registration, and is stored in the data storage unit 2 in association with the corresponding attribute information. When the grid granularity in the input condition of the condition input unit 3 and the grid granularity of the attribute information stored in the data storage unit 2 are different, the grid granularity in the input condition is set as the grid granularity of the attribute information in S2 (input condition granularity adjustment). Match. When matching the granularity, an enlargement / reduction algorithm such as the nearest neighbor method may be used. This is because the granularity of the attribute information registered in advance and the grid granularity input as the layout condition are different from each other, so that the granularity of both is the same. May be.

  When searching for one that matches or is similar to the input layout condition from the registered image data group, attribute information corresponding to the image data is acquired from the data storage unit 2 in S4 (attribute information acquisition). Next, in S5 (similarity calculation), the degree of similarity between the condition and the attribute information is calculated. The similarity obtained in S5 is compared with a preset threshold value in S6, and if it is equal to or greater than the threshold value, the image data ID and similarity are stored assuming that the image data corresponding to the attribute information is a candidate (S7). . If the degree of similarity is less than the threshold, it is determined that the candidate is not a candidate, and the process for the current attribute information is terminated without storing anything. The above processing is sequentially performed on the registered image data group (S8).

  The calculation of the similarity (S5) is obtained by performing a comparison process between the layout condition and the attribute information in units of grids at the time of condition input. When the similarity calculation process is called in S5, the process proceeds to S11 shown in FIG. 4B and the similarity calculation process is started. At this time, 0 is stored in the similar grid and the effective grid as initial values. Here, the similar grid indicates the number of grids determined to be similar among all the grids, and the effective grid indicates the number of all grids to be compared.

  When comparing the condition input for the grid of interest with the attribute value of the corresponding grid in the attribute information, the value of the effective grid is incremented by 1 in S13. Next, the attribute value and the condition are compared in S14, and if they match (S14), the value of the similar grid is incremented by 1 in S15. The above processing is sequentially performed for all the grids (S16), and when the processing for all the grids is completed, the similarity to the image data is obtained by dividing the value of the similar grid by the value of the effective grid in S17. When the similarity calculation process ends, the similarity value is set as the output result of S5 (similarity calculation in FIG. 4A).

  When the direction of the search target image is landscape (horizontal) and the condition input image is portrait (vertical) (FIG. 5A), the user rotates the layout imaged at the head by 90 degrees. It is necessary to input as conditions, and operability is poor. Further, when the shape of image data such as letter or A size is different between the search target image and the layout condition creation frame (FIG. 5B), the operability may be deteriorated. The operability can be improved by making it possible to switch the direction and shape of the layout condition creation frame displayed on the display device before inputting the conditions as shown in FIG. For example, in the case of FIG. 5A, the direction of the layout creation frame can be switched by the two buttons 16 and 17 as the image direction. Similarly, the shape of the layout condition creation frame can be switched by the user pressing two buttons 18 and 19 of A size and letter as the image shape.

  In many cases, the direction of a document is uniquely determined. However, it may be rarely recognized that a user is recognized in a different direction. Also, there may be a case where registration is not performed in the correct direction during filing. When the search unit 4 in FIG. 1 compares the layout conditions and the registered attribute information, if the directions are different, a good search result cannot be obtained.

  FIG. 6 is a diagram for explaining the processing flow for solving the problem, that is, the problem in the case where the directions are different when comparing the layout condition and the attribute information. When the search process is started (S21). First, four types of conditions are created by rotating the layout conditions given from the condition input unit 3 every 90 degrees (S22). When calculating the similarity for each attribute information corresponding to the image data, the similarity is calculated for each of the four types of conditions created above (S25), and the highest similarity is equal to or greater than a predetermined threshold value. If it is equal to or greater than the threshold value (S26), the rotation angle information corresponding to the highest similarity is stored (S27), and the comparison processing is similarly performed on the next image data. At this time, if the maximum similarity is less than the threshold value, the next image data is directly subjected to comparison processing as it is not candidate data. Although the registered attribute information may be rotated, generally, the amount of processing and the processing time or the required amount of memory are reduced by storing the rotated layout conditions.

  In order to prevent the image of the image data from being displayed on the display device in a direction different from the layout condition as a search result, the same rotation condition information stored in the flowchart of FIG. 6 is used as the layout condition input by the user. The image of the image data is rotated and displayed so as to be in the direction.

  By providing a means for changing the grid width, the user can input more detailed layout conditions when necessary. For example, when a large number of candidates are listed when a search is performed, the target document and other similar documents can be distinguished by inputting more detailed conditions. Further, if there is a layout condition input in advance when the user instructs to change the grid width, the condition is inherited so as to be suitable for the grid width newly set when the grid width is changed.

  FIG. 7 is a diagram for explaining an example of changing the grid axis. FIG. 7A shows an example in which no layout condition is input when the grid width is changed. By pressing the “arrange” button 20, the grid is reset with a coarser grain size than the current grid granularity, and when the “fine” button 21 is pressed, the grid is reset with a smaller granularity than the current grid granularity. . FIG. 7B shows an example in which layout conditions are input when the grid width is changed, and the grid whose particle size has been changed is reset by the two buttons 22 and 23 as in FIG. 7A. At that time, the conditions specified in the grid before the change are set in the corresponding grid after the change. When a grid after change consists of a plurality of grids before change, a condition that occupies a large amount in the grid after change is set.

  By providing means for designating conditions having a plurality of attributes as means for inputting layout conditions, a more ambiguous condition search can be performed. FIG. 8 is a diagram for explaining an example thereof. When the region in the attribute information of the comparison image corresponding to the grid designated by the “character or background” 24 in FIG. 8 is a character or when it is a background It is determined that the degree of similarity is high, and in the case of the figure, it is determined that the degree of similarity is low. When the condition is designated as “invalid” 25 indicating that the condition is not used as a search reference, processing is performed so that the corresponding grid does not affect the similarity.

For example, when an ambiguous condition search is performed, the similarity calculation processing flow shown in FIG.
FIG. 9 is obtained by adding the processes of S30, S31, and S32 to the flowchart of FIG. In S30, the conditions of the target grid to be compared are determined, and when the condition is designated as “invalid”, the processing of S13 to S15 is not performed. Accordingly, it is possible to perform processing so that the grid designated as “invalid” does not affect the similarity.

When all the grids are designated as “invalid”, the value of the effective grid is 0, and in order to avoid the denominator of the division in S17 being 0, if the effective grid is 0 in S31, the process of S32 To migrate to. In S32, when all grids are designated as “invalid”, the similarity is set to 1 so that all image data are candidates.
Further, in the case of a condition for designating a plurality of attributes such as “character or background”, any of the plurality of attributes designated to shift to S15 as true if the attribute value is a character or background in S15. If it matches, the processing is set to be true.

  Instructing the addition of layout conditions in an image data search method for searching arbitrary image data associated as a document from a group of image data registered by associating a plurality of consecutively registered image data as one document And a means for storing one or more layout conditions. When an instruction is given to add a layout condition, the layout condition input up to that point is stored, and the next layout condition is newly input. It can be so. As a result, documents satisfying the logical product of a plurality of layout conditions are set as candidates. It is also possible to use a document or image data that satisfies the logical sum of a plurality of layout conditions as a candidate.

It is a figure which shows one structural example (example of stand-alone) of the image data search device by this invention. It is a figure which shows the other structural example (example which uses a server) of the image data search device by this invention. It is a figure which shows the example of a layout condition input screen. It is a figure which shows the example of a flow of the search process by this invention. 6 is a diagram illustrating an example of a document direction switching screen. FIG. It is a figure which shows the other example of a flow of the search process by this invention. It is a figure which shows the example of a grid width change. It is a figure which shows the example of condition specification. It is a figure which shows the example of a similarity calculation process flow at the time of ambiguous condition designation | designated.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Data registration part, 2 ... Data storage part, 3 ... Condition input part, 4 ... Search part, 5 ... Display apparatus (display part), 11 ... Layout condition creation frame.

Claims (17)

  In an image data retrieval device that retrieves arbitrary image data from electronically filed image data groups, a condition input unit that inputs layout conditions in units of grids, and image data is retrieved from image data groups using layout conditions. An image data search apparatus comprising: a search unit that performs a search; and an output unit that outputs a search result.   The image data search apparatus according to claim 1, wherein the condition input unit takes a unit of a grid area that can be specified to be larger than a pixel unit of an original image.   The image data search apparatus according to claim 1, wherein the condition input unit includes means for designating a shape of a layout condition when the condition is input.   The image data search apparatus according to claim 1, wherein the condition input unit includes means for designating a direction of a layout condition when the condition is input.   The search unit performs a search by rotating either the attribute information filed at the time of search or the specified layout condition at rotation angles of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, respectively, and comparing with the other. 2. The image data retrieval apparatus according to claim 1, further comprising means for performing the operation.   The output unit has means for rotating and outputting the image data so that it matches the layout specified by the user when the image data corresponding to the attribute information matching the condition is presented to the user as a candidate. 6. The image data search device according to claim 5, wherein   The image data search apparatus according to claim 1, wherein the condition input unit includes means capable of changing a granularity of a grid.   8. The image according to claim 7, wherein the condition input unit includes means for reflecting a condition input screen after the granularity change so as to inherit a condition that has already been specified before the granularity change. Data retrieval device.   The image data search apparatus according to claim 1, wherein the condition input unit includes means for specifying a condition such that all or a plurality of attributes are specified for a grid when a layout condition is specified.   The image data search apparatus according to claim 1, wherein the condition input unit includes means for specifying a condition for specifying a condition not to be searched for a grid when a layout condition is specified.   The image data search apparatus according to claim 10, wherein the condition input unit preliminarily specifies conditions of an initial state of all grids when a layout condition is specified.   In an image data retrieval apparatus for retrieving arbitrary image data from an electronically filed image data group, a condition input unit for inputting a plurality of layout conditions in units of grids, and image data from the image data group using the layout conditions An image data search apparatus comprising: a search unit for searching; and an output unit for outputting a search result with a document including a page meeting a plurality of designated layout conditions as a candidate.   In a search method for searching for arbitrary image data from electronically filed image data groups, a step of inputting layout conditions in units of grids, a step of searching image data from image data groups using layout conditions, An image data search method comprising a step of outputting a search result.   The image data search method according to claim 13, wherein the grid granularity can be changed.   In an image data retrieval method for retrieving arbitrary image data from an electronically filed image data group, a step of inputting a plurality of layout conditions in units of grids, and image data is retrieved from the image data group using the layout conditions. An image data search method comprising: a step; and a step of outputting a search result with a document including a page meeting a plurality of designated layout conditions as a candidate.   13. A control program for an image search apparatus for operating the image data search apparatus according to claim 1, wherein the control program causes a computer to function as each of the means.   The computer-readable recording medium which recorded the control program of Claim 16.

Images