JP2008134954A - Information processing device, its control method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing device which easily, quickly retrieves non-text information in documents which is very important and more frequently becomes a retrieval target, and also provide its control method and a program. <P>SOLUTION: The information processing device extracts a language expression relating to non-text information from document data, generates metadata for retrieving the non-text information based on the language expression, associates the generated metadata with the non-text information registers it in a storage medium and compares retrieval conditions relating to input with the registered metadata to retrieve the non-text information. At that time, the device extracts an emphasis expression relating to the non-text information from the document data and adds emphasis information, which indicates that the emphasis expression is made, to the metadata. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information processing apparatus suitable for searching non-text information such as a chart in a document, a control method thereof, and a program.

  In recent years, an enormous amount of electronic documents has been flooded due to the reduction of storage space and the convenience of acquisition from a remote location. For this reason, it takes a long time to search for a desired electronic document.

  In particular, electronic documents related to catalogs, papers, instruction manuals for electronic devices, and the like include not only text information but also a large number of non-text information such as images, diagrams, and tables. These non-text information is also frequently used information.

However, these non-text information itself does not have character information or is fragmented character information even if it has, it is difficult to search for desired non-text information. Therefore, a technique has been developed in which metadata (text information) is associated with a document or image information in the document, and the document or image information is searched using the metadata (for example, see Patent Document 1).
JP 2003-30243 A

  However, conventionally, metadata for search cannot be extracted by analyzing the non-text information, the relationship between its constituent elements, the structure of the non-text information, etc., and the non-text information in the document or its constituent elements cannot be extracted. It was not possible to search accurately.

  The present invention has been made under such a background, and an object thereof is an information processing apparatus capable of easily and quickly searching for non-text information having a high importance in a document and a high frequency of search. The control method and program are provided.

  In order to achieve the above object, the present invention provides an information processing apparatus for retrieving non-text information using metadata related to non-text information in document data, wherein the non-text information is retrieved from the document data. Extraction means for extracting a linguistic expression related to the text, creation means for creating metadata for searching for the non-text information based on the linguistic expression extracted by the extraction means, and meta data created by the creation means Registration means for registering data in the storage medium in association with the non-text information; search means for searching for non-text information by comparing the input search condition with the metadata registered by the registration means; And the extraction means extracts an emphasized expression related to the non-text information from the document data, and the creating means extracts the emphasized expression. The emphasis information indicating that when, characterized in that added to the meta-data.

  In the present invention, non-text information can be searched using metadata to which emphasis information is added. This emphasis information generally means that the metadata is related to a linguistic expression having high importance and high frequency of search.

  Therefore, according to the present invention, by searching for non-text information using the metadata to which the emphasis information having the above characteristics is added, the non-text information having a high importance in the document and a high frequency of search. It is possible to provide an information processing apparatus that can easily and quickly search, a control method thereof, and a program.

[First Embodiment]
FIG. 1 is a block diagram showing a basic configuration of an information processing apparatus according to first to third embodiments of the present invention. This information processing apparatus is designed to search non-text information such as desired images, figures and tables or its constituent elements from multimedia document data with high accuracy, and is constructed on a personal computer (PC). Yes.

  In FIG. 1, a CPU 101 is a controller (processing unit) that controls various processes in the information processing apparatus. The control is executed based on programs stored in the ROM 102, the external storage device 105, and the like which will be described later. The CPU 101 can execute a plurality of programs in parallel.

  The CPU 101 is connected to various devices such as a ROM 102, a RAM 103, an input unit 104, an external storage device 105, a display unit 106, and a communication unit 107 via a bus 110 so as to communicate with each other. The ROM 102 stores a program such as a boot program, data, and the like. When the CPU 101 actually executes processing, the RAM 103 is used as a work area, for example, by loading a program related to the processing from the ROM 102 or the external storage device 105 and developing the program.

  The input unit 104 includes a keyboard or the like, and has various function keys such as alphabetic keys, character input keys such as hiragana keys and katakana keys, and cursor movement keys. Note that the input unit 104 may be equipped with a pointing device such as a mouse.

  The external storage device 105 stores an OS, various programs such as application programs, and various data such as document data. The external storage device 105 includes a storage medium such as a hard disk and a nonvolatile semiconductor memory. When the external storage device 105 is a hard disk, it has a drive for recording data by driving the storage medium. The external storage device 105 also stores an application program for performing processing according to the flowcharts of FIGS.

  The display device 106 is composed of a liquid crystal display or the like, and displays data input by the input unit 104, processing contents in the CPU 101, and the like. The communication unit 107 performs control for communicating with other information processing apparatuses, printers, and the like via the LAN 108 and the connector 109. Communication control by the communication unit 107 makes it possible to share application programs and data according to the first to third embodiments with other information processing apparatuses.

  Communication by the communication unit 107 may be any communication method such as wired communication such as RS232C, USB, IEEE1394, P1284, SCSI, modem, Ethernet (registered trademark), wireless communication such as Bluetooth, infrared communication, IEEE802.11b. Good. Note that the communication unit 107 is connected to a storage device, a scanner, a printer, and the like via a connector 109.

  This information processing apparatus stores metadata for searching non-text information or its components in a document when storing a multimedia document (hereinafter referred to as a document) in the external storage device 105 or an external computer. Extracting.

  FIG. 2 shows an example of document data 201 for one page. The document data 201 includes a body (text information) 202, a heading (text information) 203, and a sentence (text information) 204 following the heading.

  The document data in FIG. 2 includes a chart element (non-text information) 205, a caption (description: text information) 206 of the chart element 205, a chart element 207, and a caption (description: text information) 208 of the chart element 207, A chart element 209 and a caption (descriptive text: text information) 210 of the chart element 209 are included. The caption means an explanatory text (text information) of the corresponding figure.

  Further, the document data of FIG. 2 has a caption 211 for the entire chart area composed of three chart elements 205, 207, and 209 and a page number (text information) 212. Note that the captions 206, 208, and 210 of the individual chart elements 205, 207, and 209 are also referred to as sub-captions for the caption 211 of the entire chart area.

  FIG. 3 shows structured document data indicating the analysis result of the document data 201 of FIG. FIG. 3 is an excerpt of the basic part of the layout analysis result. In FIG. 3, (3-1) to (3-32) describe the layout analysis result in the XML format. Among these, (3-1) indicates the start of the page of FIG. 2, and it is described that the page number is “11”. (3-32) indicates the end of the page.

  (3-2) to (3-4) are description parts indicating the page size of FIG. The size (600 900) of the page shown in (3-3) is based on the XY coordinate system in which the upper left corner of the page is the origin, the Y axis is the forward direction in the lower direction, and the X axis is the forward direction in the right direction of the page X and Y coordinate values are indicated. That is, it is described that the page is 600 in the X direction and 900 in the Y direction. Similarly, information such as position and size is indicated in an XY coordinate system in which the upper left corner of the page is the origin, the Y axis is the forward direction in the lower direction, and the X axis is the forward direction in the right direction of the page.

  (3-5) to (3-31) list the constituent elements (areas) of the document information in the page of FIG. 2, and each constituent element is expressed in units of blocks. Among these, (3-6) to (3-21) are descriptions expressing blocks in the areas of the sentences 202, 203, and 204 in FIG. In (3-6), the block identification number assigned in the layout analysis is “1101”, and a description indicating that the block is in a text area in which a sentence is described.

  In (3-7) to (3-9), description indicating the upper left position coordinate of the block in the XY coordinate system is described as information indicating the position of the block in the page. . In addition, (3-10) to (3-12) describe the size of the block in the XY coordinate system. This size is indicated by the distance in the X and Y directions from the upper left position coordinate of the block.

  (3-13) to (3-20) show the contents described in the block, and are text area blocks. Therefore, the sentence 204 following the body 202, heading 203, and heading 203 in FIG. is described. These sentences (text information) are obtained by converting image information into text information by using a technique such as character recognition when scanned from a paper medium or the like.

  In (3-15), the heading 203 part of FIG. 2 is described, text information indicating that the heading number is “5” and the heading title is “Furniture A” is added. ing. In addition, in (3-18), it is described in addition to the character string of the sentence 204 that an underline is given to the part “grounding part of the leg” in the sentence 204 of FIG. 2.

  (3-22) to (3-31) describe other components in the page such as the chart element 205 of FIG. In (3-22), the block identification number assigned to the chart element 205 in FIG. 2 in the layout analysis is “1102”, which describes that the block is in the graphic format. In (3-23) to (3-25), as the information indicating the position in the page of the block related to the chart element 205, the position coordinate on the upper left of the block related to the chart element 205 in the XY coordinate system is as follows. is described. Also, in (3-26) to (3-28), the size of the block related to the chart element 205 in the XY coordinate system is described. Further, although not shown, the chart element 205 itself is described below (3-30).

  In this embodiment, in order to search for non-text information such as images, diagrams, and tables in document data stored in the external storage device 105 in FIG. 1, layout analysis is first performed on the document data in advance. The layout analysis result is described in the XML format as shown in FIG.

  Then, based on the layout analysis result, a process of analyzing the mutual relationship between the non-text information area and its constituent elements, that is, the individual non-text information in the non-text information area is executed. Based on the relationship analysis result, processing for extracting and registering non-text information and search metadata for searching the constituent elements is executed.

  For example, the CPU 101 obtains the position of the block in which each chart element or sentence is described by performing block division by layout analysis. Then, the CPU 101 recognizes a character string of about one to two lines arranged in the vicinity of the chart element 205, the chart element 207, and the chart element 209 shown in FIG. Next, the CPU 101 recognizes these character strings as the explanatory texts (captions) 206 and 208 of the chart element 205 and the chart element 207.

  If a character string of about 1 to 2 lines is not arranged in the vicinity of the target chart element, the CPU 101 determines that no caption for the chart element exists. In addition, a character string of about 1 to 2 lines is placed near the target chart element, but other chart elements are placed closer to the chart element than the text string. However, the CPU 101 determines that there is no caption for the chart element.

  In the example of FIG. 2, chart element 205, chart element 207, and chart element 209 are in a parallel relationship, caption 206 corresponds to chart element 205, caption 208 corresponds to chart element 207, and caption 210 is a chart element. It can be recognized that it corresponds to 209. The CPU 101 extracts the captions 206, 208, and 210 as search metadata for searching the chart elements 205, 207, and 209, respectively, and associates the caption elements 205, 207, and 209 with the chart elements 205, 207, and 209, respectively. Register with etc.

  In the search process, for example, the CPU 101 searches the caption 206 as metadata, thereby reading the chart element 205 as non-text information associated with the metadata (caption 206) as a search result. The data is displayed on the display unit 106.

  As a result, various types of non-text information can be searched without depending on the format of various types of non-text information such as images, diagrams, and tables. Further, at the time of searching, the target non-text information can be searched simply by searching the searching metadata, and the time required for searching can be shortened.

  FIG. 4 is a conceptual diagram showing an analysis result of layout analysis for the document of FIG. In this embodiment, the layout analysis result is actually described in XML data as shown in FIG. 3, but is shown in a conceptual diagram in FIG. 4 for easy understanding.

  The area where the chart element 205, the caption 206 of the chart element 205, the chart element 207, the caption 208 of the chart element 207, the chart element 209, and the caption 210 of the chart element 209 are arranged as an entire chart area. That is, the entire chart area includes captions as text information and sub-captions. The chart area shown in FIG. 2 is a state in which chart elements are densely packed. That is, in the present embodiment, the term “overall chart area” is used to indicate a chart area that may include not only a pure chart but also text information.

  The captions and the like shown in FIG. 4 are obtained by performing the layout analysis of the document data in FIG. As shown in FIG. 2, the number “11” is present in an isolated state on the bottom line of the page, so the CPU 101 determines that it is a page number. If there are previous and subsequent pages, and the numbers “10” and “12” continuing to “11” exist at the same position, the CPU 101 determines that the number “11” in FIG. May be determined. In FIG. 4, the number “11”, which is text information obtained from the layout analysis result, is determined to be a page number as indicated by reference numeral 401 and is set to “page No. 11”. Reference numerals 402 and 404 shown in FIG. 4 correspond to the sentences 202 and 204 shown in FIG.

  In the document data of FIG. 2, a short character string “5. Furniture A” is arranged in an isolated state between a relatively long sentence 202 and a sentence 204. Furthermore, between the sentences on the previous page of the document data in FIG. 2, the numbers “4” and “6” continuing to “5” are written together with a short character string (not shown). In this case, the CPU 101 determines that the character string “furniture A” in FIG. 2 is “title” and determines that “5.” is the section number. Therefore, as indicated by reference numeral 403 in the layout analysis result of FIG. 4, “sentence (section) number =“ 5 ”, title =“ furniture A ””.

  That is, in the present embodiment, the CPU 101 performs block division by layout analysis. As a result, when a description of “number” and “character string of about one line” is recognized in a certain area, the recognized description is arranged at a predetermined distance or more away from the previous area (upper area). Determine whether or not. And then. If it is determined that they are arranged at a predetermined distance or more, the CPU 101 regards “number” as a sentence (section) number and determines “a character string of about one line” as a title. Note that the section number or title when the font (font) of “number” and “character string of about one line” is different from those of other areas, or when the character size is larger than those of other areas. May be estimated.

  Further, in the document data of FIG. 2, a relatively short character string “product No. 100 furniture A” is described in an isolated state on the bottom line of the entire chart area. In addition, the other three character strings “side view”, “enlarged view of leg”, and “overall view” in the entire chart area are directly below the figure in a one-to-one correspondence with the three figures. Are listed. Therefore, the character string “product No. 100 furniture A” is the caption of the entire chart area (the entire chart area), and the other three character strings are the captions of the figure immediately above (in this case, the sub-captions). ). Therefore, in the layout analysis result of FIG. 4, as indicated by reference numerals 405 to 408, these character strings are used as captions.

  That is, in the entire chart area, captions are registered as follows. First, the CPU 101 performs layout analysis of document data and executes block division processing. As described above, the CPU 101 associates the character string (206, 208, 210) closest to the recognized area of the chart element (205, 207, 209) with the chart element as the caption of the chart element. Register in the external storage device 105.

  Then, the CPU 101 determines whether another chart element area exists within a predetermined distance from the top, bottom, left, and right area edges of a certain chart element area. When other chart element areas exist, the CPU 101 registers the entire area including these areas in the external storage device 105 as an entire chart area.

  Further, the CPU 101 determines whether or not there is a character string area of about 1 to 2 lines not associated with the chart element within a predetermined distance from the entire chart area. When such a character string (211) area exists, the CPU 101 registers the character string in the external storage device 105 as a caption of the entire chart area.

  As described above, in the first embodiment, layout analysis is performed, text information (caption) and non-text information (chart) are identified by the layout analysis, and the caption and the chart are determined based on their positional relationship and the like. Correspondences, relationships between captions, etc. are certified.

  In addition, after performing the layout analysis or in parallel with the layout analysis, the meta data for search for searching for the non-text information element by analyzing the characteristics such as the structure and attributes of the non-text information itself such as a chart or the like. It is also possible to extract data. For example, a character string described in itself such as a pie chart or a drawing is searched. Then, the character string is searched from the surrounding character blocks such as the pie graph and the drawing, and the sentence including the character string is analyzed to recognize the features, attributes, and the like of the pie graph and the drawing. You may comprise so that the character string etc. of a part of analyzed sentence may be extracted as search metadata, such as the said pie chart and drawing.

  Next, search metadata for searching the entire chart area constituted by the individual chart area group in the document data and chart elements (individual chart, image information) constituting the individual chart area are extracted and registered. The processing will be described based on the flowchart of FIG. It is assumed that, prior to performing the search metadata extraction / registration process, the above-described layout analysis process, registration process of chart elements, captions, and the like to the external storage device 105 are performed.

  First, the CPU 101 acquires one entire chart area from the external storage device 105 based on the analysis result of the document data layout (step S501). Here, it is assumed that the entire chart area in the lower half of FIG. 2 has been acquired as shown in FIG. 4 as a result of the layout analysis of the document data for one page of FIG.

  Next, the CPU 101 extracts caption information of the acquired entire chart area (step S502). Here, the CPU 101 determines that the text information of about 1 to 2 lines that are not registered as sub caption information in the vicinity of the entire chart area is the caption information of the entire chart area. 2 and 4, the character string “product No. 100 furniture A” 211 in FIG. 2 is extracted as caption information of the entire chart area.

  Next, the CPU 101 extracts the emphasis information indicating the emphasis expression when the entire chart area extracted in step S501 and the caption information extracted in step S502 are emphasizing (step S503). Note that the lower half of the entire chart area in FIG. 2 and the caption information of the entire chart area are not emphasized, so that the highlight information is not extracted in step S503.

  Here, as a form of emphasis expression, a special type of character (font), character size, character modification, character color and character background color, etc., in addition to emphasis expression for the character or area itself, an enclosing frame In addition, a form that is indirectly emphasized by a symbol such as an arrow is included.

  Then, the CPU 101 registers the extracted caption information of the entire chart area in the external storage device 105 as search metadata for searching the entire chart area (step S504). In this case, when the enhancement information is extracted in step S503, the enhancement information is also registered in the external storage device 105 in association with the caption information.

  Here, the registered emphasis information may be information as it is in the form of the original emphasis expression emphasized in the document data. However, in order to reduce the memory capacity, it is desirable to register information indicating the type of emphasis expression as emphasis information such as “bold”, “red”, “thick frame”, “emphasis frame”, etc. (described later). The same applies to step S511). Furthermore, as will be described later, it is also possible to register the word “emphasis” for all the emphasis types without indicating the emphasis type.

  Note that the emphasis expression is generally made for a portion having high importance and high frequency of search.

  Next, the CPU 101 determines whether or not the processing in steps S506 to S510 described later has been completed for all individual chart areas in the entire chart area acquired in step S501 (step S505). As a result, when the processing of steps S505 to S510, which will be described later, is completed for all individual chart regions, the CPU 101 ends the registration processing of the caption information and the like.

  On the other hand, if the processing in steps S506 to S510 described below for all the individual chart regions has not yet been completed, the CPU 101 takes out only one individual chart region that has not yet been subjected to the process, and The caption (subcaption) information of the chart element in the chart area is extracted (step S506).

  In the example of FIG. 2, there are chart elements 205, 207, and 209 as individual chart elements, and as sub-caption information thereof, “side view” denoted by reference numeral 206, “leg part” denoted by reference numeral 208. “Enlarged view”, “overall view” indicated by reference numeral 210 exists. Among these, the sub-title “enlarged view of the leg” is emphasized because the character size of the character string “leg” is larger than the character size of the other character strings. Further, the individual chart element indicated by reference numeral 207 is surrounded by a thicker frame line than other individual chart elements and is emphasized.

  Next, if the emphasized expression is made for the individual chart area acquired in step S506 or the subcaption information, the CPU 101 extracts the emphasized information indicating the emphasized expression (step S507).

  Here, as described above, the individual chart areas 205 and 209 shown in FIG. 2 and the caption information 206 and 210 of these chart areas are not emphasized, and therefore, in step S506, the individual chart areas 205 and 209 are not highlighted. , 209 and the sub-caption are acquired, the enhancement information is not extracted in step S507. On the other hand, the individual chart area 207 shown in FIG. 2 and the caption information 208 of the chart area 207 are emphasized as described above. Therefore, when the individual chart element 207 and its caption information 208 are acquired in step S506, the CPU 101 extracts the emphasis information indicating the emphasis expression in step S507.

  Next, the CPU 101 estimates the role and relationship of the individual chart element acquired in step S506 with respect to the entire chart area acquired in step S501 (step S508). In this estimation process, when only layout analysis is performed, for example, the individual chart element is a part of the whole chart area based on the relationship between the position and the occupied area of the individual chart element in the whole chart area. It can be estimated that there is.

  By conducting a logical analysis, it is possible to grasp the more accurate relationship and role between individual chart elements and the relationship and role between the entire chart area and individual chart elements, and the entire chart area for each chart element. You may make it estimate the more exact role and relationship with respect to.

  Next, the CPU 101 analyzes the role and relationship with respect to the caption of the entire chart area acquired in step S502 of the sub-caption of the individual chart element extracted in step S506 (step S509). Here, for example, the relationship between “Product No. 100 furniture” which is the caption of the entire chart area and “Side view” which is a sub caption of the individual chart element 205 is the simplest relationship between words in natural language. It is analyzed that there is a certain modification relationship.

  Next, the CPU 101 stores the feature information such as the relationship between the caption of the entire chart area and the sub-caption of the individual chart element as search metadata for retrieving the chart element in the external storage device 105 or the like. Registration in the medium (step S510).

  That is, the CPU 101 also registers information indicating that the caption and the sub-caption are in a modification relationship. Further, the CPU 101 associates and registers the search metadata, the chart elements to be searched with the search metadata, and captions (including sub-captions) in the external storage device 105. Further, when the CPU 101 extracts the highlight information in step S507, the CPU 101 registers the highlight information in the external storage device 105 in association with the sub-caption information.

  When the CPU 101 returns to step S505 and there remains an individual chart element that has not been subjected to the processes of steps S506 to 510, the CPU 101 performs the processes of steps S506 to 510 on the individual chart elements. The registration process of FIG. 5 is executed for each entire chart area (individual chart element) of each page of the document.

  FIG. 6 is a conceptual diagram when the CPU 101 performs layout analysis on the document data of FIG. 2 and obtains the layout analysis result of FIG. 4, and then performs the search metadata registration processing of FIG. Reference numeral 601 indicates the entire chart area and its caption. Reference numerals 602 to 604 indicate individual chart elements and their captions (subcaptions).

  As shown in FIG. 6, as the search metadata for searching the entire chart area of the document data in FIG. 2, “Product No. 100 Furniture A” which is all captions and sub-captions in the document data. , “Side view”, “enlarged view of the leg”, and “overall view” are registered. Further, as search metadata for searching the individual chart element 205 of the document data of FIG. 2, the caption “Product No. 100 Furniture A” of the entire chart area of the document data of FIG. A sub-caption “side view” is registered. In addition, it is registered that the caption “product No. 100 furniture A” and the sub caption “side view” have a relationship of “modification”.

  Further, as search metadata for searching the chart element 207 of the document data in FIG. 2, the caption “Product No. 100 Furniture A” in FIG. 2 and the sub-caption “leg” of the chart element 207 are included. Part enlarged view "is registered. In addition, it is registered that the caption “product No. 100 furniture A” and the sub-caption “enlarged view of the leg portion” have a relationship of “modification”.

  Furthermore, the character size of the character string “leg” in the sub-caption “enlarged view of the leg” is larger than other character strings “expansion” etc., and “leg” is emphasized. Emphasis information (emphasis) is added to the word “leg” constituting the subcaption. Also, the enlarged view of the leg itself, which is an individual chart element, is surrounded by a thick frame line, and the enlarged view of the leg is emphasized. Emphasis information (emphasis) to that effect is added to the whole. The emphasis information added to the search metadata is used to accurately and quickly search a desired chart in document data, as will be described later.

  Similarly, as search metadata for searching the chart element 209 of the document data of FIG. 2, the caption “Product No. 100 Furniture A” of the document data of FIG. "Overall view" is registered. In addition, it is registered that the caption “product No. 100 furniture A” and the sub-caption “overall view” have a relationship of “modification”.

  For example, “Product No. 100 “Furniture A— (Modification) —Side view” metadata and “Product No. The search metadata “100 furniture A- (modification) -overall view” indirectly indicates that there is a relationship between the “side view” and the “overall view”. Accordingly, the relationship between individual non-text information, that is, the relationship between sub-captions can be used as search metadata.

  Next, the search process will be described. When searching non-text information such as charts in document data, search metadata similar to a character string as a search condition input by a user is used.

  For example, it is assumed that the character string “legs of chair A” is input as a search condition. Of the character string as the search condition, “chair A” is input as a thick character string. In this case, as shown in FIG. 7, the CPU 101 morphologically analyzes the character string “legs of chair A”, which is a search condition, and breaks it down into words, and as search instruction information for each part of speech and classification of the word. Determine the importance of.

  At this time, since “chair A” is input as a thick character string, the CPU 101 sets the importance of the “chair A” as high as search instruction information. That is, as shown in FIG. 7, “80” is set as the importance of the noun “leg part” and the suffix “figure” input in the normal character string that is not thick. On the other hand, “90”, which is higher than “80” in the normal form, is set as the importance of the noun “chair” input in the character string in the form of meat. In addition, among the nouns input in the character string in the form of flesh, “100” is set as the importance level of the proper noun “A”. Note that “0” is set as the importance of the particle “no”.

In addition, the CPU 101 recognizes that there is a modification relationship between “A” and “leg”.
Further, although not shown in the drawing, the CPU 101 determines the importance of the search metadata as well as the search condition. As a method for determining the importance in this case, the same method as in the search condition can be used.

  Then, the CPU 101 evaluates the similarity between the search condition and the importance of the search metadata.

  That is, the CPU 101 searches for search metadata including a word whose meaning is similar to that of the word related to the search condition, and sets the similarity in meaning between the words as the similarity between the search condition and the search metadata. In addition, the more the search metadata includes words similar in meaning to the words related to the search condition, the higher the degree of similarity of the search metadata with respect to the search condition.

  Further, when the modification relationship between similar words in the search condition is similar to the modification relationship between similar words in the search metadata, the CPU 101 determines the search condition of the search metadata. The similarity to is evaluated even higher.

  As described above, in the search condition and the search metadata, high importance is set for the word related to the emphasized expression. Therefore, when evaluating the similarity between the search condition and the search metadata, when either the search condition or the search metadata is highlighted for words having the same or similar meaning Is evaluated with higher similarity than usual. Further, when words having the same or similar meaning are emphasized in both the search condition and the search metadata, the degree of similarity is evaluated even higher.

  Then, the CPU 101 displays non-text information such as a chart associated with the search metadata similar to the search condition on the display unit 106 as a search result. In this case, the CPU 101 performs search processing in the order of search metadata that is highly evaluated for similarity to the search condition, and arranges and displays non-text information associated with the search metadata in the search order. Display on the device 106.

  For example, in the case where the above-mentioned “leg view of chair A (chassis A is emphasized with thick meat)” is used as a search condition, the similarity order of the non-text information in FIG. 2 with respect to this search condition is the number shown in FIG. The order is 603, 601, 603, and 604, and the search results are displayed in this order.

  Further, when “product No. furniture A” is used as a search condition, the non-text information with the highest evaluation is number 601, and subsequently, numbers 602 to 604 have the same similarity. Accordingly, the number 601 is displayed at the top as the search result, and the numbers 602 to 604 are displayed in random order.

  As described above, the non-text information area and the individual non-text information can be searched using the search data that takes into account the relationship between the non-text information area and the individual non-text information. The non-text information area and the individual non-text information can be searched with high accuracy.

  Moreover, since the importance used for similarity evaluation is set high for the highlighted search conditions and search metadata, it is possible to easily and quickly search for important non-text information.

  Note that the search metadata for searching for non-text information can be extracted not only from the area to which the non-text information belongs, but also from a text area in the vicinity thereof. Further, not only layout analysis but also morphological analysis of captions and logical analysis of the structure and relationship of non-text information can provide more detailed search data relationships. In this case, non-text information and the like can be searched with higher accuracy with higher accuracy.

  Next, an actual search screen will be described with reference to FIG. When the user instructs a document search process by operating the input unit 104, the CPU 101 displays a document search window 801 shown in FIG. 8 on the screen of the display unit 106.

  A title bar 802 is formed in the document search window 801. In the title bar 801, “document search” is displayed as the title of the window.

  In addition, a search window 803 is formed in the document search window 801. In this window 803, a search condition input box 804 for inputting search conditions and a search result display box 805 for displaying search results are displayed. Is formed.

  In the search condition input box 804, the search condition can be input in a format such as a sentence in a natural language having relationship information between words or phrases as well as simple words. In the example of FIG. 8, the search condition input box 804 displays the character string “legs of chair A” as the search condition. Further, the character string “chair A” is emphasized by a thick character string. Such a search condition can be input to the search condition input box 804 using the input unit 104 shown in FIG.

  First, the user operates a predetermined button with the search condition input in the search condition input box 804. Then, in response to the detection of the input operation, the CPU 101 searches the external storage device 105 for search metadata similar to the search condition related to the input. Then, the CPU 101 reads the entire chart area, which is a non-text information area associated with the search metadata, and individual chart elements, etc., which are non-text information, and displays them in the search result display box 805 as search results. . At this time, the CPU 101 not only displays the search results (see reference numerals 806, 808, and 809 in FIG. 8) in the search result display box 805 in descending order of similarity, but also relates to the search result search metadata. Display metadata with selectability.

  In other words, when the search condition is “a leg diagram of chair A (chassis A is thick and emphasized)”, a character string exactly the same as the character string “leg figure” is included in the character string of this search condition. As shown in FIG. 6, the search metadata “Product No. 100 Furniture A— (Modification) —“ Expanded view of “leg part” <emphasis> ”<emphasis>” related to number 603 is included. ing. Further, the search metadata associated with the number 603 includes the character “A” in the character string of the search condition as described above.

  Moreover, “A” is highlighted in both the search condition and the search metadata. In addition, the “leg” is highlighted in a large character size form in the search metadata.

  Therefore, the CPU 101 determines that the search metadata having the highest similarity to the search condition is “product No. 100 furniture A- (modification)-“ enlarged view of “legs” <emphasis> ”<emphasis>”. Acknowledge that there is. Then, the CPU 101 displays the “enlarged view of the leg” (the drawing 207 in FIG. 2) associated with the search metadata as the search result 806 on the top of the search result display box 805.

  Further, as described above, the search metadata “enlarged view of the leg” in the search result 806 has a relationship between the search metadata “product No. 100 furniture A” and “modification”. Information is given. Therefore, the CPU 101 displays the search metadata “product No. 100 furniture A (whole figure)” on the screen of the display unit 106 as a view selection of the corresponding part in the search result 806 (see FIG. 8 807).

  In addition, a check box indicated by a mark “◯” is provided in the search metadata displayed as the view selection of the corresponding part. When a check mark is entered in this check box and a predetermined button is operated, the CPU 101 displays information associated with the search metadata corresponding to this check box instead of the search result currently displayed as the search result. To do.

  For example, the user puts a check mark in a check box corresponding to “Product No. 100 Furniture A (whole figure)” denoted by reference numeral 807, selects the entire chart area, and confirms the selection by operating a predetermined button. Suppose that When the CPU 101 displays the search result in response to this operation, the display content of the display area of the search result 806 changes from “enlarged view of the leg” to “entire figure of furniture A”. In this manner, other information related to the currently displayed search result can be easily displayed, so that desired information can be accurately searched and reused.

  In FIG. 8, black in the “◯” mark indicates a check mark. This check mark is set by default for the search metadata related to the search result when the search result is displayed first.

  When the CPU 101 first displays the search result on the display unit 106, the CPU 101 associates the search metadata with the search metadata in the descending order of the similarity with the character string (language expression) related to the search condition. Display text information as search results. However, if there is at least one non-text information (such as individual chart elements) having search metadata with a higher degree of similarity than the search metadata for the non-text information area (the entire chart area) The included non-text information (the entire chart area) is configured not to be displayed as the first search result.

  The reason is that the entire chart area related to the relatively low similarity can be displayed without omission by the view selection operation of the corresponding part for each chart element initially displayed as the search result as described above. As a result, a limited number of display screens can be used effectively to display more search results on a single display screen, which makes it possible to quickly find a desired chart and the like. Will improve.

  Further, when the similarity to the search condition is highest in the search metadata related to the chart element, the CPU 101 also displays the chart element as the first search result. In this case, the CPU 101 displays the search metadata of all the individual non-text information in the area as the search metadata related to the view selection corresponding to the search result (non-text information area).

  In the above description, only the modification relationship such as the entire chart area and individual chart elements (whole and part) is illustrated as the relationship of the search metadata. Other relationships may be used. Further, in order to select a desired search result more accurately and reuse it, it is also possible to add a similarity to the search condition to each search metadata to be displayed. . In this case, it is generally considered that the similarity is displayed with a number, but it is also possible to display the similarity with a graph or the like so that the similarity can be recognized at a glance.

[Second Embodiment]
In the first embodiment, the case where the non-text information is “drawing” is exemplified. However, even when the non-text information is “table”, the search metadata is almost the same as the first embodiment. Can be created. In this case, the item name is recognized by analyzing the structure of the table itself, and the item name may be registered as search metadata in association with the item value related to the item name. When the search result is displayed, if the item name corresponding to the search result is, for example, the item name in the horizontal column of the table, the item name in the vertical column and the entire table are selected for the above view selection. What is necessary is just to comprise so that it may display as metadata for search.

  The item names in the table are constituent elements of the table and have a character as a sub-caption (explanation of chart elements). If a table name or the like is given to this table, the table name or the like functions as a caption. In the case of a table, one item value is directly searched by using the relationship between the item name (component) in the horizontal column and the item name (component) in the vertical column as search metadata. be able to.

  Hereinafter, a process for creating search metadata for a table will be described with reference to the document of FIG. The document data 901 for one page shown in FIG. 9 has a sentence 904 (text information), a heading (text information) 903, and a sentence 904 following the heading 903.

  A table 905 is described in the lower half of the page. The table 905 includes a vertical column heading (item name) 906 and a horizontal column heading (item name) 907. This table 905 records the annual sales by product, as indicated by the caption “910 annual sales by product”. Accordingly, the year is described as the heading 906 in the vertical column, and the product name is described as the heading 907 in the horizontal column. In the item value area of the table 905, sales numbers 908 and 909 of each product for each year are recorded. A page number “11” indicated by reference numeral 911 is described below the page.

  When the first embodiment is applied to such a table 905, the headings 906 and 907 in the table 905 may be handled in the same way as captions (sub-captions) of chart elements. In addition, the character string “annual sales by product” written on the lower side of the table 905 may be handled as the caption of the table 905. Further, the sales number 909 of the product B in 2001 is shaded and emphasized. Therefore, the importance of the search metadata relating to the sales number 909 of the product B in the 2001 fiscal year may be set high.

  That is, when the search metadata is extracted from the lower half of the document data 901 in FIG. 9 by the same method as in the first embodiment, the result is as shown in FIG. In FIG. 10, the number 1001 indicates the entire table area and its caption. Numbers 1002 and 1003 respectively indicate the number of sales of “product B” in 2000 and 2001 and its caption (subcaption).

  As shown in FIG. 10, the search metadata for searching the entire table area in the lower half of the document data in FIG. 9 includes all captions and sub-captions in the table area “annual sales by product”. ”,“ 2000 ”,“ 2001 ”, etc., and“ Product A ”,“ Product B ”,“ Product C ”, etc. are registered.

  Further, as the search metadata for searching the chart element (sales number) 908 of “product B” in the table 905 of FIG. 9, as shown by the number 1002, “100”, the entire table area The caption “annual sales by product” and sub-captions “2000” and “product B” of the chart element 908 are registered. In addition, it is registered that there is a “modification” relationship between the caption “annual sales by product” and the sub-caption “2000” and between the sub-caption “2000” and the sub-caption “product B”. Yes.

  Further, the search metadata for searching the chart element (sales number) 909 of “product B” in the table 905 of FIG. 9 is registered as indicated by reference numeral 1003. That is, “1234 items”, caption “annual sales by product” of the entire table area, and sub-captions “2001” and “product B” of the chart element 909 are registered. It is also registered that there is a “modification” relationship between the caption “annual sales by product” and the subcaption “2001”, and between the subcaption “2001” and the subcaption “product B”. Yes.

  Further, the sales number “1234” of “product B” in “2001” is highlighted in a shaded form. Emphasis information (enhancement) to that effect is added to the words “2001” and “product B” constituting the subcaption, and further, the emphasis information (emphasis) is also added to the item value “1234” of the subcaption. It has been added.

  By comparing the metadata for search related to “table” with the search condition related to input and evaluating the similarity between them, search is also performed for the data described in “table” and “table”. It becomes possible to do. In the search process, the emphasized data is calculated with a high similarity to the search condition and is displayed as a search result with priority. Therefore, it is possible to easily and quickly search for important matters that are highlighted.

[Third Embodiment]
In the first and second embodiments, the search metadata for searching the chart area or individual chart elements in the chart area is extracted from the chart area itself.

  On the other hand, it is also possible to extract search metadata from text information in an area near the chart area. This is because the text information in the vicinity of the chart takes into account the document characteristic that there is a high possibility that the portion explained for the chart exists.

  As a method for extracting search metadata from text information in a region near the chart region, the same method as in the first and second embodiments can be used. For example, as in the first and second embodiments, the linguistic expression relationship between the caption of the entire chart area and the caption (subcaption) of the individual chart elements therein is estimated. Then, words similar to the linguistic expression are extracted from text information in the vicinity of the chart area, and additionally registered as search metadata for searching the chart area or the chart element.

  The search metadata extraction process in the third embodiment will be described based on the flowchart of FIG. Note that the processing in steps S1101 to S1110 in FIG. 11 is exactly the same as that in steps S501 to S510 in FIG. 5, and detailed description thereof will be omitted.

  The CPU 101 extracts and registers search metadata for searching the chart area or its chart elements from the chart area extracted in step S1101 by the processing of steps S1101 to S1110.

  Next, the CPU 101 performs linguistic expression between the caption of the graphic area (overall chart area) and the caption (subcaption) of the graphic element extracted in step S1109 from the text information in the vicinity of the chart area. A word (language expression) similar to the relationship is searched (step S1111).

  The text information in the vicinity of the chart area is not necessarily the text information of the same page as the chart area. For example, when a chart is present at the top or the last part of a page, search metadata can be extracted from text information of the previous page or the next page of the chart.

  Further, the degree (range) of the neighborhood is arbitrary, and the degree of the neighborhood may be set by the user according to the document type, characteristics, and the like.

  Further, the neighborhood can be set not only in a physical separation distance such as a line unit or a page unit, but also in a document constituent unit such as a “section” unit of the document.

  Next, the CPU 101 determines whether or not the similar language expression is present in the text information in the vicinity of the chart area (step S1112). If not, the process returns to step S1105.

  On the other hand, if a similar language expression exists in the text information in the vicinity of the chart area, if there is an emphasized expression in the similar language expression, the emphasized expression is extracted (step S1113).

  Next, the CPU 101 additionally registers the similar language expression and the emphasized expression obtained in steps S1111 and S1113 in the external storage device 105 as search metadata for searching the chart area and the chart element (step S1111, S1113). S1114). In this case, similar linguistic expressions and emphasized expressions are not registered as they are, but are morphologically analyzed as shown in FIGS. 6 and 10 and the relationship / role of captions and subcaptions are clearly indicated. It is registered with.

  Next, the above processing will be described with reference to the example of the document in FIG. In the case of the chart area of FIG. 2, from the text information 204, “here, furniture A in which the grounding part of the leg is devised as home furniture so as not to damage the floor”. A linguistic expression is found as a similar linguistic expression. In addition, it is recognized that the “grounding portion of the leg” with an underline is emphasized.

  Then, as shown in FIG. 6, the linguistic unit obtained by performing morphological analysis on this similar language expression is added with information indicating that the emphasis expression has been made, and is registered as search metadata.

  Further, in the table of FIG. 9, regarding the item value of 909, there is a linguistic expression such as “The remarkable increase in the number of sales of the product B in 2001 is ...” from the text information 904. Found. In addition, it is recognized that the underlined “Rapid expansion of sales volume of product B in 2001” is emphasized.

  Then, as shown in FIG. 10, the linguistic unit obtained by performing morphological analysis on the similar language expression is added with information indicating that the emphasis expression has been made, and is registered as search metadata.

  Note that the search metadata that has a similarity relationship only with the caption of the chart element in the chart area is registered as the search metadata for searching for the chart element. Also, search metadata that has a similar relationship with both the caption of the entire chart area and the caption of the chart element is registered as search metadata for searching the entire chart area and chart elements. To do.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer of the system or apparatus (or CPU, MPU, or the like). Is also achieved by reading and executing the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention. .

  Examples of the storage medium for supplying the program code include a floppy (registered trademark) disk, a hard disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, and a DVD. An optical disc such as RW or DVD + RW, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used. Alternatively, the program code may be downloaded via a network.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (Operating System) running on the computer based on the instruction of the program code. Includes a case where the functions of the above-described embodiments are realized by performing part or all of the actual processing.

  Furthermore, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the expanded function is based on the instruction of the program code. This includes a case where a CPU or the like provided on the expansion board or the expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows the basic composition of the information processing apparatus which concerns on the 1st-3rd embodiment of this invention. It is a figure which shows the example of a document for demonstrating 1st Embodiment. FIG. 3 is a diagram describing the analysis result of the layout analysis of the document in FIG. 2 in XML. It is a conceptual diagram which shows the analysis result of the layout analysis of the document of FIG. It is a flowchart which shows the extraction process of the metadata for a search in 1st, 2nd embodiment. FIG. 6 is a conceptual diagram illustrating a registration state of search metadata when the process of FIG. 5 is performed on the document of FIG. 2. It is a conceptual diagram which shows the example of analysis of the character string which concerns on the search condition in the case of performing a search process. It is a figure which shows the example of a display of the search result which concerns on 1st Embodiment. It is a figure which shows the example of a document for demonstrating 2nd Embodiment. FIG. 10 is a conceptual diagram illustrating a registration state of search metadata when the process of FIG. 5 is performed on the document of FIG. 9. It is a flowchart which shows the extraction process of the metadata for search in 3rd Embodiment.

Explanation of symbols

101 ... CPU
102 ... ROM
103 ... RAM
104 ... Input unit 105 ... External storage device 106 ... Display 805 ... Search result display box

Claims (7)

An information processing apparatus that searches for non-text information using metadata related to non-text information in document data,
Extracting means for extracting a linguistic expression related to the non-text information from the document data;
Creating means for creating metadata for searching for the non-text information based on the linguistic expression extracted by the extracting means;
Registration means for registering the metadata created by the creation means in a storage medium in association with the non-text information;
A search unit for searching for non-text information by comparing a search condition related to input and the metadata registered by the registration unit;
The extraction unit extracts an emphasis expression related to the non-text information from the document data, and the creation unit adds emphasis information indicating the fact to the metadata when the emphasis expression is extracted. Information processing apparatus.   The information processing apparatus according to claim 1, wherein the extraction unit extracts a linguistic expression related to the non-text information from an area of a predetermined range where the non-text information exists.   2. The information processing according to claim 1, wherein the extraction unit extracts a linguistic expression related to the non-text information from an area of text information in the vicinity of a predetermined range area where the non-text information exists. apparatus.   The search means evaluates the similarity of both by comparing the search condition according to the input and the metadata registered by the registration means, and the non-text information associated with the metadata in descending order of similarity The information processing apparatus according to claim 1, wherein the information is read from the storage medium and displayed as a search result.   The information processing apparatus according to claim 1, wherein the search unit highly evaluates the similarity of the metadata to which the emphasis information is added. A method of controlling an information processing apparatus that searches for non-text information using metadata related to non-text information in document data,
An extraction step of extracting a linguistic expression related to the non-text information from the document data;
A creation step of creating metadata for searching for the non-text information based on the linguistic expression extracted by the extraction step;
A registration step of registering the metadata created by the creation step in a storage medium in association with the non-text information;
A search step for searching for non-text information by comparing a search condition related to input and the metadata registered by the registration step;
The extracting step extracts an emphasized expression related to the non-text information from the document data, and the creating step adds emphasized information indicating the fact to the metadata when the emphasized expression is extracted. A method for controlling the information processing apparatus.   A program for executing the control method according to claim 6.

Images