JP2006065588A - Information reading device, information reading program, and information reading program recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a user to easily produce link data for a document only for reading, and to reduce a burden on a reader when a link character string is displayed. <P>SOLUTION: The information reading device comprises a storage device 3 for storing first electronic data including text data and second electronic data including data recognizable as characters, an electronic document display control section 15 for producing data for display for displaying at least the first electronic data, a display device 5 for displaying the data for display, an input device 2 allowing an input operation, a character string input processing section 10 for specifying a specific character string in the first electronic data as a selected character string based on an input from the input device 2, a data retrieval section 12 that recognizes a character from the second electronic data and retrieves a character string acting as a reference character string matching with the selected string for the recognition result, and a link data producing section 14 for producing the link data indicating an association between the selected string and the data position of the reference character string. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information browsing apparatus, an information browsing program, and an information browsing program recording medium that allow a user to easily understand an electronic document such as an electronic book and that is convenient for the user.

  Currently, many electronic documents are composed of text data, image data, sound data, and the like. For example, in a document composed of text data and image data, the text data and the image data are described separately in separate files, or are described at separate positions in the same file. Then, the document creator assigns a figure number or the like to one image data, writes the figure number in the text data, and explains in text while referring to the image data to the reader.

  However, if the user tries to view the image data while reading the text data of the electronic document in which the text data and the image data are separated from each other, the screen must be scrolled to the position where the image data exists. Don't be.

  Similarly, if the user wants to listen to sound data related to the text data and the sound data that are separated from each other while reading the text data of the electronic document, the sound data must be searched and reproduced.

  Therefore, for example, a document creator associates a character string representing a figure number in text data with image data or sound data corresponding to the character string in advance (hereinafter, the character string thus associated is referred to as a link character string). ), When the reader designates the link character string, there is a technique for jumping to the corresponding data and displaying or reproducing the data.

  As prior arts for creating such a link character string, there are JP-A-6-314230 (hereinafter referred to as the first prior art) and JP-A-2003-308314 (hereinafter referred to as the second prior art).

  In the first prior art, by simultaneously displaying reference object data referring to certain data and referenced data referenced from the reference object data, and dragging an image representing the referenced object data to the reference object data, Any two pieces of electronic data can be associated. The first prior art will be specifically described with reference to FIG.

  When the user designates the character string 1001 of FIG. 20 as reference body data using a mouse or the like, the character string is stored in the memory as a reference character string. When the user drags the image 1002 representing the referenced object data to the position of the character string 1001, the CPU stores the character string 1001 and the referenced object image 1002 in association with each other.

  In the second prior art, when an electronic document is created, the electronic document can be associated with other electronic data by presenting data related to the currently created electronic document. The second prior art will be specifically described with reference to the block diagram of FIG.

The word string input by the user in the document input unit 1101 is stored in the storage unit 1105 together with position information in the electronic document as an electronic document database 1105B, and is passed to the document information extraction unit 1102 and the information search unit 1103. The document information extraction unit 1102 extracts word data or the like serving as a search key for an electronic document from a word string passed from the document input unit 1101 by morphological analysis or the like, and stores it in the storage unit 1105 as search data 1105A. The information search unit 1103 determines whether the word string input by the document input unit 1101 matches the search key of the search data 1105A, and has a search key that matches in the electronic document database 1105B. Search for electronic documents. The document information presentation unit 1104 displays the similar electronic document corresponding to the search.
JP-A-6-314230 (published on November 8, 1994) JP 2003-308314 A (published October 31, 2003)

  However, in the prior art, when a person other than the document creator creates a link character string, it is necessary to grasp the document structure created by another person, so a great deal of effort is required to create the link character string. It becomes. Hereinafter, this problem will be described in detail for each conventional technique.

  In the first prior art, in order to associate the reference object data with the referenced object data, the user must manually search for the referenced object data, so a great amount of labor is required to create the link character string. Become.

  In the second prior art, when a user searches for an image related to a text character string in order to associate a text character string with a certain image data, the text character string at a position near the text character string and the image data. And the image data in the vicinity of the character string matched by the comparison is assumed to correspond to the search. For this reason, if there is no text character string that explains or supplements image data, the user cannot search for desired image data, and thus a great amount of labor is required to create a link character string.

  The present invention has been made to solve the above-described problem, and is an information browsing apparatus, an information browsing program, and information that allow a user to easily create a link character string even in various forms of electronic documents. The purpose is to provide a browsing program recording medium.

(Point of invention)
The main points of the invention for solving the above-mentioned problems can be summarized as follows.
1. A character string selected from the electronic document displayed on the display device is searched from the same electronic document or another electronic document, and the corresponding data portion is displayed by searching the electronic document. Therefore, the user can save time and effort to specify data to be associated with the specific character string.
2. When displaying a character string associated with specific data, the associated specific data is also displayed. By automatically displaying the associated data, the user can easily understand the displayed electronic document.

A detailed description of the correspondence between the present invention and the action / effect will be given below.
The information browsing apparatus according to the present invention includes storage means for storing first electronic data including text data and second electronic data including data recognizable as characters, and a display for displaying at least the first electronic data. Display data creation means for creating data, display means for displaying the display data, input means capable of input operation, and a specific character in the first electronic data based on the input from the input means A character string input processing means for designating a string as a selected character string; a recognition means for recognizing characters from second electronic data; and a reference character string that matches the selected character string with respect to a recognition result by the recognition means; Search means for searching for a character string, and link data creation means for creating link data indicating an association between the selected character string and the data position of the reference character string It is characterized in that.

  The first electronic data and the second electronic data may be included in the same electronic document file or included in different electronic document files.

  As described above, since the information browsing apparatus includes the recognition unit, the search unit, and the link data creation unit, link data can be easily created even if the data type to be associated is other than text data. it can. For example, link data can be created even in a read-only document that cannot be rewritten by the user.

  In the information browsing apparatus, the second electronic data may be image data including a character, and the recognition unit may recognize the character from the image data.

  As described above, since the information browsing apparatus includes the recognition unit, the user can easily create link data even if the data type to be associated is image data.

  The information browsing apparatus includes a character string presence / absence determining unit that determines whether or not a reference character string indicated by the link data exists in the display data created by the display data creating unit; When it is determined by the character string presence / absence determining means that the reference character string exists, at least a part of the second electronic data including the reference character string is used as reference data together with the first electronic data. A display control means for controlling the display data creation means so as to be displayed on the display means may be further provided.

  According to the above configuration, since the character string presence / absence determination unit and the display control unit are provided, the user clicks on the link character string with a mouse or the like and jumps to the position of the reference data associated with the selected character string. It does not take.

  The information browsing apparatus may further include a display position setting unit that sets a display position of the reference data on the display unit.

  According to the above configuration, since the display position setting means is provided, the user can set the position on the screen for displaying the reference data according to his / her preference.

  In the information browsing apparatus, the display position setting unit sets an initial position of a region to be displayed on the display unit of the reference data when the partial region of the reference data is displayed on the display unit. Also good.

  According to the above configuration, since the display position setting means is provided, the initial position of the reference data displayed on the display means can be set, and the user needs to operate the device in order to see the desired reference data position. Absent.

  In the information browsing apparatus, the display control unit may include a selected character string in the first electronic data displayed on the display unit and a reference character string in the reference data associated with the selected character string. The display data generation means for the previous period may be controlled so that it can be distinguished from the data.

  According to the above configuration, since the display control means is provided, for example, the relationship between the character string of the text data and the character string of the chart data associated with the link character string of the text data can be grasped at a glance. good. Note that “displayed in a distinguishable manner from other data” means, for example, that the data is displayed in a different display form from the other data.

  The information browsing apparatus may further include a conversion unit that converts data recognizable as characters in the area of the reference data displayed on the display unit into text data.

  According to the above configuration, since the area recognizable as a character is limited to the area displayed on the display means of the data associated with the selected character string, the amount of calculation required for recognition can be reduced. Further, since the user can create link data while simultaneously viewing the text data and the character string of the chart data associated with the selected character string of the text data, it is easy to use.

  In the information browsing apparatus, the second electronic data may be voice data, and the recognition unit may be a voice recognition unit.

  According to the above configuration, since the recognition unit is provided, it is not necessary to search for voice data related to a word in the middle of browsing an electronic document.

  In the information browsing apparatus, the link data created by the link data creating unit may include time information indicating the location of data corresponding to the reference character string in the voice data.

  According to the above configuration, since the link data creating means is provided and time information indicating the position of the sound data corresponding to the reference character string is associated, it is possible to save troubles such as fast-forwarding of the sound and cueing.

  As described above, the information browsing apparatus of the present invention includes the storage means for storing the first electronic data including text data and the second electronic data including data recognizable as characters, and at least the first electronic data. Display data creation means for creating display data for displaying the display, display means, input means, and a specific character string in the first electronic data based on the input from the input means as a selected character string A character string input processing means to be designated, a recognition means for recognizing characters from the second electronic data, and a character string to be a reference character string that matches the selected character string is searched for the recognition result by the recognition means It is a configuration comprising search means and link data creation means for creating link data indicating the association between the selected character string and the data position of the reference character string.

  Thereby, even if the data type to be associated is other than text data, link data can be easily created. For example, link data can be created even in a read-only document that cannot be rewritten by the user.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

[Hardware configuration]
A hardware configuration according to the embodiment of the present invention will be described with reference to FIG. The information browsing apparatus in FIG. 2 includes a control device 1, an input device 2, a storage device 3, an external storage medium reading / writing device 4, a display device 5, and a communication device 6, and each unit is connected by a bus 7.

  The control device 1 is configured by a preprocessor, a memory, and the like, and operates according to a control program stored in the storage device 3, for example. More specifically, the processing results of the functional units described later are stored in the storage device 3, and data stored in the storage device 3 (hereinafter abbreviated as data in the storage device 3) are displayed on the display device 5 and the communication device 6. Output to.

  The input device 2 is composed of a keyboard having various keys such as cursor keys, a jog dial, a handwritten OCR, or a pointing device (for example, a mouse). A signal input by the input device 2 (for example, a key pressed on the keyboard) Signal) to the control device 1. Further, a microphone, an A / D converter, and a voice recognition engine may be separately provided and voice input may be used.

  The storage device 3 is constituted by a RAM or a ROM, and has a work area for storing input data, temporary data, and processing data.

  The storage device 3 stores an electronic document 3B composed of text, image data, sound data, or the like. The electronic document 3B may be described in, for example, an XML (extensible Markup Language) format. Of course, a plurality of these electronic documents can be stored.

  The storage device 3 also stores link data indicating the association between the data position of the character string and the data position of electronic data such as image data. The structure of this link data will be described later.

  The external storage medium read / write device 4 is composed of a reader / writer for an SD memory card, etc., and writes data stored in the storage device 3 to the external storage medium and reads data from the external storage medium to the storage device 3. I do.

  The display device 5 includes a CRT, a liquid crystal monitor, and the like, and displays data in the storage device 3.

  The communication device 6 is configured by a network interface card or the like, and transmits / receives data to / from a server device connected via a network cable or the like.

[Functional configuration based on block diagram]
Here, the functional configuration of the information browsing apparatus in the present embodiment will be described using the block diagram of FIG. The information browsing device includes an input device 2, a storage device 3, a display device 5, a control device 1, and a character string input processing unit 10, a search range specifying unit 11, and a data search unit configured by a program executed by the control device 1. 12, a candidate data display control unit 13, a link data creation unit 14, and an electronic document display control unit 15.

  The electronic document display control unit 15 interprets the format of the electronic document and creates display data for displaying it on the display device 5. Furthermore, it is determined whether or not there is data linked to other data in the display data, and the display data is changed based on the determination result. The processing of the electronic document display control unit 15 will be described in detail later.

  The character string input processing unit 10 stores the character string selected by the user using the input device 2 in the electronic document displayed on the display device 5 and the file position of the character string in the work area of the storage device 3. To do. In the following description, a character string input by the character string input processing unit 10 is simply referred to as an input selection character string.

  The search range designation unit 11 stores the electronic document 3B designated by the user using the input device 2 or a specific position of the electronic document 3B in the work area. The range specified by the search range specifying unit 11 is a search target of the data search unit 12. Of course, when the user has not selected anything, that is, when the search range specifying unit 11 has not specified anything, all the electronic documents 3B stored in the storage device 3 may be searched.

  The data search unit 12 searches for multimedia data such as image data, sound data, and moving image data including the same character string as the input selection character string within the range of the electronic document 3B specified by the search range specifying unit 11. The processing of the data search unit 12 will be described in detail later.

  The candidate data display control unit 13 creates a candidate list including the corresponding data by the search of the data search unit 12 and displays the candidate list on the display device 5.

  The link data creation unit 14 associates the candidate selected by the user using the input device 2 in the candidate list displayed on the display device 5 by the operation of the candidate data display control unit 13 with the input selection character string. . Hereinafter, a character string associated with the input selection character string is referred to as a link character string (reference character string), and data associated with the input selection character string is referred to as linked data. As shown in FIG. 10, the linked data includes at least information indicating the data position of the link character string. Then, the link data creation unit 14 stores link data indicating their association in the storage device 3. The processing of the link data creation unit 14 will be described in detail later.

  1 may be added as a client device as a server / client system, and the link data 3A and the electronic document 3B stored in the storage device 3 may be stored in the storage device of the server device. good.

[Operation example by flowchart]
(Outline of operation of information browsing device)
An outline of the operation of the information browsing apparatus will be described with reference to the flowchart of FIG. The operation example shown in the flowchart of FIG. 3 will be described with reference to FIG.

  In S1, the electronic document display control unit 15 creates display data for displaying the electronic document on the display device 5, and the display device 5 displays the display data. FIG. 4 shows a display example of S1. Details of the processing of S1 will be described later.

  In S <b> 2, the character input processing unit 10 stores the character string (input selection character string) selected by the user using the input device 2 and the file position where the character string exists in the work area of the storage device 3. In FIG. 4, the user selects the character string 890 “FIGURE 2”, and the character input processing unit 10 stores the character string and the character string position in the work area of the storage device 3.

  In S3, the search range designation unit 11 stores the electronic document 3B designated by the user using the input device 2 or a specific range of the electronic document 3B in the work area. The range stored in the work area by the search range specifying unit 11, that is, the range specified by the search range specifying unit 11 is a search target of the data search unit 12. Note that the electronic document designated by the user in S3 may be a file other than the file displayed on the display device 5. For example, it may be a file stored in a different information browsing device.

  In S4, the data search unit 12 searches for electronic data related to the input selection character string stored in S2 from the electronic document range specified in S3 (the range specified by the search range specifying unit 11). Details of the process of S4 will be described later.

  In S5, the candidate data display control unit 13 creates display data for displaying the search result corresponding to the search process in S4. At this time, if there are a plurality of search results, a display prompting the user to select the search results may be displayed, and the search results may be determined based on the user's input.

  In S6, the link data creation unit 14 creates link data indicating the association between the input selection character string stored in S2 and the electronic data corresponding to any of the search results in S5. Details of the process of S6 will be described later.

(Processing of the data search unit 12)
First, the functional configuration of the data search unit 12 will be described in detail with reference to FIG. The data search unit 12 includes an object data detection unit 120, a text search unit 121, a character recognition unit 122, a voice recognition unit 123, and a moving image processing unit 124.

  The object data detection unit 120 detects text data, image data, sound data, and moving image data that exist in the electronic document 3B range designated by the search range designation unit 11.

  When the object data detection unit 120 detects the text data, the text search unit 121 searches the detected text data for a character string that matches the input selected character string, and uses the character string corresponding to the search as a search result. Save in the work area of the storage device 3.

  When the object data detection unit 120 detects image data, the character recognition unit 122 performs character recognition of the detected image data, and the character recognition unit 122 further matches the input selected character string in the recognition result. The column is searched, and the character string corresponding to the search is stored in the work area of the storage device 3 as a search result.

  When the object data detection unit 120 detects sound data, the voice recognition unit 123 performs voice recognition of the detected sound data, and the voice recognition unit 123 further matches the input selected character string in the recognition result. The column is searched, and the character string corresponding to the search is stored in the work area of the storage device 3 as a search result.

  When the object data detection unit 120 detects moving image data, the moving image processing unit 124 performs character recognition for each still image data of the moving image data, and further performs voice recognition for sound data of the moving image data. Then, the moving image processing unit 124 searches whether there is a match with the input selected character string in each recognition result as in the above process, and uses the character string corresponding to the search as a search result. Save to area.

  Next, processing (corresponding to S4 in FIG. 3) of the data search unit 12 will be described using the flowchart in FIG. Here, as shown in FIG. 7, when the user selects the character string 900 “FIGURE 2” in the electronic document displayed on the display device 5 using the input device 2 (that is, “FIGURE 2” is the input selection character string). And the case where the search range designation unit 11 designates the electronic document range 905 in FIG. 8 as a result of designation by the user using the input device 2 will be supplemented. The case where the electronic document range 905B described in the image data as shown in FIG. 9 is designated by the search range designation unit 11 will also be described. Note that the range specified by the search range specifying unit 11 does not have to be the same as that of the electronic document in FIG. 7, and may be a different file.

  In S20, text data exists in a partial area (hereinafter referred to as a search focus area), which is one of areas obtained by subdividing the range designated by the search range designation unit 11 in the object data detection unit 120. It is determined whether or not to do. If it is determined in S20 that text data exists in the search focus area, the process proceeds to S21. Otherwise, the process proceeds to S30.

  In S30, the object data detection unit 120 determines whether image data exists in the search focus area. If it is determined in S30 that image data exists in the search focus area, the process proceeds to S31. Otherwise, the process proceeds to S40.

  In S40, the object data detection unit 120 determines whether sound data exists in the search focus area. If it is determined in S40 that sound data exists in the search focus area, the process proceeds to S41. Otherwise, the process proceeds to S50.

  In S50, the object data detection unit 120 determines whether moving image data exists in the search focus area. If it is determined in S50 that moving image data exists in the search focus area, the process proceeds to S51. Otherwise, the process proceeds to S60.

  In S60, the object data detection unit 120 updates the area to be determined in S20, S30, S40, and S50, that is, the search focus area, and advances the search focus area to the next. For example, if the corresponding data does not exist in S20, S30, S40, and S50, the search focus area is set as one line of the electronic document, and the search focus area is advanced by one line, and the process proceeds to S20, S30, S40, or S50. If the corresponding data exists, the data amount, that is, the corresponding text data, image data, sound data, and moving image data amount may be advanced.

  In S61, it is determined whether or not the input selection character has been searched for the entire range specified by the search range specifying unit 11. If the entire range has been searched, the processing in FIG. 6 is terminated. Otherwise, the process proceeds to S20.

  In S21, the text search unit 121 determines whether the text data detected in S20 is the same as the character string (input selection character string) input by the character string input processing unit 10. If there is the same character string (reference character string), the process proceeds to S22. Otherwise, the process proceeds to S30.

  In the case of FIG. 8, text data 909, 910, and 911 are detected in S20, and it is determined in S21 that text data 910 having the same character string as the input selection character string 900 “Figure 2” in FIG. 7 exists. Accordingly, in this case, the process proceeds to S22.

  In S22, the text search unit 121 creates linked data including information on the data position corresponding to the input selected character string from the text data detected in S20, and stores it in the work area of the storage device 3. To do.

Here, the data structure of the linked data will be described with reference to FIG.
The data structure 960 of linked data stored in the work area includes “file name 961 of linked data” that is the name of the file that includes the corresponding image data, and “linked data” that is the file position of the corresponding data in the search. “Data file position 962”, coordinate position “matching character string position 963” of the character string matched with the input selected character string, “matching degree” indicating the degree of correlation with the input character string (input selected character string) 964 ”and“ data type 965 ”indicating the type of data corresponding to the search. However, as described later, the meaning of “matching character string position 963” differs depending on whether the “data type 965” is image data or sound data.

  In the process of S22, since there is text data that matches the input selection character string, the “matching degree 964” is set to a high value (for example, 100). Further, as “matching character string position 963”, the coordinate position where the matched text exists in S10 is stored. Further, since “data type 965” is text, it becomes “text data”.

  As described above, the link data indicating the association between the data position of the character string and the data position of the electronic data such as the image data is data independent of the electronic document. Therefore, the link data can be used regardless of whether the electronic document is read-only or not. A character string and a specific electronic data position can be associated with each other according to the instructions of the person.

  In S31, the character recognition unit 122 performs character recognition on the image data detected in S30.

  In S32, the character recognition unit 122 determines whether the image data includes a character string (reference character string) that matches the input selection character string. If there is a matching character string, the process proceeds to S33. Otherwise, the process proceeds to S40.

  In S33, as in S22, the character recognition unit 122 assumes that the image data detected in S30 corresponds to the search, creates linked data as a search result, and stores it in the work area of the storage device 3. If there are a plurality of matching character strings in one image, linked data is created for all of them and stored in the work area.

  However, unlike the case of S22, the “matching degree 964” in the data structure of the linked data candidate stores a normalized matching score for character recognition. For example, normalization may be performed between 0 and 100. The “matching character string position 963” is a position where the input selection character string is described in the corresponding image data, and the “data type 965” is “image data”.

  In the range specified by the search range specifying unit 11 by the above processing, the input selected character is also used when, for example, image data and text data are not separated as shown in FIG. 9 and a character string is stored as image data. You can search columns. In FIG. 9, the character string 910B is stored as image data instead of text data. As will be described later, by correctly storing “matching character string position 963”, the electronic document display control unit 15 can display the contents of the electronic document in an easy-to-understand manner to the user.

  In S41, the voice recognition unit 123 performs voice recognition of the sound data detected in S40.

  In S42, the speech recognition unit 123 determines whether or not the speech recognition result in S41 includes the same character string (reference character string) as the input selected character string. If there is the same character string, the process proceeds to S43. Otherwise, the process proceeds to S50.

  In S43, the voice recognition unit 123 assumes that the sound data detected in S40 corresponds to the search, and stores it in the work area as a candidate for linked data. Of course, when there are a plurality of character strings that match the input selection character string from the speech recognition result of one sound data (when the same word is spoken many times), all of them are stored as candidates for linked data. Store in the work area of the device 3.

  In the process of S43, the “data type 965” of the linked data is “sound data”. “Matching level 964” stores a matching score of normalized speech recognition. For example, normalization may be performed between 0 and 100. The “matching character string position 963” is the time when the same word as the input selected character string in the corresponding sound data is uttered.

  For example, in the range specified by the search range specifying unit 11, if there is audio data recording a Japanese language class explaining the meaning of a plurality of words, the specific time of the word and the class audio as described above , The user does not have to search for the voice position to find where in the class the word was explained.

  In S51, the moving image processing unit 124 performs character recognition on each image data of the image sequence of the moving image data detected in S50.

  In S52, the moving image processing unit 124 determines whether each of the image data includes a character string (reference character string) that matches the input selection character string. If there is a matching character string, the process proceeds to S53. Otherwise, the process proceeds to S54.

  In S53, the moving image processing unit 124 assumes that the moving image data detected in S50 corresponds to the search, creates linked data as a search result, and stores it in the work area of the storage device 3. Further, when there are a plurality of still images with matching input selection character strings in the moving image still image sequence, linked data is created for all of them and stored in the work area of the storage device 3. Similarly, when there are a plurality of matched character strings in one image, linked data is created for all of them and stored in the work area of the storage device 3.

  In the case of S53, the “data type 965” of the linked data is “moving image data”. “Matching level 964” stores a matching score of normalized character recognition. For example, normalization may be performed between 0 and 100. The “matching character string position 963” is a time in the moving image data of the corresponding still image data and a coordinate position in which the input selection character string in the corresponding still image is described.

  In S54, the moving image processing unit 124 performs voice recognition on the sound data of the moving image data detected in S50.

  In S55, the moving image processing unit 124 determines whether or not there is the same character string as the input selected character string in the voice recognition result in S54. If there is the same character string, the process proceeds to S56. Otherwise, the process proceeds to S60.

  In S <b> 56, the moving image processing unit 124 assumes that the moving image data detected in S <b> 50 corresponds to the search, creates linked data as a search result, and stores it in the work area of the storage device 3. Of course, when there are multiple character strings that match the input selection character string from the voice recognition result of one video data (when the same word is spoken many times), the linked data is created for all of them, Store in the work area of the storage device 3.

  In the case of S56, the “data type 965” of the linked data is “moving sound data”. “Matching level 964” stores a matching score of normalized speech recognition. For example, normalization may be performed between 0 and 100. The “matching character string position 963” is the time when the same word as the input selected character string in the corresponding sound data is uttered.

  For example, when there is moving image data that explains the meaning of multiple words using animation, the specific word and the specific coordinate position of the specific image of the image sequence are associated as described above, By associating with a specific time, the user does not have to search for a moving image position in order to find out where (at which time) the word was explained in the class.

  If it is determined in S61 that the search has been completed, the candidate data display control unit 13 indicates that the linked data candidates stored in S22, S33, S43, S53, and S56 on the display device 5 have a high degree of coincidence 964. Output in order. A candidate with a higher degree of matching 964 is a desired candidate for the user, and therefore the user does not have to take time to select the desired candidate.

(Process of link data creation unit 14)
Here, the processing of the link data creation unit 14 (corresponding to S6 in FIG. 3) will be described in detail.

  The link data creation unit 14 associates the linked data selected by the user using the input device 2 in the linked data list displayed by the candidate data display control unit 13 with the input selection character string. Here, the data structure of the link data representing the association will be described in detail with reference to FIG.

  The link data 3A includes the contents of the input selection character string “character string content 951”, “character string file position 952” indicating the file position of the input selection character string, and “link data to be linked” selected by the user. 960 ". Of these, the linked data 960 has the data structure described above. The link data indicates that the character string of the character string contents 951 at the character string file position 952 is associated with the linked data 960.

  It should be noted that after the user selects the character string 900 “FIGURE 2” of the electronic document in FIG. 7 using the input device 2 (that is, “FIGURE 2” is the input selection character string), the link data creation unit 14 selects the input selection character string. When associating 900 with certain linked data, the same linked data as the character string 900 may be automatically associated with the character string 901 “FIGURE 2”. Specifically, a process of creating a plurality of link data 3A at the same time is performed. Furthermore, the linked data may be associated with all of the character string “Figure 2” existing in the area not displayed on the electronic document display device 5. Thus, by associating the same character string at the same time, the electronic document display control unit 15 can display the contents of the electronic document in an easy-to-understand manner to the user.

(Processing of electronic document display control unit 15)
First, the functional configuration of the electronic document display control unit 15 will be described in detail with reference to FIG. The electronic document display control unit 15 includes a display data creation unit 150, a link character string detection unit 151, a default position setting unit 152, an identification display instruction processing unit 153, a partial link data creation unit 154, and a display data update unit 155. Is done.

  The display data creation unit 150 interprets the format of the electronic document and creates display data for displaying it on the display device 5.

  The link character string detection unit 151 determines whether or not a link character string indicating a link to other data exists in the display data created by the display data creation unit 150.

  The default position setting unit 152 determines from which position of the linked data to display based on an instruction given by the user using the input device 2. This process will be described later.

  The identification display instruction processing unit 153 displays the selected character string selected by the user using the input device 2 and the character string in the linked data associated with the selected character string so as to be distinguished from other data. . This process will be described later.

  The partial link data creation unit 154 searches for multimedia data such as image data, sound data, and moving image data including the same character string as the input selected character string in the area of the display device 5 where the linked data is displayed. The processing of the partial link data creation unit 154 is almost the same as that of the data search unit 12, but the search target area is different.

  The display data update unit 155 updates the display data based on instructions from the link character string detection unit 151 and the identification display instruction processing unit 153.

  Next, processing of the electronic document display control unit 15 will be described using the flowchart of FIG. Here, it is assumed that link data indicating the association between the character string 900 “Figure 2” in the display data displayed on the display device 5 and the image data 907 in FIG. 8 is created as shown in FIG. Supplement the explanation.

  In S100, the display data creation unit 150 creates display data for outputting the electronic document 3B in the storage device 3 to the display device 5.

  In S101, the link character string detection unit 151 determines whether or not there is a link character string, that is, a character string indicated in the character string content 951 of the link data in the display data area created in S100. If a link character string exists, the process proceeds to S103. If not, the process proceeds to S102, and the display data created in S100 is output to the display device 5.

  In S103, the link character string detection unit 151 acquires link data corresponding to the link character string from the storage device 3.

  In S104, the display layout is updated to present the link data obtained in S103 to the user. Here, the process of S104 will be described in detail with reference to the flowchart of FIG.

  In S201, the display data update unit 155 changes the display layout in order to display the linked data 960 of the link data acquired in S103. In S202, the display layout changed in S201 is output to the display device 5.

  FIG. 14 shows a display example of S202 when the data type 965 of the linked data 960 is “text data”. In FIG. 14, the display layout of only text data as shown in FIG. 7 is changed to a display layout that displays the linked data associated with “Figure 2” in the area 970 (hereinafter referred to as 970 in FIG. 14). Such an area where linked data to be linked is displayed is called a linked display area, and an area other than the linked display area 970 is called a text display area 972.

  Here, the text data in the text display area 972 in FIG. 14 is displayed so as not to overlap the linked display area 970 (hereinafter, such display is referred to as wraparound display). By displaying in this way, it is possible to save the trouble of switching the display / non-display of the linked data of the user. Further, the linked display area 970 can be scrolled by a scroll bar 971, and the user can easily refer to desired information. Further, the matching character string position 963 of the linked data, that is, the position of “Figure 2” is displayed so as to come to the center of the linked display area 970. Although this display position will be described below, it can be changed.

  It is not necessary to bring the linked display area 970 to the lower right of the display screen as shown in FIG. For example, the linked display area 970 may be positioned in the line immediately below the link character string as shown in FIG. If the display is as shown in FIG. 15, the position of the link character string and the data related to the link character string is close, so the reader can save the trouble of moving the line of sight.

  Further, the default position setting unit 152 may arbitrarily set the position of the linked display area 970 based on the input of the user input device 2. Since user preferences can be reflected, it is convenient for users.

  Further, even when the data type 965 of the linked data 960 is “image data”, the processing is the same as that described above in which the data type 965 of the linked data 960 is “text data”.

  FIG. 16 shows a display example of S202 when the data type 965 of the linked data 960 is “sound data”. In FIG. 16, two linked data whose data type 965 is “sound data” are associated with the character string “solar system” 980. However, it is assumed that the two linked data indicate different times of the same audio file. That is, the two linked data have the same file name 961 and file position 962, and different matching character string positions 963. In FIG. 16, playback buttons (981 and 982) that can be played from the specified time are displayed for one piece of sound data. These times are the times indicated by the matching character string position 963 of each linked data. When the user clicks the play button 981 or 982, sound data is played from the corresponding time.

As described above, by displaying the playback button that can be played back from the voice data position where the link character string is spoken, it is possible to save time and trouble such as fast-forwarding of the voice and cueing. That is, the user does not have to bother to search for the audio of the explanation of a certain word. FIG. 17 shows an example of display in S202 when the data type 965 of the linked data 960 is a plurality of “moving image data”. Show. The moving image stream data is displayed in the area 992 in FIG. All linked data is associated with the character string “solar system” 980. When the user clicks the play button 990, the moving image is played from the image position where the character string “solar system” exists.

  In FIG. 17, the file name 961 and the file position 962 are the same in all linked data when the data type 965 is “moving image data”. That is, a plurality of link data indicates the same moving image data as the linked data, and further indicates the continuous time of the moving image data. In this situation, as indicated by 991 (displayed with diagonal lines), the section corresponding to the linked data is displayed so that it can be identified. When the data type 965 is a plurality of “moving sound data”, the same processing as that when the data type 965 is a plurality of “sound data” may be performed.

  By displaying the playback button 990 that can be played back from the image position where the link character string is displayed as described above, for example, it is possible to save troubles such as fast-forwarding of the moving image and heading. In addition, as indicated by 991 in FIG. 17, by displaying so that the section in which the link character string in the moving image data is displayed can be identified to the user, it is easy for the user to know which section has an associated video. .

  When there are a plurality of link character strings in the display data area created in S100, all of the linked data associated with each link character string may be displayed at a time. However, it is desirable to display only the linked data related to the link character string located at the top of the display screen. This is because if too much linked data is displayed at once, it is difficult for the reader to view the electronic document.

  In S <b> 203, the default position setting unit 152 determines whether or not the user has instructed to scroll the position of the linked data displayed in the linked display area 970. If it is determined that the user has given such an instruction, the process proceeds to S204. Otherwise, the process proceeds to S206.

  In S204, the default position setting unit 152 sets the default position of the linked data to be displayed in the linked display area 970 designated by the user. Then, the matching character string position 963 of the linked data that is the target of S204 is replaced with the newly designated default position. For example, in the case of FIG. 14, the user can specify the default position by switching the screen using the scroll bar 971 in the linked display area 970 and then pressing the position setting button or the like of the input device 2. You can do that.

  Here, with reference to FIG. 18A to FIG. 18C, the specific processing of S203 and S204 and the effects thereof will be described.

  In FIG. 18, it is assumed that the character “Large” 995A included in the image of (A) is associated with the character string in the text. At that time, as shown in (B), when a part of the image of (A) is displayed in the referenced data area 970, the user must scroll to view the desired “large” character 995B. . Thus, for example, as shown in (C), the user scrolls the image once so that the “Large” character 995C is in the center, and presses a position setting button or the like so that the default position can be designated. Then, from the next time, the user will not have the trouble of scrolling every time to see the “Large” character.

  Further, the enlargement / reduction ratio of the image size displayed as a default may be set by the same operation as described above. However, in that case, the linked data 960 needs to have a data item of a default enlargement ratio. If the default enlargement / reduction ratio can be set, a user-friendly display is possible regardless of the original size of the image data.

  In S206, the identification display instruction processing unit 153 determines whether or not there is an instruction to identify and display a specific character from the user. If it is determined that the user has instructed specific character identification display, the process proceeds to S207. Otherwise, the process proceeds to S209.

  In S207, the identification display instruction processing unit 153 detects the character string displayed in the linked display area 970. At this time, character recognition is also performed on the image data to detect a character string. It is not necessary to limit the area for detecting the character string to the portion displayed in the linked display area 970. However, the larger the target area for character string detection, the greater the amount of calculation for character string detection.

  In S208, if any character string in the text display area 972 matches the character string detected in S207, the display data update unit 155 can identify the matched character string and other character strings. The identification display is performed as follows.

  FIG. 19 shows one example. In FIG. 19, the character strings “923” and “921” displayed in the linked display area 970 are detected, and the detected character strings are in the text display area 972. Has been. Note that FIG. 19 is an example, and the identification display may be performed by changing the background color of the character string, underlining the character string, or changing the font type of the character string. As described above, when there are the same character strings in the text display area 972 and the linked display area 970, the user can easily grasp the contents of the electronic document by identifying and displaying them.

  In S209, it is determined whether or not a user has instructed creation of partial link data. Here, the partial link data is link data indicating the association between the data displayed in the linked display area 970 and the data in the text display area 972, and is different from the link data described above. That is, partial link data is created based on a search from a limited area, and is therefore distinguished from normal link data. If it is determined that an instruction to create partial link data has been issued by the user, the process proceeds to S210. Otherwise, the process proceeds to S213.

  In S210, the character string selected by the user in the text display area 972 is input.

  In S211, the partial link data creation unit 154 searches for the character string input in S210 limited to the displayed linked display area 970. Then, if there is a corresponding character string by the search, it is displayed on the display device 5 so that the user can select the corresponding result using the input device 2. As described above, since the area to be searched is limited to the displayed linked display area 970, the amount of calculation required for the search can be reduced. In addition, since the user can create a link character string while simultaneously viewing the text data and the character string of the chart data associated with the link character string of the text data, it is easy to use. However, it is not necessary to completely limit the search range of the character string to the displayed linked display area 970, and a portion that is not displayed may be searched. However, if the search range of the character string is limited to the portion displayed in the linked display area 970, the excessive calculation amount can be reduced.

  In S212, the partial link data creation unit 154 creates partial link data indicating that the user selects the corresponding character string in S211 and the character string selected in S210.

  These processes are the same as those performed by the data search unit 12, but the user can associate the character string displayed in the linked display area 970 with reference to the text in the text display area 972. It is easier to use.

  In S213, it is determined whether or not the user has instructed to scroll the text display area. If there is a scroll instruction, the process of the flowchart of FIG.

[Examples in software]
An object of the present invention is to supply a recording medium that records a program code of software that realizes the functions of the above-described embodiments to another system or apparatus, and the computer CPU of the system or apparatus is stored in the recording medium. Needless to say, this can also be achieved by reading and executing the program code.

  In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

  As a recording medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, a nonvolatile memory card, and the like can be used.

  The program code may be downloaded from another computer system to a recording device or the like via a transmission medium such as a communication network.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU of the expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  As described above, due to the effect of the data search unit 12 of the present invention, for example, certain text data and image data and text data are not separated as shown in FIG. 9, and character strings are also stored as image data. Even when the existing data are related, the two data can be easily related.

  In addition, since the link data of the present invention has a data structure independent of the electronic document, the user can associate two data regardless of whether the document is read-only or not.

  When link data exists in the display data displayed on the electronic document display device, for example, the corresponding linked data is displayed as shown in FIGS. Therefore, the user does not need to click the link character string with a mouse or the like to jump to the corresponding image data position. In addition, since the reader can refer to the linked data while looking at the text data, the electronic document can be easily grasped.

  Further, as described above, the character string and the specific time of the sound data are associated with each other, and if there is a link character string associated with the specific time of the sound data, a reproduction button for reproducing the sound data position is displayed. Therefore, it is not necessary to search for voice data related to a word in the middle of browsing an electronic document. In addition, since the character string and the specific time of the sound data are associated with each other, it is possible to save troubles such as fast-forwarding of voice and cueing.

  When the present invention is applied to the recording medium, the recording medium stores program codes corresponding to the flowcharts described above.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The present invention can be applied to an apparatus for browsing an electronic document including multimedia data such as an electronic book.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates an embodiment of the present invention, and is a block diagram illustrating a functional configuration of an information browsing apparatus according to the present invention. It is a block diagram which shows the hardware constitutions of the information browsing apparatus shown in FIG. It is a flowchart explaining the outline | summary of the processing operation of the information browsing apparatus shown in FIG. FIG. 2 is an explanatory diagram illustrating an example of an electronic document displayed on the display device illustrated in FIG. 1. It is a block diagram which shows the detailed function structure of the data search part of FIG. It is a flowchart explaining the processing operation of the data search part of FIG. FIG. 2 is an explanatory diagram illustrating an example of an electronic document displayed on the display device illustrated in FIG. 1. It is explanatory drawing which shows an example of the designation | designated operation | movement of the electronic document by the search range designation | designated part shown in FIG. It is explanatory drawing which shows the other example of the designation | designated operation | movement of the electronic document by the search range designation | designated part shown in FIG. It is explanatory drawing which shows the structure of the link data which the link data preparation part shown in FIG. 1 produces. It is a block diagram which shows the detailed functional structure of the electronic document display control part shown in FIG. 4 is a flowchart for explaining a processing operation of the electronic document display control unit shown in FIG. 1. 13 is a flowchart for explaining in detail the processing of S104 in the electronic document display control unit shown in FIG. FIG. 13 is a diagram illustrating a display example on the display device based on the operation of S202 in the electronic document display control unit illustrated in FIG. 13, and is an explanatory diagram illustrating a state in which linked data of text data is displayed together with the electronic document of the body text It is. FIG. 15 is an explanatory diagram illustrating another example of the display example illustrated in FIG. 14. FIG. 13 shows still another display example on the display device based on the operation of S202 in the electronic document display control unit shown in FIG. 13, and the linked data of the sound data is displayed on the display device together with the electronic document of the main text. It is explanatory drawing which shows the state. FIG. 14 shows still another display example on the display device based on the operation of S202 in the electronic document display control unit shown in FIG. 13, and the linked data of the moving image data is displayed on the display device together with the electronic document of the text. It is explanatory drawing which shows the state. FIG. 18A explains a specific operation example of S203 and S204 in the default position setting unit shown in FIG. 13, and the character “Large” included in the image is converted into a character string in the text. FIG. 18B is an explanatory diagram showing a state of being associated, FIG. 18B is an explanatory diagram showing a state in which a part of the image shown in FIG. 18A is displayed in the referenced data area, and FIG. It is explanatory drawing which shows the state by which the image was scrolled so that the character of "large" might come to the center by the user. It is explanatory drawing which shows the example of a display with a display apparatus based on operation | movement of S208 in an electronic document display control part shown in FIG. It is a figure for demonstrating the prior art (1st prior art) which links | relates reference body data and referenced body data. It is a block diagram for demonstrating the prior art (2nd prior art) which links | relates reference body data and to-be-referenced body data.

Explanation of symbols

1 Control device 2 Input device (input means)
3. Storage device (storage means)
4 External storage medium read / write device 5 Display device (display means)
6 Communication device 7 Bus 10 Character string input processing section (character string input processing means)
11 Search range specification part 12 Data search part (recognition means, search means)
13 candidate data display control unit 14 link data creation unit (link data creation means)
15 electronic document display control unit 120 object data detection unit 121 text search unit 122 character recognition unit 123 voice recognition unit 124 video processing unit 150 display data creation unit (display data creation means)
151 Link character string detector (selected character string presence / absence determining means)
152 Default position setting section (display position setting means)
153 Identification display instruction processing unit 154 Partial link data creation unit (conversion means)
155 Display data update unit (display control means)

Claims (11)

Storage means for storing first electronic data including text data and second electronic data including data recognizable as characters;
Display data creating means for creating display data for displaying at least first electronic data;
Display means for displaying the display data;
Input means capable of input operation;
A character string input processing means for designating a specific character string in the first electronic data as a selected character string based on an input from the input means;
Recognition means for recognizing characters from the second electronic data;
Search means for searching for a character string to be a reference character string that matches the selected character string with respect to a recognition result by the recognition means;
An information browsing apparatus comprising link data creating means for creating link data indicating an association between the selected character string and a data position of the reference character string.   The information browsing apparatus according to claim 1, wherein the second electronic data is image data including characters, and the recognition unit recognizes characters from the image data. Character string presence / absence determining means for determining whether or not the reference character string indicated by the link data is present in the display data created by the display data creating means;
When it is determined by the character string presence / absence determining means that the reference character string exists, at least a part of the second electronic data including the reference character string is used as reference data together with the first electronic data. The information browsing apparatus according to claim 2, further comprising display control means for controlling display data creation means so as to be displayed on the display means.   The information browsing apparatus according to claim 3, further comprising display position setting means for setting a display position of the reference data in the display means.   5. The display position setting means sets an initial position of an area of the reference data to be displayed on the display means when displaying a partial area of the reference data on the display means. Information browsing device described in 1.   The display control unit can identify a selected character string in the first electronic data displayed on the display unit and a reference character string in the reference data associated with the selected character string from other data. 4. The information browsing apparatus according to claim 3, wherein the display data creation means for the previous period is controlled so as to be displayed.   4. The information browsing apparatus according to claim 3, further comprising conversion means for converting data recognizable as characters in the area of the reference data displayed on the display means into text data.   The information browsing apparatus according to claim 1, wherein the second electronic data is voice data, and the recognition unit is a voice recognition unit.   9. The information according to claim 8, wherein the link data created by the link data creation means includes time information indicating a location of data corresponding to the reference character string in the voice data. Browsing device.   An information browsing program for operating the information browsing apparatus according to any one of claims 1 to 9, wherein the information browsing program causes a computer to function as each of the above means.   A computer-readable program recording medium on which the information browsing program according to claim 10 is recorded.

Images