JP2010044767A - Document processor, document processing method, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To present information by outputting video data related to a specified part on document data. <P>SOLUTION: A document processor includes an input means for inputting first electronic document data; a presentation output control means for performing output control of presentation information for presenting contents of the first electronic document data; a specifying means for specifying a part of second electronic document data out of the second electronic document data controlled in presentation output by the presentation output control means; a video or still image selecting means for selecting one or more video data or still image data corresponding to a part of the first electronic document data input based on the input means and the second electronic document data specified by the specifying means; and a video or still image output control means for sequentially performing output control of a plurality of the video data or still image data selected by the video or still image selecting means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to electronic document processing, and more particularly to a document processing apparatus capable of outputting video data related to electronic document data.

  2. Description of the Related Art Conventionally, on the Internet, WWW (World Wide Web) is provided as an application service that provides hypertext type information in a window format.

  The WWW is a system that performs document processing for creating, publishing, or sharing a document, and shows a new style of document. However, from the viewpoint of practical use of documents, advanced document processing exceeding WWW, such as document classification and summarization based on document contents, is required. For such advanced document processing, mechanical processing of document contents is indispensable.

However, mechanical processing of document contents is still difficult for the following reasons.
First, HTML (Hyper Text Markup Language), which is a language for describing hypertext, defines the expression of a document but hardly specifies the contents of the document. Secondly, a hypertext network formed between documents is not always easy for a reader of a document to understand the contents of the document. Third, in general, the author of the text writes without regard to the convenience of the reader, but the convenience of the reader of the document is not coordinated with the convenience of the author.

As described above, the WWW is a system that indicates a new document. However, since the document is not mechanically processed, advanced document processing cannot be performed. In other words, in order to perform advanced document processing, it is necessary to mechanically process the document.

  Therefore, with the goal of mechanical processing of documents, a system that supports mechanical processing of documents has been developed based on the results of natural language research. As document processing by natural language research, mechanical document processing that uses the tag attached to the document on the premise of the attribute information on the internal structure of the document by the author of the document, so-called tag assignment, has been proposed. Yes.

By the way, with the spread of computers in recent years and the progress of networking, it is necessary to increase the functionality of document processing that creates, labels, and changes text documents with text processing and indexing depending on the contents of the document. It has been. For example, document summarization or document classification according to the user's request is desired.
In addition, processing suitable for the user, such as a display style suitable for the form of the document viewer and display unit that displays the document and its summary, a reading function that is performed along with the display, and the linkage between the display of the document and the image and reading are also possible. It has been demanded.
Furthermore, if video data such as moving images and still images can be output in relation to document data, it is expected that users will be able to provide more diverse and sophisticated information and more accurate information. Therefore, in addition to various document processing, it is desired to link with video data.

  The present invention has been proposed in view of such circumstances, and enables video data to be output in correspondence with document data presented in a document processing apparatus, particularly for a portion designated by a user. The purpose is to be able to present accurate, diverse, and advanced information to the user by outputting video data.

For this reason, the document processing apparatus of the present invention includes an input unit for inputting first electronic document data, a presentation output control unit for controlling output of presentation information for presenting the contents of the first electronic document data, and the presentation output. Of the second electronic document data presented and controlled by the control means, a designation means for designating a part of the second electronic document data , and the first input based on the input means the electronic document data and video, or still image selection means for selecting one or more video data or still image data corresponding to a part of the specified second electronic document data by said designating means, said video Alternatively, a video or still image output control unit that sequentially outputs and controls a plurality of video data or still image data selected by the still image selection unit.

When the video or still image selection means selects one or more video data or still image data corresponding to the first electronic document data input based on the input means, the first electronic Video data or still image data relating to a portion corresponding to the content of the document data is selected.

Further, the video or still image output control means controls whether the second electronic document data is output on an area where the second electronic document data is presented or output, or whether the second video or still picture is presented and output on an area where only the second video or still image is displayed. Like that.
Further, the presentation output control means presents a list of a plurality of electronic document data .

The document processing method of the present invention includes an input procedure for inputting first electronic document data, a presentation output control procedure for controlling the presentation information for presenting the contents of the first electronic document data, and the presentation output control procedure. A designation procedure for designating a part of the second electronic document data from the second electronic document data presented and output by the first electronic document inputted based on the input means a video or still image selection procedure to select one or more video data or still image data corresponding to a part of the specified second electronic document data in the data and the specified procedure, the video or still image A video or still image output control procedure for sequentially outputting a plurality of video data or still image data selected in the selection procedure is performed.

  The recording medium of the present invention records an operation control program having the above procedures.

As can be seen from the above description, the present invention provides the following effects.
That is, according to the document processing apparatus and the document processing method of the present invention, one or more corresponding to a specified portion is specified by specifying a part of the presented contents of one or more electronic document data. Video data is output controlled so that, along with the presentation of the document, a specified part, that is, a video related to an entire document or a part of the document content can be provided to the user. It is possible to provide accurate, diverse, and advanced information.
In addition, as video data, appropriate information can be presented to the user by appropriately using a moving image with a large amount of information that can be provided and a still image that can accurately convey an image.

In addition, by presenting the text or summary of one or more electronic document data, it is possible to specify a part of the presented document, or to specify a part with a keyword. Can be specified in various ways. For example, it can be specified in units of sentences, clauses, characters, etc.
Then, one or a plurality of video data is selected and output based on the video designation information (video tag) added corresponding to a part of the document designated as such, so that the part that the user wants to see Only video data can be output, and video output that can respond precisely to user requests can be realized.

In addition, by presenting title information of one or a plurality of electronic document data (or video data) so that a part of the title information can be designated, designation of electronic document data unit or designation of video data itself can be performed. Easy to implement.
Since video data related to one or a plurality of titles (that is, electronic document data or video data) designated in such a manner is output, the contents of each electronic document data and video data can be appropriately and appropriately given to the user. It can be provided easily.

  Furthermore, according to the recording medium of the present invention, a program for realizing the document processing method of the present invention can be provided. For example, the document processing apparatus of the present invention can be easily realized by using a general-purpose personal computer or the like. . Thereby, a general user can enjoy the said effect easily.

It is a block diagram of the document processing apparatus of embodiment of this invention. It is explanatory drawing of the document structure used by embodiment. It is explanatory drawing of the window which displays the sentence structure of embodiment. It is a flowchart of the manual classification process of embodiment. It is a flowchart of the index creation process of an embodiment. It is explanatory drawing of the active value of the element of embodiment. It is a flowchart of the active diffusion process of embodiment. It is a flowchart of the center active value update process of embodiment. It is explanatory drawing of the classification | category window of embodiment. It is explanatory drawing of the browsing window of embodiment. It is explanatory drawing of the classification model of embodiment. It is a flowchart of the automatic classification | category process of embodiment. It is a flowchart of automatic classification of an embodiment. It is a flowchart of the relevance calculation process between meanings of an embodiment. It is explanatory drawing of the relevance degree between meanings of embodiment. It is a flowchart of the document reading process of an embodiment. It is a flowchart of the file generation process for reading of embodiment. It is explanatory drawing of the tag file example of embodiment. It is explanatory drawing of the tag file example of embodiment. It is explanatory drawing of the example of the file for reading of embodiment. It is explanatory drawing of the example of the file for reading of embodiment. It is explanatory drawing of the reading-out window of embodiment. It is a flowchart of the summary preparation process of embodiment. It is a flowchart of the summary sentence display process of embodiment. It is a flowchart of the telop display process of an embodiment. It is explanatory drawing of the example of a display of the browsing window of embodiment. It is explanatory drawing of the example of a display containing the summary sentence of the browsing window of embodiment. It is explanatory drawing of the example of a telop display in the browsing window of embodiment. It is explanatory drawing of the telop display operation | movement of embodiment. It is explanatory drawing of the example of a display after the telop display in the browsing window of embodiment. It is explanatory drawing of the tag file and video file of embodiment. It is explanatory drawing of the tag file with a video tag of embodiment. It is explanatory drawing of the tag file with a video tag of embodiment. It is explanatory drawing of the tag file with a video tag of embodiment. It is explanatory drawing of the file for video output of embodiment. It is explanatory drawing of the video output form of embodiment. It is a flowchart of the video output process from the browsing window of embodiment. It is explanatory drawing of the video output in the browsing window of embodiment. It is explanatory drawing of the document display state in the browsing window of embodiment. It is explanatory drawing of a part specification of the document in the browsing window of embodiment. It is explanatory drawing of the document display state in the browsing window of embodiment. It is explanatory drawing of partial specification of the document by the keyword in the browsing window of embodiment. It is a flowchart of the video output process from the classification | category window of embodiment. It is explanatory drawing of the video window of embodiment. It is a flowchart of the video output process from the video list window of embodiment. It is explanatory drawing of the video list window of embodiment. It is explanatory drawing of the functional block of the document processing apparatus of embodiment.

Hereinafter, embodiments of the present invention will be described in the following order.
1. 1. Configuration of document processing apparatus 2. Document data structure 3. Manual classification processing for document data 3-1 Processing procedure 3-2 Index creation 3-3 Document browsing / classification creation / classification operation 3-4 Classification model creation / registration 4. Automatic classification process for document data 4-1 Processing procedure 4-2 Automatic classification Reading process 6. 6. Summary creation / display processing Video file output process 7-1 Video file and tag 7-2 Video output form 7-3 Video output process from viewing window 7-4 Video output process from classification window 7-5 Video output form from video list window 8 . Functional block configuration of document processing device

1. Configuration of Document Processing Apparatus As shown in FIG. 1, a document processing apparatus 1 according to an embodiment of the present invention includes a main body 10 including a control unit 11 and an interface 12, and inputs sent to the main body 10 in response to input from a user. Unit 20, a communication unit 21 for transmitting / receiving signals to / from the outside, a display unit 30 for displaying output from the main body 10, a recording / reproducing unit 31 for recording / reproducing information on / from a recording medium 32, and audio An output unit 33 and an HDD (hard disk drive) 34 are provided.

The main body 10 has a control unit 11 and an interface 12 and constitutes a main part of the document processing apparatus 1.
The control unit 11 includes a CPU 13 that executes processing in the document processing apparatus 1, a RAM 14 that is a volatile memory, and a ROM 15 that is a nonvolatile memory.
For example, according to the procedure recorded in the ROM 15, the CPU 13 temporarily stores data in the RAM 14 when necessary, and performs control for executing the program.
The operation of the control unit 11 will be described in detail later, but it is necessary for the classification processing relating to the supplied document data, the summary creation processing, the generation processing of the voice reading file for the reading operation, and these processing. Document analysis. Furthermore, output control processing of video data related to document data is also performed. Programs and application software necessary for these operations are stored in the ROM 15, the HDD 34, or the recording medium 32.
The document processing program used by the control unit 11 may be stored in the ROM 15 in advance as described above, or may be taken in from the recording medium 32 or the HDD 34. For example, from the network such as the Internet via the communication unit 21, It is also conceivable to download a document processing program provided by an external server or the like.

The interface 12 is connected to the control unit 11, input unit 20, communication unit 21, display unit 30, recording / playback unit 31, audio output unit 33, and HDD 34.
Under the control of the control unit 11, the interface 12 inputs data from the input unit 20, inputs / outputs data to / from the communication unit 21, outputs data to the display unit 30, and outputs to the recording / reproducing unit 31. Data input / output, data output to the audio output unit 33, and data input / output to / from the HDD 34 are performed. Specifically, the timing of data input / output between the control unit 11 and each of the above units is adjusted, the data format is converted, and the like.

The input unit 20 is a part that receives user input to the document processing apparatus 1. The input unit 20 is configured by a keyboard or a mouse, for example. The user can use the input unit 20 to input characters such as keywords using the keyboard, or to select an element of the electronic document displayed on the display unit 30 with the mouse.
Hereinafter, an electronic document handled by the document processing apparatus 1 is referred to as “document data” or simply “document”. An “element” is an element constituting a document, and includes, for example, a document, a sentence, and a word.

The communication unit 21 is a part that receives a signal transmitted to the document processing apparatus 1 from the outside via, for example, a communication line or transmits a signal to the communication line. The communication unit 21 receives data such as one or a plurality of documents transmitted from the outside, and sends the received data to the main body 10. Of course, data is also transmitted to an external device via a communication line.
Further, one or a plurality of video data (video files) may be taken in by the communication unit 21 accompanying or independently of the document data.

  The display unit 30 is a part that displays characters and image information as an output of the document processing apparatus 1. The display unit 30 is configured by, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), or the like, and displays, for example, one or a plurality of windows, and characters, figures, etc. are displayed on the windows. Is displayed.

The recording / reproducing unit 31 records / reproduces data on / from a recording medium 32 such as a so-called floppy disk.
Although a floppy disk (magnetic disk) is taken as an example of the recording medium 32 here, it is needless to say that a portable medium such as an optical disk, a magneto-optical disk, or a memory card can be applied as an example of the recording medium 32. The recording / reproducing unit 31 may be any recording / reproducing device (disk drive, card drive, etc.) corresponding to the medium.

If the recording medium 32 records a document processing program for processing a document, the recording / reproducing unit 31 reads the document processing program from the recording medium 32 and supplies the document processing program to the control unit 11. Can do.
If document data or video data is recorded on the recording medium 32, it can be read out by the recording / reproducing unit 31 and supplied to the control unit 11. That is, the document processing apparatus 1 has an input mode of document data and video data different from the reception of document data and video data by the communication unit 21.
Further, the control unit 11 can record the document data processed by the document processing apparatus 1 in the recording medium 32 in the recording / reproducing unit 31.

The voice output unit 33 is a part that outputs a document as an output of the document processing apparatus 1 as a reading voice.
That is, the voice output unit 33 performs output processing of the voice signal when the voice signal generated by the voice synthesis process based on the document information (a reading file to be described later) is supplied by the control unit 11, thereby displaying the voice signal. It functions as an output unit of the document processing apparatus 1 together with the unit 30.

The HDD 34 provides a large capacity recording area in the document processing apparatus 1. The HDD 34 records / reproduces information based on the control of the control unit 11.
The HDD 34 is used for storing application programs for various processes executed by the control unit 11, such as a program for speech synthesis, or document data or video captured in the document processing apparatus 1, for example. It can be used as a part for storing data or the like.

2. Document Data Structure Next, the structure of document data in this example will be described. In this example, document processing is performed with reference to tags that are attribute information given to the document. The tags used in this example are syntactic tags that indicate the structure of the document, and semantic and pragmatics that allow the understanding of the mechanical content of the document across multiple languages. There is a tag.

Some syntactic tags describe the internal structure of a document.
As shown in FIG. 2, the internal structure by tagging is configured such that each element such as a document, a sentence, and a vocabulary element is linked to each other by a normal link and a reference / referenced link.
In the drawing, a white circle “◯” indicates an element, and the lowest white circle is a vocabulary element corresponding to the word at the lowest level in the document. A solid line is a normal link indicating a connection between elements such as a document, a sentence, and a vocabulary element, and a broken line is a reference link indicating a dependency relationship by reference / reference.
The internal structure of the document is in order from top to bottom, from document (documemt), subdivision (paradivision), paragraph (sentence), subsentential segment (...), vocabulary element Composed. Of these, subdivisions and paragraphs are optional.

On the other hand, as semantic / pragmatic tagging, there is one in which information such as meaning is described like the meaning of a multiple meaning word.
Tagging in this example is based on an XML (Extensible Markup Language) format similar to HTML (Hyper Text Markup Language).

  An example of tagging is shown below, but tagging of documents is not limited to this method. Moreover, although the example of an English and Japanese document is shown below, the description of the internal structure by tagging can be applied to other languages similarly.

  For example, the sentence “Time flies like an arrow.” Can be tagged as follows. <> Is a tag assigned to the document.

<Sentence><noun phrase meaning = “time0”> time </ noun phrase>
<Verb phrase><Verb meaning = “fly1”> flies </ verb>
<Adverb phrase><adverb meaning = like0> like </ adverb><nounphrase> an
<Noun meaning = “arrow0”> arrow </ noun></ noun phrase>
</ Adverb phrase></ Verb phrase>. </ Sentence>

  Where <sentence>, <noun>, <noun phrase>, <verb>, <verb phrase>, <adverb>, and <adverb phrase> are a sentence, noun, noun phrase, verb, verb phrase, adjective / adverb, respectively. (Including prepositional or postpositional phrases), adjective / adverbic phrases. In other words, it represents the syntactic structure of the sentence.

These tags are arranged corresponding to immediately before the end of the element and immediately after the end of the element. A tag placed immediately after the end of the element indicates the end of the element by the symbol “/”. Elements are syntactic constructs: phrases, clauses, and sentences.
Note that the word sense = “time 0” indicates a plurality of meanings of the word “time”, that is, the 0th meaning of the plurality of meanings. Specifically, the word “time” has a meaning of at least a noun, an adjective, and a verb. Here, the word “time” indicates a noun (= the 0th meaning). Similarly, the word “orange” has at least a plant name, color, and fruit meaning, which can also be distinguished by meaning.

  In this example, the syntactic structure of the document data can be displayed in a window 101 on the display unit 30 as shown in FIG. In this window 101, vocabulary elements are displayed on the right half surface 103, and the internal structure of the sentence is displayed on the left half surface 102.

  For example, as shown in the figure, in this window 101, a sentence “in which the internal structure is described by tagging“ C city where Mr. A's B meeting ended ”, some public newspapers and general newspapers self-regulate the photo coverage. Part of "Clarified policy on paper" is displayed. An example of tagging this document is as follows:

<Document><sentence><adverb phrase relationship = "place"><nounphrase><adverb phrase location = "C city">
<Adverb phrase relationship = “subject”><noun phrase identifier = “B-kai”><adverb phrase relationship = “affiliation”><person name identifier = “Mr. A”> Mr. A </ person name></ adverb phrase>< Organization name Identifier = “Group B”> Group B </ Organization name></ Noun phrase> is </ adverb phrase>
</ Adverb phrase><place name identifier = “C city”> C city </ place name></ noun phrase>, </ adverb phrase><adverb phrase relationship = “subject”><noun phrase identifier = “press "Syntax =" Parallel "><nounphrase><adverbphrase> Some </ adverb phrase> popular paper </ noun phrase> and <noun> general paper </ noun></ noun phrase></ adverb phrase >
<Adverb phrase relationship = “object”><adverb phrase relationship = “content” subject = “press”><adverb phrase relationship = “object”><nounphrase><adverbphrase><noun co-reference = “Group B ”> So </ noun></ adverb phrase> Photo report </ noun phrase></ adverb phrase>
Self-regulatory </ adverb phrase> policy </ adverb phrase>
<Adverb phrase interposition = "position"> on paper </ adverb phrase>
Revealed. </ Sentence></Document>

By tagging in this way, the structure of the document is expressed by each pair of tags <> to </>.
For example, a range of one document is indicated by <document> to </ document>, and similarly, a range of one sentence is indicated by <sentence> to </ sentence>. Further, for example, the part “Mr. A's B meeting” is expressed as a noun phrase having “B meeting” as an identifier by <noun phrase identifier = “B meeting”> to </ noun phrase>.
In other words, the internal structure of the sentence shown on the left half surface 102 of FIG.

  Further, in this document, “some popular papers and general papers” are represented as parallel by the tag “Syntax =“ Parallel ””. The definition of parallel means that the dependency relationship is shared. Unless otherwise specified, for example, <noun phrase relationship = x> <noun> A </ noun> <noun> B </ noun> </ noun phrase> indicates that A is dependent on B To express. Relation = x represents a relation attribute.

Relational attributes describe the interrelationships between syntax, meaning, and rhetoric. Grammatical functions such as subject, object, indirect object, subject roles such as actors, activists, beneficiaries, and rhetorical relationships such as reasons, results, etc. are described by this relation attribute. In this example, relational attributes are described for relatively easy grammatical functions such as a subject, object, and indirect object.

  In this document, the attributes of proper nouns such as “Mr. A”, “Group B”, and “C City” are described by tags such as place names, person names, and organization names. By adding tags such as place names, person names, organization names, etc., it is expressed that the word is a proper noun.

3. Manual Classification Processing for Document Data 3-1 Processing Procedure In the document processing apparatus 1 of this example, when document data is captured from the outside by, for example, the communication unit 21 (or the recording / reproducing unit 31), the document data is converted according to the content. Perform the classification process. In the following description, document data from the outside is described as being taken in via the communication unit 21, but the description is supplied from the outside in the form of a portable medium such as a floppy disk, and the recording / playback unit The same applies to the case where document data is fetched from 31.

As the classification process, there are a manual classification process in which the user manually classifies according to the contents of the document data, and an automatic classification process in which the document processing apparatus 1 automatically classifies.
These classification processes are performed based on a classification model to be described later. However, in the document processing apparatus 1, there is no classification model in the initial state. Therefore, at the time of being in the initial state, classification processing including creation of a classification model is necessary as manual classification processing. After the classification model is generated, automatic classification processing can be performed on the input document data.
First, here, the manual classification process that needs to be executed first will be described. That is, the manual classification process is a process in which the control unit 11 creates a classification model and classifies document data based on a user operation when the document processing apparatus 1 in the initial state receives document data sent from the outside. It becomes operation to perform.

  First, FIG. 4 shows the entire processing procedure as manual classification processing. Detailed processing of each processing step will be described later.

  Step F11 in FIG. 4 shows document reception processing by the reception unit 21 of the document processing apparatus 1. In step F11, the receiving unit 21 receives one or a plurality of documents transmitted via, for example, a communication line. The receiving unit 21 sends the received document to the main body 10 of the document processing apparatus. The control unit 11 stores the supplied one or more document data in the RAM 14 or the HDD 34.

  When a document is received, a video file including video data (moving image or still image data) related to the document may be received (of course, only video data may be received). If received, the control unit 11 captures the video file and stores it in the RAM 14 or the HDD 34. Processing related to the video file will be described in detail later.

In step F12, the control unit 11 of the document processing apparatus 1 extracts features of a plurality of documents sent from the receiving unit 21, and creates feature information, that is, an index of each document. The control unit 11 stores the created index in, for example, the RAM 14 or the HDD 34.
As will be described later, the index includes proper nouns and meanings other than proper nouns that are characteristic of the document, and can be used for document classification and search.

The document browsing in step F13 is a process executed as required by the user. That is, it is performed according to the user's operation. This step F13 and the next step F14 are processes based on user operations.
With respect to the input document data, the user can view the document contents on the screen of the display unit 30 by performing a required operation.
While the document is being browsed, the user can instruct various processing such as summary creation, which will be described later, by operating the icons and the like on the screen. The process proceeds to the classification operation.
In step F14, the control unit 11 generates / displays the classification item in response to the user performing an operation of setting the classification item (in this specification, the classification item is also referred to as a category). The user also performs an operation of distributing the document data to the set classification items, and the control unit 11 distributes / displays the document data accordingly.

In step F15, the control unit 11 creates a classification model in accordance with the classification item creation and classification operation performed by the user in step F14.
The classification model is composed of a plurality of classification items (categories) for classifying documents, and the classification state is determined by associating each document with an index of each document (index of each document created in step F12). It is data to prescribe.
If such a classification model is generated, the classification model is registered in step F16. That is, the control unit 11 performs registration by storing the classification model in, for example, the RAM 14.
According to the processing of FIG. 4 described above, manual classification and classification model creation are performed for one or a plurality of document data input when the document processing state 1 is in the initial state.
The processing after step F12 in FIG. 4 will be described in detail.

3-2 Index Creation In step F14, the control unit 11 creates an index for the input document data.
First, a specific example of an index created for a certain document data is shown.

<Index date = "AAAA / BB / CC" Time = "DD: EE: FF" Document address = "1234">
<User operation history Maximum summary size = "100">
<Number of selected elements = “10”> Picturetel </ Selected>
...
</ User operation history>
<Summary> Tax reduction scale, untouched-Prime Minister X's meeting </ summary>
<Word meaning = “0003” Central activity value = “140.6”> Do not touch </ word>
<Word meaning = “0105” identifier = “X” central activity value = “67.2”> prime minister </ word><person name identifier = “X” word meaning = “6103” central activity value = “150.2”> prime X Word / Person Name>
<Word meaning = “5301” central activity value = “120.6”> determined </ word>
<Word meaning = “2350” Identifier = “X” Central activity value = “31.4”> Prime Minister </ word>
<Word meaning = “9582” Central activity value = “182.3”> Emphasized </ Word>
<Word meaning = “2595” central activity value = “93.6”> touch </ word>
<Word meaning = "9472" Central activity value = "12.0"> Noticed </ Word>
<Word meaning = "4934" Central activity value = "46.7"> I didn't touch </ Word>
<Word meaning = “0178” central activity value = “175.7”> explained </ word>
<Word meaning = “7248” identifier = “X” central activity value = “130.6”> I </ word>
<Word meaning = “3684” Identifier = “X” Central activity value = “121.9”> Prime Minister </ word>
<Word meaning = “1824” central activity value = “144.4.”> Appealed </ word>
<Word meaning = “7289” central activity value = “176.8”> showed </ word>
</ Index>

In this index, <index> and </ index> are the beginning and end of the index, <date> and <time> are the date and time this index was created, and <summary> and </ summary> are The beginning and end of the summary of the contents of this index are shown respectively.
<Word> and </ Word> indicate the beginning and end of a word.
Further, for example, meaning = “0003” indicates the third meaning. The same applies to other cases. As described above, since the same word may have a plurality of meanings, a number is predetermined for each meaning in order to distinguish them, and the corresponding meaning is represented by a number.

  <User operation history> and </ User operation history> indicate the start and end of the user operation history, and <Select> and </ Select> indicate the start and end of the selected element, respectively. ing. The maximum summary size = “100” indicates that the maximum size of the summary is 100 characters, and the number of elements = “10” indicates that the number of selected elements is 10.

As in this example, the index includes proper nouns, meanings other than proper nouns, and the like that are characteristic of the document.
For example, the process of step F12 for creating such an index will be described with reference to FIGS. FIG. 5 shows an index creation process for one document data. Therefore, when processing a plurality of document data, the process of FIG. 5 is performed for each document data.
FIG. 7 shows detailed processing in step F31 in FIG. 5, and FIG. 8 shows detailed processing in step F43 in FIG.

As the index creation processing in step F12 in FIG. 4 described above, first, active diffusion in step F31 in FIG. 5 is performed.
This active diffusion is a process for giving high central activity values to elements associated with elements having high central activity values by diffusing the central activity values of the elements based on the internal structure of the document data. It is.
That is, after giving a central activation value as an initial value to each element constituting the document, the central activation value is diffused based on the internal structure of the document, specifically, the link structure.
Since this central activity value is determined according to the internal structure by tagging, it is used for extracting document features and the like.
As step F31, the control unit 11 performs active diffusion, and stores the central active value of each element obtained as a result of the active diffusion in the RAM 14, for example.

The active diffusion in step F31 will be described in detail with reference to FIGS.
First, an example of the element-element link structure is shown in FIG.
In FIG. 6, the periphery of the elements E1 and E2 is shown as part of the structure of the elements and links constituting the document. E1 to E8 are examples of elements, and the description will be given focusing on the elements E1 and E2.

Assume that the central activity value of element E1 is e1, and the central activity value of element E2 is e2.
The elements E1 and E2 are connected by a link L12 (the above-described normal link or reference link).
An end point connected to the element E1 of the link L12 is T12, and an end point connected to the element E2 is T21.
The element E1 is further connected to elements E3, E4, and E5 via links L13, L14, and L15, respectively. The end points on the element E1 side of the links L13, L14, and L15 are denoted as T13, T14, and T15, respectively.
Element E2 is also connected to elements E6, E7, and E8 by links L26, L27, and L28, respectively. The end points on the element E2 side in each of the links L26, L27, and L28 are T26, T27, and T28, respectively.
The active diffusion process of FIGS. 7 and 8 will be described using an example of such a link structure.

In step F41 of FIG. 7, when starting the active diffusion for the document data as the index creation target, the control unit 11 first performs the initial setting of the central active value for all the elements of the document data.
As the initial value of the central activity value, for example, a high value is given to a proper noun or an element selected (clicked) by the user.
The control unit 11 sets the end point activation value of the end point T (xx) of the link connecting the elements to 0 for the reference link and the normal link. The control unit 11 stores the initial value of the end point activation value thus assigned, for example, in the RAM 14.

  In step F42, the control unit 11 initializes a counter for counting the elements Ei constituting the document. That is, the count value i of the counter for counting elements is set to 1. When i = 1, this counter refers to the first element (for example, element E1 in FIG. 6).

In step F43, the control unit 11 executes a center activity value update process for calculating a new center activity value for the element referred to by the counter.
The center activation value update process will be described in detail with reference to FIG. 8 taking the process for the element E1 as an example.
This center activation value update process is a process of updating the endpoint activation value for the element, and further calculating a new center activation value using the updated endpoint activation value and the current center activation value.

  In step F51 of FIG. 8, the control unit 11 initializes a counter that counts the number of links having one end connected to an element Ei (for example, E1 in this case) constituting the document. That is, the count value j of the counter for counting links is set to 1. When j = 1, this counter refers to the first link L (yy) connected to the element Ei. In the example of FIG. 6, for example, a link L12 is referred to as the first link for the element E1.

  In step F52, the control unit 11 determines whether the link being referred to, that is, the link L12 connecting the elements E1 and E2, is a normal link by referring to the relation attribute tag. The control unit 11 proceeds to step F53 if the link L12 is a normal link, and proceeds to step F54 if the link L12 is a reference link.

When it is determined that the link L12 is a normal link and the process proceeds to step F53, the control unit 11 performs a process of calculating a new end point activation value of the end point T12 connected to the normal link L12 of the element E1.
The endpoint activation value t12 of the endpoint T12 is the endpoint activation value of all endpoints connected to the link other than the link L12 among the endpoint activation values of the link destination element E2 (in this case, the endpoint activation values of T26, T27, and T28). t26, t27, t28) and the central activation value e2 of the element E2, and the value obtained by this addition is divided by the total number of elements included in the document.
The control unit 11 calculates a new end point activation value for the end point connected to the normal link by performing such calculation using each end point activation value and each center activation value read from the RAM 14. The end point activation value thus stored is stored in the RAM 14. That is, the endpoint activation value t12 of the endpoint T12 is updated.

On the other hand, when it is determined in step F52 that the link L12 is a reference link and the process proceeds to step F54, the control unit 11 similarly calculates a new end point activation value of the end point T12 of the element E1 connected to the normal link L12. The calculation for calculating the end point activation value is as follows.
That is, in this case, the endpoint activation value t12 of the endpoint T12 is the endpoint activation value of all endpoints connected to the link other than the link L12 among the endpoint activation values of the link destination element E2 (in this case, T26, T27, T28). The end point activation values t26, t27, and t28) are added to the center activation value e2 of the element E2. (In other words, the point that there is no division is different from the case of the above normal link)
And the control part 11 calculates a new end point active value about the end point connected with the reference link by performing such calculation using each end point active value and each center active value read from RAM14, The calculated end point activation value is stored in the RAM 14. That is, the endpoint activation value t12 of the endpoint T12 is updated.

After performing the processing of step F53 or F54, the control unit 11 proceeds to step F57 via the determination processing in step F55 (if the determination result is NO), increments the count value j, and proceeds to step F52. Return.
That is, subsequently, since the count value j = 2 is set, the second link (for example, the link L13) for the element E1 is referred to. The end point activation value t13 of the end point T13 is calculated / updated.

In step F55, the control unit 11 determines whether or not new end point activation values for all links have been calculated for the element Ei (E1) that is currently being referenced with the count value i, and branches the process. Therefore, the end point activation value update processing is performed until all end point activation values of the element Ei being referred to are updated.
That is, by repeating the processing while incrementing the count value j in step F57, for example, for the element E1, the endpoint activation values t12, t13, t14, and t15 are updated for the endpoints T12, T13, T14, and T15, respectively. When all of them are updated, the process proceeds from step F55 to F56.

In response to the determination of all the end point activation values for the element Ei, in step F56, a new center activation value ei of the element Ei is calculated using the updated end point activation value.
The new center activation value ei of the element Ei is obtained by the sum of the current center activation value ei of the element Ei and the new endpoint activation values of all the endpoints of the element Ei. For example, in the case of element E1 in FIG. 6, the new center activation value e1 (new) is
e1 (new) = e1 + t12 + t13 + t14 + t15
It becomes.

  In this way, the control unit 11 calculates the center activation value ei of the element Ei being referenced with the current count value i. Then, the control unit 11 stores the calculated new center activation value ei in the RAM 14. That is, the center activation value ei of the element Ei is updated. (However, at this point of time, the old center activation value is also retained for use in the processing of step F45 described later)

When the process as shown in FIG. 8 is performed as the central activation value update process in step F43 in FIG. 7, the process of the control unit 11 proceeds to step F44 in FIG. It is determined whether or not the center activity value update processing has been completed for the element. Specifically, the control unit 11 determines whether or not the count value i has reached the total number of elements included in the document.
When the central activation value update process has not been completed for all the elements, the control unit 11 proceeds to step F47, increments the count value i, and returns to step F43.
For example, after the processing for the element E1 is completed as described above, the count value i = 2 is set, and this time, the element E2 is referred to.
Then, for the element E2, the center activation value update process of step F43 (that is, the process of FIG. 8) is performed in the same manner as described above.
Although not described in detail because of overlapping explanation, in the example of the link in FIG. 6, in the case of element E2, in the process of FIG. 8, the endpoint activation values t21, t26, t27, After t28 is updated, the new center activation value e2 (new) is
e2 (new) = e2 + t21 + t26 + t27 + t28
Is calculated and updated.

  In the processing of FIG. 7, the central activity value update processing of step F43 is repeated while the count value i is incremented in step F47 and the reference element is changed in this way, so that the central activity of all elements included in the document is obtained. The value will be updated.

When the update of the central active value is completed for all elements in the document, the process proceeds from step F44 to F45.
In step F45, the control unit 11 calculates an average value for the change in the central activity value of all elements included in the document, that is, the change in the newly calculated central activity value with respect to the original central activity value.
For example, the control unit 11 reads the old center activity value stored in the RAM 14 and the updated new center activity value for all elements included in the document. Then, the difference between the new center activity value and the old center activity value is obtained for each element, and the sum of the differences is divided by the total number of elements, thereby calculating the average value of the change in the center activity value of all the elements.
The control unit 11 stores, for example, the RAM 14 in the average value of the change in the central activity value of all the elements calculated in this way.

Subsequently, in step F46, the control unit 11 determines whether or not the average value calculated in step F45 is within a preset threshold value.
When the average value is within the threshold value, the control unit 11 ends the series of steps as the active diffusion process. When the average value is not within the threshold value, the control unit 11 returns to Step F42 and returns to the series of processes described above. Run the process again.

This series of active diffusion processes can be said to be a process of raising the central activity value of an element related (linked) to an element having a high central activity value.
However, when this active diffusion is performed only once, when the purpose of the index creation process is considered, there is an element in which the central activity value cannot be sufficiently increased among elements that should originally increase the central activity value. There is also a possibility. For example, in a single active diffusion, the central activity value is increased to some extent for an element directly linked to an element whose initial value of the central activity value is set high. Even if the element is important as an index, the center activity value may not be sufficiently increased.
Therefore, by performing the active diffusion process a plurality of times as necessary through the determination in step F46, the central active value is converged as a whole, and the important element that the central active value cannot be raised. This is to prevent as much as possible.
It should be noted that the central activity value is converged as a whole by multiple active diffusions, based on the central activity value of each element updated by the active diffusion processing, and further by the next active diffusion processing. This is because the central activity value is updated. However, if such an active diffusion process is performed too many times, the central activity values of all elements converge and become almost the same value, which is inappropriate.
For this reason, as processing of steps F45 and F46, active diffusion suitable for index creation is determined by determining the end timing of the active diffusion processing based on the change so as to obtain the average value of the change in the central activity value. Will be realized.

When the active diffusion process as shown in FIGS. 7 and 8 (that is, step F31 in FIG. 5) is completed, the process of the control unit 11 proceeds to step F32 in FIG.
In step F32, the control unit 11 extracts elements whose center activity value exceeds a preset threshold based on the center activity value of each element obtained in step F31. The control unit 11 stores the extracted elements in the RAM 14.

Subsequently, in step F33, the control unit 11 reads out the element extracted in step F32 from the RAM 14, for example. Then, the control unit 11 extracts all proper nouns from the extracted elements and adds them to the index. Proper nouns have no special meaning and have special properties such as not being listed in the dictionary, so they are handled separately from words other than proper nouns. The word meaning corresponds to each meaning of a plurality of meanings of the word as described above.
Whether each element is a proper noun can be determined based on a tag attached to the document. For example, in the internal structure with tagging shown in FIG. 3, “Mr. A”, “Group B”, and “C City” have the relationship attributes of “person name”, “organization name”, and “place name”, respectively. It can be seen that it is a proper noun. And the control part 11 adds the taken-out proper noun to an index, and memorize | stores the result in RAM14.

  In the next step F34, the control unit 11 extracts meanings other than proper nouns from the elements extracted in step F32, adds them to the index, and stores the results in the RAM 14.

Through the above processing, for example, an index like the above-described specific example is generated. That is, the index is a feature of tagged documents, and the features are arranged, and the features of the document are determined based on the central activity value subjected to diffusion processing according to the internal structure of the document. It will be a thing.
Such an index includes meanings and proper nouns that represent features representing the document, and can be used when referring to a desired document.
The index includes a document address indicating a position where the document is stored in the RAM 14 (or HDD 34), along with a meaning and proper nouns representing the characteristics of the document.

3-3 Document Browsing / Category Creation / Classification Operation The index creation processing described above with reference to FIGS. 5 to 8 is performed in step F12 in FIG. Therefore, the manual classification process in FIG. 4 proceeds to the processes of steps F13 and F14, that is, the browsing and manual classification processes by the user as described above.

As described above, in step F13 in FIG. 4, the user can browse the document displayed on the display unit 30.
In step F14, the user can perform an operation for setting a classification item or an operation for distributing document data to the set classification item.
The operations performed in steps F13 and F14, the processing of the control unit 11 corresponding thereto, and the display example of the display unit 30 are as follows.

9 and 10 show specific examples of display on the display unit 30. FIG.
First, FIG. 9 is a display example of a classification window 201 corresponding to a classification model described later in detail. That is, it is a specific example of a graphic user interface (GUI) used for displaying the document classification.
In this classification window 201, as a button display 202 for operation, a position reset button 202a for returning the window state of the screen to an initial position, and a browser (browser: viewing window described later) for reading the contents of the document ) And a exit button 202c from this window are displayed.
Further, as will be described later in detail as the button display 202, a video button 202d for executing video output related to a document designated from the classification window 201 is displayed, and a video list button 202e for calling a video list window described later is displayed. Is displayed.

Further, the classification window 201 is formed with document classification areas 203, 204, 205... As small windows corresponding to the classification items corresponding to the classification model.
The document classification area 203 is an area for displaying “other topics”. This “other topics” document classification area 203 is an area where documents that have not been classified yet are presented. For example, each document received in step F11 in FIG. 4 (that is, a document to be classified in the future) is presented in the document classification area 203 of this “other topics”.
The document classification area 204 is an area where documents classified as “business news” are presented, for example.
The document classification area 205 is an area in which documents classified as “political news” are presented, for example.
In addition to these, the document classification areas not denoted by reference numerals in the figure are areas where documents corresponding to specific classification items are presented.

In each of these document classification areas 203, 204..., Documents classified into the classification items (categories) set in the respective document classification areas are presented by the document icon and the document title. If there is no title, a summary of one sentence is used as the title and displayed.
Further, the size of each document classification area 203, 204... Is not fixed, and each document can be moved by moving the separation frames 211, 212, 213. The areas of the classification areas 203, 204... Can be arbitrarily changed. The number of document classification areas can be arbitrarily increased or decreased by the user.

Further, item titles (for example, “political news”) of the document classification areas 203, 204... Can be arbitrarily set and changed by the user.
The number of document classification areas and the item titles correspond to the classification items of a classification model to be described later. In other words, the user can set or delete a document classification area or set an item title by operating the input unit 20 using a mouse, a keyboard, or the like in the classification window 201. Will be set.
Note that a check mark 214 is added to each document data displayed in a list in the classification window 201 when the user selects (clicks).

FIG. 10 shows an example of a browsing window 301 where the user browses the contents of document data.
For example, after the user clicks and selects a certain document in the classification window 201 of FIG. 9 (a state in which a check mark 214 is added to a certain title (document data)), the browser button By clicking 202b, the control unit 11 opens a browsing window 301 for displaying the selected document as shown in FIG.

The browsing window 301 displays a file name display unit 302 that displays the file name of the document data file, a document display unit 303 that displays the document data of the file name, and a summary sentence of the document displayed on the document display unit 303. And a keyword display unit 305 for inputting / displaying keywords.
As the button display 306 for operation, a summary creation button 306a for instructing creation of a summary sentence, an undo button 306b for performing an undo operation (operation cancellation), a reading button 306c for executing a reading operation, video data A video button 306d for instructing display of the video list, a video list button 306e for calling a video list window described later, and the like are displayed.

In such a browsing window 301, the user can browse a document displayed on the document display unit 303. When the entire document cannot be displayed, a part of the document is displayed. Of course, you can view the full text by scrolling.
In addition, the user can create a summary sentence about the document displayed on the document display unit 303 by clicking on the summary creation button 306 a and display it on the summary display unit 304.
Furthermore, the user can read out the text or summary of the document displayed in the document display unit 303 by clicking the read-out button 306c.
In addition, the user can click the video button 306d to display the text of the document displayed on the document display unit 303 or the video data related to the summary text displayed on the summary display unit 304. it can.
The processing of the control unit 11 for creating a summary sentence, the processing for the reading operation, and the processing for the video output operation will be described later.

The classification window 201 and the browsing window 301 as described above are not limited to the manual classification process of FIG. 4 and are displayed on the display unit 20 as needed according to the user's operation. The manual classification process of FIG. With regard to the above, the user can confirm the type and contents of the received document in the classification window 201 and the browsing window 301.
Specifically, one or more documents received in step F11 in FIG. 4 are stored in the document classification area 203 of “other topics” in the classification window 201 as shown in FIG. 9 after the index creation processing in step F12. Is displayed.
In this classification window 201, the user manually categorizes each document displayed in the document classification area 203. For example, when the contents are not known only by the title of the document, the browsing window 301 in FIG. Confirm the document contents with. The browsing performed according to the user's needs in this way is the processing of step F13 in FIG.

In step F14, the user can arbitrarily add, update, and delete classification items on the classification window 201, and the control unit 11 displays the displayed document classification areas 203, 204,. -Change the display mode (number, area, title, etc.).
The setting / change of the classification item (document classification area title) by the user is reflected in a classification model described later.

After setting the classification items as necessary, the user sorts each document displayed in the document classification area 203 into each document classification area. In other words, the documents are classified manually by the user.
Specifically, the icon of the document displayed in the document classification area 203 of “other topics” is dragged to the document classification area corresponding to a desired classification item (category) using the mouse of the input unit 20, for example. By doing.
For example, after setting the document classification area titled “Sports”, the user sets the sports-related document icon displayed in the “Other Topics” document classification area 203 to the “Sports” document classification area. Do something like dragging.
The icons and titles of the documents manually classified in this way are displayed in the document classification area of the dragged destination.

3-4 Creation / Registration of Classification Model When the manual classification operation by the user is performed as described above, the control unit 11 creates a classification model including a plurality of categories based on the classification operation of the user in Step F15 of FIG. . That is, the control unit 11 collects the indexes of the plurality of documents classified into each category, and generates a classification model. Then, the plurality of documents are classified into each category of the classification model.

The classification model is composed of a plurality of classification items (categories) for classifying documents. Then, for each category, a data format is shown in which classified documents are shown.
For each document, an index is formed in step F12 and the like. The classification model has, for example, a data structure in which an index of a document classified for each category is associated as shown in FIG. Become.
In FIG. 11A, “Sports”, “Company”, “Computer”, etc. are set as categories. These are the classification items set by the user in the classification window 201 as described above. Of course, there may be a category set in advance (ie, displayed as a document classification area in the classification window) without being set by the user.
Each category item is associated with an index IDX1, IDX2,..., That is, each category item is associated with an index of a document classified by the user as described above.

The index associated with each classification item is an index of a document displayed in the document classification area of the classification item in the classification window 201.
For example, the index IDX1 is associated with the category “sports” because the user creates a document classification area whose title is “sports” in the classification window 201, and further displays the icon of the document with the index IDX1 as the “sports”. This is based on the manual classification of dragging to the document classification area having “” as the title.

As described above, each document index includes proper nouns, meanings other than proper nouns, document addresses, and the like.
For example, as shown in FIG. 11A, one classification item is associated with one or a plurality of indexes. Since the index includes proper nouns, meanings, document addresses, etc., the classification model is shown in FIG. ).

That is, as shown in FIG. 11B, the classification model has a structure having columns of proper nouns, meanings other than proper nouns, and document addresses as category indexes corresponding to the respective categories.
In the classification model, for each category “sports” “society” “computer” “plant” “art” “event”, proper nouns “Mr. A, ...”, “Mr. B, ...”, Proper nouns such as “C company, G company,...”, “D type,...”, “Mr. E,.
"Baseball (4546), Ground (2343), ...", "Labor (3112), Unique (9821), ...", "Mobile (2102), ...", "Sakura 1 (11111) ), Orange 1 (9911), cherry blossom 2 (11112), orange 2 (9912), and cherry 3 (11113) are also assigned to each category.
Further, document addresses “SP1, SP2, SP3,...”, “S01, S02, S03,...”, “CO1, CO2, CO3,...”, “PL1, PL2, PL3,. , "AR1, AR2, AR3, ..." and "EV1, EV2, EV3, ..." are also assigned to each category.

“Sakura 1”, “Sakura 2”, and “Sakura 3” indicate the first meaning (11111), the second meaning (11112), and the third meaning (11113) of “Sakura”. “Orange 1” and “Orange 2” indicate the first meaning (9911) and the second meaning (9912) of “orange”. For example, “Orange 1” represents a plant orange and “Orange 2” represents an orange color.
The reason why the word meaning is used instead of the word itself in cases other than proper nouns is that the same word may have a plurality of meanings.

In step F15 of FIG. 4, for example, such a classification model is generated according to the manual classification operation of the user. In step F16, the classification model is registered, that is, recorded in the RAM 15 (or HDD 34).
Thus, the classification of the document is performed by generating / registering the classification model.

After the classification model is created / registered in steps F15 and F16 in FIG. 4, the classification is performed according to automatic classification processing described later, user classification item editing, or manual classification operation. The model will be updated sequentially.
When the classification model is updated, the update date and time is recorded in the classification model. In FIG. 11, “Dec. 10, 1998, 19:56:10” is recorded as the update date.

4). Automatic Classification Processing for Document Data 4-1 Processing Procedure In the document processing apparatus 1 of this example, once a classification model is created as described above, for example, document data imported from the outside by the communication unit 21 is automatically processed. It is possible to perform an automatic classification process of classifying into
That is, the automatic classification process described below is a process for classifying the document data with respect to the classification model when the document processing apparatus 1 receives the document data sent from the outside.
In this example, the automatic classification processing described below is performed every time one document is received, or may be performed every time a predetermined number of documents are received, or the user can display the screen shown in FIG. The automatic classification process may be performed on all the documents received so far when the operation is opened.

FIG. 12 shows the entire processing procedure as the automatic classification processing.
Step F21 in FIG. 12 shows document reception processing by the reception unit 21 of the document processing apparatus 1. In step F21, the receiving unit 21 receives one or a plurality of documents transmitted via, for example, a communication line. The receiving unit 21 sends the received document to the main body 10 of the document processing apparatus. The control unit 11 stores the supplied one or more document data in the RAM 14 or the HDD 34.
When one or a plurality of video files are received together with the document data (or independently of the document data), the control unit 11 takes in the video files and stores them in the RAM 14 or the HDD 34.

  In step F22, the control unit 11 creates an index for the document captured in step F21.

  In step F23, the control unit 11 automatically classifies each indexed document into any category of the classification model based on the classification model. Then, the control unit 11 stores the classification result in, for example, the RAM 14. Details of the automatic classification will be described later.

In step F24, the control unit 11 updates the classification model based on the result of the automatic classification of the new document in step F23.
In step F25, the control unit 11 registers the classification model updated in step F24. For example, the classification model is stored in the RAM 14.

Through the processing of FIG. 12, the automatic classification processing is performed so that the document data input to the document processing state 1 is classified on the classification model.
That is, in this automatic classification process, an index is created for the received document, and after automatic classification is performed, the proper nouns, meanings, document addresses, etc. constituting the index are shown in FIG. Thus, the classification model is associated with a certain category (the classification model is updated).

The processes in steps F21 and F22 are the same as steps F11 and F12 in the manual classification process described above. That is, as the index creation processing in step F22, the processing described with reference to FIGS. 5 to 8 is performed, and repeated description here is avoided.
The update of the classification model in step F24 is in accordance with the classification result of the automatic classification in step F23.
Hereinafter, the automatic classification in step F23 will be described in detail as a process different from the manual classification process described above.

4-2 Automatic Classification Detailed processing of the automatic classification in step F23 of FIG. 12 is shown in FIG.
In step F61 of FIG. 13, the control unit 11 performs a set of proper nouns included in the category Ci of the classification model and a set of proper nouns among the words extracted from the documents received in step F21 and put into the index. Let P (Ci) be the number of these common sets. The control unit 11 stores the number P (Ci) calculated in this way in the RAM 14.

In step F62, the control unit 11 shows a relationship between meanings between all meanings included in the index of the document and all meanings included in each category Ci. , The sum R (Ci) of the degree of association between meanings is calculated.
That is, the control unit 11 calculates the total sum R (Ci) of all-word meanings for words other than proper nouns in the classification model. And the control part 11 memorize | stores the total R (Ci) of the calculated relevance degree between meanings in RAM14.

Here, the degree of association between meanings will be described.
The degree of association between meanings may be calculated in advance for the meanings included in the electronic dictionary provided in the document processing apparatus 1 by the process of FIG. 14, and the result may be retained as shown in FIG. That is, when the control unit 11 executes the process of FIG. 14 once in advance, it can be used in the automatic classification process of FIG.

The processing of FIG. 15 that is executed in advance by the control unit 11 is as follows.
First, in step F71, the control unit 11 uses the dictionary to explain the meaning of words in the electronic dictionary and creates a word-meaning network using this dictionary.
That is, a meaning network is created from the reference relationship between the explanation of each meaning in the dictionary and the meaning that appears in this description.
The internal structure of the network is described by tagging as described above. The control unit 11 of the document processing apparatus creates a network by sequentially reading the meaning and explanation of the electronic dictionary stored in the RAM 14, for example.
The control unit 14 stores the semantic network created in this way in the RAM 14.

The network is created by the control unit 11 of the document processing apparatus using a dictionary, and is received from the outside by the receiving unit 21 and reproduced from the recording medium 32 by the recording / reproducing unit 31. Can also be obtained.
The electronic dictionary can be obtained by receiving from the outside by the receiving unit 21 or by reproducing from the recording medium 32 by the recording / reproducing unit 31.

  In step F72, the center activation value corresponding to each semantic element is diffused on the semantic network created in step F71. By this active diffusion, the central activity value corresponding to each meaning is given according to the internal structure by tagging given by the dictionary. The center activity value diffusion process is the process described with reference to FIG.

  In step F73, one word meaning Si constituting the word meaning network created in step F71 is selected, and in the next step F74, the initial value of the central activation value ei of the vocabulary element Ei corresponding to this word meaning Si is selected. The difference Δei of the central activity value at this time is calculated.

Further, in step F75, a difference Δej of the central activation value ej of the element Ej corresponding to another meaning Sj corresponding to the difference Δei of the central activation value ei of the element Ei in step F74 is obtained.
In step F76, the quotient Δej / Δei obtained by dividing the difference Δej obtained in step F75 by Δei obtained in step F74 is used as the degree of association between meanings for the meaning sj of the meaning Si.

In step F77, it is determined whether or not the calculation of the degree of association between meanings has been completed for all pairs of one meaning Si and another meaning Sj.
When the calculation of the degree of association between word meanings is not completed for all pairs of meanings, the process returns to step F73, and the calculation of the degree of association between meanings is continued for the pair whose calculation of the degree of association between meanings is not completed.
In such a loop from step F73 to step F77, the control unit 11 sequentially reads necessary values from the RAM 14, for example, and calculates the degree of association between meanings as described above. For example, the control unit 11 sequentially stores the calculated degree of association between meanings in the RAM 14, for example.
Then, when the calculation of the degree of association between meanings is completed for all meaning pairs, the series of processes is ended from Step F77.

Such calculation of the degree of association between meanings can be said to be a process in which the meaning of the change in the central activity value according to the change in the central activity value of a certain meaning has a high degree of association.
In other words, if the central activity value of a certain meaning is changed in step F74, the central activity value of the related (linked) meaning is changed accordingly. You can see the degree of relevance of each other meaning. (As described in the explanation of active diffusion, the central activity value of a certain element Ei reflects the central activity value and end point activity value of the linked element, and the element Ei end point activity value is updated. In addition, since it is obtained from the sum of the end point activation value of the element Ei and the current center activation value, the amount of change in the center activity value increases as the degree of association with the link destination increases)
By performing such processing from each meaning to all other meanings, the degree of association can be calculated for all meaning pairs (combinations).

  The degree of association between meanings calculated in this way is defined between each meaning and meaning as shown in FIG. In the table of FIG. 15, the degree of association between meanings is normalized to take a value from 0 to 1. In this table, the degree of relevance between the meanings of “computer”, “television”, and “VTR” is shown as an example. The degree of association between the meanings of “computer” and “TV” is 0.55, the degree of association between the meanings of “computer” and “VTR” is 0.25, and the degree of association between the meanings of “TV” and “VTR” is 0.60. It is.

When the processing in step F62 in FIG. 13 is performed using the pre-calculated relevance between the meanings as described above, the control unit 11 subsequently performs the relationship Rel between document classifications of the document with respect to the category Ci as step F63. (Ci) Rel (Ci) = m1P (Ci) + n1R (Ci)
Calculate as
Here, the coefficients m1 and n1 are constants and represent the degree of contribution of each value to the degree of association between document classifications.
The control unit 11 performs the calculation of the above formula using the number P (Ci) of the common set calculated in step F61 and the sum R (Ci) of the relationship between meanings calculated in step F62, and the relationship between the document classifications Rel (Ci) is calculated.
The control unit 11 causes the RAM 14 to store the inter-document classification degree of association Rel (Ci) calculated in this way.

The values of these coefficients m1 and n1 can be set to m1 = 10 and n1 = 1, for example.
The values of the coefficients m1 and n1 can also be estimated using a statistical method. That is, the control unit 11 can obtain the above-mentioned coefficient by optimization by being given the inter-document-class relevance Rel (Ci) for a plurality of pairs of coefficients m and n.

In step F64, when the inter-document classification relation Rel (Ci) for the category Ci is the maximum and the value of the inter-document classification relation Rel (Ci) exceeds a certain threshold, the category Ci Sort documents into
That is, the control unit 11 creates a document category relevance level for each of the plurality of categories, and when the maximum document category relevance level exceeds the price, the document has the maximum document category relevance level. Classify into category Ci. As a result, the document is automatically classified into a required category.
When the maximum degree of relevance between document classifications does not exceed the threshold value, document classification is not performed.

When the automatic classification in step F23 of FIG. 12 is performed as the processing of FIG. 13 as described above, a series of automatic classification is completed by updating and registering the classification model accordingly in steps F24 and F25. .
That is, the document data received by the document processing apparatus 1 is automatically classified, and the user can confirm the received document data in the required document classification area in the classification window 201 of FIG. Become.

5). Reading process Next, a process for reading the document data will be described.
As described above, the user can browse the text of the document by selecting the document and opening the viewing window 301 as shown in FIG. For example, the browsing window 301 can be opened from the classification window 201 described with reference to FIG.

For example, when the browser button 202b is clicked in a state where a certain document is selected in the classification window 201, a browsing window 301 in which the text of the selected document is displayed in the document display unit 303 is opened as shown in FIG. It is.
When the entire document cannot be displayed on the document display unit 303, a part of the document is displayed.
At the time when the summary sentence is not created, the summary display unit 304 is blank as shown in FIG. A summary sentence creation and display operation will be described later.

As described above, when the document processing apparatus 1 receives the tagged document, the document processing apparatus 1 can display the text and a summary sentence, which will be described later in detail, and present it to the user. it can.
That is, the CPU 13 activates a speech reading program among the electronic document processing programs recorded in the ROM 15 and the HDD 34, and can read the document through a series of steps as shown in FIG. .
First, each simplified process will be described here, and then each process will be described in detail using a specific document example.

  As processing of the control unit 11, step F101 in FIG. 16 is a document reception / storage process similar to step F11 in FIG. 4 (or step F21 in FIG. 12). As described above, the received document (tagged document) is classified manually or automatically. Similarly, step F101 in FIG. 16 is executed in the sense that the received document can be read out. It is written. Here, the order of the classification process and the reading process and the like are not particularly defined as a processing procedure.

  It should be noted that, as will be described later, a tag that is necessary for performing speech synthesis needs to be assigned to a document (received document) that is a target of document reading processing. However, the document processing apparatus 1 can also create a document by receiving a tagged sentence and adding a new tag necessary for speech synthesis to the document. Hereinafter, a tagged document prepared by receiving or creating in this way is referred to as a tag file.

  As the document reading process of the document processing apparatus 1, in step F102, a reading file is generated based on the tag file under the control of the CPU 13. As will be described later, the reading file is generated by deriving attribute information for reading from a tag in the tag file and embedding the attribute information.

Subsequently, in step F103, the document processing apparatus 1 performs processing suitable for the speech synthesis engine using the reading file under the control of the CPU 13.
This speech synthesis engine may be configured by hardware or may be realized by software. When the speech synthesis engine is realized by software, the application program is stored in advance in the ROM 15, the HDD 34, or the like.
In the document processing apparatus 1 of this example, a functional block configuration formed to execute a reading process, a summary creation / display process, a video output process, and the like, which will be described later, will be collectively described with reference to FIG. However, the speech synthesis engine corresponds to the speech synthesis engine 601 in FIG.

Subsequently, in step F104, the document processing apparatus 1 performs processing in accordance with an operation performed by the user using a user interface described later.
The document processing apparatus 1 can read a given document by performing such processing. Each of these steps will be described in detail below.

First, reception or creation of a tagged document in step F101 will be described.
For example, the document processing apparatus 1 receives a document (a document to which a tag necessary for performing speech synthesis has already been assigned) from the communication unit 21.
Alternatively, the document processing apparatus 1 receives a tagged document and creates a document by newly assigning a tag necessary for speech synthesis to the document.

  As an illustrative example, it is assumed that a tag file in which Japanese and English documents are tagged as described below is received or created.

First, the Japanese document that is the source of the tag file is as follows.
"[Nicely aging] / 8 cancer transfer can be suppressed !?
Cancer has been the leading cause of death in Japan for over a decade. The mortality rate is increasing with age. When you consider the health of the elderly, you cannot avoid the problem of cancer.
What characterizes cancer is cell proliferation and metastasis. In human cells, there is an “oncogene” that hits the accelerator in a car and grows more and more, and a “cancer suppressor gene” that acts as a brake.
There is no problem if both are balanced. When normal regulatory functions are lost and mutations that do not work in the cell occur, cancer begins to grow. In older people, this mutation accumulates over a long period of time, increasing the proportion of cells with conditions for canceration, leading to more cancer.
By the way, without another feature, the nature of metastasis, cancer doesn't have to be so afraid. It is because complete cure is possible only by excision. Here is the importance of suppressing metastasis.
This metastasis does not occur just by increasing the number of cancer cells. Cancer cells dissolve proteins (proteins) between cells and create a path for them to enter blood vessels and lymph vessels. In recent years, it has been elucidated that complicated movements such as searching for new “residents” and diving while circulating. "

An example of the English document that the tag file is based on is as follows: `` During its centennial year, The ABCD Journal will report events of the past century that stand as milestones of American business history.THREE COMPUTERS THAT CHANGED the face of personal computing were Iaunched in 1977. That year the PC A II, PC B and PC C came to market.The computers were crude by today's standerds.PC A ll owners, for example, had to use their television sets as screens and stored. data on audiocassettes. ''

  When the document processing apparatus 1 receives a document tagged with such a Japanese or English document, the document processing apparatus 1 displays the text as shown in FIGS. 26 and 27 described later, or displays a summary sentence. Can be created and displayed.

Here, the above Japanese or English documents are each configured as a tag file as shown in FIG.
As a tag file of a Japanese document, FIG. 18 (a) shows an excerpt of the headline part “[Nicely aging] / 8 cancer transfer, can be suppressed !?”, FIG. 18 (b), This is an excerpt of the last paragraph in the document, "This metastasis, cancer cells ... are being elucidated in recent years." The remaining paragraphs are omitted.
Note that the actual tag file is configured as one file from the heading to the last paragraph.

In the heading part shown in FIG. 18A, the tag <heading> indicates that this part is a heading.
The tag files shown in FIGS. 18A and 18B are basically tagged in the same manner as the tag file example used when the document data structure is described with reference to FIG. Although the detailed description about each tag mentioned above is abbreviate | omitted, the tag required in order to synthesize | combine speech is provided to required places.

As a tag required for speech synthesis, for example, as shown in the part of “Example 1” in the figure, information indicating a reading pseudonym is given to the original document such as “protein”. There are things that are granted when you are. That is, in this case, in order to prevent the reading from overlapping with “protein protein”, reading attribute information of pronunciation = “null” is described, and a tag prohibiting partial reading of “(protein)” is included. Has been granted.
In addition, as shown in “Example 2” and “Example 3” in the figure, tags necessary for speech synthesis are erroneously read out as technical terms such as “lymphatic vessels” and “residents”. There are things that are given to difficult parts that can be performed. That is, in this case, in order to prevent reading “Rinpakuda” or “Sumie”, the reading attribute information indicating the reading pseudonym of pronunciation = “Rinpakan” and pronunciation = “Sumika” is described. ing.

  On the other hand, as a tag necessary for speech synthesis in the tag file shown in FIG. 19, the pronunciation = “two” for a Roman numeral “II” as shown in “Example 4” in the figure. Reading attribute information is described. This is described in order to prevent “second” from being read out as “second” when it is desired to read out “II” as “two”.

  For example, when a quotation is included in a document, a tag indicating that the sentence is a quotation is attached to such a tag file, although not shown. Further, for example, when there is a question sentence in the document, a tag indicating that the sentence is a question sentence is attached to the tag file, although not shown.

  In step F101 shown in FIG. 16, the document processing apparatus 1 receives or creates a document to which a tag necessary for speech synthesis is added, for example, as in the above example.

Next, generation of the reading file in step F102 in FIG. 16 will be described.
The document processing apparatus 1 derives attribute information for reading from the tag in the tag file, and generates a reading file by embedding the attribute information.
Specifically, the document processing apparatus 1 finds tags indicating the heads of paragraphs, sentences, and phrases of the document, and embeds attribute information for reading in correspondence with these tags. In addition, when the document processing apparatus creates a summary sentence of the document, it finds the beginning of the part included in the summary sentence from the document, embeds attribute information that increases the volume at the time of reading, and includes the part included in the summary sentence It can be emphasized.

  The document processing apparatus 1 generates a reading file as shown in FIG. 20 or FIG. 21 from the tag file shown in FIG. 18 or FIG. 20A and 20B correspond to the portions shown in FIGS. 18A and 18B. Of course, the actual reading-out file consists of the above-mentioned heading part to the last paragraph as one file.

  In the read-out file shown in FIG. 20, attribute information Com = Lang *** is embedded corresponding to the head of the document. This attribute information indicates the language in which the document is described. Here, the attribute information is Com = Lang = JPN, which indicates that the language describing the document is Japanese. In the document processing apparatus, by referring to this attribute information, an appropriate speech synthesis engine corresponding to the language can be selected for each document.

In addition, attribute information such as Com = begin_p, Com = begin_s, and Com = begin_ph is embedded in this reading file. These pieces of attribute information indicate the heads of paragraphs, sentences, and phrases of the document, respectively. The document processing apparatus 1 identifies the heads of these paragraphs, sentences, and phrases based on the tags in the tag file described above.
It should be noted that in the reading file, for example, portions where tags representing the same level of syntactic structure appear continuously, such as <adjective verb phrase><nounphrase> in the tag file described above, respectively. A number of Com = begin_ph is not embedded, and one Com = begin_ph is embedded.

Further, attribute information of Pau = 500, Pau = 100, and Pau = 50 is embedded in the reading file in correspondence with Com = begin_p, Com = begin_s, and Com = begin_ph, respectively. These pieces of attribute information indicate that pause periods of 500 milliseconds, 100 milliseconds, and 50 milliseconds are provided when reading out.
That is, the document processing apparatus 1 sets a pause period of 500 milliseconds, 100 milliseconds, and 50 milliseconds at the beginning of a paragraph, sentence, and phrase of a sentence so that the document is read out by the speech synthesis engine. Information.
Note that these pieces of attribute information are embedded corresponding to Com = begin_p, Com = begin_s, and Com = begin_ph. Therefore, for example, <adverb phrase><nounphrase> in a tag file, a portion in which tags representing syntactic structures of the same level appear consecutively is regarded as one phrase, and the corresponding number of Pau = A single Pau = 50 is embedded without being embedded.
Also, for parts where tags representing different levels of syntactic structure appear consecutively, such as <paragraph><sentence><nounphrase> in the tag file, the corresponding Pau = *** is embedded. . For this reason, when reading out such a portion, the document processing apparatus 1 reads out such a portion by providing a pause period of 650 milliseconds obtained by adding the pause periods of paragraphs, sentences, and phrases of the document, for example.
As described above, the document processing apparatus 1 can perform reading without a sense of incongruity in consideration of breaks between paragraphs, sentences, and phrases by providing pause periods corresponding to paragraphs, sentences, and phrases. The pause period does not need to be 600 milliseconds, 100 milliseconds, and 50 milliseconds, respectively, at the beginning of paragraphs, sentences, and phrases of the document, and can be changed as appropriate.

  Furthermore, in the file for reading out, “(protein)” is removed corresponding to the reading attribute information described in the tag file as “phonetic” = “null”, and the phonetic = “Rinpakan”, Corresponding to the reading attribute information of pronunciation = “Sumika”, “lymph vessel” and “resident” are respectively replaced by “Rinpakan” and “Sumika”. The document processing apparatus 1 embeds such reading attribute information so as not to make a reading error due to a deficiency of a dictionary referred to by the speech synthesis engine.

Also, attribute information for designating only this quoted sentence to use another speech synthesis engine based on a tag indicating that it is a quoted sentence included in the document is embedded in the reading file. Also good.
Furthermore, attribute information for increasing the intonation of the ending of the sentence may be embedded in the reading file based on a tag indicating a question sentence.
Furthermore, attribute information for converting a so-called “Narutone” style to a “Masutone” style can be embedded in the reading file as needed. In this case, the document processing apparatus 1 does not embed such attribute information in the reading-out file, but converts the text of “Naruto” into the style of “Nadamasu” and converts the reading-out file into a voice-reading file. You may make it produce | generate.

On the other hand, in the reading file shown in FIG. 21, attribute information of Com = Lang = ENG is embedded corresponding to the head of the document, indicating that the language describing the document is English.
Also, attribute information such as Com = Vol = *** is embedded in the reading file. This attribute information indicates the volume at the time of reading. For example, Com = Vol = 0 indicates that reading is performed at the default volume of the document processing apparatus. Also, Com = Vol = 80 indicates that the default volume is read out with a volume increased by 80%. Arbitrary Com = Vol = *** is valid until the next Com = Vol = ***.
Further, in the reading file, “II” is replaced with “two” in correspondence with the reading attribute information “phonetic” = “two” described in the tag file.

The document processing apparatus 1 generates such a reading file through a series of steps shown in FIG.
First, the document processing apparatus 1 analyzes the received or created tag file by the CPU 13 in step F201. Here, the document processing apparatus 1 determines the language in which the document is described, and searches for the beginning of paragraphs, sentences, and phrases of the document and reading attribute information based on the tags.
Subsequently, in step F202, the document processing apparatus 1 causes the CPU 13 to embed attribute information such as Com = Lang = *** at the beginning of the document according to the language in which the document is described.

  Next, in step F203, the document processing apparatus 1 causes the CPU 13 to replace the beginning of the paragraph, sentence, and phrase of the document with attribute information in the reading file. That is, the document processing apparatus 1 replaces <paragraph>, <sentence>, and <*** phrase> in the tag file with Com = begin_p, Com = begin_s, and Com = begin_ph, respectively.

Further, in step F204, the document processing apparatus 1 causes the CPU 13 to combine the same level of syntactic structure and the same Com = begin _ *** into one Com = begin _ ***.
Subsequently, in step F205, the document processing apparatus 1 causes the CPU 13 to embed Pau = *** corresponding to Com = begin _ ***. That is, the document processing apparatus 1 embeds Pau = 500 before Com = begin_p, embeds Pau = 100 before Com = begin_s, and embeds Pau = 50 before Com = begin_ph.
In step F206, the document processing apparatus 1 uses the CPU 13 to replace the reading with correct reading based on the reading attribute information. That is, the document processing apparatus 1 removes “(Tanpaku)” based on the reading attribute information of pronunciation = “null”, and based on the reading attribute information of pronunciation = “Rinpakan” and pronunciation = “Sumika”. “Lymph vessel” and “resident” are replaced with “Rinpakan” and “Sumitaka”, respectively.

  The document processing apparatus 1 automatically generates a reading file by performing the processing shown in FIG. 17 in step F102 shown in FIG. The document processing apparatus 1 stores the generated reading file in the RAM 14.

Next, processing using the reading file in step F103 in FIG. 16 will be described.
The document processing apparatus 1 performs processing suitable for the speech synthesis engine stored in advance in the ROM 15, the HDD 34, or the like using the read-out file under the control of the CPU 13.
Specifically, the document processing apparatus 1 selects a speech synthesis engine to be used based on attribute information “Com = Lang = ***” embedded in the reading file.
The speech synthesis engine is assigned an identifier according to the type of language or male / female voice, and the information is recorded in the HDD 34 as an initial setting file, for example. The document processing apparatus 1 refers to the initial setting file and selects a speech synthesis engine having an identifier corresponding to the language.

The document processing apparatus 1 converts Com = begin _ *** embedded in the reading file into a format suitable for the speech synthesis engine.
For example, the document processing apparatus 1 marks Com = begin_ph with a number in the 10,000 range, such as Mark = 10000. Also, Com = begin_s is marked with a number in the 1000 range such as Mark = 1000, and Com = begin_p is marked with a number in the 100 range such as Mark = 100.
This means that the heads of <phrase>, <sentence>, and <paragraph> are indicated by numbers in the 10,000s, 1000s, and 100s, respectively, and this mark indicates <phrase>, <sentence>, and <paragraph>. The head of> can be identified.
Furthermore, since the volume attribute information is expressed as a percentage of the default volume such as Voll = *** in the reading-out file, the document processing apparatus 1 uses the attribute information as a percentage information. Is converted into absolute value information.

  In step F103 shown in FIG. 16, the document processing apparatus 1 converts the read-out file into a format that allows the speech synthesis engine to read the document by performing processing using such a read-out file. To be.

Next, the operation using the user interface in step F104 in FIG. 16 will be described.
The document processing apparatus 1 activates the speech synthesis engine when the user operates, for example, the mouse of the input unit 20 and clicks the reading button 306c shown in FIG.
Then, the document processing apparatus 1 displays a user interface reading window 401 as shown in FIG.

As shown in the figure, the reading window 401 includes a reproduction button 420 for reading a document, a stop button 421 for stopping reading, and a pause button 422 for temporarily stopping reading.
The reading window 401 includes a cue button 411, a fast reverse button 412 and a fast forward button 413 for cuing, fast rewinding and fast forwarding in sentence units, and a head for cuing, fast rewinding and fast forwarding in paragraph units. A search button 414, a fast-reverse button 415, and a fast-forward button 416, and a cue button 417, a fast-rewind button 418, and a fast-forward button 419 are provided for cuing, fast-rewinding, and fast-forwarding in phrase units.

Furthermore, the reading window 401 includes selection switches 423 and 424 for selecting whether a reading target is a full sentence or a summary sentence created as described above.
In addition, an image display area 403 for displaying an image at the time of reading is set, and an image of a person who is reading is displayed, or a telop display area 402 is provided so that characters can be displayed in telop according to the reading voice. Yes.

  Although not shown here, for example, a button for increasing / decreasing the voice, a button for increasing / decreasing the speed of reading, a button for changing the voice of male / female voice, and the like may be provided.

The document processing apparatus 1 performs a reading operation by the speech synthesis engine in response to the user clicking / selecting these various buttons / switches by operating, for example, the mouse of the input unit 20 or the like.
For example, the document processing apparatus 1 starts reading a document when the user clicks the play button 420. Specifically, the control unit 11 supplies the voice signal generated by the voice synthesis process to the voice output unit 33 and outputs it as voice.
The document processing apparatus 1 stops or pauses the reading process when the stop button 421 and the pause button 422 are clicked.

In addition, when the user presses the cue button 411 during reading, the user jumps to the head of the currently read sentence and reads it again. Similarly, for the cue buttons 414 and 417, jump to the head of the paragraph or phrase currently being read out and read out again.
The control unit 11 recognizes the jump destination with the above-mentioned mark regarding the operation of the cue buttons 411, 414, and 417. That is, when the cue button 411 relating to a sentence is operated, the control unit 11 searches the mark in the 1000s by going back from the middle position of the sentence currently being read out. When a mark in the 1000s is detected, reading is resumed from there. In the case of paragraphs and phrases, the 100th and 10,000th marks are searched for and the reading is resumed.
By such processing, for example, it is possible to respond to a request that a user wants to repeatedly reproduce a desired portion in a document.

In step F104 in FIG. 16, the document processing apparatus 1 reads a document by the speech synthesis engine in response to the user performing an operation using the user interface in the reading window 401 as described above.
In this way, the document processing apparatus 1 can read out a desired document without a sense of incongruity by the speech synthesis engine.

  By the way, the document to be read out may be the text of the document data or a summary sentence generated as described later. Whether the body text or the summary sentence is selected by clicking the selection switches 423 and 424. In any case, the processing of steps F102 and F103 in FIG. 16 is performed on the tag file as the body text or the summary sentence. The text-to-speech can be read by the speech synthesis engine.

In this example, the reading file is generated from the received or created tag file. However, the reading file may be directly read based on the tag file without generating such a reading file.
In this case, the document processing apparatus 1 uses the speech synthesis engine after receiving or creating the tag file, and based on the tag indicating the paragraph, sentence, and phrase attached to the tag file, Read aloud with a predetermined rest period. In this way, the document processing apparatus 1 can directly read out based on the tag file without generating a voice reading file.

6). Summary Creation / Display Process Next, a process for creating a summary sentence for document data will be described.
As described above, the user can browse the text of the document by selecting the document and opening the viewing window 301 as shown in FIG.
In the browsing window 301, the text and summary of the selected document are displayed.
However, when no summary sentence is created, the summary display section 304 is blank as shown in FIG.

When the summary creation button 306a is clicked in the browsing window 301, a summary sentence about the document displayed on the document display unit 303 is created and displayed on the summary display unit 304 as shown in FIG.
That is, the control unit 11 activates a summary creation engine (described later with reference to FIG. 47) configured by software or hardware, and performs summary sentence creation processing as described below according to the user's summary creation operation. After creation, control to display it is performed.
The process of creating a summary from a document is performed based on the internal structure by tagging the document.

The summary sentence is usually generated according to the size of the summary display unit 304. The area of the text display unit 303 and the summary display unit 304 can be changed by the user moving the partition frame 312.
That is, the summary sentence is created in a size (document length) corresponding to the size of the summary display unit 304 at the time when the summary creation is instructed.
For example, in FIG. 26, the summary display unit 304 is very narrow, but when the user widens the summary display unit 304 by moving the partition frame 312 and instructs the creation of the summary, for example, FIG. A summary sentence having a document length corresponding to the area size is created and displayed in the expanded summary display section 304.

However, for example, when the summary display unit 304 is very narrow as shown in FIG. 26 and a summary sentence corresponding to the size is created, the summary sentence often has insufficient contents. That is, since a summary sentence cannot be created with a sufficient document length, the sentence does not substantially function as a summary.
Therefore, in this example, the window size (document length that can be displayed in the window) as the summary display unit 304 is less than a predetermined value, for example, less than about 1 to 2 lines in terms of the number of lines on the screen as shown in FIG. In some cases, a summary sentence having a document length exceeding the size of the summary display unit 304 is created, and is displayed as a telop by a telop creation engine (described later in FIG. 47) configured by software or hardware. Like to do.

  Hereinafter, a summary sentence creation and display operation of this example including such a telop display will be described.

  When the summary creation button 306a is clicked on the browsing window 301, the summary creation processing of FIG. 23 is started by the control unit 11 (summary creation engine). When the summary creation process of FIG. 23 is completed, the summary sentence display process of FIG. 24 is subsequently executed. In some cases, the telop display process of FIG. 25 is performed during the summary sentence display process.

In the summary creation process of FIG. 23, first, the control unit 11 performs active diffusion as step F81. In this example, documents are summarized by adopting the central activity value obtained by active diffusion as the importance. That is, in a document given an internal structure by tagging, a central activity value corresponding to the internal structure by tagging can be given to each element by performing active diffusion.
The active diffusion process performed in step F81 is the same as that described with reference to FIGS. 6 to 8. However, as described above, the active diffusion is also applied to elements having a high central activity value and high central activity. It is processing that gives a value. That is, active diffusion is an operation on a central activity value in which the central activity value is equal between the anaphoric (coreference) expression and its antecedent, and otherwise the central activity value is attenuated. Since this central activity value is determined according to the internal structure by tagging, it can be used for the analysis of documents in consideration of the internal structure by tagging.

Next, in step F82, the size wsy of the summary display unit 304 at that time is confirmed, and it is confirmed whether the summary display unit size wsy is equal to or larger than a predetermined size wsth set in advance.
The predetermined size wsth is normally set as a window size capable of displaying a sufficient document length (number of characters) as a summary sentence. As the specific value as the predetermined size wsth, an appropriate value is determined in system design.

First, a case where the summary display unit size wsy is equal to or larger than a predetermined size wsth set in advance will be described. This is processing when a summary sentence corresponding to the summary display unit size wsy is created.
In this case, the process of the control unit 11 proceeds to step F83, and the control unit 11 determines the size wsy of the summary display unit 304 of the browsing window 301 currently displayed on the display unit 30 (specifically, this summary display). The maximum number of characters that can be displayed in the section 304 is set as a variable (maximum number of characters) ws.
Further, the control unit 11 initializes the character string of the summary (internal register holding the summary sentence) s and sets the initial value s (0) = “”. The control unit 11 records the maximum number of characters ws and the initial value s (0) of the character string s thus set in the RAM 14.

In step F85, the control unit 11 sets the count value i of the counter that counts the sentence skeleton extraction processing to “1”.
In step F86, based on the count value i of the counter, the control unit 11 extracts the skeleton of the sentence having the i-th highest average central activity value from the sentence.
The average central activity value is an average of the central activity values of the elements constituting one sentence.
For example, the control unit 11 reads the character string s (i-1) recorded in the RAM 14, adds the character string of the extracted sentence skeleton to the character string s (i-1), and performs S (i) And Then, the control unit 11 records the character string s (i) thus obtained in the RAM 14.
Since the character string s (i−1) is the initial value s (0) at the first time, the skeleton of the sentence extracted this time is stored in the RAM 14 as the character string S (i).
Further, when the process of step F84 is performed thereafter, the skeleton of the extracted sentence becomes the character string S (i), and the character string S (i) up to that point (that is, the character string S (i-1) at that time). ) Will be added.
At the same time, in step F84, the control unit 11 creates a list L (i) in the order of the central activation values of elements not included in the skeleton of the sentence, and records this list L (i) in the RAM 14.

  That is, in this step F86, the summary algorithm selects sentences in descending order of the average central activity value using the result of activity diffusion, and extracts the skeleton of the selected sentence. The skeleton of the sentence is composed of essential elements extracted from the sentence. The required elements can be the element head, subject, object, indirect object, possessor, cause, condition Alternatively, an element having a relation attribute of comparison and an element directly included when the coordinate structure is an essential element constitute an essential element. Then, the skeleton of the sentence is generated by connecting the essential elements of the sentence and added to the summary.

In step F87, the control unit 11 determines whether the length of the character string s (i) is larger than the maximum number of characters ws of the summary display unit 104 of the viewing window 301.
This step F87 is a determination process for creating a summary sentence according to the size of the summary display unit 304.

When the length of the character string s (i) does not reach the maximum number of characters ws, the control unit 11 advances the process to Step F88.
In step F88, the control unit 11 has the highest central activity value of the element of the sentence having the highest average central activity value in the document (i + 1) and the list L (i) created in step F86. Compare the central activity values of the elements.
That is, the sentence having the next highest average central activity value after the sentence adopted as the summary in step F86 (that is, the sentence to be added to the summary sentence next) and the sentence adopted as the summary in step F86. Compare the central activity values of elements that were excluded from the summary as not skeleton.

  In the process of step F88, it is determined whether the part to be added next to the character string as the summary sentence is selected from those not adopted as the skeleton in the sentence adopted in the immediately preceding step F86 or from other sentences. It becomes processing to do.

If the central activity value in the list L (i) has a higher central activity value than the central activity value of the element in the sentence with the (i + 1) th average central activity value, the summary sentence Next, the part to be added to the character string is selected from those not adopted as the skeleton in the sentence adopted in the immediately preceding step F86.
Therefore, the process of the control unit 11 proceeds to step F90, and the element having the highest central activation value in the list L (i) is added to the character string S (i) stored at that time, and the character string SS (i) And
At this time, the element added to the character string SS (i) is deleted from the list L (i).
In step F91, it is determined whether or not the character string SS (i) is larger than the maximum number of characters ws. If not, the process returns to step F88.

In step F88, if there is an element whose central activity value is higher than the highest central activity value in the list L (i) as the element of the sentence having the highest average central activity value in (i + 1) th, the character as the summary sentence The next part to be added to the column is selected from a sentence different from the sentence adopted in the immediately preceding step F86, and the count value i is incremented in step F89, and the process returns to step F86.
That is, in step F88, the skeleton is extracted in step F86 for the sentence that is the (i + 1) th highest average central activity value, and is added to the character string S (i).

  As described above, the character string S is added in step F87 or step F91 while adding to the character string based on the element having the high central activity value as the element that becomes the skeleton of the sentence in step F86 or step F90 or other elements. By comparing (i) or SS (i) with the maximum number of characters ws, a character string that is close to the maximum number of characters ws but does not exceed the maximum number of characters ws is created.

For example, when the character string S (i) exceeds the maximum number of characters ws in step F87, the process of the control unit 11 proceeds to step F92, and the character string S (i-1) before adding the skeleton in the immediately preceding step F86 is used. A summary sentence.
That is, since this has exceeded the maximum number of characters ws by adding the sentence skeleton in step F86, the character string S (i-1) before adding the skeleton is close to the maximum number of characters ws. Is a character string that does not exceed the maximum number of characters ws, and is used as a summary sentence.

For this reason, when the character string S (i) is generated for the first time in step F86 (i = 1 time point) and the character string S (i) exceeds the maximum number of characters ws in step F87, the character string S Since (i-1) is the character string S (0) as the initial value set in step F83, a summary sentence could not be substantially created.
This is because the size of the summary display unit 304 is too small. Therefore, the user expands the area of the summary display unit 304 on the screen and clicks the summary creation button 306a again to display the summary display unit 304 in FIG. The process may be started.
However, in this example, it is determined in step F82 that the size of the summary display unit 304 is too small. In such a case, as will be described later, the document length is longer than the document length that can be displayed with the size of the summary display unit 304. Therefore, there is almost no situation where the summary sentence cannot be created.

If the character string S (i) does not exceed the maximum number of characters ws in step F87, the process of the control unit 11 proceeds to step F88 as described above, and the next portion to be added to the character string is determined.
If the process proceeds to step F91 as described above, it is determined whether or not the character string SS (i) exceeds the maximum number of characters ws.
If the character string SS (i) exceeds the maximum number of characters ws, the process of the control unit 11 proceeds to step F93, and the character string S (i) before adding a certain element in the immediately preceding step F90 is summarized. It will be a sentence.
That is, since the maximum number of characters ws has been exceeded by adding an element in step F90, the character string S (i) before adding the element is close to the maximum number of characters ws but the maximum number of characters ws. It is determined that the character string does not exceed, and is used as a summary sentence.

The above description of the processing has been made assuming that the process proceeds from step F82 → F83 → F85. Therefore, when the processing of FIG. 23 is finished, a summary sentence having a document length that matches the size of the summary display unit 304 is created. The contents of the summary sentence are those using one or more sentence skeletons having a high average central activity value and elements having a high central activity value other than the skeleton.
The summary sentence created in this way is stored in the RAM 14, and the summary sentence as a whole is fixedly displayed on the summary display unit 304 as shown in FIG.

That is, when the summary creation process of FIG. 23 is completed, the process of the control unit 11 proceeds to the summary sentence display process of FIG. As the summary sentence display process, first, in step F94, it is confirmed whether or not the size wsy of the summary display unit 304 matches the maximum number of characters ws at that time. That is, the maximum number of characters ws used in the immediately preceding summary creation process determines whether or not the size wsy of the summary display unit 304 is substituted in step F83.
The case of wsy = ws is a case where a summary sentence suitable for the size of the summary display unit 304 at that time is created by the above processing. In this case, the process advances to step F96 to display the created summary sentence on the summary display unit 304.
As a result, the entire summary sentence is fixedly displayed as shown in FIG. That is, the user sets the size of the summary display unit 304 in advance and then instructs the creation of the summary, so that the summary of the document length corresponding to the size of the summary display unit 304 is fixed (that is, switched as telop). (Without moving).

If the user wants to see a more detailed summary sentence by looking at the displayed summary sentence, or wants to see a shorter summary sentence, the size (area) of the summary display portion 304 of the viewing window 301 is displayed. ) And the summary creation button 306a may be clicked again.
Then, as long as the size wsy of the summary display unit 304 is equal to or larger than the predetermined size wsth, a summary of a document length corresponding to the size of the summary display unit 304 at that time is created and displayed by the above-described processing of FIG. Will be.

  Incidentally, for example, when the user clicks the summary creation button 306a and the summary creation processing of FIG. 23 is started in a state where the summary display unit 304 is very small as shown in FIG. When it is determined that the size wsy of the summary display unit 304 is less than the predetermined size wsth, the following processing is performed.

In this case, the process of the control unit 11 proceeds to step F84 and substitutes a preset fixed value wsfix as the maximum number of characters ws. The fixed value wsfix is a value corresponding to the number of characters set as a minimum necessary for the summary sentence, and is set to an appropriate value in system design. For example, it is good also as wsfix = wsth.
Furthermore, the fixed value wsfix = (wsfix−i) × V, the fixed value initial value wsfix−i is held, and the user can arbitrarily input the value of the coefficient V, so that the user can input the fixed value wsfix. It may be set arbitrarily.

  In addition, the control unit 11 initializes a summary character string (an internal register holding a summary sentence) s and sets an initial value s (0) = “”. The control unit 11 records the maximum number of characters ws and the initial value s (0) of the character string s set in this way in the RAM 14.

Subsequently, the control unit 11 sets the variable i = 1 in step F85, and then proceeds to the processing after step F86. However, since this is as described above, redundant description is avoided.
That is, also in this case, a summary sentence is created as the number of characters closest to the maximum number of characters ws within a range not exceeding the maximum number of characters ws. However, in this case, since the maximum number of characters is ws = wsfix, the number of characters does not correspond to the size of the summary display unit 304 at that time, and actually the number of characters exceeding the number of characters that can be displayed on the summary display unit 304 A summary sentence of (document length) is generated.

When the summary sentence is created by the processes of steps F86 to F93, the control unit 11 executes the summary sentence display process of FIG.
In this case, since the maximum number of characters ws is not the summary display portion size wsy, the process proceeds from step F94 to F95, and the generated summary sentence is displayed as a telop using the telop creation engine.

An example of the telop display process in step F95 is shown in detail in FIG.
First, the control unit 11 (telop creation engine) sets variable n = 1 in step F501. In step F502, the nth sentence is extracted from the generated summary sentence. Therefore, the first sentence constituting the summary sentence is first extracted.

Next, in step F503, it is determined whether or not the sentence length Ls (number of words) of the extracted sentence exceeds the summary display unit size wsy.
For example, if the first sentence of one or more sentences constituting the summary sentence is less than or equal to the number of words that can be displayed in the summary display part size wsy at that time, the process of the control unit 11 proceeds to step F504. move on. Then, the telop display time T1 is calculated by multiplying the sentence length Ls by a predetermined coefficient k.

Subsequently, in step F505, the control unit 11 starts counting of the internal timer, and displays the first sentence on the summary display unit 304 in step F506. At this time, since the sentence length Ls is equal to or smaller than the summary display unit size wsy, the entire first sentence is displayed on the summary display unit 304 as a telop.
During this display operation, the control unit 11 monitors the time count value in Step F507, and proceeds to Step F515 when the T1 time is counted by the timer. In step F515, it is determined whether or not the display of the last sentence of the summary sentence has been completed. If not, the variable n is incremented in step F516, and the process returns to step F502. Therefore, the second sentence among the sentences constituting the summary sentence is extracted next.

Subsequently, the control unit 11 performs the determination of Step F503 for the second sentence. Here, if the sentence length of the second sentence exceeds the summary display unit size wsy (that is, the entire sentence cannot be displayed at once). ), The process of the control unit 11 proceeds to step F508, and the telop display time T1 is calculated by multiplying the sentence length Lwsy that can be displayed on the summary display unit by a predetermined coefficient k.
Subsequently, in step F509, the control unit 11 starts counting the internal timer, and in step F510, displays the second sentence on the summary display unit 304 as a telop. However, at this time, since the sentence length Ls exceeds the summary display part size wsy, only the displayable top part is displayed. That is, the part of the second sentence corresponding to the sentence length Lwsy from the top is displayed.
During this display operation, the control unit 11 monitors the time count value in step F511, and proceeds to step F512 when the T1 time is counted by the timer.

In step F512, the displayed sentence is scrolled by one character. In step F313, the process waits for T2. Then, in step F514, it is determined whether or not the last character up to the second minute has been displayed. If the last character has not been reached yet, the process returns to step F512 to scroll by one character.
The T2 time as the standby time is a time that defines the scroll timing, and this is also a value set in the system design. For example, when about 0.5 to 1 second, the sentence is scrolled at an appropriate speed. Will go.

  As for the sentence that cannot be displayed on the summary display unit 304 by the processes of steps F508 to F514, the head part is first displayed for T1 hours, and then the display is scrolled by one character every T2 hours. It will be displayed up to the last character of the sentence.

When the scroll has displayed up to the last character of the sentence, the control unit 11 proceeds from step F514 to F515 to determine whether or not the display of the last sentence of the summary sentence has been completed. In step F516, the variable n is incremented, and the process returns to step F502. Accordingly, the third sentence of the sentences constituting the summary sentence is extracted next.
And about the 3rd sentence, a telop display is performed by the process of step F504-F507 or step F508-F514.

  If it is determined in step F515 that display has been completed for all sentences constituting the summary sentence, the process proceeds to step F517 to display the head part of the summary sentence. In other words, a displayable head portion is displayed in the summary display portion size wsy. In step F518, a scroll bar 307, which will be described later, is displayed, and the telop display process is terminated.

The display operation realized by the telop display processing as described above is as follows.
For example, assume that the summary creation button 306a is clicked in the browsing window 301 in which the summary display unit 304 is very narrow as shown in FIG.
At this time, a summary sentence longer than the document length that can be displayed on the summary display unit 304 is created by the above-described processing of FIG.

For example, assume that the following summary sentence is generated for the text displayed in the text display unit 303 in FIG.
“[Nicely Aging] / 8 Cancer metastasis can be suppressed! What characterizes cancer is cell proliferation and metastasis. In human cells, there are“ oncogenes ”and“ cancer suppressor genes ”. is there. In the elderly, normal regulatory functions are lost over a long period of time, mutations that do not work within the cell accumulate, the proportion of cells increases, and cancer occurs frequently. Metastasis does not occur just by increasing the number of cancer cells. Cancer cells dissolve proteins, create a way, and enter blood vessels and lymphatic vessels. Expectations (nm23) are gathered from the viewpoint of cancer diagnosis and treatment. "

Such a summary sentence is composed of seven sentences [1] to [7] as follows. In the above-described telop display process of the present example, the telop is converted into sentences.
[1] [Nicely Aging] / 8 Cancer metastasis can be suppressed! ?
[2] It is cell proliferation and metastasis that characterize cancer.
[3] Human cells include “oncogenes” and “cancer suppressor genes”.
[4] In the elderly, normal regulatory functions are lost over a long period of time, mutations that do not work within the cell accumulate, and the proportion of cells increases, leading to more cancer.
[5] Metastasis does not occur just by increasing the number of cancer cells.
[6] Cancer cells dissolve proteins, create a way, and enter blood vessels and lymphatic vessels.
[7] Expectations (nm23) are gathered from the viewpoint of cancer diagnosis and treatment.

25, first sentence [1] is first extracted and displayed on summary display unit 304 as shown in FIG.
The telop display state of the summary display unit 304 after the state of FIG. 25 will be described with reference to FIGS.

Since the sentence [1] in FIG. 28 has the sentence length Ls ≦ wsy (the entire sentence can be displayed), the display state in FIG. 29A is the sentence length in the processing of steps F504 to F507 in FIG. It continues for T1 time calculated according to.
Subsequently, since the sentences [2] and [3] also satisfy the sentence length Ls ≦ wsy, in the processes of steps F504 to F507, respectively, for each T1 time calculated according to the sentence length, as shown in FIGS. 29 (b) and 29 (c). Display is executed.

Since the next sentence [4] has the sentence length Ls> wsy (the entire sentence cannot be displayed), the processes of steps F508 to F514 in FIG. 25 are performed.
That is, first, as shown in FIG. 29 (d), after the head portion of the sentence is displayed for the calculated T1 time, the display is scrolled by one character as shown in FIG. 29 (e) → (f) every T2 time. Is done.
Thereafter, scrolling proceeds in the same manner, and the display of sentence [4] is completed when the state shown in FIG.

  Subsequently, sentences [5], [6], and [7] are sequentially extracted, and since these are sentence length Ls ≦ wsy, T1 time calculated according to the sentence length in the processes of steps F504 to F507, respectively. The display of FIG. 29 (h) (i) (j) is performed one by one.

As described above, each sentence is sequentially displayed as a telop, and when the sentence length is too long to be displayed, the part that could not be displayed is displayed by scrolling.
When the telop display of all the sentences constituting the summary sentence is completed, as shown in FIG. 30, the head part of the summary sentence is displayed and the scroll bar 307 is displayed as steps F517 and F518 in FIG.
That is, after the entire summary sentence is presented to the user by the telop display, the scroll bar 307 is prepared, and then the user arbitrarily scrolls the characters on the summary display unit 304 and confirms the contents of the summary sentence. It is something that can be done.

Creation and display of the summary sentence of this example are executed as described above, and this makes it possible to present a summary suitable for the user.
First, the user can set the document length of the generated summary sentence by arbitrarily setting the size of the summary display unit 304.
That is, when a user wants to see a detailed summary sentence, the summary display section 304 is widened and the summary creation button 306a is clicked. On the other hand, when a summary sentence with simple contents is desired, the summary display section 304 is narrowed and then the summary creation button 306a Just click. As a result, a summary sentence having a size desired by the user is generated and displayed in a fixed manner instead of a telop.

Also, if the user makes the summary display section 304 too narrow, a summary sentence of the minimum required document length is created regardless of the window size and displayed as a telop as described above. become.
Therefore, for example, even if the summary display unit 304 is inevitably narrowed because the user wants to make the text display unit 303 wide, the user can view a summary sentence with appropriate contents. As a result, the contents can be accurately recognized.

In the telop display processing of this example, the summary sentence is converted into telops in units of sentences. Since the display time (T1) of one sentence is calculated according to the sentence length, each sentence (telop) is , The display time is appropriate for the user to read.
Furthermore, when the sentence is long, it is suitable for the user to recognize one sentence because the scroll display is executed (since the subsequent part is not displayed as a switching display but is moved and displayed, one sentence continues. It ’s easy for users to recognize).

Note that examples of the telop display process are not limited to the example of FIG.
For example, the display may be teloped not in sentence units but in units of phrases, phrases, etc., or the display delimiter may be changed according to the length of sentences or phrases.
It is also possible to scroll the entire summary sentence from the beginning to the end.
In any case, the entire summary sentence may be presented to the user by switching display, moving display, or a combination thereof.

Various setting of the display period, switching timing, and scroll timing as each telop is conceivable.
As described above, in this example, the text and the summary text can be read out, but the timing of the telop display performed by the telop creation engine can be set in accordance with the output timing of the read-out voice.
For example, as described above, in the reading process, a file for reading is generated from the tag file, the reading file is converted into a form suitable for the voice synthesis engine, and then a natural voice that is spoken by the voice synthesis engine. Reading is performed according to the speed of feeling and the separation of paragraphs, sentences, and phrases.
Therefore, by setting such paragraph, sentence, and phrase divisions, telop divisions similar to the reading speed, and setting the display period, it is possible to realize telop display that is easy to read for the user.
As a specific example of the technique, as described above, the speech synthesis engine performs the reading process (but does not execute the output of the reading voice), and the CPU 13 refers to the reading timing and determines the telop display division and the switching timing. You only have to set it. For example, during a period when a speech is being read out by the speech synthesis engine (actually no speech is output), the character string as the phrase is displayed as a telop, and when the read-out speech advances to the next phrase Then, the process switches to the telop display of the next phrase.
A configuration for realizing the linkage between the speech synthesis engine and the telop creation engine will be described later with reference to FIG.

7). 7-1 Video File Output Processing 7-1 Video File and Tag The above description is about processing mainly for document data in the document processing apparatus 1 of this example. In the document processing apparatus 1 of this example, the document data The video data related to can be output. Specifically, the document data displayed in the browsing window 301 or the video data related to the portion (sentence, phrase, word, etc.) designated by the user in the document data is output, or the above-described classification is performed. Video data related to the document data specified in the window 201 can be output, and further, video data specified in the video list window described later can be output.
Hereinafter, various processes relating to video data will be described.

First, a data file as video data (hereinafter referred to as a video file) and a tag structure attached to document data for associating the video file with document data will be described.
As described above, for example, when document data is captured from the communication unit 21 or the recording medium 32 into the document processing apparatus 1, a video file may be captured at the same time. A video file may be input separately from document data.
As described with reference to FIGS. 4 and 12, the document data and the video file taken into the document processing apparatus 1 are stored in the RAM 14 or the HDD 34. The control unit 11 retrieves the stored document data as necessary, and can display the text of the document, create and display the summary, and read out as described above. The video file corresponding to the designated document data or a part of the designated document data can be taken out and displayed.

FIG. 31 shows an image of document data and a video file stored in the document processing apparatus 1.
As document data, that is, tag files, various document data such as tag files TG1, TG2, TG3,.
On the other hand, various files such as video files V1, V2,... Are stored as video files.
The contents of the video file are various, and are created arbitrarily by the server supplying document data and video data, respectively. For example, the video file V1 includes moving image data VM1 to VM4 as a plurality of scenes. The video file V3 includes two moving image data VM5 and VM6, and the video file V5 includes one moving image data VM9.
The video file V2 includes a plurality of still image data VS1 to VS7.
The video file V4 includes a plurality of still image data VS8 and VS9 and a plurality of moving image data VM7 and VM8.
The video file V6 includes moving image data VM10 to VM17 as a plurality of scenes.
Of course, there are various other video files.
The video data is, for example, MPEG data.

The moving image data is assumed to include audio data. However, the audio data is basically not included in the still image data, but audio data related to the still image video may be added.
Note that the audio data here is audio data recorded corresponding to an image as a moving image or a still image, and audio data in which the text of the related document data or the content of the summary sentence is read out. (It is not a read-out voice signal generated by the voice synthesis process as described above, and is not a voice signal having the same content as the read-out voice).
That is, the audio data refers to audio data accompanying the video data, and does not directly match the document data. For example, it is audio data included in normal video data, such as ambient sounds, sound effects, announcements / descriptions, and the like when recording recorded video. (However, depending on the intention of the information provider, it is possible to add audio data having the content of reading the document data as it is to the video data corresponding to the document data.)

Each video file has a time code TC, for example, hour / minute / second / frame attached to each video file. For example, in the video file V1, as shown in the figure, a time code TC from “00: 00: 00: 00” to “00: 03: 39: 05” is attached from the first frame to the last frame.
This time code TC functions as an address of each part in the video file. For example, by specifying a time code, an arbitrary portion (arbitrary section) can be reproduced. For example, when reproducing the moving image data VM2 in the video file V1, the time code of the start frame and the end frame of the section as the moving image data VM2 may be specified.

As described above, each video file is composed of one or a plurality of moving image and still image data. Document data (tag file) is a video file name, each moving image data, A tag for designating still image data is attached.
That is, as a tag file, related video data can be specified by a tag in an arbitrary unit such as an entire document, a paragraph, a sentence, or a phrase. Video data related to document data can be determined and output based on a designated tag (referred to as a video tag).

Examples of tag files with video tags are shown in FIGS. 32, 33, and 34. FIG.
FIG. 32 is an example in which a video tag is added to the tag file having the same contents as those in FIG.
In the case of FIG. 18, <document> to </ document> are attached as tags that define the document range. In this case, as shown in (Example 11), a video indicating a document range tag is used. A tag is added,
<Document video file = “V3”><scene = “VM5” in = “00: 00: 00: 00” out = “00: 00: 03: 12”>... / </ Scene></ document >
It is said that.
By such a tag, it is expressed that the video data related to the entire document data is the moving image data VM5 included in the video file V3 shown in FIG. In addition, in = “00: 00: 00: 00” and out = “00: 00: 03: 12”, the start time (start frame) and end time (end time) of the scene as the moving image data VM5 in the video file V3. Frame), that is, a time code is designated.
With such a tag structure, the control unit 11 can discriminate video data related to the tag file, and can reproduce and output it as necessary.

FIG. 33 shows an example in which a video tag is added to the tag file having the same contents as those in FIG.
In this example, one moving image data corresponding to each sentence in the document data is designated.
First, the tag <document video file = “V6”>... </ Document> indicates that video data related to this document is included in the video file V6.
Each sentence has a tag <sentence> to </ sentence>, and the range thereof is shown. As shown in (Example 12) to (Example 16), each sentence has a tag,
<Sentence = “VM10” in = “00: 00: 00: 00” out = “00: 00: 00: 15”>... </ Sentence>
<Sentence = “VM11” in = “00: 00: 00: 16” out = “00: 00: 0: 28”>... </ Sentence>
<Sentence = “VM12” in = “00: 00: 00: 29” out = “00: 00: 00: 48”>...
<Sentence = “VM13” in = “00: 00: 00: 49” out = “00: 00: 01: 00”>...
<Sentence = “VM14” in = “00: 00: 01: 01” out = “00: 00: 01: 16”>...
That is, it is shown that the moving image data VM10, VM11, VM12, VM13, and VM14 correspond to each sentence.

By such a tag, it is represented that the video data related to each sentence of the document data is the moving image data VM10 to VM14 included in the video file V6 shown in FIG. Further, the start time of the scene as each of the moving image data VM10 to VM14 in the video file V6 by in = “00: 00: 00: 00” and out = “00: 00: 00: 15” in the moving image data VM10 ( A start frame) and an end time (end frame), that is, a time code is specified.
With such a tag structure, the control unit 11 can discriminate video data related to each sentence of the tag file and can reproduce and output it as necessary.

FIG. 34 is an example in which a video tag for still image data is added to each tag in the document data in the tag file having the same content as FIG. 19 described above.
In this case, first, the tag <document video file = “V2”>... </ Document> indicates that video data related to this document is included in the video file V2.
Each sentence has a tag <sentence> to </ sentence>, and its range is shown. As shown in (Example 17) to (Example 21), each sentence has a tag,
<Sentence = “VS1”>... </ Sentence>
<Sentence = “VS2”>... </ Sentence>
<Sentence = “VS3”> ・ ・ ・ ・ ・ ・ </ Sentence>
<Sentence scene = "VS4"> ... </ sentence>
<Sentence = “VS5”> ・ ・ ・ ・ ・ ・ </ Sentence>
That is, it is shown that the still image data VS1, VS2, VS3, VS4, and VS5 correspond to each sentence.

If it is assumed that each video data VS1 to VS7 in the video file V2 has a management structure in which reproduction names can be designated as the data names “VS1”. A video tag specified only by name may be used, but still image data may also be specified by a time code in the video tag as in the case of the moving image data.
Conversely, when the playback section of the moving image data can be specified only by the data name, it is not always necessary to specify the time code as described above.

  32 and 33, the control unit 11 can discriminate video data related to each sentence of the tag file and can reproduce and output it as necessary.

The example of FIG. 32 is an example in which one moving image data is associated with the entire document, and FIGS. 33 and 34 are examples in which moving image data or still image data is associated with each sentence of the document. Video data can be associated with tag files in various variations.
That is, if a video tag is added to tags such as <paragraph> to </ paragraph>, <** phrase> to </ ** phrase>, a certain video data can be stored in an arbitrary unit such as a paragraph unit or a phrase unit. It is possible to associate.
In addition, it is naturally possible to associate a plurality of moving image data within a single tag file, or to associate still image data and moving image data in a complex manner.

  By the way, when actually outputting video data, a video output file is generated from the tag file of the document data, and a video engine described later executes a video reproduction operation using the video output file.

A video output file is a file generated by extracting a video tag from a tag file, as shown in FIGS. 35 (a), (b), (c), and (d), for example.
FIG. 35A shows a video output file generated from the tag file of FIG. 32. In this case, the above-described video tag is extracted and becomes the file content specifying the moving image data VM5 in the video file V3 as shown. .
FIG. 35B shows a video output file generated from the tag file of FIG. 33. In this case as well, the video tag is extracted and becomes the file contents for designating the moving image data VM10 to VM14 in the video file V6 as shown. .
FIG. 35C shows a video output file generated from the tag file shown in FIG. 34. In this case, the file contents specify still image data VS1 to VS5 in the video file V2, as shown in the figure.

FIGS. 35A, 35B, and 35C are examples in which all video tags in the tag file are extracted to generate a video output file. However, some video tags are extracted and used for video output. Sometimes a file is generated.
For example, FIG. 35D shows a video output file in which a part of video tags is extracted from the tag file of FIG. 33, and the moving image data VM11 in the video file V6 is designated as shown in FIG. As will be described later, when a user specifies a part of a sentence and requests a video output, a video tag relating to the specified part is extracted in this way to form a video output file.

The video engine performs processing to display and output the video data designated by the video output file formed as in these examples.

7-2 Video output form

In this example, it is possible to output video data related to the document data displayed in the browsing window 301 or a portion (sentence, phrase, word, etc.) designated by the user in the document data.
Also, video data related to the document data specified in the classification window 201 can be output.
Furthermore, it is possible to output video data for document data (or video data) designated in a video list window described later.
These video output forms will be schematically described first with reference to FIG. Specific processing examples and display examples in each output form will be described later.

As described above, the reading window 301 is opened on the display unit 30 or the classification window 201 is opened on the display unit 30 in response to taking in document data / video data or a user operation.
Also, the video list window described later with reference to FIG. 46 is opened by a user operation from the browsing window 301 or the classification window 201 (such as clicking on the video list buttons 306e and 202e described above).
In FIG. 36, the states in which these windows are opened are schematically shown as S1, S2, and S3.

When the viewing window 301 is open, clicking the video button 306d outputs video data relating to the displayed document data (S1 → S4 → S5).
This is a case where the user clicks the video button 306d without specifying a part of the text when the text (or text and summary text) of the document data is displayed in the browsing window 301. At this time, Outputs video data corresponding to the entire document data being displayed.

  When the user designates a part of the document and clicks the video button 306d while the viewing window 301 is open, video data relating to the designated part is output (S1 → S6 → S7 →). S8).

  Further, when the user inputs a keyword and clicks the video button 306d while the viewing window 301 is open, video data relating to the portion corresponding to the keyword is output (S1 → S9 → S10 → S11). .

In this way, from the viewing window 301, with respect to the displayed document data, the part specified by the specifying operation (or the part corresponding to the input keyword) in the video data or the document data related to the entire document. The video data related to is output.
In each of the states S5, S8, and S11, the video data may be displayed and output on the browsing window 301. For example, the video window 501 described later with reference to FIG. Also good.

Next, in a state in which the classification window 201 as shown in FIG. 9 is opened, the user designates one or more desired titles from the titles of document data classified and displayed in the list. When the video button 306d is clicked, video data relating to the designated document data is output (S2->S12->S13-> S14).
As described above, by specifying the title of the displayed document data from the classification window 201, the video data relating to the specified document data can be viewed. In the state of S14, for example, a video window 501 described later with reference to FIG. 44 is opened to display and output video data.

Next, in a state where a video list window 600 shown in FIG. 46, which will be described later, is opened, the titles displayed in the list (document data titles with associated video data or titles of the video data itself) are displayed. When the user designates one or a plurality of desired titles and performs a reproduction operation, video data (or video data of the titles) related to the document data of the designated title is output (S3 → S15 → S16 → S17).
As described above, by designating the displayed title from the video list window, the video data (or the designated video data) related to the designated document data can be viewed. In the state of S17, video data is displayed and output on the video list window 600.

  In the video list window 600, the list that can be displayed and specified can be, for example, all video files (or moving image data and still image data units) captured in the document output device, or When a large number of moving image data or the like is associated with the document data, each moving image data or the like may be designated within a range related to the document data.

The titles displayed in the list in the video list window 600 and the classification window 201 may be titles actually attached to the document data (or video data) or title sentences created by summarizing the document data. . Furthermore, the top part of the document data may be extracted and used as the title.
Furthermore, keywords can be input in the video list window 600 and the classification window 201, and the video data of the part related to the keywords can be output.

7-3 Video Output Processing from Browsing Window Processing and display examples of the control unit 11 for video output from the browsing window 301 described above will be described with reference to FIGS.
First, the flow of processing of the control unit 11 relating to video output from the viewing window 301 will be described with reference to FIG. 37, and then each video display example corresponding to S5, S11, and S8 of FIG.

For example, in a state where the browsing window 301 as shown in FIG. 27 or the like is open, the control unit 11 clicks the video button 306d in steps F601, F602, and F603 of FIG. Or whether a part of the designation input is performed on the displayed document or a keyword is input.
When it is detected that a part of designation input has been performed on the document displayed in step F602, the control unit 11 stores the designation part in step F604 and presents the designated part to the user. Perform display control.
In addition, when it is detected that a keyword is input in step F603, the control unit 11 stores the keyword in step F605 and performs display control so that a portion corresponding to the keyword is presented in the displayed sentence. Do.

If it is detected in step F601 that the video button 306d has been clicked, the control unit 11 proceeds to processing for video output.
First, in step F606, it is determined whether or not a video tag is added to the currently displayed document data. If not, video data to be output, that is, video data related to the document does not exist. Therefore, the process is finished as it is. In this case, the click operation of the video button 306d by the user is an invalid operation.
Actually, when document data having no video tag is displayed, the video button 306d is displayed in an inactive state to indicate to the user that the video button 306d cannot be operated. You may make it leave.

  If the displayed document data includes a video tag, the control unit 11 branches the process after confirming a part of the sentence in the sentence or a keyword input situation in steps F607 and F608. Accordingly, a video output file is created in steps F609, F610, and F611. As described above, the video output file is a file for executing a video reproduction operation by the video engine in the document processing apparatus 1.

Then, the control unit 11 activates the video engine in step F612, and executes a video display process based on the created video output file from step F613.
For example, as shown in FIG. 38, the video display unit 330 is set on the browsing window 301 to display an image output from the video engine.
At this time, on the viewing window 301, a video bar 331 indicating the position of the video displayed on the video display unit 330, a start position display unit 332 for displaying a time code as a start position of the displayed video, and display A current position display unit 333 that displays a time code as the current playback position (current position) of the displayed video, an end position display unit 334 that displays a time code as the end position of the displayed video, and the like are also displayed. .

  The video engine may be configured by hardware or may be realized by software. When the video engine is realized by software, the application program is stored in advance in the ROM 15, the HDD 34, or the like. A functional configuration including the video engine will be described later with reference to FIG.

  When the video output being executed in step F613 is completed, the control unit 11 returns to the normal viewing window 301 in step F615. That is, the display of the video display unit 330, the video bar 331, the start position display unit 332, the current position display unit 333, the end position display unit 334, and the like shown in FIG. 38 is ended, and the screen state is returned to, for example, FIG. Then, the video display process ends.

An example of video display output realized by such processing is as follows.
First, a display example performed in the flow of S1 → S4 → S5 will be described with reference to FIG.
For example, assume that the user clicks the video button 306d without specifying a part of a certain document part or inputting a keyword when the browsing window is open as shown in FIG.
In this case, the process proceeds in steps F601, F606, F607, F608, and F609 in FIG. 37 to create a video output file for the entire displayed document.

If the tag file of the displayed document data is FIG. 32, the video tag included in the entire tag file is extracted, and a video output file as shown in FIG. 35A is formed. Become.
In step F613, the video engine performs video output processing based on the video output file, whereby video display is executed on the video display unit 330 on the viewing window 301 as shown in FIG.
Since the operation is based on the video output file of FIG. 35A, the video data displayed at this time is the moving image data VM5 of the video file V3.
Of course, if a plurality of moving image data are specified in the tag file, each moving image data is sequentially output. If one or more still image data is specified in the tag file, one or more still image data is sequentially output. Display output.
As the video output progresses, the display of the video bar 331, the start position display unit 332, the current position display unit 333, the end position display unit 334, etc. is updated.
At this time, audio data included in the video data is output from the audio output unit 33 as audio.

  As described above, when the user does not specify a particular part in the document or input a keyword, a video output operation based on all the video tags included in the displayed document data is executed.

Next, a display example performed in the flow of S1, S6, S7, and S8 in FIG. 36 will be described.
For example, when the viewing window is open as shown in FIG. 39, the user clicks the video button 306d after specifying a part of the document by an operation using the mouse of the input unit 20 or the like. Is the case.

In the document displayed as shown in Fig. 39, the user selects "THREE COMPUTERS If the operation of designating the “THAT CHANGED” part is performed, the control unit 11 stores the designated part in steps F602 and F604 of FIG. 37 and presents the designated part to the user as shown in FIG. To do. For example, it is presented by a display method such as reverse display, highlight display, or flashing display.
Thereafter, when the user clicks the video button 306d, the processing of the control unit 11 proceeds in steps F601 → F606 → F607 → F611 in FIG. 37 to create a video output file for the designated portion.

If the tag file of the displayed document data is shown in FIG. 33, the video tag corresponding to the designated portion is extracted from the tag file, and a video output file is formed.
In this case, the video tag corresponding to “THREE COMPUTERS THAT CHANGED” is the sentence “THREE COMPUTERS THAT CHANGED the face of ..launched in 1977”, as seen in Example 13 of FIG. Such a video tag, that is,
<Sentence = “VM11” in = “00: 00: 00: 16” out = “00: 00: 0: 28”>...
Therefore, this video tag is extracted, and a video output file as shown in FIG. 35D is generated.

In step F613, the video engine performs video output processing based on the video output file, whereby video display is executed on the video display unit 330 on the viewing window 301 as shown in FIG.
Since the operation is based on the video output file of FIG. 35D, the video data displayed at this time is the moving image data VM11 of the video file V6.
As the video output progresses, the display of the video bar 331, the start position display unit 332, the current position display unit 333, the end position display unit 334, etc. is updated.
Audio data included in the video data is output from the audio output unit 33 as audio.
In the display example of FIG. 41, a text display unit 303 is set at the bottom of the window so that a designated portion is presented.
If the user designates a certain part in the summary sentence from the state of FIG. 39, the summary display unit 304 may be set in the state of FIG. 41 so that the designated part is presented.

  In this way, when the user designates a certain part in the document, a video output operation based on the video tag related to the designated part is executed.

In this example, video tags were attached in sentence units, but of course, video tags were attached in word units, phrase units, etc., for example, if there is a video tag corresponding only to `` THREE COMPUTERS THAT CHANGED '', Video data (moving image or still image) is output based on the video tag.
If a plurality of moving image data or still image data is specified for the designated portion, each moving image data or still image data is sequentially output.

Next, a display example performed in the flow of S1 → S9 → S10 → S11 in FIG. 36 will be described.
For example, when the browsing window is open as shown in FIG. 39, the user inputs a certain keyword by an operation using the keyboard of the input unit 20 and then clicks the video button 306d. Is the case.

If the user performs an operation of inputting the keyword “computers” on the document displayed as shown in FIG. 39, the control unit 11 stores the keyword in the processing of steps F603 and F605 in FIG. At the same time, for example, as shown in FIG. 42, the keyword and the corresponding portion of the keyword are presented to the user.
That is, the keyword input unit 305 displays the keyword input in response to the input, and presents the corresponding portion of the keyword in the displayed document by a display method such as reverse display, highlight display, or blinking display.
Thereafter, when the user clicks on the video button 306d, the processing of the control unit 11 proceeds in steps F601 → F606 → F607 → F608 → F610 in FIG. 37 to create a video output file related to the keyword corresponding portion.

That is, as in the case of the partial designation described above, a video output file is formed by extracting video tags added in units of sentences, phrases, words, etc. including keywords from the tag file as shown in FIG. To do.
In step F613, the video engine performs a video output process based on the video output file generated in this manner, so that the video display unit 330 displays the video on the viewing window 301, for example, as in FIG. Display (one or more moving images or still images) is performed.
In this case, as shown in the display example of FIG. 41, the text display section 303 (or summary display section 304) is set at the bottom of the window so that the corresponding part of the keyword is presented.

  As described above, when the user inputs a keyword to obtain a video output, a video output operation based on the video tag related to the keyword corresponding portion is executed.

7-4 Video Output Processing from Classification Window Next, processing and display examples of the control unit 11 for video output from the classification window 201 as shown in FIG. 9 will be described with reference to FIGS. This is a video display example corresponding to the flow of S2->S12->S13-> S14 in FIG.

  In a state where the classification window 201 is opened, the user can output video data related to the specified document or documents by selectively specifying the document displayed in a list.

For this operation, the control unit 11 monitors whether the video button 306d is clicked or a document designation input is performed in steps F701 and F702 of FIG.
In this example, when no document is designated, the video button 306d is displayed in an inactive state to indicate to the user that the video button 306d cannot be operated.

  If it is detected in step F702 that the user has made an input for designating a part of the document (one or a plurality of documents) among the documents listed by title on the classification window 201, control is performed. The unit 11 stores the document specified in step F703 and performs display control so that the specified document is presented to the user. For example, as shown in FIG. 9, the designated document is presented by displaying a check mark 214 for the designated document title. At this time, the video button 306d is activated.

If the click of the video button 306d is detected in step F601 after the above designation is made, the control unit 11 proceeds to processing for video output.
First, in step F704, it is determined whether or not a video tag is added to the currently designated document data. If not, video data to be output, that is, video data related to the document does not exist. Therefore, the process is finished as it is. In this case, the click operation of the video button 306d by the user is an invalid operation.
In the process of step F703, when document data that does not have a video tag is designated, the video button 306d may remain in an inactive state.

If the designated document data includes a video tag, the control unit 11 creates a video output file in step F705. That is, video tags included in the entire tag file of the designated document data are extracted, and a video output file as shown in FIGS. 35A, 35B, and 35C, for example, is formed.
Then, the control unit 11 starts the video engine in step F706, and opens a video window 501 as shown in FIG. 44 on the display unit 30 in step F707.

The video window 501 displays a file name display unit 502 that displays the file name of the document data file, a video display unit 503 that displays video data related to the document data with the file name, and a video display unit 503. A video bar 504 indicating the position of the video (playback progress position of the moving image data), a start position display unit 521 for displaying a time code as the start position of the displayed video, and the current playback position of the displayed video (current A current position display unit 522 that displays a time code as a position), an end position display unit 523 that displays a time code as an end position of a displayed video, and the like are displayed.
Further, in order for the user to instruct an image reproduction operation, a play button 505, a pause button 506, a stop button 507, a fast-rewind button 508, a fast-forward button 509, a backward search cue button 510, and a feed direction cue button 511 Etc. are displayed.

When such a video window 501 is opened, the control unit 11 performs processing according to the operation of the user interface as step F708.
For example, according to the video output file as shown in FIG. 35A, playback of the moving image data VM5 of the video file V3 shown in FIG. 31 is designated, but the playback button 505 is clicked in the video window 501. In the case where the video data is received, the control unit 11 causes the video engine to start reproducing the moving image data VM5. As a result, an operation image of the moving image data VM5 is displayed on the video display unit 503. At this time, the audio data included in the moving image data VM5 is output from the audio output unit 33 as audio.

When the pause button 506 is clicked, the control unit 11 pauses the reproduction of the moving image data VM5 by the video engine.
When the stop button 507 is clicked, the control unit 11 stops the reproduction of the moving image data VM5 by the video engine.
When the fast reverse button 508 or the fast forward button 509 is clicked, the control unit 11 causes the video engine to perform the fast reverse or fast forward of the reproduction operation of the moving image data VM5.
When the return direction cue button 510 or the feed direction cue button 511 is clicked, the control unit 11 uses, for example, the preceding and following scenes (or a specific point to be a point) as a playback location of the video data VM5 by the video engine. Return (or send) to the beginning position, and playback from that position is executed.
Further, when the user drags the display portion of the current position on the video bar 504, the video engine 504 is moved in the return direction or the advance direction of the playback portion of the moving image data VM5 by the video engine.

By the way, when the displayed video data is still image data, for example, based on a video output file as shown in FIG. 35C, the operations of the pause button 506, fast reverse button 508, and fast forward button 509 are substantially the same. It may not be displayed because it is meaningless.
In that case, when the playback button 505 is clicked, the display of still image data is started, and when a plurality of still image data is specified by the tag file, the return direction cue button 510 or the feed direction The still image data may be switched in response to the click of the cue button 511.

The user can view the video data in an arbitrary state by such processing according to the user interface operation.
As described above, the user can designate one or a plurality of document data with respect to the document data displayed as a list in the classification window 201 and view the video data related to the document data.

7-5 Video Output Form from Video List Window Next, processing and display examples of the control unit 11 for video output from the video list window will be described with reference to FIGS. 45 and 46. FIG. This is a video display example corresponding to the flow of S3 → S15 → S16 → S17 in FIG.

First, the video list window will be described.
When the user clicks the video list button 202e while the classification window 201 in FIG. 9 is open, or clicks the video list button 306e while the browsing window shown in FIG. 10 is open. The control unit 11 causes the display unit 30 to display a video list window 600 as shown in FIG.

In the video list window 600, a list display unit 601 for displaying a list of video data (or document data having a video tag), a video display unit 602 for displaying video data, and a video display unit 602 display videos. A video bar 603 indicating a position (playback progress position of moving image data), a list operation display unit 604 used for various operations for designation in the list display unit 601, a start position of a displayed video, a current position (current A position display unit 605 for displaying a position), an end position, and the like with a time code.
Further, as buttons for video output, a play button 606, a pause button 607, a stop button 608, a return direction cue button 609, a fast rewind button 610, a fast forward button 611, a feed direction cue button 612, and the like. Is displayed.

  The list display unit 601 displays a list of video data or document data that can be selected by the user according to the titles, etc. Various examples of what kind of data are displayed in what range are considered. It is done.

For example, a list display may be performed in units of document data as in the above classification window. That is, in this case, the document data including the video tag is displayed as a list by title or the like.
Alternatively, a list may be displayed in units of video data designated by one or a plurality of video tags included in certain document data. For example, when proceeding from the viewing window 301 to the video list window 600, a list of video data specified by each video tag included in the document data displayed in the viewing window 301 is displayed.
When the classification window 201 is advanced to the video list window 600 in a state where one or more document data is designated, the video data by each video tag included in the designated one or more document data is displayed. May be displayed in a list.
Furthermore, a list of video data related to document data included in a certain classification item in the classification window 201 may be displayed.
It is also possible to display a list of all displayable video data (video file units or units of one moving image / still image data) regardless of the designation or classification of document data.

As a title on the list, when document data is presented, the title, summary sentence, or head sentence is displayed.
In addition, when presenting the title of the video data itself, if the video data has a title, use the title. If not, use the title of the corresponding document data or a summary sentence as the title. do it.

  From such a list display of the list display unit 601, the user selectively designates desired document data or video data, whereby video data related to the designated document or documents, or designated one or more documents. Video data can be output.

For this operation, the control unit 11 performs processing corresponding to the user's designated operation in step F801 in FIG.
That is, regarding the list display of the list display unit 601, setting or changing the list status according to the operation of the list operation display unit 604 (for example, clearing options, displaying all selectors, reloading the list, clearing the selection state, etc.) And a process according to the user's designation performed by clicking on the option.
That is, in response to the user specifying one or more options (titles), the video data relating to the specified options is stored, and a check mark 620 is displayed for the options as shown in FIG. , To indicate that it has been designated.
The video bar 603 displays the video data corresponding to the designated option.

In step F802, the control unit 11 extracts a video tag related to the currently specified option, and creates a video output file.
That is, a video tag corresponding to the designated video data itself or a video tag included in the whole tag file of the designated document data is extracted, and a video output file as described with reference to FIG. 35, for example, is formed.
In step F803, the control unit 11 starts the video engine, and in step F804, performs processing corresponding to the operation of the user interface.

For example, when the playback button 606 is clicked, the control unit 11 causes the video display unit 602 to start playback of video data (moving image or still image) based on the video output file by the video engine. Of course, the display state of the video bar 603 and the position display unit 605 is also updated as the playback progresses.
The audio data included in the displayed moving image data or still image data is output from the audio output unit 33 as audio.

When the pause button 607 is clicked, the control unit 11 pauses the reproduction of video data by the video engine.
When the stop button 608 is clicked, the control unit 11 stops the reproduction of video data by the video engine.
When the fast reverse button 610 or the fast forward button 611 is clicked, the control unit 11 causes the video engine to perform a fast reverse or fast forward operation for reproducing video data.
When the return direction cue button 609 or the feed direction cue button 612 is clicked, the control unit 11 uses, for example, the beginning of the preceding or succeeding scene (or a specific point to be a point) as a playback position of the video data by the video engine. Return (or send) to a location and play from that location.
Further, when the user drags the display portion of the current position on the video bar 603, the video engine is moved in the return direction or the advance direction of the playback position of the video data by the video engine accordingly.

The user can view the video data in an arbitrary state by such processing according to the user interface operation.
As described above, the user can view the video data related to the designated option by designating one or a plurality of options displayed in the video list window 600.

8). Functional Block Configuration of Document Processing Device Various operations realized in the document processing device 1 have been described above.
In the document processing apparatus 1 of this example, in order to execute these processes, for example, a functional block as shown in FIG. 47 is formed as a software configuration (or a hardware configuration) in the control unit 11 and a file group configuration. Has been.

That is, the above-described speech synthesis engine 701, telop creation engine 702, summary creation engine 703, and video engine 704 are provided. An audio control unit 705 is provided as an audio signal output processing unit for the audio output unit 33, and a display control unit 706 is provided as an image signal output processing unit for the display unit 30.
Further, the user interface 707 is provided with a portion that performs processing for user operations (operations by clicking a mouse as the input unit 20 or keyboard operations) for buttons on various displayed windows.
And the controller 700 which controls these functional blocks is formed.

The files include the reading file 708, the body tag file 709, the summary sentence tag file 710, the video file 711, and the video output file 712 described above.
As described above, the text tag file 709 and the video file 711 are taken from the communication unit 21 or the recording / playback unit 31 (recording medium 32). The summary sentence tag file 710 is generated from the body tag file 709 by the summary creation engine 703.
The reading file 708 is generated by converting the body tag file 709 or the summary sentence tag file 710.
The video output file 712 is generated by being converted from the body tag file 709 or the summary sentence tag file 710 (specifically, a video tag is extracted).

  The processing of the functional block in FIG. 47 for realizing each operation described above will be described below.

Reading process In the reading process described with reference to FIGS. 16 to 22, a reading file 708 is first generated from the text tag file 709 or the summary sentence tag file 710.
The speech synthesis engine 701 refers to the reading file 708 based on an instruction from the controller 700 and performs a speech synthesis process based on the reading file 708.
The generated synthesized speech signal (read-out speech signal) Yout is subjected to output level adjustment in the speech control unit 705, supplied to the speech output unit 33, and output.
Further, the controller outputs an image signal as a reading window 401 from the display control unit 706 and displays it on the display unit 30.
Information about user operations on the reading window 401 is captured by the user interface 707 and transmitted to the controller 700. The controller 700 controls the operation of the speech synthesis engine 701 in accordance with the user operations.

23. Summary Creation Processing In the summary creation processing described with reference to FIG. 23, the controller 700 instructs the summary creation engine 703 to execute summary creation for the body tag file 709. As a result, a summary tag file is formed.
The controller 700 transmits the size information of the summary display unit 304 to the summary creation engine 703, so that the summary generation processing according to the size of the summary display unit 304 is performed as described above.

Summary fixed display processing As described with reference to FIGS. 23, 24, and 27, when the size of the summary display unit 304 is equal to or larger than the predetermined value wsth, the summary sentence tag file 710 having a document length corresponding to the size is displayed. Will be generated and displayed in a fixed manner.
In this case, the generated summary sentence tag file 710 is processed as the display document output Sout by the summary creation engine 703 and supplied to the display control unit 706. Then, the image is combined with the image of the viewing window 300 by the display control unit 706 and displayed on the display unit 30 as shown in FIG.

Summary text / text telop display processing As described with reference to FIGS. 23 to 30, when the size of the summary display unit 304 is less than the predetermined value wsth, a summary tag file 710 having a document length exceeding the size is displayed. Will be generated and displayed as a telop.
In this case, the generated summary sentence tag file 710 is subjected to telop processing by the telop creation engine 702. And it outputs sequentially as telop display document output Tout. The display control unit 706 synthesizes the telop display document output Tout with the image of the viewing window 301 so that the display unit 30 executes the display as shown in FIGS.

-Telop display processing at the reading timing As a modification of the telop display, it has been described that the progress timing of the telop can be matched with the reading voice timing.
In that case, the reading file 708 converted from the summary sentence tag file 710 is used as a shared file of the speech synthesis engine 701 and the telop creation engine 702. (A signal system indicated by a broken line in the figure is formed.)
Based on the control of the controller 700, the speech synthesis engine 701 performs speech synthesis / readout output using the read-out file 708. However, in this case, the voice control unit 705 sets the output level of the generated synthesized voice signal Yout to zero so that no reading voice is output to the user.
The telop creation engine 702 creates a telop using the reading file 708. At this time, the telop creation engine 702 and the speech synthesis engine 701 exchange timing signals with each other. That is, the telop creation engine 702 executes the telop creation / telop display document output Tout based on the timing signal from the reading file 708. As a result, the telop display at the reading speed is executed.
Note that the reading speed (that is, the telop speed in this case) can be changed by the controller 700 changing the setting in the speech synthesis engine 701 based on information from the user interface 707.

Video Output Processing Various video output processing described with reference to FIGS. 31 to 46 is performed by processing of the video engine 704 according to instructions from the computer 700 based on information from the user interface 707.
The video engine 704 refers to the video output file 712 generated from the body tag file 709 or the summary sentence tag file 710, determines video data to be reproduced, and reads the video file 711.
The read video data is processed as a video signal Vout for output by the video engine 704 and supplied to the display control unit 706. Then, the image is combined with the image of the viewing window 301, the video window 501, or the video list window 600 by the display control unit 706 and displayed as described above on the display unit 30 (FIGS. 38, 41, 44, 46, etc.). ).
Also, audio data included in the video data is processed as an audio signal Aout for output by the video engine 704 and supplied to the audio control unit 705 to perform processing such as level adjustment. And it is output from the audio output unit 33 as reproduced audio.
During video output, user operation information for the video window 501, the video list window 600, and the like is captured by the user interface 707 and transmitted to the controller 700. The controller 700 controls the operation of the video engine 701 in accordance with the user operation. To do.

The functional block of FIG. 47 has been described above, but the functional block configuration and operation mode are not limited to this.
FIG. 47 shows only the parts related to the output operation from the display unit 30 or the audio output unit 33 described here. For example, functional blocks for realizing document reception processing, classification processing, etc. are omitted. .

  Although the embodiment of the present invention has been described above, this is merely an example, and various hardware or software configurations and processing examples of the document processing apparatus 1 can be considered.

For example, in the above example, the video output processing of this example is performed in accordance with the operation from the browsing window 301, the classification window 201, or the video list window 600 as shown in FIG. Alternatively, the video list window 600 is used, but the present invention is not limited to such an output form.
For example, the video window 501 may be opened and video output may be performed in response to an operation from the viewing window 301.
Of course, other windows may be provided.

  Various types of designation of output video data are conceivable, such as outputting video data related to a certain part of certain document data by an operation from the classification window 201.

  Further, there are various specific device examples constituting the document processing apparatus 1. For example, when the input unit 20 in the document processing apparatus 1 is taken as an example, not only a keyboard and a mouse but also a tablet, a light pen, an infrared ray, or the like is used. Other devices such as a wireless commander device are conceivable.

  Further, in the embodiment, it has been described that a document or a video file is transmitted to the communication unit 22 from the outside via a telephone line or the like, but the present invention is not limited to this. For example, the present invention can be applied to a case where a document or video file is transmitted via a satellite or the like, and can be read from the recording medium 32 in the recording / playback unit 31 or a document or video file written in the ROM 15 in advance. Good.

In the embodiment, an example of a tagging method for a document has been described. However, the present invention is not limited to this tagging method.
Furthermore, in the above-described embodiment, Japanese and English sentences are exemplified, but it goes without saying that the present invention is not limited to these languages.
Thus, it goes without saying that the present invention can be modified as appropriate without departing from the spirit of the present invention.

Furthermore, in the present invention, it is possible to easily provide the recording medium 32 such as a disk-shaped recording medium or a tape-shaped recording medium in which the electronic document processing program described above is written.
That is, the recording medium of the present invention can be realized. The HDD 34 shown in FIG. 1 can be similarly used as the recording medium of the present invention.
Such a recording medium 32 can provide a program for realizing the document processing method described above. For example, the document processing apparatus of the present invention can be easily realized by using a general-purpose personal computer or the like.
Of course, the recording medium 32 may be an optical disk, a magneto-optical disk, a magnetic tape, a memory card such as a flash memory, a memory chip, or the like in addition to the floppy disk.
Furthermore, the program for realizing the document processing method of the present invention can be provided via network communication such as the Internet, and therefore the present invention can be applied to the program server side or as a recording medium in the communication process. Is.

In addition to an operation control program that realizes the above processing, a recording medium that also records document data and video data to be processed by the program is useful.
That is, when electronic document data is provided from the recording medium to the document processing device 1, the operation processing program and the video file are also provided at the same time, so that the document processing device can handle the document data as described above. The video output process can be executed.

  DESCRIPTION OF SYMBOLS 1 Document processing apparatus 10 Main body, 11 Control part, 12 Interface, 13 CPU, 14 RAM, 15 ROM, 20 Input part, 21 Communication part, 22 Portable terminal reader, 30 Display part, 31 Recording / reproducing part, 32 Recording medium, 33 voice output unit, 34 HDD, 101 window, 201 classification window, 301 browsing window, 303 document display unit, 304 summary display unit, 401 reading window, 501 video window, 600 video list window, 700 controller, 701 speech synthesis engine, 702 Telop creation engine, 703 Summary creation engine, 704 Video engine, 705 Audio control unit, 706 Display control unit, 707 User interface

Claims (6)

Input means for inputting first electronic document data;
Presentation output control means for controlling the output of presentation information for presenting the contents of the first electronic document data;
Designation means for designating a part of the second electronic document data from the second electronic document data subjected to presentation output control by the presentation output control means;
One or a plurality of video data or still image data corresponding to a part of the first electronic document data input based on the input means and the second electronic document data specified by the specifying means A video or still image selection means to select;
A document processing apparatus comprising: a video or still image output control unit that sequentially outputs and controls a plurality of video data or still image data selected by the video or still image selection unit. When the video or still image selection means selects one or more video data or still image data corresponding to the first electronic document data input based on the input means, the first electronic 2. The document processing apparatus according to claim 1, wherein video data or still image data relating to a portion corresponding to the content of the document data is selected.   The video or still image output control means controls whether to output the second electronic document data on an area where the second electronic document data is presented and output, or whether to present and output it on an area where only the second video or still image is displayed. 1. The document processing apparatus according to 1.   The document processing apparatus according to claim 1, wherein the presentation output control means presents a list of a plurality of electronic document data. An input procedure for inputting the first electronic document data;
A presentation output control procedure for outputting and controlling presentation information for presenting the contents of the first electronic document data;
A designation procedure for designating a part of the second electronic document data from the second electronic document data presented and output by the presentation output control procedure;
One or a plurality of video data or still image data corresponding to a part of the first electronic document data input based on the input means and the second electronic document data specified in the specifying procedure. A video or still image selection procedure to select;
A video or still image output control procedure for sequentially outputting a plurality of video data or still image data selected in the video or still image selection procedure;
Document processing method in which is performed. An input procedure for inputting the first electronic document data;
A presentation output control procedure for outputting and controlling presentation information for presenting the contents of the first electronic document data;
A designation procedure for designating a part of the second electronic document data from the second electronic document data presented and output by the presentation output control procedure;
One or a plurality of video data or still image data corresponding to a part of the first electronic document data input based on the input means and the second electronic document data specified in the specifying procedure. A video or still image selection procedure to select;
A video or still image output control procedure for sequentially outputting a plurality of video data or still image data selected in the video or still image selection procedure;
A recording medium on which is recorded an operation control program.