JP2002032397A - Retrieving device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a retrieving device which enables a user to retrieve a document, even when the user can not recall the document, title, and retrieval expression having contents and even when the user can not express the title and contents of the document well in a natural language. SOLUTION: This device is provided with a 1st storing means for storing data such as the document, sound and image as an object of an operation of inputting, editing and retrieving, etc., as operation object data; an acquiring means for acquiring time series image data obtained by photographing a self- device for performing operations of the operation object data and the vicinity of the self-device; an extracting means for extracting the image data of a scene, having a characteristic for making environment specifiable in operating the operation object data as environment information; a selecting means for selecting the environment information, when retrieval is carried out; a retrieving means for retrieving the operation object data made to correspond to the environment selected by the selecting means among pieces of the operation object data stored in the 1st storing means; and an outputting means for outputting the operation object data which are retrieved by the retrieving means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a search device.

[0002]

2. Description of the Related Art In a conventional document search apparatus or word processor, in order for a user to obtain a document retrieved in the past, the user can recall the document by recalling the title or contents of the retrieved document or use it in the past retrieval. It was necessary to search the data by remembering the search formula that was used. At this time, it is necessary to input a search expression as a character string, and to input a natural language expression representing the document itself or a natural language expression similar thereto as the title or contents of the document. Therefore, if the search formula, the title and the contents of the document cannot be remembered, or if they cannot be represented by a character string or a natural language expression, the search cannot be performed.

[0003]

As described above, when the retrieval formula, the title and contents of the document cannot be remembered, or when they cannot be represented by a character string or natural language expression, they are retrieved and extracted. I couldn't do that.

[0004] Therefore, an object of the present invention is to provide a search device that enables a search even when a search formula, a title or contents of a document cannot be remembered, or a title or contents of a document cannot be expressed in a natural language expression. And

[0005]

SUMMARY OF THE INVENTION The present invention provides an input / edit / edit
First storage means for storing data such as a document, a sound, an image, etc., as an operation target data as an operation target such as a search; Means for acquiring unique image data, and among the image data acquired by the acquiring means, image data of a characteristic scene for enabling an environment at the time of operation on the operation target data to be specified is extracted as environment information. Extracting means; second storage means for storing the image data extracted as the environment information corresponding to the time of the operation on the operation target data in association with the operation target data; and A selection unit for selecting, and an operation associated with the environment information selected by the selection unit from the operation target data stored in the first storage unit And retrieval means for retrieving the elephant data,
Output means for outputting the operation target data searched by the search means.

[0006]

The present invention inputs the contents expressing the situation, scene, environment, and the like when the user has created and searched for data in the past, analyzes the contents, and stores the analysis results together with the data. Therefore, desired data is searched based on the stored situation, scene, environment, and the like.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Before describing a specific embodiment, the basic concept of the retrieval apparatus according to the present invention and the definition of terms will be described.

In many information devices today, as represented by word processors, data input to the system is often processed and reused later. However, there is a problem that as the number of input data increases, it takes time to search for necessary data when it is desired to reuse the data.

The search device according to the present invention is used as a search key for data to be searched when accessing the data, such as when inputting data into the system or when processing or editing the data. The information (referred to as "environment information") representing the user himself / herself, the system or the surrounding situation (referred to as "environment information") is used, and this environment information is extracted semi-automatically.

Conventionally, when searching for necessary data from among many data already input to the system, there is no label such as a data file name that the user intentionally adds to the data, or an inappropriate label is used. In such cases, there is no other way than to use relatively simple physical labels automatically added by the system, such as the data generation time, as a search key. The part that relied on the user's memory for mapping to was large.

On the other hand, in a situation where a human needs a search, when he / she thinks of the data he / she wants to search, he / she is more likely to use “when” and “when doing” than a physical label such as time. , “When we were at…” and “when we were with…” are often related to the user or the surrounding situation when accessing the system, Searching was sometimes inconvenient.

The search apparatus according to the present invention has a device for taking in physical label data from which a user, a system, or a surrounding condition at the time of accessing the search data can be obtained. By analyzing data (referred to as “environmental analysis target data”), “environmental information” that is more easily used as a search key is stored in association with the data to be searched, and is used during the search.

An embodiment will be described below with reference to the drawings.

First, the basic configuration of the system is shown in FIG.

This system includes a search target data input unit 1, a search target data storage unit 2, a search target data output unit 3, an environment analysis target data input unit 4, an environment information analysis unit 5,
It comprises an environment information storage unit 6, an environment information output unit 7, a search key input unit 8, and a search unit 9.

As shown in FIG. 3, the data to be searched refers to general data input to a system realized by a computer or the like. For example, texts and data files consisting of texts and images input using a keyboard, mouse, pen, etc., image data such as still images and moving images input using image scanners and cameras,
After being input and stored in the system, such as recording using a microphone or electronically synthesized acoustic data,
Reusable data such as editing, processing, and reference.

As shown in FIG. 2, the search target data input unit 1 includes an input device for inputting the search target data and information on input data necessary for later analysis (for example, when there are a plurality of input devices, Is composed of a data conversion unit for adding a device identifier, a quantization bit number, a sampling frequency, and the like.

The search target data storage unit 2 stores the search target data input from the search target data input unit 1.

The search target data output unit 3 extracts the search target data stored in the search target data storage unit 2, and presents and outputs the data to the user. Here, the form of presentation and output depends on the application. When a document or image data is handled as search target data, it has means for displaying or printing on a display, and when handling audio data, it has an audio output means for reproducing and outputting to a speaker or the like.

As shown in FIG. 3, the environmental analysis target data is data from which environmental information is extracted by analysis, and is automatically input into the system when the user uses the system.

For example, a history of input operations using a keyboard, a mouse, a pen, etc., image data such as a still image and a moving image input using a scanner camera, etc., recorded using a microphone, or electronically synthesized Acoustic data, position measurement data using position measurement devices such as GPS and magnetic sensors, data for measuring the user's condition and external conditions when using the system, such as measurement value data obtained from temperature, humidity, and pressure sensors, pulse and blood pressure , Body temperature, blood glucose level, and other measured value data indicating the user's health condition.

In addition, an input made by the user without intending to be used for retrieval when using the system can also be used as data to be analyzed. For example, a workstation is intended to be used by multiple users, so it is necessary to enter a user name at the beginning of using the system,
As a result, the user ID is determined, so that it is possible to determine who has used the system.

Further, by acquiring the identifier of the computer connected on the network, it is possible to determine which system was used. If the installation location of the system is fixed, it is possible to know which system was used.

Information on who and where the data is input can be obtained by analyzing input data other than the search target, such as a schedule table and other documents. It is also possible to use search target data as environment analysis target data.

The environment analysis target data input unit 4 includes an input device for inputting the environment analysis target data,
And a data conversion unit for adding information relating to input data necessary for later analysis.

Correspondence information indicating which search target data corresponds to the environment analysis target data is added. For example, if the search target data is time-series data, the input time of the search target data can be used as the corresponding information. That is,
For each of the search target data and the environmental analysis target data,
The input time can be added, and the correspondence can be made based on this time information.

If the data to be searched is not data that changes with time or changes are slight, and if the data to be analyzed for the environment does not change with time or changes are slight, an identifier is assigned to the data. This can be added and used as correspondence information. That is, a data identifier is added to each of the search target data and the environmental analysis target data, and the corresponding data can be identified based on the data identifiers.

The data to be analyzed include, for example, extraction of utterance contents by voice recognition, identification of a system use place by acoustic characteristics, identification of a speaker by speaker recognition, identification of a system use place by image recognition, layout of document data. Obtain environmental information by analyzing, analyzing schedules and data access information.

Analysis such as detection, analysis, recognition, etc.
This is performed by the environment information analysis unit 5. The analysis differs depending on the form of the environmental analysis target data, and higher-order environmental information can be obtained by further analyzing and structuring a plurality of analysis results.

Specific analysis contents and obtained environment information for each environment analysis target data will be described in respective embodiments described later.

It is desirable that these analyzes be performed automatically when the system is used. However, if the analysis process is complicated and real-time performance cannot be ensured, even if the user does not use the system, the analysis is performed. Good.

It is also possible to correct errors in automatic analysis,
If the user intentionally wants to add environmental information, the analysis result and manual correction or addition may be made possible.

As described above, correspondence information indicating which search target data corresponds to the search target data is added to the environmental analysis target data. Correspondence information indicating whether or not it has been obtained is added. Needless to say, the correspondence information may be information for directly associating the environment information with the search target data, instead of associating the environment information with the environment analysis target data. This correspondence information can use the input time of the search target data as the correspondence information when the temporal change of the data is significant, similarly to the correspondence between the search target data and the environmental analysis target data. However, when the temporal change has no particular meaning, the association can be performed using the data identifier.

When the obtained environment information corresponds to a part of the data to be searched or the data to be analyzed, the corresponding information indicating which part of the whole corresponds. Can be searched more accurately and easily.

By following such correspondence information between the environment information and the search target data or the environment analysis target data, the search target data can be searched using the environment information as a search key.

Furthermore, information indicating what kind of analysis the environment information has been obtained or how much likelihood has been obtained can be added to the environment information. In this case, even when similar environmental information is obtained by a plurality of analysis methods, it can be used as a material for determining which environmental information is most reliable.

The environment information referred to here means that the user inputs search target data using the system as shown in FIG.
Information about the user or his / her surroundings (time, place) when editing, searching, or outputting. Not only the physical time and place, but also when attending a meeting with a certain person. When you do, on the commuter train, in the living room at home, when you are, where you are, where you are, what you are doing, where you are,
This refers to structured information that is meaningful to the user who is searching, such as "place", "when and where". By linking the environment information with the search target data, the search target data required by the user can be easily found using the environment information as a search key.

Each of the environmental information includes "who entered the data to be searched", "where the data to be searched was input", "how the data to be searched was input",
By adding an identifier corresponding to the topic obtained by the analysis, such as "Why did you enter the search data?", To the specific analysis result, to show the contents of the environmental information to the user in a way that is easy to understand. Can be.

Such environmental information can be obtained by analyzing physical data such as sound and video signals automatically taken while the user is using the system, and measured values of position and the like. Such data to be analyzed to obtain environmental information is referred to as environmental analysis target data as shown in FIG.

The environment information analysis unit 5 analyzes and structures the environment analysis target data input to the system from the environment analysis target data input unit 4, and generates environment information.

The environment information storage unit 6 stores environment information including analysis results and its structured information.

The environment information output unit 7 outputs environment information including analysis results stored in the environment information storage unit and its structured information.

The search key input unit 8 includes an input device such as a keyboard, a touch panel, a pen, and a mouse for receiving an input from the user, and an environment information conversion unit for converting the input device into environment information to be searched.

The search unit 9 searches the environment information storage unit 6 for environment information designated by the user through the search key input unit 8,
The search target data linked to the environment information is extracted.

The search unit 9 can also retrieve relevant environment information by following the environment information and the structured information stored in the environment information storage unit 6.

Existing multimedia data input / output applications, such as word processors, electronic publishing systems, and audio / video recording systems, simply implement functions for input / output and storage of data to be searched. When expressed in correspondence with the configuration of FIG. 1, only a search target data input unit, a search target data storage unit, and a search target data output unit are provided.

According to the present invention, information (environmental information) representing the user and surrounding conditions when the user is operating the search target data using the system is linked to the search target data, and the environment information is used. It provides a framework for searching for search target data. Further, a major feature is that the system automatically generates environment information.

Next, details of each section will be described.

Input of data from an input device The input device of the data input unit for search 1 or the data input unit for environmental analysis 4 is continuous data such as audio media or video media, and the relative position between the input device indicating the pointing position and the system. The position data, the absolute position data of the system itself obtained by using the position measurement system, and the like are taken into the system.

The data conversion section of the search target data input section 1 or the environmental analysis target data input section 4 shown in FIGS. 2 and 4 converts these input data into a format that can be handled by the system. For example, continuous data is
It is stored and processed in a quantized format.

Here, the discrete / quantized information is added to the discrete / quantized search target data and the environmental analysis target data. This discrete / quantized information is used when reproducing or outputting all or a part of the search target data.

Further, the term "search target data" or "environment analysis target data" refers to data after conversion.

Handling of time information as environment analysis target data If an operation on the search target data such as input, edit, search, and output is taken as an event, the occurrence time of each event obtains environment information on the search target data. Can be used as an analysis target. That is, an event occurrence time series such as time information indicating the time when the search target data is input to the system can be handled as the environmental analysis target data. As a result, information indicating when the search target data is operated and when a certain part of the search target data is operated are recorded.

Similarly, time information indicating the input time can be treated as environmental analysis target data.

As described above, by having time information for each input data, it is possible to correspond to each time. Add a time link between the search target data or a part of it and the environmental analysis target data,
It can be used when linking the operation event for the search target data and the environmental analysis target data.

Storage of search target data and environmental analysis target data The search target data storage unit 2 stores the search target data, the environment analysis target data storage unit 6 stores the environmental analysis target data, together with additional information such as discrete / quantized information. Store.

The entity of the search target data or the environmental analysis target data and the additional information may be linked to each other, for example. Alternatively, the data to be searched or the data to be analyzed in the environment generally requires a huge storage capacity, whereas the additional information is relatively common for each input device and each input data. Alternatively, a frame may be prepared for each of the data to be searched and the data to be analyzed, and the frame may have a pointer indicating the entity thereof and a pointer indicating the additional information or the entity of the additional information.

Timing of Environmental Analysis (Environmental Information Analyzing Unit 5) The environmental information analyzing unit 5 analyzes environmental analysis target data to obtain environmental information. That is, environmental information is obtained by parameter extraction, pattern recognition / understanding, and the like.

The analysis of the environmental analysis target data is automatically performed when the environmental analysis target data is input, as shown in FIG. As described above, the environment analysis target data is always input when the user operates the search target data using the system. Therefore,
The analysis will always be performed while the user is using the system.

Analysis of Environmental Information (Environmental Information Analyzing Unit 5) The information obtained by extraction or recognition / understanding is directly or indirectly used when the user uses the system or the surroundings. This is information indicating the situation.

For example, an example in which the location where the search target data is created is a living room at home as environment information will be described. This environment information can be obtained by, for example, using a position data sequence sent from a position measurement system such as a GPS as environment analysis target data and recognizing that the position data is a living room at home. Environmental information indicating the place of use can also be obtained from environmental analysis target data other than GPS, and an example is shown in FIG.

By the way, in the environmental information analysis unit 5, which environmental information is desired to be obtained, from which analysis result the environmental information is obtained, and from which environmental analysis target data the analysis result is obtained by an analysis method. Must be set prior to automatic analysis.

In the above example, as environment information relating to the use place of the system, a use place to be identified, such as a living room at home, or a work place of one's own, and further information for identifying the use place. GP as analysis target data
It is necessary to set the use of the S position measurement data, the correspondence between the position data value and the place of use to be identified, etc. before analysis.

Of these, "what analysis method should be used to analyze which data to obtain certain environmental information" is a part that is hardly dependent on application scenes and users. Therefore, the system may be provided with default knowledge on analysis. Referring to FIG. 6, in order to obtain environmental information indicating a place of use,
The analysis of the reverberation pattern of the reverberation data and the ambient sound pattern of the ambient sound data, etc. may be handled as preliminary knowledge built into the system regardless of the application.

On the other hand, “what instance of environmental information (instance of)
You want to get, and its environment information (instance of)
Which analysis result can be obtained? "Depends greatly on the application scene and the user. That is, in the above example, the former is based on knowledge about what to identify, such as identifying three types of living room at home, workplace, and moving vehicle as environmental information indicating a place of use. The latter distinguishes a living room at home by analyzing the reverberation pattern obtained as a result of the analysis, the type of ambient sound pattern obtained, etc. The knowledge related to the understanding of the results is all knowledge that largely depends on the application and the demands of the user. Speaking of Table 2, the part for obtaining individual environmental information from the analysis result needs to be set by the user prior to the analysis.

Specifically, sound, image, position, schedule, access information, and the like are conceivable, and the details will be described later.

Search Mechanism In addition, to each of the obtained analysis results, information relating to the analysis such as which part of the environmental analysis target data has been analyzed is added. As a result, the obtained individual environmental information is linked to all or analyzed portions of the environmental analysis target data. As described above, since there is a link between the search target data and the environment analysis target data, the search target data can be searched from the environment information. At the time of a search, search is performed from environmental information linked to the search target data. For example, environment information indicating a use place, a use time, a use state, and the like are displayed through the environment information output unit 7, and the user selects the environment information to be used as a search key by referring to the environment information.

The environment information used as a search key is specified through the search key input unit 8.

The search section 9 searches the specified environment information in the environment information storage section 6, and the search target data linked thereto is output from the search target data output section 3.

Addition of likelihood to environment information (environmental information analysis unit 5 and environment information output unit 7) The certainty (likelihood) of the analysis result depends on the analysis method.
Further, the likelihood differs for each analysis result, even if they are obtained from the same analysis method.

Therefore, as the information relating to the above-mentioned analysis, information indicating which analysis method has been obtained and how reliable the analysis result is according to the analysis method are added.

This likelihood may be used by fixing a value subjectively determined by the user, or may be set semi-automatically by the system.

For example, the higher the frequency of using environment information obtained by a certain analysis method as a search key, the higher the likelihood of the analysis method is set. If the analysis is pattern recognition, the similarity with the standard pattern may be used as the likelihood.

When there are a plurality of analysis methods and the analysis results obtained by the respective methods are inconsistent, by referring to the information on the analysis method and the likelihood of the analysis result, the search target data can be retrieved. A search with a high likelihood can be preferentially handled and a search can be efficiently advanced. It is also possible to give priority to environmental information obtained with high likelihood from many analysis methods.

Structure of Analysis Result (Environmental Information Analysis Unit 5) If multimedia data can be used as environmental analysis target data, analysis is first performed separately for each medium.

Next, the analysis result of each medium is structured for each desired environment information.

For example, as shown in FIG. 6, environmental information on where the search target data is input can be obtained by a positioning system such as a GPS, as well as by acoustic data such as reverberation characteristics and ambient noise. It is also possible to make an estimation by referring to data such as identification, scene recognition using images, a history of schedules and daily activities.

The analysis results regarding the use locations of each of the plurality of media are structured as environmental information having the same contents.

Use of search target data as environmental analysis target data Although the search target data and the environmental analysis target data have been described above as separate data, they may be the same data. . That is, using the search target data as it is as the environmental analysis target data,
This may be analyzed to obtain environmental information. For example, when a speech of a specific person is reproduced using recognition results obtained by performing speaker recognition using voice data input from a microphone as environment information, both search target data and environment analysis target data are the same voice data.

As described above, within the system, the same data can be a search target from a search viewpoint, and at the same time, can be an analysis target from an environment analysis viewpoint.

Handling of Device Information as Environment Analysis Target Data The system can include a plurality of input devices for the search target data input unit or the environment analysis target data input unit. If a plurality of input devices can be used, the device information itself indicating from which device each input data is input can also be used as the environmental analysis target data.
For example, whether it is audio media, video media, data input from a device installed anywhere,
Environmental information can also be obtained from information such as where the microphone is directed and the data captured by the camera.

Addition of likelihood by device information.

When the quality of the data to be analyzed is different for each device, the likelihood can be changed by referring to the device information.

Next, an example in which an environment information input unit 10 that enables the user to input environment information will be described with reference to FIG.

The environment information input unit 10 includes an input device such as a keyboard, a touch panel, a pen, and a mouse for receiving an input from a user, and an environment information conversion unit for converting the input device into environment information.

In general, it is unavoidable that the result obtained by the automatic analysis contains an error, and if the result is used without being corrected, the efficiency of the search may be reduced. By directly inputting the environment information by the user, it becomes possible to correct or delete an analysis error by the system or to add necessary environment information.

The environment information that can be input by the user here includes the analysis result of the system, its structured information, or the likelihood.

It is also possible to add a high likelihood to the environment information directly input by the user.

An analysis method in which a user frequently deletes an analysis result is an analysis method in which the likelihood of the analysis method is reduced, or conversely, correction is frequently repeated so as to obtain a large amount of environmental information used as a search key. Since it can be determined that there are many analysis results that the user pays attention to, the likelihood of the analysis method can be increased.

Next, the environment analysis target data is stored.
An example including an environment analysis target data storage unit will be described with reference to FIG.

The environment analysis object data storage unit 11 stores the environment analysis object data input from the environment analysis object data input unit 4. The environment analysis target data stored here is extracted as necessary, and is analyzed and structured by the environment information analysis unit 5.

By analyzing the environmental analysis target data once input to the system, it is possible to perform a complicated analysis offline compared to the processing capacity of the hardware or to re-execute when the already obtained environmental information is not sufficient. It becomes possible to analyze. In other words, when analyzing environmental analysis target data to obtain environmental information, it is necessary to determine in advance what to extract by analysis (analysis result) and how to analyze (analysis method including necessary parameter settings). However, depending on the purpose at the time of the search, the analysis result or analysis method already obtained may not be sufficient. By storing the environment analysis target data inside the system and resetting the analysis method and the analysis result to be obtained, it is possible to obtain environmental information as needed.

Next, the above specific example will be described in detail.

First, the first invention will be described. First
Of the invention, when creating and searching data such as documents,
The present invention relates to a search device that stores an ID of this data in association with position information of a position where the data is created and searched, and extracts desired data based on the position information. For example, as the position information, a GPS capable of collecting a user's action history is used, and data is extracted based on information on a map and a room plan. The details will be described below.

Embodiment 1 The search target is performed as a document (data).

The document search device of the present invention is a small portable device. FIG. 9 illustrates a configuration of a device for configuring the apparatus of the present invention. Central processing means 101 for performing control and processing based on the present invention, semiconductor memory for storing programs and data, magnetic disk, optical disk, CD-RO
M, a storage means 102 such as an IC card, a display means 104 such as a liquid crystal display or a plasma display for displaying a search result or a document content, a display controller 103 for controlling the display means, and a user's input of a search command Input means 106 such as a keyboard and a mouse for performing
And input controller 105 for controlling the same, GPS
It comprises communication means 107 such as an antenna and an infrared receiver.

FIG. 10 shows a functional configuration of the present embodiment. In the present embodiment, a document data storage unit 25 for storing document data and the like, an input unit 20 for inputting a search command for searching for a document, a search unit 23 for searching for a document according to the search command, a search A search result display unit 22 for displaying the result, a position measurement unit 24 for measuring the position where the computer apparatus body is located, a position document storing information on the measured position, the name of the searched document, and the like. It comprises an information storage unit 26, a place name storage unit 27 for storing information on the measured position and a place name representing the position, and a control unit 21 for controlling these components.

FIG. 11 shows the format of document data in the document data storage unit and an example thereof. The document data includes a pair of a document name and a document ID of a file storing data of the document. In FIG. 11, file8, file11, file23, etc. represent the document ID. The document name is, for example, JIS
They are arranged in code order. The document data storage is implemented on the storage unit 102 in FIG.
-It can be replaced and used as in the case of a ROM or an IC card.

First, processing for searching for a specific document will be described, second, setting of the name of an arbitrary place will be described, and thirdly, documents that have been browsed, modified, and created in the past using the place name will be described. The process of searching for will be described.

First, the user inputs the name of a document to be searched using a keyboard or the like from the input unit, and designates “search” on a window (not shown) with a mouse or the like, or presses the “search” key on the keyboard. Press. FIG. 12 shows a processing flow of the control unit 21 in this case, and the search unit 23
13 is shown in FIG.

When "search" is selected, the control unit 21
Receiving an input character string from the input unit 20 (process 401);
The search unit 23 is activated (process 402), and the input character string is transferred to the search unit (process 403). The search unit 23 includes the control unit 2
The document data is searched using the input character string transferred from No. 1 (process 501). When the corresponding document name exists, the obtained document ID is transferred to the control unit (process 50).
3). The control unit 21 receives the document name and the document ID transferred from the search unit 23 (process 404), activates the search result display unit 22 (process 405), and transfers the search result to the search result display unit 22 (process). 406). Search result display section 22
Generates a search result display window and displays the contents of the document data in the window. The user can browse the document in this way. It is easy to modify the document as well as browse.

On the other hand, the search unit 23 activates the position measuring unit 24 (process 504). The position measurement unit 24 calculates the latitude and longitude of the point where the computer is located, transfers position information including the calculated latitude and longitude to the search unit 23, and
Receives the position information (step 505). The position measurement method is not the gist of the present invention. For example, a GPS (global positioning system) method using a positioning satellite ("Car Navigation System" Takashi Yamaguchi et al., Toshiba Review, Vol. 48, No. 2, 1993) may be used. In the case of GPS, the position can be measured in the field where a positioning satellite can be used.

The search unit 23 stores the position information transferred from the position measurement unit 24 and the document ID of the searched document in the position document information storage unit 26 in association with each other (process 50).
6). FIG. 14 shows an example of the data format and data in the position document information storage unit 26. Here, the document ID of the searched document is stored. However, when a plurality of documents are searched, the document ID of only the document that the user has selected and displayed in the search result display section is displayed. It can also be stored.

In the above, a case has been described in which the user searches for, browses, and modifies an existing document. However, the user may input and create a new document. In this case, the input unit stores the user's input in the document data storage unit 25, and stores the document ID and the document name set by the user. Further, the document ID is transferred to the control unit 21. The control unit 21 activates the search unit 23 and transfers the document ID. The search unit 23 receives the document ID from the control unit as shown in FIG.
1), the position measuring unit 24 is activated (processing 1602). The position information is received from the position measuring unit 24 (step 1603).
Then, the position information and the document ID are stored in the position document information storage unit 26 (process 1604).

Second, the user inputs a name representing the place at an arbitrary place from the input unit 20, and designates “place registration” on a window (not shown) with a mouse or the like, or Press the "place registration" key above. FIG. 15 shows a processing flow of the control unit in this case, and FIG. 14 shows a processing flow of the position measurement unit.

When "place registration" is selected, the control unit
An input character string is received from the input unit 20 (process 701),
The position measuring unit 24 is activated (process 702), and the input character string is transferred to the position measuring unit 24 (process 703).

The position measuring section 24 receives the transferred input character string (process 1401). Next, the latitude and longitude of the point where the computer is located are calculated (step 1402), and the transferred input character string is associated with the position information including the calculated latitude and longitude in the place name storage unit 27. It is stored (process 1403). FIG. 16 shows an example of the data format and data in the place name storage unit.

Third, the user designates “place search” on a window (not shown) with a mouse or the like at an arbitrary time or place, or presses a “place search” key on a keyboard.

FIG. 17 shows the flow of processing of the control unit in this case. The control unit 21 activates the search result display unit 22 (process 901). FIG. 18 shows a flow of processing of the search result display unit 22. The search result display section 22 stores the place name storage section 2
7, all the place names are taken out (process 1001), all the taken out place names are displayed in the search result display window (process 1002), and the place names are displayed from the place names displayed by the user with the mouse or the like. Is selected (processing 1003). FIG. 19 shows an example of display in the search result display window. In FIG. 19, the black portions indicate that the user has selected. The search result display unit 22 extracts the position information corresponding to the selected place name from the place name storage unit 27 (step 1004), and transfers the extracted position information to the control unit 21 (step 100).
5).

When the control unit 21 receives the position information from the search result display unit 22 (step 902), it activates the search unit 23 (step 903) and transfers the position information to the search unit 23 (step 904). FIG. 20 shows a flow of processing of the search unit 23. The search unit 23 searches the position document information storage unit 26 using the position information transferred from the control unit 21 (Process 1).
201).

FIG. 23 shows the details of the search processing. FIG.
As shown in FIG. 3, whether the latitude and longitude match is not a perfect match but is determined based on whether the latitude and longitude fall within a predetermined value range. If the corresponding document ID is found, the document data storage unit 25 is searched using the document ID in the position document information (process 1203). The retrieved document name and document ID are transferred to the control unit (process 1204). The control unit 21 receives the transferred document name and document ID (process 905) and transfers them to the search result display unit 22 (process 906).

The search result display section 22 receives the document name and the document ID from the control section 21 (step 1006) and displays them on the window for the search result display section (step 1007). If there is one document ID, the contents of the document having the document name and the document ID are displayed. If there are a plurality of document IDs, a list of document names is displayed as shown in FIG. In the latter case, the user can select a required document name from the displayed document names. The search result display section displays the document I corresponding to the document name selected by the user.
The contents of the document D are displayed in the search result display window.

In the display stage of FIG. 19, it is possible to display all the document names corresponding to all the place names at the same time. In particular,
FIG. 20 shows not only the location information of the place selected by the user but also the location information of all the place names in the place name storage unit.
May be performed.

As described below, whether or not a place name corresponding to the position information of the document has been set for a document that the user has browsed, modified, or created, and for all the created documents under predetermined conditions. Is checked, and if there is a document for which the corresponding place name is not set, the user can be prompted to register the place name. FIG. 25 shows the flow of the process of the control unit in this process.

The control section 21 activates the search result display section 22 when the number of documents browsed, modified, and created by the user exceeds a predetermined number (step 1702). FIG. 26 shows the flow of processing of the search result display unit 22. Search result display section 22
Extracts all the position information from the place name storage unit 27 (step 1801) and transfers it to the control unit 21 (step 1801).
2). The control unit 21 receives the position information from the search result display unit 22 (Step 1703), activates the search unit 23 (Step 1704), and transfers the position information (Step 170).
5).

As shown in FIG. 27, the search unit 23 receives the position information transferred from the control unit 21 (step 190).
1), search the position document information storage unit 26 with the position information (process 1902). If the position information in the position document information storage unit 26 includes position information that is not within a predetermined range from the transferred position information, that is, the position information in which the place name is registered, the position information document The document data storage unit 25 is searched by the ID (process 1904). The searched document name, document ID, and position information are transferred to the control unit 21 (step 1905).

The control unit 21 retrieves the document name, document ID, and position information transferred from the retrieval unit 23,
2 (processing 1706). Search result display section 22
Receives the document name, the document ID, and the position information from the control unit 21 (step 1803), extracts the location name of the position information closest to the position information from the place name storage unit 27 (step 1804), and displays the search result. A message prompting the registration of the place name, the document name and the place name closest to the document name and the place name extracted from the place name storage unit 27 are displayed on the copy window (step 1805).

FIG. 31 shows an example of the display of the search result display window. When the user inputs a place name, the place name storage unit 27 is searched by the input place name (process 18).
07). If there is no matching place name, the location information of the document corresponding to the input place name is stored in the place name storage unit 27 (process 1809), and the specified location information stored in the displayed document is stored. If there is a document whose position information does not fall within the range, the place name closest to the document name of the document is displayed again (step 1811). FIG. 32 shows an example of the display after the user has registered the place name with respect to the display of FIG. In the present embodiment, the latitude and longitude are measured using GPS as the position measurement method and the information is stored, but the position measurement method is not limited to GPS. For example, an infrared transceiver can be used. Specifically, for example, buildings and houses on different sites and premises, or different buildings on the same site and premises,
An infrared transmitter that oscillates a bit pattern unique to a place such as a conference room or a work place on a house or floor is installed. In this case, an infrared receiver may be used as the position measuring means in the present embodiment, and instead of measuring the latitude and longitude, the bit pattern emitted by the infrared transmitter may be received, and the place name and the room name may be set. In this manner, the same processing as in the present embodiment can be performed.

In this embodiment, in the search processing,
Although the method of selecting the displayed place name has been described, the search may be performed by inputting the place name. More specifically, the user inputs a place name, compares the inputted place name with the place name in the place name storage unit 27, and if there is a matching place name, determines the latitude and longitude of the place name. The position document information storage unit 26 is searched by the value, and the document ID of the longitude and latitude values within a predetermined value range
Should be displayed.

Further, a search may be made by inputting values of latitude and longitude. Specifically, the user inputs the latitude and longitude values, searches the position document information storage unit 26 with the input latitude and longitude values, and obtains the document IDs of the longitude and latitude values within a predetermined value range.
Should be displayed. At this time, it is also possible to search the place name storage unit with the input latitude and longitude values and simultaneously display the place names of the longitude and latitude values within a predetermined value range.

Although the place name and the document name are stored and displayed in this embodiment, the time may be measured at the same time, and the place name, the document name and the time may be stored in combination. In this case, the document name can be displayed together with the time for each place name.

In this embodiment, the case of browsing, modifying, and creating a document has been described. However, the present invention is not limited thereto. For example, in combination with a mobile phone, it is possible to store information on the location where the phone was used, so that it is possible to retrieve later where, where or with whom. In addition, in combination with electronic still cameras and video recorders, it is possible to store information on the locations where photos and videos were taken, so that it is possible to search for where and what was taken later. In addition, it can be installed on vehicles and moving objects such as automobiles, and stores information on the position during movement, so that it is possible to search later where it has passed. Example 2 The device configuration of this example is the same as that of FIG. FIG. 28 shows a functional configuration. In the first embodiment, registration of a place name is independent of browsing, correction, and creation of a document. In the present embodiment, when a user browses, modifies, or creates a document, it checks whether there is a registered place name. If not, the user is prompted to register the place name.

FIG. 29 shows the flow of processing of the search unit 23 when a user views and modifies a document, and FIG. 30 shows the flow of processing of the search unit when the user creates a document. FIG.
As shown in FIG. 9, it is checked in step 2107 whether or not there is a registered place name, and if not, the registration of the place name is prompted in step 2109 or the like. Similarly, as shown in FIG. 30, it is checked whether or not there is a registered place name in step 2205, and if not, the registration of the place name is prompted in step 2207 or the like.

Third Embodiment In the first embodiment, the document search device has a means for measuring the position. In the third embodiment, the document search device has a means for browsing, modifying and creating a document, and a means for transmitting information. It comprises a small portable first device and a second device having a means for receiving information and a means for searching. The device configurations of the first device and the second device of the present embodiment are the same as those in FIG. However, the communication unit 107 is an infrared transmitter or the like in the first device, and is an infrared receiver or the like in the second device. FIG. 33 illustrates a functional configuration of the first device of the present embodiment. The first device is a document data storage unit 2 for storing document data and the like.
5. Input unit 2 for inputting a search command for searching a document
0, search unit 2 for searching a document according to a search command
3. A search result display unit 22 for displaying search results, a clock unit 29 for counting time, and an operation history storage unit 3 for storing time data and a document ID and a document name of a document of a search result.
1. Communication unit 30 for notifying the presence of the first device, output unit 2 for outputting data in operation history storage unit 31
8 and a control unit 21 for controlling each of these components.

FIG. 34 shows the functional configuration of the second device of this embodiment. The second device includes an input unit 20 for inputting a search command for inputting data and searching for a document, a search unit 23 for searching according to the search command, a search result display unit 22 for displaying a search result, A communication unit 30 for receiving information transmitted from one device; a position operation information storage unit 3 for storing information transmitted from the first device;
2. It comprises a place name storage unit 27 for storing a place name expressing a position, and a control unit 23 for controlling each of these components.

The format of the data in the document data storage unit 25 of the first device in FIG. 33 is the same as that in FIG.

The processing flow of the control unit of the first device in FIG. 33 is the same as that in FIG.

FIG. 35 shows the flow of processing of the search unit 23 of the first apparatus in FIG. The search unit 23 searches for document data using the input character string transferred from the control unit 21 (process 2701). If the corresponding document name exists, the document ID of the document is transferred to the control unit 21 (step 2703), and the time data at that time is received from the clock unit 30 (step 2704). Further, the communication unit 30 is activated (process 2705), and the time data is transferred to the communication unit 30 (process 2706). The activated communication unit 30
Transmits the user ID of the user operating the first device and the transferred time data. FIG.
Shows the format of data to be transmitted and an example thereof. Finally, the search unit 23 stores the time data and the document ID and the document name of the searched document in the operation history storage unit 31 (processing 270).
7). FIG. 29 shows the format of data in the operation history storage unit 31 and an example thereof. In this case, the document ID and the document name of the searched document are stored. However, when a plurality of documents are searched, the user selects and displays the contents in the search result display unit. Only the document ID can be stored.

In the above, the case where the user searches for, browses, and modifies an existing document has been described. However, the user may input and create a new document. In this case, the input unit stores the user's input in the document data storage unit 25, and stores the document ID and the document name set by the user. Further, the document ID is transferred to the control unit. The control unit 21 activates the search unit 23 and transfers the document ID. The search unit 23 receives the document ID from the control unit 21 as shown in FIG.
1) The time data is received from the clock unit 29 (processing 30).
02). Further, the communication unit 30 is started (step 300
3) Transfer the time data to the communication unit 30 (process 300)
4). Finally, the time data and the document ID and the document name of the input and created document are stored in the operation history storage unit 31 (process 3005).

The second device performs a process when data transmitted from an arbitrary first device is received, and a process of searching for a document that has been browsed, corrected, or created in the past using a place name. The communication unit of the second device is provided with a place ID of a place where each is installed.

When the arbitrary communication unit i receives the data transmitted by the arbitrary first device, the control unit 21 pairs the received data and the location ID assigned to the communication unit i30 with the position operation information. It is stored in the storage unit 32. FIG. 39 shows an example of the data format in the position operation information storage unit 32 and an example of data immediately after the processing is performed in <Example 1>. <Example 1>
Then, the document ID and the document name have not been set yet. This is set in the processing described later.

Next, a process of searching for a document using a place name will be described. First, the user inputs data in the operation history storage unit 31 of the first device from the output unit of the first device to the input unit 20 of the second device. Specifically, for example, the output unit of the first device and the input unit of the second device are connected by a code, and "operation history input" on a window (not shown) of the second device is designated by a mouse or the like. Alternatively, an "operation history input" key on the keyboard is pressed. At this time, optical communication may be used instead of connection using a cord. When “operation history input” is selected and the data in the operation history storage unit 31 of the first device is input, the control unit 21 of the second device transfers the input data to the search unit 23. .

The search unit 23 performs the processing shown in FIG. 40 for all the transferred data. That is, for all the transferred data in the operation history storage unit 31 of the first device, the data in the position operation information storage unit 32 of the second device is searched, and when the user ID matches the time, The arrangement of the document ID and the document name in the data of the operation history storage unit 31 is added to the data in the position operation information storage unit 32. <Example 2> in FIG. 39 illustrates an example of data immediately after the processing is performed.

Next, the user designates “location search” on a window (not shown) of the second device with a mouse or the like, or presses the “location search” key on the keyboard. When the “location search” is selected, the control unit 21 of the second device activates the search unit 23. The search unit 23 extracts data matching the user ID of the user from the position operation information storage unit 32. The location ID of the extracted data is searched in the location name storage unit 27, and the location ID is stored in the location name storage unit 27.
Replace with the location name stored in. FIG. 41 shows a format of data in the place name storage unit 27 and an example thereof. Search unit 23
Transfers the search results to the control unit 21. Control unit 2
1 activates the search result display unit 22 and transfers the search result to the search result display unit 22. Search result display section 22
Displays the search result in a search result display window. FIG. 42 illustrates an example of a display in the search result display window.

Fourth Embodiment In the third embodiment, the communication means of the first device only transmits by the infrared transmitter or the like, and the communication means of the second device only receives by the infrared receiver or the like. . In this embodiment,
The communication means of the first device and the second device is an infrared transceiver or the like, and performs transmission and reception.

The device configurations of the first device and the second device of the present embodiment are the same as those in FIG. However, the communication means 107 is
Such as an infrared transceiver.

FIG. 43 shows the functional configuration of the first device of this embodiment. The first device includes a document data storage unit 25 for storing document data and the like, an input unit 20 for inputting a search command for searching for a document, a search unit 23 for searching for a document according to the search command, a search result. , A timekeeping unit 29 that measures time, an operation history storage unit 31 that stores time data, a place ID, and a document ID and a document name of a document of a search result, and a place ID.
, A communication unit 30 for notifying the second device of the presence of the first device and receiving the location ID from the second device, It comprises a control unit 21 for controlling each component.

FIG. 44 shows a functional configuration of the second apparatus of this embodiment. The second device receives the information transmitted from the first device, and communicates with the communication unit 3 for transmitting the location ID.
0, a transmission / reception storage unit 33 for storing received data and a location ID, and a control unit 21 for controlling these components.

The format of the data in the document data storage unit 25 of the first apparatus in FIG. 43 is the same as that in FIG.

The processing flow of the control unit 21 of the first apparatus in FIG. 43 is the same as that in FIG.

FIG. 45 shows the flow of processing of the search unit 23 of the first apparatus in FIG. The search unit 23 searches for document data using the input character string transferred from the control unit (process 3701). If the corresponding document name exists, the document ID of the document is transferred to the control unit 21 (process 3703), and the time data at that time is received from the clock unit 29 (process 3704). Further, the communication unit 30 is activated (step 3705), and the time data is transferred to the communication unit 30 (step 3706). The started communication unit transmits the user ID of the user operating the first device and the transferred time data. The format and an example of the data transmitted by the communication unit 30 are the same as those in FIG. In FIG. 36, the data transmitted by the communication unit 30 includes the user ID and the time data, but may include only the user ID, only the time data, or other data. In short, it suffices if the presence of the first device can be notified to the second device.

The communication unit 30 further receives the location ID transmitted from the communication unit 30 of the second device, and transfers it to the search unit 23. The search unit 23 receives the transferred place ID (Step 3707). Finally, the search unit 23 inputs the place ID,
The time data and the document ID and the document name of the retrieved document are stored in the operation history storage 31 (processing 370).
8). FIG. 46 shows the format of data in the operation history storage unit 31 and an example thereof. In this case, the document ID and the document name of the searched document are stored. However, when a plurality of documents are searched, the user selects and displays the contents in the search result display unit. Only the document ID can be stored.

In the above, a case has been described in which the user searches for, browses, and modifies an existing document. However, the user may input and create a new document. In this case, the input unit stores the user's input in the document data storage unit 25, and stores the document ID and the document name set by the user. Further, the document ID is transferred to the control unit 21. The control unit 21 activates the search unit 23 and transfers the document ID. The search unit 23 receives the document ID from the control unit 21 as shown in FIG.
01), and receives time data from the timer 29 (Process 3)
902). Further, the communication unit 30 is activated (processing 390).
3) Transfer the time data to the communication unit 30 (Process 390)
4). The activated communication unit 30 transmits the user ID of the user operating the first device and the transferred time data, and further receives the location ID transmitted from the communication unit 30 of the second device. Transfer to search unit 23. The search unit 23 receives the transferred place ID (Step 3905). Finally, the search unit 23 stores the location ID in the operation history storage unit 31 together with the time data, the document ID and the document name of the input and created document (Step 3906).

The communication unit 30 of the second device is provided with a place ID of a place where each is installed. When any communication unit i30 of the second device receives data transmitted by any first device, the control unit 21 transmits the received data and the location ID assigned to the communication unit i30 to the transmission / reception storage unit 33. And the location ID is transmitted from the communication unit i30.

Next, a process of searching for a document using a place name will be described. The user specifies “location search” on a window (not shown) of the first device with a mouse or the like, or presses a “location search” key on a keyboard. When the “location search” is selected, the control unit 21 of the first device activates the search unit 23. The search unit 23 extracts data from the position operation information storage unit 32. The location ID of the extracted data is searched in the location name storage unit 27, and the location I is searched.
D is replaced with the place name stored in the place name storage 27. FIG. 4 shows the format and an example of the data in the place name storage 27.
Same as 1. The search unit 23 transfers the search results to the control unit 21. The control unit 21 includes a search result display unit 22
Is started, and the search result is transferred to the search result display unit 22. The search result display section 22 displays the search result in a search result display section window. An example of the display in the search result display window of the first device is the same as that in FIG.

In this embodiment, the time is measured, and the time data, the location ID, and the document ID and the document name of the document are stored.
Although displayed, the time may not be measured, and only the place ID and the document ID and the document name of the document may be stored and displayed.

Further, the first device and the second device are connected to the location I.
Although D is transmitted and received, a place name may be transmitted and received. In this case, the place name storage 27 is not required.

Fifth Embodiment In the third embodiment, transmission is performed by communication means such as an infrared transmitter of the first device, and reception is performed by communication means 30 such as an infrared receiver of the second device. In this embodiment,
Conversely, the first device performs reception using communication means 30 such as an infrared receiver, and the second device performs transmission using communication means 30 such as an infrared transmitter.

The device configuration of the first device and the second device of the present embodiment is the same as that of FIG. However, the communication means 107 is
The first device is an infrared receiver, and the second device is an infrared transmitter. The functional configuration of the first device of this embodiment is the same as that of FIG. 43 of the fourth embodiment. However, the communication unit 30 only receives the location ID from the second device.

FIG. 48 shows the functional configuration of the second device of this embodiment. The second device includes a communication unit 30 for transmitting the location ID, a transmission storage unit 33 for storing the transmitted location ID, and a control unit 21 for controlling these components.

The format of the data in the document data storage unit 25 of the first device is the same as that of FIG.

The processing flow of the control unit 21 of the first device is the same as that of FIG.

The processing flow of the search unit 23 of the first device is as follows.
This is the same as the processing flow of the search unit 23 of the first device in the fourth embodiment. However, the communication unit of the second device transmits the place ID at all times or at regular intervals.
706 and processing 3904 in FIG. 47 are not performed.

In addition, the document data storage unit 2 of the first device
5, the processing flow of the control unit 21 of the first device, the data format in the operation history storage unit 31, the data format in the place name storage unit 27, and the like are the same as those in the fourth embodiment. is there.

Embodiment 6 Embodiments 1 to 5 allow a document that has been browsed, modified, or created in the past to be searched at any location based on positional information. In the first, second, fourth, and fifth embodiments, the computer was able to know the position of the computer itself.

In this embodiment, the information of the position of the computer is compared with the information of the position of the document browsed, corrected or created in the past, and the document of the information of the matched position is displayed.

FIG. 49 shows a processing flow of the search unit 23 of the embodiment corresponding to the first and second embodiments.

FIG. 50 shows a processing flow of the search unit 23 of the present embodiment corresponding to the fourth and fifth embodiments. In FIG. 50, by checking not only the place ID but also the time of the day, for example, when different meetings are held in the same meeting room in the morning and afternoon, the minutes corresponding to each meeting can be opened. In addition, it is easy to extend to store and match the day of the week.

In addition, the processing flow of the control unit 21, the format of each data, and the like are the same as those in the first, second, fourth, and fifth embodiments, respectively, and thus description thereof will be omitted.

Seventh Embodiment In the display of the search result display unit 22 shown in FIGS. 11 and 13 in the first embodiment and the display of the search result display unit 22 shown in FIG. It only displayed the name of the place. In this embodiment, an image is displayed in addition to the name of the place.

The functional configuration of this embodiment is the same as that of FIG. 10 of the first embodiment, and the search result display section 22 has a map database. The processing flow of the search result display unit 22 is the same as that in FIG. FIG. 51 shows details of the display process of the process 1002 in FIG. FIG. 52 illustrates an example of the display of the search result display window. FIG. 52A shows an example in which a map of Japan is displayed, and FIG. 52B shows an example in which a world map is displayed. Here, the search result display unit 22 displays the X and Y coordinates of the search result display window so as to correspond to the longitude and the latitude. Further, FIG. 53 shows an example of the display after “Laboratory (Kobe)” is selected in FIG. 52 (a). Other processes and the like are the same as those in the first embodiment, and a description thereof will not be repeated. Example 1
52, it is possible to simultaneously display the corresponding document names at the positions of all the place names at the display stage of FIG. In both cases,
A predetermined number of document names are displayed, and when there is a document exceeding the predetermined number, another document name can be displayed by designating a mouse or the like.

Similarly, in the third and fourth embodiments, the search result display section 22 has a map database and can perform display processing. FIG. 54 shows an example of the search result display window. FIG. 54A shows a character string representing a department name of a company organization, and a location name, a document name, and a time are displayed in association with them. FIG. 55 shows the format and an example of the data in the map database in this case. In FIG. 54A, the organization names are displayed in the form of a tree structure in the order of the numerical values of the organization levels in FIG. 54 of FIG.
In (b), the place name is displayed in association with the video of the place. The display of the image of the place and the association with the place name may use computer graphics technology.

Further, it is possible to display documents in a place in a certain area as a group, and to display the position of each document when the place is selected. For example, when displaying on a world map, documents in Japan are displayed together at one location, and when this location is designated with a mouse or the like, a map of Japan is displayed instead of the world map, and each document is It is displayed at the corresponding position in each place on the Japan map. Alternatively, for example, in the display of the company-wide organization chart of the company, the documents of the head office are collectively displayed at one position, and when this position is designated with a mouse or the like, the head office organization chart is displayed instead of the company-wide organization chart, Each document is displayed at a corresponding position on the head office organization chart.

In the above embodiment, a method of searching for a document using position information as a search key has been described. Conversely, for a document displayed by a user, the document can be viewed, modified,
Or by displaying the information of the created position,
It can be used as a clue for the user to remember the contents of the document or the history of browsing, modifying, or creating the document in the past. Specifically, the location document information storage unit 26 or the operation history storage unit 31 is searched using the document ID of the searched document, and the location name may be displayed according to the latitude and longitude of the document ID or the location ID.

Further, in the above embodiment, the document is searched, but the search target is not limited to the document. For example, a program may be used. For example, in the case of a program,
Specifically, a program ID and a program may be used instead of the document ID and the document.

Embodiment 8 In the above embodiments, information on the position of the apparatus is used as a clue for retrieval. In the present embodiment, a document group is displayed using information such as an address included in the document, thereby making it easier to search for the document.

The device configuration of this embodiment is the same as that of FIG.

FIG. 56 shows a functional configuration of this embodiment. In the present embodiment, a document data storage unit 25 for storing document data and the like, an input unit 20 for inputting a display command and a search command for document data, a search unit 23 for searching for a document according to the search command, a document The display unit 34 includes a display unit 34 for displaying data and search results, a map database 36 and a position database 37 used for displaying document data and search results, and a control unit 21 for controlling these components.

First, the case where the address of the writer is written in the document as shown in FIG. 57 will be described. The extraction of the address will be described in detail later, but the character string in the first document frame may be analyzed using document chapter frame recognition. The method of analysis may be a method used in a machine translation system or the like. Thus, the writer's address is extracted from the document, and the document ID and the address are stored in the document data storage as shown in FIG.

FIG. 59 shows a format of the position database and an example thereof.

The display unit 34 compares the address of each document with the address in the position database 37 for the searched documents.
When the address constituting the document address exists in the position database 37, the latitude and longitude values are given to the document. After that, the map in the map database 36 is displayed,
The document name is displayed at the coordinate position corresponding to the latitude and longitude values of the document. Further, it is possible to display not only the searched documents but also all the documents in the document data storage unit 25. At this time, in the same manner as in the seventh embodiment, a predetermined number of document names are displayed, and when there is a document exceeding the predetermined number, another document name is displayed by designating a mouse or the like. Can be.

Next, as shown in FIG. 60, a case where only the affiliation of the writer or the affiliation of the writer is described in the document will be described. The extraction of the affiliated organization or affiliated department name can be performed as described above. FIG. 61 shows the format of the document data storage unit 25 and an example thereof.

FIG. 62 shows a format of the position database 37 and an example thereof.

The display unit 34 compares the name of the institution or department of the writer of each document with the name of the institution or department in the location database 37 for the retrieved documents. If present, the latitude and longitude values are added to the document. Thereafter, the map in the map database 36 is displayed, and the document name is displayed at a coordinate position corresponding to the latitude and longitude values of the document. Further, it is possible to display not only the searched documents but also all the documents in the document data storage unit 25. At this time, in the same manner as in the seventh embodiment, a predetermined number of document names are displayed, and when there is a document exceeding the predetermined number, another document name is displayed by designating a mouse or the like. Can be.

As in the case of the seventh embodiment, documents in a place in a certain area can be displayed as a group, and when the place is selected, it can be displayed at the position of each document.

As in the case of the address, it is possible to extract the creation date, the announcement date, or the issuance date written in the title portion of the document or the like, and display the date and the document list as shown in FIG. it can. Similarly, when there are a plurality of document data storage units 25, the date, the name of the document data storage unit 25, and the document name assigned to the document data storage unit 25 where the document was present are displayed as shown in FIG. be able to.

Next, the second invention will be described with reference to FIG.

The second invention relates to a search device that stores a schedule created using a known schedule management system and data such as documents in association with each other, and extracts desired data from schedule items.

The method of associating the data with the schedule includes: (1) a method of manually associating a user without using a time; (2) a method of analyzing a date or the like written in the data and associating the data; and (3) data creation. There are methods using the time, data change time, access time, and the like.

As a method of storing the schedule item specified by the above means in association with the target data, there is a method of adding link information to the target data to the schedule and storing the schedule.

Further, as a method of retrieving data stored in association with the above method, there are the following methods. (1) A method of directly searching for desired desired data by designating schedule items using a schedule table as an interface ( 2) There is a method such as searching for “…… meeting” from the schedule table first, and using the extracted schedule items to find the associated data, such as “Material used for“ …… meeting ”]. .

The conventional schedule management device can manage only the schedule, and does not have a function of managing related materials such as materials prepared in advance and records of meetings actually held according to the schedule.

In the present embodiment, in addition to the conventional schedule management device, it has link information to the target data, determines a schedule to be linked, and follows the link to the target data from the result of the schedule search. .

A conventional schedule management device will be described with reference to FIG.

The data structure of the schedule is as follows.

Schedule: = (date and time, date and time) date and time: = [from start time] to [end time] (at least one is required) message: = one or more keywords keyword: = {location | schedule name | Other | Content | Other memos} Schedule table: = A table in which schedules are arranged in chronological order The structure of the above schedule table manages schedules in order of date and time (time), so the directory structure is the same as the year, month, and day. The schedules are stored one by one in individual files, using the time of each schedule as a file name.

'92 --- Jan --- 1 --- 0700, 1300, | -Feb | -2 --- 1000, | -... |-.. | -Dec | -31-FIG. 65 In, the schedule creation unit creates a schedule based on data input from the input unit 41 such as a keyboard. The created schedule is stored in the schedule storage unit 48.

The basic functions are to store the schedule by specifying the date and time, and to retrieve the schedule by specifying the date and time. In some cases, there is a function of searching for a schedule that includes a key using a message as a key.

The link information will be described. When recording the target data, link information that associates the schedule with the target data is added to the schedule. For example,
In the case of the schedule table having the above-described structure, link information to the corresponding data in the target data storage device is written in a file corresponding to each schedule. It is also easy to create link information from the target data to the schedule and extend it to a bidirectional link.

There are the following three types of link information depending on the order of the linking time and the time of the target schedule. Based on the schedule, there is an "advance" link indicating that it was created in the past, a "current" link indicating that it was recorded at the same time as the schedule progressed, and a "post" link indicating that it was recorded after the schedule ended is there.

FIG. 66 shows an example. 60 is a schedule table. Suppose that the material 61 of the meeting to be held from 13:00 on March 2 was created in the morning of the 1st. Since the material creation time is before the schedule, "Advance" as shown by the solid line
Create a link. Since the quick recording or recording tape 62 during the meeting at 13:00 on the second day is created at the same time as the schedule, a “present” link is provided as shown by a double line. The "current" link is useful for keeping track of what is happening. Since the minutes 63 created on the day after the meeting are created after the schedule, a “post-mortem” link indicated by a broken line is provided.

Next, means for associating a schedule with target data using the link will be described. As a method of specifying a schedule item to be associated with the input target data, there are three methods: a method of explicitly instructing by a user, a method of specifying from target data, and a method of specifying from environmental information.

When recording the target data, the user explicitly indicates which schedule is to be linked.
If the schedule is shown as a candidate and is selected by the user, it becomes easier to give an instruction. Material 61 in Figure 66
If it was created in the morning of the first day of the month, there is no schedule at that time. Therefore, when recording the material, the preceding and following schedules are displayed as in the schedule table 60, and the user is allowed to select which schedule is linked. If the schedule item is specified, the type of link can be determined by comparing the times. For example,
When the user selects the meeting on March 2, the times are compared and a “prior” link is established.

Instead of creating target data and then associating it with a schedule, it is also possible to edit a document after selecting a schedule in advance. For example, in the morning of March 1, the schedule 60
Is displayed, and when the creation of materials is started after the meeting on March 2 is designated, the links are automatically linked when the creation is completed.

The schedule is specified by analyzing the target data and recognizing the title and date. When creating a notice of holding a meeting in a certain format or creating minutes, a meeting name and a holding date and time can be easily specified. This is particularly effective when recording target data having a fixed format. For example, if the minutes 63 are created on March 3, the minutes indicate that the meeting is from 13:00 to 15:00 on March 2, so a “post-event” link may be provided. it can. Specifically, in the basic drawing, for example, a character string “minutes” and the next date and time may be extracted from the target data from the target data input unit 47 by the basic information layout extracting unit 54. In addition, the date and time after the character string “holding date and time” may be extracted. Also, recognize the text frame,
What is necessary is just to refer to the character string in the first text frame and the character string in the schedule. For example, in the case of the “development meeting minutes” and the “development meeting”, it can be seen that the “development meeting” matches, so that the links can be made.

It is also conceivable to specify a schedule using environment information other than the target data. The other party is identified from the audio information, the conference name is read from the image information, and items likely to be described in the schedule are extracted. If a sufficient recognition result cannot be obtained, a schedule is searched using this item and the current time as keys, and the schedule is presented to the user in order from the schedule with the highest matching degree.

When the schedule for linking is determined by any of the above methods or a combination thereof,
Decide which type of link will be "advance", "present", or "post".

By comparing the current time at which the target data was created with the time of the schedule, a “prior” link for a schedule after the current time, a “current” link for the same time as the present, In that case, it is a “post-mortem” link. Also, for example, when determining the type of link information after a certain time, not when the target data is created, the “data new creation time” in the access information of the target data described later is compared with the schedule time. do it.

Next, a method of searching for data associated with a schedule by the means will be described.

A schedule is searched by specifying a keyword written in the subject such as date and time, place, partner, subject, and the like. This function is used in the conventional schedule management device.
It has already been realized. In addition to the conventional functions, by associating them with access information and position information, it is possible to attach a modifier to a keyword and use a plurality of keywords connected to each other, or to reduce the degree of limitation. For example,
Search methods such as "accessed around x time" and "created after returning from xx (place)" can also be realized.

After the schedule is specified in this manner, the target information is reached by following the link information. The point that the link to the target data is followed in conjunction with the search is a major difference from the conventional schedule management device.

Modifiers other than keywords described in the schedule can also be used to limit the search range. For example, it is possible to judge that the “advanced” link to the xxx conference from the “designed for” part of the search specification “material created for the xxx conference”. From "after returning" of "created after returning from xxx", it can be determined that the target data created after the time of returning to xxx should be searched.

An example of a search in the search unit 50 is shown in FIG.

FIG. 66 (b) shows a state in which the schedule table is displayed at the conference start time on March 2 at 13:00. In the schedule table, "Advance" link column, "Current"
There is a link column and a "post-mortem" link column. The mark 64 indicates that there is a material prepared in advance. When 64 is designated by an input device such as a mouse or a pen, a material selection screen 65 is displayed and a desired material can be selected. 66
Is selected, the contents are displayed as 66. If there is only one linked document, 6
It is also possible to set so that the content of 66 is displayed immediately without displaying the material selection screen of 5.

FIG. 67 (a) shows only the column on March 2 when the schedule table was opened at 16:00 on March 2. A tape mark 67 is displayed in the “current” link field, indicating that a recording tape of the conference exists. When this mark is selected, the tape can be played and heard.

FIG. 67 (b) shows the column for March 2 in the schedule table after the minutes have been created. There is a mark 68 indicating that there is a material in the "post-mortem" link field. By selecting 69, it is possible to view materials prepared in advance, or select 67 to reproduce the state of the meeting recorded on the recording tape. The target data may be an electronic mail document. In the case of an e-mail document, the link to the schedule described above can be realized by using the transmission date and time and the subject (title) described in the e-mail.

A search other than the search using the XXX conference as a keyword is also possible. For example, when "materials of a development meeting opened after head office" is input, "head office", "after", and "development meeting" are first extracted from the input character string. Next, the schedule table is searched using the keyword "head office". In the schedule table 60 of FIG. 66, it can be seen that March 1 corresponds. Next, a character string of "development meeting" is searched after March 1, and it is found that March 2 is applicable. Then, the names of the conference materials linked on March 2 may be displayed.

As a modified example, the target data is a TV program,
If a schedule is read as a program guide, it can be applied to a video recorder. Recording reservation is made in the same manner as in the past. A program table (schedule) is stored by another input means.
When extracting a desired program from a recorded tape, search by program name or performer.

As another modified example, when a relative time from a certain reference is used as a method of specifying a time, a birthday or a joining date is set as a reference for timekeeping. You can search for records by specifying "xx year job".

Further, it is possible to search for a schedule. For example, when "development meeting" is entered as a search key and the character string of "development meeting" is included in the schedule, the schedule and the title of the document linked to the schedule can be displayed. FIG. 68 shows a display example.

Next, the third invention will be described. Third
The invention of the present invention is a search device that extracts layout information of data when creating and storing data such as documents, stores the data in association with the layout information, and extracts desired data with the layout information. is there. The layout information includes the number of columns, the size and position of the document frame, the number of lines,
The number of figures and tables, the size and position of the figures and tables, and the type of figures and tables can be searched using the layout information. However, several types of patterns are prepared for the size and position of figures and tables. , Or similar data can be searched based on the layout of a diagram drawn by hand on a tablet or the like. The details will be described below.

Embodiment 1 FIG. 69 is a block diagram showing an embodiment.

The image data, character data, graphic data, and the like constituting the document are input to a document data input unit 1 composed of, for example, an image scanner, a communication cable, a CDROM player, and a floppy (registered trademark) disk drive.
01, and stored in a document data storage unit 102 including a memory and a hard disk. The layout information extraction unit 103 extracts layout information in the document data from the contents of the document data storage unit 102. For example, layout information includes the number of pages, the size of pages, the number of columns, the size and position of a text frame, the number of lines, the size and position of a line frame, the size of characters, the number of figures, the size and position of figures and tables. , Chart types, etc. are extracted. These are stored in the layout information storage unit 104.

The user inputs the layout image of the document to be searched through the search key input unit 105. The search unit 106 uses the search key input from the search key input unit 105 to search for a corresponding document from the layout information storage unit 104. The document output unit 107 includes the search unit 10
The document found in step 6 is presented to the user.

The operation will be described below using the input data example of FIG.

The data input from the document data input unit 101 shown in FIG. 69 is, for example, document data composed of character data and graphic data or image data as shown in FIG. When the document data is viewed by the user, the document data is read into the document data storage unit 102 via the document data input unit 101, and displayed on the document output unit 107 including a display, a printer, and the like.

The document data once browsed by the user is sent to the layout information extracting unit 103. Regarding the operation of the layout information extraction unit 103, extraction from image data is described in Japanese Patent Application No. 63-7518, and extraction from code information such as character data and graphic data is described in Japanese Patent Application No. 61-20.
It is omitted here because it is familiar with No. 6835.

For example, assuming that the positions and sizes of the text frame and the chart as shown in FIG. 71 are extracted, the columns (1 column, 2 columns, 2 titles) of each page are obtained by the following method. Determine the column layout and the type of chart layout and classify them.

FIG. 72 shows an example of a column template and constraints. First, if there is a case where the text frames are very close to each other in the extracted layout information, the comprehensive text frame is redefined. Next, a column template having columns higher than the number of text frames is selected. The value of the layout information such as the position of the extracted text frame is substituted for the variable of each column in the column template. However, if there is a chart close to the text frame, the side facing the adjacent side is the boundary of the text frame. As a result of the substitution, if the constraint condition unique to each column template is not satisfied, matching is performed with the next column template. As shown in FIG. 71, in this example, the text frame is 2
One has been extracted. Since the distance between the adjacent text frame and chart frame is 20 mm to 30 mm, it is not considered to be close. Therefore, it is applied to a template having two or more columns. When applied to a two-column template, SX1 = 17
0, SX2 = 100, SY1 = 50, SY2 = 200, PY1 =
30, PY2 = 100, PX1 + SX1 = 40 + 170 = 21
0, PX2 = 20, which does not satisfy the constraint condition at all. On the other hand, assuming that the first sentence frame is a title column, the second sentence frame is a second column, and the first and second tables are combined into a third sentence frame by applying to a two-column set with a title, SX2 = SX3
= 100, SY2 = SY3 = 200, PY2 = PY3 = 100,
PX2 + SX2 = 120 <PX3 = 130 <PX1 + SX1 = 21
0, PY1 + SY1 = 80 <PY2 = 100, which satisfies the constraint. The column of the first page is determined to be a column with title.

Next, classification is made according to the diagram layout type. FIG. 73 shows an example of a list of diagram layout types in two columns with titles. In the example of FIG. 71, there are two sentence frames, so that F, G,
H ... The third column can be classified into the G type, which is composed only of charts.

The title of the document data and the extracted layout information and classification are stored in the layout information storage unit 104, for example, as shown in FIG.

The search key input unit 105 inputs not only the title and author name of document data but also layout information via, for example, the following interface. First, a list of document columns as shown in FIG. 75 (a) is presented to the user, and an item close to the user's memory is selected using, for example, a mouse or a keyboard. Assume that the user has selected a two-column system with a title. The search key input unit shows a list of chart layout types in two columns with a title as shown in FIG. 75 (b), and prompts the user to select again. In the search unit to which these keys are sent, a G-type page is searched in two columns with a title from the data in the layout information storage unit, sent to the document output unit 107, and presented to the user. When a plurality of corresponding pages are found, the pages may be presented sequentially or reduced and several pages may be presented at a time and selected. Alternatively, the layout of a diagram drawn by the user himself using a tablet or the like may be used as a search key, and a document may be searched based on the search key.

In some cases, the storage of the user's layout is ambiguous. For example, the G type in FIG. 73 may be confused and stored with the C type, D type, or H type by the user. Therefore, it is conceivable to store the easily confused diagram layout type as a similar diagram layout type in the search unit, search for a page having a similar layout, and present the page to the user.

When searching by chart type, the search key input unit 105 inputs the chart type via an interface as shown in FIG. 76, for example. For example, FIG.
As shown in (a), the type of the chart is indicated, and from those, the one that stores the page to be searched is selected. If there are multiple charts on the page, it is sufficient to specify multiple charts. As shown in FIG. 76 (b) or (c), a screen for inputting detailed classification for each of the selected items is shown. here,
Since the graph is selected, the type of the graph is shown. Here, when the bar graph is selected, the information is passed from the search key input unit to the search unit. Layout information storage unit 104
Is shown to the user on the page on which the bar graph is laid out in comparison with the chart type. Here, the image of the page is 1
It may be shown one by one or several at once. Also,
You may cut out and present only the part of the chart. For example, as shown in FIG. 77A, only the bar graph portion of the searched page is shown to the user. When a plurality of searches are made, if the user presses a button such as "next candidate", a bar graph of the next searched page is displayed as shown in FIGS. 77 (b) and (c). Alternatively, as shown in FIG. 78, only graphs of a plurality of search pages may be displayed in a list at a time.

The layout information extraction unit 103 has features such as the page that the user has been browsing for the longest time, the first page, and the page with figures and tables, instead of all the pages of the input document. Layout information may be extracted only for a page. Also,
It is also conceivable that the user specifies a page from which layout information is to be extracted. Of course, for a document for which layout information has not been fetched in advance, a corresponding page is searched for while extracting layout information at the time of search.

Next, the fourth invention will be described. 4th
The present invention uses a predetermined device capable of creating, storing, and retrieving data such as a document, and inputs and stores the data, or the data used for each operation when retrieving the stored data. And image information input through an image input device installed at a predetermined position of the device at the time of input, storage, or search, and stores them in association with each other, and extracts desired data using the image information. It is a search device. At the time of data extraction, a change point of the input image information is used, and the change point uses a scene change or brightness of an image as a search for environmental information. The details will be described below.

Embodiment 1 Hereinafter, an embodiment in which a camera is used for input will be described with reference to the drawings.

FIG. 79 is a diagram showing a configuration of an environment information extracting method for extracting a scene change point (time) as environment information. An image memory 70 to which a video is given as an input,
Enter 71.

A histogram of the luminance of the image memories 70 and 71 is calculated by the histogram calculator 72. This calculation is performed for each field. That is, the output of the arithmetic unit 72 is latched by the vertical synchronization code VSYNC,
Internal registers are cleared. If there is no change in the scene, there will be no significant change in luminance between successive fields. Conversely, if the change in luminance is large, it means that the scene has changed. Therefore, if the sum of the absolute values of the differences between the outputs of the histogram calculator 72 between the continuous fields is larger than a certain threshold value, the scene change point Judge that there is, and record the time at that time.

FIG. 80 is a flowchart for explaining the operation of the histogram calculator 72. Here, the screen size is assumed to be M × N.

FIG. 81 is a diagram showing another method of determining a scene change. Instead of using the absolute value of the difference between the outputs of the histogram calculator 72 between the continuous fields, the histogram is recorded, that is, the degree of spread of the distribution is shown. Variance X
Is used as a criterion for determining a scene change. Focusing on the shape of the brightness distribution, it is not affected by changes in the height of the distribution, so that when the brightness of the screen changes in the same scene, there is no need to erroneously determine that there is a scene change. There is.

FIG. 82 is a flow chart for explaining the operation of the distributed X calculator 73.

FIG. 83 is a diagram showing a method for detecting the case where the movement of the scene has stopped.

If the scene does not change for a certain period of time or longer, it can be determined that the movement of the camera has stopped. Therefore, by measuring the time after the scene change,
Stop can be determined.

FIG. 84 shows the structure of the scene change judging section.
FIG. Using color information instead of the brightness described above,
A change in the color of the entire screen is a change in the scene.
You. The input composite color image signal (NTSC
The signal etc.) is converted by a color decoder 74 into a lightness (L) and X
X / Y signals of Y color system are separated and A / D converted respectively.
Digitized by the device 75. Digitized L, X,
Each of the Y signals is supplied to a histogram calculator,
Average value P with togram_L, P_X, P_YAnd variance X _L, X
_X, X_YIs calculated. With the brightness histogram described above,
Similarly, these values are required on a field-by-field basis.
I do. In the scene change determination unit, X_L, X_X, X_YOf the value of
A scene change is determined based on the change.

FIG. 85 is a diagram showing a configuration example of environment information extraction based on still another scene change detection method. A difference between successive fields of an input video signal is always obtained, and a scene change is determined by relaxing its absolute value. The operation between consecutive fields is performed by using a field memory while holding the image one field before.
The figure shows a case where the operation is performed only with the luminance. If the relaxation of the absolute value is larger than the set threshold value, it is determined that a scene change has occurred. The threshold value is held in a rewritable threshold value register.

FIG. 86 shows a case where color information is used. By using the difference between the X and Y signals, a change in color tone can be detected.

In the above description, it is assumed that scene change detection of an input image is performed on the spot in real time (or almost real time). In some cases, scene change detection may be performed later.

In this case, if the time required for detecting the scene change is not real time and may be long, the calculation speed for detecting the scene change can be reduced, and the apparatus scale and cost are reduced. be able to. In this case, as shown in FIG. 87, the raw data (video) from the environment information input unit is first stored in the environment information storage unit, and the raw data is sent to the environment information extraction unit therefrom. Thus, scene change detection is performed, and information on the resulting scene change point sequence is sent back to the environment information storage unit and stored.

If images have been stored, a plurality of scene change detecting means can be applied to one stored image to detect scene changes in parallel, as shown in FIG. 88.
Analysis time can be reduced.

In the above description, a change point of a scene in a video is assumed as a search key, and the detection method has been described. However, in addition to the change point of a scene, for example, a scene where a human appears on a screen may be used. For example, a scene in which a specific individual appears can be assumed as a search key.

The detection of a person is performed, for example, by using the above-described color information to detect a scene in which a flesh color appears. In this case, as a method of specifying the skin color,
For example, a color distribution diagram of the xy color system as shown in FIG. 89 is displayed on the screen of the display, and a point or a region of a desired color (in this case, skin color) is selected from the display with a pointing device or the like. By doing so, a method of designating is conceivable.

As another method of designating a skin color, for example, as shown in FIG. 90, an image is stopped in a certain scene where a human is shown, and a skin color area such as a human face or a hand shown in the scene is removed. A method of specifying by cutting out using a pointing device or the like is also considered.

In order to detect a specific individual, a face area is first extracted from a scene, and then a determination is made by matching the face image of the individual to be searched. In order to extract a face region from a scene, a skin color region is detected by the above-described method, and the area of the region is determined.
The eyes, nose, etc. are searched to determine whether the face is present. Matching methods include correlation calculation, SSDA (Sequential
Similarity Detection Algorithm) is known.

As a method of specifying an individual to be searched, a method in which a scene in which the person is shown is stopped, and a face area is cut out using a pointing device and specified is considered.
When images of a plurality of individuals' faces are registered together with their names in advance, they can be designated by their names instead of by images. Of course, a method of listing and displaying registered faces and selecting them with a pointing device is also conceivable.

If personal face images have been registered in advance, a search is performed using each of them as a template to obtain information about who was in the video instead of "who is it?" Can also.

It goes without saying that the above-described information such as scene change of a video, a scene in which a person is captured, a specific individual, and who is present includes time information.

Next, the operation at the time of retrieval will be described.

One of the central ideas of the present invention is to store a document or a program together with environmental information when it was created / registered / previously referred / last referenced / The document or the program is searched using the environment information. And here, it is assumed that a video is used as the environmental information.

One of the methods of using video for search is shown in FIG.
As shown in FIG. 1, still images, such as scene change points, can be listed in a row on the screen, and when an image is selected from the images, a document or program corresponding to the image is searched.

As shown in FIG. 92, this is realized by attaching a link to a document or program from a characteristic scene in a video such as a scene change point. Most simply, a scene in a video corresponding to a time (creation, registration, modification, reference, etc.) meaningful to a document or a program is set as a feature scene.

Conversely, in order to reproduce the video at each relevant time point for a document or program searched by using means other than the video, as shown in FIG. You just need to put a reverse link to the scene.

In the case of a video, not all scenes to be used as search keys, such as scene change points, can always be cut out.
In particular, in the case of scenes related to human sensibilities, such as "seeing and remembering images", human intervention is essential instead of automatic extraction.

In this case, the searched video is searched for while fast-forwarding or the like. At this time, not only one screen but also the entire video is divided into n and n screens as shown in FIG. , The time required for the search can be reduced to 1 / n. This means that even if the video is on sequential media such as magnetic tape,
This can be realized by simultaneously reading / writing n tracks from / to the tracks.

Alternatively, if n magnetic tape devices are used to store video images in order of 1 / n of the total time and n devices are simultaneously played back at the time of playback, the same applies to sequential media. The effect of can be realized.

As another example of the retrieval, as shown in FIG. 137, the video captured by the camera at the start and the end of the input of the document and the still image at the time when the scene is changed during the input are arranged side by side. indicate. The user can select a required document by looking at these images.

It is also possible to output not only the still image being input, but also the title of the mail that came in between, and the voice recorded during that time.

In the above description, a video image is used for searching a document using a camera installed in a computer. Similarly, a small camera can be attached to a name tag or a batch to accumulate images, and the accumulated images can be transferred to a computer to perform the above-described processing, thereby being used as an action record.

Next, the fifth invention will be described. Fifth
According to the invention, a search device that stores the data creation start time or the data creation end time required when creating data such as a document together with the data, and extracts desired data using these times It is. From the start time and end time of the created data,
The degree of urgency is obtained, and this degree of urgency is used as a search key. The time interval provided for the creation from the data creation start and end times is schematically represented. For example, a schematic diagram is used as a search key. According to the present invention, when editing stored data such as a document, an access start time or an access end time to the data is stored together with the data, and desired data is extracted using these times. Search device. The number of access starts is counted to obtain a reference frequency, and this frequency is used as a search key (a weekly report is accessed every week). The access frequency (density) of predetermined data over time is schematically represented. The schematic diagram is used as a search key, and data edited based on the designated schematic diagram is extracted. The details will be described below.

Embodiment 1 FIGS. 96 to 99 show a basic diagram and an example of the configuration, contents and processing of the retrieval key analysis unit 53 in FIG. 95. In FIG. 99, "-1" in the execution unit indicates the relative position from the position of the word in the execution part, and means that the execution unit extracts the word immediately before the word "society" shown in the condition unit. .

In the basic diagram and FIG. 95, the access information detecting section 45 extracts information related to data access operation when creating / editing text data, and stores it in the environment information storage section 48 as access information. Information storage unit 4
8 is deleted. Table 1 shows the operation events detected by the access information detection unit 45.

[0262]

[Table 1]

With these operations, access information elements having the attributes shown in Table 2 are generated or deleted.

[0264]

[Table 2]

Of these, the data amount at the time of new data creation is generally zero, but when new data is created by changing the data file name, it has some amount from the beginning. The access information is configured by adding the number of references, the degree of urgency, the file name of the copy source, the total number of printouts, and the like as other attribute information of the document to the above list of access information elements. Here, the number of times of reference indicates the total number of access information elements whose data contents do not change. Further, the urgency is a value representing the amount of description per unit time, and when the urgency of one sentence is E, E is derived by the following equation.

[0266]

(Equation 1)

Here, each variable has the meaning shown in Table 3.

[0268]

[Table 3]

The total number of printouts is the sum of the number of printouts in each access information element.

FIG. 100 shows an example of the access information. The document 1 and the document 1-access information are linked by a link. The document 1-access information is described by an access information element list and other attribute information. Each access information element is generated each time one access occurs, and is added to the list.

FIG. 101 shows the flow of processing performed by the access information detecting section in response to the "new data creation" operation and the "save after data name change" operation.

FIG. 102 shows the flow of processing performed by the access information detecting section in response to the "data access start" operation.

FIG. 103 shows the flow of processing performed by the access information detecting section in response to the "data access end" operation.

FIG. 104 shows the flow of processing performed by the access information detecting section for data printout.

FIG. 105 shows a flow of processing performed by the access information detecting section for data deletion.

In the basic diagram, the search unit 50 on the plane 95
Using the access information stored in the environment information storage unit 48, a search for document data using keywords as shown in Table 4 is performed.

[0277]

[Table 4]

FIG. 106 is a flowchart of a search process using the information on the timing (timing pattern 1) in Table 4.
As a result, by comparing the range from time S to time E with the time information included in the access information, it is possible to search for document data that has been created, created, started, or created.

FIG. 107 is a flowchart of a search process using information on a timing (timing pattern 2) in Table 4.
Thus, by comparing the range from the time (T-dT) to the time (T + dT) with the time information included in the access information, it is possible to search for the document data that has been created, or the creation start or the creation finished.

FIG. 108 is a flowchart of a search process using the information on the period in Table 4, and the time taken from the new creation time of the access information to the latest access end time is equal to the period T with an error of e%. Document data that can be included can be searched.

FIG. 109 is a flowchart of a search process using the information on the reference frequency in Table 4, and it is possible to search for document data having access information whose reference count satisfies the range D. Here, the number of times of reference indicates the total number of access information elements whose data contents do not change. Further, Noften is a certain number of times that can be empirically regarded as “referred many times”, and has a different value depending on the user.

FIG. 115 is a flowchart of a search process using the information on the print frequency in Table 4, and it is possible to search for document data having access information such that the total number of printouts satisfies D. Noften is a fixed number of times that can be empirically regarded as “printed many times”, and has a different value depending on the user.

FIG. 110 is a flowchart of a search process using the information on the urgency of creation in Table 4, and the document data can be searched according to the urgency at the time of creation.

[Search Example 1] "A sentence that started writing around February and was referred to several times" [Search Example 2] "A sentence created around March and written in a hurry in a day" [Search example 3] “Sentences created by elaborating over a one-month period” FIG. 111 is an example in which the attributes of the access information element are displayed. The left end indicates the new creation time or the access start time, and the right end indicates the access. Indicates the end time. The height of the figure indicates the amount of document data at each time of new creation, access start, and access end. Furthermore, the presence or absence of a change in the data content is represented by a pattern of the figure, and the new creation time and the latest access end time are represented by colored small circles. The link to the copy source file is represented by an arrow.

FIG. 112 shows an example in which access information of several sentences is displayed on a monthly basis based on the display method shown in FIG. Sentence 1, sentence 2, and sentence 3 in FIG. 112 have characteristics that can be searched by the above [Search example 1], [Search example 2], and [Search example 3], respectively.

FIG. 113 shows the same data as in FIG. 112 displayed again on a weekly basis. Also, the search unit in the basic drawing uses the scheduling information generated by the schedule creation unit as time information,
The search can be performed using words described in the schedule instead of the absolute time. Thus, for example, a search such as “a sentence created at the time of the dissertation of the XX Society” can be performed. Specifically, as shown in FIG. 97, a society, a dissertation, and a deadline are extracted from the search key, and a character string of no schedule data including these character strings is specified. The search process may be performed as described above.

FIG. 114 shows document access data in combination with schedule data.

Next, the sixth invention will be described. Sixth
According to the invention, when inputting data such as a document or a voice, the voice characteristic data is detected from the voice data, and the detected characteristic data is stored in association with the data. Is a search device that extracts desired data by using. The characteristic data is a speaker or the number of speakers obtained by performing speaker recognition from voice data. The characteristic data is data of an environment (room size, inside a train, or the like) in which data such as documents and voices are input. The characteristic data schematically represents the density and tension (corresponding to laughter, liveliness, or quietness) of data such as voice. This will be described in detail below with reference to the basic diagram of FIG.

Next, the case where acoustic media such as surrounding sounds and voices are used as environment analysis target data will be described.

The environment information input section 80 has an acoustic media input device such as a microphone, takes in acoustic data such as ambient sound and voice, performs A / D conversion via a low-pass filter, and performs sampling and quantization. I do. If information on discretization processing such as sampling frequency and quantization bit number and information on input devices are added to the acoustic data, data extraction processing when input data of different formats due to various discretization processing are mixed Useful for

As a general rule, the data to be analyzed is always input while the user is using the system. That is, it is always input when inputting, editing, searching and outputting search target data, or when the user adds or corrects environmental information. However, if the data to be analyzed is always taken in, the amount of data will be enormous, so if there is no need to search using environmental information obtained from the data to be searched at the time of its use, omit the input of the data to be analyzed. Is also good.

The input environment analysis data may be multi-channel acoustic data using a plurality of microphones. For example, in a conference where multiple speakers are present, each speaker has a microphone, or the sound data reconstructed by using a microphone array to increase or decrease the sound emitted from a sound source at a specific position or direction. Can also be entered. In this case, information indicating which microphone is input, which position or which direction of arrival is emphasized or attenuated is added to the acoustic data as information on the input device.

The environment information extracting unit 43 immediately and automatically analyzes the input environment analysis target data. If the environmental analysis target data is acoustic data,
For example, identification of the ambient sound level, the type of the ambient sound, identification of the place of use based on the reverberation characteristics of the surroundings, speaker recognition, speech recognition, and speech understanding.

Ambient sound level is determined by inputting search target data,
It can be obtained by calculating the signal power of the audio data at the time of editing, searching and outputting. The level of the ambient sound is
Depending on whether the threshold value is exceeded or lower than a preset threshold value, environmental information about the noisy surroundings, such as the location where the system was used is noisy or quiet, can be obtained. Can be

The type of the ambient sound can be identified by calculating the similarity between the spectrum pattern data obtained by frequency analysis of the acoustic data and the frequency spectrum pattern data corresponding to the type of the ambient sound. For example, it responds to the sounds of applause, laughter, etc., the sounds of the user or the person present at the place of use, the running sounds of air conditioners and automobiles, the audible sounds of medium-wave radios, telephone calls, etc. Using standard frequency spectrum pattern data, the type of sound source around the place where the system was used,
Furthermore, environmental information about the place of use is obtained. The method of identifying surrounding sounds is the same as that used in word speech recognition. In word speech recognition using waveform data of surrounding sounds instead of word speech, a standard for matching that was created for each word was used. What is necessary is just to create typical frequency spectrum pattern data for each type of ambient sound.

When the system is used, a test sound is emitted, the reverberation sound is collected, and the reverberation characteristic is obtained, so that the use place of the system can be identified. If the user does not move while using the system, a test sound such as a beep sound may be output at the start of use of the system to collect the reverberation sound. In this case, in discriminating the ambient sound, standard frequency spectrum pattern data may be individually created for each use location. As a result, the use place is identified, for example, the place where the system was used is outdoors, a hall where reverberation sound remains for a long time, or a seat where the user always sits, and environmental information on the use place is obtained.

When used in a place where a plurality of speakers are present, it is possible to know who the utterance is by speaker recognition. Further, as described above, a speaker can be specified without having to perform recognition processing by providing a microphone for each speaker and inputting from different channels, or by giving directivity using a microphone array. Such information on the speaker provides not only the utterance of the utterance but also environmental information such as who was nearby.

If a standard frequency spectrum pattern of a phoneme or a word to be recognized is prepared, the similarity of the sound data as the environmental analysis target data to the spectrum pattern data by the frequency analysis is calculated. And words.

If a phoneme is detected, a word detection result is obtained using a method such as dynamic programming (DP). When detecting a word, word spotting is performed to obtain a word detection result. In addition, multiple possible word detection result strings that can be combined with detected words
The word recognition result can be determined by performing the semantic analysis.
In addition, it is possible to obtain the meaning of the utterance made by the word string forming the recognition result.

Here, the meanings of the words and word strings to be recognized need to be set prior to analysis. That is, only the words and meanings set before the analysis are automatically obtained as an analysis result when the environment analysis target data is input. Therefore, when performing a search using environmental information that uses the meanings of words and word strings that were not set when the environmental analysis target data was input, perform the settings again and replace the environmental analysis target data with the environmental analysis target data. Retrieve it from the storage unit. As described above, it is necessary to set information to be obtained as a recognition result in advance, not only for speech recognition but also for analysis of speaker, recognition of surrounding sound, recognition of surrounding sound, and the like, and analysis in general. If the environment information required at the time of the search is insufficient, the reanalysis of the environment analysis target data may be performed similarly.

As an example of a personal multimedia filing system using acoustic data as environmental information, a case where the present invention is applied to a conference file will be described.

Data input to the system is multimedia data such as sound, images, characters, and the like.

Here, the input multimedia data is the data to be searched.

Also, at the same time as inputting or outputting the search target data, audio data for extracting environmental information is taken in. The environmental information at the time of input and output here means who,
This is information indicating who, when, where, what, and how the target data was input or output. Among them, as the environmental information at the time of input, the same acoustic data as the target data may be used as the acoustic data for extracting the environmental information.

By adding such environment information, the search becomes easier when the input multimedia data is searched and output again later.

[0306] The system automatically extracts environmental information from the acoustic data and stores the environmental information by linking it to the target data.

The target data input unit 47 is, for example, a microphone,
It has means for inputting target data such as sound, images, characters, etc. of a scanner, camera, pen, mouse, keyboard, and the like.

The input of sound data to the system is taken in through a microphone. If it is not necessary to extract the environment information in real time, the acoustic data taken in from the microphone may be temporarily stored in the target data storage unit 49 and used when extracting the environment information.

In order to improve the quality of sound data and to improve the accuracy of extracting environmental information when there are a plurality of sound sources, such as when there are a plurality of speakers, the sound data is fetched using a plurality of microphones and a multi-channel It can also be an input.

[0310] Search target data other than acoustic data, for example, document / video / image data such as materials distributed at a conference, image data such as scenery of a conference hall, and text / image data such as a memo, are, for example, a scanner, a camera, a pen, a mouse, and the like. Input via an input device.

[0311] The target data storage unit 49 stores the multimedia data fetched by the target data input unit 49 together with the input time.

[0312] The environment information input section 80 has sound input means such as a microphone.

Similarly to the target data input unit 47, acoustic data such as the voice of the speaker, the voice of the audience, and the ambient noise
Capture through a microphone. Similarly, when there are a plurality of speakers, a plurality of microphones can be used and multi-channel input can be performed.

As shown in FIG. 118, the environment information extracting section 43 includes a sound source estimating section 90, a voice section estimating section 91, a voice recognition understanding section 92, a speaker information managing section 93, and a conference state estimating section 94.
Having.

The sound source estimating unit 90 determines the position and direction of a sound source included in the sound data captured by a plurality of microphones in the environment information input unit 80, and estimates a sound data component emitted by a specific sound source. The simplest configuration example of the sound source estimating unit is a method of estimating a voice component of a speaker by preparing a microphone for each speaker and associating the speaker with acoustic data captured by each microphone. Also, even when the number of speakers and the number of microphones do not match, it is possible to control the acoustic directivity by convolving the acoustic data captured by each microphone and controlling the convolution coefficient, A plurality of speakers can be discriminated from the difference in the direction of arrival of the sound wave, and a component of a specific speaker can be estimated, or a component other than the specific speaker can be estimated.

The sound source referred to here is not necessarily limited to the voice uttered by a human, but may be ambient environmental noise.

[0317] The voice section estimation unit 91 estimates a temporal section in which a voice exists in the time series of the audio data. For example, when determining the beginning of the sound, a certain power threshold and a certain time threshold are set in advance, and when the power of the signal exceeding the power threshold is maintained for more than the time threshold in the sound data, the power threshold is exceeded. Is determined as the starting point.

The termination can be determined in the same manner. In addition, by setting the power threshold and the time threshold to different values for the start end determination and the end determination, the voice section can be estimated with higher accuracy. From the speech signal component of the specific speaker obtained by the sound source estimator,
The beginning and end of the speaker's utterance can be estimated.

[0319] The speech recognition / understanding section 92 performs word recognition and speech understanding by acoustic analysis, syntax, and semantic analysis of acoustic data, and extracts the meaning content of the utterance. In the acoustic analysis, frequency analysis of a short section is performed, for example, at intervals of several milliseconds, and a phoneme sequence or a word sequence is obtained by pattern matching using a statistical method such as HMM or a composite similarity method.
Using these and history information, we perform syntactic and semantic analysis,
Obtain final speech recognition and understanding results. Further, from the voice signal component of a specific speaker obtained by the sound source estimating unit, the vocabulary and meaning content uttered by the speaker can be extracted. (FIG.) The speaker information management unit 93 includes, for each speaker estimated by the sound source estimation unit 90, a speech section estimated by the speech section estimation unit 91, a word recognition result output by the speech recognition understanding unit 92, and the meaning of the utterance It manages the content and manages environmental information indicating which speaker spoke what and when. These pieces of information are managed in the speaker information management table. In the speaker information management table,
In addition to targeting only the voice spoken by a speaker,
It is also possible to target environmental noise emitted by a certain sound source. If these data are managed together with the time information, it is easy to associate the data with the search target data.

[0320] The conference situation estimating section 94 has a speaker turnaround time difference measuring section, an ambient noise measuring section, a non-speaker generated sound volume measuring section, and a situation estimating section. The speaker replacement time difference measuring unit obtains the elapsed time from the end of one speaker to the start of another speaker by referring to the speaker information management table.
The ambient noise measuring unit obtains the level of the ambient noise from the volume level of the non-voice section. The non-speaker generated sound volume measurement unit obtains the level of the ambient sound from a sound source other than the speaker in the voice section. The situation estimation unit determines the smoothness of the discussion from the elapsed time measured by the speaker turnaround time difference measurement unit, from the ambient noise level measured by the ambient noise measurement unit and the ambient sound level measured by the non-speaker generated volume measurement unit. Estimate the conference status such as liveliness of the conference. These conference statuses are managed together with time information in a conference status information management table.

The input to the environment information extraction unit 43 described above is for the case where the acoustic data captured by the environment information input unit 80 is used, but the audio data stored in the target data storage unit 49 is used. To extract environmental information.

The environment information storage unit 48 stores the environment information extraction unit 4
3 and the contents of the conference status information management table. All these pieces of information have time information added.

The search key input section 52 has input means such as a microphone, a pen, a mouse, and a keyboard.

A search key relating to the contents of the speaker information management table and the conference status information management table stored in the environment information storage unit 48, such as the speaker, vocabulary, meaning content, and atmosphere, is
Input is performed by voice input or pointing operation using these input means.

The search unit 50 searches the contents of the speaker information management table and the contents of the conference status information management table based on the search key received by the search key input unit, and finds matching environmental information and related information. The target data is output as a search result by a visual method such as displaying characters or icons or an auditory method such as voice or sound effect.

The target data output unit 51 acoustically or visually reproduces and outputs the environment information and the contents of the multimedia data stored in the environment information storage unit 48 and the target data storage unit 49, and the search result of the search unit. .

For example, an output example of environment information will be described. Using the speaker and time as axes, it is possible to visually indicate who spoke and when. In addition, if the meaning content and the speaker are used as axes, it is possible to visually show who made what kind of meaning and who made the similar content.

Some examples using sound as environment information will be described.

(Example) Conference file: The target data is voice
Documents such as materials At the time of input The environment information input unit 80 captures the voice of the speaker and the surrounding sounds via a microphone.

The environment information extraction unit 43 performs speech recognition and speech understanding from these speech data when, 1) who utters or 2) what topic, vocabulary, and meaning. Extract. 3) The atmosphere of the meeting based on the elapsed time from when one speaker finishes speaking to when another speaker starts speaking, the ambient noise level when there is no speaking, the voice and sound level of other speakers, etc. The meeting status such as is extracted.

-Who made the statement: It would be easy if multi-channel inputs could be made using a plurality of microphones.

For example, if at least the number of microphones is equal to the number of speakers, it is possible to know who has spoken by associating the microphones with the speakers. Even if the number of microphones is smaller than the number of speakers, if a microphone array is configured and the direction of the sound source is known, the speaker can be generally determined.

-Topic, vocabulary, meaning content: A word is recognized from an utterance by continuous speech recognition, and the meaning content is determined from a combination of the words. Some words to be recognized can be specified in advance from expected topics, and can be preferentially treated when determining a recognition result over other words.

-Start and end of speech: A speech section can be detected from a speech level and duration.

The environment information storage unit 48 records the audio data or the environment information extracted by the environment information extraction unit 43 together with the input time.

The target data output section 51 visually presents environmental information, or reproduces sound, sound reproduction, and documents as target data.

At the time of a search: In the search key input section 52, a pen, a keyboard, a mouse,
A search request input from a user is received from an input device such as a camera. For example, in the environment information output unit 43, FIG.
As shown in, if a list of environmental information is displayed, and the user selects the environmental information to be used as a search key with the input device from the list, the environmental analysis target data that obtained the corresponding environmental information was obtained, and further linked to that. Search target data can be obtained.

Example of search focused on speaker: What is said, vocabulary, and meaning are displayed in order of time. For example, it is assumed that moving image data of a movie or the like is flown as search target data, and each person describes his / her impression in each scene when viewed by a plurality of people. It is assumed that an audio tape recording the utterance of the place is used as the environment analysis target data, and the analysis is performed using the estimation result of the speaker by the speaker recognition and the recognition result of the words included in the utterance as environment information. By displaying the obtained environmental information, that is, the recognition results for each speaker, arranged in chronological order of utterance, it is possible to know the utterance of a specific person and its transition from only the environmental information, or to find a certain word Search target data can also be searched for as the utterance scene.

Example of Search Focusing on Vocabulary: Vocabulary included in repeated remarks is treated as a keyword of the meeting.

An example of a search focused on a topic: The opinion of who has a certain topic is displayed.

Example of Search Focusing on Meeting Atmosphere: If the elapsed time from the end of one speaker to the start of another speaker is short, the discussion is considered to be heated.
Also, from the voices and sound levels of the voices other than the speaker, it can be determined whether the voice was lively or quiet.

(Example) Document file: The target data is other than voice, and the environmental information is ambient noise information (level). At the time of input. The environmental information input unit 80 takes in ambient sounds at the time of use via a plurality of microphones.

The environment information extraction unit 43 extracts the level of the ambient noise from the sound data. Further, the frequency of key input for document creation is extracted. (Alternatively, by extracting the level of the noise close to the spectrum of the key input sound and the level of the noise other than the noise separately, the noise generated for creating a document is excluded from the ambient noise.) The voice data or the environment information extracted by the environment information extraction unit is recorded together with the input time.

In the target data output unit 51, for each document,
Alternatively, the noise information is visually presented in units of paragraphs, sentences, phrases, sections, characters, and the like.

At the time of a search: In the search key input section 52, a pen, a keyboard, a mouse,
A search request input from a user is received from an input device such as a camera.

Example of search focusing on the degree of concentration of document creation:
Based on the level and type of ambient noise, a document created in a quiet environment is judged to have high quality.

Next, the seventh invention will be described. Seventh
The present invention is a search device that stores a number, name, or nickname unique to a computer used (logged in) by a user, and extracts and searches for data such as a desired document using the stored number, name, or nickname.

At present, many computers such as personal computers and workstations can be connected via a network. In such an environment, data existing in a storage device of a certain computer can be referred to from other computers. , Can be changed. In the seventh aspect, even in such an environment, it is possible to extract and search desired data by using which computer has created, referred to, or changed the data.

Embodiment 1 FIG. 120 shows the basic configuration.

This search device includes an input unit 100, a computer I
D setting unit 101, search target data storage unit 102, search target data output unit 103, data and computer ID storage unit 104, environment information output unit 105, search key input unit 10
6, a search unit 107 and a communication unit 108.

FIG. 121 shows the format of data and information stored in the computer ID storage unit 104 and examples thereof. FIG.
2 shows an example of a correspondence table between data names and document names stored in the search target data storage unit 102. FIG.
2 shows a flow of a process in the computer ID setting unit 101 for generating the information of FIG.

When a search key on a window (not shown) of the search key input unit 106 is pressed, a search process is performed by the search unit 107.

FIG. 124 shows the flow of the search process. Processing 1
FIG. 125 shows an example of the screen displayed by 231. In FIG. 125, when the computer name is selected, the processing of the processing 1233 and the subsequent processing in FIG. 124 is performed. FIG. 126 shows an example of a screen displayed by the process 1233.

Embodiment 2 The configuration of the apparatus is the same as that of FIG. In this example,
A computer-specific number or name or nickname and a layout drawing in the room where the computer is placed are stored, and it is possible to search for desired data based on the location of the location as a clue.

FIG. 127 shows information on a computer and its layout. This information is created in advance and stored in the data of each computer and the computer ID storage unit 102 as shown in FIG.
1 together with the information shown in FIG.

The processing flow of the computer ID setting unit 101 for generating the information of FIG. 127 is the same as that of FIG. FIG.
8 shows the flow of the processing of the search unit 107. Processing 12
FIG. 129 shows an example of the screen displayed at 71. In addition,
In the process 1271, all the place names are displayed, but the data of all the computers and the information shown in FIG. 121 in the computer ID storage unit 104 are searched by the user IDs of the user, and the computer IDs are searched. May display only the name of the zone of the room where the calculators are located. FIG. 130 shows an example of the screen displayed by the process 1273. FIG. 126 shows an example of a screen displayed by processing 1279.
Is the same as

Note that the correspondence table in FIG. 127 may be automatically created when a computer is connected by giving a location display such as an AI to an outlet of a network to which the computer is connected. . In this case, the correspondence table can be automatically created and changed even if the computer is moved.

Next, an example in which the environment information output unit 7 shown in FIG. 1 performs a display range determination process will be described. Here, temperature is used as an example of the environmental analysis target data. For the measurement of the temperature, a technique used in a digital thermometer or the like may be used. Also, a document is used as search target data.

FIG. 131 shows the format and an example of information stored. FIG. 132 shows a processing flow of the environment information output unit 7,
FIG. 133 shows an example of the display.

Similarly, when the brightness (illuminance) is used as the data for environmental analysis, it may be displayed as shown in FIG. If the voice is used as the environmental analysis target data and the volume of the voice is used as the search key, it is displayed as shown in FIG. 135, and if the type of voice is used as the search key, FIG.
May be displayed as shown in FIG.

[0361]

According to the present invention, data can be searched even when the retrieval formula, title, or content cannot be remembered, or when the title or content cannot be expressed in a natural language expression.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of a system according to the present invention.

FIG. 2 is a configuration diagram of a search target data input unit according to the present invention.

FIG. 3 is a diagram showing data levels according to the present invention.

FIG. 4 is a configuration diagram of an environment analysis target data input unit according to the present invention.

FIG. 5 is a diagram showing analysis timing according to the present invention.

FIG. 6 is an explanatory diagram relating to acquisition of environment information according to the present invention.

FIG. 7 is a schematic configuration diagram of a system according to the present invention.

FIG. 8 is a schematic configuration diagram of a system according to the present invention.

FIG. 9 is a device configuration diagram according to the first invention.

FIG. 10 is a functional configuration diagram according to the first invention.

FIG. 11 is a diagram showing a format in a document data storage unit and an example thereof according to the first invention;

FIG. 12 is a diagram showing a processing flow of a control unit according to the first invention.

FIG. 13 is a diagram showing a processing flow of a search unit according to the first invention;

FIG. 14 is a diagram showing a format in a position document information storage unit according to the first invention and an example thereof;

FIG. 15 is a diagram showing a processing flow of the control unit when the “place registration” key according to the first invention is pressed.

FIG. 16 is a diagram showing a format in a place name storage unit according to the first invention and an example thereof;

FIG. 17 is a diagram showing a processing flow of a control unit according to the first invention;

FIG. 18 is a diagram showing a processing flow of a search result display unit according to the first invention.

FIG. 19 is a view showing an example of display in a search result display window according to the first invention;

FIG. 20 is a diagram showing a processing flow of a search unit according to the first invention;

FIG. 21 is a view showing an example of a search result display window display according to the first invention;

FIG. 22 is a diagram showing a processing flow of the position measurement unit when the “place registration” key according to the first invention is pressed.

FIG. 23 is a diagram showing a flow of a search process of a position document information storage unit according to the first invention.

FIG. 24 is a diagram showing a processing flow of a search unit in the case of inputting a new document according to the first invention.

FIG. 25 is a diagram showing a processing flow of a control unit in the case of promoting place name registration according to the first invention.

FIG. 26 is a diagram showing a processing flow of a search result display unit in the case of promoting place name registration according to the first invention.

FIG. 27 is a diagram showing a processing flow of a search unit in the case of promoting place name registration according to the first invention.

FIG. 28 is a diagram showing a functional configuration in a second embodiment according to the first invention;

FIG. 29 is a diagram showing a processing flow of a search unit in the second embodiment according to the first invention.

FIG. 30 is a diagram showing a processing flow of a search unit in the case of inputting a new document in the second embodiment according to the first invention.

FIG. 31 is a diagram showing an example of display in the case of promoting place name registration according to the first invention;

FIG. 32 is a diagram showing an example of a display after an operation of promoting place name registration according to the first invention;

FIG. 33 is a diagram showing a functional configuration of a first device in a third embodiment according to the first invention.

FIG. 34 is a diagram showing a functional configuration of a second device in the third embodiment according to the first invention.

FIG. 35 is a diagram showing a processing flow of a search unit of the first device in the third embodiment according to the first invention.

FIG. 36 is a diagram showing a format and an example of data transmitted by a communication unit of the first device in the third embodiment according to the first invention.

FIG. 37 is a diagram showing a format in an operation history storage unit of the first device and an example of the format in the third embodiment according to the first invention;

FIG. 38 is a diagram showing the flow of processing of the search unit of the first device in the case of inputting a new document in the third embodiment according to the first invention.

FIG. 39 is a diagram showing a format in a position operation information storage unit of the second device and an example thereof in the third embodiment according to the first invention;

FIG. 40 is a diagram showing a processing flow of a search unit of the second device in the third embodiment according to the first invention.

FIG. 41 is a diagram showing a format in a place name storage unit of the second device and an example thereof in the third embodiment according to the first invention;

FIG. 42 is a diagram showing an example of a display on a search result display unit of the second device in the third embodiment according to the first invention.

FIG. 43 is a functional configuration diagram of a first device in a fourth embodiment according to the first invention.

FIG. 44 is a functional configuration diagram of the second device in the fourth embodiment according to the first invention.

FIG. 45 is a diagram showing a processing flow of a search unit of the first device in the fourth embodiment according to the first invention.

FIG. 46 is a diagram showing a format in the operation history storage unit of the first device and an example thereof in the fourth embodiment according to the first invention.

FIG. 47 is a diagram showing a processing flow of a search unit of the first device in the case of inputting a new document in the fourth embodiment according to the first invention.

FIG. 48 is a functional configuration diagram of the second device in the fifth embodiment according to the first invention.

FIG. 49 is a diagram showing a processing flow of a search unit in Embodiment 6 according to the first invention.

FIG. 50 is a view showing a processing flow (No. 2) of the search unit in the sixth embodiment according to the first invention.

FIG. 51 is a diagram showing a flow of a display process of a search result display unit in the seventh embodiment according to the first invention.

FIG. 52 is a view showing an example of display of a search result display section window in the seventh embodiment according to the first invention.

FIG. 53 is a view showing an example (part 2) of displaying a window for a search result display unit in embodiment 7 according to the first invention;

FIG. 54 is a view showing an example (part 3) of display of a window for a search result display section in embodiment 7 according to the first invention.

FIG. 55 is a diagram showing a format and an example of data in a map database in a seventh embodiment according to the first invention.

FIG. 56 is a functional configuration diagram in an eighth embodiment according to the first invention.

FIG. 57 is a diagram showing an example of a document in an eighth embodiment according to the first invention.

FIG. 58 is a diagram showing a format and an example of a document data storage unit in an eighth embodiment according to the first invention.

FIG. 59 is a diagram showing a format of a location database and an example thereof in an eighth embodiment according to the first invention.

FIG. 60 is a view showing an example (part 2) of a document in the eighth embodiment according to the first invention.

FIG. 61 is a diagram showing a format of a document data storage unit and an example thereof (part 2) in an eighth embodiment according to the first invention;

FIG. 62 is a diagram showing a format of a position database and an example thereof (part 2) in the eighth embodiment according to the first invention;

FIG. 63 is a view showing an example of display in the eighth embodiment according to the first invention.

FIG. 64 is a view showing an example (part 2) of a display in the embodiment 8 according to the first invention.

FIG. 65 is a basic diagram according to the second invention;

FIG. 66 is a diagram showing an example of a schedule table according to the second invention;

FIG. 67 is a diagram showing an example of a schedule table according to the second invention;

FIG. 68 is a diagram showing an example of a schedule table according to the second invention;

FIG. 69 is a basic diagram according to the third invention;

FIG. 70 is a view showing a document example according to the third invention;

FIG. 71 is a view showing an example of a layout according to the third invention;

FIG. 72 is a diagram showing a column template and constraints according to the third invention;

FIG. 73 is a diagram showing a chart arrangement type according to the third invention;

FIG. 74 is a diagram showing a storage example of a layout information storage unit according to the third invention;

FIG. 75 is a diagram showing a list of columns according to the third invention;

FIG. 76 is a diagram showing types of charts according to the third invention.

FIG. 77 is a view showing a search result according to the third invention;

FIG. 78 is a view showing a search result according to the third invention.

FIG. 79 is a diagram showing an environment information extraction unit according to the fourth invention.

FIG. 80 is a view showing a processing flow according to the fourth invention;

FIG. 81 is a diagram showing a scene change determination unit according to the fourth invention;

FIG. 82 is a view showing a processing flow according to the fourth invention;

FIG. 83 is a view showing an example of scene change detection according to the fourth invention;

FIG. 84 is a view showing an example of scene change detection according to the fourth invention;

FIG. 85 is a view showing an example of scene change detection according to the fourth invention;

FIG. 86 is a view showing an example of scene change detection according to the fourth invention;

FIG. 87 shows another embodiment according to the fourth invention.

FIG. 88 is a view showing an example of scene change detection according to the fourth invention;

FIG. 89 is a view showing an example of scene change detection according to the fourth invention;

FIG. 90 is a view showing an example of scene change detection according to the fourth invention;

FIG. 91 is a view showing an example of scene change detection according to the fourth invention;

FIG. 92 is a view showing an example of scene change detection according to the fourth invention;

FIG. 93 is a view showing an example of scene change detection according to the fourth invention;

FIG. 94 is a view showing an example of scene change detection according to the fourth invention;

FIG. 95 is a basic view according to the fifth and sixth inventions;

FIG. 96 shows a search key analysis unit.

FIG. 97 is a view showing search key analysis processing.

FIG. 98 is a view showing a search word dictionary.

FIG. 99 is a diagram showing search term extraction rule words.

FIG. 100 shows an example of access information.

FIG. 101 is a view showing a processing flow of an access information detecting unit in FIG. 95;

102 is a diagram showing the flow of the process of the access information detection unit in FIG. 95.

FIG. 103 is a view showing a processing flow of an access information detecting unit in FIG. 95;

104 is a diagram showing the flow of the process of the access information detection unit in FIG. 95.

FIG. 105 is a diagram showing the flow of the process of the access information detection unit in FIG. 95

106 is a diagram showing the flow of the processing of the search unit in FIG. 95.

FIG. 107 is a view showing a processing flow of a search unit in FIG. 95;

108 is a view showing a processing flow of a search unit in FIG. 95.

109 is a diagram showing the flow of the processing of the search unit in FIG. 95.

110 is a view showing a processing flow of a search unit in FIG. 95.

FIG. 111 is a diagram showing a display example of an access information element.

FIG. 112 is a diagram showing an example of access information display.

FIG. 113 is a diagram showing a display example of access information.

FIG. 114 is a diagram showing an example of access information display combined with schedule data;

FIG. 115 is a diagram showing the flow of the processing of the search unit in FIG. 95.

FIG. 116 shows an example of access information.

FIG. 117 is a diagram showing an example of a document display

FIG. 118 is a diagram showing an example of an environment information extraction unit.

FIG. 119 is a diagram showing a display example of an environment information list.

FIG. 120 is a basic configuration diagram according to a seventh invention.

FIG. 121 is a diagram showing a format of data and information stored in a computer ID storage unit and an example thereof.

FIG. 122 is a diagram illustrating an example of a correspondence table between data names and document names stored in a search target data storage unit;

FIG. 123 is a diagram showing a processing flow in a computer ID setting unit.

FIG. 124 is a diagram showing a flow of a search process.

FIG. 125 shows an example of a screen displayed by the same processing.

FIG. 126 is a view showing an example of a screen displayed by the same processing.

FIG. 127 is a diagram showing an example of information regarding computers and their arrangement.

FIG. 128 is a diagram showing a flow of processing of a search unit.

FIG. 129 is a view showing an example of a screen displayed by the same processing.

FIG. 130 shows an example of a screen displayed by the same processing.

FIG. 131 is a diagram showing a format and an example of storage information in the device shown in FIG. 1

FIG. 132 is a view showing a processing flow of an environment information output unit of the apparatus shown in FIG. 1;

FIG. 133 is a diagram showing an example of a display in which the processing is solved.

FIG. 134 is a view showing a display example when brightness is set as environmental analysis target data.

FIG. 135 is a diagram showing a display example when the loudness of a sound is set as data to be analyzed for the environment.

Fig. 136 is a diagram showing a display example when the type of audio is set as the data to be analyzed for the environment.

FIG. 137 is a diagram showing a search application example of the device shown in FIG. 65;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Search target data input part 2 ... Search target data storage part 3 ... Search target data output part 4 ... Environment analysis target data input part 5 ... Environment information analysis part 6 ... Environment information storage part 7 ... Environment information output part 8 ... Search Key input section 9 ... Search section

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yasushi Kawakura 1 Kosuka Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture Inside the Toshiba R & D Center (72) Inventor Mika Fukui Toshiba Komukai-shi, Kawasaki-shi, Kanagawa No. 1 in the Toshiba R & D Center, Inc. (72) Inventor Hisako Tanaka No. 1, Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa Incorporated In Toshiba R & D Center, Inc. 1 Toshiba R & D Center, Komukai Toshiba-ku, Ward (72) Inventor Hiroshi Mizoguchi 1 Toshiba R & D Center, Komukai Toshiba-cho, Kawasaki-shi, Kanagawa Pref. (72) Inventor Satoshi Ito Kanagawa No. 70 Yanagicho, Sachi-ku, Kawasaki-shi, Japan Inside the Toshiba Yanagicho factory (72) Inventor Shigenori Seto 1 Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa Inside the Toshiba R & D Center (72) Inventor Yoshikuni Matsumura 1 Kodamu Toshiba-cho, Koyuki-ku, Kawasaki-shi, Kanagawa F-term in the Toshiba R & D Center 5B050 BA10 BA16 EA18 GA08 5B075 ND16 ND23 PP01 PP13 PP28 PQ02 PR03 QM08 UU40

Claims (14)

[Claims] 1. A first storage means for storing data such as a document, a sound, an image, etc. as an operation target such as an input, an edit, and a search as operation target data, and a self apparatus for performing an operation on the operation target data Means for acquiring time-series image data obtained by photographing the periphery of the apparatus and its own device; and a characteristic scene for enabling an environment at the time of operation on the operation target data to be specified among the image data acquired by the acquisition means. Extracting means for extracting the image data of the operation target as environment information, and second storage means for storing the operation target data in association with the image data extracted as the environment information corresponding to the time of the operation on the operation target data Selecting means for selecting the environment information at the time of searching; and selecting from the operation target data stored in the first storage means by the selecting means Search means for searching an operation object data associated with the boundary information, the search apparatus characterized by comprising output means for outputting the operation target data retrieved by the search means. 2. The retrieval apparatus according to claim 1, wherein said extracting means extracts image data having a scene change from the acquired time-series image data as the environment information. 3. The retrieval apparatus according to claim 1, wherein said extracting means extracts, as the environment information, image data in which a person is detected from the acquired time-series image data. 4. The apparatus according to claim 1, wherein said extracting means extracts, from the acquired time-series image data, detected image data of a specific person designated in advance as the environment information. Search device. 5. Acquiring, as the environment information, position information of a location where the own device that performs an operation on the operation target data exists, and storing the position information together with the image data extracted as the environment information in association with the operation target data. The search device according to claim 1, wherein the search is performed. 6. A method according to claim 1, wherein information relating to a computer device performing an operation on said operation target data is obtained as said environment information, and stored together with said image data extracted as said environment information in association with said operation target data. The search device according to claim 1, wherein 7. The retrieval apparatus according to claim 6, wherein the information on the computer device is identification information of the computer device. 8. The search apparatus according to claim 6, wherein the information on the computer device is a location of the computer device. 9. A schedule management means for storing and managing a schedule registered by a user; and a user's schedule at the time of operation on the operation target data is extracted from the schedule management means, and is extracted as the operation target data. A third storage unit configured to associate and store the schedule with the environment information; and the search unit selects one of the operation target data stored in the first storage unit. 2. The retrieval apparatus according to claim 1, wherein the retrieval unit retrieves the operation target data associated with the environment information and the schedule selected by the selection unit. 10. A means for extracting information indicating a structural characteristic of the operation target data, and a fourth storage means for storing the extracted information indicating the structural characteristic in association with the operation target data. The apparatus further comprises: the selection unit selects information indicating the structural feature together with the environment information; and the search unit selects from the operation target data stored in the first storage unit by the selection unit. 2. The search device according to claim 1, wherein the search unit searches for the operation target data associated with the environment information and the information indicating the structural feature. 11. The information indicating the structural feature includes the number of columns,
11. The retrieval apparatus according to claim 10, wherein the information is information indicating the size and position of the document frame, the number of lines, the number of figures and tables, the size and position of figures and tables, and the type of figures and tables. 12. An operation start time and an operation end time at the time of operation on the operation target data are set as the environment information.
2. The search device according to claim 1, wherein the search device is stored in association with the operation target data together with the image data extracted as the environment information. 13. An urgency of the operation target data is obtained from an operation start time and an operation end time at the time of operation on the operation target data, and this urgency is used as the environment information.
2. The search device according to claim 1, wherein the search device is stored in association with the operation target data together with the image data extracted as the environment information. 14. The retrieval apparatus according to claim 1, wherein an operation frequency of the operation target data is stored as the environment information in association with the operation target data together with the image data extracted as the environment information. .