JP3340208B2 - Multimedia information collection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information collecting apparatus for inputting multimedia information to a computer. The present invention relates to a multimedia information collection device that can present estimated link information to a user.

[0002]

2. Description of the Related Art In recent years, multimedia information devices have become widespread. Collected information has been digitized, stored in an information device such as a computer, and the stored information has been increasingly used later as a multimedia document. Is coming.

FIG. 17 is an explanatory diagram of a conventional multimedia information collecting apparatus. In FIG. 17, 3 is a memory circuit,
4 is a data storage circuit, 21 is a CPU, 22 is a system bus, 101 is an imaging device such as a CCD camera, 102 is an image signal processing circuit, 111 is a microphone, 113 is a speaker, 112 is an audio signal processing circuit, and 122 is handwriting input. Display integrated type tablet in which a digitizer 121 for use and a flat display circuit 5a such as an LCD (liquid crystal display) are integrated,
Reference numeral 123 denotes a stylus pen for performing handwriting input. Then, the CPU 21 uses the system bus 22 to store the memory circuit 3, the data storage circuit 4, the image signal processing circuit 102,
Audio signal processing circuit 112, input display integrated tablet 1
22 are connected to the handwriting input digitizer 121 and the flat panel display circuit 5a.

Conventionally, when collecting multimedia information such as photographing, recording, and writing (handwriting), the CPU 21 first takes a photograph on the input / display integrated tablet 122 according to a program stored in the memory circuit 3 in advance. , Recording, writing (handwriting) and other operation menus. Next, the user operates the stylus pen 123 on the input / display integrated tablet 122 to select an operation menu.

[0005] When the user selects the writing menu, handwriting input is performed on the input / display integrated tablet 122 using the stylus pen 123. Accordingly, the CPU 21 reads the input coordinates from the handwriting input digitizer 121, displays the handwriting at the corresponding position on the flat display circuit 5a, and stores the data input by handwriting in the data storage circuit 4.

When the user selects the photographing menu, the image photographed by the imaging device 101 is converted into a digital signal in the image signal processing circuit 102, and the image signal is processed by the CPU.
21 reads the digital signal from the image signal processing circuit 102 and stores it in the data storage circuit 4.

When the user selects the recording menu,
The sound input from the microphone 111 is converted into a digital signal by the sound signal processing circuit 112 and
1 reads the digital signal from the audio signal processing circuit 112 and stores it in the data storage device 4.

In the above conventional multimedia information collecting apparatus, the user provides relevance between a plurality of pieces of relevant information such as handwriting input, photographing, sound recording, etc., in order to make the collected information easy to use. Linking was performed when multiple pieces of information were imported. For example, when linking a handwritten input and a photo image, a stylus pen is used to attach a photo image to a part of the handwritten input screen or to provide an icon indicating that there is a photo image related to the handwritten input screen. 123 was going. In this way, when information is taken in, a link is provided between a plurality of pieces of information, and the taken-in information is structured. Then, the link information designated by the user, that is, the structured information, is stored in the data storage circuit 4 together with the information of each media taken in.

[0009]

The above-mentioned conventional apparatus has the following problems. It is very convenient to create a link between the collected and related information at the stage of using the collected information.However, it is necessary to perform the operation of creating the link at the time of collecting the information. The burden on the user was increasing. In particular, when collecting temporary information, an operation for establishing a link sometimes misses the timing of information collection.

[0010] In addition, when an operation of reading out each piece of information and establishing a link after collecting a plurality of pieces of information is performed, the collected information may be forgotten, and the link may not be created properly.

The present invention solves such a conventional problem, in which the CPU estimates a relationship between a plurality of pieces of information fetched from a user operation (operation) at the time of information collection, and a plurality of pieces of information automatically fetched. An object of the present invention is to reduce a user burden when collecting information by providing a link between them.

[0012]

To achieve the above object, the present invention has the following constitution. FIG. 1 is a diagram for explaining the principle of the present invention. In FIG. 1, a multimedia information collecting apparatus includes a memory circuit 3, a data storage circuit 4, and a display circuit 5.
, An association determination rule storage circuit 6, an information collection operation status holding circuit 7, an estimated link information storage circuit 8, a medium A information input circuit 11 and a medium B information input circuit as information input means for a plurality of different media. 12, the medium C information input circuit 13, and the CPU 21 are connected by a system bus 22.

The relevancy determination rule storage circuit 6 stores in advance rules for determining relevance between media. For example, the media C information input circuit 13
When the data input operation of the media A information input circuit 11 or the media B information input circuit 12 is performed during the data input operation of the above, the data of the media C information input circuit 13 and the media A information input circuit 11 or the media B information input The rule that the data is related to the circuit 12 is stored.

The information collection operation status holding circuit 7 holds a user operation status at the time of information collection performed by the user. The estimated link information storage circuit 8 is configured to, when it is estimated from the user operation situation at the time of collecting information that there is a relationship between the data of the plurality of media information input circuits, store the relationship as estimated link information. .

[0015]

The information is collected by a medium A information input circuit 11 and a medium B which are means for collecting data of a plurality of media.
Data from each medium digitized by the information input circuit 12 and the medium C information input circuit 13 is stored in the data storage circuit 4. At this time, the CPU 21 refers to the user operation status at the time of information collection held in the information collection operation status holding circuit 7 according to the relevancy determination rule stored in the relevancy determination rule storage circuit 6 in advance.
As a result, the CPU 21 estimates the relevance between the collected data of the plurality of media, and stores the estimated link information in the estimated link information storage circuit 8 when the relevance is presumed.

Later, when referring to the information collected by the user, the CPU 21 reads out the information specified by the user from the data storage circuit 4 and displays it on the display circuit 5 and also displays the estimated link information in the estimated link information storage circuit 8. Upon checking, if there is information related to the information specified by the user, the fact is displayed to the user.

For this reason, the user can efficiently refer to the information later by using the estimated link information without performing the operation of linking the information of the plurality of collected media at the time of collecting the information.

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIGS. 2 is a block diagram showing one embodiment of the present invention, FIG. 3 is a block diagram showing a more specific embodiment of FIG. 2, FIGS. 4 to 9 are diagrams for explaining the operation of one embodiment of the present invention, and FIGS. 2 shows a processing flowchart of one embodiment.

In FIGS. 2 and 3, reference numeral 3 denotes a memory circuit;
Is a data storage circuit, 4a is a hard disk, 5a is a flat display circuit, 6 is a relevancy determination rule storage circuit, 7 is an information collection operation status holding circuit, 8 is an estimated link information storage circuit,
21 is a CPU, 22 is a system bus, 101 is an imaging device, 101a is a CCD camera, 102 is an image signal processing circuit, 102a is an image A / D conversion circuit, 111 is a microphone, 112 is an audio signal processing circuit, and 112a is audio A
/ D conversion circuit, 112b is an audio D / A conversion circuit, 121
Is a digitizer for handwriting input which is a coordinate input device, 122
Denotes an input display integrated tablet, and 123 denotes a stylus pen. 2 and 3 are denoted by the same reference numerals.

The multimedia information collecting apparatus shown in FIG. 2 includes an image signal processing circuit 102 for performing signal processing from the image pickup apparatus 101, a microphone 111 and a speaker 113.
The signal processing circuit 112 connected to the input terminal, the input display integrated tablet 122 integrating the handwriting input digitizer 121 and the flat display circuit 5a, the CPU 21, the memory circuit 3, the data storage circuit 4, The sex determination rule storage circuit 6, the information collection operation status holding circuit 7, and the estimated link information storage circuit 8 are connected by a system bus 22.

The relevancy determination rule storage circuit 6 stores a relevancy determination rule in advance for estimating the relevance between multimedia data collected by the imaging device 101, the microphone 111, the handwriting input digitizer 121, and the like. It is something to keep. For example, a determination rule is stored that image data or voice data recorded while the handwriting input sheet is open is related to the handwriting input data on the handwriting input sheet.

The information collection operation status holding circuit 7 is a circuit for holding a user operation status at the time of information collection performed by the user. The estimated link information storage circuit 8 is a circuit that, when it is estimated that there is a relationship between the collected multimedia information, stores the relationship as the estimated link information.

Next, the operation of collecting multimedia information shown in FIG. 2 will be described. An image captured by the imaging device 101 is digitized by the image signal processing circuit 102 and the data is stored in the data storage circuit 4.
And digitizes the voice input by the microphone 111 by the voice signal processing circuit 112 and stores the data in the data storage circuit 4.
The data of the digitizer 121 for handwriting input by the handwriting input stylus pen 123 is stored in the data storage circuit 4. At the time of collecting such information, the CPU 21 uses the relevancy rules stored in the relevancy determination rule storage circuit 6 in advance to store the data collected from the user operation status at the time of information collection held in the information collection operation status holding circuit 7. Is estimated, and the result is stored in the estimated link information storage circuit 8.

Later, when referring to the information collected by the user, the CPU 21 reads out the information specified by the user from the data storage circuit 4 and displays it on the flat display circuit 5a of the input / display integrated tablet 122. At this time, if the information is audio information, the audio converted into an analog signal by the audio signal processing circuit 112 is reproduced by the speaker 113. Also, CPU
21 checks the estimated link information in the estimated link information storage circuit 8, and if there is estimated link information relating to the information referred to by the user, this is displayed on a part of the screen of the flat display circuit 5a, for example. Present to the user. The user can use the stylus pen 12 based on the displayed estimated link information.
After the link between the information is determined by an operation such as 3, the determined link information is stored in the data storage circuit 4.

Next, a specific configuration diagram of one embodiment shown in FIG. 3 will be described. In FIG. 3, the CCD camera 101
a, which outputs an NTSC video signal, corresponds to the imaging apparatus 101 of FIG.
2a is for converting an analog video signal to a digital signal and corresponds to the image signal processing circuit 102 in FIG. 2, and the hard disk 4a is for the data storage circuit 4 in FIG. Circuit 112a
The audio D / A conversion circuit 112b corresponds to the audio signal processing circuit 112 in FIG.

The input / display integrated tablet 122 is a unit in which a handwriting input digitizer 121 as a coordinate input device and the flat display circuit 5a are integrated. The input coordinates are read from the digitizer 121 and handwriting is displayed at a corresponding position on the flat display circuit 5a.

The relevancy determination rule storage circuit 6 states that "the image data and the voice data recorded while the handwriting input sheet is open are related to the handwriting input data on the handwriting input sheet." The rule is stored in advance.

The information collection operation status holding circuit 7 holds a user operation status at the time of information collection performed by the user. The estimated link information storage circuit 8 stores the relevance as estimated link information when it is estimated that there is a relationship between the collected image data, audio data, and handwritten input data.

The CPU 21 sends the image A
/ D conversion circuit 102a, audio A / D conversion circuit 112a,
An audio D / A conversion circuit 112b, an input / display integrated tablet 122, a memory circuit 3, a hard disk 4a, an association determination rule storage circuit 6, an information collection operation status holding circuit 7,
It is connected to the estimated link information storage circuit 8.

Hereinafter, the operation of the multimedia information collecting apparatus of FIG. 3 will be described with reference to the flow of the service screens of FIGS. In the figure, A to V are input / display integrated tablet 122.
1 shows a display screen and the display contents thereof.

In FIG. 4A, first, the CPU 21 displays a screen A of the flat display circuit 5a of the input / display integrated tablet 122 according to a program stored in the memory circuit 3 in advance.
The operation menu of “photograph” B, “recording” C, and “handwriting” D is displayed above. FIG. 4 shows a case where the user touches “handwriting” D with the stylus pen 123 and selects the handwritten menu.
The display changes to that shown in FIG.

In FIG. 4B, the CPU 21 displays the handwriting input sheet F and registers the user operation status during the handwriting input in the information collecting operation status holding circuit 7. Incidentally, the handwriting input sheet F can be transferred to the page or the next page before touch the arrow E ₁ or E ₂ with the stylus pen 123. In this screen, the menu of "handwriting" D is deleted. However, for example, black and white reverse display can be performed.

In FIG. 4C, the user operates the stylus pen 1
23, the CPU 21 reads the coordinates from the handwriting input digitizer 121 in the input / display integrated tablet 122, displays the handwriting at the corresponding position on the flat display circuit 5a, and outputs the handwritten data. It is stored in the hard disk 4a. At this time, the CPU 21
Reads the user operation status from the information collection operation status holding circuit 7 and estimates the relevance according to the rules in the relevancy determination rule storage circuit 6. In this case, there is no relationship between the handwritten input data and the photographic image or audio data, and nothing is stored in the estimated link information storage circuit 8.

In FIG. 5D, when the user touches "Photo" B with the stylus pen 123 during handwriting input and selects a photo menu, the CPU 21 displays the screen shown in FIG. 5E. In FIG. 5E, a photographing window G is displayed on the screen A of the flat display circuit 5a. At this time, the CCD camera 10
1a is converted to an image A / D conversion circuit 102
a, the image is converted into digital data, and the CPU 21
Data is read out from the D conversion circuit 102a and displayed on the photographing window G of the flat panel display circuit 5a,
The user operation status during photographing is registered in the information collection operation status holding circuit 7. Here, when the user touches "Shooting" H with the stylus pen 123 and presses the shooting button, the CPU 21 stores the image data read from the image A / D conversion circuit 102a on the hard disk 4a. The CPU 21 reads the user operation status from the information collection operation status holding circuit 7 and estimates the relevance according to the rules in the relevancy determination rule storage circuit 6. In this case, since the image data and the voice data recorded while the handwriting input sheet is open are related to the handwriting input data on the handwriting input sheet, the CPU 21 The estimated link information storage circuit 8 stores that there is a relationship between the handwritten data on the handwritten input sheet F displayed on the input / display integrated tablet 122 and the captured image data. Then, the CPU 21 registers that the photographing is completed in the information collecting operation status holding circuit 7, and closes the photographing window G. Note that, in the case of FIG. 5E, the menu display of the “photograph” B is displayed in reverse black and white.
The menu display can be erased like the menu of "handwriting" D in (b).

As shown in FIG. 6 (f), when the recording menu is selected by touching "recording" C with the stylus pen 123 while the handwritten sheet F is being displayed, the CPU 21 displays the display of FIG. 6 (g).

In FIG. 6 (g), the CPU 21 displays the recording window I on the screen A of the flat display circuit 5a and registers the user operation status during recording in the information collection operation status holding circuit 7. Here, when the user touches “Start” J with the stylus pen 123 and presses the start button, the audio signal input to the microphone 111 is converted into digital data by the audio A / D conversion circuit 112a, and the CPU 21 Data is read from the D conversion circuit 112a and stored in the hard disk 4a.

When the user touches "start" J again with the stylus pen 123 and presses the stop button, the CP
U21 stops reading data from the audio A / D conversion circuit 112a, and the information collection operation status holding circuit 7
, The user operation status is read out, and the relevancy is estimated according to the rules in the relevancy determination rule storage circuit 6. In this case, "the image data and the voice data recorded while the handwriting input sheet is open are related to the handwriting input data on the handwriting input sheet."
The CPU 21 stores in the estimated link information storage circuit 8 that there is a relationship between the handwritten data on the handwritten input sheet F displayed on the display-integrated tablet 122 and the recorded voice data.

Thereafter, the CPU 21 registers the end of the recording in the information collecting operation status holding circuit 7 and closes the recording window I. Next, a case in which the user refers to the collected information will be described.

As shown in FIG. 7H, the handwriting input sheet F collected by the user by operating the stylus pen 123 later.
Is designated, the CPU 21 causes the handwriting input sheet F
Is read from the hard disk 4a and displayed,
The link information in the estimated link information storage circuit 8 is checked. The user searches the handwriting input sheet F by touching the arrow E ₁ or E ₂ with the stylus.

If there is estimated link information in the handwritten input sheet designated by the user, the display shown in FIG. In FIG. 7 (i), it is displayed to the user that there is image information of a photographed picture which is estimated link information K related to the displayed handwritten input sheet F and recorded voice information.

In FIG. 7 (j), the user operates the stylus pen 12
When the user touches the "photo" L in 3 and selects the image of the estimated link information K, the CPU 21 displays the image shown in FIG. FIG.
In (k), the photographed image is displayed on the image data window together with the date and time. When the user touches "Yes" O for link execution with the stylus pen 123, the CPU 21 displays the display of FIG. When there are a plurality of photographed images, the up and down arrows Q ₁ and Q ₂ on the image data window can be touched with the stylus pen 123 to search for a predetermined image.

In FIG. 8 (l), the CPU 21 attaches a "photo" R, which is an icon of image information, to the page of the handwritten sheet F being displayed on the flat display circuit 5a, and stores the determined link information in the hard disk 4a. I do. And CPU
21 temporarily displays the screen of FIG.
The screen of (m) is displayed.

FIG. 9 (m) indicates to the user that there is recorded voice information which is estimated link information K related to the displayed handwritten input sheet F. When the user touches “voice” M with the stylus pen 123 and selects the voice of the estimated link information K, the CPU 21 displays the voice data of FIG. 9 (n).

In FIG. 9 (n), the date and time of recording as the audio data specified by the estimated link information K are displayed.
When the user touches “play” S on the audio data window with the stylus pen 123 and presses the audio data reproduction button, the CPU 21 sends the audio data to the audio D / A conversion circuit 112 b and reproduces the audio with the speaker 113. I do. When the user touches “Name” T on the audio data window with the stylus pen 123 and presses the name button, the user can give a name to the audio data by handwritten character recognition. Further, when the user selects “Yes” O for link execution with the stylus pen 123, the CPU 21 displays a display of FIG. 9 (o). Note that when there are a plurality of recorded audio data, the up and down arrows Q ₁ and Q ₂ on the audio data window can be touched with the stylus pen 123 to search for predetermined audio data.

In FIG. 9 (o), the CPU 21 attaches "voice" V, which is a voice information icon, to the page of the handwritten sheet F being displayed on the flat display circuit 5a, and stores the determined link information in the hard disk 4a. I do.

In FIG. 7 (j) and FIG. 9 (m), the CPU
If the user selects “ignore” N with the stylus pen 123 after presenting to the user that there is estimated link information K related to the information specified by the user, the CPU 21
Deletes the estimated link information K presented to the user from the estimated link information storage circuit 8.

The display positions of the image and sound icons “photo” R and “voice” V can be changed by operating the stylus pen 123 by the user. Further, when the user refers to the determined link information, the user touches the “photo” R or “voice” V icon with the stylus pen 123.

Next, the operation of the embodiment of the present invention will be described with reference to flowcharts shown in FIGS. In the figure, S1
S49 indicates a processing step. FIG. 10 shows FIG.
(D) corresponds to the process when selecting the photo menu.

This process starts when the user touches "photo" B with the stylus pen 123 and selects a photo menu during handwriting input. First, the CPU 21 records a user operation status indicating that a photograph is being taken in the information collection operation status holding circuit 7 (S1).

Then, the CPU 21 displays the photographing window G (S2). Next, the CPU 21 outputs the image A /
The data is read out from the D conversion circuit 102a and the display of the monitor screen is started on the photographing window G (S
3) This process ends.

FIG. 11 corresponds to the processing when the photographing button in FIG. 5E is pressed. This processing starts when the user touches the "photographing" H on the photographing window G with the stylus pen 123 and presses the photographing button.

First, the CPU 21 proceeds to the step (S3).
Display of the monitor screen is stopped (S4). Then C
The PU stores the photograph data of the monitor screen whose display has been stopped on the hard disk 4a (S5).

Next, the CPU 21 reads from the information collecting operation status holding circuit 7 the user operation status, which is the information collecting status of the CCD camera 101a, the microphone 111, and the handwriting input digitizer 121 as each medium (S6).

Next, the CPU 21 reads out a judgment rule from the association rule judgment circuit 6 (S7). Next, CPU
Reference numeral 21 determines the relevancy based on the read determination rule of the relevance rule determination circuit 6 and the user operation status of the information collection operation status holding circuit 7, and stores it in the memory circuit 3 (S8).

Then, the CPU 21 checks the memory circuit 3 and, if the relevance is stored (S9), stores the relevance information in the estimated link information storage circuit 8 as estimated link information (S10). If no relevance is stored in No. 3, nothing is stored in the estimated link information storage circuit 8.

Next, the CPU 21 records that the photographing has been completed in the information collecting operation status holding circuit 7 (S1).
1). Thereafter, the CPU 21 deletes the photographing window G (S12) and ends the processing when the photographing button is pressed.

FIG. 12 shows a process for determining the relevancy of a photographed photograph, which is an example of the relevancy determination process in the step (S8). First, when the photographing button is pressed to start the relevancy determination process, the CPU 21 reads the user operation status from the information collection operation status holding circuit 7 and determines whether the entry surface of the handwriting input sheet F is open. (S13).

When the entry surface of the handwriting input sheet F is open, it corresponds to the determination rule from the relevance rule determination circuit 6, so that the currently open handwriting input sheet F
The handwritten memo on the surface is related to the photographed image.

The CPU 21 stores this relevance information in the memory circuit 3 (S14). When the entry surface of the handwriting input sheet F is not opened, nothing is stored in the memory circuit 3 because there is no relation.

Thereafter, the CPU 21 performs the next relevancy determination process, or terminates the process if there is no next relevancy determination process (S15). FIG. 13 corresponds to the process when selecting the recording menu in FIG.

This process starts when the user touches "recording" C with the stylus pen 123 and selects a recording menu during handwriting input. First, the CPU 21 records the user operation status indicating that recording is in progress in the information collection operation status holding circuit 7 (S16).

Then, the CPU 21 sets the recording window I
Is displayed (S17). Next, the CPU 21 starts displaying the recording level meter on the recording window I (S1).
8) Then, this processing is completed.

FIG. 14 corresponds to the processing when the recording button in FIG. 6G is pressed. This process starts when the user touches “Start” J on the recording window I with the stylus pen 123 and presses the record button.

The audio signal input by the microphone 111 is converted into digital data by an audio A / D conversion circuit 112a, and the CPU 21 transmits the digital signal to the audio A / D conversion circuit 112a.
Digital audio data from the hard disk 4
a (S19).

The user “starts” with the stylus pen 123.
CPU2 until you touch J again and press the recording stop button
1 stores the audio data from the audio A / D conversion circuit 112a on the hard disk 4a, and presses a recording stop button, and the CPU 21 stops storing the audio data from the audio A / D conversion circuit 112a on the hard disk 4a. (S20).

Next, the CPU 21 reads from the information collection operation status holding circuit 7 the user operation status, which is the information collection status of the CCD camera 101a, microphone 111, and handwriting input digitizer 121 as each medium (S21).

Next, the CPU 21 reads out a judgment rule from the association rule judgment circuit 6 (S22). Next, CP
U21 determines the relevancy from the read determination rule of the relevancy determination circuit 6 and the user operation status of the information collection operation status holding circuit 7, and stores it in the memory circuit 3 (S23).

Next, the CPU 21 checks the memory circuit 3 and, if the relevance is stored (S24), stores the relevance information in the estimated link information storage circuit 8 as estimated link information (S25). If no relevance is stored in No. 3, nothing is stored in the estimated link information storage circuit 8.

Next, the CPU 21 records the end of the recording in the information collecting operation status holding circuit 7 (S26). After that, the CPU 21 deletes the recording window I (S
27) The process when the record button is pressed ends.

FIG. 15 shows the relevancy judgment process of the recorded voice, which is an example of the relevancy judgment process in the step (S23). First, when the recording process is started by pressing the recording stop button, the CPU 21 reads the user operation status from the information collection operation status holding circuit 7 and determines whether the entry surface of the handwriting input sheet F is open. (S28).

When the entry surface of the handwriting input sheet F is open, it corresponds to the determination rule from the relevance rule determination circuit 6, and therefore the currently open handwriting input sheet F
The handwritten memo on the surface and the recorded voice are related. The CPU 21 stores this relevance information in the memory circuit 3 (S29). When the entry surface of the handwriting input sheet F is not opened, nothing is stored in the memory circuit 3 because there is no relation.

Thereafter, the CPU 21 performs the next relevancy determination process, or terminates the process if there is no next relevancy determination process (S30). Next, a flowchart when referring to information collected by the user will be described with reference to FIG.

FIG. 16 corresponds to the display processing of the handwritten page when referring to the collected information of FIGS. 7 (i) to 9 (o). The user operates the stylus pen 123 to select, for example, handwriting from the operation menu screen of FIG. 4A and display the handwriting input sheet F of FIG.
Next, the user moves the arrow E ₁ or E ₂ on the handwriting input sheet F.
This process is started by designating the display of the handwritten input sheet F collected before.

First, the CPU 21 reads out the handwritten page data designated by the user from the hard disk 4a (S31) and displays the read out handwritten page data on the handwritten input sheet F (S32).

Next, the CPU 21 reads the estimated link information from the estimated link information storage circuit 8 (S33). CPU
21 judges whether or not the read estimated link information includes data related to the handwritten page data (S34).

If there is data related to the handwritten page data, the CPU 21 displays a message as shown in FIG. 7 (i) (S35). Next, as shown in FIG. 7 (j), when the user touches the "photo" L with the stylus pen 123 and selects the link processing of the photo (S36), the CPU 21 reads the photo data as the estimated link information from the hard disk 4a. It is displayed in the photo link processing window as shown in FIG. 8 (k) (S39).

Next, the user instructs whether or not to execute a link on the photo link processing window (S4).
0). When the user instructs to execute the link, the CPU 21 stores the estimated link information of the handwritten page data and the photograph data on the hard disk 4a as finalized link information (S41).

Next, the CPU 21 displays a photo icon on the handwritten page of the handwritten sheet F as shown in FIG. 8 (l) (S42). Next, when the photograph icon is displayed in the step (S42) or when the link is not executed in the step (S40), the CPU 21 stores the estimated link information of the handwritten page data and the photograph data from the estimated link information storage circuit 8. It is deleted (S43).

Further, the CPU 21 checks whether or not there is still information related to the handwritten page in the estimated link information storage circuit 8 (S49). If the related information still exists, the process returns to step (S35), and a message as shown in FIG. 8M is displayed (S35).

Next, as shown in FIG. 9 (m), the user touches the "voice" M with the stylus pen 123 to select a voice link process (S37), and the CPU 21 sends the voice as the estimated link information from the hard disk 4a. The data is read out and displayed in the audio link processing window as shown in FIG. 9 (n) (S44).

Next, the user instructs whether or not to execute a link on the audio link processing window (S4).
5). Here, when the user instructs to execute the link, the CPU
21 stores the link information between the handwritten data and the voice data in the hard disk 4a as finalized link information (S
46).

Next, the CPU 21 displays a voice icon on the handwritten page of the handwritten sheet F as shown in FIG. 9 (o) (S47). Next, when the voice icon is displayed in the step (S47) or the link is not executed in the step (S45), the CPU 21 stores the estimated link information of the handwritten page data and the audio data from the estimated link information storage circuit 8. Delete (S48) and proceed to the next step (S4
Proceed to 9).

In step (S49), the CPU 21 further checks whether or not there is still information related to the handwritten page in the estimated link information storage circuit 8. If there is no related information, the process ends.

If there is no data related to the handwritten page data in the step (S34), or if the user does not perform the link processing of the photograph and the sound in the step (S36) and the step (S37), the CPU performs the estimation link. If the estimated link information in the information storage circuit 8 has been deleted (S38), this processing ends.

Although the embodiment of the present invention has been described above, the present invention does not limit the information to be collected to the above-mentioned media, but can provide, for example, a keyboard input, a communication data input, and the like. It is not limited to the rules and can be provided by the user as appropriate.

[0086]

According to the present invention, the CPU estimates the relationship between information of a plurality of media taken in from a user operation at the time of information collection, and links a link between the information of a plurality of media taken automatically. Since the information can be structured and structured, the burden on the user when collecting information can be reduced.

When the information collected later is used, the information can be used efficiently by the estimated link information between the information of the plurality of media taken in estimated by the present apparatus.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a configuration diagram of an embodiment of the present invention.

FIG. 3 is a specific configuration diagram of an embodiment of the present invention.

FIG. 4 is an operation explanatory diagram (part 1) of the embodiment of the present invention;

FIG. 5 is an operation explanatory view (part 2) of the embodiment of the present invention;

FIG. 6 is an operation explanatory view (part 3) of the embodiment of the present invention;

FIG. 7 is an operation explanatory view (part 4) of the embodiment of the present invention;

FIG. 8 is an operation explanatory diagram (part 5) of the embodiment of the present invention;

FIG. 9 is an operation explanatory diagram (part 6) of the embodiment of the present invention;

FIG. 10 is a processing flowchart when a photo menu is selected in one embodiment of the present invention.

FIG. 11 is a processing flowchart when a shooting button is pressed in one embodiment of the present invention.

FIG. 12 is a flowchart illustrating a process for determining the relevancy of a photographed image according to an embodiment of the present invention.

FIG. 13 is a processing flowchart when a recording menu is selected in one embodiment of the present invention.

FIG. 14 is a processing flowchart when a record button is pressed in one embodiment of the present invention.

FIG. 15 is a flowchart illustrating a process of determining the relevance of a recorded voice according to an embodiment of the present invention.

FIG. 16 is a flowchart illustrating a display process of a handwritten page according to an embodiment of the present invention.

FIG. 17 is an explanatory diagram of a conventional example.

[Explanation of symbols]

 Reference Signs List 3 memory circuit 4 data storage circuit 5 display circuit 6 association determination rule storage circuit 7 information collection operation status holding circuit 8 estimated link information storage circuit 11 media A information input circuit 12 media B information input circuit 13 media C information input circuit 21 CPU 22 System bus

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G06F 17/30 G06F 12/00 JICST file (JOIS)

Claims (3)

(57) [Claims] 1. A multimedia information collecting apparatus comprising: information input means for collecting data of a plurality of media; and data storage means for storing collected data. An information collecting operation for storing a user operation at the time of data collection. Status holding means (7); relevancy determining means (6) for holding a rule for estimating the relevance between a plurality of media data from the user operation; and the relevancy between the plurality of media data as the rule. A multimedia information collecting device, comprising: estimated link information storage means (8) for storing a result of estimation based on the information. 2. An input display integrated tablet (122) in which a flat display circuit (5a) and a coordinate input device (121) for performing handwriting input are integrated, an imaging device (101), and imaging by the imaging device. A multimedia signal collecting circuit comprising: an image signal processing circuit (102) for converting the converted image signal into a digital signal; and a data storage circuit (4) for storing input data from the coordinate input device and the imaging device. An information collection operation status holding circuit (7) for storing a user operation at the time of the input data; and an association for holding a rule for estimating an association between the coordinate input device and the input data from the imaging device from the user operation. Determining means (6) for describing a result of estimating the association between the input data of the coordinate input device and the input data of the imaging device based on the rule; A multimedia information collecting apparatus comprising: an estimated link information storage circuit (8). 3. An input display integrated tablet (122) in which a flat display circuit (5a) and a coordinate input device (121) for performing handwriting input are integrated, an imaging device (101), and imaging by the imaging device. An image signal processing circuit (102) for converting the converted image signal into a digital signal; a microphone (111) for inputting audio; an audio signal processing circuit (112) for converting an audio signal of the microphone into a digital signal; In a multimedia information collecting apparatus provided with a data storage circuit (4) for storing input data from the input device, the imaging device and the microphone, an information collecting operation status holding circuit (7) for storing a user operation at the time of the input data ), From the user operation the coordinate input device, the imaging device and the microphone A relevancy determining means (6) for holding a rule for estimating a relevance between the input data, and a relevancy between input data from the coordinate input device, the imaging device, and the microphone is estimated based on the rule. Estimated link information storage circuit for storing results (8)
A multimedia information collecting device, comprising: