JP2004258745A - Event acquisition device, indexing device, event acquisition method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible, when reproducing the contents of a lecture, meeting or the like, to display a supplement reference which was not presented in the lecture or the like simultaneously with such reproduction. <P>SOLUTION: While a motion picture is being reproduced by a motion picture reproducing application 130, when a user inputs an instruction of creating a new slide image to an application 120 for presentation via a user interface 110, an event data acquiring application 140 acquires reproduction time by the motion picture reproducing application 130 at issuance of the instruction to create event data wherein the reproduction time is associated with a page number identifying the newly-created slide image. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an event acquisition device, an indexing device, an event acquisition method, and a program.
[0002]
[Prior art]
Conventionally, when conducting presentations, lectures, meeting meetings, etc., the state of presentations, lectures, etc. is taken with a video camera or recorded with a recording device, and recorded data taken with a video camera or recorded data recorded on a recording medium etc. It may be recorded and distributed. A person who has received such a recording medium or the like reproduces the recorded data with a predetermined reproducing device, so that the presentation or lecture can be viewed or listened to.
[0003]
In presentations and lectures, a projector or the like that displays materials and the like on a screen is often used. Therefore, when a video or the like obtained by shooting a presentation or the like is reproduced, if the materials and the like displayed on the screen are displayed in synchronization with the reproduction, the contents of the presentation and the lecture can be more easily understood. Become.
[0004]
Accordingly, a technique has been proposed in which the state of a presentation is recorded on a video, and at the time of the presentation, a presenter obtains timing information for displaying each material image on a screen (for example, see Patent Document 1). .
[0005]
In such a technique, when playing back a video in which a presentation is recorded, the display timing information of each acquired image is referred to, and when the playback time of the presentation video reaches the display timing of a certain image, the certain image is displayed. Is displayed based on the image data.
[0006]
[Patent Document 1]
JP-A-10-55391
[0007]
[Problems to be solved by the invention]
In some cases, a participant in a lecture or a meeting may want to provide supplementary materials or the like that were not prepared at the time of the lecture or the meeting in order to easily convey the contents to a third party or the like. For example, at the time of the meeting, supplementary information that is not included in the recorded video and audio, such as what the atmosphere of the other party was when talking, was synchronized with the reproduction of these video and audio. It is considered that the presentation and the like can make it easier for viewers and listeners to understand the contents of the meeting.
[0008]
In the above-described conventional technology in which the presenter obtains the timing at which the prepared material is displayed in the lecture or the like in the above-described lecture and performs synchronous playback display, when the user wants to add a new supplementary material after the end of the lecture or the like Even could not add it.
[0009]
The present invention has been made in view of the above, and when reproducing the contents of a lecture or a meeting or the like, it is easy to display supplementary materials not presented in the lecture or the like in synchronization with the reproduction of the lecture or the meeting or the like. It is an object to obtain an event acquisition device, an indexing device, an event acquisition method, and a program.
[0010]
[Means for Solving the Problems]
In order to achieve the above object, the invention according to claim 1 provides a new application for an application that realizes an image display function from a user while reproducing a moving image or audio based on moving image data or audio data. An event recording means is provided for recording, when an instruction to create editable image display data is issued, a reproduction time at the time of the instruction and identification information for identifying the newly created image display data in association with each other. An event acquisition device characterized in that:
[0011]
According to the first aspect of the present invention, if an instruction to create new image display data is given to an application at a timing when it is desired to add an image display such as supplementary material while referring to a moving image display to be reproduced, The reproduction time at the time of the new creation instruction and the information for identifying the newly created image display data are recorded in association with each other. Therefore, by referring to such a record in which the reproduction time is associated with the identification information, the image display based on the image display data created by the application at the timing desired by the user at the time of reproducing the moving image or the like is performed. It is possible to easily create data to be performed.
[0012]
The invention according to claim 2 is the configuration according to claim 1, wherein the event recording unit is configured to determine whether or not the image display data created by the application already exists and when a timing instruction is given from a user. The reproduction time at the time of the instruction and the identification information for identifying the image display data are recorded in association with each other.
[0013]
According to the second aspect of the present invention, when there is image display data created in advance before a moving image or the like is reproduced, the user can perform the operation at a desired timing while the moving image is being reproduced. When the instruction is issued, the reproduction time at the time of the instruction and the information for identifying the image display data created in advance are recorded in association with each other. Therefore, by referring to such a record in which the reproduction time and the identification information are associated with each other, the reproduction of the moving image or the like is performed based on the image display data created in advance by the application at the timing desired by the user. Creation of data for displaying an image can be facilitated.
[0014]
The invention according to claim 3 is configured such that, when a start instruction is issued from the user and an instruction to create new editable image display data is issued from the user to an application for realizing the image display function, An event acquisition device, comprising: an event recording unit that records an elapsed time from the start instruction time at the time of execution and identification information for identifying the newly created image display data in association with each other. is there.
[0015]
According to the third aspect of the present invention, if an instruction to create new image display data is given to the application at a timing at which an image such as supplementary material is to be added, the start time specified by the user at the time of the new creation instruction Is recorded in association with information that identifies the newly created image display data. Therefore, if the user gives a start instruction at the same time as the start of the shooting of a meeting or a lecture while participating in a lecture or a meeting that has been separately shot, a record in which the elapsed time is associated with the identification information is recorded. , It is possible to easily create data for performing image display based on the image display data created by the application at a timing desired by the user when reproducing a captured image or the like.
[0016]
According to a fourth aspect of the present invention, in the configuration of the third aspect of the invention, the event recording means includes a step of: receiving a timing instruction from a user when image display data created by the application already exists; It is characterized in that an elapsed time from the start instruction time at the time of the instruction and the identification information for identifying the image display data are recorded in association with each other.
[0017]
According to the invention according to claim 4, when there is image display data created in advance before the user gives a start instruction, the user who participates in a meeting or lecture that is being photographed at a desired timing. When the instruction is issued, the elapsed time from the start instruction at the time of the instruction and the information for identifying the image display data created in advance are recorded in association with each other. Therefore, by referring to such a record in which the elapsed time and the identification information are associated with each other, an image created in advance by the application at the timing desired by the user at the time of reproduction of a moving image or the like photographed at a meeting or the like. It is possible to easily create data for performing image display based on the display data.
[0018]
According to a fifth aspect of the present invention, while recording moving image data and audio data supplied in real time, new editable image display data for an application realizing an image display function from a user. When an instruction to create the image is given, an event recording means for recording the elapsed time from the start of recording at the time of the instruction and identification information for identifying the newly created image display data is provided. It is an event acquisition device characterized by the following.
[0019]
According to the fifth aspect of the present invention, while recording a moving image or the like obtained by capturing a meeting or a lecture provided in real time, a new image is added to the application at a timing at which an image display such as supplementary material is to be added. When the display data creation instruction is issued, the elapsed time from the start of recording at the time of the new creation instruction and the information for identifying the newly created image display data are recorded in association with each other. Therefore, by referring to the recording in which the elapsed time and the identification information are associated with each other, the image display based on the image display data created by the application at the timing desired by the user when reproducing the recorded moving image or the like. Can be easily created.
[0020]
According to a sixth aspect of the present invention, in the configuration of the fifth aspect of the present invention, the event recording means includes a step of, when there is a timing instruction from a user when image display data created by the application already exists, It is characterized in that the elapsed time from the start of recording at the time of the instruction and the identification information for identifying the image display data are recorded in association with each other.
[0021]
According to the invention according to claim 6, when there is image display data created in advance before recording a meeting or a moving image, the user can select a desired timing while recording the moving image. In this case, the elapsed time from the start of recording at the time of the instruction and the information for identifying the image display data created in advance are recorded in association with each other. Therefore, by referring to the recording in which the elapsed time and the identification information are associated with each other, the image display data created in advance by the application at the timing desired by the user when reproducing the recorded moving image or the like. It is possible to easily create data for displaying an image based on the image.
[0022]
In addition, the invention according to claim 7 is a system in which, while reproducing a moving image or a sound based on the moving image data or the sound data, a new image display data is transmitted from the user to an application for realizing the image display function. When a creation instruction is issued, an event recording unit that associates and records a playback time at the time of the instruction and identification information for identifying the newly created image display data, and the reproduced moving image data or Indexing means for associating the audio data with event content including identification information associated with the playback time recorded by the event recording means in a time series based on the playback time recorded by the event recording means. An indexing apparatus characterized in that:
[0023]
According to the invention according to claim 7, if an instruction to create new image display data is given to the application at a timing when it is desired to add an image display such as supplementary material while referring to the moving image display to be reproduced, The reproduction time at the time of the new creation instruction and the information for identifying the newly created image display data are recorded in association with each other. Then, the indexing unit associates the moving image data with the identification information of the image display data in chronological order by the reproduction time included in the event content. Therefore, according to such association, it becomes easy to create data for performing image display based on the image display data created by the application at a timing desired by the user when reproducing a moving image or the like. be able to.
[0024]
Further, according to the invention, while recording moving image data and audio data supplied in real time, a user instructs an application for realizing an image display function to create new image display data. Event recording means for recording the elapsed time from the start of recording when instructed, and identification information for identifying the newly created image display data, and the recorded moving image Indexing means for associating data or the audio data with event content including identification information associated with the elapsed time recorded by the event recording means, in a time series based on the elapsed time recorded by the event recording means; An indexing device comprising:
[0025]
According to the invention according to claim 8, while recording a moving image or the like taken of a meeting or a lecture provided in real time, a new image is added to the application at a timing at which an image display such as supplementary material is to be added. When the display data creation instruction is issued, the elapsed time from the start of recording at the time of the new creation instruction and the information for identifying the newly created image display data are recorded in association with each other. Then, the moving image data and the identification information of the image display data are associated with the time series by the elapsed time included in the event content by the indexing means. Therefore, according to such association, it is easy to create data for performing image display based on the image display data created by the application at a timing desired by the user when reproducing the recorded moving image or the like. Can be
[0026]
According to a ninth aspect of the present invention, there is provided a reproducing step of reproducing a moving image or audio based on moving image data or audio data, and an application for realizing an image display function from a user during the reproducing in the reproducing step. Event recording that, when an instruction to create new editable image display data is issued, associates the playback time at the time of the instruction with identification information for identifying the newly created image display data and records the event. And an event acquiring method.
[0027]
According to the ninth aspect of the present invention, if an instruction to create new image display data is given to the application at a timing when it is desired to add an image display such as supplementary material while referring to a moving image display to be reproduced, The reproduction time at the time of the new creation instruction and the information for identifying the newly created image display data are recorded in association with each other. Therefore, by referring to such a record in which the reproduction time and the identification information are associated with each other, the data for performing the image display based on the image display data created by the application at the timing desired by the user at the time of reproducing the moving image. Can be easily created.
[0028]
The invention according to claim 10 is configured such that, when an instruction to create new editable image display data is issued to the application for realizing the image display function from the user after the start instruction from the user, An event acquisition method characterized by comprising an event recording step of recording an elapsed time from the time of the start instruction at the time of execution and identification information for identifying the newly created image display data in association with each other. is there.
[0029]
According to the tenth aspect of the present invention, when an instruction to create new image display data is given to the application at a timing at which an image such as supplementary material is to be added, the start time designated by the user at the time of the new creation instruction Is recorded in association with information that identifies the newly created image display data. Therefore, if the user gives a start instruction at the same time as the start of the shooting of a meeting or a lecture while participating in a lecture or a meeting that has been separately shot, a record in which the elapsed time is associated with the identification information is recorded. , It is possible to easily create data for performing image display based on the image display data created by the application at a timing desired by the user when reproducing a captured image or the like.
[0030]
The invention according to claim 11 provides a recording step for recording moving image data and audio data supplied in real time, and an application for realizing an image display function from a user during recording in the recording step. When there is an instruction to create a new editable image display data, the elapsed time from the start of recording at the time of the instruction and the identification information for identifying the newly created image display data are recorded in association with each other. And an event recording step.
[0031]
According to the eleventh aspect of the present invention, while recording a moving image or the like obtained by capturing a meeting or a lecture provided in real time, a new image is added to the application at a timing at which it is desired to display an image such as supplementary material. When the display data creation instruction is issued, the elapsed time from the start of recording at the time of the new creation instruction and the information for identifying the newly created image display data are recorded in association with each other. Therefore, by referring to the recording in which the elapsed time and the identification information are associated with each other, the image display based on the image display data created by the application at the timing desired by the user when reproducing the recorded moving image or the like. Can be easily created.
[0032]
According to the twelfth aspect of the present invention, a user can newly edit an application for realizing an image display function from a user while reproducing a moving image or a sound based on the moving image data or the sound data. Functioning as event recording means for associating and recording the reproduction time at the time of the instruction and the identification information for identifying the newly created image display data, It is a program characterized by the following.
[0033]
By causing a computer to execute the program according to the twelfth aspect, the same operation as the invention according to the first aspect is performed, and an image display created by the application at a timing desired by the user when reproducing a moving image or the like. It is possible to easily create data for displaying an image based on the data.
[0034]
Further, in the invention according to claim 13, after an instruction to start the computer is issued from the user, an instruction to create new editable image display data is issued from the user to an application for realizing the image display function. In this case, the program is caused to function as an event recording unit that records the elapsed time from the start instruction time at the time of the instruction and the identification information for identifying the newly created image display data in association with each other. It is.
[0035]
By causing a computer to execute the program according to claim 13, the same operation as the invention according to claim 3 is performed, and the image display created by the application at a timing desired by the user when reproducing a moving image or the like. It is possible to easily create data for performing image display based on the application data.
[0036]
Further, the invention according to claim 14 enables a user to newly edit an application for realizing an image display function from a user while recording the moving image data and the audio data supplied in real time. Recording means for associating the elapsed time from the start of recording at the time of the instruction with the identification information for identifying the newly created image display data when the instruction to create the new image display data is issued. It is a program characterized by functioning as.
[0037]
By causing a computer to execute the program according to claim 14, the same operation as the invention according to claim 5 is performed, and the image display created by the application at a timing desired by the user when reproducing a moving image or the like. It is possible to easily create data for performing image display based on the application data.
[0038]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of an event acquisition device, an indexing device, an event acquisition method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.
[0039]
A. First embodiment
FIG. 1 shows a functional configuration of the indexing device according to the first embodiment of the present invention. The indexing device 100 shown in FIG. 1 can be configured by a device that realizes various functions by executing an application by a CPU or the like, for example, a PC (Personal Computer), and includes a user interface (I / F) 110 and a presentation. Application 120, a moving image reproduction application 130, an event data acquisition application 140, an indexing application 150, and a storage unit 160.
[0040]
The user interface 110 has input devices such as a keyboard and a mouse, and the user can give instructions to various applications operating on the indexing device by appropriately operating these devices.
[0041]
The storage unit 160 is a storage device such as a hard disk drive capable of recording and reading data, and stores data created and used by various applications described later.
[0042]
The presentation application 120 creates slide image display data corresponding to slide images (tables, graphs, documents, patterns, etc.) to be displayed on a screen or the like when performing a presentation or the like.
[0043]
As illustrated in FIG. 2, the presentation application 120 manages a plurality of slide image display data created for use in one presentation as one presentation material file. Each slide image display data included in the slide image file includes data such as title data and a page number of the slide image in addition to display data corresponding to the slide image.
[0044]
When the user inputs an instruction to create new slide image display data via the user interface 110, the new slide image display data editable by the presentation application 120, that is, the blank slide image is displayed. Data to be displayed is created, and then the user issues an appropriate instruction via the user interface 110 to edit the slide image display data so that an image of a desired character, table, graph, pattern, or the like is displayed. You can do it.
[0045]
Also, when new slide image display data is created as described above according to the user's instruction, the title data is blank until the user inputs the instruction, and the desired title can be set by the user's instruction as appropriate. Can be described.
[0046]
In addition, as for the page number data of the newly created slide image display data, information indicating the number next to the largest page number created in the presentation material file so far is automatically described. For example, if the slide image display data to be newly created is the first slide image display data in the presentation material file, the page number is “1”, and three slide image display data have already been created. After that, when a new creation instruction is issued, the page number of the newly created slide image display data is “4”. According to the presentation application 120, the page numbers are automatically described, but this order can be edited, that is, changed to an arbitrary page number according to a user's instruction.
[0047]
The presentation application 120 causes the storage unit 160 to store the presentation material file created according to the user operation as described above.
[0048]
In addition, the user operates the user interface 110 as appropriate to instruct the presentation application 120 to select one of the plurality of slide image display data included in the created presentation material file, and At the timing, an image based on the slide image display data can be displayed on a display unit such as an LCD (Liquid Crystal Display).
[0049]
The moving image reproduction application 130 reproduces a moving image based on moving image data stored in the storage unit 16 in accordance with the MPEG (Moving Pictures Experts Group) 2 standard or the like. In addition, the moving image reproduction application 130 reproduces a moving image in accordance with an instruction supplied from the user via the user interface 110 such as reproduction start, pause, reproduction end, and fast forward.
[0050]
The event data acquisition application 140 monitors the state of the presentation application 120 while the moving image data is being reproduced based on the moving image data by the moving image reproduction application 130, and instructs the user to create new slide image display data. Creates and records event data when is performed on the presentation application 120.
[0051]
More specifically, as shown in FIG. 3, when the moving image reproduction by the moving image reproduction application 130 is started, the event data acquisition application 140 transmits a new slide image display data to the presentation application 120. It is monitored whether or not a creation instruction has been issued (step Sa1). If there is a new creation instruction, page number data is acquired as information for identifying slide image display data created by the new creation instruction (step Sa1). Step Sa2).
[0052]
Here, the following method can be used as a method by which the event data obtaining application 140 obtains a notification of a new creation instruction from the presentation application 120, page number data of a newly created slide image, and the like. For example, when the presentation application is Microsoft's “PowerPoint”, information indicating various states of the application can be acquired using a technique called OLE (Object Linking and Embedding). In this case, by calling a function so that data can be exchanged between the event data acquisition application 140 and the presentation application supporting OLE, it is possible to acquire information indicating the application state as described above. it can.
[0053]
Note that a method other than acquiring the information indicating the state of the presentation application 120 using the OLE technology may be used. For example, when a new creation of the presentation application is instructed, The process of notifying the event data acquisition application 140 of the fact and the page number of the created slide image may be executed.
[0054]
When the presence / absence of a new creation instruction and the page number data are acquired by the method described above, the event data acquisition application 140 acquires the playback time of the moving image data by the moving image playback application 130 at the time when the new creation instruction is issued ( Step Sa3). For example, while the moving image reproduction based on 35 minutes of moving image data such as “0: 00: 00: 00” to “35: 00: 0” shown in FIG. Is "0:07:12", the event data acquisition application 140 acquires "0:07:12" as the reproduction time. Further, when the moving image reproduction by the moving image reproduction application 130 is started at an intermediate position, for example, “0:13:15”, a new creation instruction is issued when the reproduction position is “0:25:18”. Sometimes, the event data acquisition application 140 acquires the reproduction time “0:25:18” of the moving image data. That is, the elapsed time from the start of the reproduction is 12 minutes and 3 seconds, but the event data acquisition application 140 is not the elapsed time from the start of the reproduction, but the time when a new creation instruction is given from the user. The reproduction time on the moving image data is obtained.
[0055]
When the user issues a new creation instruction to the presentation application 120, the event data acquisition application 140 acquires the playback time at the time of the instruction and the page number of the slide image created by the instruction. The time and the page number data are associated and recorded as event data (step Sa4). That is, as shown in FIG. 4, when a new creation instruction is issued at the playback time “0:07:12” and the slide image display data of page number “2” is newly created by the presentation application 120, Creates and records event data in which the reproduction time “0:07:12” is associated with “2 pages”. When a new creation instruction is issued at the playback time “0:25:18” and the slide image display data of the page number “3” is created by the presentation application 120, the playback time “0:25:18” is displayed. Event data in which “18” and “3 pages” are associated with each other is created and recorded.
[0056]
The event data acquisition application 140 performs the above-described event data recording / creating process from step Sa2 to step Sa4 every time the user issues a new slide image creation instruction to the presentation application 120.
[0057]
On the other hand, if there is no new creation instruction from the user (determination “No” in step Sa1), it is determined whether or not the moving image reproduction by the moving image reproduction application 130 has been completed (step Sa5). The case where the reproduction of the moving image is completed means whether the reproduction based on a certain moving image data has been completed, that is, the reproduction position has reached “0:35:00” in the case of the moving image data shown in FIG. Alternatively, this is a case where the user instructs the moving image reproduction application 130 to end reproduction through the user interface 110.
[0058]
When the moving image reproduction by the moving image reproduction application 130 has been completed, the event data acquisition application 140 executes the processing of steps Sa2 to Sa4 and records the event data recorded while performing the moving image reproduction. The output is output to the indexing application 150 (step Sa6).
[0059]
The indexing application 150 shown in FIG. 1 converts the event data supplied from the event data acquisition application 140 after the moving image reproduction ends, and the slide image display data created by the presentation application 120 during the moving image reproduction as described above. Based on the presentation material file including the moving image data and the moving image data reproduced by the moving image reproducing application 130, synchronous reproduction data capable of synchronously reproducing a moving image based on the moving image data and a slide image is generated. .
[0060]
As a data format for controlling and synchronizing data of various different formats such as moving images, still images, and characters, for example, there is a SMIL (Synchronized Multimedia Integration Language) format, and the indexing application 150 in the present embodiment performs synchronous reproduction. The SMIL format data is created as data.
[0061]
By reproducing such synchronized reproduction data by a predetermined reproduction software (for example, software for reproducing SMIL format data) on a PC or the like, a display as shown in FIG. 5 is made on the display screen.
[0062]
As shown in the figure, the display screen includes a video display column 500 for displaying a moving image reproduced by the moving image reproduction application described above, and a slide included in a presentation material file created during the reproduction of the moving image. A slide image column 510 for displaying slide images based on the image display data and a table of contents column 520 for displaying a list of titles of these slide images are displayed.
[0063]
Synchronous reproduction data is data that enables the above-described characters such as moving images, slide images, and titles to be reproduced and displayed in synchronization with each other. For details of the synchronous reproduction data creation processing, see FIG. It will be explained while doing so.
[0064]
First, the indexing application 150 extracts character information such as title data and page number data of each slide image display data from the presentation material file stored in the storage unit 160 (step Sb1). In addition, the indexing application 150 extracts slide image display data from the presentation material file and saves the data as slide image display data in a predetermined format (for example, a general-purpose format such as JPEG (Joint Photographic Experts Group)). (Step Sb2).
[0065]
Next, based on the event data, the document information such as the extracted title data and page number data, the presentation material file and the moving image data, the indexing application 150 generates a video based on the moving image data as shown in FIG. Synchronized playback data is created that allows the slide images based on the slide image display data in the presentation material file and character information such as the titles of these slide images to be displayed in synchronization (step Sb3).
[0066]
As described above, in the present embodiment, SMIL format data is created as such synchronized playback data, and the indexing application 150 acquires the following data from the supplied data and converts them into SMIL format data. , To create SMIL-format synchronized playback data.
(1) New creation timing of each slide image
(2) Title data of each slide image, etc. (title, body text, page number, etc.)
(3) Storage location (file path information) for each slide image (JPEG, etc.)
(4) Storage location of moving image data corresponding to each slide image (file path information)
(5) Layout data indicating how to arrange the video display column 500, the slide image column 510, and the table of contents column 520 (see FIG. 5).
[0067]
The layout information of (5) is not data supplied from event data or the like, but data created according to a layout specified by the user.
[0068]
FIG. 7 schematically shows the correspondence between the synchronization of the title, the slide image, and the video file in the synchronously reproduced data thus created. In the example shown in the drawing, at the time of reproduction of a moving image, the user's instruction to create a new slide image for the first page to the presentation application 120 is performed at the playback time “00:00:00”, and the instruction to create the new slide image for the second page is issued. At the playback time “0:01:15”, a new slide image creation instruction for the third page is performed at the playback time “0:02:45”, and a new slide image creation instruction for the fourth page is issued at the playback time “0”. : 03:53 ", and the event data at the instruction timing is acquired by the event data acquisition application 140.
[0069]
When a new slide image creation instruction is issued at the above timing during video playback, a period from when a new slide image creation instruction for the first page is instructed to a new slide image creation instruction for the second page is given. The moving image data reproduced at (time 0:00:00 to 0:01:15) is associated with the slide image of the first page and its title “Introduction”. Also, during the period from when a new slide image creation instruction for the second page is issued to when a new slide image creation instruction for the third page is issued (time 0:01:15 to 0:02:45). The moving image data is associated with the slide image of the second page and its title “background”. Further, the moving image reproduced between the time when the new slide image creation instruction for the third page is issued and the time when the new slide image creation instruction for the fourth page is issued (time 0:02:45 to 0:03:53). The image data is associated with the slide image of the third page and its title “Summary”.
[0070]
The indexing application 150 can control and synchronize the reproduction of data in different formats, such as character information such as a title, still images such as a slide image, and moving image data, based on the time-series correspondence as described above. It creates data in SMIL format.
[0071]
By reproducing the synchronous reproduction data by a predetermined reproduction software on a PC or the like, a display as shown in FIG. 5 is made on the display screen.
[0072]
If the user viewing the display clicks on "background", the synchronized playback data includes a path to the slide image of the second page and the corresponding moving image data (start time 0:01:15). Since the link to the title is described, the corresponding slide image of the second page is displayed in the slide image column 510 by the playback software, and at the time “0:01:15” corresponding to the title. The reproduction of the moving image data is started and is displayed in the video display column 500.
[0073]
If the user clicks on “Overview”, the slide image on the third page and the path to the moving image data (start time 0:02:45) corresponding to the title are linked. Is displayed in the slide image column 510, and the reproduced video based on the moving image data from the time “0:02:45” is displayed in the video display column 500.
[0074]
Of course, the slide image corresponding to the designated reproduction time can be displayed in the slide image column 510 by specifying the reproduction time to be displayed in the video display column 500 by the user. For example, when the user instructs to reproduce a moving image from the reproduction time “0:02:30”, the reproduction software performs the reproduction of the moving image from that time according to the synchronous reproduction data, and performs the reproduction at the designated time “0:02:30”. The slide image of the second page (corresponding to the video of 0:01:15 to 0:02:45) associated with “02:30” is read out and displayed in the slide image column 510.
[0075]
The reproduction by the synchronized reproduction data created through the above-described process includes, in addition to the video based on the moving image data, the slide image of the newly created page at the timing of the new creation instruction performed with reference to the moving image reproduction. The displayed slide image of the page up to the timing at which a new page creation instruction is issued is displayed.
[0076]
Therefore, the user of the indexing apparatus 100 issues a new creation instruction to the presentation application 120 at a timing at which supplementary materials and the like are desired to be added while referring to the moving image playback of the meeting or the lecture, and then newly creates the new creation. By performing operations such as editing the slide image into a slide image that displays the material to be supplemented, as described above, the supplementary material image of the intended content is created at the timing intended by the editor according to the progress of the moving image playback as described above. Synchronous reproduction data that can be displayed can be created.
[0077]
Further, in the present embodiment, the event data acquisition application 140 acquires the reproduction time of the moving image reproduction by the moving image reproduction application 130 at the time when the new creation instruction is given by the user, and starts from the middle of the moving image data. Even when the reproduction is started, the reproduction time of the moving image data is acquired instead of the elapsed time after the start of the halfway reproduction (see FIG. 4).
[0078]
In this way, the user temporarily stops playback based on the moving image data in order to perform a newly created slide image editing operation, and then gives a moving image playback instruction from the point of the stop again, and refers to the playback video. However, even when a new creation instruction is issued at a desired timing, the reproduction time of the moving image data at the time of the instruction can be acquired as event data. Therefore, a slide that presents the created supplementary material or the like at the timing desired by the user at the time of reproducing the moving image data by generating the synchronous reproduction data using the event data thus obtained and reproducing the synchronous reproduction data. Images can be displayed.
[0079]
B. Second embodiment
Next, an indexing device according to a second embodiment of the present invention will be described with reference to FIG. As shown in the figure, the indexing device 200 according to the first embodiment is different from the indexing device 100 according to the first embodiment in that a moving image recording / reproducing unit 180 is provided instead of the moving image reproducing application 130 shown in FIG. Is different.
[0080]
The moving image recording / playback unit 180 is a function realized by a capture board or software, and converts moving image data captured by a DV (Digital Video) camera 190 and supplied in real time into a predetermined format (for example, MPEG2 or the like). The image data is converted into moving image data, and the converted moving image data is stored in the storage unit 160. The moving image recording / playback unit 180 executes the moving image playback based on the moving image data supplied from the DV camera 190 in real time while recording as described above, and displays the playback image on a display unit (not shown) or the like. Let it.
[0081]
In the present embodiment, when the synchronized playback data is created using the indexing device 200 in the same manner as the indexing device 100 according to the first embodiment, the DV camera 190 shoots, for example, a meeting or a lecture, and shoots the image. The moving image recording / reproducing unit 180 records and reproduces the moving image in real time. By referring to the playback video, the editor issues a new creation instruction to the presentation application 120 at a timing at which supplementary material or the like is to be presented, so that the same synchronized playback data as in the first embodiment can be created. it can.
[0082]
In other words, the event data acquisition application 140 according to the second embodiment sends the user to the presentation application 120 while the moving image data supplied from the DV camera 190 is recorded in real time by the moving image recording / reproducing unit 180. When a new slide image creation instruction is issued from the server, the page number of the newly created slide image is acquired, and the recording time of the moving image recording / playback unit 180 at the time when the instruction is issued is acquired. Record as data (see FIG. 3).
[0083]
Here, the recording time by the moving image recording / reproducing unit 180 is an elapsed time from the point in time when the shooting by the DV camera 190 is started and the shot image is supplied in real time to start the recording of the moving image data. . Therefore, when a new slide image creation instruction for the third page is issued 3 minutes and 20 seconds after the start of recording of the moving image captured by the DV camera 190, the event data acquisition application 140 records the recording time “0:03: 20 "is acquired and recorded in association with a page number such as three pages.
[0084]
The synchronous reproduction data creation processing performed by the indexing application 150 in the second embodiment is the same as that in the first embodiment. Therefore, when the reproduction based on the synchronized reproduction data generated by the indexing application 150 is performed, the reproduction time of the moving image based on the moving image data recorded by the moving image recording / reproducing unit 180 becomes “0:03:20”. At this point, the slide image of the third page is displayed in the slide image column 510 (see FIG. 5).
[0085]
As described above, the user of the indexing apparatus 200 shoots and records a meeting or a lecture or the like, actually refers to a video reproduced in real time, or actually participates in a lecture or a meeting to be shot. Meanwhile, a new creation instruction is given to the presentation application 120 at a timing when the supplementary material is to be displayed, and then the newly created slide image is edited into a slide image displaying the material to be supplemented, thereby performing the above-described operation. As described above, when playing back a moving image, it is possible to create synchronous playback data that can display a supplementary material image of the intended content at the timing intended by the editor.
[0086]
Note that the recording of moving image data supplied in “real time” in the present specification does not include a case where a file such as moving image data is supplied and copied, but a video captured by the DV camera 190 or the like is included. Is supplied to the indexing apparatus 200 as it is, and is supplied at the same time as when the moving image data previously shot by the DV camera 190 or the like is reproduced to the indexing apparatus 300, that is, with 35 minutes of moving image data. This includes the case where the video data is supplied in 35 minutes, and also the case where the video recording / reproducing unit 180 receives video data such as a broadcasted program and records it.
[0087]
C. Third embodiment
Next, an indexing device 300 according to a third embodiment of the present invention will be described with reference to FIG. As shown in the figure, the indexing device 300 according to the third embodiment is different from the indexing device 100 according to the first embodiment in not having the moving image reproduction application 130 shown in FIG.
[0088]
In the indexing device 300, the user instructs the event data acquisition application 140 via the user interface 110 to start acquiring event data. Upon receiving such an instruction, the event data acquisition application 140 starts measuring time.
[0089]
When a new creation instruction is issued to the presentation application 120 after the acquisition start instruction is issued, the event data acquisition application 140 acquires the elapsed time from the acquisition start instruction when the instruction is issued, The slide image is recorded as event data in association with the page number of the slide image created by the new creation instruction.
[0090]
Based on the event data acquired by the event data acquisition application 140 in this manner, the indexing application 150 creates synchronized playback data as in the first embodiment. Here, the moving image data to be reproduced in synchronization with the display of the slide image is recorded by a moving image recording device different from the indexing device 300 or the like. Then, the moving image data recorded by the moving image recording device is supplied to the indexing device 300 and stored in the storage unit 160, so that the indexing application 150 performs processing based on the moving image data as in the first embodiment. Synchronous reproduction data that can synchronize the reproduction of a moving image with the display of a slide image can be created.
[0091]
That is, the user of the indexing apparatus 300 that is actually participating in a lecture or a meeting gives an instruction to start shooting to a moving image recording apparatus (not shown) that shoots the state of the lecture or the meeting at the same time or almost simultaneously. What is necessary is just to give an acquisition start instruction to the event data acquisition application 140 of the indexing device 300. Thereafter, when the user of the indexing apparatus 300 wants the moving image recording apparatus to photograph and record a meeting or a lecture, and to display supplementary material while actually participating in the lecture or meeting to be photographed. In the same manner as in the first embodiment described above, a new creation instruction is issued to the presentation application 120, and then the newly created slide image is edited into a slide image that displays the material to be supplemented. In accordance with the progress of the image reproduction, it is possible to create synchronous reproduction data capable of displaying the supplementary material image of the intended content at the timing intended by the editor.
[0092]
D. Modified example
Note that the present invention is not limited to the above-described embodiment, and various modifications as exemplified below are possible.
[0093]
(Modification 1)
In the first embodiment described above, the event data acquisition application 140 acquires the reproduction time of the moving image reproduction by the moving image reproduction application 130 at the time when a new creation instruction is given by the user. Further, when one piece of synchronous reproduction data is created, when another piece of moving image data is added to one piece of moving image data, the event data acquisition application 140 acquires the event data as follows. You may.
[0094]
As shown in FIG. 10, after the moving image data A for 35 minutes is reproduced by the moving image reproducing application 130, when the moving image data B to be added thereto is further reproduced, the reproduction time of the moving image data B is changed. When the user issues an instruction to newly create an n-page slide image at “0:05:12”, the event data acquisition application 140 sets the playback time of the moving image data A to the playback time of the moving image data B. 35 minutes are added and adjusted, and the adjusted playback time “0:40:12” is obtained to create event data.
[0095]
By doing so, even when synchronous playback data such as continuous playback of the video data A and the video data B is created, the event data acquisition application 140 takes the playback time of the video data A into consideration. Since the playback time has been acquired, when the playback position for continuous playback reaches “0:05:12” of the moving image data B, that is, when the continuous playback time reaches “0:40:12”, n Synchronous reproduction data capable of synchronous reproduction such that a slide image of a page is displayed in a slide display field 510 shown in FIG. 5 can be created.
[0096]
(Modification 2)
In the above-described first to third embodiments, when the event data acquisition application 140 receives a new slide image creation instruction from the presentation application 120, the timing (playback time, the time from the start of recording). Event data including an elapsed time, an elapsed time from a user's start instruction, and the like are obtained.
[0097]
Not only when all the slide images are newly created as described above, but also when one or a plurality of slide images have already been created before moving image reproduction, recording, user start instruction, etc., are performed. When a new creation instruction is issued to add the event data, the event data acquisition application 140 may create event data including the instruction timing, as in the above embodiments.
[0098]
Further, when one or a plurality of slide images have already been created, the event data acquisition application 140 transmits a page related to the timing designation instruction among the slide images already created based on the timing designation instruction from the user. The event data may be created by associating the number with the timing such as the reproduction time at which the timing designation instruction was issued.
[0099]
For example, as shown in FIG. 11, when a reproduction based on moving image data is started in a state where a slide image of one page has already been created, a timing designation instruction is issued from the user to the event data acquisition application 140. If there is, the event data acquisition application creates event data in which the reproduction time “0:08:14” at the time of the instruction is associated with one page. Thereafter, when a new creation instruction is given to the presentation application 120 at the playback time “0:13:24” of the playback based on the moving image data, the event data acquisition application 140 operates in the same manner as in the first embodiment. Then, event data is created in which the reproduction time “0:13:24” at the time of the instruction is associated with two pages that are newly created pages.
[0100]
(Modification 3)
In each of the above-described embodiments, when the user issues a new slide image creation instruction to the presentation application 120, the event data acquisition application 140 creates event data including the instruction timing. However, event data may be created when a new creation instruction is issued to an application having an image display function other than the presentation application 120.
[0101]
For example, an application having a function of displaying a document image created and edited, such as a document data creation and editing application (word processor application), and an application having a function of displaying a created table and a graph, such as a spreadsheet application, Is also good. When the user issues a new document data creation instruction to the document data creation / editing application, the event data acquisition application 140 sends the instruction timing and information (file name, etc.) for identifying the newly created document data. ) May be created.
[0102]
(Modification 4)
In each of the above-described embodiments, reproduction of data in different formats such as moving image data obtained by capturing a lecture or a meeting, slide image display data, and character information such as a title of each slide image is controlled and synchronized. Created the SMIL format data that can be synchronized, but not only the title of the slide image synchronized with the video playback, but also the synchronized playback that synchronizes the audio playback with the slide image display based on the audio data that recorded the lectures and meetings. For data creation, the same event data acquisition processing as in the above embodiments may be performed.
[0103]
(Modification 5)
Further, in the above-described embodiment, the indexing application 150 generates a moving image, a slide image, and the like based on event data, document information such as extracted title data and page number data, a presentation material file, and moving image data. SMIL format data has been created as synchronous reproduction data that can synchronously display character information such as a title of a slide image.
[0104]
As described above, data in SMIL format may be created as synchronous reproduction data. However, even if data other than SMIL is used, video (or audio or the like), still images such as slide images, and characters such as titles may be used. Any data may be used as long as the data can be controlled and synchronized in different formats such as information, and data in a format other than the SMIL format may be created.
[0105]
(Modification 6)
In each of the above-described embodiments, the presentation application 120, the moving image reproduction application 130, the event data acquisition application 140, and the indexing application 150 operate in a single indexing device. A plurality of PCs or the like may share the functions of the application to realize an indexing device that generates synchronized playback data.
[0106]
(Modification 7)
In each of the above-described embodiments, a computer such as a CPU incorporated in a PC or the like reads out and operates a program stored in an external storage device or the like to execute the event acquisition process (see FIG. 3) and the indexing process. (See FIG. 6). However, the same processing as the above processing may be realized by a hardware circuit, or a program for causing a computer to execute such processing may be provided. The program may be provided to the user via a communication line such as the Internet, or the program may be recorded on a computer-readable recording medium such as a CD-ROM (Compact Disc-Read Only Memory) and provided to the user. It may be.
[0107]
【The invention's effect】
As described above, according to the first aspect of the present invention, an image display based on image display data created by the application at a timing desired by the user when reproducing a moving image or the like obtained by taking a lecture or a meeting or the like. This makes it possible to easily create data for performing the above.
[0108]
According to the second aspect of the present invention, an image display based on image display data created in advance by the application at a timing desired by the user when reproducing a moving image or the like obtained by taking a lecture or a meeting or the like. This makes it possible to easily create data for performing the following.
[0109]
According to the third aspect of the present invention, when a user gives a start instruction at the same time as the start of shooting of a meeting or a lecture when the user is participating in a lecture or a meeting that is separately shot, the reproduction of a shot image or the like can be performed. In this case, it is possible to easily create data for performing image display based on the image display data created by the application at a timing desired by the user.
[0110]
According to the invention of claim 4, when reproducing a moving image or the like obtained by photographing a meeting or the like, an image display based on image display data created in advance by the application at a timing desired by the user is performed. There is an effect that the creation of data to be performed can be facilitated.
[0111]
According to the invention according to claim 5, when a moving image or the like obtained by recording a meeting or a lecture is reproduced, an image is displayed based on the image display data created by the application at a timing desired by the user. This has the effect of facilitating data creation and the like.
[0112]
Further, according to the invention according to claim 6, an image based on image display data created in advance by the application at a timing desired by the user at the time of reproduction of a moving image or the like obtained by photographing and recording a meeting or a lecture. This has the effect of facilitating creation of data to be displayed.
[0113]
According to the seventh aspect of the present invention, when reproducing a moving image or the like obtained by capturing a meeting or the like, generating data for performing image display based on the image display data generated by the application at a timing desired by the user. And the like can be facilitated.
[0114]
Further, according to the invention according to claim 8, when reproducing a moving image or the like in which a meeting or the like is recorded, data for displaying an image based on the image display data created by the application at a timing desired by the user. This has the effect of facilitating creation and the like.
[0115]
According to the ninth aspect of the present invention, creation of data for performing image display based on the image display data created by the application at a timing desired by the user in response to playback of a moving image obtained by capturing a meeting or the like. And the like can be facilitated.
[0116]
According to the tenth aspect of the present invention, when the user participates in a lecture or a meeting that has been separately photographed or the like and gives a start instruction at the same time as the start of the shooting of the meeting or the lecture, the photographed image or the like can be obtained. This has the effect of facilitating the creation of data for performing image display based on the image display data created by the application at the timing desired by the user during reproduction.
[0117]
According to the eleventh aspect of the present invention, when reproducing a moving image or the like in which a meeting or the like is captured and recorded, data for performing image display based on image display data created by the application at a timing desired by the user is provided. This has the effect of facilitating creation and the like.
[0118]
Further, by causing a computer to execute the program according to claim 12, the same operation as that of the invention according to claim 1 is performed, and the program created by the application at a timing desired by the user when reproducing a moving image or the like. This has the effect of facilitating the creation of data for performing image display based on the image display data.
[0119]
By causing a computer to execute the program according to claim 13, the same operation as the invention according to claim 3 is performed, and the application executes the program at a timing desired by the user in accordance with the progress of reproduction of a moving image or the like. It is possible to easily create data for performing image display based on the created image display data.
[0120]
Further, by causing a computer to execute the program according to claim 14, the same operation as the invention according to claim 5 is performed, and the program created by the application at a timing desired by the user when reproducing a moving image or the like. Creation of data for performing image display based on the image display data can be facilitated.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an indexing device according to a first embodiment of the present invention.
FIG. 2 is a diagram showing the contents of a presentation material file created by a presentation application operating on the indexing device.
FIG. 3 is a flowchart showing a procedure of event data acquisition processing by an event data acquisition application operating on the indexing device.
FIG. 4 is a diagram for explaining an event data acquisition process by the event data acquisition application.
FIG. 5 is a diagram showing an example of a reproduced image based on synchronous reproduction data created by the indexing device.
FIG. 6 is a flowchart showing a procedure of processing by an indexing application operating on the indexing device.
FIG. 7 is a diagram schematically showing a correspondence relationship between a moving image and a slide image in the synchronously reproduced data for synchronously reproducing data of different formats such as a moving image and a slide image.
FIG. 8 is a block diagram illustrating a configuration of an indexing device according to a second embodiment of the present invention.
FIG. 9 is a block diagram illustrating a configuration of an indexing device according to a third embodiment of the present invention.
FIG. 10 is a diagram for explaining the content of event data acquisition processing by a modification of the indexing device according to the first embodiment.
FIG. 11 is a diagram for explaining the content of event data acquisition processing by a modification of the indexing device according to each embodiment.
[Explanation of symbols]
100 indexing device
110 User Interface
120 Presentation Application
130 Video playback application
140 Event data acquisition application
150 Indexing Application
160 storage unit
180 Video recording and playback unit
200 indexing device
300 indexing device

Claims (14)

While reproducing a moving image or audio based on the moving image data or audio data, when the user instructs an application that realizes the image display function to create new editable image display data, An event acquisition apparatus, comprising: an event recording unit that associates and records a reproduction time when instructed and identification information for identifying the newly created image display data. The event recording means, when a timing instruction is given by a user when image display data created by the application already exists, a reproduction time at the time of the instruction and identification information for identifying the image display data The event acquisition device according to claim 1, wherein the event acquisition device records the event in association with the event. If there is an instruction from the user to create new editable image display data for the application realizing the image display function after the user issues an instruction to start the application, the instruction from the time of the start instruction at the time of the instruction is given. An event acquisition device, comprising: an event recording unit that records an elapsed time in association with identification information for identifying the newly created image display data. The event recording means, when there is a timing instruction from the user when the image display data created by the application already exists, an elapsed time from the start instruction time at the time when the instruction is issued, The event acquisition device according to claim 3, wherein the event acquisition device records the data in association with identification information for identifying data. While recording moving image data and audio data supplied in real time, if the user instructs an application that realizes the image display function to create new editable image display data, the instruction is issued. An event acquisition device, comprising: an event recording unit that records an elapsed time from the start of recording at the time of recording and identification information for identifying the newly created image display data in association with each other. The event recording means, when there is a timing instruction from the user when the image display data created by the application already exists, the time elapsed from the start of recording at the time of the instruction, and the image display data 6. The event acquisition device according to claim 5, wherein the event acquisition device records the identification information in association with the identification information. When a user instructs an application for realizing an image display function to create new image display data while reproducing a moving image or audio based on the moving image data or audio data, the instruction is issued. Event recording means for recording the reproduction time at the time and the identification information for identifying the newly created image display data in association with each other;
The reproduced moving image data or the audio data and the event content including the identification information associated with the reproduction time recorded by the event recording means are timed based on the reproduction time recorded by the event recording means. An indexing device, comprising: indexing means for associating with a stream. While recording moving image data and audio data supplied in real time, if the user issues an instruction to create new image display data to an application that implements the image display function, Event recording means for recording the elapsed time from the start of recording in association with identification information for identifying the newly created image display data,
Based on the elapsed time recorded by the event recording unit, the recorded moving image data or the audio data and the event content including the identification information associated with the elapsed time recorded by the event recording unit are timed. An indexing device, comprising: indexing means for associating with a stream. A reproducing step of reproducing a moving image or a sound based on the moving image data or the sound data;
During the playback in the playback step, when a user issues an instruction to create new editable image display data to an application that implements an image display function, the playback time at the time of the instruction and the new An event recording step of recording the created image display data in association with identification information for identifying the image display data. If there is an instruction from the user to create new editable image display data for the application realizing the image display function after the user issues an instruction to start the application, the instruction from the time of the start instruction at the time of the instruction is given. An event acquisition method comprising an event recording step of recording an elapsed time in association with identification information for identifying the newly created image display data. A recording step of recording moving image data and audio data supplied in real time,
During the recording in the recording step, when a user issues an instruction to create new editable image display data for an application that implements an image display function, the elapsed time from the start of recording at the time of the instruction. An event recording step of recording in association with identification information for identifying the newly created image display data. Computer
While reproducing a moving image or audio based on the moving image data or audio data, when the user instructs an application that realizes the image display function to create new editable image display data, A program that functions as an event recording unit that records a reproduction time at the time of an instruction and identification information for identifying the newly created image display data in association with each other. Computer
If there is an instruction from the user to create new editable image display data for the application realizing the image display function after the user issues an instruction to start the application, the instruction from the time of the start instruction at the time of the instruction is given. A program that functions as an event recording unit that records an elapsed time in association with identification information for identifying the newly created image display data. Computer
While recording moving image data and audio data supplied in real time, if the user instructs the application that realizes the image display function to create new editable image display data, A program that functions as an event recording unit that records an elapsed time from the start of recording when instructed and identification information for identifying the newly created image display data in association with each other.

Images