JP2008017171A - Recording device and method of multimedia data, and creation system and method of multimedia contents - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a recording device which simply and on real time records material data and information required for creating multimedia contents including what is called a voice file with a chapter or a voice file equivalent to it. <P>SOLUTION: An indicator 110 changes with time an image of lecture data created by presentation software, or switches, and indicates it on a display screen. A data recorder 150 records digital voice data from a voice input 140 continuously during a predetermined time, and stores the file of the voice data 151 in a memory storage. Further, responding to the output of a record indication detector 160, the data recorder 150 records the image data of a slide image in a specific time together with time information on the specific time among a large number of the slide images displayed by the indicator 110 in a predetermined time, and stores each of those image data 152 in the memory storage as an image file with time information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus and method for recording necessary material data and information in real time on a computer for creating multimedia contents in which one or more image data is incorporated in one audio file, and It relates to a system and method for creating such multimedia content.

  In recent years, the number of educational institutions that electronically archive lecture contents as audio data or video data and distribute them via the Internet is increasing.

When an audio file containing audio data recording the lecture content is distributed, the amount of data is small, but in many cases the state of the lecture cannot be sufficiently communicated only by audio. For this reason, a proposal has been made to create a moving image file including digital moving image data of lecture content and distribute it via a network (for example, see Patent Document 1).
JP 2005-79913 A

  When moving image files are distributed as in Patent Document 1, it is good to accurately convey the state of the lecture, but there is a problem that the amount of data is enlarged. The enlarged data increases the upload time on the distributor side, the download time on the student side, and the amount of traffic on the Internet, and also increases the consumption of data storage media on both the student side and the distributor side. Leads to.

  As a method for improving the above-mentioned problems, use of a so-called chapter-attached audio file can be mentioned. An audio file with chapters is a data format in which a chapter (chapter) is added to an arbitrary portion of the audio file, and programmable information such as images and URL links, character information, and the like can be inserted into the chapter. The "Enhanced Podcast" that can be downloaded from the iTunes Music Store, a music download service site that the company has developed around the world, using "iTunes" software provided free of charge by Apple Computer.

  An audio file with chapters has an advantage that the amount of data is small while the amount of information is as large as that of a moving image file. However, in order to produce this, special software and corresponding hardware (for example, “GarageBand 3” and an Apple computer that meets the requirements for operating it) are required, and the operation of the software has been achieved. You need a professional operator. If a user (for example, a lecturer or his assistant) wants to create a voice file with chapters, knowledge and skill equivalent to that of a dedicated operator are required.

  In addition, existing audio file creation software with chapters can record audio data, but no interface for recording image data has been prepared. Therefore, after creating the original audio data file, it is necessary to import the image data created by the user himself / herself or obtained from a third party by the user's mouse operation etc. and insert it into the audio data. It was.

  Therefore, one object of the present invention is to easily and in real time record necessary material data and information for creating multimedia content including a so-called chapter-attached audio file or an equivalent audio file. It is to provide an apparatus and method.

  Another object of the present invention is to provide a system and method that includes a user terminal and a server and can create multimedia contents with simple and few operations on the user terminal.

  In order to achieve the above object, a multimedia data recording apparatus according to one aspect of the present invention includes a display unit for changing or switching an image over time, a voice input unit for inputting voice, Audio data from the audio input means is continuously recorded for a predetermined time, and image data at one or more specific times among the images displayed by the display means within the predetermined time is displayed as time information of the specific time. And data recording means for recording together.

  According to the multimedia data recording apparatus of the present invention, for example, audio data of lecture contents while an image of a lecture material created by presentation software is changed or changed over time on a lecturer computer and displayed on a display means. During recording of audio data, image data at a specific time among a large number of images of lecture materials is recorded together with time information at a specific time, so that an extended audio file such as a so-called chapter audio file can be recorded. Audio data and image data necessary for creation, and time information about the image data can be recorded easily and in real time during the lecture.

  The multimedia data recording apparatus of the present invention may include that the image data recorded by the data recording unit is still image data obtained by storing an image displayed on the display unit at the specific time. According to the present invention, for example, a screen capture process provided by an OS (Operating System) of the computer or a screen capture software installed in the computer is provided while switching and displaying an image of a lecture material on a lecturer computer. By executing the screen capture process at a desired timing, it is possible to easily record still image data at a specific time and store image data at a specific time.

  The multimedia data recording apparatus of the present invention further includes a detecting unit that detects a signal instructing recording of the image data, and the data recording unit detects the image at the specific time according to the output of the detecting unit. It may include recording data. According to the present invention, for example, while the image of the lecture material is changed or switched and displayed on the instructor computer, the instruction is issued at a timing desired by the user or determined by the instructor computer according to preset conditions and algorithms. The image data at a specific time can be recorded by detecting the recording instruction signal.

  Furthermore, the multimedia data recording apparatus of the present invention may further include an input device connected to the detection means, and may include providing the recording instruction signal to the detection means in response to an operation of the input device by a user. . According to the present invention, for example, a desired image data is recorded by operating an input device such as a keyboard, mouse, or remote controller connected via an infrared interface or Bluetooth or other wireless interface provided in the instructor's computer. Can do.

  Furthermore, the multimedia data recording apparatus of the present invention further includes monitor means for monitoring the state of the image displayed by the display means, and the monitor means receives the recording instruction signal when the image is changed or switched. May be provided to the detection means. According to the present invention, for example, still image data after an image has been changed or switched can be automatically recorded, so that it is possible to record data in a fully automatic manner, saving the user's trouble such as operation of an input device. Is possible.

  A multimedia content creation system according to another aspect of the present invention is a system for creating multimedia content in which one or more image data is incorporated in one audio file, the user terminal and the user terminal Are connected via a communication line, and the user terminal includes a display unit that changes or switches an image over time, a voice input unit that inputs voice, and a voice input unit. And recording the image data at one or more specific times among the images displayed by the display means within the predetermined time together with the time information of the specific time. Including a data recording means, a transfer means for transferring the audio data and the image data at the specific time, The time information is extracted from the image data transferred from the user terminal, the time position detecting means for detecting the time position from the start position of the audio data, and the time position detected for the image data, Content creation means for creating an expanded audio file by adding image data and marker information indicating the time position where the image data is incorporated to the audio data is included.

  According to the present invention, for example, in an instructor computer corresponding to a user terminal, audio data and image data necessary to create an extended audio file such as an audio file with chapters, and time information about the image data, It is easily recorded in real time during the lecture, and the material data and time information are transferred to the server device. In the server device, based on the time position detected for each image data, the image data and marker information indicating the time position in which the image data is incorporated are added to the sound data, and so-called chapter sound files are expanded. Create an audio file. Therefore, for example, when creating an extended audio file related to lecture contents, the operation of the instructor's computer can be limited to the display of lecture materials and the recording and transfer of audio data and image data at a specific time. There is an advantage that investment of software resources and hardware resources necessary for the instructor's computer can be reduced.

  A multimedia content creation system according to still another aspect of the present invention is a system for creating multimedia content in which one or more image data is incorporated in one audio file, the user terminal and the user A server device to which a terminal is connected via a communication line, wherein the user terminal displays a display unit that changes or switches an image over time, a voice input unit that inputs voice, and the voice input unit Is recorded continuously for a predetermined time, and image data at one or more specific times among the images displayed by the display means within the predetermined time is recorded together with time information of the specific time. A data recording means for extracting the time information from the image data and detecting the time position from the start position of the sound data; Creating project data including position detection means, the detected time position information, and information for identifying the corresponding image data; project data creation means; audio data; image data at the specific time; and Transfer means for transferring project data, and the server device, based on the time position included in the project data transferred from the user terminal, the image data and the time position where the image data is incorporated. And content creation means for creating an expanded audio file by adding marker information indicating the above to the audio data.

  According to the present invention, a program that creates an extended audio file such as a chaptered audio file that operates on a server device sets data ("project data") in which the time position and the addition target of information to be added to the audio file are set. If it is necessary to create and input a file including the file, the project data is created by the instructor's computer, and is transferred to the server apparatus together with the audio data and the image data, and the time information about the image data. The server device can execute a program for creating an expanded audio file without having to perform an operation for detecting a time position after receiving data from the instructor computer. Therefore, even when data transfer from a large number of users to the server device is concentrated, there is an advantage that it is possible to reduce the load of the extended sound file creation processing in the server device.

  According to the multimedia content creation system of the present invention, for example, in a lecturer computer used to display lecture materials during a lecture, audio data and image data at a specific time, and time information about the image data are recorded, It is possible to construct a system that transfers to a server device and creates an expanded audio file such as a chapter-attached audio file in the server device. Therefore, the instructor computer can be created even by an educational institution that does not have dedicated software and hardware and a specialized operator.

  The above objects and advantages of the present invention and other objects and advantages will be more clearly understood through the following description of embodiments. However, the embodiments described below are merely examples, and the present invention is not limited to these.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a schematic configuration of a multimedia content creation system according to an embodiment of the present invention. As shown in FIG. 1, the multimedia content creation system includes an instructor computer 10 and a content creation server 20. The instructor computer 10 is connected to the content creation server 20 via a communication line 30 such as the Internet. In the following description, the instructor computer 10 corresponds to a computer that displays lecture materials created using various presentation software via a projector, but this is an example, and the present invention is not limited thereto. It is not limited.
Further, the communication line 30 may be configured to include various communication lines such as a dedicated line, a public telephone line, a satellite communication line, and various servers (not shown), and a specific aspect thereof is not particularly limited. A typical example of the communication line 30 is the Internet, but it may be a LAN (Local Area Network) or another network.

  As will be described later, the content creation server 20 includes a client computer 11, a content distribution server 40, a file server 50, and a mail server 60 that enables transmission / reception of electronic mail on the client computer 12, such as the Internet. It is connected via a line 30 so as to be accessible.

  Next, the configuration of the instructor computer 10 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration of the instructor computer 10. The instructor computer 10 includes a display unit 110, an I / O unit 120, a voice input unit 140, a data recording unit 150, a recording instruction detection unit 160, and an input. A device 170, a transfer unit 180, and a monitor unit 190 are included.

  The instructor computer 10 naturally includes a processor and a storage device connected to a bus (not shown). A storage device such as a hard disk (HDD) or memory (RAM) stores an OS (Operating System) and a program of presentation software to be described later, and processing on the client side in the multimedia content creation system of the present invention. A program to be executed (hereinafter also referred to as “client software”) is stored, and the operation of each unit including the monitor unit 190 is controlled from the display unit 110 by executing these programs.

  The display unit 110, for example, presents a slide image of lecture content or a slide image of lecture content in PDF (Portable Document Format) format created by presentation software such as Microsoft Power Point or Apple Computer Keynote. Under the above operation, it is displayed on a display screen (not shown). The presentation software performs a process of changing or switching a slide image displayed on the display screen in response to an operation input of the input device 170 described later.

  The I / O unit 120 outputs a video signal of the slide image of the lecture content to the projector 130. The projector 130 projects or shows a slide image on a large screen or the like installed in the classroom.

  The voice input unit 140 is a component that performs A / D conversion on the voice of the lecturer picked up by a microphone (not shown) and inputs the voice as digital voice data.

The data recording unit 150 continuously records the digital audio data from the audio input unit 140 for a predetermined time, for example, 40 minutes if it is a scheduled lecture time of 40 minutes, and the audio data 151 as one audio file. The data is stored in a storage device such as a hard disk (HDD) or memory (RAM).
Here, the encoding format of the audio data to be recorded is not particularly limited. For example, there are audio formats such as MPEG-4 AAC (Advanced Audio Coding) and WAV (RIFF waveform Audio Format), and particularly the former AAC audio. The format (extension m4a, m4b or m4p) is a preferable audio file format suitable for reproduction of an extended audio file in an MPEG-4 audio player which is now becoming mainstream.

Further, the data recording unit 150 receives image data of a slide image at a specific time among a large number of slide images displayed on the display unit 110 within the predetermined time according to an output of a recording instruction detection unit 160 described later. Are recorded together with time information at the specific time, and each of the image data 152 (image data 1, image data 2,... In FIG. 2) is stored as an image file with time information in a storage device such as a hard disk (HDD) or the like. Store in memory (RAM).
Here, the compression format of the image data to be recorded is not particularly limited. For example, a JPEG or PNG (Portable Network Graphics) image file format is suitable.

  The recording instruction detection unit 160 is a component that detects a signal instructing recording of image data from the input device 170 or from a monitor unit 190 described later and outputs the signal to the data recording unit 150.

The input device 170 includes a keyboard 171 and a mouse 172. In response to an operation of the keyboard 171 or the mouse 172 by a user (for example, a lecturer or an assistant thereof), a corresponding signal or command is input to the lecturer computer 10.
As shown in FIG. 2, the input device 170 may include a remote controller (RC) 173 that is held and operated by a user (for example, a lecturer or an assistant thereof). In this case, the instructor computer 10 naturally includes an appropriate interface (for example, an infrared port, a wireless interface such as Bluetooth) for receiving a command transmitted from the remote controller 173.

As is well known, various commands can be assigned to operation keys or operation buttons of the keyboard 171, mouse 172, or remote controller 173 in accordance with various processes executed by the OS, presentation software, and client software. .
According to one embodiment of the present invention, main processes such as switching of a lecture slide using presentation software, start and end of recording of audio data using client software, and recording of image data at a specific time are performed by a keyboard. 171, mouse 172, or remote controller 173 can be configured to be executed in response to an operation of an operation key or operation button.

  The transfer unit 180 collects a file of audio data 151 and a file of image data 152 (image data 1, image data 2,...) With time information recorded by the data storage unit 150 and stored in the storage device. In response to a command input by the user, the content is transferred to the content creation server 20 via the communication line 30 such as the Internet.

The monitor unit 190 is a component that constantly monitors the state of an image displayed on the display unit 110 during recording of audio data in the data recording unit 150 described above. The monitor unit 190 is designed to give a signal instructing recording of image data to the recording instruction detecting unit 160 when detecting that the slide image of the lecture content has changed or switched, for example.
Such a monitor unit 190 can be realized by using an appropriate comparison circuit or algorithm for monitoring a change in the video signal output for display on the display screen. An example of detection of a change or switching of a slide image by the monitor unit 190 will be described later.

  Next, the configuration of the content creation server 20 will be described with reference to FIG. 3 is a block diagram showing the configuration of the content creation server 20. The content creation server 20 includes a communication control unit 210, a data storage unit 220, a time position detection unit 230, a project data creation unit 240, a content creation unit 250, A preview editing unit 260 and a transfer unit 270 are included.

  The content creation server 20 naturally includes a processor and a storage device connected to a bus (not shown). In a storage device such as a hard disk (HDD) or memory (RAM), an OS (Operating System) and a program for causing the content creation server 20 to execute server-side processing in the content creation system of the present invention (hereinafter referred to as “server software”). The above-described communication control unit 210 controls the operation of each unit including the transfer unit 270 by executing these programs.

  The communication control unit 210 uses the audio data 151 file transferred by the transfer unit 180 of the instructor computer 10 and the image data 152 (image data 1, image data 2,...) With time information for the instructor. Receive from computer 10. Further, the communication control unit 210 uses the function of the transfer unit 270 described later to transfer the expanded audio file created by the content creation unit 250 described later to the client computer 11, the content distribution server 40, the file server 50, and the mail server. A communication interface is also provided when transferring to any one or more of 60.

  The data storage unit 220 receives data received from the instructor computer 10 via the communication control unit 210, that is, the audio data 151 file and the image data 152 with time information recorded and transferred in the instructor computer 10. A file of (image data 1, image data 2,...) Is stored in a storage device. In addition, the data storage unit 220 stores project data 222 created by a project data creation unit 240 (to be described later) and content 251 created by the content creation unit 250 in a storage device.

  The time position detection unit 230 extracts the recording time of each image (creation time of each image file) from the image data 152 (image data 1, image data 2,...) With time information, which will be described later. The time position from the start position of the audio data is detected by a predetermined calculation process.

  The project data creation unit 240 creates project data 222 (FIG. 8) including time position information for each image detected by the time position detection unit 230 and information for specifying corresponding image data.

  Based on the time position included in the project data 222, the content creation unit 250 corresponds to the corresponding image data 152 (image data 1, image data 2,...) And the time position (hour / min / sec) at which each image data is incorporated. The expanded audio file in which the marker information indicating) is added to the audio data is created as the content 251. Details of processing in the content creation unit 250 will be described later.

The preview editing unit 260 sends audio data, image data (image data 1, image data 2,...), And project to the client computer 11 (FIG. 1) connected to the content creation server 20 via the communication line 30. A browsing service for the preview screen 400 (FIG. 10) for browsing data is provided. The preview editing unit 260 can add, change, and delete image data by an operation through a preview screen on the client computer 11. This preview screen is XML (eXtensible
Markup language (HTML), HyperText Markup Language (HTML), CompactHTML, DHTML (Dynamic HTML), WML (Wireless Markup Language), HDML (Handheld Device Markup Language), etc. . In this case, the client computer 11 naturally includes browser software for browsing such a preview screen.

  The transfer unit 270 transmits the content 251 created by the content creation unit 250 and stored in the data storage unit 220 to any one of the client computer 11, the content distribution server 40, and the file server 50 via the communication control unit 210. Forward to one or more. The transfer unit 270 can also attach the content 251 to an e-mail, transfer it to the mail server 60, and further transfer the content 251 from the mail server 60 to the client computer 12. It is obvious that an appropriate communication protocol is selected in the communication control unit 210 according to the transfer destination designated by the transfer unit 270, and data or file transfer processing of the content 251 is performed, which is obvious to those skilled in the art. Detailed explanation is not given here.

  The operation of the multimedia content creation system configured as described above will be described in the order of the operation of the instructor computer 10 and the operation of the content creation server 20.

  FIG. 4 is a flowchart showing the entire multimedia data recording process executed in the instructor computer 10.

  First, client software in the multimedia content creation system of the present invention is activated on the instructor computer 10 (step S101). Further, the presentation software is activated in the instructor computer 10 (step S102). The slide image file of the lecture content created in advance by the user and stored in the portable storage medium is read by appropriate presentation software corresponding to the file, and the slide image of the lecture content is displayed on the display screen by the display unit 110. In addition, the image is projected or transmitted by the projector 130 via the I / O unit 120.

In response to an operation of the input device 170, for example, a predetermined key operation on the keyboard 171, the data recording unit 150 starts recording (recording) of audio data, and at the same time, the first slide displayed on the display screen by the display unit 110. Image still image data (image data 1) is stored in a memory (not shown). This is performed by executing the screen capture process provided by the OS of the instructor computer 10 or the capture process provided by the screen capture software installed in the instructor computer 10 simultaneously with the start of recording (step S103). Usually, the image data file saved by the screen capture process includes information (date / month / day and hour / minute / second information) indicating the time when the image data was created. Therefore, the file of the first image data (image data 1) generated as a result of the screen capture process being executed simultaneously with the start of recording contains time information equal to the time when recording was started.
The data recording unit 150 stores the first image data (image data 1) file with time information in a hard disk (not shown) (step S104).

  When a presentation using the lecturer computer 10 progresses and the user operates the input device 170 to switch the slide image of the lecture content to the next image, the image data is changed according to the image switching operation through the input device 170. A signal for instructing recording, that is, a signal for instructing screen capture (screen capture instruction signal) is supplied to the recording instruction detection unit 160.

  The screen capture instruction signal may be given to the recording instruction detection unit 160 by the monitor unit 190 that monitors the state of the image displayed on the display screen. That is, the screen capture instruction signal may be given to the recording instruction detection unit 160 by the monitor unit 190 detecting a change or switching of the image. As already described, the monitor unit 190 can be realized by using an appropriate comparison circuit or algorithm for monitoring changes in the video signal output for display on the display screen. Therefore, the screen capture instruction signal is both a specific electric signal generated by a specific circuit and a specific instruction generated by executing a monitoring process according to a specific algorithm of the client software. Of course.

  FIG. 5 is a diagram showing how the slide image is switched or changed. For example, when the monitor unit 190 detects that the image has been switched from FIG. 5A to FIG. 5B, a screen is displayed to record the image data of the image shown in FIG. A capture instruction signal is provided from the monitor unit 190 to the recording instruction detection unit 160. Further, when the image changes due to an animation or the like, the image shown in (c) of FIG. 5 also has an image (a state shown in (d) of FIG. 5) after a certain period of time has elapsed since the end of the change. When no change is seen, a screen capture instruction signal is given to record the image data of the image shown in FIG. It should be noted that a person skilled in the art determines how much the image has changed or what time has elapsed since the end of the image change, and whether the screen capture instruction signal is given from the monitor unit 190 to the recording instruction detection unit 160. This is an item that can be set arbitrarily.

  Referring again to FIG. 4, the recording instruction detection unit 160 detects the presence / absence of a screen capture instruction signal (step S105). When a screen capture instruction signal is detected (step S105: Yes), a screen capture process is performed in which still image data (image data 2) of the slide image displayed on the display screen by the display unit 110 is stored in a memory (not shown). Execute (Step S106). Further, the data recording unit 150 stores a file of image data (image data 2) with time information in a hard disk (not shown) (step S107).

  After the data recording unit 150 stores a file of image data with time information (image data 2), or when the recording instruction detection unit 160 does not detect a screen capture instruction signal (step S105: No), recording is not performed. It is determined whether or not to end (step S108). Specifically, when the user operates the input device 170 at the time when the lecture is finished and the recording is instructed (step S108: Yes), the lecturer's voice recording is finished, and the processing after the next step S110 is performed. Migrate to If the end of recording is not instructed (step S108: No), the process returns to step S105.

  Hereinafter, among the slide images of the lecture contents that are sequentially switched and displayed on the display screen according to the progress of the presentation using the instructor computer 10, the image (or the monitor unit 190) selected by the user by operating the input device 170. The image data of the image automatically detected or changed after the image data is saved as image data 3, image data 4,..., Image data n by screen capture processing, and a file of image data with time information. (Step S105 to step S109).

  When the end of recording is instructed (step S108: Yes) and the instructor's voice recording ends, the data recording unit 150 stores a file of the voice data in the storage device.

  With the above processing, recording of the audio data file and recording of the image data file with specific time information in the instructor computer 10 are basically completed.

  In the multimedia content creation system in this embodiment, the data recording process in the instructor computer 10 particularly includes the transfer process in step S111. That is, in step S111, the transfer unit 180 transfers the audio data file stored in the storage device and the image data file with specific time information to the content creation server 20 as a group.

Next, the operation of the content creation server 20 will be described with reference to FIG.
FIG. 6 is a flowchart showing the entire multimedia content creation process executed in the content creation server 20.

  First, the communication control unit 210 in the content creation server 20 includes a file of audio data 151 transferred by the transfer unit 180 of the instructor computer 10 and image data 152 with time information (image data 1, image data 2,... ) Is received (step S201). The data storage unit 220 stores these received data 221 files in a storage device.

  Next, the time position detection unit 230 extracts the recording time of each image (image file creation time) from the received file of image data 152 (image data 1, image data 2,...) With time information. (Step S202). This extraction process can be easily performed by reading the creation time from the index information of the image data.

  Subsequently, the time position detection unit 230 detects a relative time based on the creation time of the image data 1 based on the creation time of each image (step S203). As described above, in the instructor computer 10, the image data 1 is created simultaneously with the start of recording (recording of audio data), so the process of detecting relative time detects the time position from the start position of the audio data. It corresponds to doing.

  FIG. 7 shows an example in which the relative time based on the creation time of the image data 1 is obtained based on the creation time of the image data 1, the image data 2,.

  Referring to FIG. 6 again, the project data creation unit 240 creates project data 222 including time position information for each image detected by the time position detection unit 230 and information for specifying corresponding image data. (Step S204).

FIG. 8 is an explanatory diagram showing a main part of information set in the project data 222.
For example, when creating an audio file with a chapter based on the MPEG-4 AAC audio format that can be played back by an Apple computer audio player widely known under the product name “iPod”, the project data 222 is preferably Is the chapter time (time position from the start position of the audio) described in the format “<chapter startitime =” time ”> and the image described in the format“ <picture> image file name </ picture> ” It is an XML (eXtensible Markup Language) file including information for specifying data. In the example of FIG. 8, the image data 1 (file name “image 1.jpg”), image data 2 (file name “image 2.jpg”), image data 3 (file name “image 3” shown in FIG. .jpg "), ..., the relative time detected for each piece of image data is set as the chapter time.

  The project data 222 is created as a setting file for the process executed in the next step S205. On the other hand, the project data 222 provides a file base that can be changed in the client computer 11 in the preview editing process described later. . The created project data is stored in the storage device by the data storage unit 220.

  In step S205, the content creation unit 250 indicates the corresponding image data 152 (image data 1, image data 2,...) And the time position where each image data is incorporated, based on the time position included in the project data 222. An expanded audio file in which the marker information is added to the audio data is created as the content 251.

FIG. 9 is a flowchart showing the processing contents executed in the content creation unit 250. In step S2051, the relative time (time information) for each image data is read from the project data 222. In the following step S2052, based on the relative time for each read image data, each image data and marker information (marker information corresponds to a chapter in a chaptered audio file) is added to the audio data file. Desired content (for example, an audio file with a chapter such as “Enhanced Podcast”) is created. The created content is stored in the storage device by the data storage unit 220.
With the above processing, content creation in the content creation server 20 is completed.

  In addition to providing a content creation service to the instructor computer 10 as described above, the content creation server 20 according to the present embodiment further includes a preview editing unit 260, thereby creating content via the communication line 30. A browsing service for a preview screen 400 of a desired content is provided to the client computer 11 connected to the server 20. That is, as can be seen from the example shown in FIG. 10, the preview screen 400 includes a file name of audio data, an audio file playback button, a title for each marker information (corresponding to a chapter), a time position, and image data incorporated with the marker. This is a page screen including information such as a file name and a preview image of an image corresponding to each marker information. This preview screen 400 corresponds to project data to which a preview image of image data is added, and the user accesses the content creation server 20 using the client computer 11 and browses the preview screen 400 to arbitrarily view the preview data 400. There is an advantage that the finished audio file with chapters can be confirmed from a remote location.

  In addition, the content creation server 20 includes the preview editing unit 260, so that the project data created by the project data creation unit 240 for the user using the client computer 11 and stored in the storage device by the data storage unit 220 is stored. 222 is given permission to edit (add, change, delete) afterwards. As a result, various modifications are made to the XML source of the project data 222 while browsing the preview screen 400, for example, a change in the position where an image is incorporated by appropriately adjusting the time position of a certain marker, It is possible to delete all information including image data attached thereto.

  Further, as already described, the transfer unit 270 transfers the content 251 stored in the data storage unit 220 to the client computer 11, the content distribution server 40, the file server 50, and the mail server via the communication control unit 210. The content 251 may be attached to an e-mail, transferred to the mail server 60, and further transferred from the mail server 60 to another client computer 12. . As a result, there is an advantage that, for example, when the audio files with chapters that can be reproduced with lecture images are distributed widely to students, the workload and investment on the distributor side can be greatly reduced. .

  In the above-described embodiment, the content creation server 20 includes the time position detection unit 230 and the project data creation unit 240. However, these components are changed to be included in the instructor computer 10 instead of the content creation server 20. You may do it. That is, project data 222 including time position information for each image detected by the time position detection unit 230 and information for specifying corresponding image data is created on the instructor computer 10 side, and the project data 222 is You may make it transfer to the content creation server 20 with the file of the audio | voice data 151 recorded with the computer 10 for lecturers, and the file of the image data 152 (image data 1, image data 2, ...) in specific time.

  As a specific example of creating an extended audio file in the multimedia content creation system of the above-described embodiment, a so-called chaptered audio file based on the MPEG-4 AAC audio format (for example, “Enhanced Podcast”) However, the present invention is not limited to this, and the present invention is not limited to this, and includes a plurality of extended audio files that can include image data, such as “Flash”. Of course, it is possible to correspond to these formats.

  The instructor's computer 10 may be a method of setting up client software for executing client-side processing in the multimedia content creation system of the present invention by installing the software in a computer and making the software resident. Alternatively, the software may be provided from the content creation server 20 or another server through the communication line 30 and the software stays in the memory only when it is necessary for processing by the instructor computer 10. It may be provided on the creation server 20 or another application server, and by using it through the communication line 30, the software is not allowed to stay in the instructor computer 10, and may be provided by an ASP (Application Service Provider), Also al, may be a combination of two or more of these methods is a method using the client software.

  In the above embodiment, the instructor computer 10 stores and records image data at a specific time by screen capture processing, but includes a lecturer's figure and notes written by the instructor. An imaging device such as a video camera that captures projections or projected images may be connected to the instructor computer 10 to store and record image data captured by the imaging device at a specific time. A method of storing and recording image data at a specific time in combination may be used.

  Furthermore, in the above embodiment, an example in which the instructor computer 10 is a personal computer (PC) has been described. However, the instructor computer 10 is not limited to the PC, and other devices such as a mobile phone with a camera, a recording function, etc. In the digital camera having the data transfer function or the digital video camera having the data transfer function, the multimedia data recording process according to the present invention may be executed, and other components of the present invention are also described in the present invention. Needless to say, modifications, substitutions or changes can be made as appropriate within the scope of the above.

  Since the present invention can be widely used as a multimedia data creation tool, for example, not only educational institutions such as universities, vocational schools, and prep schools, but also various lectures and seminars, training of companies and public offices, broadcasting media, etc. It can be used in various places and uses, such as content reuse, event / event recording, etc.

It is a figure which shows schematic structure of the multimedia content production system in embodiment of this invention. It is a block diagram which shows the structure of the computer 10 for lecturers shown in FIG. It is a block diagram which shows the structure of the content creation server 20 shown in FIG. 4 is a flowchart showing the entire data recording process executed by the instructor computer 10. It is a figure which shows the mode of the change or change of the slide image displayed on the display screen of the computer 10 for lecturers, (a) And (b) shows the example in which a slide image is switched, (c) And (d) An example in which the slide image does not change for a certain time is shown. 4 is a flowchart illustrating an entire content creation process executed by the content creation server 20. It is a chart explaining the detection of the relative time about image data. It is explanatory drawing which shows the principal part of the information set to project data. It is a flowchart which shows the detail of the production process of the extended audio | voice file among the content production processes in the content production server. It is a figure explaining the preview function and edit function which are provided by the content creation server.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Instructor computer 11, 12 Client computer 20 Content creation server 30 Communication line 40 Content distribution server 50 File server 60 Mail server 110 Display unit 120 I / O unit 130 Projector 140 Audio input unit 150 Data recording unit 151 Audio data 152 Image data 160 recording instruction detection unit 170 input device 180 transfer unit 190 monitor unit 210 communication control unit 220 data storage unit 221 received data 222 project data 230 time position detection unit 240 project data creation unit 250 content creation unit 251 content 260 preview editing unit 270 transfer 400 Preview screen

Claims (18)

Display means for changing or switching images over time; and
Voice input means for inputting voice;
The sound data from the sound input means is continuously recorded for a predetermined time, and the image data at one or more specific times among the images displayed by the display means within the predetermined time is the time of the specific time. Data recording means for recording together with information;
A multimedia data recording apparatus comprising:   2. The multimedia data recording apparatus according to claim 1, wherein the image data recorded by the data recording unit is still image data in which an image displayed on the display unit is stored at the specific time. And further comprising detection means for detecting a signal instructing recording of the image data,
The multimedia data recording apparatus according to claim 1, wherein the data recording unit records image data at the specific time according to an output of the detection unit. An input device connected to the detection means;
4. The multimedia data recording apparatus according to claim 3, wherein the recording instruction signal is provided to the detection means in accordance with an operation of the input device by a user. And further comprising monitor means for monitoring the state of the image displayed by the display means,
4. The multimedia data recording apparatus according to claim 3, wherein the monitor means gives the recording instruction signal to the detection means when the image is changed or switched. A method executed by a computer comprising display means for changing or switching images over time and voice input means for inputting voice,
Recording audio data from the audio input means continuously for a predetermined time;
Recording the image data at one or more specific times among the images displayed by the display means within the predetermined time together with the time information of the specific time;
A method for recording multimedia data, comprising:   7. The multimedia data recording method according to claim 6, wherein the image data recording step records still image data in which an image displayed on the display means is stored at the specific time. The image data recording step includes
Detecting a signal instructing recording of the image data;
8. The multimedia data recording method according to claim 6, wherein the image data at the specific time is recorded in accordance with the recording instruction signal detected in the detecting step.   9. The multimedia data recording method according to claim 8, wherein the detecting step detects the recording instruction signal given in accordance with an operation of an input device by a user. The image data recording step includes
Further comprising the step of monitoring the state of the image displayed by the display means;
9. The multimedia data recording method according to claim 8, wherein the detecting step detects the recording instruction signal given in accordance with a change or switching of the image in the monitoring step. In a computer comprising display means for changing or switching images over time and voice input means for inputting voice,
Processing for continuously recording audio data from the audio input means for a predetermined time;
A process of recording image data at one or more specific times among images displayed by the display means within the predetermined time together with time information at the specific times;
A multimedia data recording program including instructions for executing the program.   12. The multimedia data recording program according to claim 11, wherein the image data recording process records still image data in which an image displayed on the display means is stored at the specific time. The image data recording process includes:
A process for detecting a signal instructing recording of the image data;
13. The multimedia data recording program according to claim 11, wherein the image data at the specific time is recorded in accordance with the recording instruction signal detected in the detection process.   14. The multimedia data recording program according to claim 13, wherein the detection processing detects the recording instruction signal given in accordance with an operation of an input device by a user. The image data recording process includes:
A process of monitoring the state of the image displayed by the display means;
14. The multimedia data recording program according to claim 13, wherein the detection processing detects the recording instruction signal given in accordance with a change or switching of the image in the monitoring step. A system for creating multimedia content in which one or more image data is incorporated in one audio file,
Including a user terminal and a server device to which the user terminal is connected via a communication line;
The user terminal is
Display means for changing or switching images over time; and
Voice input means for inputting voice;
The sound data from the sound input means is continuously recorded for a predetermined time, and the image data at one or more specific times among the images displayed by the display means within the predetermined time is the time of the specific time. Data recording means for recording together with information;
Transfer means for transferring the audio data and the image data at the specific time,
The server device
Time position detection means for extracting the time information from the image data transferred from the user terminal and detecting a time position from a start position of the audio data;
Content for creating an extended audio file in which the image data and marker information indicating the time position in which the image data is incorporated are added to the audio data based on the time position detected for the image data Creating means,
A multimedia content creation system characterized by A system for creating multimedia content in which one or more image data is incorporated in one audio file,
Including a user terminal and a server device to which the user terminal is connected via a communication line;
The user terminal is
Display means for changing or switching images over time; and
Voice input means for inputting voice;
The sound data from the sound input means is continuously recorded for a predetermined time, and the image data at one or more specific times among the images displayed by the display means within the predetermined time is the time of the specific time. Data recording means for recording together with information;
Time position detecting means for extracting the time information from the image data and detecting a time position from a start position of the audio data;
Project data creating means for creating project data including the detected time position information and information for identifying the corresponding image data;
Transfer means for transferring the audio data, the image data at the specific time, and the project data;
The server device
Based on the time position included in the project data transferred from the user terminal, the image data and marker information indicating the time position in which the image data is incorporated are added to the audio data. Including content creation means for creating a sound file,
A multimedia content creation system characterized by A method for creating multimedia content in which one or more image data is incorporated in one audio file,
Changing or switching the image over time and displaying on the display means;
Inputting voice to the voice input means;
The sound data from the sound input means is continuously recorded for a predetermined time, and the image data at one or more specific times among the images displayed by the display means within the predetermined time is the time of the specific time. Recording with information,
Extracting the time information from the image data and detecting a time position from a start position of the audio data;
Creating an extended audio file in which the image data and marker information indicating the time position in which the image data is incorporated are added to the audio data based on the time position detected for the image data; When,
A method of creating multimedia content, comprising:

Images