JPH04179997A - Method and device for generating multi-media software - Google Patents

Abstract

PURPOSE:To easily make modification, etc., by processing and editing respective medium materials into data format used for software according to timing data extracted according to a sound and an image after layout by temporary editing. CONSTITUTION:Even when a medium material is not converted into a video signal, the image is converted into a video signal and recorded together with a time code and the sound is recorded together with the time code. While those recorded image and sound are reproduced, a sound and an image which are necessary are extracted and put in layout (edited temporarily) on the time base of time code. The timing data is extracted according to the sound and image in the layout and respective medium materials are processed and edited into the data format used for the software. Consequently, the software, etc. can easily be modified.

Description

[Detailed description of the invention] <Industrial application field> The great invention is CD-1 software (title), CD-ROM
A method for creating multimedia software such as software, CD-ROMXA/software, etc., and an apparatus for creating the multimedia software.

<Prior art> Multimedia software is CD-L CD-R
A multi-purpose computer that can perform sophisticated information transmission by handling character code data, image data, sound data, etc. in different expression formats in a unified manner using large-capacity storage media such as OM and the computing power of computers. This software is capable of media output.

For example, a CD-ROMXA disc is manufactured through a process as shown in FIG.

First, a plan is made as to what kind of disc to produce (5901). The production concept is decided.

Then, a scenario is created by the designer based on the plan (S902). In this case, in creating multimedia software, the scenario, which is the blueprint, is created by the designer by hand on paper. .

This multimedia software uses a computer program (playback program) to control the display and changes of images and characters in precise timing with the progression of sound. However, it is difficult even for experienced designers to accurately design the display timing and various visual effects in advance.

Therefore, in reality, after a conceptual scenario is written on paper (S902), it goes through a material collection process (S903), and then computer engineers make appropriate judgments and process the data for each material (S904). , and develops its program (5905).

Furthermore, a correction feedback loop is repeated in which the player reproduces the result, looks at it, and gives instructions for correction again.

In this data processing (S904), the analog material of the sound data is converted into digital data, and data compression processing is performed (5906). The image data is obtained by digitally converting analog materials such as photographs (5907), and then performing data compression processing. Furthermore, after the layout editing step (3908), these image data are interleaved with the sound data (5909).

Further, a document as a material is subjected to, for example, CTS (Computer Type Setting System) processing (S910) and database creation (S911).

Furthermore, after changing the program, compile it and
It is a linkage.

Then, each data processed and edited in this way is formatted according to the media to be loaded (S912), and pre-mastered (5913).
The data is then stored on the medium, such as a CD-ROMXA disk.

Socite 1. :(7) CD-ROMXA disk is CD-
It is driven by a ROM playback device, and its stored data is played back and displayed on a monitor or speaker based on a program.

<Problems to be solved by the invention> In such conventional multimedia software creation methods, the material processing (data processing) process is complicated and very long, so even a slight change in a part of the software requires correction work. The problem was that it took a long time. If it took too much time for the corrective feedback loop to converge and the software to be created in this way, the cost would be high, and there would also be problems with the content of the work being impossible to produce.

So this i! ! The following methods can be considered to solve this problem.

One is the shortening of the material processing process mentioned above. For example, simultaneous processing of sound digitization processing and ADPCM compression processing is possible.

Or, it is a simplification of the editing process. In particular, CD-ROM
In data editing on XA discs and CD-January discs, advanced multimedia expressions are possible by appropriately interleaving image data and compressed sound data. However, since this interleaving process takes time, an attempt is made to shorten the feedback loop by creating a virtual interleaved situation without actually performing the interleaving process.

However, even with these methods, the following drawbacks still occur.

In other words, even if the above improvements are made, after making the computer playable (5913 above), you will be able to view the playback results and select audio materials and image (pictures, text, etc.) materials. There are many cases in the feedback loop where you want to replace some parts. In this case, it is necessary to digitize and compress the material (data processing) each time. With conventional creation methods, it is not possible to make changes on the spot and display or generate the results right in front of the designer's eyes.

Furthermore, simply changing the display timing or the way pictures are displayed requires modification of the program at the same time.

Therefore, the present invention allows designers and others to make corrections to the hour wheel by temporarily editing the video image without processing the material.1. The object of the present invention is to provide a multimedia software creation method and a multimedia software creation device that can be created accurately in a short time and can further reduce costs.

<Means for solving WH> The method for creating multimedia software according to the present invention is a method for creating multimedia software that is created by processing and editing a plurality of media materials including sounds and images into a format that can be used by a computer. A method for creating software, which includes the steps of recording an image as a video signal along with a time code, recording sound along with a time code, and reproducing the recorded image and sound while creating necessary sounds and images. a temporary editing step of extracting and laying out on the time axis of the time code; a step of extracting timing data based on the sound and image laid out by the temporary editing; and a step of extracting timing data based on the extracted timing data. This is a method for creating multimedia software including a processing/editing step of processing/editing each media material into a data format used by the software.

Further, the multimedia software creation device according to the present invention is a multimedia software creation device that is created by processing and editing a plurality of media materials including sounds and images into a format that can be used on a computer. , has an image input means for inputting an image as a video signal together with a time code, a sound output means for inputting sound together with a time code, and a reproduction function for the input sound and image, and has the necessary sound and image. a temporary editing means for extracting and laying out timing data on the time axis of the time code; a timing data extraction means for extracting timing data from the temporary editing means; and a timing data extraction means for extracting timing data from the temporary editing means; This is a multimedia software creation device comprising processing/editing means for converting data into a data format used in the software.

Further, the multimedia software creation device according to the present invention processes and edits a plurality of media materials including sounds and images into a format that can be used by a computer, and arranges each media material in an interleaved manner to create sounds and images. A multimedia software creation device capable of simultaneously displaying images, comprising: image input means for inputting an image as a video signal together with a time code; sound input means for inputting sound together with the time code; Temporary editing means that has an image playback function and extracts necessary sounds and images and lays them out on the time axis of the time code; a timing data extraction means that extracts timing data from the temporary editing means; processing/editing means for converting the material data into a data format used by the software based on the timing data; and interleaving means for interleaving the processed/edited data based on the timing data. This is a multimedia software creation device comprising:

<Operation> In the multimedia software creation method according to the present invention, even if the media material is not converted into a video signal, the image is converted to a video signal and recorded together with the time code, and the sound is recorded together with the time code. While playing these recorded images and sounds, necessary sounds and images are extracted and laid out on the time axis of the time code (temporary editing). Then, timing data is extracted based on the laid out sounds and images, and based on this timing data, each media material is processed and edited into a data format used by the software.

Therefore, a strict scenario in the conventional process is no longer necessary, and necessary information can be defined while performing simulation in the temporary editing process.

Further, in the multimedia software creation device according to the present invention, sound and images are once recorded and then laid out on the time axis of the time code while being played back.

Timing data is then extracted, and the media material is processed and edited based on this timing data.

Furthermore, the multimedia software creation device according to the present invention interleaves data that has been processed and edited based on timing data.

<Example> Embodiments of the present invention will be described below with reference to the drawings.

FIGS. 1 to 8 are diagrams for explaining an embodiment of the method for creating multimedia software according to the present invention.

As shown in Figure 1, the method for creating multimedia software involves a rough scenario creation step (5 steps) after the planning step.
101), material collection process (S102), temporary editing process (
5103), material processing step (5104), etc.

The rough scenario creation step (5101) is a step of creating a scenario in which sounds and images change over time. In this case, the scenario does not have to be exact, and may be one that develops a general story that represents a concept.

The material collection step (5102) is to collect a plurality of media data such as sound data and image data based on the above scenario. For example, as shown in FIG.
Collect PCM sound sources (magnetic tapes), videotapes, color photographs (positive film), typesetting letters, drawings, and character data (documents, CTS tapes), etc. according to the theme of the scenario. These are taken in by a scanner or the like depending on the material.

In the temporary editing step (S103), a temporary editing device that can output and edit over time the materials related to these images using video images and the materials between sounds using audio synchronized with the video images is used. This method simulates the layout of these materials on the time axis.

Further, in the material processing step (S104), the material temporarily edited in this temporary editing step is processed in accordance with the media in which the software is used. For example, C
When producing a D-ROMXA disk, audio data is encoded as ADPCM, and video tapes, color photographs, etc. are encoded as CLUT data.

FIG. 2 is a diagram illustrating the concept of a temporary editing device used in the above temporary editing step (5103).

As shown in this figure, pictures, films, and the like are recorded as image data using video signals by a camera 21, and input to and stored in a control device 22 comprised of a computer.

In addition, the characters are recorded as image data using a video signal by the video captioner 23, and this computer 2
2 and stored.

Further, video tapes on which other moving pictures and the like are recorded are inputted to the computer 22 along with time codes by a video tape recorder 24.

As for sound (magnetic tape), a time code and sound data are input to the computer 22 by an audio recorder 25.

The computer 22 includes, for example, 8 MB of RAM, a video frame buffer, a video digitizer, an audio digitizer, a 1.2 GB hard disk, and the like.

Therefore, each media data input to the computer 22 can be selected and extracted by the computer 22, image data can be displayed on the monitors 26 and 27, and sound data can be played back by a speaker (not shown). becomes.

These video data and sound data are then edited while being played back and output in real time.

For example, extracting the necessary parts from each video and sound data,
Layout editing is performed based on time codes (automatically managed by the computer 22), and effects are added using a video effects device 28 or the like. Therefore, a designer or the like can use this temporary editing system based on the rough scenario, refer to the monitor display and audio output, and perform temporary editing in real time while easily simulating.

In this temporary editing system, timing data from the data thus temporarily edited and created is output as a floppy disk 30 via the video effect device operation monitor means 29.

Here, timing data specifies which part of which type of material is to be used, and the specified material is
This indicates when it will be used.

Furthermore, in this temporary editing system, at the same time as the timing data 30 is created, a proofreading video code 132 that is temporarily edited using the video recording device 31 can be created.

Note that this temporary editing system does not perform data processing (data compression, formatting, etc.) for CD-■ media, CD-ROM media, etc.

FIG. 3 is a conceptual diagram showing the process of extracting each piece of information after completion of temporary video editing.

Based on the timing data 30, the material list creation means 331 extracts the following editing list. For example, a floppy disk 34 stores a list showing display materials such as pictures and films, a floppy disk 35 stores a list showing text materials to be displayed, and a floppy disk 36 stores a list showing video frames of video images to be extracted. Create.

Furthermore, for the sound, a floppy disk 37 is created in which a list (specified by time code) indicating sound materials to be extracted is stored.

Furthermore, interleap parameter creation means 332 creates a floppy disk 38 storing parameters for interleaving.

FIG. 4 is a conceptual diagram showing the steps of material processing.

In this material processing process, using the list of each material above, for materials such as video tapes and audio magnetic tapes, only the parts that are actually used are extracted based on the time code.
Performs data digitization and data compression. For the film, the film number to be actually used is selected from the time code given during temporary editing, and the data is digitized, data compressed, etc.

For example, picture 41 is extracted by list 34 and scanned by scanner 4.
After the signal is digitized in step 2, it is compressed into a DYUV signal and held on the magnetic disk 45. Reference numeral 43 indicates a magnetic disk that holds digitized data, and 44 indicates a step of compressing the data on this magnetic disk 43 into a DYUV signal.

Further, the sound tape 46 is extracted according to the above lists 36 and 37, and then digitized 47 to produce a tape 48 using a PCM sound source or the like, and further compressed into an ADPCM signal in step 49. 50 is a magnetic disk that stores this compressed signal.

For characters 51 written on a document or the like, character pages are extracted from the list 35, digitized by a word processor 52 or CTS, and stored on a floppy disk 53. When this disc 530 character data is treated as character data, a B&K disc 55 is created for CD-I or CD-ROM through steps 54 such as code conversion and font extraction.

If the character data on the disk 530 is treated as a picture (image data), it is stored on the magnetic disk 58 in a digitizing and CLUT forming step 57 after phototypesetting and a telop camera photographing step 56 .

The video frames 59 extracted from the video tape or the like by the list 36 are stored in a frame buffer 60 of the computer.
After being stored in the magnetic disk 61, the DY
It is compressed 62 into UV data and held on a magnetic disk 63.

! FIG. 5 conceptually shows the data editing process. Each digital data processed in the material processing process is read from each magnetic disk 45, 50, 55 (or 58), 63, and the interleave parameter 38 is used. Then, these data are processed in an automatic interleap processing step 65 and a file editing processing step 66, respectively.

This automatic interleap processing is performed as follows. First, the file name of the real-time file on the CD-1 disc is determined in advance. The time data is the SAMPTE time code (30
1 frame/sec) to the number of sectors of the CD-I disk (75 sectors/sec). Then, coding information is automatically generated. When processing the material, add the following information to the material data. If it is image data, the image compression format and type (for example, DYUV, CLUT
8, etc.), and if it is sound data, it is the sound compression format, type R (for example, B level stereo, etc.). Furthermore, once the compression level of a sound is determined, its layout pitch is uniquely determined. This is because it is necessary to output the sound without interruption; for example, in B-level stereo, sound data is stored in one sector out of every four sectors.

Further, the image is divided into empty sectors between the sound sectors, laid out and stored. In this case, all images must be laid out before they are displayed as images. If this layout is not possible, an error will occur. Furthermore, a mark indicating the end of the file shall be attached to the last divided sector.

Furthermore, in the temporary editing step before the automatic interleaving step, it is also possible to check whether or not the image data can be completely read from the disk before the designated image display time.

First, the timing of image display based on timing parameters, for example, the time from image A to image B, is calculated using the SAMPTE time code, and this calculated value is
Convert to the number of sectors on the D-1 disk. It is calculated whether this converted value can accommodate all eight image data. For example, if 8 image data is 300 bytes,
(300×1024)÷2324=133 sectors are calculated (one sector consists of 2324 bytes). During this time, a sector is required to store audio, and in the B-level stereo, 25% of the sectors are used for audio. 133X1.25=
There are 167 sectors. Furthermore, 167÷75=2.23 seconds.

Therefore, the sound level and image type are input in advance, and the above interval (time between A image display and B image display) is set.
If the time is less than 2.23 seconds, a warning message will be issued,
Warn the operator that CD-I or CD-ROMXA media cannot be created with this software. As a result, the scenario can be made stricter on the spot in the temporary editing process before interleap processing is performed, and the work efficiency can be significantly improved compared to the case where this check is not performed.

When these data are interleaved and edited as described above, a pre-mastering processing step 67 further performs processing to add error correction data necessary for CD-I, CD-ROM, etc., and then a disc creation step. Test disc 69 for CD-I or CD-ROM in 68
is created.

FIG. 6 shows the process of generating playback parameters that provide the display timing of the interleaved data to a multimedia playback program to be operated on a playback device for CD-I or CD-ROM.

The playback parameter generation means 71 generates a time code in units of video frames (for example, a SAMPTE time code of 30 frames/second) obtained from the timing data 30.
It includes a means for converting the time code into a different time code (e.g., 75 sectors per second trigger count) used on CD-1 or CD-ROM, etc., and converts this time code and image display method (visual effects, etc.) A playback parameter consisting of a list is generated and the playback parameter is stored on the disk 72.

In this way, the playback parameters are generated and held separately from the multimedia playback program, and define the processing sequence (image display timing, display method, etc.) of the multimedia playback program.

As a result, as shown in FIG. 7, in the playback test of the test disc 69, a multimedia playback program 74 and playback parameters 72 including the display method and display timing are loaded into the CD-I player 73 or the like. Then, a playback procedure is created, and then the test disk 69 is controlled to perform a final playback test.

In this case, the video media simulated by the temporary editing device and the CD-ROM media and CD-1 media have different data and hardware specifications, so the playback and
There may be some timing errors in the display. At this time, it is possible to adjust the timing of the hour wheel simply by comparing the time difference with the calibration video tape 32, counting this time difference as the number of frames on the video frame, and modifying the playback parameters 72 using an editor.

FIG. 8 is a conceptual diagram showing the main editing process of data.

If the timing is right in the above test process, CD-I, C
Manufacture a mask disk for the D-ROM disk.

This is final edited using the corrected playback parameters 81, test disk 69, and multimedia playback program 74.

Then, premastering is performed again, and finally, for example, 3
/4 inch U-matic tape in PCM1630 format and pressed into a disc 82 at a mastering factory.

<Effects> As explained above, according to the present invention, it is possible to temporarily edit materials before processing them, so it is possible to create accurate scenarios that could not be expressed with conventional paper scenarios. It becomes possible to give instructions regarding data processing at the right time.

As a result, the quality (content) of the software work can be improved.

Furthermore, when changing a part of the scenario during editing, there is no need to spend a long time processing the material each time, and it is possible to easily modify the software as a whole. Therefore, it is possible to achieve a significant cost reduction in software creation.

Furthermore, the designer who created the scenario can perform the vertical collection of materials, thereby improving the quality of the work.

[Brief explanation of drawings]

FIG. 1 is a flowchart showing the creation procedure of a method for creating multimedia software according to an embodiment of the present invention, FIG. 2 is a conceptual diagram for explaining a temporary editing device according to an embodiment, and FIG. FIG. 4 is a conceptual diagram showing a material list and interleap parameter creation process according to one embodiment. FIG. 4 is a conceptual diagram showing a material processing process according to one embodiment.
The figures are -> A conceptual diagram for explaining the process from data editing to test disc creation according to the embodiment, FIG. 6 is a conceptual diagram showing the playback parameter generation process according to one embodiment, and FIG. 7 is a conceptual diagram for explaining the reproduction parameter generation process according to one embodiment. FIG. 8 is a conceptual diagram for explaining the playback test process, FIG. 8 is a conceptual diagram for explaining the data book collection process according to one embodiment, and FIG. 9 is a process showing the conventional CD-ROMXA title production process. It is a diagram. 5101 Rough scenario creation process, 510
2...Φ・Material collection process, 5103...
- Temporary editing process, 5104...Material processing process. Patent Applicant Toppan Printing Co., Ltd. Agent Patent Attorney Kiyoshi Kuwai-Jil Figure $101 Figure 3 S4 Figure 6 Figure 7

Claims (3)

[Claims] (1) A method for creating multimedia software that is created by processing and editing multiple media materials, including sound and images, into a format that can be used on a computer, and recording the images as video signals along with time codes. a step of recording the sound together with the time code; a temporary editing step of extracting the necessary sound and images and laying them out on the time axis of the time code while playing back the recorded images and sounds; A process of extracting timing data based on the sounds and images laid out by the temporary editing, and a process of processing and editing each media material into a data format used by the software based on the extracted timing data.・
A method for creating multimedia software, comprising: an editing process. (2) A device for creating multimedia software created by processing and editing multiple media materials including sound and images into a format that can be used on a computer, and inputting images as video signals along with time codes. It has an image input means, a sound input means for inputting sound together with a time code, and a playback function for the input sound and image, and extracts the necessary sounds and images and lays them out on the time axis of the time code. Temporary editing means; Timing data extraction means for extracting timing data from the temporary editing means; Processing/editing means for converting each media material into a data format used by the software based on the extracted timing data. A multimedia software creation device comprising: and. (3) Creation of multimedia software that can display sound and images simultaneously by processing and editing multiple media materials including sound and images into a format that can be used on a computer, and interleaving each media material. The device has an image input means for inputting an image as a video signal together with a time code, a sound input means for inputting sound together with a time code, and a reproduction function of the input sound and image, and has a function of reproducing the input sound and image, and outputs necessary sounds and images. a temporary editing means for extracting an image and laying it out on the time axis of a time code; a timing data extraction means for extracting timing data from the temporary editing means; and a timing data extraction means for extracting timing data from the temporary editing means; A multimedia software comprising: a processing/editing means for converting the data into a data format used in the software; and an interleaving means for interleaving the processed/edited data based on the timing data. Creation device.