WO2001086350A1

WO2001086350A1 - System and method for translation of motion picture

Info

Publication number: WO2001086350A1
Application number: PCT/IL2001/000416
Authority: WO
Inventors: Oren J. Maurice; Ami Vinter
Original assignee: Kalifi, Ben
Priority date: 2000-05-10
Filing date: 2001-05-10
Publication date: 2001-11-15
Also published as: AU5663001A; IL143083A0

Abstract

System and methol for generating subtitles for a moving picture, the system comprising a detector (16), a processor (20) and an output device (22). The detector (16) detects the advancing of the moving picture and generates a respective indication output. The processor (20) selects a subtitle from a database in response to the indication output and generates a subtitle signal. The output device (22) puts out the subtitle for human interface according to the subtitle signal. The moving picture may comprise a film strip (14) or stored in a memory. According to a preferred embodiment, the detection is performed with respect to a soundtrack (242) or a key code of the moving picture, both of which may be incorporated in the film strip (14).

Description

SYSTEM AND METHOD FOR TRANSLATION OF MOTION PICTURE

FIELD OF THE INVENTION

The present invention relates to motion picture exhibition in general and to display of subtitles in particular.

BACKGROUND OF THE INVENTION

When screening motion pictures, especially in public theaters, it is often desirable to simultaneously display subtitles which follow the soundtrack of the motion picture. Such is the case when there is a need to translate a foreign language in the soundtrack into a familiar language or when the hearing impaired does not hear the soundtrack. A motion picture for movie theaters is usually filmed on a cinematograghic strip of cellular frames that are projected successively to create images on a screen. The soundtrack is timed with the succession of the projected frames, and the subtitles must be synchronized with the soundtrack.

Various methods for generation of subtitles for motion pictures are known in the art. For example, US Pat. No. 4,673,266 issued to Fiumi discloses a general method and apparatus for displaying of subtitles by which the film strip has coded signals thereon, for timing the display by using a microprocessor decoder. However, the '266 patent does not suggest any specific display to be used, it requires the addition of coded signals on the film strip which may damage the film strip, and furthermore it requires substantial resources.

Further methods are described in US Pat. No. 4,859,994 to Zola et al. and 5,648,789 to Beadles et al. The '994 Patent discloses a coding and decoding means similar to that of the '266 patent, and teaches a specific subtitles display device, namely, polarized light emitted by liquid crystal display that is seen only by those wearing a pair of articulated polarized glasses. The '789 Patent discloses wearable glasses having receiving and decoding circuits, and provides projection of display of the subtitles. However, these patents require the manufacture and maintenance of expensive and sophisticated glasses or polarized screening which is uncomfortable to the viewer. Another prior art subtitle technique involves burning-in of permanent printed subtitles on frames - a method which is time consuming, demands expensive laboratory resources and which is impossible to apply to protected printed subtitles and films that are intended for future use in other countries, or for screening without the subtitles.

Therefore, an object of the present invention is to respond to the long felt need of a simple method for generating subtitles for motion pictures filmed on cellular strips, that requires little time and laboratory resources, that obviates the need to alter the film strip or burning-in of subtitles, is simple to operate, and allows the use of digital equipment for projection of the subtitles.

It is therefore, an object of the present invention, to provide a novel apparatus and method that eliminates the drawbacks of the prior art and satisfies the above mentioned requirements.

These and other objectives are provided by the present invention, to be described below.

SUMMARY OF THE INVENTION

There is thus provided in accordance with the present invention, a system for generating subtitles for a moving picture, the system including a detector, a processor and an output device. The detector detects the advancing of the moving picture and generates an indication output according to the advancing of the moving picture. The processor is connected to the detector and contains a subtitle database. The processor selects a subtitle from the subtitle database in response to the indication output, and generates a subtitle signal corresponding to the subtitle.

The output device is connected to the processor and puts out the subtitle for human interface according to the subtitle signal. The moving picture may be recorded on a film strip, or stored in a memory, such as a magnetic tape, a video tape, a DVD, a CD-ROM, a phonograph, a volatile or nonvolatile computer memory, and the like.

The film strip includes marks detectable by the detector. The marks may be image frames, conveying holes, magnetic marks, variable amounts of ferrous particles, an analog soundtrack, a digital soundtrack, a key code, and the like. The film strip includes a plurality of conveying holes, wherein a conveying holes track according to the present invention, is recorded on the film strip between every two consecutive holes of the conveying holes.

The conveying holes track may include either a soundtrack or a key code. The conveying holes track is incorporated in the film strip in addition to a plurality of conventional soundtracks. The detector can read the conveying holes track according to the present invention, or one of the conventional soundtracks known in the art.

The detector may be a mechanical counter, a wheel counter, a toothed wheel counter, an optic detector, a proximity sensor, a magnetic head, a microphone, an analog-sound detector, a digital-sound detector, a key code detector, and the like. The indication output may have integer value(s), wherein the integer value(s) is a digital representation of an analog signal of a soundtrack of the moving picture.

The detector receives a signal from a source external to the moving picture, wherein the source is in synclirony with the advancement of the moving picture. The source external to the moving picture may be a magnetic tape, a video tape, a DVD, a CD-ROM, a speaker, a phonograph, a volatile computer memory, a nonvolatile computer memory, a radio transmitter, a telephone communication, and the like.

The detector detects a location in the soundtrack of the moving picture before displaying a portion of the moving picture corresponding to that location.

The processor translates the soundtrack from an original language to a translated language, within the period of time between the detection of the location by the detector, and display of a portion of the moving picture corresponding to the location. The processor selects a subtitle from a translated output of the processor, within the period of time between detection of the location by the detector, and display of a portion of the moving picture corresponding to the location.

The system further includes a converter connected between the detector and the processor, wherein the indication output is an analog signal, the converter converts the analog signal into a digital signal, and the converter applies the digital signal to the processor. The converter may be integral with either the detector or the processor.

The processor directs the output device to display the subtitle for a predetermined period of time. The processor further includes a translation program. The translation program translates original character-strings from an original language, to translated character-strings in a translated language. Each of the original character-strings represents a phrase in the original language, and a compatible string of the translated character-strings represents a phrase in a translated language, wherein each of the phrases includes at least one word.

The processor further includes a voice recognition program for differentiating between human voices and non-human sounds. The voice recognition program converts a digital signal, representative of the soundtrack of the moving picture, into a character-string, wherein the character-string represents a spoken word.

The processor provides translation of the moving picture in synchrony with a current spoken phrase. The translation is stored in a memory such as a magnetic tape, a video tape, a DVD, a CD-ROM, a phonograph, a volatile computer memory, a nonvolatile computer memory, a memory including a radio transmitter, a memory including a telephone communication, and the like.

The translation and the moving picture may be stored in two separate memories. The translation may either be prepared off-line prior to display of the moving picture, or it may be a real time simultaneous translation.

The system may further include a signal generator for video, connected between the processor and the output device, wherein the output device includes a video projector, and the signal generator converts a digital signal received from the processor, to an analog video signal, and applies the analog video signal to the video projector.

The output device may be a video projector, a loud speaker, an earphone, a computer display, a TV display, and the like. The putting out operation by the system, may include broadcasting, visually displaying, sounding, transmitting, and the like.

The output device displays a subtitle image of the subtitle on a lower portion of a screen below the projection of an image on the screen. Alternatively, the output device may display an image on an upper portion of the screen where the moving picture is projected, wherein the image is displayed only when there is no projection of the moving picture on the upper portion.

Alternatively, the output device may broadcast the subtitle acoustically via a communication channel, wherein the communication channel is wireless. The output device may include a liquid crystal display, wherein the liquid crystal display displays a subtitle image of the subtitle on a lower portion of the screen. The output device may further include a lens system located in sequence with- a light source and the liquid crystal display, in a line of sight of the light source and the liquid crystal display. The output device may further include a shutter located between the liquid crystal display and the lens system, for obscuring at least partially the liquid crystal display. The shutter obscures the liquid crystal display when the liquid crystal display does not display the subtitle image.

The output device outputs a subtitle image of the subtitle, wherein the subtitle image is a subtitle, a message, an advertisement, and the like.

The communication channel for broadcasting the moving picture may be a wireless television network, a cable television network, a satellite network, a high definition television network, a high definition (digital) motion picture system, a plain simple telephone network (PSTN), Internet, an intranet, a closed circuit audio-visual apparatus, and the like. The closed circuit audio-visual apparatus may incorporate at least one device such as a video tape player, a television camera, a DVD, a computer main frame, and the like.

The connection between the detector, the processor, the output device, the converter, the memory, the external source signaling the detector location of the moving picture, the signal generator, and the closed circuit audio-visual apparatus, may be wired electronic, wired optic, wireless optic, infrared, radio frequency, and the like.

In accordance with another aspect of the present invention, there is thus provided a method for generating and reporting subtitles for a moving picture. The method includes the procedures of creating a subtitle table, detecting the moving picture, generating an indication output, matching the indication output to a selected subtitle within the subtitle table, and reporting the selected subtitle. The procedure of creating a subtitle table may include the sub-procedures of computing a set of reference vectors for each of a plurality of sound frames in a soundtrack of the moving picture, linking the set of reference vectors to a set of subtitles, and storing the set of reference vectors and the set of subtitles in a memory.

The procedure of detecting may include the sub-procedure of computing a representative vector for each of the sound frames when detecting the soundtrack.

The procedure of matching may include the sub-procedure of matching the representative vector with a reference vector of the set of reference vectors. The procedure of reporting may include the sub-procedure of reporting a selected subtitle linked to the reference vector.

The method may further include a procedure of converting the indication output to a digital signal. The procedure of detecting may include a sub-procedure of reading the soundtrack of the moving picture. The soundtrack may be read from a medium such as a cellulose film strip, a magnetic tape, a video tape, a DVD, a CD-ROM, a speaker, a phonograph, a volatile computer memory, a nonvolatile computer memory, a radio transmitter, telephone communication, and the like. The method may further include a preliminary procedure of digitizing the soundtrack.

The procedure of matching may further include the sub-procedures of computing at least one error function, and determining if at least one of the error functions is less than a threshold. The subtitles according to the method may be a message, an advertisement, and the like.

The method may further include the preliminary procedures of digitizing the soundtrack, and outputting the selected subtitle, wherein the procedure of outputting may include broadcasting, visually displaying, sounding, transmitting, and the like. The procedure of outputting may include the display of a subtitle image of each of the subtitles on a lower portion of a screen, for a predetermined period of time. The procedure of outputting may further include the sounding of each of the subtitles via a communication channel.

The procedure of storing may include storing in a memory, wherein the memory is a magnetic tape, a video tape, a DVD, a CD-ROM, a phonograph, a volatile computer memory, a nonvolatile computer memory, and the like.

The procedure of matching may include the sub-procedures of computing at least one error function, and determining if at least one of the error functions is less than a threshold. The reference vector may include a minimum vector and a maximum vector, wherein a set of the minimum vector and the maximum vector is associated with a time code, and the time code is associated with a footage in the film strip of the moving picture.

The procedure of storing may include a sub-procedure of constructing a minimum matrix and a maximum matrix, and the procedure of computing may further include a procedure of constructing a match matrix. The representative vector may include at least one value of at least one representative function, wherein the representative function is a zero crossing function, a

Maurice characteristic function, a minimum error match basic frequency determination function, a discrete Fourier-transform based band-specific amplitude function, and the like. The procedure of matching may include a pseudo-code as described herein below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be further understood and appreciated from the following detailed description, taken in conjunction with the following enclosed drawings in which like numerals designate correspondingly analogous elements or sections throughout, and in which:

Figure 1 is a schematic illustration of a system, constructed and operative in accordance with a preferred embodiment of the present invention;

Figure 2 is a schematic illustration of a film strip travelling between a pair of wheels;

Figure 3 is a schematic illustration of a film strip and a spur wheel;

Figure 4 is a schematic illustration of a system, constructed and operative in accordance with another preferred embodiment of the present invention;

Figure 5 is a schematic illustration of a section of a film strip;

Figure 6 is a schematic illustration of a system, constructed and operative in accordance with a further preferred embodiment of the present invention;

Figure 7 is a schematic illustration of a video projector which is a particular example of projector of Figure 1;

Figure 8 is a schematic illustration of a method for display of subtitles during the projection of a motion picture, operative in accordance with another preferred embodiment of the present invention;

Figure 9 is a schematic illustration of a translation system, constructed and operative in accordance with a further preferred embodiment of the present invention;

Figure 10A is a schematic illustration of a method for preparing subtitles for a moving picture in edition mode, operative in accordance with another preferred embodiment of the present invention; Figure 1 OB is a schematic illustration of a method for accessing subtitles of a moving picture in trace mode, operative in accordance with a further preferred embodiment of the present invention; and

Figures 11A and 11B together constitute a flow chart of a preferable computer algorithm of step 656 of Figure 10A, and step 706 of Figure 10B.

DETAILED DESCRIPTION OF THE INVENTION

The present invention overcomes the disadvantages of the prior art, by providing a system which continuously broadcasts subtitles of a moving picture, each subtitle corresponding to a spoken phrase of the moving picture. A subtitle database corresponding to the moving picture, is stored in the processor. The detector detects a soundtrack in film strip of the moving picture, and generates an indication output corresponding to the soundtrack. The processor selects a subtitle from the subtitle database, the subtitle corresponding to the indication output, and generates a signal. The output device broadcasts the subtitle according to the signal.

Reference is now made to Figure 1, which is a schematic illustration of a system, generally referenced 10, constructed and operative in accordance with a preferred embodiment of the present invention. System 10 includes a main projector 12 for projecting a film strip 14, a detector 16, a converter 18, a processor 20, and a video projector 22. Main projector 12 and video projector 22 project images on a screen 24. Main projector 12 is a device which projects the images developed on a reel of cellulose film strip, on screen 24.

Film strip 14 is a cellulose type film strip conventionally wound on two reels (not shown). Film strip 14 includes a plurality of frames 38, and a plurality of conveying holes 40. Conveying holes 40 are an inherent part of film strip 14, and are used for advancement of film strip 14 through main projector 12, in conjunction with cogged wheel(s) (not shown). Detector 16 may be a counter of a type such as mechanical, optical, magnetic, and the like. Processor 20 is a PC or a computer of any type which includes a CPU and a memory or storage unit (not shown). The storage unit contains a subtitle database. Converter 18 is a device, which converts an analog output signal of detector 16, to a digital signal input to processor 20. The output of detector 16 may be pulse modulated signals, such as "TTL" (transistor-transistor logic digital pulse modulated signal), which is unfit for communication to processor 20. Converter 18, thus converts the output of detector 16 to a transmission code, such as "RS232", adapted for communication with processor 20. Converter 18 can be eliminated if detector 16 outputs a digital signal or if a converter is already integrated with detector 16 or processor 20. In such a case the digital output of detector 16 is fed directly from detector 16 to processor 20 via a direct connection.

Video projector 22 may incorporate either a digital signal entry or an analog video signal entry. A digital video projector receives a digital signal from processor 20, whereas an analog video projector may receive an analog video signal or a digital signal from processor 20. In case an analog video projector is employed, and processor 20 outputs a digital signal, a video-signal generator 21 is connected between processor 20 and video projector 22 or incorporated integrally with video projector 22. The video-signal generator 21 converts a digital signal to an analog signal.

Converter 18 is connected to detector 16 and processor 20. Video projector 22 is connected to processor 20. It is noted that a part or all of the connections between detector 16, converter 18, processor 20, generator 21, and video projector 22 may be optical, infrared, RF, and the like, wired or wireless.

When a motion picture is projected by system 10, film strip 14 continuously travels through main projector 12, while unwinding from one reel (not shown), and winding on a take-up reel (not shown). Film strip 14 travels through main projector 12 in a direction such as the one designated by arrow 26. Thus, main projector 12 projects a motion picture confined by rays 28 and 30, on screen 24, on upper portion 48 thereof. Detector 16 identifies the sequential order of frames 38 of film strip 14, corresponding to the length of film strip 14 unwound from a reel (not shown), and inputs the indication output to converter 18, corresponding to the identity of or order of a frame detected. The indication output is fed to processor 20, directly or via converter 18, as the case may be. Processor 20 identifies the subtitle corresponding to signal received from converter 18 or detector 16, and retrieves a preselected subtitle from the storage unit therein. Processor 20, then inputs a video signal or a digital signal (via generator 21), as the case may be, to video projector 22, corresponding to the preselected subtitle. Video projector 22 then projects the preselected subtitle confined by rays 32 and 34, on a lower portion 46 of screen 24. Video projector 22 can project an image on screen 24, confined by rays

34 and 36, thus covering a lower portion 46 of screen 24, as well as an upper portion 48 thereof, overlapping the projection produced by main projector 12. This may be especially useful for projecting with video projector 22 over portion 48 or entire screen

24, images such as commercial advertisements or other projections, when main projector 12 is inactive. It will be noted that video projector 22 is much more suitable for manipulation for such advertisements or other projections than the main projector 12 which requires celluloid film - that are usually available only for high quality motion pictures.

The preselected subtitle selected by processor 20, corresponds to the current set of frames 38 being projected by main projector 12. Processor 20 contains in a storage unit therein, a subtitle database. The subtitle database contains the subtitles which correspond to the spoken phrases of the moving picture. At the pre-editing stage, an operator intermittently runs the moving picture, and creates a subtitle for each spoken phrase. The operator then stores the subtitle in the subtitle database.

For example, the operator runs the moving picture from the beginning, hears a spoken phrase, stops the moving picture, and creates a subtitle for the phrase. The operator then stores the subtitle in the subtitle database, continues to run the moving picture from the point the moving picture was stopped, hears the next spoken phrase, and creates a subtitle for the new spoken phrase. The operator repeats the cycle described herein above, until the moving picture reaches the end. Furthermore, the operator stores in processor 20, together with each subtitle, a period of time during which the subtitle will be projected on screen 24. Thus, system 10 continuously projects the subtitles corresponding to a motion picture, at the bottom of the motion picture (portion 46 of screen 24).

It is noted that detector 16 allows the synchronization of the subtitle projected on screen 24, with a set of frames 38 which is projected on screen 24, simultaneously. It is furthermore noted, that detector 16 allows projection of a subtitle, which corresponds to a spoken phrase, simultaneously with acoustic broadcast of the spoken phrase. Detector 16 may include a mechanical frame counter. A mechanical counter, generally referenced 100 in Figure 2, may include a simple wheel 15 pressed against another wheel 17 both rotating by film strip 14 travelling there between. According to another preferred embodiment of the present invention, a mechanical frame counter, generally referenced 150 in Figure 3, includes a spur wheel or a toothed wheel 19, teeth 21 of wheel 19 engaged in conveying holes 40 (Figure 1) disposed along the edge of film strip 14. Film strip 14 is run through main projector 12, conveying holes 40 pull teeth 21 and rotate wheel 19. Since the correlation between the number of conveying holes 40 and the number of frames 38 is known, the correlation between the revolution of wheel 19 and the number of passed frames 38 is easily computable. Thus, detector 16 continuously reports to processor 20, the index of frame 38 (footage) which is projected by main projector 12, at any moment during projection of the moving picture.

It will be appreciated that other means for measuring the amount or rate of the passing of film strip 14, such as optical means that read the alternating conveying holes 40 or frames 38, may be applicable.

It is noted that in addition to conveying holes 40, film strip 14 may include continuous lengthwise apertures or other mechanical elements deployed along its edge, in which case detector 16 is able to sense such elements and follow advancement of film strip 14. it is possible to detect film strip 14, mechanically, such that film strip 14 includes variable amounts of ferrous particles, wherein detector 16 is a proximity sensor. Analogously, film strip 14 may include optic or magnetic marks, wherein detector 16 is appropriately optic or magnetic. It is further noted that mechanical elements, ferrous particles, optic marks, and magnetic marks may be incorporated to the original film strip 14 or a copy thereof, in a motion picture laboratory, during production of the original strip of the motion picture, or of a duplicate strip.

It will be appreciated that the terms "projection", "travel", "passing", "conveying", advancing and the like, of film strip 14 by or through main projector 12, respectively, refer also to any other running of the motion picture - with or without a projector or projection thereof, such as running the motion picture "off-line" for editing or synchronization purposes.

The preselected subtitles are previously prepared with or without monitoring of film strip 14 traveling through main projector 12. Prior to display of the motion picture to the audience, film strip 14 is run "off-line" for editing purposes, wherein each specific subtitle is matched and conjugated to a specific output of detector 16. After the editing stage, processor 20 can rematch each specific subtitle to a specific output signal of detector 16.

Processor 20 is further capable of generating digital or video signal of the specific subtitle so matched into video projector 22 for displaying a video image of the digital or video signal of the specific subtitle on screen 24.

Reference is now made to Figure 4, which is a schematic illustration of a system, generally referenced 200, constructed and operative in accordance with a further preferred embodiment of the present invention. System 200 includes a main projector 212, a film strip 214, a detector 216, a converter 218, a processor 220, and an output device 222. Output device 222 is a device which broadcasts a subtitle either graphically on a screen, or acoustically from a loud speaker. Thus, output device 222 can be either a video projector as described herein above with reference to Figure 1, or a transducer. Film strip 214 includes a plurality of frames 238, and a soundtrack 242. Detector 216 is a sound detector, such as a magnetic head, and the like, which reads magnetically or optically recorded analog sound. Converter 218 is an analog to digital converter, as mentioned above with respect to Figure 1. Converter 218 is connected to detector 216 and processor 220. Processor 220 is connected to output device 222. All the parts of system 200, except detector 216, are analogous to their equivalents in system 100 of Figure 1 and therefore are not described again in detail.

Detector 216 reads soundtrack 242 recorded previously on film strip 214, and outputs an analog (or digital) signal 224, corresponding to the sound on soundtrack 242. Soundtrack 242 may be analog or digital, and detector 242 may output analog or digital signal, irrespective of the nature of the soundtrack.

Soundtrack 242 runs continuously along frames 238 on film strip 214. Detector 216 reads the soundtrack 242 along a predefined number of frames 238, and detector 216 outputs signal 224 corresponding to the sound along a frame 238. If signal 224 is analog, converter 218 converts analog signal 224 to digital signal 226. Digital signal 226 has an array of integer values, which is a digital representation of the analog value of soundtrack 242 along a frame 238.

Processor 220 processes digital signal 226, and computes representative value(s) according to a designated function. It is noted that such function should be determined to be immune to background noise. The value(s) computed by the designated function correspond to a specific set of frames 238. These value(s) for each set of frames within film strip 214 are computed at the pre-editing stage, and stored in a memory or storage unit (not shown) of processor 220 and are referred to as "stored" value(s).

During operation, processor 220 computes representative value(s) for a set of frames 238, referred to as "computed" value(s). Processor 220 compares the computed value(s) with the stored value(s), and identifies a set of frames corresponding to the stored value(s). Processor 220 then retrieves a preselected subtitle from the storage unit corresponding to the computed value(s), and inputs a signal to output device 222 corresponding to the preselected subtitle, output device 222 then projects the preselected subtitle on screen 24 (Figure 1) or generates human voice in the translating language, as further explained below.

It is noted that detector 216 may receive analog or digital audio input from a source external to film strip 214, which is synchronized with the advancement of film strip 214 through projector 212. The source external to the film strip 214 may include a magnetic tape, video tape, DVD, CD-ROM, speaker, phonograph, volatile or nonvolatile computer memory, radio transmitter, telephone, and the like. Thereby the input from the external source provides for the synchronization of the projection of a preselected subtitle with a set of frames 238.

It may be appreciated by those skilled in the art, that an advantage of system 200 is that by employing the computation method described herein above, much storage space in memory is saved, and processing time is substantially reduced. Another advantage of system 200 is that the computation method allows separation of human sounds from background non-human sounds recorded on film strip 214.

Technologies and methods for separation of human voice from the soundtrack are known, and can be employed with processor 220 or in connection thereto. Furthermore, technologies and methods for providing automated translation - especially for suggested translation that may aid a human translator in off-line sessions, may be used with processor 220 or in conjunction thereto. The human translator can consider the suggested translation and correct it if necessary. Moreover, technologies and methods for generating synthetic human voice for a given text are known (in the translating language in the context of this invention).

Therefore, processor 220 may incorporate any of the above technologies or be connected to devices providing such technologies, and instead or in addition to projection of subtitles, sound the translated text with output device 222. The non-translated voices may be wiped off the soundtrack by means of said separation of human voice from the soundtrack.

It will further be appreciated that an obstacle to providing a simultaneous translation via system 200, projected or sounded by output device 222, is the need to check and correct, off-line, the suggested automated translation. A real-time automated simultaneous translation may be provided by system 200 once the automated translation is relied upon without further intervening by a human operator. In such a case, detector 216 reads soundtrack 242, a few seconds or even a fraction of a second, prior to its sounding together with the projection of the motion picture. Such a short time delay between the reading of the soundtrack by detector 216 and the projection of the motion picture through film strip 214, suffices for the system to prepare and project or sound the translation through output device 222.

It is noted again that in some cases, although currently still a minority, soundtrack 242 is in digital format, wherein converter 218 may be eliminated from system 200. It is further noted that by employing system 200, it is possible to trace a given soundtrack, with the ability to skip forward or backward within the soundtrack.

Reference is further made to Figure 5, which is a schematic illustration of a section of a film strip, generally referenced 250.

Soundtracks 252 and 254 are conventional SONY soundtracks, and are incorporated on two edges of film strip 250. Soundtracks 252 and 254 are prevalent mainly in the US, and include information such as digital sound, footage location, name of the film, reel number, and the like. Either of soundtracks 252 or 254 may include a key code, the key code containing information such as footage location, and the like. Soundtrack 256 is a conventional DOLBY soundtrack, and it is incorporated between successive conveying holes 258. Soundtrack 256 is prevalent mainly in Europe and the US, and includes digital sound information. Key code track 260 is a conventional DTS key code, and it is incorporated at one edge of frames 262. Key code track 260 is a time code. The DTS system conventionally includes an external device that provides for the soundtrack of the motion picture in response to the reading of key code track 260 while the picture is projected. Conventional soundtracks 264 and 266 are incorporated at one edge of frames 262 (beyond the gap in which DTS key code track 260 is incorporated), on generally all types of conventional film strips 250. Conventional soundtracks 264 and 266 are of optic analog type, and contain sound information.

It is noted that a different detector 216 especially designed to read the different types of each of tracks 252, 254, 256, 260, 264, and 266, is used to read each soundtrack. It is further noted that the film producer or the film distributor can record any or all of the tracks described herein above, on the original film strip or a duplicate thereof, in a laboratory dedicated to perform such recording.

According to a preferred embodiment of the present invention, key code or sound track 268 is recorded between conveying holes 270 on film strip 250. Thus, track 268 may be recorded on film strip 250 in isolation, or in addition to any or all of conventional soundtracks 252, 254, 256, 260, 264, and 266. Detector 216 of system 200 is designed to read track 268. Therefore, the designated track 268 or any of the conventional tracks mentioned above, allow system 200 to project a preselected subtitle corresponding to a set of frames 262.

According to another aspect of the invention, the synchronization of system 10 (Figure 1) and system 200 (Figure 4) to the "projection" of the motion picture, may be employed in dubbing of motion pictures. Thus, it is possible to sound a translation of a motion picture in synchrony with a set of frames 38 (Figure 1). The sounding of the translation replaces the projection of subtitles and is synchronized with a processor equivalent to processor 20 in Fig. 1. Dubbing is performed following a pre-editing stage, in which the background non-human sound is separated from the human sound, as mentioned above. The translation source may be magnetic tape, video tape, DVD, CD-ROM, cellulose film strip, speaker, phonograph, volatile or nonvolatile computer memory, radio transmitter, telephone, and the like.

Likewise, it is possible to simultaneously sound different translations of a motion picture, on different communication channels. The channels may be wired or wireless in a theatre in which the movie is displayed. According to a broader aspect of the invention, the motion picture is broadcast via TV or alternative casting means. Each such sounding of a translation is simultaneously synchronized with projection or display of a movie, and is transmitted in a separate channel. Thus, a viewer is able to switch to a selected channel, and listen to human sounds of a motion picture, in a selected language. Furthermore, according to another variation of the invention, a group of people speaking different languages, may participate in joint activity, for example - play a game, wherein controller of the activity communicates with a player in a language comprehended by the player.

When casting outside a theatre, the communication channel may be provided by television broadcast, cable television, satellite communication, high definition television network, high definition (digital) motion picture system, plain simple telephone network (PSTN), Internet, intranet, closed circuit audio-visual system, and the like.

By employing the high definition (digital) motion picture system, the need to compress all audio translations and video information on a single medium, is obviated. Thus, a high quality video is obtained. Accordingly, it is possible to incorporate only a single language and a single time code, with the video medium, and then store a plurality of translations or subtitles in a different medium. It is then possible to broadcast the motion picture from one medium, and broadcast the human voice or the subtitles in a selected language, from another medium.

Furthermore, a clipping in a web page may be translated to a language comprehended by a user. A closed circuit audio-visual system may be a system operated by video tape player, television camera, DVD, computer main frame, and the like. An audio DVD which includes a plurality of translations of a video DVD, may operate in coordination with a video DVD. Thus, a viewer can watch a motion picture on the video DVD, select a language, and listen to the human voices of the motion picture in the selected language, by incorporating the audio DVD with the video DVD.

Reference is now made to Figure 6, which is a schematic illustration of a system, generally referenced 300, constructed and operative in accordance with another preferred embodiment of the present invention. System 300 includes a main projector 312, a film strip 314, a speaker 316, a microphone 318, a converter 319, a processor 320, and a video projector 322. Converter 319 is an analog to digital converter as mentioned above with reference to converter 18 of Figure 1. Speaker 316 generates the sound corresponding to film strip 314, as known in the art. Microphone 318 detects the sound generated by speaker 316, and sends an analog signal 324 to converter 319. Converter 319 converts analog signal 324 to a digital signal 326, and forwards digital signal 326 to processor 320. Processor 320 retrieves a preselected subtitle, according to digital signal 326.

It is noted that operation of system 300 is analogous to operation of system 200 (Figure 2), wherein signals 324 and 326 correspond to signals 224 and 226, respectively. Thus, system 300 projects preselected subtitles on screen 24 (Figure 1), each preselected subtitle corresponding to a sound generated by speaker 316. It will be appreciated that a "key string" or samples of the digital representation of the soundtrack may be stored in a storage unit so that less memory and processing resources are used in comparison with storage of the soundtrack in its entirety. Moreover, such samples may also refer, among others, to arbitrary sections of the soundtrack or to specific sounds isolated from the soundtrack, such as frequencies or sounds of human voice only.

Reference is now made to Figure 7, which is a schematic illustration of the video projector 22 which is a particular example of projector 22 of Figure 1. Video projector ΥΣ includes a light source 39, an LCD 41, a shutter 42, and a lens system 44. LCD 41 is located at the line of sight between light source 39 and shutter 42. Shutter 42 is located at the line of sight between LCD 41 and lens system 44.

LCD 41 generates an image according to a signal from processor 20. Light source 39 projects the image generated by LCD 41 on screen 24, through lens system 44. For the purpose of eliminating undesired projections on upper portion 48 - which interferes with the projection of the motion picture by projector 12 (as in Figure 1), shutter 42 obscures an appropriate part of LCD 41.

It is noted that LCD 41 generates light also when LCD 41 generates no subtitle or generates subtitle on the lower portion 46 only. Therefore, there is a need to prevent projection of light on screen 24, at times when no subtitle is projected on screen 24. Shutter 42 thus obscures LCD 41 entirely - including the area obscured by perforated section 43 of shutter 42 - when no subtitle is projected, or partially - excluding section 43 - when subtitle is projected, in order to prevent projection of unnecessary light on screen 24, which interferes with the projected motion picture. Furthermore, it is noted that shutter 42 may be fully opened, such that video projector 22" will project an image on the lower portion 46 of screen 24, as well as on upper portion 48 thereof. LCD 41 is thus able to project an image on the entire viewable portions 46 and 48 of screen 24. The image may be a message, an advertisement, and the like, which is projected on screen 24, at times when main projector 12 does not project a motion picture on screen 24, such as during intermissions, and the like. Alternatively, shutter 42 may be fixed, such that shutter 42 invariably obscures a fixed portion of LCD 41, and video projector 22 constantly projects preselected subtitles, or an opaque strip on lower portion 46 of screen 24.

Reference is now made to Figure 8, which is a schematic illustration of a method for display of subtitles during audition of a moving picture, generally referenced 400, operative in accordance with a further preferred embodiment of the present invention.

In step 402, the film strip is run. With reference to Figure 1, film strip 14 is run through main projector 12 and detector 16, in direction 26. In step 404 subtitles are created. A user (not shown), listens to the human voice of running film strip 14 (Figure 1), and creates subtitles corresponding to a set of frames 38. In step 406 a subtitle table is created. With reference to Figure 1, the user stores the subtitles in a memory unit of processor 20, and creates a subtitle table in processor 20. The subtitle table includes the sequential index of frames 38 (footage), each set of frames 38 pointing to the subtitle which the user created for the set of frames 38. It is noted that steps 404 and 406 constitute the pre-editing stage.

In step 408 the frames are counted, or the soundtrack is read, and a signal 409 is generated. With reference to Figures 1, 2, and 3, mechanical frame counter 100 or 150, counts frames 38 of film strip 14, travelling through main projector 12. Mechanical frame counter 100 or 150 then generates signal 409, corresponding to the frame number 38 in film strip 14. Alternatively, with reference to Figure 4, detector 216 reads soundtrack 242, and outputs signal 409. Soundtrack 242 may be read from magnetic tape, video tape, DVD, CD-ROM, cellulose film strip, speaker, phonograph, volatile or nonvolatile computer memory, radio transmitter, telephone, and the like.

In step 410 the signal is digitized or converted to an indication output 410. With reference to Figure 1, converter 18 converts the analog signal output by detector 16, to a digital signal. Alternatively, with reference to Figure 4, converter 218 digitizes analog signal 224 to digital signal 226.

In step 412 the indication output is matched with a subtitle, and a signal 414 is generated. With reference to Figure 1, processor 20 matches indication output 410 to a subtitle, by referring to the subtitle table which the user created in step 406. The integer value of indication output 410 represents the sequential number of frame 38. Processor 20 looks up in the subtitle table, and locates the sequential index of frame 38 stored in subtitle table in step 406, the sequential index corresponding to indication output 410. Processor 20 then accesses from the memory unit, the subtitle which points to the sequential index of frame 38, and outputs a signal 414 corresponding to the subtitle accessed.

In step 416, a visual or an audio output of a subtitle is provided according to signal 414. With reference to Figure 4, processor 220 outputs a signal corresponding to an accessed subtitle, to output device 222. Output device 222 either projects an image of the subtitle on lower portion 46 of screen 24, or sounds the subtitle through a loud speaker (not shown).

Reference is now made to Figure 9, which is a schematic illustration of a translation system, generally referenced 600, constructed and operative in accordance with a further preferred embodiment of the present invention.

Translation system 600 includes a main projector 612, a film strip 614, a detector 616, a converter 618, a processor 620, and an output device 622. Processor 620 further includes a voice recognition program or unit 624, and a translation program 626.

Main projector 612, film strip 614, and converter 618, are analogous to main projector 12, film strip 14, and converter 18, respectively (Figure 1). Detector 616 detects human voices recorded on either of soundtracks 252, 254, 256, 260, 264, and 266 (Figure 3). Voice recognition program or unit 624 is a computer program or a device, which converts a digital signal to a character-string, the character-string representing a spoken word in a selected language. Translation program 626 is a computer program, which converts an original character-string to a translated character-string. The original character-string represent a spoken word, expression, idiom, sentence and the like in an original language recorded on either of soundtracks 252, 254, 256, 260, 264, and 266 (Figure 3). The translated character-string represent a spoken word, expression, idiom, sentence and the like, in a translated language corresponding to the original text in the original language. Output device 622 is either an audio or a visual output device, such as a loud speaker, earphone, computer display, video projector, and the like.

Converter 618 is connected to detector 616 and processor 620. Processor 620 is connected to output device 622. Film strip 614 travels through main projector 612 and detector 616, at a velocity U, in a direction designated by arrow 626.

Main projector 612 projects an image of a frame (not shown) at point 628 in film strip 614, on a screen (not shown). Detector 616 detects either of soundtracks 252, 254, 256, 260, 264, and 266 (Figure 3), at a point 630 in film strip 614, a distance L from point 628 in direction 626. The sound corresponding to point 628 of a frame in film strip 614, is recorded at point 630 in film strip 614. Therefore, detector 616 detects the sound corresponding to point 628, a time T = L / U ahead of projection of the frame at point 628 on a screen (not shown).

Human voices are recorded in an . original language on film strip 614. Detector 616 detects the human voice in the original language, and outputs an analog signal to converter 618. Converter 618 converts the analog signal to a digital signal, and outputs the digital signal to voice recognition program 624. Voice recognition program outputs an original character-string, representing a spoken word, expression, idiom, sentence and the like in the original language, to translation program 626. Translation program 626 converts the original character-string to translated character-string. The translated character-string represent a spoken word, expression, idiom, sentence and the like in a language different than the original language. Processor 620 outputs a signal, which represents the translated character-string, to output device 622. Output device 622 converts the signal to an audio or a visual signal. The user then observes the audio or visual signal, as a translated sound or a translated text of the original language.

It is noted that the user indicates to translation program 626, the language of the original language and the language of the translated language. It is furthermore noted, that if translation system 600 translates a spoken word (or expression, idiom, sentence and the like) in T units of time, then it outputs a translated word corresponding to point 628 in film strip 614, generally when a frame at point 628 is projected on a screen. Thus, translation system 600 can be employed as a simultaneous translation system, whereby the original language of the motion picture is presented to the audience, either by subtitles at the bottom of the screen, or by an audible sound, at audition time.

Alternatively, the user can employ translation system 600 as an off-line translation system. The user observes the translated words visually or audibly, prior to audition, at which time he can edit the translation. The user then records the edited translation as subtitles on a text storage device such as a CD-ROM, DVD, volatile or nonvolatile computer memory, and the like. Alternatively, the user records the edited translation as spoken sounds or computer generated sounds, on a sound storage device such as a CD-ROM, DVD, magnetic tape, soundtrack 268 (Figure 3), and the like.

Reference is now made to Figures 4, 10A, 10B. Figure 10A is a schematic illustration of a method for preparing subtitles for a moving picture in edition mode, generally referenced 650, operative in accordance with another preferred embodiment of the present invention. Figure 10B is a schematic illustration of a method for accessing subtitles of a moving picture in trace mode, generally referenced 700, operative in accordance with a further preferred embodiment of the present invention.

With reference to Figure 10A, an operator runs the moving picture in edition or analysis mode, wherein subtitles are created for the moving picture and stored in memory. Edition mode is a one time operation. With reference to Figure 10B, the operator runs the moving picture in trace mode, wherein the soundtrack of the moving picture is traced during audition. In trace mode, the subtitles which were stored in memory in the edition mode, are accessed from memory and either displayed graphically on a screen, or broadcast from a loud speaker. Thus, the moving picture is run in trace mode during each audition.

With reference to Figure 10A, in step 652 the soundtrack is detected. With reference to Figure 4, detector 216 detects film strip 214, and generates analog signal 224. In step 654 an analog signal is digitized or converted to a digital signal. With reference to Figure 4, converter 218 either digitizes analog signal 224, or converts analog signal 224 to a digital signal 226. In step 656 the min-max vector is computed and a subtitle is prepared. The method for computing the min-max vector is described herein below in detail, in conjunction with step 706 of Figure 10B. It is noted that Figure 10A illustrates a method for running the moving picture in edition mode. In the edition mode, the operator runs the moving picture intermittently, listens to a selected spoken phrase, creates a subtitle for the selected spoken phrase, enters the subtitle in processor 220, and processor 220 concurrently computes a set of min-max vectors for the selected spoken phrase.

In step 658 the set of min-max vectors is linked to a subtitle. Thus, processor 220 links each set of min-max vectors to each subtitle, and creates a record for each spoken phrase. In step 660 the set of min-max vectors and the subtitle are stored. Thus, processor 220 stores the record in memory.

The operator repeats steps 652-660 for each section of the moving picture which contains a selected spoken phrase, while the operator advances film strip 214 one step at a time, in order to create a record for the next spoken phrase. The operator repeats steps 652-660 from the start to the end of the moving picture.

Steps 702 and 704 (Figure 10B), are analogous to steps 652 and 654 (Figure 10A). Step 706 is a matching process which includes steps 708, 710, 712, and 714. In step 706 processor 220 chooses a record corresponding to a current spoken phrase, accesses the record from memory, and reports the selected subtitle of the record, wherein processor 220 outputs a signal. It is noted that a current spoken phrase, is a spoken phrase which a subject who views the moving picture, hears, while the subject views the moving picture.

In step 716 the selected subtitle is broadcast. With reference to Figure 4, processor 220 inputs a signal to output device 222, wherein output device 222 either displays the selected subtitle on a screen, or broadcasts the selected subtitle by a loud speaker. Step 706 is described herein below in detail, in conjunction with step 656 (Figure 10A).

According to one aspect of the present invention, a mathematical method is used in conjunction with a computer algorithm, in order to create a subtitle for each spoken phrase in the moving picture, and to broadcast the subtitle for each spoken phrase during audition.

When running the moving picture in either edition mode (Figure 10 A), or in trace mode (Figure 10B), detector 216 (Figure 4) detects soundtrack 242 in film strip 214 (steps 652 and 702 in Figure 10A and 10B, respectively). Detector 216 outputs analog signal 224 corresponding to soundtrack 242.

Converter or audio digitizer 218 converts analog signal 224 to digital sound frames, herein below referred to as a "DS frame". Converter 218 samples signal 224, and creates a sequence of DS frames, and inputs the sequence of DS frames to processor 220. Converter 218 thus performs pulse code modulation (PCM) of analog signal 224. A DS frame is composed of a predetermined number of digital pulses, n. For instance if at a sampling rate of 11,025 Hz, analog signal 224 is sampled for 0.2 seconds, 2,205 digital pulses are created, and thus, each DS frame includes 2,205 digital pulses. The height of each digital pulse corresponds to voltage level of analog signal 224, at each sampling instance.

Processor 220 converts each digital pulse to a binary number, and creates an array Aj (i = 0, 1, ... n-1) for each DS frame, herein below referred to as a "DS frame A". According to the mathematical method described herein below, processor 220 creates a min-max vector (V^mιn, V^max) for each DS frame, while running in edition mode, and further constructs two matrices from the min-max vectors (a minimum matrix from the minimum vectors, and a maximum matrix from the maximum vectors).

Thus, each row of the minimum matrix is equal to a minimum vector, and each row of the maximum matrix is equal to a maximum vector. Alternatively, instead of a minimum matrix and a maximum matrix,- only one min-max matrix is constructed. Each row of a min-max matrix is then equal to a minimum vector followed by a maximum vector, or a maximum vector followed by a minimum vector. Further alternatively, each row of the min-max matrix is alternately equal to a minimum vector and a maximum vector.

Each set of min-max vectors is associated with a time code, the time code designating the footage in film strip 214. Likewise, processor 220 creates a representative vector V(A) for each DS frame, while running in trace mode.

As described herein above with respect to Figure 10 A, step 658 in edition mode, processor 220 creates a record by linking a set of min-max vectors with a subtitle. In trace mode, processor 220 compares a representative vector with a restricted set of min-max vectors, until an acceptable match is found. Processor 220 then accesses the record from memory (step 712, Figure 10B), and reports the selected subtitle corresponding to the current DS frame.

The method to compute a representative vector V(A), is described herein below. A representative vector V(A) is in the form

V(A) = ZX(A), MF(A), MEM (A), BA(A,1), BA(A,2),- ■ ■ , BA(A,-)

where ZX(A) is a zero crossing function, MF(A) is a Maurice characteristic function, MEM(A) is minimum error match basic frequency determination function, and BA(A,b) is a DFT-based band-specific amplitude function.

In this case, V(A) is composed of four specific functions. It is noted however, that any type of function, and any number of functions may be used to construct a representative vector V(A). In the extreme case, V(A) may be composed of a single function, wherein V(A) is a scalar.

Herein below, each of the four functions which compose the representative vector, are described. It is noted that the index i of the array A; may extend beyond the range (0, 1, ..., n-1). The calculation of the functions is still possible, because the array A is merely a window of n consecutive scalars within a continuous immensely large array of seal ars.

(1) A "Zero Crossing" Function ZX. ZX(A) is a simple representative function that can be applied with relatively slow or weak processors requiring minimal computing resources, and it yields results that may border inaccuracies. It is given by:

ZX(A) = ∑f(i) i = 0 where:

(2) A "Maurice Characteristic" function MF. MF(A) is a more complex representative function, named herein "Maurice" after its author, which requires moderate computing resources, and yields relatively accurate results. It is given by:

(3) A "Minimum Error Match Basic Frequency Determination" Function MEM: MEM(A) is another moderately complex representative function, that requires moderate computing resources and yields relatively accurate results. It is given by:

MEM (A) — the x for which g(A, x) has the minimal value, whereby x is an integer between 1 and n, and g(A, x) is given by:

where (i mod x) is i modulo x

(4) A DFT-based band-specific amplitude function BA.

BA(A, b) is a discrete Fourier transform (DFT) that transforms time-dimension variables into frequency-dimension variables. BA is a generalized function that can theoretically deal with all frequencies, demands high computing resources, and can yield excellent results. It is given by:

n where b is the frequency harmonics selector, b can range from 2 to — . b renders the

BA function to yield a real positive value. Such value represents the relative amplitude

of the signal in the frequency of Hz, where SR stands for the sampling rate in Hz of b the vector A. It will be appreciated that if the relevant frequencies are known, as is often the case - for example those b's that are characteristic to human voice, only the relevant b's could be selected for saving unnecessary computing. Herein below, the method for computing the min-max vector (V^mm, v^max) is described. A minimum vector V^mm and a maximum vector V^max is computed for every

DS frame in edition or analysis mode.

V^mιn and V^max vectors (the "min-max" vectors) are given by the following equations:

where A' is a DS frame of n terms. A' is located in an array A". A" contains two DS frames: the current DS frame A, and the next DS frame A+1. Thus, the A" array contains 2n terms, while the index i runs from zero to 2n-l . p is the number of scalars in representative vector V(A), and in our case, n where V(A) is composed of four functions, it is equal to — 1- 3 .

Herein below, a method is described for determining whether a match exists while in trace mode, and when the location of the last frame is known. Three error functions EF_prev, EF_cur, and EE„_e^ are calculated, and if at least one of them is below a threshold, then a match exists. The values of the error functions, and the computer algorithm for determining whether a match exists or not, is illustrated in Figures 11A and 1 IB.

An error function is generally in the form: p - 1

EF (V, V -m¹¹¹m¹¹¹ , τFina^dx^Λ ) = ∑ ω .ef(V.,V -m"^lm^ul , _τ/ma^clx^A i = 0 where:

and where ω is a "weighing" vector in the length of p scalars, with each scalar ω, being a non-negative coefficient for a term V; in V, thus setting the significance of an error in specific term i in the vector V - for a match-finding decision. ω_{ may be zero, in which case the Vj terms are entirely ignored.

It is noted that an error exists when a scalar Vj in the representative vector V is greater than the corresponding scalar in the maximum vector V^ma , or when a scalar Vi in the representative vector V is less than the corresponding scalar in the minimum vector V^mιn. When such a scalar is found, the error-scalar is the difference between the scalar, and the corresponding scalar which it has exceeded in the min-max vector set.

Reference is now made to Figures 11A and 11B. Figures 11A and 11B together constitute a flow chart of a preferable computer algorithm of step 656 of Figure 10A, and step 706 of Figure 10B, generally referenced 800.

It will be appreciated that the algorithm of Figures 11A and 1 IB is designed for two modes: acquisition mode and trace mode. In an "acquisition" mode, the DS frame index (FRAME) is defined to be zero, and the position of the DS frame A in the soundtrack is not known. In trace mode, the last DS frame index is known , and only minor modifications to the DS frame index are required (up to one DS frame movement backwards or forwards), in order to readjust for time-drifts.

However, if a DS frame is erroneous in trace mode and does not match any min-max vector set in the close neighborhood of its designated frame, it is possible to resort to acquisition mode, and to re-acquire the location, such as in cases where it is assumed that the film strip has been cut and pasted, incorrectly. In such a case it is also possible to perform further tracing searches through further DS frames, hoping that the error is temporary, and that one of the following DS frame(s) will match its designated min-max vector set.

A match matrix for the match algorithm is a matrix with p columns, where p is the length of V, and each row of the matrix is a representative vector. When searching for a match, and a new representative vector is generated, the new representative vector is appended to the match matrix, as the last row of the matrix. A match is sought for the whole match matrix, by checking each row against successive min-max vectors, and checking the case where the error is less than the threshold defined for the required tolerance for each and every line of the match matrix. With reference to Figures 11 A and 1 IB in sequence, a pseudo-code is herein below presented:

Empty the match matrix (set it to 0 rows) (step 802)

2. Set DS-frame positioning number("FRAME"=0) (step 804)

3. While the film strip/soundtrack is running do (step 806):

4. Get DS vector A from the audio digitizer (wait until a DS-frame is available) (step 808)

5. Compute V(A) (step 810)

6. if FRAME ? 0 then (step 812)

7 Computed = EF(V, V ^m , V™^ax ) (step prev brame Frame

814)

Compute EF = EF(V, V -m¹"m^UX _Λ , V ri"riax cur trame + 1 Frame + 1'

(step 816)

Compute E ) next

+ 2

(step 818) 10. If EFprev is the smallest among the error values (EF_prev, EF_cur, EF_ne_xt), do not change FRAME (steps 820 and 822)

11. if EF_cur is the smallest among the error values, set FRAME-FRAME+1 (steps 824 and 826)

12. if EF_next is the smallest among the error values, set FRAME=FRAME+2 (steps 828 and 830)

13. if all the error values are greater than a specific threshold for more than a specific number of frames, set FRAME=0 and empty the match matrix (set it to 0 rows) (steps 832 and 834)

14. else (6) (step 812)

15. Append V as the last row to the match matrix (step 836)

16. Scan the min-max vectors for a sequence (of min-max vectors) that agrees with the match matrix (step 838) 17. If one such sequence was found, set FRAME=(frame number of the min-max vector that agrees with the first row in the match matrix) + (number of rows in the match matrix) (steps 840 and 842)

18. If no such sequence was found, remove the first row from the matrix, and go back to line 16 (steps 840 and 844)

19. End (6)

20. if FRAME ≠ 0 then look for a matching set of subtitles (by the DS FRAME positioning number or time code), and send them to the digital/video signal generator (step 846)

21. wend

It will be appreciated by those skilled in the art that the invention is not limited to what has been shown and described hereinabove by way of example. Rather, the scope of the invention is limited solely by the claims which follow.

Claims

CLAIMS:

1. System for generating subtitles for a moving picture, the system comprising: a detector, detecting the advancing of said moving picture and generating an indication output according to the advancing of said moving picture; a processor connected to said detector, said processor contains a subtitle database and selects a subtitle from said subtitle database in response to said indication output and generates a subtitle signal corresponding to said subtitle; and an output device connected to said processor, for putting out said subtitle for human interface according to said subtitle signal.

2. The system according to claim 1, wherein said moving picture is stored in a memory, which is selected from a list consisting of: a film strip; a magnetic tape; a video tape; a DVD; a CD-ROM; a phonograph; a volatile computer memory; and a nonvolatile computer memory.

3. The system according to claim 1, wherein said moving picture comprises a film strip.

4. The system according to claim 3, wherein said film strip comprises marks detectable by said detector and selected from a list consisting of: image frames; conveying holes; magnetic marks; variable amounts of ferrous particles; an analog soundtrack; a digital soundtrack; and a key code;

5. The system according to claim 3, wherein said film strip comprises a plurality of conveying holes, and wherein a conveying holes track is recorded on said film strip between consecutive holes of said conveying holes.

6. The system according to claim 5, wherein said conveying holes track comprises a soundtrack.

7. The system according to claim 5, wherein said conveying holes track comprises a key code.

8. The system according to claim 5, wherein said conveying holes track is incorporated in said film strip in addition to a plurality of conventional soundtracks.

9. The system according to claim 8, wherein said detector reads said conveying holes track.

10. The system according to claim 8, wherein said detector reads at least one of said conventional soundtracks.

11. The system according to claim 1, wherein said detector is selected from a list consisting of: a mechanical counter; a wheel counter; a toothed wheel counter; an optic detector; a proximity sensor; a magnetic head; a microphone; an analog-sound detector; a digital-sound detector; and a key code detector.

12. The system according to claim 1, wherein said indication output includes a digital representation of an analog signal of a soundtrack of said moving picture.

13. The system according to claim 1, wherein said detector receives a signal from a source external to said moving picture, said source is in synchrony with the advancement of said moving picture.

14. The system according to claim 13, wherein said source is selected from a list consisting of: a magnetic tape; a video tape;

DVD;

CD-ROM; a speaker; a phonograph; a volatile computer memory; a nonvolatile computer memory; a radio transmitter; and a telephone communication..

15. The system according to claim 1, wherein said detector detects a location in the soundtrack of said moving picture before display of a portion of said moving picture corresponding to said location.

16. The system according to claim 15, wherein said processor translates said soundtrack from an original language to a translated language within the period of time between the detection of said location by said detector and the display of a portion of said moving picture corresponding to said location.

17. The system according to claim 16, wherein said processor selects a subtitle from a translated output of said processor, within the period of time between the detection of said location by said detector and the display of a portion of said moving picture corresponding to said location.

18. The system according to claim 1, further comprising a converter connected between said detector and said processor, wherein said indication output is an analog signal, said converter converts said analog signal into a digital signal and applies it to said processor.

19. The system according to claim 18, wherein said converter is integral with said detector.

20. The system according to claim 18, wherein said converter is integral with said processor.

21. The system according to claim 1, wherein said processor directs said output device to display said subtitle for a predetermined period of time.

22. The system according to claim 1, wherein said processor further comprises a translation program.

23. The system according to claim 22, wherein said translation program translates original character-strings from an original language to translated character-strings in a translated language, wherein each of said original character-strings represents a phrase in said original language, wherein a compatible string of said translated character-strings represents a phrase in a translated language, correspondingly, and wherein each of said phrases includes at least one word.

24. The system according to claim 1, wherein said processor further comprises a voice recognition program for differentiating between human voices and non-human sounds, wherein said voice recognition program converts a digital signal, representative of the soundtrack of said moving picture, into a character-string, and wherein said character-string represents a spoken word.

25. The system according to claim 24, wherein said processor provides a translation of said moving picture in synchrony with a current spoken phrase.

26. The system according to claim 25, wherein said translation is stored in a memory selected from a list consisting of: a magnetic tape; a video tape;

DVD;

CD-ROM; a phonograph; a volatile computer memory; a nonvolatile computer memory; a memory comprising a radio transmitter; and a memory comprising a telephone communication.

27. The system according to claim 22, wherein said translation and said moving picture are stored in two separate memories.

28. The system according to claim 22, wherein said translation is prepared off-line prior to display of said moving picture.

29. The system according to claim 22, wherein said translation is a real time simultaneous translation.

30. The system according to claim 24 wherein said voice recognition program provides for a real time simultaneous translation by integrating a digital representation of said non-human sounds with a compatible strings of said translated character-strings representing a phrase in a translated language for dubbing said soundtrack.

31. The system according to claim 1 , further comprising a signal generator for video, connected between said processor and said output device, wherein said output device includes a video projector and said signal generator converts a digital signal received from said processor to an analog video signal and applies it to said video projector.

32. The system according to claim 1, wherein said output device is selected from a list consisting of: a video projector; a loud speaker; an earphone; a computer display; and a TV display.

33. The system according to claim 1, wherein said putting out is selected from a list consisting of: broadcasting; visually displaying; sounding; and transmitting.

34. The system according to claim 1, wherein said output device displays a subtitle image of said subtitle on a lower portion of a screen below the projection of an image on said screen.

35. The system according to claim 1, wherein said output device displays an image on an upper portion of a screen where the moving picture is projected, and wherein said image is displayed only when there is no projection of the moving picture on said upper portion.

36. The system according to claim 1, wherein said output device broadcasts said subtitle acoustically via a communication channel.

37. The system according to claim 36, wherein said communication channel is wireless.

38. The system according to claim 1, wherein said output device further comprises a liquid crystal display, said liquid crystal display displays a subtitle image of said subtitle on a lower portion of a screen.

39. The system according to claim 38, wherein said output device further comprises a lens system located in sequence with a light source and said liquid crystal display, in a line of sight of said light source and said liquid crystal display, said output device further comprises a shutter located between said liquid crystal display and said lens system, for obscuring at least partially said liquid crystal display.

40. The system according to claim 39, wherein said shutter obscures said liquid crystal display when said liquid crystal display does not display said subtitle image.

41. The system according to claim 1, wherein said output device outputs a subtitle image of said subtitle, and wherein said subtitle image is selected from a list consisting of: a subtitle; a message; and an advertisement.

42. The system according to claim 1, wherein said moving picture is broadcast via a communication channel selected from a list consisting of: wireless television network; cable television network; satellite network; high definition television network; high definition (digital) motion picture system; plain simple telephone network (PSTN);

Internet;

Intranet; and closed circuit audio-visual apparatus.

43. The system according to claim 42, wherein said closed circuit audio-visual apparatus incorporates at least one device selected from a list consisting of: a video tape player; a television camera; DVD; and a computer main frame.

44. The system according to claim 1, wherein a connection between any two elements thereof, is selected from a list consisting of: wired electronic; wired optic; wireless optic; infrared; and radio frequency.

45. The system according to claim 44, wherein said two elements are selected from the list consisting of: said detector; said processor; said output device; a converter; a memory; an external source signaling said detector location of said moving picture; a signal generator for video; and a closed circuit audio-visual apparatus.

46. Method for generating and reporting subtitles for a moving picture, the method comprising the procedures of: creating a subtitle table; detecting said moving picture and generating an indication output; matching said indication output to a selected subtitle within said subtitle table; and reporting said selected subtitle.

47. The method according to claim 46, wherein said procedure of creating comprises a sub-procedure of linking a set of frames in a film strip of said moving picture, with each subtitle in said subtitle table.

48. The method according to claim 46, wherein said procedure of detecting comprises a sub-procedure of counting a plurality of frames in a film strip of said moving picture.

49. The method according to claim 46, further comprising a procedure of converting said indication output to a digital signal.

50. The method according to claim 46, wherein said procedure of detecting comprises a sub-procedure of reading a soundtrack of said moving picture.

51. The method according to claim 50, wherein said soundtrack is read from a medium selected from a list consisting of: a cellulose film strip; a magnetic tape; a video tape;

DVD;

CD-ROM; a speaker; a phonograph; a volatile computer memory; a nonvolatile computer memory; a radio transmitter; and telephone communication.

52. The method according to claim 46, wherein said procedure of matching further comprises the sub-procedures of: computing at least one error function; and determining if at least one of said at least one error functions is less than a threshold.

53. The method according to claim 46, wherein said subtitles are selected from a list consisting of: a subtitle; a message; and an advertisement.

54. The method according to claim 46, further comprising a preliminary procedure of digitizing said soundtrack.

55. The method according to claim 46, further comprising a procedure of outputting said selected subtitle.

56. The method according to claim 55, wherein said procedure of outputting includes a sub-procedure is selected from a list consisting of: broadcasting; visually displaying; sounding; and transmitting.

57. The method according to claim 55, wherein said procedure of outputting comprises displaying a subtitle image of each of said subtitles on a lower portion of a screen, for a predetermined period of time.

58. The method according to claim 55, wherein said procedure of outputting comprises sounding each of said subtitles via a communication channel.

59. The method according to claim 46, wherein: said procedure of creating a subtitle table includes the sub-procedures of: computing a set of reference vectors for each of a plurality of sound frames in a soundtrack of said moving picture; linking said set of reference vectors to a set of subtitles; and storing said set of reference vectors and said set of subtitles in a memory; said procedure of detecting includes the sub-procedure of computing a representative vector for each of said sound frames when detecting said soundtrack; said procedure of matching includes the sub-procedure of matching said representative vector with a reference vector of said set of reference vectors; and said procedure of reporting includes the sub-procedure of reporting a selected subtitle linked to said reference vector.

60. The method according to claim 59, wherein said sub-procedure of storing comprises storing in a memory selected from a list consisting of: a magnetic tape; a video tape;

DVD;

CD-ROM; a phonograph; a volatile computer memory; and a nonvolatile computer memory.

61. The method according to claim 59, wherein said procedure of matching further comprises the sub-procedures of: computing at least one error function; and determining if at least one of said at least one error functions is less than a threshold.

62. The method according to claim 59, wherein said reference vector comprises a minimum vector and a maximum vector.

63. The method according to claim 62, wherein a set of said minimum vector and said maximum vector is associated with a time code, and wherein said time code is associated with a footage in a film strip of said moving picture.

64. The method according to claim 59, wherein said sub-procedure of storing further comprises a sub-procedure of constructing a minimum matrix and a maximum matrix.

65. The method according to claim 59, wherein said sub-procedure of computing further comprises a sub-procedure of constructing a match matrix.

66. The method according to claim 59, wherein said representative vector comprises at least one value of at least one representative function.

67. The method according to claim 66, wherein said representative function is selected from a list consisting of: a zero crossing function; a Maurice characteristic function; a minimum error match basic frequency determination function; and a discrete Fourier-transform based band-specific amplitude function.

68. The method according to claim 59, wherein said procedure of matching includes a sub-procedure comprising a pseudo-code as follows:

1. Empty the match matrix (set it to 0 rows)

2. Set DS-frame positioning number("FRAME"=0)

3. While the film strip/soundtrack is running do

4. Get DS vector A from the audio digitizer (wait until a DS-frame is available)

5. Compute V(A)

6. if FRAME ? 0 then

7. Compute EF = EF(V,V™^m ,V™^ax ) prev Frame Frame

8. Compute EF = EF(V, V™^m V™^ax J cur Frame + 1 Frame 4- 1

9. Compute EF = EF(V,V ^{ _;F™^ax _ ) next Frame + 2 Frame + 2

10. If EFprev is the smallest among the error values (EF_prev, EF_CUr, EF_next), do not change FRAME

11. if EF_cur is the smallest among the error values, set FRAME=FRAME+1

12. if EFnext is the smallest among the error values, set FRAME=FRAME+2

13. if all the error values are greater than a specific threshold for more than a specific number of frames, set FRAME=0 and empty the match matrix (set it to 0 rows)

14. else (6)

15. Append V as the last row to the match matrix

16. Scan the min-max vectors for a sequence (of min-max vectors) that agrees with the match matrix

17. If one such sequence was found, set FRAME=(frame number of the min-max vector that agrees with the first row in the match matrix) + (number of rows in the match matrix)

18. If no such sequence was found, remove the first row from the matrix, and go back to line 16

19. End (6)

20. if FRAME ≠ 0 then look for a matching set of subtitles (by the DS

FRAME positioning number or time code), and send them to the digital/video signal generator

21. wend