CN113989281B - Film sound reproduction system and method based on optical imaging - Google Patents
Film sound reproduction system and method based on optical imaging Download PDFInfo
- Publication number
- CN113989281B CN113989281B CN202111616095.6A CN202111616095A CN113989281B CN 113989281 B CN113989281 B CN 113989281B CN 202111616095 A CN202111616095 A CN 202111616095A CN 113989281 B CN113989281 B CN 113989281B
- Authority
- CN
- China
- Prior art keywords
- image
- vector
- audio
- damaged
- film
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000012634 optical imaging Methods 0.000 title claims abstract description 62
- 238000001514 detection method Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000005516 engineering process Methods 0.000 claims abstract description 30
- 230000008439 repair process Effects 0.000 claims abstract description 27
- 238000006243 chemical reaction Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 28
- 238000003384 imaging method Methods 0.000 claims description 16
- 230000000694 effects Effects 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 13
- 239000000126 substance Substances 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 230000010354 integration Effects 0.000 claims description 4
- 229920001220 nitrocellulos Polymers 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 claims description 3
- QTBSBXVTEAMEQO-UHFFFAOYSA-M Acetate Chemical compound CC([O-])=O QTBSBXVTEAMEQO-UHFFFAOYSA-M 0.000 claims description 2
- 230000004907 flux Effects 0.000 abstract description 8
- 238000005070 sampling Methods 0.000 description 9
- 238000005457 optimization Methods 0.000 description 5
- 238000012937 correction Methods 0.000 description 3
- 238000002485 combustion reaction Methods 0.000 description 2
- 238000002425 crystallisation Methods 0.000 description 2
- 230000008025 crystallization Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002845 discoloration Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000002269 spontaneous effect Effects 0.000 description 2
- 159000000021 acetate salts Chemical class 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 150000002823 nitrates Chemical class 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G06T5/77—
Abstract
The invention provides a film sound reproduction system and method based on optical imaging, firstly, generating a sound track image according to a sound track identifier on a film through an optical imaging device, then, carrying out damage detection judgment on the sound track image according to a chrominance identification technology and a symmetry judgment technology, and taking the sound track image which is confirmed to be damaged as a damaged image; then, carrying out repair processing on the damaged image to form a repaired image; and then, the restored image is subjected to sound reproduction processing so as to convert the restored image into a time domain waveform and form a restored audio, and the sound reproduction mode is based on light beam scanning and photoelectric conversion in the traditional method, converts the moving scanning light beam into a traversing pixel-level light seam and converts the light flux of a local area into a full-local luminosity integral, so that the sound reproduction speed and the sound reproduction quality can be greatly and remarkably improved.
Description
Technical Field
The present invention relates to the field of media restoration, and more particularly, to a motion picture film sound reproduction system and method based on optical imaging.
Background
The historical video and audio data has important artistic value. In recent years, due to the progress of screen technology and projection technology, many famous old movies, such as "robbers", "piano players at sea", and "thousand and thousand seek", all undergo 4K repair and re-show, and have also obtained good public praise. Motion picture film can be divided into two parts: a track region and a picture region. Soundtracks are an important component of film, and soundtrack restoration has a crucial impact on old film restoration, which is a large impact compared to picture portion restoration, however, film soundtrack restoration research has been neglected for a long time, and the source of film restoration has long been focused on the image domain.
Film of a film soundtrack is damaged from various aspects and links, the humidity and temperature of air can affect the film during the storage period, and mildew, spontaneous combustion, natural discoloration, crystallization and the like are common damages of the film. In practice, films recorded on nitrate film in the early days of the film industry are not allowed to be shown using current projectors due to low flash point, which is a huge loss for the cultural industry, and the film itself is damaged each time the film is shown. At present, the repair of the movies is mainly manual repair. Repairing a 90 minute movie requires more than 3 months of 15 professional teams; moreover, the soundtrack film is essentially a modulated audio time domain waveform recorded in the image domain, and the occurring damage often includes both image damage and audio damage, and the image domain damage is the main one, and the traditional method can only repair the damage from the sound domain.
Therefore, there is a need for a soundtrack film restoration method mainly based on image domain restoration and assisted by sound domain restoration, and there is a need for a motion picture film sound reproduction system and method based on optical imaging, which can perform cross-mode sound reproduction restoration on soundtrack film on the basis of ensuring the "old and worn principle" of historical audio and video, so as to better improve sound reproduction efficiency and sound reproduction effect.
Disclosure of Invention
In view of the above problems, the present invention aims to provide a film sound reproduction system based on optical imaging, so as to solve the problems that the existing film restoration method in the prior art still mainly adopts manual restoration, and needs a professional team of 15 people to work for more than 3 months to restore a 90-minute film; moreover, the soundtrack film is essentially a modulated audio time domain waveform recorded in the image domain, the occurring damage usually includes both image damage and audio damage, and the image domain damage is taken as the main, and the traditional method can only repair the damage from the sound domain.
The invention provides a cinematographic film sound reproduction system based on optical imaging, which comprises an optical imaging device, an imaging detection module, an image restoration module and a sound reproduction module, wherein,
the optical imaging device is used for generating a music track image according to the music track identification on the film;
the imaging detection module is used for carrying out damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology, and taking the audio track image confirmed to be damaged as a damaged image;
the image restoration module is used for restoring the damaged image to form a restored image;
and the sound reproduction module is used for carrying out sound reproduction processing on the restored image so as to convert the restored image into a time domain waveform and form a restored audio.
Preferably, an audio repair module is further included;
the audio restoration module is used for optimizing the restored audio to form film audio.
Preferably, the film includes at least a damaged motion picture soundtrack film, a nitrate salt film, an acetate salt film.
Preferably, the imaging detection module comprises an initial detection unit, a threshold acquisition unit and a damage judgment unit; wherein the content of the first and second substances,
the initial detection unit is used for carrying out initial detection on the audio track images to determine whether the audio track images are damaged or not and taking the audio track image with the lightest damage as a reference frame;
the threshold acquisition unit is used for extracting the features of the reference frame to acquire the feature parameters of the reference frame;
the damage judging unit is used for carrying out gray level clustering on the audio track images to obtain a gray level clustering result, obtaining an upper reference threshold and a lower reference threshold based on the gray level clustering result, comparing the upper reference threshold and the lower reference threshold with the characteristic parameters to obtain a specific damage degree of each audio track image, and mapping the specific damage degree with the audio track images to form damaged images.
Preferably, the image restoration module includes an area locking unit and a restoration unit; wherein, the first and the second end of the pipe are connected with each other,
the region locking unit is used for determining a first vector, a second vector, a third vector and a fourth vector in the damaged image according to the upper reference threshold and the lower reference threshold; taking a region enclosed by the first frame vector and the second vector as a first region, taking a region enclosed by the third vector and the fourth vector as a second region, constructing track functions related to the first vector, the second vector, the third vector and the fourth vector based on the first region and the second region, and determining an ideal function meeting all track envelope requirements according to the track functions;
and the repairing unit is used for repairing the color space of the damaged image according to the ideal function so as to form a repaired image.
Preferably, the sound reproduction module comprises a unit to be modulated and a modulation unit;
the unit to be modulated performs high-precision conversion on the envelope curve of a light and dark area on the repaired image pixel by using a traversing pixel-level light slit, and performs amplified envelope curve processing on the whole domain of the repaired image according to integration so as to convert the repaired image into a curve to be modulated;
the modulation unit is used for carrying out pre-emphasis or de-emphasis processing on the curve to be modulated to obtain an original recording curve, and the original recording curve is used as film audio frequency reduction audio frequency to be output; wherein, the first and the second end of the pipe are connected with each other,
the modulation unit comprises A-ring modulation and B-ring modulation; wherein the content of the first and second substances,
the A ring modulation at least comprises an optical sound reduction head, a photoelectric transducer and a preamplifier of the projector;
the B-ring modulation at least comprises an input end of a main attenuator, an equalizer, a power amplifier, a frequency division network, a loudspeaker and a sound field from a screen to a sitting area of an audience hall;
the initial recording curve is a time domain waveform.
Preferably, the sound reproduction module further comprises a personality module, and the personality module is used for providing different audio effects for the restored audio.
In another aspect, the present invention provides a method for playing back a film based on optical imaging, which is characterized in that the method for playing back a film based on optical imaging performs a playing back process on the film based on the aforementioned system for playing back a film based on optical imaging, and includes:
generating, by the optical imaging device, a soundtrack image from the soundtrack identification on the film;
performing damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology, and taking the audio track image with damage confirmed as a damaged image;
performing a repair process on the damaged image to form a repaired image;
and performing sound reproduction processing on the restored image to convert the restored image into a time domain waveform and form a restored audio.
Preferably, the process of performing damage detection judgment on the audio track image according to a chromaticity identification technology and a symmetry judgment technology, and determining a damaged audio track image as a damaged image, includes:
performing preliminary detection on the audio track images to determine whether the audio track images are damaged or not, and taking the audio track image with the lowest damage as a reference frame;
performing feature extraction on the reference frame to acquire feature parameters of the reference frame;
performing gray level clustering on the audio track images to obtain a gray level clustering result, obtaining an upper reference threshold and a lower reference threshold based on the gray level clustering result, comparing the upper reference threshold and the lower reference threshold with the characteristic parameters to obtain a specific damage degree of each audio track image, and mapping the specific damage degree with the audio track images to form damaged images.
Preferably, the process of performing a repair process on the damaged image to form a repaired image includes:
determining a first vector, a second vector, a third vector and a fourth vector in the damaged image according to the upper reference threshold and the lower reference threshold; taking a region enclosed by the first frame vector and the second vector as a first region, taking a region enclosed by the third vector and the fourth vector as a second region, constructing track functions related to the first vector, the second vector, the third vector and the fourth vector based on the first region and the second region, and determining an ideal function meeting all track envelope requirements according to the track functions;
and carrying out chroma space restoration on the damaged image according to the ideal function to form a restored image.
From the above technical solutions, the present invention provides a motion picture film sound reproduction system and method based on optical imaging, including an optical imaging device, an imaging detection module, an image restoration module, and a sound reproduction module, wherein the optical imaging device is configured to generate a sound track image according to a sound track identifier on a film; the imaging detection module is used for carrying out damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology and taking the audio track image confirmed to be damaged as a damaged image; the image restoration module is used for restoring the damaged image to form a restored image; the invention relates to a method for processing a restored image, which comprises the steps of carrying out sound reproduction processing on the restored image to convert the restored image into a time domain waveform to form a restored audio, replacing an original scanning method by using an optical imaging method, carrying out digital processing on a sound reproduction process, analyzing the damage characteristics of a film, converting a moving scanning light beam into a traversal pixel level light seam on the basis of light beam scanning and photoelectric conversion in the traditional method, and converting the light flux of a local area into a full-local luminosity integral, wherein the sound reproduction speed and the sound reproduction quality can be greatly improved, compared with the traditional method, the method disclosed by the invention has the advantages that the speed is improved by more than 2 times, the audio effect after sound reproduction can be freely selected, and the sampling rate can reach 162.08% of the traditional method.
Drawings
Other objects and results of the present invention will become more apparent and more readily appreciated as the same becomes better understood by reference to the following specification taken in conjunction with the accompanying drawings. In the drawings:
fig. 1 is a system block diagram of a motion picture film soundtrack system based on optical imaging in accordance with an embodiment of the present invention;
fig. 2 is a schematic diagram of damaged images involved in a graphics repair module in a motion picture film sound reproduction system based on optical imaging according to an embodiment of the present invention;
fig. 3 is a flowchart of a motion picture film soundtrack method based on optical imaging in accordance with an embodiment of the present invention.
Detailed Description
Film of a movie soundtrack is damaged from all aspects and all links, and during the storage period, the humidity and temperature of the air affect the film, and mildew, spontaneous combustion, natural discoloration, crystallization, and the like are common damages of the film. In practice, films recorded on nitrate film in the early days of the film industry are not allowed to be shown using current projectors due to low flash point, which is a huge loss for the cultural industry, and the film itself is damaged each time the film is shown. At present, the repair of the movies is mainly manual repair. Repairing a 90 minute movie requires more than 3 months of 15 professional teams; moreover, the soundtrack film is essentially a modulated audio time domain waveform recorded in the image domain, the occurring damage often includes both image damage and audio damage, and the image domain damage is taken as the main, and the traditional method can only repair the damage from the sound domain.
In view of the above problems, the present invention provides a motion picture film sound reproduction system and method based on optical imaging, and the following describes in detail the specific embodiments of the present invention with reference to the attached drawings.
To illustrate the optical imaging based motion picture film sound reproduction system provided by the present invention, fig. 1 schematically illustrates an optical imaging based motion picture film sound reproduction system according to an embodiment of the present invention; fig. 3 illustrates an exemplary embodiment of the present invention for a motion picture film soundtrack method based on optical imaging.
The following description of the exemplary embodiment(s) is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses. Techniques and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but are intended to be considered a part of the specification where appropriate.
As shown in fig. 1, the present invention provides a motion picture film sound reproduction system 100 based on optical imaging, which comprises an optical imaging device 110, an imaging detection module 120, an image restoration module 130 and a sound reproduction module 140, wherein the optical imaging device 110 is configured to generate a soundtrack image according to the soundtrack identification on the film; the imaging detection module 120 is configured to perform damage detection and judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology, and use the audio track image determined to be damaged as a damaged image; the image restoration module 130 is used for performing restoration processing on the damaged image to form a restored image; the audio reproduction module 140 is configured to perform audio reproduction processing on the restored image to convert the restored image into a time-domain waveform, so as to form a restored audio.
In the embodiment shown in fig. 1, the optical imaging device 110 is based on an optical imaging method, that is, the system is a film soundtrack data acquisition system based on an optical imaging method, which uses an optical imaging device to replace a laser scanning system based on light flux in a conventional projector to acquire film soundtrack information, and by introducing an optical imaging system using a non-heat-generating light source, most films (including flammable nitrate film, photolyzable acetate film, etc.) that cannot be processed by the conventional projector can be normally acquired, and at the same time, the optical imaging device is no longer limited to use of a blue light source during sound reproduction, so that a wider choice can be made in the visible light spectrum, so that the acquired film has better light flux, and the light source is improved to facilitate scratches on the film surface, The fading of dust and other damages, and the sound quality is improved.
In the embodiment shown in fig. 1, the imaging detection module 120 includes a preliminary detection unit 121, a threshold acquisition unit 122, and a damage determination unit 123; wherein the content of the first and second substances,
the initial detection unit 121 is configured to perform initial detection on the audio track image to determine whether the audio track image is damaged, and use the audio track image with the least damage as a reference frame;
the threshold obtaining unit 122 is configured to perform feature extraction on the reference frame to obtain feature parameters of the reference frame;
the damage determination unit 123 is configured to perform grayscale clustering on the audio track images to obtain a grayscale clustering result, obtain an upper reference threshold and a lower reference threshold based on the grayscale clustering result, compare the upper reference threshold and the lower reference threshold with the characteristic parameters to obtain a specific damage degree of each audio track image, and map the specific damage degree with the audio track images to form a damaged image;
more specifically, the imaging detection module 120 detects the damage of the film picture according to the chrominance identification and symmetry judgment of the image, and first, performs preliminary detection on the input audio track image frame sequence, selects a frame of image with the minimum damage as a reference frame, performs gray level clustering after the image is mapped according to the image characteristics of the reference frame, and obtains an upper reference threshold according to the gray level clustering result
And a lower reference threshold
And according to the previously input mildew gray level threshold
To calculate the final threshold value
As shown in equation (1).
Wherein the content of the first and second substances,
in order to be the final threshold value, the threshold value is set,
in order to be the upper reference threshold value,
is a lower reference threshold value for the lower threshold value,
is a constant weighting factor. Please note that the other letters involved in the formula are defined in this figure so as to obtain the upper reference threshold and the lower reference threshold, and obtain the damage degree of the damaged image.
In the embodiment shown in fig. 1, the image repair module 130 includes an area locking unit 131 and a repair unit 132; wherein, the first and the second end of the pipe are connected with each other,
the region locking unit 131 is configured to determine a first vector, a second vector, a third vector and a fourth vector in the damaged image according to an upper reference threshold and the lower reference threshold; taking a region enclosed by the first vector and the second vector as a first region, taking a region enclosed by the third vector and the fourth vector as a second region, constructing track functions related to the first vector, the second vector, the third vector and the fourth vector based on the first region and the second region, and determining an ideal function meeting the envelope requirements of all tracks according to the track functions;
the repairing unit 132 is configured to repair the damaged image in the chrominance space according to the ideal function to form a repaired image;
more specifically, as shown in fig. 1 and 2 together, the module 130 performs image damage repair, film correction, audio track envelope optimization, and other tasks. Two-bit modeling for a film soundtrack image to be repaired is as follows, let
The frame picture is
，
And
is an upper reference threshold
The divided boundary envelope curve is divided into a plurality of boundary envelope curves,
and
is a lower reference threshold
The divided boundary envelope curve is divided into a plurality of boundary envelope curves,
is composed of
And
the area of the air duct is surrounded by the air duct,
is composed of
And
the enclosed area, as shown in figure 2,
、
、
、
is composed of
、
、
、
The corresponding curve function is bound to have an envelope curve
、
Satisfy the formula (2), provided
As shown in equation (3), then there must be
Satisfy all envelope curves for audio tracksIs required, of
For the desired envelope, then the problem can be relaxed to an optimization problem (4); in addition, because the problem is non-convex and the complexity of second-order derivation is high, a numerical method is adopted for violent search in actual processing, and different algorithms are developed for different tasks to solve based on the optimization problem; in the task of removing the massive damage of the image, the repairing can be carried out based on a chromaticity space so as to achieve the total repairing of the mildew; aiming at the jitter removal task, a repair algorithm based on symmetry axis correction can be used for repairing and correcting slight jitter and medium jitter, and meanwhile, severe jitter is greatly weakened; for the task of removing the overlapping region, the method can be based on the method for identifying and removing the overlapping region of the envelope waveform; aiming at the sound reproduction effect improvement task, an optimized envelope generation and optimization mode can be designed by referring to the quality after sound reproduction, so that missing information can be effectively supplemented under the condition that the information after sound reproduction of an audio track is not changed, the sound reproduction quality is optimized, and the method is integrated in the repairing unit 132, namely the repairing unit 132 is used for repairing the color space of the damaged image according to an ideal function to form a repairing image, and can also be used for repairing and correcting slight jitter and medium jitter of the damaged image based on a repairing algorithm of symmetry axis correction, and meanwhile, the serious jitter is greatly weakened to form the repairing image; the method can also be used for forming a repair image aiming at the damaged image of the overlapping area based on the overlapping area identification and removal method of the envelope waveform; the method can also be used for complementing missing information aiming at the damaged image to be improved in the sound reproduction effect based on an optimized envelope line generation and optimization mode under the condition that the information after sound reproduction of the audio track is not changed, and optimizing the sound reproduction quality to form a restored image, so that the sound reproduction and restoration tasks of the audio track are effectively completed; wherein, the formula involved is as follows:
wherein the content of the first and second substances,
、
、
、
is composed of
、
、
、
The corresponding function of the curve is used,
、
is the envelope function;
represent
And
the area of the air duct is surrounded by the air duct,
represents
And
a region enclosed;
in the embodiment shown in fig. 1, the sound reproduction module 140 includes a unit to be modulated 141 and a modulation unit 142;
the unit to be modulated 141 performs high-precision conversion on the envelope of the light and dark region on the restored image pixel by using the traversal pixel-level light slits, and performs amplified envelope processing on the whole domain of the restored image according to the integral to convert the restored image into a curve to be modulated;
the modulation unit 142 is configured to perform pre-emphasis or de-emphasis processing on a curve to be modulated to obtain an original recording curve, and output the original recording curve as a film audio restoration audio;
wherein the content of the first and second substances,
the modulation unit comprises an A-ring modulation and a B-ring modulation; wherein the content of the first and second substances,
the A ring modulation at least comprises an optical sound reduction head, a photoelectric transducer and a preamplifier of the projector;
the B-ring modulation at least comprises an input end of a main attenuator, an equalizer, a power amplifier, a frequency division network, a loudspeaker and a sound field from a screen to a sitting area of an audience hall;
the initial recording curve may be a time domain waveform;
in this embodiment, the audio playback module further includes a personality module, and the personality module is configured to provide different audio effects for the reproduced audio to present rich sound effects.
Specifically, the sound reproduction module 140 is configured to convert the restored image into a time domain audio that can be played, and a film track image-sound conversion algorithm based on global luminosity integration performs high-precision conversion on an envelope of a light and dark region on a film track image pixel by using a traversal pixel-level light seam, and converts the film track image into a curve to be modulated. Therefore, the sound reproduction speed and the sound reproduction quality in the whole process can be greatly improved, compared with the traditional method, the speed of the module is improved to more than 2 times, the audio effect after sound reproduction can be freely selected, and the sampling rate can reach 162.08% of that of the traditional method. Then, referring to the A-ring modulation and the B-ring modulation of the traditional method for the converted curve to be modulated, simulating a signal pre-emphasis de-emphasis process aiming at the curve to be modulated, and restoring the amplitude of the curve to be modulated to the original recording state through the processing of the process; the processed curve can be output as final movie audio;
in the traditional method, a laser scanning and photoelectric conversion element and the voice returning module 140 adopt an algorithm form to convert a moving scanning light beam into a traversing pixel-level light slit and convert the light flux of a local area into a full-local luminosity integral, so that the precision during audio sampling is effectively improved. The whole image is processed by the luminosity integral of the whole area, and the whole image is counted, amplified by an envelope curve according to the integral and converted into a curve to be modulated. The whole-area luminosity integration can effectively improve the processing speed.
Experiments prove that the sound reproduction speed is set to be twice of the normal acquisition speed, under the speed, the audio frequency after sound reproduction can still obtain better effect than the traditional method, meanwhile, because the optical imaging method can obtain the film sound track image with high resolution, the audio frequency sampling rate after sound reproduction is correspondingly greatly improved, the traditional method uses a light beam with the width of 8 mils to scan the film, and the radial width of a single frame of the film is 0.75 feet; the audio sampling rate of the obtained film audio is up to 77.8kHz, which is specified by the film industry, and the audio sampling rate of the obtained optical imaging method is improved by 62.09 percent compared with the standard of the film industry.
In addition, in the embodiment shown in fig. 1, an audio repair module 150 is also included;
the audio restoration module is used for optimizing the restored audio to form film audio, and is based on an audio restoration method based on DNN (digital noise network), namely, the audio which is played back is restored to be more suitable for human hearing.
In summary, the present invention provides an optical imaging-based cinema film sound reproduction system, including an optical imaging device, an imaging detection module, an image restoration module, and a sound reproduction module, wherein the optical imaging device is configured to generate a soundtrack image according to a soundtrack identifier on a film; the imaging detection module is used for carrying out damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology and taking the audio track image confirmed to be damaged as a damaged image; the image restoration module is used for restoring the damaged image to form a restored image; the invention relates to a method for processing a restored image, which comprises the steps of carrying out sound reproduction processing on the restored image to convert the restored image into a time domain waveform to form a restored audio, replacing an original scanning method by using an optical imaging method, carrying out digital processing on a sound reproduction process, analyzing the damage characteristics of a film, converting a moving scanning light beam into a traversal pixel level light seam on the basis of light beam scanning and photoelectric conversion in the traditional method, and converting the light flux of a local area into a full-local luminosity integral, wherein the sound reproduction speed and the sound reproduction quality can be greatly improved, compared with the traditional method, the method disclosed by the invention has the advantages that the speed is improved by more than 2 times, the audio effect after sound reproduction can be freely selected, and the sampling rate can reach 162.08% of the traditional method.
As can be seen from the above embodiments, the present invention provides a motion picture film sound reproducing system based on optical imaging, which includes an optical imaging device, an imaging detection module, an image restoration module, and a sound reproducing module, wherein the optical imaging device is configured to generate a sound track image according to a sound track identifier on a film; the imaging detection module is used for carrying out damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology and taking the audio track image confirmed to be damaged as a damaged image; the image restoration module is used for restoring the damaged image to form a restored image; the invention relates to a method for processing a restored image, which comprises the steps of carrying out sound reproduction processing on the restored image to convert the restored image into a time domain waveform to form a restored audio, replacing an original scanning method by using an optical imaging method, carrying out digital processing on a sound reproduction process, analyzing the damage characteristics of a film, converting a moving scanning light beam into a traversal pixel level light seam on the basis of light beam scanning and photoelectric conversion in the traditional method, and converting the light flux of a local area into a full-local luminosity integral, wherein the sound reproduction speed and the sound reproduction quality can be greatly improved, compared with the traditional method, the method disclosed by the invention has the advantages that the speed is improved by more than 2 times, the audio effect after sound reproduction can be freely selected, and the sampling rate can reach 162.08% of the traditional method.
As shown in fig. 2, the present invention further provides a method for playing back a motion picture film based on optical imaging, which is based on the system 100 for playing back a motion picture film based on optical imaging, and comprises:
s1: generating, by the optical imaging device, a soundtrack image from the soundtrack identification on the film;
s2: performing damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology, and taking the audio track image with damage confirmed as a damaged image;
s3: performing repair processing on the damaged image to form a repaired image;
s4: performing sound reproduction processing on the restored image to convert the restored image into a time domain waveform to form a restored audio;
the process of detecting and judging the damage of the audio track image according to the chrominance identification technology and the symmetry judgment technology and taking the confirmed damaged audio track image as the damaged image comprises the following steps:
performing preliminary detection on the audio track image to determine whether the audio track image is damaged or not, and taking the audio track image with the lowest damage as a reference frame;
extracting features of a reference frame to obtain feature parameters of the reference frame;
the method comprises the steps of carrying out gray level clustering on audio track images to obtain a gray level clustering result, obtaining an upper reference threshold and a lower reference threshold based on the gray level clustering result, comparing the upper reference threshold and the lower reference threshold with the characteristic parameters to obtain specific damage degree of each audio track image, and mapping the specific damage degree with the audio track images to form damaged images.
A process for inpainting a damaged image to form an inpainted image, comprising:
determining a first vector, a second vector, a third vector and a fourth vector in the damaged image according to an upper reference threshold and the lower reference threshold; taking an area enclosed by a first frame vector and the second vector as a first area, taking an area enclosed by a third vector and a fourth vector as a second area, constructing track functions related to the first vector, the second vector, the third vector and the fourth vector based on the first area and the second area, and determining an ideal function meeting envelope requirements of all tracks according to the track functions;
and carrying out chroma space restoration on the damaged image according to the ideal function to form a restored image.
The detailed steps are not limited herein, and reference may be made to the above-mentioned embodiments of the optical imaging-based motion picture film sound reproduction system, which are not described herein again.
As described above, the method for playing back a film based on optical imaging provided by the present invention comprises the steps of generating a soundtrack image according to a soundtrack identifier on a film by an optical imaging device, performing damage detection judgment on the soundtrack image according to a chrominance identification technology and a symmetry judgment technology, and using the soundtrack image with damage confirmed as a damaged image; then, carrying out repair processing on the damaged image to form a repaired image; and then, performing sound reproduction processing on the restored image to convert the restored image into a time domain waveform to form a restored audio, converting the moving scanning light beam into a traversal pixel level light seam and converting the light flux of a local area into a full-local luminosity integral in a sound reproduction mode on the basis of light beam scanning and photoelectric conversion in the traditional method, so that the sound reproduction speed and the sound reproduction quality can be greatly improved, compared with the traditional method, the speed is improved by more than 2 times, the audio effect after sound reproduction can be freely selected, and the sampling rate can reach 162.08% of that of the traditional method.
The optical imaging-based cinema film playback system, method proposed according to the present invention is described above by way of example with reference to the accompanying drawings. However, it will be appreciated by those skilled in the art that various modifications can be made to the optical imaging based motion picture film sound reproduction system and method of the present invention without departing from the spirit of the invention. Therefore, the scope of the present invention should be determined by the contents of the appended claims.
Claims (6)
1. A motion picture film sound reproduction system based on optical imaging, which is characterized by comprising an optical imaging device, an imaging detection module, an image restoration module and a sound reproduction module, wherein,
the optical imaging device is used for generating a music track image according to the music track identification on the film;
the imaging detection module is used for carrying out damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology, and taking the audio track image confirmed to be damaged as a damaged image; the imaging detection module comprises an initial detection unit, a threshold value acquisition unit and a damage judgment unit; wherein the content of the first and second substances,
the initial detection unit is used for carrying out initial detection on the audio track images to determine whether the audio track images are damaged or not and taking the audio track image with the lightest damage as a reference frame;
the threshold acquisition unit is used for extracting the features of the reference frame to acquire the feature parameters of the reference frame;
the damage judging unit is used for carrying out gray level clustering on the audio track images to obtain a gray level clustering result, obtaining an upper reference threshold and a lower reference threshold based on the gray level clustering result, comparing the upper reference threshold and the lower reference threshold with the characteristic parameters to obtain a specific damage degree of each audio track image, and mapping the specific damage degree with the audio track images to form damaged images;
the image restoration module is used for carrying out restoration processing on the damaged image to form a restored image; wherein the image repair module includes an area locking unit and a repair unit; wherein the content of the first and second substances,
the region locking unit is used for determining a first vector, a second vector, a third vector and a fourth vector in the damaged image according to the upper reference threshold and the lower reference threshold; taking a region enclosed by the first vector and the second vector as a first region, taking a region enclosed by the third vector and the fourth vector as a second region, constructing track functions related to the first vector, the second vector, the third vector and the fourth vector based on the first region and the second region, and determining an ideal function meeting all track envelope requirements according to the track functions;
the restoration unit is used for restoring the color space of the damaged image according to the ideal function to form a restored image;
and the sound reproduction module is used for carrying out sound reproduction processing on the restored image so as to convert the restored image into a time domain waveform and form a restored audio.
2. The optical imaging-based motion picture film soundtrack system of claim 1, further comprising an audio repair module;
the audio restoration module is used for optimizing the restored audio to form film audio.
3. The optical imaging-based motion picture film soundtrack system of claim 2,
the film includes at least a damaged motion picture soundtrack film, a nitrate film, an acetate film.
4. The motion picture film based audio reproduction system on optical imaging of claim 3, wherein said audio reproduction module comprises a unit to be modulated and a modulation unit;
the unit to be modulated performs high-precision conversion on the envelope curve of a light and dark area on the repaired image pixel by using a traversing pixel-level light slit, and performs amplified envelope curve processing on the whole domain of the repaired image according to integration so as to convert the repaired image into a curve to be modulated;
the modulation unit is used for carrying out pre-emphasis or de-emphasis processing on the curve to be modulated to obtain an original recording curve, and the original recording curve is used as film audio frequency reduction audio frequency to be output; wherein, the first and the second end of the pipe are connected with each other,
the modulation unit comprises A-ring modulation and B-ring modulation; wherein the content of the first and second substances,
the A ring modulation at least comprises an optical sound reduction head, a photoelectric transducer and a preamplifier of the projector;
the B-ring modulation at least comprises an input end of a main attenuator, an equalizer, a power amplifier, a frequency division network, a loudspeaker and a sound field from a screen to a sitting area of an audience hall;
the original recording curve is a time domain waveform.
5. The motion picture film sound reproduction system based on optical imaging of claim 4,
the audio reproduction module also comprises a personality module, and the personality module is used for providing different audio effects for the reproduced audio.
6. An optical imaging-based cinema film sound reproduction method, characterized in that, the optically imaged cinema film sound reproduction processing is carried out based on the optical imaging-based cinema film sound reproduction system according to any one of claims 1-5, comprising:
generating, by the optical imaging device, a soundtrack image from the soundtrack identification on the film;
performing damage detection judgment on the audio track image according to a chrominance identification technology and a symmetry judgment technology, and taking the audio track image with damage confirmed as a damaged image; the process of detecting and judging the damage of the audio track image according to the chrominance identification technology and the symmetry judgment technology and taking the confirmed damaged audio track image as a damaged image comprises the following steps:
performing preliminary detection on the audio track images to determine whether the audio track images are damaged or not, and taking the audio track image with the lowest damage as a reference frame;
extracting features of the reference frame to obtain feature parameters of the reference frame;
performing gray level clustering on the audio track images to obtain a gray level clustering result, obtaining an upper reference threshold and a lower reference threshold based on the gray level clustering result, comparing the upper reference threshold and the lower reference threshold with the characteristic parameters to obtain a specific damage degree of each audio track image, and mapping the specific damage degree with the audio track images to form damaged images;
performing a repair process on the damaged image to form a repaired image; wherein the process of performing a repair process on the damaged image to form a repaired image comprises:
determining a first vector, a second vector, a third vector and a fourth vector in the damaged image according to the upper reference threshold and the lower reference threshold; taking an area surrounded by the first vector and the second vector as a first area, taking an area surrounded by the third vector and the fourth vector as a second area, constructing track functions related to the first vector, the second vector, the third vector and the fourth vector based on the first area and the second area, and determining an ideal function meeting all track envelope requirements according to the track functions;
carrying out chroma space restoration on the damaged image according to the ideal function to form a restored image;
and performing sound reproduction processing on the restored image to convert the restored image into a time domain waveform and form a restored audio.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111616095.6A CN113989281B (en) | 2021-12-28 | 2021-12-28 | Film sound reproduction system and method based on optical imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111616095.6A CN113989281B (en) | 2021-12-28 | 2021-12-28 | Film sound reproduction system and method based on optical imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113989281A CN113989281A (en) | 2022-01-28 |
| CN113989281B true CN113989281B (en) | 2022-05-31 |
Family
ID=79734610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111616095.6A Active CN113989281B (en) | 2021-12-28 | 2021-12-28 | Film sound reproduction system and method based on optical imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113989281B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1791937A (en) * | 2003-05-20 | 2006-06-21 | 弗兰霍菲尔运输应用研究公司 | Apparatus and method for synchronizing an audio signal with a film |
| CN102129670A (en) * | 2011-02-25 | 2011-07-20 | 上海交通大学 | Method for detecting and repairing movie scratch damage |
| CN112308790A (en) * | 2020-07-29 | 2021-02-02 | 中科汇金数字科技(北京)有限公司 | Method for extracting and analyzing track of sound groove of old record |
| CN113344813A (en) * | 2021-06-02 | 2021-09-03 | 中国工商银行股份有限公司 | Image restoration method and device and server |
-
2021
- 2021-12-28 CN CN202111616095.6A patent/CN113989281B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1791937A (en) * | 2003-05-20 | 2006-06-21 | 弗兰霍菲尔运输应用研究公司 | Apparatus and method for synchronizing an audio signal with a film |
| CN102129670A (en) * | 2011-02-25 | 2011-07-20 | 上海交通大学 | Method for detecting and repairing movie scratch damage |
| CN112308790A (en) * | 2020-07-29 | 2021-02-02 | 中科汇金数字科技(北京)有限公司 | Method for extracting and analyzing track of sound groove of old record |
| CN113344813A (en) * | 2021-06-02 | 2021-09-03 | 中国工商银行股份有限公司 | Image restoration method and device and server |
Non-Patent Citations (2)
| Title |
|---|
| An Adaptive Face Image Inpainting Algorithm Based on Feature Symmetry;Zuodong Niu;《Symmetry》;20200131;第1-3页 * |
| An Adaptive Image Inpainting Method Based on the Weighted Mean;Nguyen Hoang Hai;《Informatica》;20191231;第1-3页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113989281A (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5752133B2 (en) | Method and system for converting a digital image from a low dynamic range (LDR) image to a high dynamic range (HDR) image | |
| CN100459685C (en) | Information processing apparatus, imaging apparatus, information processing method, and program | |
| US20090010623A1 (en) | Information encoding method, information playback method, and information storage medium | |
| CN101490747B (en) | Audio watermarking technique for motion picture presentations | |
| CN113989281B (en) | Film sound reproduction system and method based on optical imaging | |
| JP2006340066A (en) | Moving image encoder, moving image encoding method and recording and reproducing method | |
| JP5497420B2 (en) | Image processing apparatus and imaging apparatus | |
| Svanberg | The EDCF guide to digital cinema production | |
| JP2004120606A (en) | Video reproducing device | |
| JP2005503635A (en) | Repair of variable area film soundtrack | |
| JP2007110189A (en) | Image quality evaluation apparatus and image quality evaluation method | |
| Lauber | History and Ethics of Film Restoration | |
| JP4551734B2 (en) | Variable speed reproduction apparatus and variable speed reproduction method | |
| EP2805230A1 (en) | Method for backing up digital cinematographic content | |
| Kurttila | Digital Film Restoration and Remastering | |
| JP2010206609A (en) | Image playback apparatus, control method of the same, and program | |
| Liu et al. | A novel audio extraction and restoration system for optical soundtrack | |
| Behura et al. | Impact Of Digital Cinematography On Craftsmanship | |
| Lauber | Archival Ethics and Digital Film Restoration | |
| Clark et al. | The Status of Cinematography Today | |
| JP2006186786A5 (en) | ||
| Stotzer et al. | Phonographic sound extraction using photography and signal processing | |
| Dawson et al. | The 6K restoration of Orson Welles’ Chimes at Midnight | |
| Taquet et al. | Detection and correction of under-/overexposed optical soundtracks by coupling image and audio signal processing | |
| Besserer et al. | Restoration of under/overexposed optical soundtracks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |