JP2005341122A - Device and method for recording and reproducing image, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To faithfully record and reproduce color information of ambient light in photography along with image information and audio information. <P>SOLUTION: A video signal acquired from an imaging apparatus 1, a sound signal acquired from a microphone 4, and a lighting signal acquired from a colorimeter 6 are digitized, and are supplied to a digital signal processing circuit 8. The signals, supplied to the digital signal processing circuit 8, are multiplexed for performing prescribed signal processing, such as addition of error correction codes, and are supplied to a photography information recording device 9 for recording. An image-reproducing device performs the reproduction output of a signal, based on the signal recorded to the photography information recording device 9. In this case, a luminaire is controlled, according to the lighting signal recorded along with the video and sound signals, thus reproducing a lighting environment in the photography at reproduction. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention records illumination environment information related to ambient light at the time of camera shooting on a recording medium together with a video signal and an audio signal, reproduces the illumination information in synchronization with the recorded video signal and audio signal, and displays video. At the same time, it relates to a technology that makes it possible to reproduce the state of ambient light during shooting.

  For example, Patent Document 1 discloses a video recording apparatus and a playback apparatus that record information at the time of video shooting and that can reproduce the environment at the time of video playback. This video recording / reproducing apparatus records video and audio by a CCD and a microphone at the time of video shooting, performs predetermined signal processing, and records them on a magnetic tape. Simultaneously with video shooting, the environment detection device detects the environment at the time of shooting and acquires environmental information. The acquired environmental information is subjected to predetermined signal processing by a circuit and recorded in a predetermined area of the magnetic tape.

When reproducing recorded information for spectators, etc., video / audio is reproduced with a monitor or speaker, for example, temperature and humidity are reproduced with an air conditioner, wind direction and wind speed are reproduced with a blower, and a telescopic column is controlled. The column control device that reproduces the inclination and acceleration of the small room in which the chair where the audience sits is installed, and the illuminance is reproduced by the illuminating lamp. In this way, the scene environment at the time of shooting is reproduced together with video / audio, so that the audience can experience a higher sense of reality.
JP-A-9-205607

  However, in the technique of Patent Document 1, it is possible to detect illuminance information and color temperature information using a color temperature illuminometer, but since there is no information on chromaticity values, it is used for stage lighting or the like. It is impossible to detect illumination that reproduces the three primary colors of RGB. Moreover, even if the color temperature information is detected by the color temperature illuminometer, the specific control method of the illumination lamp with the color filter is not described, and the color of the illumination at the time of photographing cannot be reproduced.

  The present invention has been made in view of the above-mentioned problems, and the object thereof is to faithfully record color information of ambient light at the time of photographing together with video information and audio information, and recorded at the time of reproducing the photographing information. By controlling the lighting device based on the lighting information, it is possible to faithfully reproduce the color of the ambient light at the time of shooting, and to make you feel as if you are on the spot. A recording apparatus and a recording method, a video reproducing apparatus and a reproducing method, a program, and a recording medium are provided.

  The first technical means is a video recording apparatus that records a video signal and an audio signal acquired by shooting on a recording medium, and includes illumination environment information acquisition means for acquiring illumination information related to ambient light at the time of shooting, The illumination environment information acquired by the illumination environment information acquisition means is recorded on a recording medium together with a video signal and an audio signal.

  A second technical means includes, in the first technical means, an imaging device that acquires a video signal and a microphone that acquires an audio signal, and the video signal output from the imaging device and the audio signal output from the microphone. Are a video signal and an audio signal to be recorded on a recording medium.

  The third technical means performs signal processing and separation of the recording information of the recording medium on which the video signal and the audio signal and the illumination environment information regarding the ambient light at the time of shooting when the video signal and the audio signal are acquired are recorded. Signal processing means, display means for performing reproduction processing on the video signal separated by the signal processing means, and displaying; audio output means for performing reproduction processing on the audio signal separated by the signal processing means and outputting sound; And a reproduction illumination conversion means for generating a control signal for reproducing the illumination environment at the time of photographing based on the illumination environment information separated by the signal processing means.

  According to a fourth technical means, in the third technical means, the reproduction illumination conversion means described above generates a control signal for controlling the illumination environment by converting chromaticity value information in the uniform color space into RGB values. One of the control process and the process for controlling the illumination environment can be selected by changing the chromaticity value information in the uniform color space to the correlated color temperature value.

  The fifth technical means holds in advance a conversion table in which the relationship between the brightness of each RGB illumination and the chromaticity value in the uniform color space is recorded in the fourth technical means, and the reproduction illumination conversion means A chromaticity value in the uniform color space is converted into an RGB value using a table.

  The sixth technical means obtains a video signal and an audio signal by photographing, obtains illumination information related to ambient light at the time of photographing, and illumination environment information obtained in the illumination environment information obtaining step. And a recording step of recording on a recording medium together with a video signal and an audio signal.

  The seventh technical means performs signal processing on the recording information of the recording medium on which the video signal and the audio signal and the illumination environment information regarding the ambient light at the time of photographing when the video signal and the audio signal are acquired are recorded. A separation signal processing step, and a reproduction output step for reproducing and outputting the respective signals separated by the signal processing step. The reproduction output step is based on the illumination environment information separated by the signal processing step. It includes a reproduction illumination conversion step for generating a control signal for reproducing the illumination environment.

  According to an eighth technical means, in the seventh technical means, the reproduction conversion step for reproduction described above generates a control signal for controlling the illumination environment by converting the chromaticity value information in the uniform color space into an RGB value. It is possible to select one of a process to generate and a process to generate a control signal for controlling the illumination environment by changing the chromaticity value information in the uniform color space to a correlated color temperature value. It is a thing.

  According to a ninth technical means, in the eighth technical means, the reproduction illumination conversion step uses a conversion table in which the relationship between the brightness of each RGB illumination and the chromaticity value in the uniform color space is recorded. It is characterized by converting the chromaticity value of the space into an RGB value.

  The tenth technical means is a program that causes a computer to execute the reproduction conversion step for reproduction in any one of the seventh to ninth technical means.

  The eleventh technical means is a recording medium in which the tenth program is recorded in a computer readable manner.

  According to the present invention, the color information at the shooting site at the time of video shooting is recorded together with the video information and audio information by the video recording device, so that the color information of the ambient light at the time of shooting can be faithfully recorded. Also, since the lighting device is controlled based on the recorded lighting information when shooting information is played back, the color of the ambient light at the time of shooting can be faithfully reproduced, as if it were as if it were on the spot. It becomes possible to feel.

  FIG. 1 is a block diagram showing an embodiment of a video recording apparatus according to the present invention. The image pickup apparatus 1 includes, for example, an optical system including a zoom lens, a focus lens, a diaphragm, and the like, and an image pickup element using a CCD, and a subject image is incident on the CCD through the optical system. The subject image incident on the CCD is converted into an electric signal to be a video signal, supplied to the A / D converter 2 and converted into a digital signal.

  The video signal converted into a digital signal by the A / D converter 2 is supplied to the encoder 3 and compressed and encoded. The compression encoding method in the encoder 3 includes MPEG-2, MPEG-4, MPEG-4 ASP, JPEG, Motion-JPEG, H.264, and so on. 263, H.M. H.264 or the like can be applied, but the present invention is not limited to this, and an optimal compression encoding method can be adopted as appropriate. The video signal compressed and encoded by the encoder 3 is supplied to the digital signal processing circuit 8.

  On the other hand, the sound at the time of photographing is converted into an electric signal by the microphone 4 to be an audio signal. This audio signal is supplied to the A / D converter 5 and sampled at a predetermined sampling frequency, for example, and converted into a digital signal. The audio signal converted into the digital signal is supplied to the digital signal processing circuit 8.

  The image pickup apparatus 1 and the microphone 4 provided with the CCD and the processing system of these signals are configured to be capable of acquiring images, as represented by a movie function of a video camera or a digital camera, for example. The signals of each other are synchronized.

  The colorimeter 6 is an apparatus for measuring an illumination signal such as brightness and color information of the surrounding illumination environment at the time of photographing in photographing of an image using the imaging device 1 and the microphone 4 described above. This corresponds to environmental information acquisition means. For example, when photographing outdoors, the illuminometer 6 acquires illumination environment information such as ambient brightness and color information. The illumination environment information acquired by the colorimeter 6 can be output to the outside as an analog signal or a digital signal. Here, the output signal output from the colorimeter 6 is supplied to an A / D and MPU unit 7 comprising a microprocessor. The A / D and MPU unit 7 has an A / D conversion function. When an analog signal is input, the A / D and MPU unit 7 converts the analog signal into a digital signal.

  Here, the output signal from the colorimeter 6 is continuously supplied to the A / D and MPU unit 7, but the A / D and MPU unit 7 outputs a signal at a predetermined interval. Here, by controlling the A / D and MPU unit 7, for example, signal output is performed once every several video signal frames. Signals output from the A / D and MPU unit 7 at predetermined intervals are supplied to the digital signal processing circuit 8.

  Through the above processing, the digital signal processing circuit 8 is supplied with the video signal, the audio signal, and the illumination signal as digital signals. For example, the video signal is subjected to predetermined processing such as compression coding and supplied to the digital signal processing circuit 8. These signals supplied to the digital signal processing circuit 8 are multiplexed, subjected to predetermined signal processing such as addition of an error correction code, and supplied to the photographing information recording device 9.

  In the photographic information recording device 9, the supplied signal is subjected to recording signal processing such as recording encoding and modulation, and recorded on a recording medium such as a magnetic tape. When recording is performed on the magnetic tape, a subcode area is secured on the magnetic tape together with recording of the video signal and the audio signal, and the illumination signal is recorded as metadata. Of course, the recording medium is not limited to the magnetic tape described above, and a disk medium such as a magnetic disk, an optical disk, or a magneto-optical disk, and other recording media can be appropriately applied.

FIG. 2 is a block diagram showing an embodiment of a video reproduction apparatus according to the present invention. The video playback apparatus according to the present embodiment is a playback apparatus that inputs signals recorded by the video recording apparatus shown in FIG. 1 and performs playback processing for each signal and outputs the signals.
The photographic information input unit 10 receives the multiplexed signal from the recording medium on which the digital illumination signal is multiplexed and recorded together with the digital video signal and the digital audio signal as described above. That is, the photographing information input unit 10 may be a reading device that reads a signal from the recording medium, or may be an input interface that inputs the multiplexed signal as described above from the outside.
Further, the video reproduction apparatus may be provided with a recording apparatus for inputting a multiplexed signal acquired by the video recording apparatus as shown in FIG. 1 and recording the multiplexed signal on a recording medium such as a fixed magnetic disk. In this case, the imaging information input unit 10 reads the multiplexed signal from the recording device and outputs the multiplexed signal to the digital signal processing circuit 11.

  In the video reproduction apparatus, the signal input from the photographing information input unit 10 is supplied to the digital signal processing circuit 11. In the digital signal processing circuit 11, the supplied reproduction signal is subjected to processing such as error correction, and separated into a digital video signal, a digital audio signal, and a digital illumination signal multiplexed at the time of recording. That is, the digital signal processing circuit 11 corresponds to the signal processing means of the present invention for performing signal processing on the recording information of the recording medium and separating it.

  Of the signals separated by the digital signal processing circuit 11, the digital video signal is supplied to the decoder 12 and decompressed, and then supplied to the D / A converter 13 to be converted into an analog video signal. This analog signal is sent to the display device 14 functioning as the display means of the present invention and reproduced as a video.

  Of the signals separated by the digital signal processing circuit 11, the digital audio signal is supplied to the D / A converter 15, where it is converted into an analog signal to be an analog audio signal. This analog audio signal is reproduced as audio by the speaker 16. The speaker 16 functions as an audio output unit of the present invention.

  Further, among the signals separated by the digital signal processing circuit 11, the illumination signal is supplied to the MPU and D / A unit 17. The MPU and D / A unit 17 is composed of a microprocessor and has a D / A conversion function. The illumination signal input to the MPU / D / A unit 17 is composed of colorimeter measurement data, and is converted into an analog signal by the MPU / D / A unit 17. The converted analog signal is supplied to the reproduction illumination conversion device 18, and is supplied as a control signal to the illumination device 19 after performing the reproduction illumination conversion processing described later. The illumination device 19 reproduces the color information of the illumination in the shooting environment by controlling the color of the illumination for reproduction based on the control signal. The reproduction illumination conversion device 18 corresponds to the reproduction illumination conversion unit of the present invention that generates a control signal for reproducing the illumination environment at the time of photographing based on the illumination environment information separated by the signal processing unit.

  Next, the procedure of the reproduction illumination conversion process will be described with reference to the flowchart of FIG. 2 is supplied with chromaticity values (x, y, z) as an example of photographing illumination light data (step S1). The reproduction illumination conversion device 18 converts the chromaticity values (x, y, z) into the RGB values (R, G, B) of the reproduction illumination when color illumination is used for the reproduction illumination (step S2). , S3). The chromaticity value (x, y, z) is an example of a uniform color space that does not depend on the device, and may be another uniform color space. When general illumination color temperature information is used without using color illumination, the chromaticity value (x, y) is converted to the correlated color temperature value of reproduction illumination (steps S2 and S4). The reproduction illumination conversion device 18 is one of a process of converting the chromaticity value information in the uniform color space into an RGB value and a process of changing the chromaticity value information in the uniform color space to a correlated color temperature value. Can be selected.

  Here, a case where RGB three primary color illumination is used as an example of color illumination will be described. Examples of the RGB three primary color type illumination include a color LED and a color fluorescent lamp, but are not limited thereto. Then, the illumination intensity of each RGB is divided into 256 levels to emit light, the chromaticity values (x, y, z) at that time are measured, an RGB-xyz conversion table is created, and an illumination conversion apparatus for reproduction 18 is stored.

  An example of a conversion table creation method will be described using the case where the present invention depicted in FIG. 4 is applied to a home theater. In FIG. 4, the display device 14 and the speaker 16 are disposed at a predetermined position in the room, and a reproduction illumination device 19 is disposed above the speaker 16. In this room, a sofa 20 is arranged facing forward so that a viewer can sit there. Then, a standard white plate (not shown) is installed instead of the display device 14 at the position where the display device 14 in front of the viewer is arranged, and a color meter is installed at the position of the sofa 20.

  The illumination device 19 on the speaker 16 is irradiated with 256 levels of RGB illumination intensity toward the standard white plate, and the chromaticity values (x, y, z) at that time are measured with a colorimeter. To do. Then, an RGB-xyz conversion table is created based on the chromaticity values obtained by the colorimeter. Thereby, the RGB value (R, G, B) of the color illumination corresponding to the chromaticity value (x, y, z) of the photographing illumination light can be determined.

  The reproduction illumination conversion device 18 determines the RGB value based on the obtained chromaticity value, and supplies the control signal to the illumination device 19 to emit light, thereby reproducing the illumination state at the time of photographing. . Here, the conversion from RGB to xyz is not limited to the use of the RGB-xyz conversion table, and matrix conversion may be performed. The following is an example of matrix conversion.

  FIG. 5 is an xy chromaticity diagram showing the color range of the reproduction illumination according to this embodiment. FIG. 6 shows a color temperature curve and a correlated color temperature curve on the xy chromaticity diagram. As shown in FIG. 5 and FIG. 6, when the color range for reproduction illumination is made wider than the color temperature curve, correlation temperature curve, and color range that can be reproduced by the current television, it is applied to a broadcasting system. In addition, it is possible to reproduce the entire color range of the photographing illumination light.

When general illumination color temperature information is used, the chromaticity value (x, y) is converted into the correlated color temperature value T of the reproduction illumination as described above. At this time, as a simple calculation method for obtaining the correlated color temperature value T from the chromaticity value (x, y), there is a McCamy correlated color temperature conversion formula described below. (Color Res. Application 17, 142-144 (1992))
T = −437n ³ + 3601n ² −6861n + 5514.31 (2)
Then, the reproduction illumination conversion device 18 selects the illumination having the correlated color temperature T corresponding to the chromaticity value (x, y) of the photographic illumination light, supplies the control signal, and emits the light, so The lighting state can be reproduced.

  With the system as described above, for example, for viewers sitting on the sofa 20 in the room shown in FIG. 4, not only video and audio, but also the color of the lighting at the scene taken is reproduced. High sense of presence can be provided.

  In the above-described embodiment, the color of the illumination is measured by the colorimeter 6 at the time of shooting, and the measured information is supplied to the digital signal processing circuit 8 via the A / D and MPU unit 7 to record the shooting information. Although recorded in the apparatus 9, the present invention is not limited to such a configuration. For example, as another example, the video recording apparatus having the digital signal processing function according to the present invention as described above is provided with an input unit that allows a user to input data, and another imaging apparatus or microphone, a color meter, and the like. The environment information acquired by the above may be manually input from this input unit. At this time, the input environmental information may not be actually detected.

  In addition, the video recording method according to the present invention acquires the video signal and the audio signal by shooting, and acquires the lighting environment information acquisition step of acquiring the illumination information related to the ambient light at the time of shooting, and the lighting information acquisition step. A recording step of recording the lighting environment information on a recording medium together with the video signal and the audio signal. The illumination environment information acquisition step corresponds to processing for acquiring a video signal, an audio signal, and illumination information by the imaging device 1, the microphone 4, and the colorimeter 6 shown in FIG. Further, the recording step includes an A / D converter 2 for video signals, an encoder 3, an A / D converter 5 for audio signals 5, an A / D for illumination information, an MPU unit 7 in FIG. The processing by the digital signal processing circuit 8 for performing the signal processing and the processing for recording the signal processed signal in the photographing information recording device 9 are included.

  In addition, the video reproduction method according to the present invention includes recording information on a recording medium on which a video signal and an audio signal and illumination environment information related to ambient light at the time of shooting when the video signal and the audio signal are acquired are recorded. It has a signal processing step for signal processing and separation, and a reproduction output step for reproducing and outputting each signal separated by the signal processing step. The reproduction output step includes a reproduction illumination conversion step for generating a control signal for reproducing the illumination environment at the time of photographing based on the illumination environment information separated by the signal processing step.

  The above-described signal processing step corresponds to a process of separating each signal by performing predetermined signal processing such as error correction on the signal input to the imaging information input unit 10 in the digital signal processing circuit 11 shown in FIG. . The reproduction output step is a step for performing reproduction processing of each signal separated by the digital signal processing circuit 11. Here, the audio signal includes processing by the decoder 12 and the D / A converter 13 and display output processing by the display device 14, and the audio signal is processed by the D / A converter 15 and output from the speaker 16. Is included. Further, the illumination signal includes processing by the MPU and D / A unit 17 and generation processing of an illumination control signal by the reproduction illumination conversion device 18. The illumination device controls illumination according to the control signal generated by the reproduction illumination conversion device 18.

  The video playback method of the present invention can be realized by executing a program by a computer. The program according to the present invention can be recorded on a computer-readable recording medium and can be distributed as a portable recording medium, or can be provided to a user via an electric communication line.

  The recording medium on which the program is recorded is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy (registered trademark) disk or a hard disk, or a CD-ROM / MO / Disk system of optical disks such as MD / DVD, card system such as IC card (including memory card) / optical card, or mask ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory) flash It may be a medium that carries a fixed program including a semiconductor memory such as a ROM.

In either case, the stored program may be configured to be accessed and executed by the microprocessor, or the program read into the program storage area of the microcomputer is downloaded and the program is stored. An executed method may be used. The video recording apparatus and the video playback apparatus of the present invention may be configured to be connected to a communication network including the Internet, and the program may be downloaded from the communication network. This download program may be stored in the main device in advance, or may be installed from another recording medium.
Finally, the embodiment described above does not limit the scope of the present invention, and various modifications can be made within the scope of the present invention.

It is a block diagram which shows one Embodiment of the video recording apparatus by this invention. It is a block diagram which shows one Embodiment of the video reproduction apparatus by this invention. It is a flowchart for demonstrating the procedure of the illumination conversion process for reproduction. It is a figure which shows an example of the production method of a conversion table. It is a chromaticity diagram showing the color range of the illumination for reproduction according to the present embodiment. It is a figure which shows the color temperature curve on a xy chromaticity diagram, and a correlation color temperature curve.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging device, 2 ... A / D converter, 3 ... Encoder, 4 ... Microphone, 5 ... A / D converter, 6 ... Colorimeter, 7 ... A / D, MPU part, 8 ... Digital signal processing circuit, DESCRIPTION OF SYMBOLS 9 ... Shooting information recording device, 10 ... Shooting information input part, 11 ... Digital signal processing circuit, 12 ... Decoder, 13 ... D / A converter, 14 ... Display apparatus, 15 ... D / A converter, 16 ... Speaker 17 ... MPU, D / A section, 18 ... Reproduction lighting conversion device, 19 ... Lighting device, 20 ... Sofa.

Claims (11)

  A video recording apparatus for recording a video signal and an audio signal acquired by shooting on a recording medium, comprising: illumination environment information acquiring means for acquiring illumination information regarding ambient light at the time of shooting, the lighting environment information acquiring means The video recording apparatus, wherein the lighting environment information acquired in (1) is recorded on the recording medium together with the video signal and the audio signal.   The video recording apparatus according to claim 1, comprising: an imaging device that acquires a video signal; and a microphone that acquires an audio signal; and a video signal output from the imaging device and an audio signal output from the microphone. Is a video signal and an audio signal to be recorded on the recording medium.   Signal processing means for performing signal processing and separating the recording information of the recording medium on which the video signal and the audio signal and the illumination environment information regarding the ambient light at the time of photographing when the video signal and the audio signal are acquired are recorded; Display means for performing reproduction processing on the video signal separated by the signal processing means and displaying; audio output means for performing reproduction processing on the audio signal separated by the signal processing means and outputting sound; and the signal processing And a reproduction illumination conversion means for generating a control signal for reproducing the illumination environment at the time of photographing based on the illumination environment information separated by the means.   4. The video reproduction apparatus according to claim 3, wherein the reproduction illumination conversion means controls a control signal for controlling an illumination environment by converting chromaticity value information in a uniform color space into an RGB value. A video reproduction apparatus characterized in that any one of processing for controlling the illumination environment can be selected by changing chromaticity value information in the uniform color space to a correlated color temperature value.   5. The video reproduction apparatus according to claim 4, wherein a conversion table in which a relationship between brightness of RGB illumination and a chromaticity value in a uniform color space is recorded in advance is stored, and the reproduction illumination conversion unit includes the conversion table. A video reproducing apparatus, wherein the chromaticity value of the uniform color space is converted into an RGB value by using a color space.   The video signal and the audio signal are acquired by shooting, the lighting environment information acquisition step for acquiring the illumination information regarding the ambient light at the time of the shooting, and the lighting environment information acquired in the lighting environment information acquisition step are the video signal. And a recording step of recording on the recording medium together with an audio signal.   A signal processing step of performing signal processing and separating the recording information of the recording medium on which the video signal and the audio signal and the illumination environment information regarding the ambient light at the time of photographing when the video signal and the audio signal are acquired are recorded; A reproduction output step for reproducing and outputting the respective signals separated by the signal processing step, and the reproduction output step is based on the illumination environment information separated by the signal processing step. A reproduction method comprising: a reproduction illumination conversion step for generating a control signal for reproducing the image.   8. The video reproduction method according to claim 7, wherein the reproduction illumination conversion step includes a process of generating a control signal for controlling an illumination environment by converting chromaticity value information in the uniform color space into an RGB value. And a process for generating a control signal for controlling the illumination environment by changing chromaticity value information in the uniform color space to a correlated color temperature value. Playback method.   9. The video reproduction method according to claim 8, wherein the reproduction illumination conversion step uses the conversion table in which the relationship between the brightness of each RGB illumination and the chromaticity value in the uniform color space is recorded. A video reproduction method characterized by converting the chromaticity value of the image into an RGB value.   The program which makes a computer perform the illumination conversion step for reproduction in the video recording method of any one of Claim 7 thru | or 9.   A recording medium comprising the program according to claim 10 recorded in a computer-readable manner.

Images