JP4651307B2 - Illumination environment reproduction apparatus and method, and video reproduction apparatus - Google Patents

Description

The present invention controls the illumination environment at the time of shooting by controlling the lighting device based on the lighting environment information regarding the lighting environment at the time of shooting when the video signal and the audio signal are acquired at the time of reproducing the video signal and the audio signal. It is related to the technology that can reproduce.

  For example, Patent Document 1 discloses a video recording apparatus and a playback apparatus that record information at the time of video shooting and that can reproduce the environment at the time of video playback. This video recording / reproducing apparatus records video and audio by a CCD and a microphone at the time of video shooting, performs predetermined signal processing, and records them on a magnetic tape. Simultaneously with video shooting, the environment detection device detects the environment at the time of shooting and acquires environmental information. The acquired environmental information is subjected to predetermined signal processing by a circuit and recorded in a predetermined area of the magnetic tape.

When reproducing recorded information for spectators, etc., video / audio is reproduced with a monitor or speaker, for example, temperature and humidity are reproduced with an air conditioner, wind direction and wind speed are reproduced with a blower, and a telescopic column is controlled. The column control device that reproduces the inclination and acceleration of the small room in which the chair where the audience sits is installed, and the illuminance is reproduced by the illuminating lamp. In this way, the scene environment at the time of shooting is reproduced together with video / audio, so that the audience can experience a higher sense of reality.
JP-A-9-205607

However, in the technique of Patent Document 1, it is possible to detect illuminance information and color temperature information using a color temperature illuminometer, but since there is no information on chromaticity values, it is used for stage lighting or the like. It is impossible to detect illumination that reproduces the three primary colors of RGB. Moreover, even if the color temperature information is detected by the color temperature illuminometer, the specific control method of the illumination lamp with the color filter is not described, and the color of the illumination at the time of photographing cannot be reproduced.

The present invention has been made in view of the above-described problems, and its purpose is to control the lighting device based on the environmental lighting information related to ambient light at the time of shooting when reproducing shooting information, thereby An object of the present invention is to provide an illumination environment reproduction device and method, and an image reproduction device that can faithfully reproduce the color of light and feel a high sense of presence as if the person is on the spot.

The first technical means controls the lighting device based on the lighting environment information related to the lighting environment at the time of shooting when the video signal and the audio signal are acquired during the reproduction of the video signal and the audio signal. In a lighting environment reproduction device that reproduces a lighting environment at the time of shooting, when a color lighting device controlled by RGB values is used as the lighting device , the lighting environment information represented by chromaticity values is converted to RGB values. When the lighting device controlled by a color temperature value is used as the lighting device, the lighting environment information represented by the chromaticity value is converted into a color temperature value. It outputs to the said illuminating device, It is characterized by the above-mentioned .

The second technical means includes the illumination environment reproduction device according to the first technical means, display means for performing reproduction processing on the video signal and displaying video, and audio for performing reproduction processing on the audio signal and outputting sound. Output means .

The third technical means controls the lighting device based on the lighting environment information related to the lighting environment at the time of shooting when the video signal and the audio signal are acquired during the reproduction of the video signal and the audio signal. In a lighting environment reproduction method for reproducing a lighting environment at the time of shooting, when a color lighting device controlled by RGB values is used as the lighting device , the lighting environment information represented by chromaticity values is converted to RGB values. When the lighting device controlled by a color temperature value is used as the lighting device, the lighting environment information represented by the chromaticity value is converted into a color temperature value. It outputs to the said illuminating device, It is characterized by the above-mentioned .

According to the present invention, at the time of photographing information reproduction, by controlling the illumination device on the basis of the ambient light information on ambient light shooting Kageji, can be faithfully reproduced color only ambient light at the time of shooting, as if It is possible to feel a high sense of presence as if you were on the spot .

  FIG. 1 is a block diagram showing an embodiment of a video recording apparatus according to the present invention. The image pickup apparatus 1 includes, for example, an optical system including a zoom lens, a focus lens, a diaphragm, and the like, and an image pickup element using a CCD, and a subject image is incident on the CCD through the optical system. The subject image incident on the CCD is converted into an electric signal to be a video signal, supplied to the A / D converter 2 and converted into a digital signal.

  The video signal converted into a digital signal by the A / D converter 2 is supplied to the encoder 3 and compressed and encoded. The compression encoding method in the encoder 3 includes MPEG-2, MPEG-4, MPEG-4 ASP, JPEG, Motion-JPEG, H.264, and so on. 263, H.M. H.264 or the like can be applied, but the present invention is not limited to this, and an optimal compression encoding method can be adopted as appropriate. The video signal compressed and encoded by the encoder 3 is supplied to the digital signal processing circuit 8.

  On the other hand, the sound at the time of photographing is converted into an electric signal by the microphone 4 to be an audio signal. This audio signal is supplied to the A / D converter 5 and sampled at a predetermined sampling frequency, for example, and converted into a digital signal. The audio signal converted into the digital signal is supplied to the digital signal processing circuit 8.

  The image pickup apparatus 1 and the microphone 4 provided with the CCD and the processing system of these signals are configured to be capable of acquiring images, as represented by a movie function of a video camera or a digital camera, for example. The signals of each other are synchronized.

Colorimeter 6, in the imaging of the image with the imaging device 1 and the microphone 4 above, Ru apparatus der for measuring illumination signal such brightness and color information of the illumination environment around the time of shooting. For example, when photographing outdoors, the illuminometer 6 acquires illumination environment information such as ambient brightness and color information. The illumination environment information acquired by the colorimeter 6 can be output to the outside as an analog signal or a digital signal. Here, the output signal output from the colorimeter 6 is supplied to an A / D and MPU unit 7 comprising a microprocessor. The A / D and MPU unit 7 has an A / D conversion function. When an analog signal is input, the A / D and MPU unit 7 converts the analog signal into a digital signal.

  Here, the output signal from the colorimeter 6 is continuously supplied to the A / D and MPU unit 7, but the A / D and MPU unit 7 outputs a signal at a predetermined interval. Here, by controlling the A / D and MPU unit 7, for example, signal output is performed once every several video signal frames. Signals output from the A / D and MPU unit 7 at predetermined intervals are supplied to the digital signal processing circuit 8.

  Through the above processing, the digital signal processing circuit 8 is supplied with the video signal, the audio signal, and the illumination signal as digital signals. For example, the video signal is subjected to predetermined processing such as compression coding and supplied to the digital signal processing circuit 8. These signals supplied to the digital signal processing circuit 8 are multiplexed, subjected to predetermined signal processing such as addition of an error correction code, and supplied to the photographing information recording device 9.

  In the photographic information recording device 9, the supplied signal is subjected to recording signal processing such as recording encoding and modulation, and recorded on a recording medium such as a magnetic tape. When recording is performed on the magnetic tape, a subcode area is secured on the magnetic tape together with recording of the video signal and the audio signal, and the illumination signal is recorded as metadata. Of course, the recording medium is not limited to the magnetic tape described above, and a disk medium such as a magnetic disk, an optical disk, or a magneto-optical disk, and other recording media can be appropriately applied.

FIG. 2 is a block diagram showing an embodiment of a video reproduction apparatus according to the present invention. The video playback apparatus according to the present embodiment is a playback apparatus that inputs signals recorded by the video recording apparatus shown in FIG. 1 and performs playback processing for each signal and outputs the signals.
The photographic information input unit 10 receives the multiplexed signal from the recording medium on which the digital illumination signal is multiplexed and recorded together with the digital video signal and the digital audio signal as described above. That is, the photographing information input unit 10 may be a reading device that reads a signal from the recording medium, or may be an input interface that inputs the multiplexed signal as described above from the outside.
Further, the video reproduction apparatus may be provided with a recording apparatus for inputting a multiplexed signal acquired by the video recording apparatus as shown in FIG. 1 and recording the multiplexed signal on a recording medium such as a fixed magnetic disk. In this case, the imaging information input unit 10 reads the multiplexed signal from the recording device and outputs it to the digital signal processing circuit 11.

In the video reproduction apparatus, the signal input from the photographing information input unit 10 is supplied to the digital signal processing circuit 11. In the digital signal processing circuit 11, the supplied reproduction signal is subjected to processing such as error correction, and separated into a digital video signal, a digital audio signal, and a digital illumination signal multiplexed at the time of recording .

  Of the signals separated by the digital signal processing circuit 11, the digital video signal is supplied to the decoder 12 and decompressed, and then supplied to the D / A converter 13 to be converted into an analog video signal. This analog signal is sent to the display device 14 functioning as the display means of the present invention and reproduced as a video.

  Of the signals separated by the digital signal processing circuit 11, the digital audio signal is supplied to the D / A converter 15, where it is converted into an analog signal to be an analog audio signal. This analog audio signal is reproduced as audio by the speaker 16. The speaker 16 functions as an audio output unit of the present invention.

Further, among the signals separated by the digital signal processing circuit 11, the illumination signal is supplied to the MPU and D / A unit 17. The MPU and D / A unit 17 is composed of a microprocessor and has a D / A conversion function. The illumination signal input to the MPU / D / A unit 17 is composed of colorimeter measurement data, and is converted into an analog signal by the MPU / D / A unit 17. The converted analog signal is supplied to the reproduction illumination conversion device 18, and is supplied as a control signal to the illumination device 19 after performing the reproduction illumination conversion processing described later. The illumination device 19 reproduces the color information of the illumination in the shooting environment by controlling the color of the illumination for reproduction based on the control signal. The reproduction lighting converter 18 that generates a control signal for reproducing the illumination environment at the time of shooting based on the illumination environment information separated by the digital signal processing circuit 11.

Next, the procedure of the reproduction illumination conversion process will be described with reference to the flowchart of FIG. 2 is supplied with chromaticity values (x, y, z) as an example of photographing illumination light data (step S1). The reproduction illumination conversion device 18 converts the chromaticity values (x, y, z) into the RGB values (R, G, B) of the reproduction illumination when color illumination is used for the reproduction illumination (step S2). , S3). The chromaticity value (x, y, z) is an example of a uniform color space that does not depend on the device, and may be another uniform color space. When general illumination color temperature information is used without using color illumination, the chromaticity value (x, y) is converted to the correlated color temperature value of reproduction illumination (steps S2 and S4). Reproduction lighting converter 18 are all a process of converting the chromaticity value information within the uniform color space into RGB values, and processing to convert the correlated color temperature value chromaticity value information uniform color space of Can be selected.

  Here, a case where RGB three primary color illumination is used as an example of color illumination will be described. Examples of the RGB three primary color type illumination include a color LED and a color fluorescent lamp, but are not limited thereto. Then, the illumination intensity of each RGB is divided into 256 levels to emit light, the chromaticity values (x, y, z) at that time are measured, an RGB-xyz conversion table is created, and an illumination conversion apparatus for reproduction 18 is stored.

  An example of a conversion table creation method will be described using the case where the present invention depicted in FIG. 4 is applied to a home theater. In FIG. 4, the display device 14 and the speaker 16 are arranged at predetermined positions in the room, and a reproduction illumination device 19 is arranged above the speaker 16. In this room, a sofa 20 is arranged facing forward so that a viewer can sit there. A standard white plate (not shown) is installed instead of the display device 14 at a position where the display device 14 in front of the viewer is arranged, and a color meter is installed at the position of the sofa 20.

  The illumination device 19 on the speaker 16 is irradiated with 256 levels of RGB illumination intensity toward the standard white plate, and the chromaticity values (x, y, z) at that time are measured with a colorimeter. To do. Then, an RGB-xyz conversion table is created based on the chromaticity values obtained by the colorimeter. Thereby, the RGB value (R, G, B) of the color illumination corresponding to the chromaticity value (x, y, z) of the photographing illumination light can be determined.

  The reproduction illumination conversion device 18 determines the RGB value based on the obtained chromaticity value, and supplies the control signal to the illumination device 19 to emit light, thereby reproducing the illumination state at the time of photographing. . Here, the conversion from RGB to xyz is not limited to the use of the RGB-xyz conversion table, and matrix conversion may be performed. The following is an example of matrix conversion.

  FIG. 5 is an xy chromaticity diagram showing the color range of the reproduction illumination according to this embodiment. FIG. 6 shows a color temperature curve and a correlated color temperature curve on the xy chromaticity diagram. As shown in FIG. 5 and FIG. 6, when the color range for reproduction illumination is made wider than the color temperature curve, correlation temperature curve, and color range that can be reproduced by the current television, it is applied to a broadcasting system. In addition, it is possible to reproduce the entire color range of the photographing illumination light.

When general illumination color temperature information is used, the chromaticity value (x, y) is converted into the correlated color temperature value T of the reproduction illumination as described above. At this time, as a simple calculation method for obtaining the correlated color temperature value T from the chromaticity value (x, y), there is a McCamy correlated color temperature conversion formula described below. (Color Res. Application 17, 142-144 (1992))
T = −437n ³ + 3601n ² −6861n + 5514.31 (2)
Then, the reproduction illumination conversion device 18 selects the illumination having the correlated color temperature T corresponding to the chromaticity value (x, y) of the photographic illumination light, supplies the control signal, and emits the light, so that The lighting state can be reproduced.

  With the system as described above, for example, for viewers sitting on the sofa 20 in the room shown in FIG. 4, not only video and audio, but also the color of the lighting at the scene taken is reproduced. High sense of presence can be provided.

  In the above-described embodiment, the color of the illumination is measured by the colorimeter 6 at the time of shooting, and the measured information is supplied to the digital signal processing circuit 8 via the A / D and MPU unit 7 to record the shooting information. Although recorded in the apparatus 9, the present invention is not limited to such a configuration. For example, as another example, the video recording apparatus having the digital signal processing function according to the present invention as described above is provided with an input unit that allows a user to input data, and another imaging apparatus or microphone, a color meter, etc. The environment information acquired by the above may be manually input from this input unit. At this time, the input environmental information may not be actually detected.

In addition, the video recording method that can be implemented according to the embodiment of the present invention includes a lighting environment information acquisition step of acquiring a video signal and an audio signal by shooting, and acquiring lighting information related to ambient light at the time of shooting, and the lighting information A recording step of recording the lighting environment information acquired in the acquisition step on a recording medium together with the video signal and the audio signal. The illumination environment information acquisition step corresponds to processing for acquiring a video signal, an audio signal, and illumination information by the imaging device 1, the microphone 4, and the colorimeter 6 shown in FIG. Further, the recording step includes an A / D converter 2 for video signals, an encoder 3, an A / D converter 5 for audio signals 5, an A / D for illumination information, an MPU unit 7 in FIG. The processing by the digital signal processing circuit 8 for performing the signal processing and the processing for recording the signal processed signal in the photographing information recording device 9 are included.

In addition, the video reproduction method that can be performed according to the embodiment of the present invention includes a recording in which a video signal and an audio signal and lighting environment information related to ambient light at the time of shooting when the video signal and the audio signal are acquired are recorded. It has a signal processing step for performing signal processing to separate recording information of the medium, and a reproduction output step for reproducing and outputting each signal separated by the signal processing step. The reproduction output step includes a reproduction illumination conversion step for generating a control signal for reproducing the illumination environment at the time of photographing based on the illumination environment information separated by the signal processing step.

  The above-described signal processing step corresponds to a process of separating each signal by performing predetermined signal processing such as error correction on the signal input to the imaging information input unit 10 in the digital signal processing circuit 11 shown in FIG. . The reproduction output step is a step for performing reproduction processing of each signal separated by the digital signal processing circuit 11. Here, the audio signal includes processing by the decoder 12 and the D / A converter 13 and display output processing by the display device 14, and the audio signal is processed by the D / A converter 15 and output from the speaker 16. Is included. Further, the illumination signal includes processing by the MPU and D / A unit 17 and generation processing of an illumination control signal by the reproduction illumination conversion device 18. The illumination device controls illumination according to the control signal generated by the reproduction illumination conversion device 18.

Upper Kiutsu image reproducing method can be realized by executing a program by a computer. The program according to the present invention can be recorded on a computer-readable recording medium and can be distributed as a portable recording medium, or can be provided to a user via an electric communication line.

  The recording medium on which the program is recorded is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy (registered trademark) disk or a hard disk, or a CD-ROM / MO / Disk system of optical disks such as MD / DVD, card system such as IC card (including memory card) / optical card, or mask ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory) flash It may be a medium that carries a fixed program including a semiconductor memory such as a ROM.

In either case, the stored program may be configured to be accessed and executed by the microprocessor, or the program read into the program storage area of the microcomputer is downloaded and the program is stored. An executed method may be used. The video recording apparatus and the video playback apparatus of the present invention may be configured to be connected to a communication network including the Internet, and the program may be downloaded from the communication network. This download program may be stored in the main apparatus in advance, or may be installed from another recording medium.
Finally, the embodiment described above does not limit the scope of the present invention, and various modifications can be made within the scope of the present invention.

It is a block diagram which shows one Embodiment of the video recording apparatus by this invention. It is a block diagram which shows one Embodiment of the video reproduction apparatus by this invention. It is a flowchart for demonstrating the procedure of the illumination conversion process for reproduction. It is a figure which shows an example of the production method of a conversion table. It is a chromaticity diagram showing the color range of the illumination for reproduction according to the present embodiment. It is a figure which shows the color temperature curve on a xy chromaticity diagram, and a correlation color temperature curve.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging device, 2 ... A / D converter, 3 ... Encoder, 4 ... Microphone, 5 ... A / D converter, 6 ... Colorimeter, 7 ... A / D, MPU part, 8 ... Digital signal processing circuit, DESCRIPTION OF SYMBOLS 9 ... Shooting information recording device, 10 ... Shooting information input part, 11 ... Digital signal processing circuit, 12 ... Decoder, 13 ... D / A converter, 14 ... Display apparatus, 15 ... D / A converter, 16 ... Speaker 17 ... MPU, D / A section, 18 ... Reproduction lighting conversion device, 19 ... Lighting device, 20 ... Sofa.

Claims (3)

When playing back video and audio signals, the lighting environment at the time of shooting is reproduced by controlling the lighting device based on the lighting environment information related to the lighting environment at the time of shooting when the video and audio signals are acquired. In the lighting environment reproduction device to
When using a color lighting device controlled by RGB values as the lighting device , the lighting environment information represented by chromaticity values is converted into RGB values and output to the lighting device,
When using a lighting device controlled by a color temperature value as the lighting device, the lighting environment information represented by a chromaticity value is converted into a color temperature value and output to the lighting device. Lighting environment reproduction device. The lighting environment reproduction device according to claim 1;
Display means for performing reproduction processing on the video signal and displaying the video;
A video reproduction apparatus comprising: audio output means for performing reproduction processing on the audio signal and outputting the audio. When playing back video and audio signals, the lighting environment at the time of shooting is reproduced by controlling the lighting device based on the lighting environment information related to the lighting environment at the time of shooting when the video and audio signals are acquired. In the lighting environment reproduction method to
When using a color lighting device controlled by RGB values as the lighting device , the lighting environment information represented by chromaticity values is converted into RGB values and output to the lighting device,
When using a lighting device controlled by a color temperature value as the lighting device, the lighting environment information represented by a chromaticity value is converted into a color temperature value and output to the lighting device. How to reproduce the lighting environment.

Images