JP4709897B2 - Viewing environment control system, viewing environment control device, viewing environment lighting control system, and viewing environment control method - Google Patents

Description

【Technical field】
[0001]
The present invention can control the illumination light around the video display device when the video is displayed on the video display device in accordance with the atmosphere and scene setting of the shooting scene of the video. Viewing environment control system, Viewing environment control device, viewing environment illumination The present invention relates to a control system and a viewing environment control method.
[0002]
For example, when displaying an image with an image display device such as a television receiver, or when projecting and displaying an image using a projector device, the surrounding illumination light is adjusted according to the display image. There is known a technique for providing a viewing effect such as enhancing the sense of presence.
[0003]
For example, in Japanese Patent Laid-Open No. 2-158094, a mixed light illuminance ratio of the three primary colors of a light source is calculated for each frame from a color signal (RGB) and a luminance signal (Y) of a color television display image, and linked to the image. Thus, there is disclosed a light color variable illumination device that performs dimming control. This variable-light-color illuminating device extracts a color signal (RGB) and a luminance signal (Y) from a display image of a color television, and uses the three color lights (red light, green) used as a light source from the color signal and the luminance signal. Light, blue light) is calculated, the illuminance of the three-color light is set according to the illuminance ratio, and the three-color light is mixed and output as illumination light.
[0004]
Also, for example, in Japanese Patent Laid-Open No. 2-253503, a video effect lighting apparatus that performs illumination control around a dividing unit by dividing a television image into a plurality of portions and detecting an average hue of the corresponding dividing unit. Is disclosed. This video effect lighting device includes illumination means for illuminating the surroundings of the installation location of the color television, divides the video displayed on the color television into a plurality of parts, and divides the video corresponding to the portion illuminated by the illumination means The average hue of the part is detected, and the illumination means is controlled based on the detected hue.
[0005]
Further, for example, in Japanese Patent Laid-Open No. 3-184203, the skin color of a human face or the like is not obtained from an image displayed on the screen of the image display device, but merely by calculating the average chromaticity and average luminance of the entire screen of the image display device. The remaining portion after removing the pixels of the portion is considered as the background portion, and only the RGB signal and luminance signal of each pixel in the background portion are taken out to obtain the average chromaticity and average luminance, and the color of the wall surface on the back of the image display device There is disclosed a method for controlling illumination so that the brightness and brightness are the same as the average chromaticity and average brightness of the entire screen or the background portion excluding human skin color.
DISCLOSURE OF THE INVENTION
[Problems to be solved by the invention]
[0006]
In the conventional viewing environment control apparatus described above, the feature amount (color signal and luminance signal) for each frame (screen) in the video signal to be displayed is detected and the illumination light is controlled. It is difficult to generate illumination light that matches the field (atmosphere) of the image. For example, if you illuminate the surrounding area with improper color illumination light due to the influence of the clothes of the subject person included in the video signal or artifacts in the background of the subject, the atmosphere of each scene can be reproduced. , Can not maintain the sense of reality for each scene. In other words, viewing environment lighting that deviates significantly from the lighting situation at the time of shooting a video scene, on the contrary, impairs the sense of reality.
[0007]
In the above-mentioned Japanese Patent Laid-Open No. 3-184203, the remaining portion of the skin color portion is removed as a background portion, and the viewing environment space is based on the feature amount (color signal and luminance signal) of each pixel in the background portion. Although the illumination light is controlled, for example, in the case of an image in which the ground and buildings occupy a lot in the background, the illumination light of an inappropriate color will be irradiated to the surroundings. The feeling and atmosphere will be impaired. That is, in order to estimate the lighting condition (atmosphere) at the time of shooting the display image from the video signal, it is not sufficient to remove the skin-colored pixels, and only the background pixels representing the lighting condition (atmosphere) at the time of shooting are appropriate. It is necessary to extract and use.
[0008]
In the above-described conventional technology, the illumination light state changes according to the change of the luminance and hue of the video signal for each frame, and particularly when the degree of change in luminance and hue between frames is large. The problem is that the light changes complicatedly and the viewer feels uncomfortable with the flicker. Furthermore, it is not preferable that the illumination light fluctuates in accordance with changes in luminance and hue for each frame during the display of one scene in which the illumination state at the time of shooting does not change, which adversely inhibits the atmosphere for each scene. .
[0009]
FIG. 1 is a diagram for explaining an example of the problem of illumination control according to the above-described conventional technique, and shows a part of continuous moving images. In the example shown in FIG. 1 (A), a scene of an image shot with a scene setting of outdoors in a sunny day is created. This scene consists of a video obtained by a series of camera work shooting without switching the camera. In this example, an image of a skier sliding down from above the camera toward the vicinity of the camera is taken. Skiers are dressed in red and the sky is clear.
[0010]
That is, as shown in FIG. 1B, this video scene includes the camera position (a), angle (b), number of subjects (d), camera movement (e), and camera lens type (f). Each was taken in a certain camera work situation of low position, low angle, one shot (1S), fixed, and standard lens.
[0011]
The video of this scene has a large blue sky area in the initial frame (frame A), and the skier's red clothing area gradually increases as the skier slides down and approaches the camera. That is, as the video in the scene progresses, the ratio of the colors constituting each frame changes. That is, regarding the subject size (c), the frames A to D are log shots, and the frame E is a full figure.
[0012]
In such a case, when the illumination light is controlled using the chromaticity and luminance for each frame, the blue light changes from strong illumination light to red illumination light. In other words, even though the video was shot under constant natural light illumination, illumination light that does not take into account the lighting conditions at the time of shooting is generated and irradiated. Will give a sense of incongruity. In addition, if the color of the illumination light changes in a single segment of scenes in which one scene setting (atmosphere) is continuous, the atmosphere of the scene is still disturbed and the viewer feels uncomfortable.
[0013]
FIG. 2 is a diagram for explaining another example of the problem of the illumination control according to the conventional technique. In the example shown in FIG. 2A, a scene of an image shot with a scene setting of moonlight night is created. This scene is composed of three shots (1, 2, 3) with different camera work. In Shot 1, the camera takes a long shot of the target ghost. Then, when switching to shot 2, the ghost is shot with a bust shot. In shot 3, the camera returns to the camera position of shot 1 again. These shots are intended and configured as a segment of a scene in which one atmosphere continues even though the camera work is different.
[0014]
That is, as shown in FIG. 2B, the camera work situation used for shooting this scene includes the camera position (a), the angle (b), the camera movement (e), and the camera lens type (f ) Are eye height, horizontal angle, fixed, standard lens, subject size (c), number of subjects (d) are frames A to B (shot 1) and frames E to F (shot 3). Long shot, two shot (2S), and frames C to D (shot 2) are bust shot and one shot (1S).
[0015]
In such a case, since relatively dark images of moonlight night are continuous in Shot 1, if the illumination light is controlled according to the luminance and chromaticity of each frame of these images, the illumination light becomes relatively dark. Then, when shot 1 is switched to shot 2, the ghost photographed by the bust shot becomes a relatively bright image. At this time, if the illumination light is controlled for each frame according to the above-described conventional technique, the illumination light control is largely switched at the time of switching shots, resulting in bright illumination light. When switching to shot 3 again, it returns to the dark illumination light similar to shot 1.
[0016]
In other words, even though it is a series of images shot under a certain lighting condition, illumination light that does not take into account the lighting condition at the time of shooting is generated and irradiated, so the atmosphere of the scene is disturbed on the contrary. This will make the viewer feel uncomfortable. Also, if the illumination light becomes darker or brighter in a single segment where a single scene setting (atmosphere) is continuous, the atmosphere of the scene is also hindered and the viewer feels uncomfortable.
[0017]
FIG. 3 is a diagram for explaining another example of the problem of the illumination control according to the conventional technique. In the example shown in FIG. 3A, a scene of a video shot with a scene setting of outdoors in a sunny day is created. This scene is made up of images obtained by a series of camera work shootings without switching the camera, but the brown dog that is the subject (foreground) is gradually changing from a long shot to an up shot by zoom shooting.
[0018]
That is, as shown in FIG. 3B, the camera work situation used for shooting this scene includes the camera position (a), the angle (b), the number of subjects (d), and the camera movement (e ) Are high position, high angle, one shot (1S), zoom, subject size (c), frames A to B are long shots, frame C is full figure, frame D is bust shot, frame E is In the up shot, camera lens type (f), the frame A is a standard lens, and the frames B to E are telephoto lenses.
[0019]
When the illumination light is controlled according to the luminance and chromaticity of each frame of these images, the green light changes from strong illumination light to brown illumination light. In other words, even though the video was shot under constant natural light illumination, illumination light that does not take into account the lighting conditions at the time of shooting is generated and irradiated. Will give a sense of incongruity. In addition, if the color of the illumination light changes in a single segment of scenes in which one scene setting (atmosphere) is continuous, the atmosphere of the scene is still disturbed and the viewer feels uncomfortable.
[0020]
FIG. 4 is a diagram for explaining another example of the problem of the illumination control according to the conventional technique. In the example shown in FIG. 4A, a scene of a video shot with a scene setting of outdoors in a sunny day is created. This scene consists of images obtained by a series of camera work shooting without switching the camera, but a person wearing pink clothes as the subject (foreground) by zoom shooting is taken from a long shot (frame A) to a bust shot. It gradually changes to (Frame E).
[0021]
That is, as shown in FIG. 4B, the camera work situation used for shooting this scene includes the camera position (a), the angle (b), the number of subjects (d), and the camera movement (e ) Are eye height, horizontal angle, one shot (1S), zoom, subject size (c) is a long shot for frames A to B, full figure for frame C, waist shot for frame D, and frame E for frame E. In the bust shot and camera lens type (f), the frame A is a standard lens, and the frames B and thereafter are telephoto lenses.
[0022]
When the illumination light is controlled according to the luminance and chromaticity of each frame of these images, the blue light changes from strong illumination light to pink illumination light. In other words, even though the video was shot under constant natural light illumination, illumination light that does not take into account the lighting conditions at the time of shooting is generated and irradiated. Will give a sense of incongruity. In addition, if the color of the illumination light changes in a single segment of scenes in which one scene setting (atmosphere) is continuous, the atmosphere of the scene is still disturbed and the viewer feels uncomfortable.
[0023]
FIG. 5 is a diagram for explaining another example of the problem of the illumination control according to the conventional technique. In the example shown in FIG. 5 (A), a scene of an image shot with a scene setting of outdoors in a sunny day is created. This scene consists of images obtained by a series of camera work shooting without switching the camera, but by switching the camera lens and shooting, a person wearing pink clothes as the subject (foreground) is a long shot Has changed from a bust shot to a bust shot.
[0024]
That is, as shown in FIG. 5B, the camera work situation used for shooting this scene includes the camera position (a), the angle (b), the number of subjects (d), and the camera movement (e ) Are eye height, horizontal angle, one shot (1S), and fix, subject size (c) is long shot in frames A to C, bust shot is in frames D to E, and camera lens type (f) is Frames A to C are standard lenses, and frames D and after are telephoto lenses.
[0025]
When the illumination light is controlled in accordance with the luminance and chromaticity of each frame of these images, the blue light suddenly changes to pink illumination light. In other words, even though the video was shot under constant natural light illumination, illumination light that does not take into account the lighting conditions at the time of shooting is generated and irradiated. Will give a sense of incongruity. In addition, if the color of the illumination light changes in a single segment of scenes in which one scene setting (atmosphere) is continuous, the atmosphere of the scene is still disturbed and the viewer feels uncomfortable.
[0026]
The present invention has been made in view of the above problems, and can realize illumination control in an optimal viewing environment according to the camera work situation at the time of shooting a display image. Glance Listening environment control system, Viewing environment control device, viewing environment lighting control system, It is another object of the present invention to provide a viewing environment control method.
[0027]
The first invention of the present application is , Movie Image data Shows the camera work situation at the time of shooting each frame that composes Add camera work data to the video data and send De Data transmitter And receiving means for receiving camera work data from the data transmitting device; and restricting a video frame or a screen area for detecting a video feature amount of the video data based on the camera work data; A viewing environment control device having control means for detecting the video feature amount of the video data in a frame or a screen area and controlling illumination light of the lighting device based on the detected video feature amount. It is characterized by that.
[0028]
The second invention of the present application is characterized in that the camera work data includes at least information indicating a camera position at the time of photographing each frame.
[0029]
The third invention of the present application is characterized in that the camera work data includes at least information representing a camera angle at the time of photographing each frame.
[0030]
The fourth invention of the present application is characterized in that the camera work data includes at least information indicating a size of a subject at the time of photographing each frame.
[0031]
The fifth invention of the present application is characterized in that the camera work data includes at least information indicating the number of subjects at the time of photographing each frame.
[0032]
The sixth invention of the present application is characterized in that the camera work data includes at least information indicating the movement of the camera at the time of photographing each frame.
[0033]
The seventh invention of the present application is characterized in that the camera work data includes at least information indicating a type of a camera lens used for photographing each frame.
[0034]
According to an eighth aspect of the present application, in response to an external request, video data Shows the camera work situation at the time of shooting each frame that composes The camera work data is transmitted together with the start timing of each frame constituting the video data. De Data transmitter And receiving means for receiving the camera work data; and restricting a video frame or a screen area to be detected based on the camera work data for a video feature amount of the video data; and in the restricted video frame or screen area A viewing environment control device including a control unit that detects the video feature amount and controls illumination light of the lighting device based on the detected video feature amount. It is characterized by that.
[0035]
According to a ninth aspect of the present invention, the camera work data includes at least information representing a camera position at the time of photographing each frame.
[0036]
According to a tenth aspect of the present invention, the camera work data includes information representing at least a camera angle at the time of photographing each frame.
[0037]
The eleventh invention of the present application is characterized in that the camera work data includes at least information indicating a size of a subject at the time of photographing each frame.
[0038]
In a twelfth aspect of the present invention, the camera work data includes at least information representing the number of subjects at the time of photographing each frame.
[0039]
The thirteenth invention of the present application is characterized in that the camera work data includes at least information indicating the movement of the camera at the time of photographing each frame.
[0040]
In a fourteenth aspect of the present invention, the camera work data includes at least information indicating a type of a camera lens used for photographing each frame.
[0041]
According to a fifteenth aspect of the present invention, video data to be displayed on a display device Video A viewing environment control device for controlling illumination light of a lighting device based on a feature amount, wherein the video data Shows the camera work situation at the time of shooting each frame that composes Receiving means for receiving camera work data; and Based on the detected video feature quantity, the video frame or screen area to be detected is limited based on the video data, and the video feature quantity in the limited video frame or screen area is detected. And a control means for controlling the illumination light of the illumination device.
[0042]
A sixteenth aspect of the present invention is characterized in that the control means switches and controls the illumination light of the illumination device in units of scenes constituting the video data.
[0043]
According to a seventeenth aspect of the present invention, the camera work data includes at least information representing a camera position at the time of photographing each frame.
[0044]
The eighteenth invention of the present application is characterized in that the camera work data includes at least information indicating a camera angle at the time of photographing each frame.
[0045]
According to a nineteenth aspect of the present invention, the camera work data includes at least information indicating the size of the subject at the time of photographing each frame.
[0046]
The twentieth invention of the present application is characterized in that the camera work data includes at least information indicating the number of subjects at the time of photographing each frame.
[0047]
According to a twenty-first aspect of the present invention, the camera work data includes at least information representing the movement of the camera at the time of photographing each frame.
[0048]
In a twenty-second aspect of the present invention, the camera work data includes at least information indicating a type of a camera lens used for photographing each frame.
[0051]
No. of this application 23 A viewing environment comprising the above-described viewing environment control device and a lighting device whose viewing environment illumination light is controlled by the viewing environment control device. illumination It is a control system.
[0054]
No. of this application 24 The video data to be displayed on the display device The viewing environment control method for controlling the illumination light of the illumination device based on the video feature amount, Video data Shows the camera work situation at the time of shooting each frame that composes Camera work day T Receive Based on the camera work data, the target video frame or screen area for detecting the video feature quantity of the video data is limited, the video feature quantity in the limited video frame or screen area is detected, and the detected video Based on features Installed around the display device Said The illumination light of the illumination device is controlled.
【The invention's effect】
[0055]
ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to implement | achieve the optimal viewing-and-listening environment according to the place (atmosphere) at the time of imaging | photography of an image | video.
[Brief description of the drawings]
[0056]
FIG. 1 is a diagram for explaining an example of a problem of illumination variation according to the prior art.
FIG. 2 is a diagram for explaining another example of the problem of illumination variation according to the prior art.
FIG. 3 is a diagram for explaining another example of the problem of illumination variation according to the prior art.
FIG. 4 is a diagram for explaining another example of the problem of illumination variation according to the prior art.
FIG. 5 is a diagram for explaining another example of the problem of illumination variation according to the prior art.
FIG. 6 is a block diagram showing a schematic configuration of a main part of a video transmission device in the viewing environment control system according to the first embodiment of the present invention.
FIG. 7 is an explanatory diagram showing an example of an output bit stream of the video transmission device in the viewing environment control system according to the first embodiment of the present invention.
FIG. 8 is an explanatory diagram showing an example of camera work data in the viewing environment control system according to the first embodiment of the present invention.
[FIG. 9] It is explanatory drawing for demonstrating a camera position and a camera angle.
FIG. 10 is an explanatory diagram for explaining a subject size (screen size).
FIG. 11 is an explanatory diagram for explaining the number of subjects.
FIG. 12 is an explanatory diagram for explaining components of a video.
FIG. 13 is a block diagram showing a schematic configuration of a main part of a video reception device in the viewing environment control system according to the first embodiment of the present invention.
FIG. 14 is a block diagram showing a configuration of a lighting control data generation unit in the viewing environment control system according to the first embodiment of the present invention.
FIG. 15 is an explanatory diagram showing an example of a field (atmosphere) estimation target region determined by a position and an angle in the viewing environment control system according to the first embodiment of the present invention.
FIG. 16 is an explanatory diagram showing an example of a field (atmosphere) estimation target region determined by the subject size and the number of subjects in the viewing environment control system according to the first embodiment of the present invention.
FIG. 17 is a flowchart showing an operation of a lighting control data generation unit in the viewing environment control system according to the first embodiment of the present invention.
FIG. 18 is a block diagram illustrating a schematic configuration of main parts of an external server device in a viewing environment control system according to a second embodiment of the present invention.
FIG. 19 is an explanatory diagram showing an example of a camera work data storage table in the viewing environment control system according to the second embodiment of the present invention.
FIG. 20 is a block diagram illustrating a schematic configuration of a main part of a video reception device in a viewing environment control system according to a second embodiment of the present invention.
[Explanation of symbols]
[0057]
DESCRIPTION OF SYMBOLS 1 ... Data multiplexing part, 2 ... Transmitting part, 21, 51 ... Reception part, 22, 52 ... Data separation part, 23, 53 ... CPU, 24 ... Illumination control data generation part, 25 ... Video display apparatus, 26 ... Sound reproduction Device: 27 ... Lighting device, 28, 29 ... Delay generation unit, 31 ... Scene section detection unit, 32 ... Field (atmosphere) estimation unit, 41 ... Reception unit, 42 ... Data storage unit, 43 ... Transmission unit, 54 ... Transmission 55, a receiving unit.
BEST MODE FOR CARRYING OUT THE INVENTION
[0058]
FIG. 6 is a block diagram showing a schematic configuration of a main part of the video transmission device in the viewing environment control system according to the embodiment of the present invention. The video transmission device (data transmission device) in the present embodiment is as shown in FIG. And a data multiplexing unit 1 that divides and multiplexes each of video data, audio data, and camera work data supplied as additional information into a transport stream packet (TSP) format, and an output of the data multiplexing unit 1 A transmission unit 2 is provided that modulates the data after adding an error correction code and sends the data to the transmission line as broadcast data.
[0059]
FIG. 7 is an explanatory diagram showing an outline of the structure of a transport stream packet (TSP) defined by MPEG-2 (Moving Picture Experts Group 2) -Systems, and 11 is the data content of TSP and others defined by MPEG2-Systems. A header in which information is described, 12 is an extension header (adaptation field) that can describe information determined by the sender, and 13 is a payload composed of data such as video and audio. In the present embodiment, for example, video data and audio data are transmitted by the payload 13, and camera work data as additional information can be transmitted by the extension header (adaptation field) 12. Note that different data streams of video data, audio data, and camera work data may be multiplexed and transmitted.
[0060]
FIG. 8 is a diagram for explaining camera work data indicating a camera work situation at the time of shooting each frame of the video data. In the present embodiment shown in FIG. 8, as camera work data, position information (a) representing the camera position at the time of photographing each frame, angle information (b) representing the camera angle at the time of photographing each frame, and photographing of each frame. Subject size information (c) representing the size of the subject at the time, subject number information (d) representing the number of subjects at the time of photographing each frame, camera motion information (e) representing the movement of the camera at the time of photographing each frame And lens type information (f) indicating the type of camera lens used for photographing each frame. All of these pieces of information are useful information for estimating the lighting environment at the time of shooting each frame and producing the atmosphere and the realistic sensation of each scene with illumination light. Hereinafter, each information is demonstrated.
[0061]
First, the camera position (a) at the time of video shooting is classified into (a1) high position, (a2) eye height, and (a3) low position. As shown in FIG. 9A, the high position (a1) is a position that is often used for a long shot, which will be described later, and is a high position that expresses a vast expanse or is used for crossing a fence. . The eye height (a2) is a normal position and is the eye height position of the subject (O). The low position (a3) is a low position such as the eyes of a child.
[0062]
For outdoor shooting, the height of the horizontal line (horizon line) in the shooting screen tends to be low at the high position and high at the low position. The presence of the display video scene can be improved by limiting the screen area to be detected and controlling the viewing environment illumination described later. Therefore, in the present embodiment, as the position information, a 2-bit description indicating whether the camera position at the time of shooting each video frame belongs to high position / eye height / low position is included in the camera work data. is doing.
[0063]
The camera angle (b) at the time of video shooting is classified into (b1) high angle (overhead view), (b2) horizontal angle (line of sight), and (b3) low angle (tilt). As shown in FIG. 9B, the high angle (b1) is a photographing method at an angle looking down, and is used for an overall objective situation description. The horizontal angle (b2) is a natural and standard angle in which the camera is directed in the horizontal direction. The low angle (b3) is a shooting method at an angle looking upward, and is used when expressing intimidation, superiority, victory, and the like.
[0064]
For outdoor shooting, the horizontal line (horizon) in the shooting screen tends to be high at high angles and low at low angles. The presence of the display video scene can be improved by limiting the screen area to be detected and controlling the viewing environment illumination described later. Therefore, in the present embodiment, as the angle information, a 2-bit description indicating whether the camera angle at the time of shooting each video frame belongs to high angle (overhead view), horizontal angle (line of sight), or low angle (tilt). Are included in the camera work data.
[0065]
Further, as the subject (screen) size (c) at the time of video shooting, long shot (c1), full figure (c2), knee shot (c3), waist shot (c4), bust shot (c5), upshot (c6) ), Close-up shot (c7), and big close-up shot (c8). As shown in FIG. 10, a long shot (LS: c1) is a studio landscape or a wide-angle shot in the outdoors, and is often used for the overall positional relationship and the beginning and end of a story. The full figure (F.F .: c2) has a size in which the entire body of the subject (a whole body image from the toes to the head when standing, a whole body image when sitting) is included in the shooting screen. The knee shot (K.S .: c3) has a size in which the area from the knee to the head of the subject is included in the shooting screen.
[0066]
The waist shot (WS: c4) is the size of a half-body image with the subject's waist or hips to the head in the shooting screen, and is used for basic shots of news and speakers. Is a feature. The bust shot (B.S.:c5) is the size of the subject's chest or upper body with the subject's chest or the top of the chest up to the head, which is the most commonly used size for human images. The up shot (US: c6) is a shot that expresses the expression and excitement of the subject. The close-up shot (C.U .: c7) is a shot that emphasizes up, and has a size enough to cut the head (hair) of the subject. The big close-up shot (B.C.U .: c8) is used to emphasize close-up.
[0067]
In this way, the area of the background changes depending on the subject size (screen size) at the time of video shooting, so the frames for detecting video feature values can be limited according to the subject size (screen size) at the time of video shooting. By limiting the screen area for detecting the video feature amount and controlling the viewing environment illumination described later, it is possible to improve the realism of the display video scene. Therefore, in the present embodiment, as subject size information, the subject size at the time of shooting each video frame is long shot / full figure / knee shot / waist shot / bust shot / up shot / close up shot / big close up shot. Is included in the camera work data.
[0068]
As the number of subjects (d) at the time of video shooting, only one shot (1S: d1), two shot (2S: d2), three shot (3S: d3), group shot (GS: d4), and landscape (background) only (d5) As shown in FIG. 11, one shot (d1) has only one subject (one) as the foreground, two shots (d2) have two subjects (two) as the foreground, and three shots (d3) have the foreground. There are three subjects (three), and the group shot (d4) is a shot when there are four or more subjects as foregrounds.
[0069]
In this way, the area of the background changes depending on the number of subjects at the time of video shooting. Therefore, the screen area for detecting video feature values is limited to control the viewing environment illumination described later according to the number of subjects at the time of video shooting. By doing so, the presence of the displayed video scene can be improved. Therefore, in this embodiment, as the number-of-subjects information, the number of subjects at the time of shooting each video frame is one-shot (1S) / two-shot (2S) / three-shot (3S) / group shot (GS) / landscape (background). ) Includes only 3 bits in the camera work data.
[0070]
Further, the camera movement (e) during video shooting is classified into fix (e1), pan (e2), tilt (e3), roll (e4), zoom (e5), dolly (e6), and follow (e7). Is done. The fix (e1) is a shooting method that does not move the camera position without performing a zoom operation for changing the angle of view. Pan (e2) is a photographing method in which the camera is shaken in the horizontal direction to show the surroundings, and is used for explaining the situation and showing the positional relationship between the left and right objects. Tilt (e3) is a shooting method in which the camera is shaken vertically, such as shooting from bottom to top (from top to bottom), or swinging up along the trunk of a tree to shoot branches and leaves. Sometimes used.
[0071]
The roll (e4) is a photographing method for rotating the camera, and the zoom (e5) is a photographing method for making a large image or a wide size with a zoom lens. The dolly (e6) is a technique for shooting while the camera itself moves, and the follow (e7) is a technique for shooting while following the movement of the subject when shooting a moving subject such as a running person or a vehicle.
[0072]
In this way, the background area and feature amount change abruptly depending on the movement of the camera at the time of video shooting, so the frames for detecting the video feature amount are limited in accordance with the movement of the camera at the time of video shooting. By controlling the viewing environment illumination to be performed, it is possible to improve the presence of the displayed video scene. Therefore, in this embodiment, the camera motion information includes a 3-bit description indicating whether the camera motion at the time of shooting each video frame belongs to fix / pan / tilt / roll / zoom / dolly / follow. , Contained in camera work data.
[0073]
Further, the lens work (lens type information; f) used at the time of video shooting is classified into a standard lens (f1), a wide-angle lens (f2), a telephoto lens (f3), and a macro lens (f4). The standard lens (f1) is a lens that is close to the human visual field and capable of capturing a natural perspective. The wide-angle lens (f2) is a lens having a wide angle of view and capable of photographing a wide range, and is often used for taking a landscape or a group photo. The telephoto lens (f3) is a lens that can shoot a large subject by attracting a far object, and is often used for sports scenes that cannot be approached actually, portrait photography, and the like. The macro lens (f4) is a lens that allows close-up photography, and is suitable for close-up photography of flowers and insects.
[0074]
In this way, the area of the background changes depending on the lens work at the time of video shooting, so the frames for detecting video feature quantities are limited to control the viewing environment illumination described later, adapting to the movement of the camera at the time of video shooting By doing so, the presence of the displayed video scene can be improved. Therefore, in this embodiment, as the lens type information, a 2-bit description indicating whether the type of the camera lens used at the time of shooting each video frame belongs to a standard lens / a wide-angle lens / a telephoto lens / a macro lens. , Contained in camera work data.
[0075]
It should be noted that various types of information included in the camera work data are not limited to those described above, and it is obvious that more detailed information may be added. The camera work data described above is created based on a script (also referred to as a scenario or a script) and can be generated using a camera work program or the like used in a video shooting site. It becomes possible to omit the work of creating data.
[0076]
By the way, the configuration of a video including scenes and shots will be described with reference to FIG. Video data constituting a continuous moving image can be divided into three layers (layers) as shown in FIG. The first layer (# 1) constituting the video (Video) is a frame. A frame is a physical layer and refers to a single two-dimensional image. Frames are usually obtained at a rate of 30 frames per second. The second layer (# 2) is a shot. A shot is a sequence of frames taken by a single camera. The third layer (# 3) is a scene. A scene is a sequence of shots having a story-like connection.
[0077]
Here, the above-mentioned camera work data is added in units of video data frames. However, in addition to adding the above-mentioned camera work data to all frames, there is a change in the camera work status at the time of shooting. The above camera work data may be added only to a certain frame. This also makes it possible to add camera work data indicating the camera work status at the time of shooting each frame of the video data to the video data.
[0078]
Next, a video reception device (data reception device) that receives broadcast data transmitted from the video transmission device, displays and reproduces video and audio, and controls the viewing environment illumination at that time will be described.
[0079]
As shown in FIG. 13, the video receiving apparatus according to the present embodiment receives and demodulates broadcast data input from a transmission path, performs error correction, and outputs video from the output data of the receiving unit 21. A data separation unit 22 for separating / extracting each of video data and TC (time code) output to the display device 25, audio data and TC (time code) output to the audio reproduction device 26, and camera work data as additional information; The CPU 23 receives the camera work data separated by the data separation unit 22 and outputs control data to the illumination control data generation unit 24. The feature amount of the video data / audio data is determined according to the control data from the CPU 23. Detecting and outputting illumination control data (RGB data) based on the detection result to the illumination device 27 that illuminates the viewing environment space That an illumination control data generating unit 24, the video data by processing time in the illumination control data generating unit 24, and a delay generating unit 28, 29 and outputting the delayed audio data.
[0080]
Here, based on the camera work data, the CPU 23 of the present embodiment outputs control data for controlling a video frame and a screen area to be detected by the illumination control data generation unit 24 for detecting a feature amount of the video data.
[0081]
The time code (TC) is information added to indicate the reproduction time information of each of the video data and the audio data. For example, the time (h): minute (m): second (s) of the video data: It consists of information indicating the frame (f).
[0082]
The illumination device 27 can be configured by an LED that is installed around the video display device 25 and emits light of, for example, three primary colors of RGB having a predetermined hue. However, the illumination device 27 may be configured to be able to control the illumination color and brightness of the surrounding environment of the video display device 25, and is not limited to the combination of LEDs that emit a predetermined color as described above, and is a white LED. And a color filter, or a combination of a white light bulb, a fluorescent tube and a color filter, a color lamp, or the like can be applied. Further, it is sufficient that one or more lighting devices 27 are installed.
[0083]
Next, a specific configuration of the illumination control data generation unit 24 in the present embodiment will be described with reference to FIG. In FIG. 14, reference numeral 31 denotes a scene section detection unit that detects a start point TC and an end point TC of a scene section from video data and audio data. Various methods including a known one are used as a method for detecting a scene change point. Can be used. Note that here, in addition to the feature amount of the video data, the feature amount of the audio data is also used for the detection of the scene section. This is to further improve the detection accuracy of the scene change point. The scene section may be detected from only the feature amount. Further, when information indicating a scene change point is added to the broadcast data, this may be used.
[0084]
Reference numeral 32 denotes a field (atmosphere) estimation unit that estimates illumination conditions and scene settings (atmosphere) at the shooting site from video data and audio data, and outputs illumination control data for controlling the illumination device 27 according to the estimation results. As a method for estimating the state of ambient light at the time of shooting, various techniques including known ones can be used. Here, in addition to the feature amount of the video data, the feature amount of the audio data is also used for estimating the place (atmosphere) at the time of shooting, but this further improves the accuracy of estimating the place (atmosphere). Therefore, the field (atmosphere) of the shooting scene may be estimated from only the feature amount of the video data.
[0085]
As the feature amount of the video data, for example, as in the conventional example described above, the color signal and luminance signal in a predetermined area of the screen can be used as they are, and the color temperature of ambient light at the time of video shooting is obtained from these. May be used. Furthermore, these may be configured to be switchable and output as feature amounts of video data. Further, as the feature amount of the audio data, a volume, an audio frequency, or the like can be used.
[0086]
Furthermore, in this embodiment, in order to keep the color and intensity of the viewing environment illumination light substantially constant in the same scene, the field (atmosphere) estimation unit 32 uses the scene section detection unit 31 to change the scene (starting point). Is detected, video feature amounts and audio feature amounts are detected in a predetermined number N (N is an arbitrary natural number set in advance, for example, N = 100), and illumination control is performed in accordance with the feature amounts. Data (RGB data) is configured to be output until switching to the next scene.
[0087]
Here, among the predetermined number N of frames that are continuous from the time point when the scene change point (start point) is detected, the frame used for detecting the feature amount of the video data and its screen area are controlled by the CPU 23. Determined by data. For example, in a frame whose subject size is one of bust shot, up shot, close-up shot, and big close-up shot, the subject (foreground part) occupies a lot of area in the screen, so it can be used for video feature detection. Should not be used. Also, a frame whose lens type is either a telephoto lens or a macro lens is not used for detecting the video feature amount because the subject (foreground part) occupies a large area in the screen.
[0088]
In addition, frames with camera movements of pan, tilt, roll, zoom, dolly, and follow are likely to change the background to the extreme with a series of camera movements. Other than the frame, it is not used for the detection of the video feature amount. For example, when two or more frames in which the camera motion is pan are consecutive, the second and subsequent frames are not used for detecting the video feature amount. If the above-mentioned camera motion frames are continuous, only the first few predetermined frames may be used for detecting the video feature amount.
[0089]
In addition, even for a frame used for video feature amount detection, the screen area to be detected for video feature amount differs depending on the camera work at the time of video shooting. For example, as shown in FIG. 15, a screen region suitable for estimating the color and brightness of ambient light at the time of shooting based on the camera position and angle at the time of shooting the frame (shown by hatched portions in FIG. 15). ) Is decided. In other words, the screen area suitable for estimating the color and brightness of ambient light at the time of shooting is usually the area at the top of the screen, but the horizontal line that defines the size of the area at the top of the screen shows the camera position and angle. It is decided by.
[0090]
In addition, as shown in FIG. 16, the screen area suitable for estimating the color and brightness of the ambient light at the time of shooting (in the shaded portion in FIG. 16) also depending on the subject size and number at the time of shooting of the frame. Determined). In other words, the screen area suitable for estimating the color and brightness of the ambient light at the time of shooting is desirably an area excluding the foreground subject, and this subject is highly likely to be located. The area to be excluded from the area is determined by the subject size and the number. When the number of subjects is only the background, the entire screen area is set as a video feature amount detection target.
[0091]
Therefore, in the present embodiment, the target region for detecting the video feature amount is limited based on the camera work information of the position, angle, subject size, and number of subjects. Specifically, the video feature amount is calculated using only the video data of the screen area obtained by the logical product of the hatched area in FIG. 15 determined by the position and angle and the hatched area in FIG. 16 determined by the subject size and the number of subjects. Trying to detect.
[0092]
In this way, by using camera work data, by limiting and extracting only the frames and screen areas suitable for video scene field (atmosphere) estimation, and detecting the video feature quantities in the extracted frames and screen areas, The estimation accuracy of the place (atmosphere) can be improved. As a result, it is possible to prevent an illumination control that impairs the sense of presence or the atmosphere due to an estimation error of the place (atmosphere), and to always realize an optimal viewing environment.
[0093]
Furthermore, operation | movement of the illumination control data generation part 24 of this embodiment is demonstrated with the flowchart of FIG. First, the scene section detection unit 31 reads a newly input frame (step S1), and detects whether or not the frame is a scene start point (change point) from the video feature amount and audio feature amount of the frame (step S1). S2). If no scene start frame is detected, the process returns to step S1 to read the next frame. When a scene start frame is detected, the field (atmosphere) estimation unit 32 reads video data and audio data for N frames continuous from the scene start frame (step S3).
[0094]
Based on the subject size information, camera motion information, and lens type information of the camera work data, an estimation target frame used for field (atmosphere) estimation is determined and extracted from the read N frames (step S4). . Here, it is determined whether or not a field (atmosphere) estimation target frame exists in the read N frame (step S5). One frame is read (step S6). Then, based on the position information, the angle information, the subject size information, and the subject number information of the camera work data, an estimation target screen region used for field (atmosphere) estimation in the read estimation target frame is determined and extracted (step S7). Then, the video (data) feature amount of the field (atmosphere) estimation target area is detected and the field (atmosphere) is estimated (step S8).
[0095]
Next, it is determined whether or not the above-described field (atmosphere) estimation process has been completed for all of the field (atmosphere) estimation target frames (step S9). ) When the estimation process is completed, an average value per frame of the field (atmosphere) estimation result is calculated (step S10), and illumination control data (RGB data) corresponding to this is generated and output (step S11). The detection of the audio data feature amount is performed over the above-described N frame period, and is used for the place (atmosphere) estimation process together with the video data feature amount calculated as described above.
[0096]
If it is determined in step S5 that the field (atmosphere) estimation target frame does not exist in the N frames read in step S3, illumination control data based on a default value prepared in advance is generated. Output (step S11). Here, when the field (atmosphere) estimation target frame does not exist, for example, a slightly weak white illumination light is illuminated in the viewing environment space. However, the present invention is not limited to this, and the illumination control data related to the immediately preceding scene is used as it is. It may be outputted continuously to maintain the color and intensity of the viewing environment illumination light.
[0097]
Then, it is determined whether or not the process is finished (step S13). For example, when video data is finished or when stop of viewing environment lighting control is instructed, scene segment detection and field (atmosphere) estimation are performed. The process ends, otherwise, the process returns to step S1 to acquire a new frame.
[0098]
In the present embodiment, as described above, illumination control data is generated based on the feature amounts of video data and audio data included in N frames from the scene start point, and a scene change is then detected from the illumination control data. Therefore, the color and intensity of the viewing environment illumination light can be switched in units of scenes, and the color and intensity of the viewing environment illumination light can be kept substantially constant in the same scene.
[0099]
The video data and audio data separated by the data separation unit 22 are delayed by the delay generation units 28 and 29 by the time required for the estimation process by the field (atmosphere) estimation unit 32, and then the video data Since it is reproduced by the display device 25 and the audio reproduction device 26, the color and intensity of the viewing environment illumination light can be appropriately controlled in accordance with the reproduction timing of the video / audio.
[0100]
Further, a field (atmosphere) estimation process for each of the video scene examples shown in FIGS. 1 to 5 will be described. In the video scene shown in FIG. 1, the frames A to E-1 (the frame immediately before the frame E) are indicated by the hatched area in FIG. 15G and the hatched line in FIG. A video feature amount is detected using video data in a screen area that overlaps the area. For frame E and subsequent frames, the video feature amount is detected using video data in a screen area where the area shown by the oblique lines in FIG. 15G and the area shown by the oblique lines in FIG. To do. As a result, it is possible to generate illumination control data using only the video feature amount of the background portion, and irradiate the appropriate viewing environment illumination light reflecting the illumination state at the time of photographing the scene over the display reproduction period of this scene. It becomes possible.
[0101]
In the video scene shown in FIG. 2, since the subject size is a bust shot for frames C to E-1, the video data feature quantity is not detected. Then, for the frames A to C-1 (the frame immediately before the frame C) and the frames E and thereafter, there are a hatched area in FIG. 15 (e) and a hatched area in FIG. 16 (b). A video feature amount is detected using video data in the overlapping screen area. Accordingly, it is possible to generate illumination control data using only the video feature amount of the background portion, and it is possible to irradiate appropriate viewing environment illumination light reflecting the illumination state at the time of photographing the scene.
[0102]
In the video scene shown in FIG. 3, since zoom photography is performed, the video data feature quantity is detected only for the frame A. Here, for the frame A, the video feature amount is detected by using the video data of the screen area where the area shown by the oblique lines in FIG. 15C and the area shown by the oblique lines in FIG. To do. As a result, it is possible to generate illumination control data using only the video feature amount of the background portion, and irradiate the appropriate viewing environment illumination light reflecting the illumination state at the time of shooting the scene over the display reproduction period of this scene. It becomes possible.
[0103]
In the video scene shown in FIG. 4, since zoom photography is also performed, the video data feature quantity is detected only for the frame A. Here, for the frame A, the video feature amount is detected by using the video data of the screen area where the area shown by the oblique line in FIG. 15E and the area shown by the oblique line in FIG. To do. As a result, it is possible to generate illumination control data using only the video feature amount of the background portion, and irradiate the appropriate viewing environment illumination light reflecting the illumination state at the time of shooting the scene over the display reproduction period of this scene. It becomes possible.
[0104]
In the video scene shown in FIG. 5, the video data feature amount is not detected for frames after the frame D which is switched to the telephoto lens and becomes a bust shot, and the frames A to D-1 (frames immediately before the frame D) are detected. Only for this, the video data feature quantity is detected. Here, for the frames A to D-1, the video data of the screen area where the area shown by the oblique lines in FIG. 15E and the area shown by the oblique lines in FIG. Detect feature values. As a result, it is possible to generate illumination control data using only the video feature amount of the background portion, and irradiate the appropriate viewing environment illumination light reflecting the illumination state at the time of shooting the scene over the display reproduction period of this scene. It becomes possible.
[0105]
As described above, in the present embodiment, the illumination control data is used to appropriately reproduce the lighting situation (atmosphere) at the time of shooting each scene by using various information contents input as camera work data together with video data. Thus, it is possible to control the illumination light of the illumination device 27. Therefore, it is possible to realize a natural and uncomfortable viewing environment illumination without being affected by the video content of the foreground part (subject) and the like, and it is possible to increase a sense of reality when viewing the video. In addition, since the lighting control data is switched and controlled in units of scenes, it is possible to prevent the viewing environment lighting from changing drastically in the same scene and impairing the sense of reality.
[0106]
The screen area patterns shown in FIGS. 15 and 16 can be set as appropriate. In addition, the histogram (frequency distribution) of the video data in the field (atmosphere) estimation screen area obtained from this screen area pattern is detected, and the video feature quantity is detected only from the video data with a high distribution ratio. ) By performing the estimation, it is possible to further improve the accuracy of the field (atmosphere) estimation.
[0107]
In the present embodiment, camera work data relating to the camera work status at the time of shooting each frame is transmitted and received, so that a viewing environment such as searching for or editing a desired frame using this camera work data. Various functions can be realized in addition to the illumination control.
[0108]
For example, by selecting and extracting a frame where the camera position is eye height, the angle is horizontal angle, the subject size (screen size) is a long shot, the camera motion is fixed, and the lens type is taken using a standard lens An index that collects typical video scenes (key frames, key shots) can be created.
[0109]
Furthermore, in the first embodiment of the present invention, the case where camera work data is transmitted while being multiplexed with broadcast data has been described. However, when camera work data is not added to broadcast data, a video to be displayed is displayed. By transmitting and receiving camera work data corresponding to the data from an external server device or the like, it is possible to realize an optimal viewing environment according to the lighting situation at the time of shooting the video. This will be described below.
[0110]
FIG. 18 is a block diagram showing a schematic configuration of a main part of the external server device in the viewing environment control system of the present embodiment. The external server device (data transmission device) in the present embodiment receives video as shown in FIG. A receiving unit 41 that receives a request for transmission of camera work data relating to specific video data (content) from the device (data receiving device) side; and a data storage unit 42 that stores camera work data for each video data (content) A transmission unit 43 that transmits the camera work data that has received the transmission request to the requesting video reception device (data reception device).
[0111]
Here, as shown in FIG. 19, the camera work data stored in the data storage unit 42 of the present embodiment is associated with the start time code (TC) of the frame in which the camera work situation at the time of shooting changes. The camera work data (a) to (f) described in the table format and corresponding to the video data (program content) for which a transmission request has been received is represented by a TC (frame indicating a change in the camera work status at the time of video shooting. The time code) is transmitted from the transmitter 43 to the requesting video receiver.
[0112]
Here, the camera work data may be described for all the frames constituting the video data. However, as shown in FIG. 19, the camera work data is used only for the frames in which the camera work status at the time of shooting changes. Thus, the amount of data stored in the data storage unit 42 can be reduced.
[0113]
Next, a video receiving device (data receiving device) that receives the camera work data sent from the external server device and controls the viewing environment illumination will be described. As shown in FIG. 20, the video receiving apparatus in the present embodiment receives and demodulates broadcast data input from a transmission path, performs error correction, and outputs video from the output data of the receiving unit 51. A data separation unit 52 that separates and extracts each of the video data output to the display device 25 and the audio data output to the audio reproduction device 26, and a transmission request for camera work data corresponding to the video data (content) to be displayed A transmission unit 54 that transmits to an external server device (data transmission device) via a network, and a reception unit 55 that receives the camera work data requested for transmission from the external server device via a communication network are provided.
[0114]
The CPU 53 stores the camera work data received by the receiving unit 55 and outputs control data obtained from the camera work data of the corresponding frame in synchronization with the TC (time code) of the video data. A lighting control data generation unit 24 that outputs lighting control data (RGB data) corresponding to the video feature amount of the frame and the screen area determined according to the output control data to the lighting device 27 that illuminates the viewing environment space. ing.
[0115]
That is, the CPU 53 compares the start time code of each frame of the camera work data storage table received from the external server device and stored therein with the time code of the video data separated by the data separation unit 52, The camera work data corresponding to the coincidence can be read out, and the control data for limiting the frame and the screen area adapted to the field (atmosphere) estimation of the display video scene can be generated and output from the camera work data. .
[0116]
Thereby, even if camera work data is not added to the broadcast data, camera work data corresponding to the display video data (program content) is obtained from the external server device, and illumination control is performed using this camera work data. Data can be generated, and the display switching timing of the video scene and the switching timing of the viewing environment lighting can be synchronized with a simple configuration, thus realizing an optimal viewing environment according to the lighting conditions during video shooting It becomes possible to do.
[0117]
Note that the viewing environment control apparatus, method, and viewing environment control system of the present invention can be realized by various embodiments without departing from the gist of the present invention described above. For example, the viewing environment control device may be provided in the video display device, and it is needless to say that an external lighting device may be controlled based on various information included in the input video data. Yes.
[0118]
In addition, the camera work data described above is not limited to the case where the camera work data is separated / acquired from broadcast data or the case where it is acquired from an external server device. When displaying information, the camera work data added in the media medium may be read out and used.

Claims (24)

The camerawork data indicating the camera work condition at the time of shooting of each frame constituting the movies image data, and Lud over data transmission device to transmit in addition to the video data,
Receiving means for receiving camera work data from the data transmitting device;
Limiting a target video frame or screen area for detecting video feature quantity of the video data based on the camera work data, detecting the video feature quantity of the video data in the limited video frame or screen area, and A viewing environment control device having control means for controlling the illumination light of the lighting device based on the detected video feature amount;
A viewing environment control system characterized by comprising: In the data viewing environment control system according to claim 1,
The viewing environment control system , wherein the camera work data includes at least information indicating a camera position at the time of photographing each frame. In the viewing environment control system according to claim 1 or 2,
The viewing environment control system , wherein the camera work data includes at least information representing a camera angle at the time of photographing each frame. In the viewing environment control system according to any one of claims 1 to 3,
The viewing environment control system , wherein the camera work data includes at least information indicating a size of a subject at the time of photographing each frame. In the viewing environment control system according to any one of claims 1 to 4,
The viewing environment control system , wherein the camera work data includes at least information indicating the number of subjects at the time of photographing each frame. In the audio-visual environment control system according to any one of claims 1 to 5,
The viewing environment control system , wherein the camera work data includes at least information representing a camera movement at the time of photographing each frame. In the viewing environment control system according to any one of claims 1 to 6,
The camerawork data, audio-visual environment control system characterized in that it contains data indicating the type of the camera lens used for shooting of at least the frame. Receiving a request from outside, the camera work data showing the camera work condition at the time of shooting of each frame constituting the video data, and transmitting to Lud over data transmission device with the start timing of each frame constituting the video data ,
Receiving means for receiving the camera work data;
Based on the camera work data, the target video frame or screen area for detecting the video feature quantity of the video data is limited, the video feature quantity in the limited video frame or screen area is detected, and the detected video A viewing environment control device having control means for controlling the illumination light of the lighting device based on the feature amount;
A viewing environment control system characterized by comprising: In the viewing environment control system according to claim 8,
The viewing environment control system , wherein the camera work data includes at least information indicating a camera position at the time of photographing each frame. In the viewing environment control system according to claim 8 or 9,
The viewing environment control system , wherein the camera work data includes at least information representing a camera angle at the time of photographing each frame. The viewing environment control system according to any one of claims 8 to 10,
The viewing environment control system , wherein the camera work data includes at least information indicating a size of a subject at the time of photographing each frame. The viewing environment control system according to any one of claims 8 to 11,
The viewing environment control system , wherein the camera work data includes at least information indicating the number of subjects at the time of photographing each frame. The viewing environment control system according to any one of claims 8 to 12,
The viewing environment control system , wherein the camera work data includes at least information representing a camera movement at the time of photographing each frame. The viewing environment control system according to any one of claims 8 to 13,
The camerawork data, audio-visual environment control system characterized in that it contains data indicating the type of the camera lens used for shooting of at least the frame. A viewing environment control device that controls illumination light of a lighting device based on a video feature amount of video data to be displayed on a display device,
Receiving means for receiving camera work data indicating a camera work situation at the time of photographing each frame constituting the video data;
Based on the camera work data, the target video frame or screen area for detecting the video feature quantity of the video data is limited, the video feature quantity in the limited video frame or screen area is detected, and the detected video A viewing environment control device, comprising: a control unit that controls illumination light of the illumination device based on a feature amount . In the viewing environment control device according to claim 15,
The viewing environment control apparatus, wherein the control means performs switching control of illumination light of the illumination apparatus for each scene constituting the video data. In the viewing environment control device according to any one of claims 15 and 16,
The viewing environment control apparatus, wherein the camera work data includes at least information indicating a camera position at the time of photographing each frame. The viewing environment control device according to any one of claims 15 to 17,
The viewing environment control apparatus, wherein the camera work data includes at least information representing a camera angle at the time of photographing each frame. The viewing environment control device according to any one of claims 15 to 18,
The viewing environment control apparatus, wherein the camera work data includes at least information indicating a size of a subject at the time of photographing each frame. The viewing environment control device according to any one of claims 15 to 19,
The viewing environment control apparatus, wherein the camera work data includes at least information indicating the number of subjects at the time of photographing each frame. The viewing environment control device according to any one of claims 15 to 20,
The viewing environment control apparatus according to claim 1, wherein the camera work data includes at least information representing a camera movement at the time of photographing each frame. The viewing environment control device according to any one of claims 15 to 21,
The viewing environment control apparatus, wherein the camera work data includes at least information indicating a type of a camera lens used for photographing each frame. Wherein the viewing environment control device according to any one of claims 15 to 22, the audio-visual environment illumination control system characterized in that it comprises a lighting device being controlled viewing environment illumination light by the audio-visual environment control device. A viewing environment control method for controlling illumination light of a lighting device based on a video feature amount of video data to be displayed on a display device ,
Receiving a camera work data showing the camera work condition at the time of shooting of each frame constituting the video data,
Limiting the video frame or screen area to be detected based on the camera work data, the video feature amount of the video data,
Detecting the video feature in the restricted video frame or screen area;
A viewing environment control method, comprising: controlling illumination light of the illumination device installed around the display device based on the detected video feature amount .

Images