JP2009081482A - Data transmitter, data transmission method, and unit, system, and method for controlling viewing environment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a viewing environment control system enabling the optimum lighting control of a viewing environment according to the atmosphere and setting of a photographing scene intended by a video producer. <P>SOLUTION: A data transmitter comprises: a data multiplexing section 1 for dividing video data into segments composed of not less than one arbitrary number of frames, and for multiplexing segment break information indicating the start time and/or section of each segment and lighting control type information indicating the control type of viewing environment lighting in the display of each frame of the video data to the video data in segment units; and a transmission section for demodulating the video data, where the segment break information and the lighting control type information are multiplexed, for transmission. A data receiver comprises: a data separation section 32 for separating the segment break information and the lighting control type information from the video data; and a lighting control data generation section 35 for controlling illumination light in a lighting system 38 by the lighting control type information. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a data transmission apparatus and data capable of controlling illumination light around a video display device in accordance with the atmosphere and scene setting of the video shooting scene when the video is displayed on the video display device. The present invention relates to a transmission method, a viewing environment control device, a viewing environment control system, and a viewing environment control method.

  For example, when displaying an image with an image display device such as a television receiver, or when projecting and displaying an image using a projector device, the surrounding illumination light is adjusted according to the display image. There is known a technique for providing a viewing effect such as enhancing the sense of presence.

  For example, Patent Document 1 calculates the mixed light intensity ratio of the three primary colors of the light source for each frame from the color signal (RGB) and luminance signal (Y) of the display image of a color television, and performs dimming control in conjunction with the image. A light color variable illumination device is disclosed. This variable-light-color illuminating device extracts a color signal (RGB) and a luminance signal (Y) from a display image of a color television, and uses the three color lights (red light, green) used as a light source from the color signal and the luminance signal. Light, blue light) is calculated, the illuminance of the three-color light is set according to the illuminance ratio, and the three-color light is mixed and output as illumination light.

  Further, for example, Patent Document 2 discloses a video effect lighting apparatus that performs illumination control around a division unit by dividing a television image into a plurality of portions and detecting an average hue of the corresponding division unit. Yes. This video effect lighting device includes illumination means for illuminating the surroundings of the installation location of the color television, divides the video displayed on the color television into a plurality of parts, and divides the video corresponding to the portion illuminated by the illumination means The average hue of the part is detected, and the illumination means is controlled based on the detected hue.

Furthermore, for example, Patent Document 3 does not simply calculate the average chromaticity and average luminance of the entire screen of the image display device, but instead of pixels of skin color portions such as human faces from the image displayed on the screen of the image display device. The remaining portion is considered as the background portion, and only the RGB signal and luminance signal of each pixel in the background portion are taken out to obtain the average chromaticity and average luminance, and the chromaticity and luminance of the back wall of the image display device are A method of controlling illumination so as to be the same as the average chromaticity and average luminance of the entire screen or the background portion excluding human skin color is disclosed.
Japanese Patent Laid-Open No. 2-158094 JP-A-2-253503 JP-A-3-184203

  Usually, a video scene is created as a segmented video based on a series of scene settings, for example, by the intention of a video producer (screenwriter, director, etc.). Therefore, it is desirable to irradiate the viewing space with illumination light according to the scene status of the displayed video in order to increase the presence and atmosphere when viewing the video.

  However, in the above-described conventional technology, the state of the illumination light changes according to the change in the luminance and hue of the video signal for each frame, particularly when the degree of change in luminance and hue between frames is large. There is a problem that the illumination light changes complicatedly and the viewer feels uncomfortable with flicker. Furthermore, it is not preferable that the illumination light fluctuates in accordance with changes in luminance and hue for each frame during the display of one scene with no change in scene setting, which adversely inhibits the atmosphere for each scene.

  FIG. 11 is a diagram for explaining an example of a problem of illumination control according to the conventional technique. In the example shown in FIG. 11, a scene of a video shot with a scene setting of moonlight night is created. This scene is composed of three shots (1, 2, 3) with different camera work. In Shot 1, the camera takes a long shot of the target ghost. Then, when switching to shot 2, the ghost is shot in an up shot. In shot 3, the camera returns to the camera position of shot 1 again. These shots are intended and configured as a segment of a scene in which one atmosphere continues even though the camera work is different.

  In such a case, in shot 1, a relatively dark image of moonlight night is continuous. When the illumination light is controlled according to the luminance and chromaticity of each frame of these images, the illumination light becomes relatively dark. When shot 1 is switched to shot 2, the ghost captured in the up shot becomes a relatively bright image. At this time, if the illumination light is controlled for each frame according to the above-described conventional technique, the illumination light control is largely switched at the time of switching shots, resulting in bright illumination light. When switching to shot 3 again, it returns to the dark illumination light similar to shot 1.

  That is, if the illumination light becomes darker or brighter in a single segment of scenes where one scene (atmosphere) is continuous, the atmosphere of the scene is disturbed and the viewer feels uncomfortable.

  FIG. 12 is a diagram for explaining another example of a problem caused by a variation in illumination in a scene. In the example illustrated in FIG. 12, a scene of a video shot with a scene setting of outdoors in a sunny day is created. This scene is made up of images obtained by shooting through a series of camera work without switching the camera. In this example, an image of a skier sliding down from above the camera toward the vicinity of the camera is taken. Skiers are dressed in red and the sky is clear.

  The video of this scene has a large blue sky background in the initial frame, and the skier's red clothing area gradually increases as the skier slides down and approaches the camera. That is, as the video in the scene progresses, the ratio of the colors constituting each frame changes.

  In such a case, when the illumination light is controlled using the chromaticity and luminance for each frame, the blue light changes from strong illumination light to red illumination light. In other words, the color of the illumination light changes within a single segment of a scene (atmosphere) that is continuous, and on the contrary, the atmosphere of the scene is hindered and the viewer feels uncomfortable.

  As described above, in the same scene, it is desirable to keep the viewing environment illumination light substantially constant rather than switching the viewing environment illumination light for each frame according to the video feature amount of each frame.

  In addition, shots frequently change in a short time during the opening (summary), trailers, and commercial message (CM) periods of dramas, movies, music programs, etc., so that the video signal for each frame as in the above-described conventional technique. When the illumination light is changed in accordance with changes in brightness and hue, the illumination light changes in a complicated manner, causing a problem that the viewer feels uncomfortable.

  FIG. 13 is a diagram for explaining an example of a problem due to a change in illumination at the time of opening display. The example shown in FIG. 13 is a summary video in which particularly important shots (1, 2, 3, 4) are switched in a short time (several seconds) on the story of the drama program. Here, shot 1 is a sunny day. Inside shot, shot 2 is taken in the early morning on a clear day, shot 3 is taken outdoors in a cloudy day, and shot 4 is taken in an outdoor scene setting on a sunny day.

  In other words, the video feature amount of the frame constituting this video is different for each shot, and when the illumination light is controlled using the chromaticity and luminance for each frame as in the above-described conventional technique, The illumination light also frequently changes in a short time in synchronization with the switching of the display, and on the contrary, the atmosphere of the viewing environment is hindered and the viewer feels uncomfortable.

  In this way, when displaying images in which multiple shots change frequently in a short time, instead of switching the viewing environment illumination light for each frame according to the image feature amount of each frame, a predetermined intensity is used. It is desirable to irradiate the viewing environment space with certain illumination light such as white light.

  And in television programs, it seems like an eye-catching for a short time between scenes (for example, program credits inserted before and after commercials, excerpts and telops of contents taken up in the next corner of variety programs, etc.) May be inserted. In such a case, if the illumination light is changed according to the change in the luminance or hue of the video signal for each frame as in the conventional technique, the short-time video inserted between the scenes can be used. In some cases, the illumination light changes and the viewer feels uncomfortable.

  FIG. 14 is a diagram for explaining an example of a problem caused by a change in illumination when displaying a video in which a short shot video is inserted between scenes. The example shown in FIG. 14 is an image in which a shot of a lake at night is inserted for a short time between a scene 1 consisting of an outdoor scene setting in a sunny day and a scene 2 consisting of an indoor scene setting.

  In such a case, as in the above-described conventional technique, when the illumination light is controlled using the chromaticity and luminance for each frame, and when the scene 1 is switched to the shot, the illumination light is largely switched from the bright illumination light to the dark illumination light. When switching to the scene 2 immediately in a short time, the illumination light is switched again in accordance with this, and on the contrary, the atmosphere of the viewing environment is hindered and the viewer feels uncomfortable.

  Thus, when displaying a video in which a short shot is inserted between scenes, instead of switching the viewing environment illumination light according to the video feature amount of each frame constituting the shot, Either keep the viewing environment illumination light corresponding to the video feature of the previous scene as it is and irradiate the viewing environment space, or irradiate the viewing environment space with a certain illumination light such as white light of a predetermined intensity. May be desirable.

  Furthermore, scenes with special video processing, such as reminiscence scenes in dramas and movie programs, may be inserted. In such a case, special video effects are added in the conventional technology described above. The illumination light is changed according to the change in the luminance and hue of the video signal for each frame, which may cause a problem that the viewer feels uncomfortable.

  In this way, when displaying a video that has undergone special video processing, the viewing environment illumination light is not switched according to the video feature quantity of each frame, but the viewing environment according to the video feature quantity of the immediately preceding scene. It may be desirable to irradiate the viewing environment space while maintaining the illumination light as it is, or to irradiate the viewing environment space with a certain illumination light such as white light of a predetermined intensity.

  For example, a sports program competition scene and a studio recording scene such as news / reports and information / wide show are images shot under a certain illumination in a sports venue or a recording studio. However, when the illumination light is changed according to the change in the luminance and hue of the video signal for each frame as in the conventional technique, the background artifacts, the human face, clothes, etc. included in the video signal Under the influence, the illumination light of an inappropriate color is radiated to the surroundings, and on the contrary, the atmosphere of the scene may be obstructed and the viewer may feel uncomfortable.

  In this way, when displaying an image shot under constant white light illumination, the viewing environment illumination light is not switched according to the image feature amount of each frame, but white light with a predetermined intensity, etc. Either illuminate the viewing environment space with a certain amount of illumination light, or maintain the viewing environment illumination light according to the video feature amount of the frame including the lighting of the stadium or studio shot with a long shot as it is in the viewing environment space. It is desirable to irradiate.

  On the other hand, there are cases where it is desirable to switch the viewing environment illumination light according to the video feature quantity of each frame, such as a scene where lightning runs in the night sky, so the viewing environment illumination light is updated according to the video content to be displayed It is necessary to change the timing to be performed. However, the display image that requires switching of the viewing environment illumination light in units of frames has a low appearance frequency, and indicates whether the viewing environment illumination light should be updated or maintained for every frame constituting the video data. When the lighting control type information to be added is added, there is a problem that the amount of data is unnecessarily increased.

  The present invention has been made in view of the above problems, and realizes optimal illumination control of the viewing environment by appropriately controlling the timing of switching the illumination light of the viewing environment while suppressing an increase in the amount of data. It is an object to provide a data transmission device, a data transmission method, a viewing environment control device, a viewing environment control system, and a viewing environment control method.

  According to a first aspect of the present invention, in a data transmitting apparatus that transmits video data composed of one or more frames, the video data is divided into segments composed of an arbitrary number of one or more frames. Segment delimiter information indicating a start time and / or section and illumination control type information indicating a control type of viewing environment illumination when displaying each frame of the video data are added to the video data in units of segments. It is characterized by transmitting.

  According to a second aspect of the present invention, the lighting control type information includes information indicating a maintenance mode for maintaining and maintaining the immediately preceding viewing environment illumination light when displaying all the frames in the segment. Features.

  The third invention of the present application is characterized in that the illumination control type information includes information indicating a full update mode for updating the viewing environment illumination when displaying all frames in the segment.

  In the fourth invention of the present application, the illumination control type information updates only the viewing environment illumination when displaying the first frame in the segment, and maintains the viewing environment illumination when displaying the subsequent frames. It includes information indicating an update mode to be kept constant.

  The fifth invention of the present application is characterized in that the illumination control type information includes information indicating a frame control mode for changing the control type of the viewing environment illumination for each frame in the segment.

  In a sixth aspect of the present application, when the lighting control type information indicates the frame control mode, the viewing environment lighting when displaying the frame is updated for each frame in the segment, It includes information instructing whether to maintain the previous viewing environment illumination.

  In a seventh aspect of the present application, the lighting control type information includes information indicating a default lighting mode in which the viewing environment illumination light is maintained in a predetermined state when displaying all the frames in the segment. It is characterized by that.

  The eighth invention of the present application is characterized in that the illumination control type information includes information indicating an extinguishing mode for keeping the viewing environment illumination in an extinguished state when displaying all the frames in the segment.

  The ninth invention of the present application is characterized in that the illumination control type information includes information indicating a non-control mode in which control of viewing environment illumination is stopped when all the frames in the segment are displayed.

  According to a tenth aspect of the present invention, video data to be displayed on a display device, segment delimiter information indicating a start time and / or a section of a segment obtained by dividing the video data into one or more arbitrary numbers of frames, and the segment Receiving means for receiving illumination control type information indicating a control type of viewing environment illumination when displaying each frame constituting the video data, which is associated with the video data in units; and in the illumination control type information And a control means for controlling the illumination light of the illumination device installed around the display device.

  An eleventh invention of the present application includes the viewing environment control device and a lighting device whose viewing environment illumination light is controlled by the viewing environment control device.

  According to a twelfth aspect of the present invention, in a data transmission method for transmitting video data composed of one or more frames, the video data is divided into segments composed of an arbitrary number of one or more frames. Segment delimiter information indicating a start time and / or section and illumination control type information indicating a control type of viewing environment illumination when displaying each frame of the video data are added to the video data in units of segments. It is characterized by transmitting.

  A thirteenth invention of the present application is a data transmission method for transmitting illumination control type information indicating a control type of viewing environment illumination when displaying each frame constituting video data in response to an external request, Segment delimiter information indicating a start time and / or section of a segment obtained by dividing the video data by an arbitrary number of one or more frames and the lighting control type information are transmitted in units of segments.

  A fourteenth aspect of the present invention provides video data to be displayed on a display device, segment delimiter information indicating a start time and / or a section of a segment obtained by dividing the video data by an arbitrary number of one or more frames, and the segment Lighting control type information indicating a control type of the viewing environment lighting when displaying each frame constituting the video data, which is associated with the video data in units, and based on the lighting control type information, The illumination light of the illumination device installed around the display device is controlled.

  According to the present invention, it becomes possible to appropriately control the illumination light of the viewing environment by adapting to the atmosphere of the shooting scene intended by the video producer and the scene setting, giving the viewer a sense of realism and a more sophisticated experience. A video effect can be obtained.

<First Embodiment>
Hereinafter, a first embodiment of the viewing environment control system of the present invention will be described in detail with reference to FIGS. Here, FIG. 1 is a block diagram illustrating a schematic configuration of a main part of a video transmission device in the viewing environment control system of the present embodiment, and FIG. 2 illustrates an example of a data structure of additional data in the viewing environment control system of the present embodiment. FIG.

  FIG. 3 is an explanatory diagram showing an example of scene break information in the viewing environment control system of this embodiment, FIG. 4 is an explanatory diagram showing an example of lighting control type information in the viewing environment control system of this embodiment, and FIG. 5 is this embodiment. 6 is a diagram for explaining the operation of the illumination control mode in the viewing environment control system according to the embodiment, FIG. 6 is a diagram for explaining the components of the video, and FIG. 7 is a diagram of the video receiving device in the viewing environment control system of the present embodiment. It is a block diagram which shows a part schematic structure.

  As shown in FIG. 1, a video transmission device (data transmission device) in the present embodiment includes a data multiplexing unit 1 that multiplexes each of additional data including video data, audio data, segment delimiter information, lighting control type information, and the like. And a transmission unit 2 that performs modulation after adding an error correction code to the output data of the data multiplexing unit 1 and sends the data to the transmission line as broadcast data.

  FIG. 2 is a diagram for explaining an example of the data structure of the additional data. In the present embodiment, video data is divided into segments each composed of an arbitrary number of one or more frames, and various additional information related to each segment is described in segment units.

  As shown in FIG. 3, the segment delimiter information includes a segment number, a start time, and section information. The segment start time is specified by a time code added to indicate playback time information of each of the video data and audio data. For example, as shown in FIG. 3, the time (h): minute (m) of the video data : Second (s): It is composed of information indicating a frame (f). The segment section is composed of information indicating a time (h): minute (m): second (s): frame (f) indicating a period from the time indicating the start time code of the segment to the start time code of the next segment. Has been.

  Here, both the segment start time (start frame) and section (number of frames) are described, but at least one of them may be described. The segment break in the case of only one of them can be uniquely determined if the segment is continuous with the next segment and there is no overlap. For example, a frame in which certain lighting control type information is set may be set as the start frame of the segment, and a frame up to the frame in which lighting control type information is set next may be set as the segment segment.

  Here, the segment is defined as a configuration in which video content is divided by an arbitrary length in the time direction. For example, the segment is a segment corresponding to a scene, a shot, a frame, or the like. For example, when a segment corresponds to a scene, information existing in shot units is collectively described in the segment. When a segment corresponds to a shot, the same information is repeatedly described in all the corresponding segments as information existing in a scene unit. In addition, when a segment corresponds to a frame, the same information is repeatedly described in all the corresponding segments as information existing in scene units and shot units.

  Note that the viewing environment illumination is mainly controlled in units of scenes in which the situation of the scene is constant, but is often controlled in units of shots in which the situation of shooting is constant. In some cases, fine-grained control in units of frames is sometimes desirable, so the amount of data to be added can be reduced by dividing the segments into units that often require the viewing environment lighting to be controlled to be constant depending on the content of the display data. The additional data can be described efficiently while suppressing the increase.

  The illumination control type information indicates the control type of the viewing environment illumination when displaying each frame included in the segment, and details thereof will be described later with reference to FIG.

  The scene setting information indicates the scene scene setting status of the video data included in the segment, and specifically, the season information indicating the season on the story of the scene, the time information indicating the time, and the place indicating the place Information, period information representing the era, weather information representing the weather, area information representing the area, and the like are described.

  By using this scene setting information, it is possible to estimate the illumination light according to the scene status of the displayed video (scene setting status on the story) and irradiate the viewing space regardless of the feature amount of the video to be displayed (For details, see PCT / JP2007 / 057458 specification previously filed by the applicant of the present application). Since the scene setting information changes in units of scenes, the scene setting information can be efficiently described in units of segments by dividing the segments in accordance with the scenes.

  The shooting illumination information indicates the lighting status at the time of shooting of the video data included in the segment. Specifically, the lighting type information indicates the lighting type of the shooting lighting used for shooting the video data. Describes lighting fixture information representing the nature of the photographic illumination, irradiation direction information representing the direction of the photographic illumination light, illumination intensity information representing the intensity of the photographic illumination light, illumination color information representing the chromaticity of the photographic illumination light, etc. Has been.

  By using this shooting illumination information, it is possible to estimate the illumination light according to the lighting situation at the time of video shooting and irradiate the viewing space regardless of the feature amount of the video to be displayed. PCT / JP2007 / 057760 specification previously filed by the applicant of the present application). In addition, since the photographing illumination information often changes in shot units or scene units, the photographing illumination information can be efficiently described in segment units by dividing the segments according to shots and scenes.

  Furthermore, the camera work information indicates the camera work status at the time of shooting of the video data included in the segment, and specifically indicates the position information indicating the camera position at the time of shooting the video data, and the camera angle. Angle information, subject size information representing the size of the subject, subject number information representing the number of subjects, camera motion information representing camera motion, lens work information representing the type of camera lens, and the like are described.

  By using this camera work information, for example, it is possible to prevent inappropriate viewing environment lighting by estimating illumination light using a video feature amount such as a foreground part (subject) (for example, PCT / JP2007 / 058035 specification previously filed by the applicant of the present application). Since this camera work information often changes in units of shots, the camera work information can be efficiently described in units of segments by dividing the segments in accordance with the shots.

  In the present embodiment, scene setting information, shooting illumination information, and camera work information are included in the additional data in order to obtain more appropriate viewing environment illumination. However, these pieces of information are not necessarily required.

  Next, the details of the illumination control type information of this embodiment will be described with reference to FIG. As shown in FIG. 4, in this embodiment, seven types of lighting control modes are defined: a maintenance mode, a full update mode, an update mode, a non-control mode, a turn-off mode, a default lighting mode, and a frame control mode. One mode is described and specified in the mode specification area for one segment. Only when the frame control mode is designated, control type designation data for all frames included in the segment is described in the control type designation area.

  Each illumination control mode will be described with reference to FIG. The maintenance mode is an operation mode in which when the video data corresponding to the segment section is displayed, the immediately previous viewing environment illumination is maintained and kept constant. The all update mode is an operation mode for updating the viewing environment illumination when displaying all the frames corresponding to the segment section. The update mode is an operation mode in which only the viewing environment illumination when displaying the first frame in the segment section is updated, and the viewing environment illumination when displaying subsequent frames is maintained and kept constant.

  The non-control mode is an operation mode in which the control of the viewing environment illumination when displaying all the frames corresponding to the segment section is stopped. The extinguishing mode is an operation mode in which the viewing environment illumination is switched to the extinguishing state when the video data corresponding to the segment section is displayed. The default illumination mode is an operation mode in which, when displaying the video data corresponding to the segment section, for example, the viewing environment illumination (default illumination) having a predetermined brightness and color such as bright white illumination is switched and maintained. is there.

  Furthermore, the frame control mode is an operation mode for changing the control type of the viewing environment illumination in units of each frame in the segment section, and is used when displaying the frame described for each frame in the segment. The viewing environment illumination is controlled according to the control type of the viewing environment illumination. Here, the viewing environment illumination when displaying each frame can be controlled based on any one of maintenance, update, non-control, turn-off, and default described for each frame. .

  The additional data is transmitted by an extension header (adaptation field) of a transport stream packet (TSP) defined by, for example, MPEG2 (Moving Picture Experts Group 2) -Systems, and is composed of data such as video and audio. Embedded in the payload to be transmitted. Further, the data streams of video data, audio data, and additional data may be multiplexed and transmitted. In the present embodiment, segment delimiter information and lighting control type information may be added to video data or audio data, and the data structure at that time is not limited to the above. Further, the additional data may be described in a text format, or may be encoded and described in a binary format.

  Here, the configuration of a video including scenes and shots will be described with reference to FIG. Video data constituting a continuous moving image sequence can be divided into three layers as shown in FIG. The 1st layer which comprises an image | video (Video) is a flame | frame (Frame). A frame is a physical layer and refers to a single two-dimensional image. Frames are usually obtained at a rate of 30 frames per second. The second layer is a shot. A shot is a sequence of frames taken by a single camera. The third layer is a scene. A scene is a sequence of shots having a story-like connection.

  Here, as described above, the video data is divided into segments corresponding to scenes or shots composed of an arbitrary number of one or more frames, and the above-mentioned illumination control type information (illumination control mode) is set for each segment. Because it is added, the viewing environment lighting when displaying each frame is optional as described later according to the intention of the video producer (screenwriter, director, etc.) while suppressing an increase in the amount of data It is possible to control appropriately by switching at the timing.

  Even in the same scene and shot, when it is desirable to control the switching of the viewing environment illumination in units of frames, instead of making the viewing environment illumination constant, the frame is not increased without increasing the number of segment divisions. This can be dealt with by the description of the control mode.

  Next, a video reception device (data reception device) that receives broadcast data transmitted from the video transmission device, displays and reproduces video and audio, and controls the viewing environment illumination at that time will be described.

  As shown in FIG. 7, the video receiving apparatus in the present embodiment receives and demodulates broadcast data input from the transmission path, and performs error correction, and receives video from the output data of the receiving unit 31. Separate and extract each of additional data including video data to be output to the display device 36, TC (time code), audio data to be output to the audio playback device 37, TC (time code), segment delimiter information, lighting control type information, etc. And a data separator 32.

  Further, based on the additional data separated by the data separation unit 32 and / or the feature amount of the video data and the audio data, illumination control data (RGB data) suitable for displaying each frame is generated, and the viewing environment space is set. An illumination control data generation unit 35 that outputs to the illumination device 38 that illuminates, and delay generation units 33 and 34 that delay and output video data and audio data by the processing time in the illumination control data generation unit 35 are provided.

  Here, the illumination device 38 can be configured by an LED that is installed around the video display device 36 and emits light of, for example, three primary colors of RGB having a predetermined hue. However, the illumination device 38 may be configured to be able to control the illumination color and brightness of the surrounding environment of the video display device 36, and is not limited to the combination of LEDs that emit the predetermined color as described above. Alternatively, a white LED and a color filter may be used, or a combination of a white light bulb, a fluorescent tube and a color filter, a color lamp, or the like may be applied. Further, it is sufficient that one or more lighting devices 38 are installed.

  The time code is information added to indicate the reproduction time information of each of the video data and the audio data. For example, the time (h): minute (m): second (s): frame ( f).

  The illumination control data generation unit 35 of the present embodiment performs illumination estimation processing suitable for displaying each frame based on the feature amounts of video data and audio data, or scene setting information and / or shooting included in additional data. Based on the illumination information, an illumination estimation process suitable for displaying each scene and shot is performed. Furthermore, when camerawork information is included in the additional data, illumination estimation processing suitable for each frame display can be performed based on the camerawork information and the feature amounts of the video data and audio data. Further, the illumination estimation processing may be performed by weighted averaging the illumination estimation result based on the scene setting information and / or the photographic illumination information and the illumination estimation result based on the feature amount of the video data and audio data. .

  Note that the illumination control data generation unit 35 can use various techniques including those known in the art for estimating the state of ambient light during photographing. Note that here, in addition to the feature amount of the video data, the feature amount of the audio data is also used for the estimation of the place (atmosphere), but this is for improving the estimation accuracy of the place (atmosphere). The field (atmosphere) of the shooting scene may be estimated from only the feature amount of the video data.

  As the feature amount of the video data, for example, as in the conventional example described above, the color signal and luminance signal in a predetermined area of the screen can be used as they are, and the color temperature of ambient light at the time of video shooting is obtained from these. May be used. Furthermore, these may be configured to be switchable and output as feature amounts of video data. Further, as the feature amount of the audio data, a volume, an audio frequency, or the like can be used.

  As described above, in the present embodiment, the mode for performing the illumination estimation process using the feature amount of the video data and the audio data, and the scene setting information included in the additional data without using the feature amount of the video data and the audio data. And / or modes for performing illumination estimation processing only from photographic illumination information, and these modes can be switched and set by the viewer on the receiving device side.

  Furthermore, the illumination control data generation unit 35 controls the switching timing to the viewing environment illumination according to the result of the illumination estimation process described above, based on the illumination control type information included in the additional data. In other words, when the lighting control mode described and specified for each segment is the maintenance mode, all the segments in the segment section, regardless of the content of the video data, audio data features, scene setting information, and shooting lighting information Control is performed so that the previous viewing environment illumination is maintained without change as the viewing environment illumination when the frame is displayed. Thereby, even when a segment included in the same scene as the immediately preceding segment is displayed, it is possible to realize viewing environment illumination that maintains the atmosphere when displaying the immediately preceding segment.

  In addition, when the lighting control mode described and specified for each segment is the full update mode, the viewing environment lighting when displaying all the frames in the segment is updated according to the result of the above-described lighting estimation processing. Control to do. As a result, it is possible to control the dynamic viewing environment illumination that closely follows the content of the video to be displayed.

  Furthermore, when the lighting control mode described and specified for each segment is the update mode, only the viewing environment lighting when displaying the first frame in the segment is updated according to the result of the above-described lighting estimation processing. Then, the viewing environment illumination when displaying subsequent frames is controlled so as to be maintained until the segment ends. Thereby, even when the content of the video in the segment changes frequently, it is possible to realize a certain viewing environment illumination that matches the atmosphere of the segment.

  Further, when the illumination control mode described / designated for each segment is the non-control mode, the control of the viewing environment illumination when displaying all the frames in the segment is stopped. As a result, when displaying a video that is inappropriate to change the viewing environment lighting, such as a video of a news program, for example, the viewing environment lighting control linked to the content of the video is not performed and the viewer desires. Arbitrary viewing environment lighting can be realized.

  Furthermore, when the lighting control mode described / designated for each segment is the extinguishing mode, regardless of the contents of the video data, audio data feature amount, scene setting information, and shooting lighting information, Control is performed so that the viewing environment illumination when displaying the frame is switched to the off state and held. As a result, it is possible to realize a viewing environment in which the influence of lighting is eliminated and the viewer is concentrated only on the video.

  In addition, if the lighting control mode described and specified for each segment is the default lighting mode, it is included in the segment regardless of the contents of the video data, audio data feature amount, scene setting information, and photographing lighting information. Control is performed so that the viewing environment illumination when displaying all the frames is switched to a predetermined default illumination and maintained. As a result, it is possible to reproduce a specific viewing environment illumination that is determined in advance.

  In addition, when the lighting control mode described / designated for each segment is the frame control mode, viewing when displaying each frame based on the control type described corresponding to each frame in the segment. Control ambient lighting. That is, when displaying a frame for which maintenance is specified, control is performed so as to maintain the previous viewing environment illumination. When displaying the frame for which the update is specified, the viewing environment illumination is switched according to the result of the above-described illumination estimation process. In addition, when displaying a frame for which non-control is designated, the control of the viewing environment illumination is stopped. When displaying a frame designated to be turned off, the viewing environment illumination is controlled to be turned off. When displaying a frame for which default illumination is specified, control is performed so that the viewing environment illumination has a predetermined brightness and color. As a result, it is also possible to perform fine viewing environment illumination switching control in units of frames.

  As described above, according to the present embodiment, when the lighting control type information instructs to switch the viewing environment illumination, the video feature amount / audio feature amount of each frame, or scene setting information, Based on the content of the photographic illumination information, the illumination state and scene setting (atmosphere) of the photographic field are estimated, and illumination control data for controlling the illumination device 38 is output based on the estimation result. When the lighting control type information instructs to maintain the viewing environment lighting, the immediately preceding lighting control data is repeatedly output to the lighting device 38 without performing the field (atmosphere) estimation process.

  In addition, the illumination control data generation unit 35 stores illumination control data corresponding to one or more predetermined brightness and color illumination lights in a storage unit (not shown). When the lighting control type information instructs to switch to a predetermined predetermined viewing environment lighting, the above-mentioned lighting control data is not stored, but the above-described place (atmosphere) estimation process is not performed. Is read from the storage unit and output.

  Similarly, when the lighting control type information instructs to switch the viewing environment illumination to the extinguished state, the above-described place (atmosphere) estimation processing is not performed, and the viewing environment illumination is turned off. Output control data.

  As a result, regardless of the video feature value / audio feature value of each frame, or the contents of the scene setting information and shooting illumination information, the predetermined illumination light is determined in advance at the timing specified by the illumination control type information. By switching, the viewing environment space can be irradiated or the viewing environment space can be turned off.

  On the other hand, the video data and the audio data output to the video display device 36 and the audio reproduction device 37 are generated by the delay generation units 33 and 34 for the time necessary for the field (atmosphere) estimation process using the video data and the audio data. Due to the delay, the illumination control data output from the video reception device to the illumination device 38 is synchronized with the video data and audio data output to the video display device 36 and the audio reproduction device 37, and the video display is performed. It becomes possible to switch the illumination light of the illumination device 38 at the synchronized timing.

  As described above, in the present embodiment, the viewing environment illumination is controlled based on the illumination control type information (illumination control mode) added in units of segments including one or more arbitrary frames. Thus, it is possible to perform the viewing environment illumination switching control at an arbitrary timing according to the intention of the video producer while suppressing an increase in the data amount.

  In other words, the viewing environment illumination light can be held in the same state for any period regardless of changes in the feature amount of video data, audio data, scene setting information, or shooting illumination information. In the same scene where the field (atmosphere), that is, the illumination state does not change, the viewing environment illumination light can be kept substantially the same.

  Further, in a period when the control of the viewing environment illumination based on the video feature amount / audio feature amount is inappropriate, for example, it is possible to hold and irradiate a certain white illumination light. Accordingly, it is possible to prevent the viewer from feeling uncomfortable due to inappropriate viewing environment lighting, and to realize an optimal viewing environment.

  Furthermore, since the lighting control type information (lighting control mode) is added in units of segments corresponding to scenes, shots, etc., the amount of data is greatly reduced compared to the case where the lighting control type information is added for each frame. It is possible. In addition, by providing a frame control mode as one of the lighting control modes, the number of segment divisions can be used even in the same scene, shot, etc. This can be accommodated without unnecessarily increasing.

  In the first embodiment of the present invention, the segment delimiter information and the illumination control type information are described as being added to the broadcast data and transmitted. However, when the additional data is not added to the broadcast data, By transmitting and receiving segment delimiter information and illumination control type information corresponding to video data to be displayed from an external server device or the like, it is possible to realize an optimum viewing environment during video playback. This will be described below as a second embodiment of the present invention.

<Second Embodiment>
Hereinafter, the second embodiment of the viewing environment control system of the present invention will be described in detail with reference to FIG. 8 to FIG. . Here, FIG. 8 is a block diagram showing a schematic configuration of the main part of the external server device in the viewing environment control system of the present embodiment, and FIG. 9 shows an example of a lighting control type information storage table in the viewing environment control system of the present embodiment. FIG. 10 is an explanatory diagram, and FIG. 10 is a block diagram showing a schematic configuration of a main part of the video receiving apparatus in the viewing environment control system of the present embodiment.

  As shown in FIG. 8, the external server device (data transmitting device) in the present embodiment receives a request for transmitting illumination control type information related to specific video data (content) from the video receiving device (data receiving device) side. Unit 51, a data storage unit 52 that stores lighting control type information for each video data (content), and a lighting control type information that has received a transmission request, together with segment delimiter information, a request source video receiving device (data A transmission unit 53 for transmission to a reception device.

  Here, the illumination control type information stored in the data storage unit 52 of the present embodiment is described in a table format in association with the start time information of each segment, as shown in FIG. The lighting control type information of the received video data (program content) is transmitted from the transmitting unit 53 to the requesting video receiving device together with the TC (time code) indicating the start time of the segment.

  Next, a video receiving device (data receiving device) that controls the viewing environment lighting by receiving the segment break information and the lighting control type information sent from the external server device will be described. As shown in FIG. 10, the video receiving apparatus according to the present embodiment receives and demodulates broadcast data input from a transmission path, performs error correction, and outputs video from the output data of the receiving unit 61. A data separation unit 62 that separates and extracts each of the video data output to the display device 36 and the audio data output to the audio playback device 37, and a transmission request for lighting control type information corresponding to the video data (content) to be displayed. A transmission unit 67 that transmits to an external server device (data transmission device) via a communication network, and a reception unit 68 that receives the illumination control type information requested for transmission from the external server device via the communication network together with segment delimiter information. And.

  Further, the segment delimiter information and the illumination control type information received by the receiving unit 68 are temporarily stored. The start time TC (time code) described as the segment delimiter information and the video data extracted by the data demultiplexing unit 62 are stored. TC (time code) is compared, and when they match, the CPU 66 outputs the corresponding lighting control type information, the lighting control type information from the CPU 66, and the feature amount of the video data and the audio data On the basis of this, an illumination control data generation unit 65 that generates illumination control data (RGB data) and outputs the illumination control data to the illumination device 38 that illuminates the viewing environment space is provided.

  That is, the CPU 66 receives the start time code of each segment of the lighting control type information storage table received from the external server device and stored therein, and the time code of the video data input to the lighting control data generation unit 65. When these are matched, the lighting control type information corresponding to the segment is output to the lighting control data generation unit 65. The illumination control data generation unit 65 generates and outputs illumination control data (RGB data) suitable for displaying each frame according to the illumination control type information.

  That is, when the lighting control type information instructs to switch the viewing environment lighting based on the video feature value / audio feature value of each frame, shooting is performed based on the video data and audio data of the frame. On-site lighting conditions and scene settings (atmosphere) are estimated, and lighting control data for controlling the lighting device 38 is output based on the estimation result.

  If the lighting control type information instructs to maintain and control the previous viewing environment lighting (no lighting switching) regardless of the video feature value / audio feature value of each frame, The process of estimating the field (atmosphere) is not performed, and the illumination control data output immediately before is repeatedly output.

  Further, when the lighting control type information instructs to switch to predetermined predetermined viewing environment lighting regardless of the video feature amount / audio feature amount of each frame, The field (atmosphere) estimation process is not performed, and illumination control data prepared in advance is read and output.

  Thereby, even if the illumination control type information is not added to the broadcast data, the illumination control type information corresponding to the display video data (program content) is obtained from the external server device, and based on the illumination control type information. Since the configuration is such that the viewing environment illumination is controlled, as in the first embodiment described above, the viewing environment illumination switching control is performed at an arbitrary timing according to the intention of the video producer while suppressing an increase in the data amount. It is possible to achieve optimal illumination control of the viewing environment by appropriately switching between viewing environment lighting according to the feature amount of the video data and predetermined predetermined viewing environment lighting. Become.

  In this embodiment, only segment segment information and illumination control type information are transmitted / received to / from an external server device, and illumination estimation processing based on only the video feature amount / audio feature amount of each frame is described. However, as in the first embodiment, in addition to segment delimiter information and illumination control type information, scene setting information, shooting illumination information, camera work information, and the like are simultaneously transmitted and received, so that various illumination estimation processes can be performed. Needless to say, the configuration can be performed.

  The viewing environment control apparatus, method, and viewing environment control system of the present invention can be realized by various embodiments without departing from the gist of the present invention described above. For example, the environmental lighting control device may be provided in the video display device, and it is needless to say that an external lighting device may be controlled based on various information included in the input video data. Yes.

  Furthermore, the segment delimiter information and the illumination control type information described above are not limited to the case of being separated / acquired from broadcast data or the case of being acquired from an external server device. For example, external devices (DVD player, Blu-ray disc player, etc.) When displaying the video information reproduced in the above, segment segment information and lighting control type information recorded in the media medium may be read out and used. In this case, the segment delimiter information and the illumination control type information may be added to the video information, or may be recorded in a different area from the video information in the media medium. Further, the segment delimiter information and the lighting control type information may be obtained from a media medium other than the media medium on which the video information is recorded or an external server device. Needless to say, it is only necessary to be associated with the video data.

It is a block diagram which shows the principal part schematic structure of the video transmission apparatus in the viewing-and-listening environment control system which concerns on 1st Embodiment of this invention. It is a figure for demonstrating an example of the data structure of the additional data in the viewing-and-listening environment control system which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows an example of the scene break information in the viewing-and-listening environment control system which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows an example of the illumination control classification information in the viewing-and-listening environment control system which concerns on 1st Embodiment of this invention. It is a figure for demonstrating operation | movement of the illumination control mode in the viewing-and-listening environment control system which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the component of an image | video. It is a block diagram which shows the principal part schematic structure of the video receiver in the viewing-and-listening environment control system which concerns on 1st Embodiment of this invention. It is a block diagram which shows the principal part schematic structure of the external server apparatus in the viewing-and-listening environment control system which concerns on 2nd Embodiment of this invention. It is explanatory drawing which shows an example of the illumination control classification information storage table in the viewing-and-listening environment control system which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the principal part schematic structure of the video receiver in the viewing-and-listening environment control system which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating an example of the problem of the illumination fluctuation | variation by a prior art. It is a figure for demonstrating the other example of the problem of the illumination fluctuation | variation by a prior art. It is a figure for demonstrating the other example of the problem of the illumination fluctuation | variation by a prior art. It is a figure for demonstrating the other example of the problem of the illumination fluctuation | variation by a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Data multiplexing part 2 ... Transmission part 31, 61 ... Reception part 32, 62 ... Data separation part 33, 34 ... Delay generation part 35, 65 ... Illumination control data generation part 36 ... Video display device 37 ... Audio reproduction device 38 ... Illumination device 51... Reception unit 52... Data storage unit 53.
67 ... Transmitter 68 ... Receiver

Claims (14)

In a data transmission apparatus for transmitting video data composed of one or more frames,
The video data is divided into segments composed of an arbitrary number of one or more frames, segment delimiter information indicating the start time and / or section of each segment, and the viewing environment when displaying each frame of the video data A data transmission apparatus, wherein illumination control type information indicating a control type of illumination is added to the video data and transmitted in units of segments.   The said lighting control classification information contains the information which instruct | indicates the maintenance mode which maintains the last viewing environment illumination and is kept constant, when displaying all the frames in the said segment. Data transmission device.   The data transmission according to claim 1 or 2, wherein the lighting control type information includes information indicating a full update mode for updating the viewing environment lighting when displaying all frames in the segment. apparatus.   The illumination control type information updates only the viewing environment illumination when displaying the first frame in the segment, and indicates an update mode for maintaining the viewing environment illumination when displaying subsequent frames and keeping it constant. 4. The data transmission device according to claim 1, further comprising information.   The said lighting control classification information contains the information which instruct | indicates the frame control mode which changes the control classification of viewing environment lighting for each frame in the said segment, The said Claim 1 thru | or 4 characterized by the above-mentioned. Data transmission device.   When the frame control mode is instructed, the illumination control type information updates, for each frame in the segment, the viewing environment illumination when displaying the frame, or maintains the previous viewing environment illumination. The data transmitting apparatus according to claim 5, further comprising information indicating whether or not.   2. The lighting control type information includes information indicating a default lighting mode that maintains a predetermined state of viewing environment lighting when displaying all frames in the segment. The data transmission device according to any one of 1 to 6.   8. The lighting control type information includes information indicating a light-off mode for keeping the viewing environment light in a light-off state when displaying all the frames in the segment. The data transmission device described.   9. The lighting control type information includes information indicating a non-control mode for stopping the control of the viewing environment lighting when displaying all the frames in the segment. The data transmission device described in 1. Video data to be displayed on a display device, segment delimiter information indicating the start time and / or section of a segment obtained by dividing the video data by an arbitrary number of one or more frames, and the video data in association with each segment Receiving means for receiving the illumination control type information indicating the control type of the viewing environment illumination when displaying each frame constituting the video data,
A viewing environment control device comprising: a control unit that controls illumination light of a lighting device installed around the display device based on the lighting control type information.   11. A viewing environment control system comprising: the viewing environment control device according to claim 10; and a lighting device whose viewing environment illumination light is controlled by the viewing environment control device. In a data transmission method for transmitting video data composed of one or more frames,
The video data is divided into segments composed of an arbitrary number of one or more frames, segment delimiter information indicating the start time and / or section of each segment, and the viewing environment when displaying each frame of the video data A data transmission method characterized in that illumination control type information indicating a control type of illumination is added to the video data and transmitted in units of segments. In response to an external request, a data transmission method for transmitting illumination control type information indicating a control type of viewing environment illumination when displaying each frame constituting video data,
Data transmission characterized by transmitting segment delimiter information indicating a start time and / or section of a segment obtained by dividing the video data by an arbitrary number of one or more and the lighting control type information in units of segments. Method. Video data to be displayed on a display device, segment delimiter information indicating the start time and / or section of a segment obtained by dividing the video data by an arbitrary number of one or more frames, and the video data in association with each segment Lighting control type information indicating a control type of viewing environment lighting when displaying each frame constituting the video data,
A viewing environment control method, comprising: controlling illumination light of an illumination device installed around the display device based on the illumination control type information.