JP5442643B2 - Data transmission device, data transmission method, viewing environment control device, viewing environment control method, and viewing environment control system - Google Patents

Description

  The present invention relates to a data transmission device, a data transmission method, a viewing environment control device, a viewing environment control method, and a viewing environment control system, and in particular, a data transmission device and a data transmission method for controlling peripheral devices in a user's viewing environment space. The present invention relates to a viewing environment control device, a viewing environment control method, and a viewing environment control system.

  In recent years, due to the rapid advancement of electronic technologies related to video and audio, the display has been increased in size, wide viewing angle and high definition, and the surround system has been improved. As a result, realistic video and audio can be enjoyed. For example, in a home theater system that is now widely used, a system that can provide a high sense of realism is realized by a combination of a large display or screen and multi-channel audio / acoustic technology.

  In particular, recently, a system that combines a plurality of media to enhance the realism has been actively developed. For example, a system that allows users to view a wide-field image by combining multiple displays, and a system that links the image displayed on the display with the illumination light of the lighting device, rather than simply enjoying the image on a single display device Has been proposed.

  In particular, in a technique for realizing a high sense of realism by linking a display and a lighting device, a high sense of realism can be obtained without using a large display. For this reason, there are few restrictions such as cost and installation space, and great expectations are placed, which attracts great attention.

  According to this technique, the illumination light of a plurality of illumination devices installed in a viewer's room (viewing environment space) is controlled to a color or intensity corresponding to an image displayed on a display. As a result, it is possible to give the viewer a sense and effect as if the viewer actually exists in the video space displayed on the display. For example, Patent Document 1 discloses a technique for linking an image displayed on such a display with illumination light of an illumination device.

  The illumination system disclosed in Patent Document 1 is intended to provide a high sense of presence, and is an illumination system that controls a plurality of illumination devices in conjunction with an image to be displayed. In this system, illumination control data for a plurality of illumination devices is generated from feature amounts of video data such as representative colors and average luminance. Specifically, a feature amount of video data in a predetermined screen area is detected according to the installation position of each lighting device, and illumination control data for each lighting device is generated based on the detected feature amount. Yes. The lighting control data is not limited to the data calculated from the feature amount of the video data, but may be used alone or with the video data distributed via the Internet or the like and distributed by the carrier wave. This is described in Patent Document 1.

  Further, for example, in Patent Document 2, in a system for controlling ambient light, the ambient light is not simply set to reflect the average color of the entire screen of video data, but reacts to the color at a specific screen position. It describes that ambient light can be set.

Japanese Patent Publication “JP 2001-343900 A” (published on December 14, 2001) Japanese Patent Gazette "Special Table 2005-531909 Gazette" (announced on October 20, 2005)

  In the conventional technique described above, the feature amount (color signal and luminance signal) at a specific screen position for each frame (screen) in the video signal to be displayed is detected, and the illumination light is controlled. For this reason, it is difficult to generate illumination light that matches the field (atmosphere) of the video depending on the content of the displayed video. For example, if an inappropriate color illumination light is irradiated around the subject under the influence of clothes of the subject person included in the video signal or artifacts in the background of the subject, the atmosphere of each scene is reproduced. , And the realistic sensation of each scene cannot be maintained. Viewing environment lighting that deviates significantly from the lighting situation at the time of shooting a video scene, on the contrary, impairs the sense of reality.

  In the conventional technique, the state of the illumination light changes in accordance with the change in the luminance or hue of the video signal for each frame. Therefore, particularly when the degree of change in luminance and hue between frames is large, there is a problem that illumination light changes complicatedly and viewers feel uncomfortable due to flicker. Furthermore, when a single scene is displayed with no change in the lighting situation at the time of shooting, the illumination light greatly fluctuates according to changes in luminance and hue for each frame, which adversely inhibits the reproduction of the atmosphere in each scene. This is not preferable.

  Also, video scenes are usually created by shooting under appropriate lighting conditions as a single segment based on a set of scene settings, for example, for the purpose of video producers (screenwriters, directors, etc.) Is done. Therefore, it is desirable to irradiate the viewing space with illumination light in accordance with the illumination status at the time of scene shooting of the displayed video in order to enhance the sense of presence and the atmosphere when viewing the video.

  Therefore, the present invention has been made in view of the above problems, and its purpose is to provide a viewing environment in which an optimal viewing environment can be realized by appropriately controlling a viewing environment device that adjusts the viewing environment. A control device and a data transmission device are provided.

  A viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames in order to solve the above-described problem. Receiving means for receiving reference information indicating a calculation area when calculating the feature quantity of the video data or audio data; and calculation means for calculating the feature quantity of the video data or audio data in the calculation area indicated by the reference information It is the structure provided with.

  The viewing environment device is a device for adjusting the surrounding environment when the user views the video, and is intended to be, for example, a lighting device and an air conditioner.

  According to the above configuration, the viewing environment control device receives the reference information indicating the calculation area when calculating the feature amount of the video data or the audio data, and the video data or the audio data in the calculation area indicated by the received reference information The feature amount is calculated. The viewing environment control device controls the viewing environment device based on the calculated feature amount.

  Therefore, the viewing environment device can be controlled based on the feature amount in a specific area of the video data or audio data specified by the reference information received from the outside. Therefore, for example, when calculating the feature amount of the video data, it is possible to eliminate the influence of the subject person's clothes included in the video data or the artifacts in the background of the subject. It is possible to control the viewing environment device according to the atmosphere or scene setting. As a result, it is possible to provide the user with a viewing environment that provides a high sense of realism.

  A viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames in order to solve the above-described problem. , Receiving means for receiving calculation designation information indicating a calculation method for calculating the feature quantity of the video data or audio data, and calculating the feature quantity of the video data or audio data by the calculation method indicated by the calculation designation information And a calculation means.

  According to the above configuration, the viewing environment control device receives the operation designation information indicating the calculation method when calculating the feature amount of the video data or the audio data, and the video data or the audio data is calculated by the calculation method indicated by the received operation designation information. The feature amount of the audio data is calculated. The viewing environment control device controls the viewing environment device based on the calculated feature amount.

  Therefore, the viewing environment device can be controlled based on the feature amount of the video data or audio data obtained by the specific calculation method designated by the computation designation information received from the outside. Therefore, for example, when calculating the feature amount of the video data, it is possible to obtain an appropriate feature amount according to the video content, and the viewing environment device is adapted to the shooting scene atmosphere or scene setting intended by the content creator. Can be controlled. As a result, it is possible to provide the user with a viewing environment that provides a high sense of realism.

  A viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames in order to solve the above-described problem. Receiving means for receiving correction information indicating an allowable range for calculating the feature amount of the video data or audio data; and calculating for calculating the feature amount of the video data or audio data within the allowable range indicated by the correction information And means.

  According to the above configuration, the viewing environment control device receives the correction information indicating the allowable range when calculating the feature amount of the video data or the audio data, and the video data or the video data within the allowable range indicated by the received correction information. The feature amount of the audio data is calculated. The viewing environment control device controls the viewing environment device based on the calculated feature amount.

  Therefore, the calculated feature amount falls within the allowable range specified by the correction information received from the outside. Therefore, it is possible to prevent the feature amount used for controlling the viewing environment device from being repeatedly increased and decreased. Thereby, it is possible to prevent the driving intensity of the viewing environment device from fluctuating violently and prevent the viewing environment from changing greatly.

  A viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on control reference data received from the outside in order to solve the above-described problem. It comprises a calculating means for calculating a feature amount of data and a correcting means for correcting the control reference data using the feature amount of the video data or audio data.

  According to the above configuration, the viewing environment control apparatus calculates a feature amount of video data or audio data received from the outside, and corrects control reference data received from the outside using the calculated feature amount. The viewing environment control device controls the viewing environment device based on the corrected control reference data.

  Therefore, the control reference data for the viewing environment device received from the outside can be corrected using the feature amount calculated from the video data or the audio data, and the viewing environment device can be controlled based on this data. It is possible to provide a viewing environment that follows changes in the contents of video and audio without greatly deviating from the atmosphere or scene setting of the shooting scene.

  In order to solve the above-described problem, a data transmission device according to the present invention provides data to a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data including one or more frames. Is a configuration including a transmission unit that transmits reference information indicating a calculation area when calculating the feature amount of the video data or audio data.

  According to the above configuration, the data transmission device transmits the reference information to the viewing environment control device. The reference information indicates a calculation area when the viewing environment control apparatus calculates a feature amount of video data or audio data. The calculated feature amount is used when the viewing environment control apparatus controls the viewing environment device.

  Therefore, when the reference information is transmitted to the viewing environment control apparatus, the viewing environment control apparatus calculates the feature amount of the video data or the audio data in the calculation area indicated by the reference information. The viewing environment control device controls the viewing environment device based on the calculated feature amount. Therefore, it is possible to control the viewing environment device based on the feature amount in a specific area of video data and audio data designated in advance. Accordingly, the viewing environment device can be controlled in accordance with the atmosphere or scene setting of the shooting scene intended by the content creator. Therefore, it is possible to provide the user with a viewing environment that can provide a high sense of realism.

  In order to solve the above-described problem, a data transmission device according to the present invention provides data to a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data including one or more frames. Is a data transmission device that includes transmission means for transmitting calculation designation information indicating a calculation method for calculating the feature amount of the video data or audio data.

  According to the above configuration, the data transmission device transmits the calculation designation information to the viewing environment control device. The calculation designation information indicates a calculation method used when the viewing environment control apparatus calculates the feature amount of the video data or audio data. The calculated feature amount is used when the viewing environment control apparatus controls the viewing environment device.

  Therefore, when the calculation designation information is transmitted to the viewing environment control device, the viewing environment control device calculates the feature amount of the video data or the audio data based on the calculation method indicated by the calculation designation information. The viewing environment control device controls the viewing environment device based on the calculated feature amount. Therefore, it is possible to control the viewing environment device based on the feature amount of video data or audio data obtained by a specific calculation method specified in advance. Accordingly, the viewing environment device can be controlled in accordance with the atmosphere or scene setting of the shooting scene intended by the content creator. Therefore, it is possible to provide the user with a viewing environment that can provide a high sense of realism.

  In order to solve the above-described problem, a data transmission device according to the present invention provides data to a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data including one or more frames. Is a configuration including transmission means for transmitting correction information indicating an allowable range when calculating the feature amount of the video data or audio data.

  According to the above configuration, the data transmission device transmits the correction information to the viewing environment control device. The correction information indicates an allowable range when the viewing environment control apparatus calculates the feature amount of the video data or audio data. The calculated feature amount is used when the viewing environment control apparatus controls the viewing environment device.

  Therefore, when the correction information is transmitted to the viewing environment control device, the viewing environment control device calculates the feature amount of the video data or the audio data within the allowable range indicated by the correction information. For this reason, the calculated feature amount falls within an allowable range designated in advance. The viewing environment control device controls the viewing environment device based on the feature amount falling within the allowable range. Thereby, it is possible to prevent the feature amount used for controlling the viewing environment device from being repeatedly increased and decreased. Therefore, it is possible to prevent the drive strength of the viewing environment device from fluctuating drastically and prevent the viewing environment from changing greatly.

  In order to solve the above problems, a data transmission device according to the present invention is a data transmission device that transmits control reference data for controlling a viewing environment device to a viewing environment control device, wherein the control reference data is This is a configuration set for each segment which is a set of frames constituting video data or audio data.

  According to the above configuration, the data transmission device transmits the control reference data set for each segment, which is a set of frames constituting video data or audio data.

  Therefore, when the control reference data is transmitted to the viewing environment control device, the viewing environment control device controls the viewing environment device based on the control reference data. At this time, control can be performed for each segment unit which is a set of frames. By dividing the segments in correspondence with scenes, shots, frames, etc., the viewing environment device can be controlled in accordance with the scene setting. As a result, it is possible to provide the user with a viewing environment that provides a high sense of realism.

  A viewing environment control method according to the present invention is a viewing environment control method for controlling a viewing environment device based on a feature amount of video data or audio data composed of one or more frames in order to solve the above problem. A receiving step of receiving reference information indicating a calculation area when calculating a feature amount of the video data or audio data, and a calculation step of calculating a feature amount of the video data or audio data in the calculation region indicated by the reference information It is the structure containing these.

  In addition, a viewing environment control method according to the present invention is a viewing environment control method for controlling a viewing environment device based on a feature amount of video data or audio data composed of one or more frames in order to solve the above problem. A receiving step of receiving calculation designation information indicating a calculation method for calculating the feature amount of the video data or audio data; and a feature amount of the video data or audio data by the calculation method indicated by the calculation designation information. And a calculation step for calculating.

  In addition, a viewing environment control method according to the present invention is a viewing environment control method for controlling a viewing environment device based on a feature amount of video data or audio data composed of one or more frames in order to solve the above problem. A receiving step of receiving correction information indicating an allowable range when calculating the feature amount of the video data or audio data, and calculating the feature amount of the video data or audio data within the allowable range indicated by the correction information. It is also possible to have a configuration including a calculating step.

  In addition, a viewing environment control method according to the present invention is a viewing environment control method for controlling a viewing environment device based on control reference data received from the outside in order to solve the above-described problem. Alternatively, it may be configured to include a calculation step of calculating a feature amount of audio data and a correction step of correcting the control reference data using the feature amount of the video data or audio data.

  According to the above configuration, the viewing environment control method according to the present invention has the same effects as the viewing environment control apparatus according to the present invention.

  According to the viewing environment control apparatus according to the present invention, it is possible to control appropriate viewing environment equipment according to video data or audio data, and thus it is possible to provide a viewing environment suitable for video or audio to the user.

It is a schematic block diagram which shows one Embodiment of the viewing-and-listening environment control system which concerns on this invention. It is a schematic block diagram which shows one Embodiment of the data transmitter which concerns on this invention. It is a figure which shows an example of the data structure of metadata. It is a figure which shows the structure of segment delimiter information. It is a figure which shows the example of the description content of each data in the apparatus control information for manual control. It is a figure which shows an example of arrangement | positioning in the viewing environment of the viewing environment control system which concerns on this invention. It is a figure which shows the specific example of the area | region information which changes over several frames. It is the table which represented the area | region information shown in FIG. 7 as the coordinate of a block. It is a figure showing the color information of the apparatus control information for automatic control classified according to color temperature. It is a figure showing the acoustic information of the apparatus control information for automatic control. It is a figure regarding the threshold value setting of the illumination data according to the reference illumination data of automatic setting mode. It is a flowchart which shows the flow of the process which determines illumination data from color signal data. It is a figure showing the change range of reference illumination data with a color difference. It is a figure showing the impression which the user receives by the difference in color difference. It is a temperature change table for converting into temperature data based on color temperature. It is a block diagram which shows the principal part structural example of the external server apparatus which concerns on this Embodiment. It is a schematic block diagram which shows another embodiment of the viewing-and-listening environment control system which concerns on this invention.

[Embodiment 1]
An embodiment of the present invention will be described below with reference to FIGS. In the present embodiment, a lighting device, an air conditioner, a blower, and a vibration device are described as examples of peripheral devices arranged in the viewing environment space. However, any device that controls the viewing environment is limited to these devices. is not. For example, the present invention can be applied to a scent generator.

  First, the data transmission apparatus according to the present embodiment will be described below with reference to FIG.

(Data transmission device)
FIG. 2 is a block diagram showing a schematic configuration showing an embodiment of the data transmission apparatus according to the present invention. The data transmission device is a device that transmits video data, audio data, and metadata to a viewing environment control system described later. As shown in FIG. 2, the data transmission device 1 includes a data multiplexing unit 11, a transmission unit (transmission means) 12, and a data encoding unit 13.

  The data encoding unit 13 compresses and encodes the input video data and outputs it to the data multiplexing unit 11. For video coding, various compression methods such as ISO / IEC 13818-2 (MPEG-2 Video), ISO / IEC 14496-2 (MPEG-4 Visual), ISO / IEC 14496-10 (MPEG-4 AVC), etc. Is available.

  In addition, the data encoding unit 13 compresses and encodes the input audio data and outputs it to the data multiplexing unit 11. Various compression schemes such as ISO / IEC 13818-7 (MPEG-2 AAC) and ISO / IEC 14496-3 (MPEG-4 Audio) can be used for audio encoding.

  In addition, the data encoding unit 13 compresses and encodes the input metadata and outputs it to the data multiplexing unit 11. Details of the metadata will be described later. As a metadata description method, for example, an XML (Extensible Markup Language) format is used. As a metadata compression method, a BiM (Binary format for MPEG-7) method in ISO / IEC 15938-1 (MPEG-7 Systems) can be used. Note that the XML format may be output without compression.

  The data multiplexing unit 11 multiplexes the encoded video data, audio data, and metadata. As the multiplexing method, for example, MPEG-2 transport stream packet (TSP), IP packet, RTP packet, etc. in ISO / IEC 13818-1 (MPEG-2 Systems) can be used.

  For example, when using a transport stream packet (TSP) defined in MPEG-2, metadata is described in an extension header portion following a header in which information defined in MPEG-2 is described, It is possible to transmit video data and audio data using the payload following the header. Alternatively, the metadata may be transmitted using a payload in the same manner as video data and audio data. Further, different data streams of video data, audio data, and metadata may be multiplexed and transmitted.

  The transmission unit 12 transmits or stores the multiplexed video data, audio data, and metadata as broadcast data.

  In this embodiment, the three types of data, that is, video data, audio data, and metadata, are multiplexed and transmitted as broadcast data. However, multiplexing is not an essential requirement. And an appropriate transmission method may be selected. For example, the respective data may be transmitted separately without being multiplexed, or only the video data and audio data may be multiplexed and the metadata may be transmitted independently.

  Further, as will be described later, the metadata is stored in an external server device accessible via the Internet or the like, and a URL (Uniform Resource Locator) for identifying the stored metadata is multiplexed with the video data. May be transmitted. When the metadata itself is transmitted through a network different from the video / audio multiplexed data, the information (identification data) for associating the metadata with the video / audio multiplexed data is not limited to the URL, but is a CRID (Content) in the TV-Anytime standard. Any specific information may be used as long as the correspondence between the metadata and the video / audio multiplexed data can be specified, such as Reference ID) and content name.

  Alternatively, only the metadata may be recorded and distributed on another recording medium. For example, video / audio data may be recorded and distributed on a large-capacity recording medium such as a Blu-ray Disc or DVD, and metadata may be recorded and distributed on a small semiconductor recording medium or the like. In this case, when a plurality of contents are recorded and distributed, specific information that can clarify the correspondence between the video / audio data and the metadata is included together with the video / audio data.

  In the present embodiment, metadata is handled as separate data, but it goes without saying that it may be described in one data format including the data contents of both.

(Metadata)
Next, metadata will be described below with reference to FIGS.

  FIG. 3 is a diagram illustrating an example of a data structure of metadata. In the present embodiment, video data is divided into segments each composed of an arbitrary number of one or more frames, and various additional information related to each segment is described in segment units. Various additional information includes segment delimiter information Ds, control information Dr, automatic control device control information Da, and manual control device control information Dm.

  Although details will be described later, automatic control refers to viewing using the output video data or audio data according to a predetermined processing method (for example, a region calculation designation or acoustic calculation designation processing method). It determines and controls the driving strength of environmental equipment. On the other hand, the manual control is to control the viewing environment device by directly using the control data designated in advance (manually) directly from the transmission side.

[Segment information]
FIG. 4 shows the configuration of the segment delimiter information Ds. The segment delimiter information is information for specifying a segment. In the present embodiment, as shown in FIG. 4, the segment delimiter information Ds is composed of a segment number, a start time (start point), and section information. The segment start time is specified by a time code added to indicate reproduction time information of each of video data and audio data. For example, as shown in FIG. 4, the video data is composed of information indicating time (h): minute (m): second (s): frame (f). The segment section is constituted by information indicating a time (h): minute (m): second (s): frame (f) indicating a period from the time indicating the start time code of the segment to the start time code of the next segment. Has been.

  Here, both the start time of the segment (start frame) and the section (number of frames) are described, but at least one of them may be described. Even in the case of only one of them, the segment break can be uniquely determined if the segment is continuous with the next segment and there is no overlap. For example, a frame in which information related to certain lighting control is set may be set as a start frame of the segment, and a frame in which information related to lighting control is set next may be set as a section of the segment.

  In this specification, a segment is defined as a configuration of video content divided into arbitrary lengths in the time direction, and is divided in accordance with, for example, a scene, a shot, or a frame. For example, when a segment corresponds to a scene, information existing in shot units is collectively described in the segment. When a segment corresponds to a shot, information existing in a scene unit is repeatedly described in all corresponding segments. Further, when a segment corresponds to a frame, information existing in a scene unit and a shot unit is repeatedly described in all corresponding segments.

  In the control of the viewing environment device, there are many controls mainly in scene units in which the situation of the scene is constant, but there are not a few cases in which control is performed in shot units in which the situation of shooting is constant. In some cases, fine control on a frame basis is desirable. By dividing the segment into units in which the viewing environment device should be controlled to be constant depending on the contents of the video data, it is possible to efficiently describe metadata while suppressing an increase in the amount of data to be added.

[Control information]
The control information Dr defines which data of manual control device control information Dm and automatic control device control information Da is used to control the viewing environment device in each segment. That is, it shows a control method for each viewing environment device when each frame included in each segment is output to the video display device. As shown in FIG. 3, in the present embodiment, two types of control methods, an automatic control mode and a manual control mode, are defined, and one mode is designated as control information Dr for one segment as control information Dr. Described and specified in the area.

[Manual control device control information]
The manual control device control information Dm is control data designated by the content creator for each viewing environment device in order to reproduce the environmental situation at the time of shooting the video data included in each segment. Specifically, illumination data representing photographing illumination used for photographing video data, wind power data representing wind power, temperature data representing temperature, vibration data representing vibration, and the like are described.

  FIG. 5 is a diagram showing an example of the description contents of each data in the manual control device control information Dm. As shown in FIG. 5, the effect control type and condition (value indicating the effect) to be added to the content are input to the manual control device control information Dm.

  In the case of the lighting effect, the brightness of lighting (unit: lx) and the color temperature of lighting (unit: K) in the viewing environment recommended by the content creator or content provider are specified as lighting conditions. In addition, for expression of brightness, not only looks (lx) but also candela (Cd) and lumen (lm) may be used. For color expression, not the color temperature but an XYZ color system, an RGB color system, a YCbCr color system, or the like may be used.

  The video data can produce the atmosphere and presence of each scene with illumination light. Therefore, the illumination conditions for the illumination device are useful information. For example, by controlling the lighting in the viewing environment, the realistic sensation of each scene of contemporary and historical drama can be improved. That is, the color temperature of light of a lighting device generally used in modern dramas is about 5000K for a fluorescent lamp (day white), about 6700K for a fluorescent lamp (daylight color), and about 2800K for an incandescent light bulb. On the other hand, the color temperature of the flame of a candle often used as a light source at night in a historical drama is about 1800-2500K. In addition, lighting intensity tends to be high in modern plays and low lighting intensity in historical plays. Therefore, when an indoor scene at dusk is projected in a historical drama, viewing in an environment where the color temperature of the lighting is high and the lighting intensity is strong does not provide a sense of reality, and the intention of the content creator or content provider It will be viewed in a viewing environment contrary to.

  For this reason, the content creator or the content provider has a drive condition for appropriately controlling the peripheral viewing environment device in the content viewing environment when viewing video data and audio data. It is desirable to specify. As this condition, the content creator or the content provider actually views the video data and audio data, and manually specifies the lighting conditions (brightness, color temperature) in each scene, for example. Can be created. Alternatively, by analyzing video data or video encoded data, the average brightness and dominant color in the entire screen or around the screen are automatically extracted, and the brightness and color temperature of the lighting are based on the extracted brightness information and color information. It is also possible to determine illumination conditions such as

  In the present embodiment, lighting, wind, temperature, and vibration are shown as effect types, but in addition to these, a certain sense of realism is provided in the viewing space in conjunction with content playback, including fragrance. Anything can be given.

  By using the device control information Dm for manual control, the viewing environment device can create an appropriate viewing space by recording the environmental state at the time of capturing the image in segments, regardless of the feature amount of the image to be displayed. Can be controlled. Note that the manual control device control information Dm often changes in units of frames or shots, so that the shooting environment information can be described in units of segments by dividing the segments in accordance with the frames or shots.

  Each segment only needs to include at least one of automatic control device control information Da and manual control device control information Dm. From the viewpoint of suppressing an increase in the amount of data, May be included. For example, when the control information Dr indicates the automatic control mode, describe only the automatic control device control information Da, and when the control information Dr indicates the manual control mode, describe the automatic control device control information Da. Instead, only the manual control device control information Dm may be described.

[Device control information for automatic control]
The automatic control device control information Da indicates a restriction to be applied when the data receiving device described later generates control data for each viewing environment device using video data and audio data included in each segment in the automatic control mode. Compensation information, reference information used to extract a reference feature value when the data receiving apparatus applies a restriction, and operation specification indicating an operation specifying method for calculating a reference feature value when the data receiving apparatus applies a restriction Information or control reference information (control reference data) that is control data designated by the content creator.

  Specifically, the reference information is a target area that is an area for extracting (calculating) a video feature amount (also referred to as an image feature amount) that is a feature amount of video data in one or more representative frames in each segment. Target area (calculation area) for extracting area information indicating (calculation area, screen area) and acoustic feature quantity (also referred to as voice feature quantity) that is a feature quantity of voice data in one or more representative frames in each segment , Frequency domain).

  In the correction information, color correction information representing a color range permitted as a video feature amount and acoustic correction information representing a frequency region of an audio level permitted as an acoustic feature amount are described in the generated control data. Although not shown, luminance correction information representing a luminance range allowed as a video feature amount may be included.

  The calculation designation information includes area calculation designation information indicating that the area calculation method is specified, acoustic calculation designation information indicating that the acoustic calculation method is specified, and the like. The representative frame of each segment is determined by selecting in advance for each segment or by automatically selecting some frame of each segment.

  The control reference information indicates color reference information and acoustic reference information predetermined by the content creator, which is information of the same type as the video feature quantity and the acoustic feature quantity extracted from each frame.

  As will be described later, in the automatic control mode, the video feature amount or the audio feature amount is appropriately calculated from the video data to be displayed or the audio data to be reproduced using the automatic control device control information Da, and the feature amount is calculated. Based on this, control data for the viewing environment device that defines the driving strength of the viewing environment device is generated. Then, the viewing environment device is controlled using the control data. Therefore, it is possible to appropriately determine the driving strength of the viewing environment device according to the content of the display video or audio (scene setting status on the story) and output it to the viewing environment device.

  The area information relates to information obtained by dividing the display screen of the video display device as shown in FIGS. 7 and 8 into one or more predetermined blocks. For example, as shown in FIG. 8, information such as block coordinates (x1, y1) and (x2, y2) may be stored. Of course, the area information is not limited to the information related to the blocks divided in units of pixels as described above. Even in the case of division in units of pixels, it is not necessary to make the number of pixels the same in the x direction or the y direction, and the size of the block may be changed freely. Also, area information such as a RegionLocator descriptor of ISO / IEC 15938-3 (MPEG-7 Visual Descriptions) or a Spatial Temporal Locator descriptor may be referred to. Furthermore, as shown in FIG. 7, the block may change within a segment. In this case, as shown in FIG. 8, information on the coordinates (x1_s, y1_s) and (x2_s, y2_s) of the block at the start of change, and the coordinates (x1_e, y1_e) and (x2_e, y2_e) at the end of the change. Record. Various linear interpolations or nonlinear interpolations are performed on the size and shape of the block in the section where there is a change.

  When the area information is described in the automatic control device control information Da, the image feature amount in the screen area specified by the area information is extracted from the representative frame and the output frame, and the extracted image feature is extracted. Control data for controlling a specific viewing environment device is generated using the amount. That is, as will be described later, the data receiving apparatus calculates the video feature quantity in the screen area specified by the area information in the representative frame and the video feature quantity in the screen area specified by the same area information in each frame. The final feature amount in each frame is calculated and control data is generated.

  Next, details of the color correction information (also referred to as color information) will be described. In the automatic control mode section in the video data, the above-described color information can be transmitted as the automatic control device control information Da for metadata after designating a desired color. For example, brightness (unit: lx), illumination color temperature (unit: K), and the like are designated as illumination conditions. In addition, for expression of brightness, not only looks (lx) but also candela (Cd) and lumen (lm) may be used. Also, as shown in FIG. 9, the color temperature ranges are categorized in advance, and terms such as “hot”, “warm”, “moderate”, and “cool” are used. You may express using. Furthermore, the color expression may use an XYZ color system, an RGB color system, a YCbCr color system, or the like instead of the color temperature. Note that both the color information and the area information may be transmitted, or only one of them may be transmitted.

  As described above, the color correction information represents a color range allowed as a final feature amount. Therefore, in the threshold processing of the viewing environment control method to be described later, the color correction information specified by the content creator and the color extracted in each frame are used without using the threshold set and held by the user in the threshold setting unit. From the feature amount, a final feature amount that falls within the allowable range can be obtained. Then, the control data of the viewing environment device is generated from the final feature amount within the allowable range. That is, in the same segment, the change of the color range in illumination can be limited to a desired allowable range.

  Next, details of the acoustic correction information (also referred to as acoustic information) will be described with reference to FIG. In the automatic control mode section in the video data, it is possible to specify the desired sound frequency and transmit it as metadata reference information. The acoustic information is specified as a frequency (unit: Hz), for example. Here, the acoustic information relates to information specifying one or more sound frequencies as shown in FIG. For example, as shown in FIG. 10, information such as the wind power application frequency Fw and the vibration application frequency Fv may be stored. Of course, the acoustic information is not limited to the frequency designation. Even if the frequency is specified, the frequency range may be freely changed. For example, information such as Fw_s to Fw_e for the range of the applied wind frequency and Fv_s to Fv_e for the range of the vibration applied frequency may be stored.

  In the case where a frequency range is stored as the acoustic correction information, this range is the final feature amount allowable range. Therefore, in the threshold processing of the viewing environment control method to be described later, the sound correction information specified by the content creator and the audio corresponding to the frame to be displayed are used without using the threshold set and held by the user in the threshold setting unit. From the sound pressure level (signal intensity) that is the feature amount of the data, a final feature amount that falls within the allowable range can be obtained. Then, the control data of the viewing environment device is generated from the final feature amount within the allowable range. That is, in the same segment, changes in wind power and vibration in the wind power device and the vibration device can be limited to a desired tolerance.

  Next, calculation designation information will be described. The calculation designation information indicates a method for calculating the video feature quantity or the acoustic feature quantity.

  When the area calculation method is specified by the content creator, the video feature amount in the screen area is calculated according to the specified area calculation method. Examples of the area calculation method include calculation of an average color in a predetermined screen area, calculation of a high luminance area color, calculation of a color corresponding to an illumination color temperature (correlated color temperature), and a color corresponding to a memory color (for example, Calculation of blue sky, sunset, fresh green color, etc.), but is not limited to this.

  In addition, when the sound calculation method is designated by the content creator, the sound pressure level (signal intensity) within the frequency range is calculated according to the designated sound calculation method. Examples of acoustic calculation methods include, but are not limited to, calculation of an average sound pressure level and a maximum sound pressure level within a frequency range.

  The device control information for automatic control is used to generate control data for controlling driving of the viewing environment device in an effect control data generating unit of the data receiving apparatus described later. In this embodiment, in order to suppress the data amount of the automatic control device control information Da, each of the automatic control device control information Da is set only for the representative frame in the segment. It goes without saying that automatic control device control information Da may be set for each frame.

(Viewing environment control system)
The viewing environment control system according to the present embodiment will be described below with reference to FIG.

  FIG. 1 is a schematic configuration diagram showing an embodiment of a viewing environment control system 100 according to the present invention. As shown in FIG. 1, the viewing environment control system 100 includes a data receiving device (viewing environment control device) 2, a video display device (display device) 3, an audio playback device 4, a lighting device (viewing environment device) 5, and a wind power device. (Viewing environment device) 6, an air conditioner (viewing environment device) 7, and a vibration device (viewing environment device) 8 are configured. The viewing environment control system 100 is used together with the data transmission device 1.

  The data receiving device 2 receives broadcast data provided from the data transmitting device 1 provided outside the viewing environment control system 100 and receives video data and audio data included in the broadcast data as a video display device 3 and an audio playback device 4. Output to. Further, after the metadata is converted into drive control data for driving and controlling each viewing environment device, the metadata is output to each viewing environment device such as the illumination device 5 in synchronization with the corresponding video data or the like. Thus, a viewing environment suitable for the video displayed on the video display device 3 is formed.

(Data receiver)
The data receiving apparatus 2 includes a receiving unit (receiving unit) 21, a data separating unit 22, delay generating units 23 and 24, an effect control data generating unit (calculating unit, correcting unit) 25 as a drive control data generating unit, and a threshold setting unit. 26.

  The receiving unit 21 receives broadcast data in which video data, audio data, and metadata are multiplexed transmitted from the transmission device 1. The receiving unit 21 demodulates broadcast data input from the transmission path and performs error correction.

  The data separator 22 separates and extracts video data, audio data, and metadata from the broadcast data received by the receiver 21. The separated video data and audio data are transmitted to the delay generators 23 and 24 together with a TC (time code) indicating the start time (start point) of the video data and audio data. The video data is sent to the video display device 3 via the delay generator 23. The audio data is sent to the audio reproduction device 4 via the delay generator 24. Also, the separated video data and audio data are sent to the effect control data generation unit 25 together with the separated metadata.

  The effect control data generation unit 25 is based on the video data, audio data, and metadata separated by the data separation unit 22, the lighting device 5, the wind device 6, and the air conditioner that are installed in the viewer's actual viewing environment space. Illumination dimming data (RGB data), wind power data (wind speed data), temperature data, and vibration data are generated as drive control data for appropriately driving and controlling the device 7 and the vibration device 8, respectively. Then, the lighting dimming data, wind power data, temperature data, and vibration data are output to the lighting device 5, the wind power device 6, the air conditioner 7, and the vibration device 8, respectively.

  The video feature amount and the acoustic feature amount used when generating the drive control data are extracted by the effect control data generation unit 25.

  The threshold setting unit 26 holds the threshold used by the effect control data generation unit 25 so that the effect control data generation unit 25 can use it. The effect control data generation unit 25 uses the threshold value stored in the threshold setting unit 26 to generate data for driving and controlling the viewing environment device installed in the viewing environment space of the user. Here, the threshold value held by the threshold value setting unit 26 is set in advance. However, for example, the setting may be changed according to a user operation, or according to the content type (genre) of the video to be displayed. May be variably set.

  The delay generators 23 and 24 delay the video data and audio data separated by the data separator 22, and synchronize each drive control data with the video data and audio data. For example, the video data and the audio data separated in the data separation unit 22 are delayed by the time required for the conversion to the drive control data in the effect control data generation unit 25. Thereby, the illumination dimming data sent to the lighting device 5, the wind power data sent to the wind power device 6, the temperature data sent to the air conditioner 7, and the vibration data sent to the vibration device 8, the video data and the audio data The output timing can be matched.

  The data receiving device 2 of the viewing environment control system 100 may be provided integrally with the video display device 3 and the audio reproduction device 4, or may be provided separately from each other.

(Viewing environment equipment)
As the illuminating device 5, for example, LED light sources of R (red), G (green), and B (blue) colors that can be independently controlled to emit light are arranged with a certain interval. it can. The illumination device 5 emits illumination light having a desired color and brightness using the LED light sources of these three primary colors. However, the illumination device 5 is not limited to the combination of LED light sources that emit light of a predetermined color, as long as the illumination color and brightness of the surrounding environment of the video display device 3 can be controlled. . For example, you may comprise by white LED and a color filter, the combination of a white light bulb or a fluorescent tube and a color filter, a color lamp, etc. can also be applied. Furthermore, the expression is not limited to R (red), G (green), and B (blue) colors, but may be expressed using, for example, illumination color temperature (unit: K).

  As the wind power device 6, a fan, a fan, or a blower can be used.

  As the air conditioner 7, an air conditioner (air conditioner) can be used.

  As the vibration device 8, a vibration chair or a vibration mat can be used.

  FIG. 6 shows an example in which the lighting devices 5a, 5b, 5c, the wind power devices 6a, 6b, the air conditioner 7, and the vibration device 8 are driven in conjunction with each other.

(Viewing environment control method)
Next, a method of controlling each viewing environment device using metadata will be described below with reference to FIGS.

  The effect control data generation unit 25 determines whether the control information Dr is the automatic control mode or the manual control mode, and derives effect control data suitable for each mode.

  Specifically, when the manual control mode is designated by the control information Dr, the effect control data generation unit 25 corrects the illumination data, wind data, temperature data, and vibration data, which are the device control information Dm for manual control. Instead, the lighting device 5, the wind power device 6, the air conditioning device 7, and the vibration device 8 are converted into data for driving control, and the data is output to each device.

  On the other hand, when the automatic control mode is designated by the control information Dr, the effect control data generation unit 25 uses reference information, calculation designation information, color correction information, acoustic correction information, and the like, which are automatic control device control information Da. Then, the video feature amount of the frame to be displayed and the acoustic feature amount of the audio data corresponding to the frame to be displayed are calculated. The effect control data generation unit 25 drives and controls the lighting device 5, the wind power device 6, the air conditioner 7, and the vibration device 8 using the video feature amount and the audio feature amount calculated using the automatic control device control information Da. Data is generated, and the generated data is output to each device.

  First, as an example, data generation for driving and controlling the lighting device 5 in the automatic control mode will be described.

  The effect control data generation unit 25 stores the video data of a predetermined pixel stored in the region specified by the region information in the representative frame in the segment including the frame output from the data receiving device 2 to the video display device 3. A block average value is calculated and used as reference lighting data (when there is no reference lighting data specified by the content creator). Next, the reference illumination data is converted into data indicating lightness (Lightness), saturation (Chroma) (color), and hue (Hue) (color) (hereinafter, brightness, saturation, and hue information are referred to as “ Called color signal data). Next, an embodiment of a method for generating illumination data for each frame in the corresponding segment will be described.

  FIG. 11 is a diagram for explaining threshold setting of illumination data in the automatic setting mode. The effect control data generation unit 25 calculates the color signal data in the region of each frame specified by the region information and performs threshold processing, that is, the threshold set by the user or set by the content creator. The final color signal data of each frame is obtained by calculating the video feature quantity or the acoustic feature quantity within the allowable range indicated by the correction information. The threshold value obtained based on the color correction information is set and held in the threshold value setting unit 26 in the same manner as the threshold value specified by the user.

  The illumination data corresponding to each frame for controlling the illumination color is set based on the color signal data of the reference illumination data. Here, the upper limit of the change from the color signal data of the reference illumination data is defined using Thresh_max and the lower limit is defined using Thresh_min using a threshold value. For example, if Thresh_max = 10 and Thresh_min = 20 are set for the lightness axis, the upper limit of lightness: (lightness of reference illumination data) +10, and the lower limit of lightness: (lightness of reference illumination data) −20. It means that each illumination data corresponding to each frame can change.

  FIG. 12 is a flowchart illustrating an example of a process flow for determining illumination data from color signal data. The effect control data generation unit 25 acquires the metadata and video data output from the data separation unit 22, and acquires the color signal data from the metadata and video data by the above method (step S1).

  Next, the acquired color signal data is compared with Thresh_max (step S2). Since the color signal data is generated for each video frame, the color signal data in step S1 is acquired for each frame, and the comparison of the color signal data in step S2 is performed for each frame. And If the acquired color signal data is larger than Thresh_max, Thresh_max is set as final color signal data in the frame (step S5). If the acquired color signal data is not larger than Thresh_max, it is determined whether or not the acquired color signal data is smaller than Thresh_min (step S3). If the acquired color signal data is smaller than Thresh_min, Thresh_min is set as final color signal data in the frame (step S6). If the acquired color signal data is not smaller than Thresh_min, the acquired color signal data is directly used as the final color signal data in the frame (step S4).

  The final color signal data determined by this threshold processing is the illumination data for that frame. When the illumination data of a certain frame is determined, the effect control data generation unit 25 determines whether there is a next frame in the same segment (step S7). If there is a next frame, the process returns to step S1. The color signal data of the next frame is acquired, and the above process is repeated.

  According to this threshold processing, when determining illumination data, the illumination data is determined within the threshold range by referring to the threshold according to the increase or decrease of the color signal data representing the video feature amount of each frame. Thereby, the illumination data is set so as to be within an allowable range defined for each segment. Therefore, even when the color signal data of each frame repeats a large increase and decrease, it is possible to prevent the illumination data from fluctuating violently following this. Thereby, stable switching control of the lighting device 5 can be realized.

  In the above example, the reference illumination data is calculated from the video data and the audio data in the data receiving device 2, but when the reference data preset by the content creator can be received, the reference illumination data received from the outside Can be used. Even in this case, final color signal data can be acquired by the processing shown in FIG. Note that the reference illumination data specified by the content creator is described as color reference information in the automatic control device control information Da as shown in FIG.

  Note that the color signal data in segment units received as reference illumination data from the outside, the color signal data that is the feature amount of each frame calculated from the video data, and the threshold value are used to make the final color signal within an allowable range. The processing method can be rephrased as follows. That is, the color signal data acquired from the outside as the reference illumination data is compared with the color signal data of each frame calculated from the video data, and the color signal data acquired as the reference illumination data is calculated for each frame calculated from the video data. A correction process is performed so as to approach the color signal data. When the color signal data of the reference illumination data becomes larger than Thresh_max by this processing, the final color signal data is set to Thresh_max, and when the color signal data of the reference illumination data becomes smaller than Thresh_min, Let the color signal data be Thresh_min. On the other hand, when it is between Thresh_max and Thresh_min, the color signal data after the correction processing is set as final color signal data. Even in this case, even if the color signal data of each frame is repeatedly repeatedly increased and decreased, it is possible to prevent the illumination data from fluctuating greatly following this. In addition, it is possible to improve the sense of reality more effectively by changing the reference control data for each segment reflecting the intentions of the content creator in units of frames following the content of the video data to be displayed. Become.

  As described above, according to the automatic setting mode used in the present invention, the illumination by the illumination device 5 installed around the video display device 3 is kept substantially constant based on the reference data in the same segment section. can do. Here, “substantially constant” means that the variation in illumination light within the same segment is within a range that does not impair the presence of the viewer.

  It is well known that there is a color tolerance in human vision. FIGS. 13 and 14 are diagrams illustrating another example for explaining threshold setting of illumination data according to reference illumination data in the automatic setting mode. In FIG. 13, the change range is designated as the color difference ΔE with respect to the reference illumination data. FIG. 14 shows the level division of the color difference ΔE and the general degree of vision. As a substantially constant range in the present invention, it should be within a range that can be treated as a color difference that is indistinguishable with the system colors in FIG. 14, that is, within a level range where the color difference ΔE is less than 13. A range that can be handled, that is, a level range in which the color difference ΔE = 6.5 or less is preferable. As described above, the allowable change range may directly specify threshold values for lightness, saturation, and hue, or may use the color difference ΔE.

  As described above, when color correction information is used, an allowable range is already defined as a color range. Therefore, when the illumination data is to be acquired using the color correction information, the color correction information is used as it is to indicate the upper and lower limits of the reference illumination data without using the threshold value set and held in the threshold value setting unit 26. Can be used.

  Next, data generation for driving and controlling the wind power device 6 and the vibration device 8 in the automatic control mode will be described.

  The effect control data generation unit 25 calculates each sound pressure level (dB) at a predetermined wind force application frequency (Hz) and vibration application frequency (Hz) stored in the acoustic information, and each of them is used as reference wind data (dB). ) And reference vibration data (dB). As in the case of the above-described reference illumination data, the reference wind power data and the reference vibration data may be received and acquired by the segment creator in advance. In this case, the reference wind power data and the reference vibration data are described as acoustic reference information in the automatic control device control information Da. As an example of a wind power application frequency, it is 10 Hz when the frequency range is not specified, and 0 to 10 Hz when the frequency range is specified. As an example of the vibration application frequency, 50 Hz is specified when the frequency range is not specified, and 0 to 125 Hz when the frequency range is specified. The reference wind power data is converted into wind power data indicating the wind speed (m / s). As an example of a conversion method to wind power data, there is a case where “50 dB → 0.5 m / s” and “80 dB → 2 m / s” are converted with reference to a lookup table, but the conversion method is limited to this. It is not a thing. The converted value may be specified not only when the absolute value of the wind speed is specified, but also as “strong wind” and “weak wind”.

  For the reference wind power data and the reference vibration data in the automatic setting mode, a threshold is set and an allowable range is set in the same manner as the reference illumination data described above. The effect control data generation unit 25 calculates the sound pressure level of the wind force application frequency and the vibration application frequency of each frame, that is, the wind force data and the vibration data corresponding to each frame, and finally performs the final processing corresponding to each frame by threshold processing. Wind power data and vibration data are acquired, and control data for controlling the viewing environment device is generated. The threshold used for the threshold processing is set and held in the threshold setting unit 26.

  Wind power data corresponding to each frame used for controlling the wind force and vibration data corresponding to each frame used for controlling the vibration are set according to the reference wind data and the reference vibration data, respectively. Here, an upper limit and a lower limit of the change from each reference data, that is, an allowable range is defined. For example, the upper limit of the sound pressure level: (reference wind data or reference vibration data) +15, the lower limit of the sound pressure level: (reference wind data or reference vibration data) −10, and finally obtained wind data and vibration data. Refers to things that can change.

  Next, generation of control data for driving and controlling the air conditioner 7 in the automatic control mode will be described.

  The effect control data generation unit 25 calculates a block average value of video data of predetermined pixels stored in a region specified by the region information in a representative frame in a segment displayed by the video display device 3, and Is the reference illumination data. Next, the color temperature (correlated color temperature) is estimated from the reference illumination data. As described above, the color temperature estimation process is to estimate the illumination state of the scene where the video is shot based on the feature amount of the video data. In the present invention, the processing method is not limited. For example, non-patent literature “scene illumination estimation process”, Shoji Tominaga, Satoru Sakurai, B.H. A. Wandell, IEICE Technical Report, PRMU 99-184, 1999. Can be applied.

  In the sensor correlation method, the color gamut occupied by the sensor output for each color temperature in the sensor space is obtained in advance, and the color temperature is estimated by examining the correlation between the color gamut and the acquired image pixel distribution.

  For example, in this embodiment, it is possible to estimate the color temperature of illumination at the time of shooting the video from the video data of the representative frame by applying the sensor correlation method.

  As a procedure of the processing method, (1) a color gamut occupied by the sensor output is obtained in advance. (2) Normalize all pixels of the target pixel. (3) The normalized (R, B) coordinate values are plotted on the RB plane. (4) The color gamut having the highest correlation with the (R, B) coordinate value of the target image is estimated as the color temperature of the target image. The color gamut is obtained every 500K, for example.

  In estimating the color temperature, a color gamut that can be occupied by the sensor output for each color temperature in order to classify the scene illumination is defined in the color space. Here, RGB values of sensor outputs for various object surfaces are obtained under the spectral distribution of each color temperature. A two-dimensional illumination color gamut obtained by projecting these RGB convex hulls onto the RB plane is used. This illumination color gamut can be formed by the color gamut for every 500K occupied by the sensor output as described above.

In the sensor correlation method, scaling processing of image data is necessary to adjust the overall luminance difference between images. The luminance of the i-th pixel among the target pixels is I _i and the maximum value is I _max . Then, in order to adjust the brightness between different images, the sensor output is normalized with respect to RGB and maximum values as follows.

(RGB) = (R / I _max , G / I _max , B / I _max )
L _max = max (R _i ² + G _i ² + B _i ² ) ^1/2
Then, normalized (R, B) coordinate values are plotted against the RB plane on which the illumination color gamut is projected. This illumination color gamut is used as a reference color gamut and compared with the coordinate values of the plotted target image. Then, the reference color gamut having the highest correlation with the coordinate value of the target image is selected, and the color temperature is determined based on the selected reference color gamut.

  Note that, as the video feature quantity of each scene used for the color temperature estimation process, for example, the color signal and the luminance signal of the predetermined screen area included in the video data to be displayed can be used as they are as in the conventional example described above. Needless to say, it is good.

  The calculated color temperature is converted into temperature data indicating temperature (° C.). As an example of the conversion method to temperature data, referring to the temperature conversion table shown in FIG. 15, “1667 ≦ color temperature <2251 → Hot”, “2251 ≦ color temperature <4171 → Warm”, “4171 ≦ color temperature < There are cases where conversion is performed such as “8061 → Moderate” and “8061 ≦ color temperature ≦ 25000 → Cool”, but the conversion method is not limited to this.

  Note that the data generation method for driving and controlling the data receiving device 2 in the automatic control mode is not limited to this. For example, data for driving and controlling the data receiving device 2 is created based on keywords included in character information such as EPG (Electronic Program Guide), caption information, and superimpose of a television program. Also good.

  As described above, according to the present invention, it is possible to control the driving strength of the viewing environment device within an allowable range according to the intention of the content creator without transmitting metadata as in the manual control mode. It becomes possible.

[Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIGS. For convenience of explanation, members having the same functions as those used in the above-described embodiment are denoted by the same reference numerals, and description thereof is omitted.

  In the above-described embodiment, metadata is added to video data and audio data and transmitted as part of broadcast data.

  In the present embodiment, the broadcast data does not include metadata itself. By receiving metadata corresponding to video data or audio data to be viewed from an external server device or the like, the data receiving device can realize an optimal viewing environment when reproducing the video data or audio data. In this case, the data receiving apparatus receives identification data for identifying metadata in advance.

  FIG. 16 is a block diagram showing a configuration of the external server apparatus according to the present embodiment. The external server device 9 bears a part of the function of the data transmission device 1. As shown in FIG. 16, the external server device 9 includes a request reception unit 91, a data storage unit 92, and a metadata transmission unit 93.

  The request receiving unit 91 receives a transmission request for metadata relating to specific video data or audio data (content) from the data receiving device (viewing environment control device) 2 ′ described later.

  The data storage unit 92 stores metadata for each content.

  The metadata transmission unit 93 transmits the metadata for which the transmission request has been received to the requesting data receiving device.

  The metadata stored in the data storage unit 92 describes a start time code of a segment (for example, a scene or a shot) of an arbitrary control unit intended by the content creator or the like. The metadata is transmitted from the metadata transmitting unit 93 to the requesting data receiving apparatus 2 ′ together with the TC (time code) indicating the start time of the segment.

  In addition, an ID is added to a segment (for example, a scene or a shot) of an arbitrary control unit intended by the content creator, and the metadata of the content that has received the transmission request is sent from the metadata transmission unit 93 together with the segment ID. You may make it transmit to the data receiver 2 'of a request origin. In this case, the metadata can be transmitted to the receiving device 2 'in units of segments.

  As described above, the external server device 9 can transmit metadata in units of segments when receiving a request from the outside.

  Next, a system configuration of the viewing environment control system 100 ′ including the data receiving device 2 ′ that receives the metadata sent from the external server device 9 and controls the viewing environment device will be described.

  As shown in FIG. 17, the viewing environment control system 100 ′ includes a data receiving device 2 ′, a video display device 3, an audio reproduction device 4, an illumination device 5, a wind power device 6, an air conditioner 7, and a vibration device 8. Has been.

  The data reception device 2 ′ has substantially the same configuration as the data reception device 2 described above, but unlike the data reception device 2, a CPU (Central Processing Unit) 27, a request transmission unit 28, and a metadata reception unit 29 Is further provided.

  Based on an instruction from the effect control data generation unit 25, the request transmission unit 28 transmits a transmission request for metadata corresponding to video data (content) to be displayed to the external server device 9 via the communication network. An instruction from the effect control data generation unit 25 to the request transmission unit 28 is performed via the CPU 27.

  The metadata receiving unit 29 receives metadata transmitted from the external server device 9 via the communication network. The received metadata is transmitted to the effect control data generation unit 25 via the CPU 27.

  The effect control data generation unit 25 in the data reception device 2 ′ is the same as the effect control data generation unit 25 in the data reception device 2 except that a metadata transmission request is made based on the received identification data.

  As described above, the viewing environment control system 100 ′ obtains metadata corresponding to the content from the external server device 9 even when the broadcast data does not include metadata, and based on this metadata, This configuration controls the viewing environment device. Therefore, similarly to the above-described embodiment, the viewing environment device switching control is performed at an arbitrary timing according to the intention of the video producer while suppressing an increase in the data amount. This makes it possible to realize device control in an optimal viewing environment.

  In the above-described embodiment, the lighting device 5, the wind power device 6, the air conditioner 7, and the vibration device 8 have been described as examples of the peripheral devices arranged in the virtual viewing environment space, but are limited to these viewing environment devices. It is not a thing. In addition, it goes without saying that the present invention can be applied to peripheral devices that affect the viewing environment, such as a scent generator. When the present invention is applied to these peripheral devices that affect the viewing environment, presentation effect information such as how strong the fragrance is generated may be defined by metadata.

  In the present invention, content that is video data or audio data is not limited to content related to a television program transmitted by television broadcasting, but relates to a work stored on a media medium such as a Blu-ray Disc or a DVD. It may be content. That is, the input video data is not limited to that obtained by receiving a television broadcast, and the present invention can also be applied when video data reproduced by an external reproduction device is input.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. For example, the data receiving devices 2 and 2 ′ may be provided in the video display device 3, and the external viewing environment device can be controlled based on various information included in the input video data. Needless to say. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  As described above, the viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data including one or more frames, and the video Receiving means for receiving reference information indicating a calculation area when calculating the feature quantity of data or audio data; and calculating means for calculating a feature quantity of video data or audio data in the calculation area indicated by the reference information. It is a configuration.

  In addition, it is preferable that the reference information indicates a screen area when the feature amount of the video data is calculated.

  Alternatively, it is preferable that the reference information indicates a frequency region when the feature amount of the audio data is calculated.

  Here, the reference information may be set in units of frames constituting video data or audio data, or in units of segments that are sets of the frames.

  Further, as described above, the viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames. Receiving means for receiving calculation designation information indicating a calculation method for calculating the feature quantity of video data or audio data, and calculation means for calculating the feature quantity of video data or audio data by the calculation method indicated by the calculation designation information It is the structure provided with.

  The calculation designation information preferably specifies calculation of an average color within a predetermined screen area as a feature amount of the video data.

  Alternatively, it is preferable that the calculation designation information designates calculation of a high luminance area color within a predetermined screen area as a feature amount of the video data.

  Alternatively, it is preferable that the calculation designation information designates calculation of a color corresponding to a correlated color temperature in a predetermined screen area as a feature amount of the video data.

  Alternatively, the calculation designation information preferably designates calculation of a color corresponding to a memory color in a predetermined screen area as a feature amount of the video data.

  Here, the calculation designation information may be set in units of frames constituting video data or audio data, or in units of segments that are sets of the frames.

  Further, as described above, the viewing environment control device according to the present invention is a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames. Receiving means for receiving correction information indicating an allowable range for calculating a feature amount of video data or audio data; and calculating means for calculating a feature amount of video data or audio data within the allowable range indicated by the correction information; It is the structure provided with.

  It is preferable that the correction information indicates an allowable range related to at least one of brightness and color, which is calculated as a feature amount of the video data.

  Alternatively, it is preferable that the correction information indicates an allowable range related to at least one of a frequency and a sound pressure level, which is calculated as a feature amount of the audio data.

  Here, the correction information may be set in units of frames constituting video data or audio data, or in units of segments that are a set of the frames.

  Also, as described above, the viewing environment control device according to the present invention is a viewing environment control device that controls viewing environment equipment based on control reference data received from outside, and that is used for video data or audio data received from outside. It comprises a calculating means for calculating a feature quantity and a correcting means for correcting the control reference data using the feature quantity of the video data or audio data.

  Further, it is preferable that the control reference data is set for each segment which is a set of frames constituting the video data or audio data.

  Here, a segment is defined as video data or audio data composed of one or more frames divided into arbitrary lengths in the time direction.

  According to the above configuration, the viewing environment device can be controlled for each segment unit. By dividing the segments in correspondence with scenes, shots, frames, etc., the viewing environment device can be controlled in accordance with the scene setting. As a result, it is possible to provide the user with a viewing environment that provides a high sense of realism.

  The control reference data is preferably color reference information related to at least one of brightness and color.

  Alternatively, it is preferable that the control reference data is acoustic reference information related to at least one of a frequency and a sound pressure level.

  Further, as described above, the data transmission device according to the present invention transmits data to the viewing environment control device that controls the viewing environment device based on the feature amount of the video data or audio data including one or more frames. The data transmission device performs transmission, and includes transmission means for transmitting reference information indicating a calculation area when calculating the feature amount of the video data or audio data.

  In the data transmitting apparatus, it is preferable that the reference information indicates a screen area when the feature amount of the video data is calculated.

  Alternatively, it is preferable that the reference information indicates a frequency region when the feature amount of the audio data is calculated.

  In addition, as described above, the data transmission device according to the present invention transmits data to the viewing environment control device that controls the viewing environment device based on the feature amount of video data or audio data including one or more frames. A data transmission apparatus that performs transmission, and includes a transmission unit that transmits calculation designation information indicating a calculation method when calculating the feature amount of the video data or audio data.

  In the data transmission apparatus, it is preferable that the calculation designation information designates calculation of an average color in a predetermined screen area as a feature amount of the video data.

  Alternatively, it is preferable that the calculation designation information designates calculation of a high luminance area color within a predetermined screen area as a feature amount of the video data.

  Alternatively, it is preferable that the calculation designation information designates calculation of a color corresponding to a correlated color temperature in a predetermined screen area as a feature amount of the video data.

  Alternatively, the calculation designation information preferably designates calculation of a color corresponding to a memory color in a predetermined screen area as a feature amount of the video data.

  Further, as described above, the data transmission device according to the present invention transmits data to the viewing environment control device that controls the viewing environment device based on the feature amount of the video data or audio data including one or more frames. The data transmission device performs transmission, and includes a transmission unit that transmits correction information indicating an allowable range when calculating the feature amount of the video data or audio data.

  In the data transmitting apparatus, it is preferable that the correction information indicates an allowable range related to at least one of brightness and color, which is calculated as a feature amount of the video data.

  Alternatively, it is preferable that the correction information indicates an allowable range related to at least one of a frequency and a sound pressure level, which is calculated as a feature amount of the audio data.

  In addition, as described above, the data transmission device according to the present invention is a data transmission device that transmits control reference data for controlling a viewing environment device to the viewing environment control device, and the control reference data includes video This is a configuration set for each segment, which is a set of frames constituting data or audio data.

  In the data transmission device, the control reference data is preferably color reference information regarding at least one of brightness and color.

  Alternatively, it is preferable that the control reference data is acoustic reference information related to at least one of a frequency and a sound pressure level.

  The viewing environment control system including the viewing environment control device according to the present invention is also included in the category of the present invention.

  That is, the viewing environment control system according to the present invention includes a viewing environment control device according to the present invention, a display device that displays the video data, and a viewing environment device that is installed in the same viewing environment as the video display device. It is the structure equipped with.

  The viewing environment control device may be realized by a computer. In this case, a viewing environment control program for realizing the viewing environment control device in the computer by operating the computer as each of the above means, and a computer-readable recording medium recording the viewing environment control program are also included in the scope of the present invention. enter.

  Finally, each unit included in the data transmission device 1 and the data reception devices 2 and 2 ′ may be configured by hardware logic. Alternatively, it may be realized by software using a CPU (Central Processing Unit) as follows.

  That is, the data transmission device 1 and the data reception devices 2 and 2 ′ each include a CPU that executes instructions of a control program (viewing environment control program) that realizes each function, a ROM (Read Only Memory) that stores the control program, and the above-described control. A RAM (Random Access Memory) for expanding the program into an executable format, and a storage device (recording medium) such as a memory for storing the control program and various data are provided.

  With this configuration, the object of the present invention can also be achieved by a predetermined recording medium. In this recording medium, the program codes (execution format program, intermediate code program, source program) of the control program of the data transmission device 1, the data reception device 2, and 2 ′, which are software for realizing the functions described above, can be read by a computer. It only has to be recorded. This recording medium is supplied to the data transmission device 1 and the data reception devices 2 and 2 '. Thereby, the data transmission device 1 and the data reception devices 2 and 2 ′ (or CPU or MPU) as a computer may read and execute the program code recorded on the supplied recording medium.

  The recording medium that supplies the program code to the data transmission device 1 and the data reception devices 2 and 2 ′ is not limited to a specific structure or type. That is, the recording medium includes, for example, a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, or an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. System, a card system such as an IC card (including a memory card) / optical card, or a semiconductor memory system such as a mask ROM / EPROM / EEPROM / flash ROM.

  Further, even if the data transmission device 1 and the data reception devices 2 and 2 'are configured to be connectable to a communication network, the object of the present invention can be achieved. In this case, the program code is supplied to the data transmission device 1 and the data reception devices 2 and 2 'via the communication network. The communication network is not limited to a specific type or form as long as it can supply program codes to the data transmission device 1 and the data reception devices 2 and 2 ′. For example, the Internet, an intranet, an extranet, a LAN, an ISDN, a VAN, a CATV communication network, a virtual private network, a telephone line network, a mobile communication network, a satellite communication network, or the like may be used.

  The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, wired communication such as IEEE 1394, USB (Universal Serial Bus), power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared light like IrDA or remote control, Bluetooth (registered trademark), 802. 11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, etc. can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

(Additional notes)
The embodiment described above can also be expressed as follows.

(First configuration)
In a data transmitting apparatus for transmitting video data composed of one or more frames, the video data is divided into segments composed of an arbitrary number of one or more frames, and the start time and / or section of each segment is indicated. Segment delimiter information, control information indicating the control method of the viewing environment device when displaying each frame of the video data, and control indicating the reference of the control method of the viewing environment device when displaying each frame of the video data A data transmitting apparatus, wherein reference information is added to the video data in units of segments and transmitted.

(Second configuration)
The control information includes information indicating an automatic control mode for controlling a viewing environment device based on control reference information when displaying all frames in the segment. Data transmission device.

(Third configuration)
The data transmission device according to the first or second configuration, wherein the control reference information includes a feature amount of video data to be displayed when all the frames in the segment are displayed.

(Fourth configuration)
The data transmission apparatus according to the third configuration, wherein the feature amount of the video data includes information on a screen area.

(Fifth configuration)
The data transmission device according to the third configuration, wherein the feature amount of the video data includes information on brightness or color of the video data.

(Sixth configuration)
The data transmission device according to the first or second configuration, wherein the control reference information includes a feature amount of acoustic data to be reproduced when displaying all the frames in the segment.

(Seventh configuration)
Video data to be displayed on a display device, segment delimiter information indicating the start time and / or section of a segment obtained by dividing the video data by an arbitrary number of one or more frames, and the video data in association with each segment Control information indicating a control method of the viewing environment device when displaying each frame constituting the video data, and a control indicating a reference of the control method of the viewing environment device when displaying each frame of the video data A viewing environment control apparatus comprising: receiving means for receiving reference information; and control means for controlling driving of viewing environment equipment installed around the display device based on the control information.

(Eighth configuration)
The viewing environment control apparatus according to the seventh configuration, wherein the control unit controls driving of the viewing environment device using a feature amount of video data.

(Ninth configuration)
The viewing environment control apparatus according to the eighth configuration, wherein the feature amount of the video data includes information on a screen area.

(Tenth configuration)
The viewing environment control apparatus according to the eighth configuration, wherein the feature amount of the video data includes information on brightness or color of the video data.

(Eleventh configuration)
The viewing environment control apparatus according to the seventh configuration, wherein the control unit controls driving of the viewing environment device using a feature amount of acoustic data.

(Twelfth configuration)
The viewing environment control device according to any one of the seventh to eleventh configurations, a video / audio reproduction device for reproducing the video data and / or audio data, and a periphery of the video / audio reproduction device. A viewing environment control system comprising a peripheral device.

(13th configuration)
A viewing environment control method for controlling driving of a viewing environment device according to a feature amount of video data or audio data to be displayed, by performing a correction process on the feature amount of the video data or audio data, A viewing environment control method, comprising: generating control information of the viewing environment device.

(14th configuration)
In a data transmission method for transmitting video data composed of one or more frames, the video data is divided into segments composed of an arbitrary number of one or more frames, and the start time and / or section of each segment is indicated. Segment delimiter information, control information indicating the control method of the viewing environment device when displaying each frame of the video data, and control indicating the reference of the control method of the viewing environment device when displaying each frame of the video data A data transmission method comprising: adding reference information to the video data in units of segments and transmitting the reference information.

(15th configuration)
A data transmission method for transmitting control information indicating a control method of a viewing environment device when displaying each frame constituting video data in response to an external request, wherein the video data is an arbitrary number of one or more A data transmission method characterized by transmitting segment delimiter information indicating a start time and / or a section of a segment divided by a frame, control information, and control reference information in units of segments.

(Sixteenth configuration)
When displaying video data to be displayed on a display device, segment delimiter information indicating the start time and / or section of a segment obtained by dividing the video data into one or more arbitrary frames, and each frame of the video data Control information indicating a control method of the viewing environment device and control reference information indicating a reference of the control method of the viewing environment device when displaying each frame of the video data, and based on the control information, A viewing environment control method, comprising: controlling driving of a viewing environment machine installed around a display device.

  Further, the above-described embodiment can also be expressed as follows.

(17th configuration)
Receiving means for receiving video data composed of a plurality of segments, and control means (delay generating section, effect control data generating section) for controlling viewing environment equipment, and outputting each frame of the video data to a display device Control means for determining the control content for the viewing environment device at the time of determination based on the frame feature amount that is the feature amount of the frame and the segment feature amount that is the feature amount of the segment to which the frame belongs. The receiving means includes descriptive data (metadata) including each segment feature quantity of the plurality of segments and reference information that the control means refers to extract a segment feature quantity from each of the plurality of segments. The viewing environment control apparatus is configured to further receive (data).

  The frame feature amount includes, but is not limited to, frame color information, luminance information, acoustic information corresponding to the frame, and the like, and can vary from frame to frame.

  The segment feature amount includes, but is not limited to, color information, luminance information, and acoustic information, and is predetermined for each segment.

  According to the above configuration, the viewing environment control device receives video data composed of a plurality of segments, and each segment feature amount of the plurality of segments and the control means extract the segment feature amount from each of the plurality of segments. At least one of reference information to be referred to is received. The segment feature is associated with each segment and is information used to control the viewing environment device. The reference information is information that is referenced when the viewing environment control device extracts the segment feature amount in the corresponding segment. The viewing environment control device controls the viewing environment device based on the segment feature value extracted from the frame with reference to the received segment feature value or reference information and the frame feature value corresponding to the frame output to the display device .

  Therefore, the viewing environment device can be controlled in accordance with the contents of the segment, compared to the case where the viewing environment device is controlled using only the feature amount extracted from the frame output to the display device. Therefore, the viewing environment device can be appropriately controlled for each segment of the video, and the viewing environment device can be controlled in accordance with the atmosphere or scene setting of the shooting scene intended by the video producer. As a result, it is possible to provide the user with a viewing environment that provides a high sense of realism.

(18th configuration)
The control means performs a correction process on the frame feature value of each frame belonging to the corresponding segment using each segment feature value, and a corrected feature value (post-correction) that is a frame feature value after the correction process. The viewing environment control apparatus determines the control content based on the data.

According to the above configuration, when the viewing environment device is controlled based on the frame feature corresponding to the frame output to the display device, a predetermined restriction is applied to the frame feature by the correction processing using the segment feature. Can do. Accordingly, even when the frame feature amount in the output frame is repeatedly increased and decreased, it is possible to prevent the driving intensity of the viewing environment device from fluctuating greatly following this. This makes it possible to control the viewing environment without disturbing the atmosphere when displaying images in the same segment.
(19th configuration)
The segment feature amount indicates a permissible range of the frame feature amount of each frame belonging to the corresponding segment, and the control unit corrects the frame feature amount of each frame belonging to the segment so as to be within the permissible range. It is preferable to perform processing.

According to the above configuration, the viewing environment control apparatus performs a correction process on the frame feature value in each frame, and generates a corrected feature value that falls within the allowable range indicated by the segment feature value. Then, the viewing environment device is controlled using the corrected feature amount. Since all the corrected feature values used for controlling the viewing environment devices are within the allowable range set for each corresponding segment, the viewing environment becomes large when images in the same segment are displayed. It can be prevented from changing.
(20th configuration)
The segment feature amount is a representative value of the frame feature amount of each frame belonging to the corresponding segment (image feature amount in the representative frame), and the control means uses the representative value and predetermined threshold data. An allowable range of the frame feature value of each frame belonging to the corresponding segment is set, and the control means performs the correction process so that the frame feature value of each frame belonging to the segment falls within the allowable range. Viewing environment control device.

  According to the above configuration, the viewing environment control device has a range allowed for the variation of each frame feature amount based on the segment feature amount that is a representative value of the frame feature amount and the predetermined threshold data. Set the allowable range. The viewing environment control apparatus performs a correction process on the frame feature amount in each frame, and generates a corrected feature amount that falls within the set allowable range. Then, the viewing environment device is controlled using the corrected feature amount. Since all of the post-correction feature values (final feature values) used for controlling the viewing environment device are within the allowable range set for each corresponding segment, the images in the same segment are displayed. It is possible to prevent the viewing environment from changing drastically when Therefore, even if the segment feature amount does not have a spread that defines the allowable range, the allowable range can be set, and the drive strength of the viewing environment device can be prevented from fluctuating. This makes it possible to control the viewing environment without disturbing the atmosphere when displaying images in the same segment.

  Moreover, since the allowable range can be designated as a desired range by changing the threshold data, the allowable range can be changed on the user side.

(21st configuration)
The descriptive data further includes designation data (control information) for designating a segment to be corrected by the control means, and the control means performs correction processing in the segment designated by the designation data. Viewing environment control device.

  According to the above configuration, the viewing environment control device detects the designated data and executes the correction process for the frame feature amount only in the segment designated by the designated data. For this reason, the viewing environment device is controlled without performing the correction process for a segment that does not require the correction process for each frame or that the content provider does not desire the correction process for each frame.

(Twenty-second configuration)
The viewing environment control device, wherein the reference information is region information that defines a region in the frame from which the control means extracts the segment feature value.

  According to the above configuration, an area in the frame can be defined, and a segment feature amount can be extracted from video information recorded in the area. Therefore, the amount of data to be handled is smaller than when video information of the entire frame is used, and the control process can be executed more quickly.

(23rd configuration)
The viewing environment control device, wherein the frame feature amount is color information related to at least one of brightness and color of the frame.

  According to the above configuration, the viewing environment control device can control the viewing environment device by using at least one of lightness and color information in the frame. Accordingly, it is possible to appropriately control the brightness, hue, and temperature of the viewing environment, and it is possible to provide the user with a viewing environment that is more suitable for viewing video.

(24th configuration)
The viewing environment control apparatus, wherein the receiving means further receives audio data, and the frame feature amount is acoustic information of the audio data corresponding to an output frame.

  According to the above configuration, the viewing environment control device can control the viewing environment device using the acoustic information. Accordingly, the viewing environment control apparatus can appropriately control the airflow and vibration of the viewing environment, and can provide the user with a viewing environment that is more suitable for viewing images.

(25th configuration)
A data transmission device that transmits video data composed of a plurality of segments to a viewing environment control device that controls a viewing environment device, wherein each of the plurality of segments is a feature amount of each frame belonging to the segment. Descriptive data including at least one of a segment feature amount representing a frame feature amount and reference information used to extract a segment feature amount from each of the plurality of segments, or for identifying the description data A data transmission device configured to transmit identification data to the viewing environment control device.

  According to the above configuration, the data transmission device transmits the video data composed of a plurality of segments to the viewing environment control device, and the description data including at least one of the segment feature amount and the reference information, or the description data Identification data for identification is transmitted. The segment information represents the frame feature amount of each frame belonging to the segment for each of the plurality of segments. The reference information is information used to extract a segment feature amount from each of a plurality of segments.

  Therefore, when data is transmitted to the viewing environment control device by the data transmission device, the viewing environment control device can control the viewing environment device using the segment information defined for each segment. Thereby, it is possible to control the viewing environment device according to the contents of the segment, compared to the case where the viewing environment device is controlled using only the feature amount extracted from the frame output to the display device. Therefore, the viewing environment device can be appropriately controlled for each segment of the video, and the viewing environment device can be controlled in accordance with the atmosphere or scene setting of the shooting scene intended by the video producer. As a result, it is possible to provide the user with a viewing environment that provides a high sense of realism.

(26th configuration)
Data including an adding means (data multiplexing unit) for adding the description data or the identification data to the video data in units of segments, and transmitting data generated by the adding means to the viewing environment control device Transmitter device.

According to the above configuration, the data transmission device can add the description data to the video data and transmit the data generated by the addition to the viewing environment control device.
In the control of the viewing environment device, there are many controls mainly in scene units in which the situation of the scene is constant, but there are not a few cases in which control is performed in shot units in which the shooting situation is constant. In some cases, fine control on a frame basis is desirable. By dividing the segment into units that often require the viewing environment device to be controlled consistently depending on the content of the video data, it is possible to efficiently describe description data or identification data while suppressing an increase in the amount of data to be added. .

(27th configuration)
The data transmission device, wherein the segment feature value indicates an allowable range of the frame feature value of each frame belonging to the corresponding segment.

According to the above configuration, when data is transmitted to the viewing environment control device by the data transmitting device, the viewing environment control device extracts data that falls within the allowable range indicated by the segment feature value from the frame feature value of each frame. Can be generated. The viewing environment control device can control the viewing environment device using the generated data. All the data used for controlling the viewing environment device is within the allowable range set for each corresponding segment, so when the video in the same segment is displayed, the viewing environment changes greatly. Can be prevented.
(28th configuration)
The data transmission device, wherein the segment feature value is a representative value of the frame feature value of each frame belonging to the corresponding segment.

  According to the above configuration, when data is transmitted from the data transmission device to the viewing environment control device, the viewing environment control device uses each frame feature value and a predetermined threshold value data for each frame. It is possible to set an allowable range of the frame feature amount. The viewing environment control device can generate data that falls within the set allowable range from the frame feature amount of each frame. The viewing environment control device can control the viewing environment device using the generated data. All the data used for controlling the viewing environment device is within the allowable range set for each corresponding segment, so when the video in the same segment is displayed, the viewing environment changes greatly. Can be prevented. Therefore, even if the segment feature amount does not have a spread that defines the allowable range, the viewing environment control apparatus can set the allowable range, and can prevent the driving intensity of the viewing environment device from fluctuating greatly. . This makes it possible to control the viewing environment without disturbing the atmosphere when displaying images in the same segment.

  Moreover, since the allowable range can be designated as a desired range by changing the threshold data, the allowable range can be changed on the user side.

(29th configuration)
The data transmission device, wherein the description data further includes designation data for designating a segment to be processed for the frame feature value using the segment feature value in the viewing environment control device.

  According to the above configuration, when data is transmitted to the viewing environment control apparatus by the data transmission apparatus, the viewing environment control apparatus can execute the correction process for the frame feature amount only within the segment designated by the designated data. it can. Therefore, it is possible to control the viewing environment device without performing the correction process for a segment that does not require the correction process for each frame or that the content provider does not want the correction process for each frame.

(30th configuration)
The data transmission device, wherein the reference information is region information that defines a region in the frame from which the viewing environment control device extracts the segment feature.

  According to the above configuration, when data is transmitted to the viewing environment control device by the data transmission device, the viewing environment control device defines an area in the frame and from the video information recorded in the area. It becomes possible to extract the segment feature amount. Therefore, the amount of data to be handled is smaller than when video information of the entire frame is used, and the control process can be executed more quickly.

(31st configuration)
The data transmission device, wherein the frame feature amount is color information related to at least one of brightness and color of the frame.

  According to the above configuration, when data is transmitted to the viewing environment control device by the data transmission device, the viewing environment control device can control the viewing environment device using at least one of the lightness and color information of the frame. It becomes possible. Accordingly, it is possible to appropriately control the brightness, hue, and temperature of the viewing environment, and it is possible to provide the user with a viewing environment that is more suitable for viewing video.

(Thirty-second configuration)
The data transmission device, wherein the frame feature amount is sound information related to sound.

  According to the above configuration, when data is transmitted to the viewing environment control device by the data transmission device, the viewing environment control device can control the viewing environment device using the acoustic information. Accordingly, the viewing environment control apparatus can appropriately control the airflow and vibration of the viewing environment, and can provide the user with a viewing environment that is more suitable for viewing images.

(33rd configuration)
A viewing environment control system comprising: a viewing environment control device according to the present invention; a display device for displaying the video data; and a viewing environment device installed in the same viewing environment as the video display device. .

(34th configuration)
The viewing environment control device according to the present invention, a display device that displays the video data, an audio playback device that plays back the audio data, and the video display device and the audio playback device are installed in the same viewing environment. A viewing environment control system comprising a viewing environment device.

(35th configuration)
A viewing environment control device receives a video data composed of a plurality of segments, and a viewing environment control device controls a viewing environment device, and each frame of the video data is displayed on a display device. Including a control step of determining control contents for the viewing environment device at the time of output based on a frame feature amount that is a feature amount of the frame and a segment feature amount that is a feature amount of a segment to which the frame belongs. In the reception step, description data including at least one of the segment feature amounts of the plurality of segments and reference information that the viewing environment control device refers to extract the segment feature amounts from each of the plurality of segments. The viewing environment control method, which is a configuration for further receiving a message.

  According to the above configuration, the same effects as those of the viewing environment control apparatus according to the present invention can be obtained.

(Thirty-sixth configuration)
A data transmission method for transmitting video data composed of a plurality of segments to a viewing environment control device that controls a viewing environment device, wherein the viewing environment control device belongs to each of the plurality of segments Description data including at least one of a segment feature amount that represents a frame feature amount that is a feature amount of each frame and reference information used to extract a segment feature amount from each of the plurality of segments, or A data transmission method, which is configured to transmit identification data for identifying description data to the viewing environment control device.

  According to the said structure, there exists an effect similar to the data transmitter which concerns on this invention.

(37th configuration)
A data transmission method for transmitting video data composed of a plurality of segments to a viewing environment control apparatus that controls a viewing environment device, wherein each of the plurality of segments is a feature amount of each frame belonging to the segment. Descriptive data including at least one of a segment feature amount representing a frame feature amount and reference information used to extract a segment feature amount from each of the plurality of segments, or for identifying the description data A data transmission method comprising: an addition step of adding identification data to the video data for each segment; and a transmission step of transmitting data generated by the addition step.

(38th configuration)
A data transmission method for transmitting to a viewing environment control device that controls a viewing environment device when displaying video data composed of a plurality of segments, wherein for each of the plurality of segments, for each frame belonging to the segment An external request is made for descriptive data including at least one of a segment feature amount that represents a frame feature amount that is a feature amount and reference information used to extract the segment feature amount from each of the plurality of segments. A data transmission method that is configured to receive and transmit in units of segments.

  The present invention can be suitably used in a viewing environment control system that controls the viewing environment of a user.

1 Data transmission device 2, 2 'Data reception device (viewing environment control device)
3 Video display device (display device)
4 Audio playback device 5, 5a, 5b, 5c Lighting device (viewing environment equipment)
6, 6a, 6b Wind power device (viewing environment equipment)
7 Air conditioner (viewing environment equipment)
8 Vibration device (viewing environment equipment)
9 External server device 11 Data multiplexing unit 12 Transmission unit (transmission means)
13 Data Encoding Unit 21 Receiving Unit (Receiving Unit)
22 data separator 23 delay generator 24 delay generator 25 effect control data generator (calculation means, correction means)
26 threshold setting unit 27 CPU
28 Request Transmission Unit 29 Metadata Reception Unit 91 Request Reception Unit 92 Data Storage Unit 93 Metadata Transmission Unit 100, 100 ′ Viewing Environment Control System

Claims (11)

A viewing environment control apparatus that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames, and a calculation method for calculating the feature amount of the video data or audio data. A viewing environment control device comprising: receiving means for receiving calculation designation information to be displayed; and calculation means for calculating a feature amount of video data or audio data by a calculation method indicated by the calculation designation information. The viewing environment control apparatus according to claim 1, wherein the calculation designation information designates calculation of an average color in a predetermined screen area as a feature amount of the video data. 2. The viewing environment control apparatus according to claim 1, wherein the calculation designation information designates calculation of a high luminance area color within a predetermined screen area as a feature amount of the video data. The viewing environment control apparatus according to claim 1, wherein the calculation designation information designates calculation of a color corresponding to a correlated color temperature in a predetermined screen area as a feature amount of the video data. 5. The viewing environment control apparatus according to claim 4, wherein the calculation designation information designates calculation of a color corresponding to a memory color in a predetermined screen area as a feature amount of the video data. A data transmission device that transmits data to a viewing environment control device that controls a viewing environment device based on a feature amount of video data or audio data composed of one or more frames, the video data or audio data A data transmission apparatus comprising: transmission means for transmitting calculation designation information indicating a calculation method for calculating the feature amount of the data. The data transmission apparatus according to claim 6, wherein the calculation designation information designates calculation of an average color in a predetermined screen area as a feature amount of the video data. 7. The data transmission apparatus according to claim 6, wherein the calculation designation information designates calculation of a high luminance area color within a predetermined screen area as a feature amount of the video data. The data transmission apparatus according to claim 6, wherein the calculation designation information designates calculation of a color corresponding to a correlated color temperature in a predetermined screen area as a feature amount of the video data. 7. The data transmission apparatus according to claim 6, wherein the calculation designation information designates calculation of a color corresponding to a memory color in a predetermined screen area as a feature amount of the video data. A viewing environment control method for controlling a viewing environment device based on a feature amount of video data or audio data composed of one or more frames, the calculation method for calculating the feature amount of the video data or audio data. A viewing environment control method comprising: a reception step of receiving calculation designation information to be displayed; and a calculation step of calculating a feature amount of video data or audio data by a calculation method indicated by the calculation designation information.

Images