WO2009101998A1 - 放送システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体 - Google Patents

放送システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体 Download PDF

Info

Publication number
WO2009101998A1
WO2009101998A1 PCT/JP2009/052364 JP2009052364W WO2009101998A1 WO 2009101998 A1 WO2009101998 A1 WO 2009101998A1 JP 2009052364 W JP2009052364 W JP 2009052364W WO 2009101998 A1 WO2009101998 A1 WO 2009101998A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
information
data
presentation
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2009/052364
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
Tetsujiro Kondo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to US12/866,684 priority Critical patent/US20100325678A1/en
Priority to CN2009801072485A priority patent/CN101960849A/zh
Priority to EP09710743A priority patent/EP2242270A1/en
Publication of WO2009101998A1 publication Critical patent/WO2009101998A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/16—Analogue secrecy systems; Analogue subscription systems
    • H04N7/162—Authorising the user terminal, e.g. by paying; Registering the use of a subscription channel, e.g. billing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/09—Arrangements for device control with a direct linkage to broadcast information or to broadcast space-time; Arrangements for control of broadcast-related services
    • H04H60/13—Arrangements for device control affected by the broadcast information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/29—Arrangements for monitoring broadcast services or broadcast-related services
    • H04H60/33—Arrangements for monitoring the users' behaviour or opinions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/35—Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users
    • H04H60/48—Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for recognising items expressed in broadcast information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/61—Arrangements for services using the result of monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
    • H04H60/65—Arrangements for services using the result of monitoring, identification or recognition covered by groups H04H60/29-H04H60/54 for using the result on users' side
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/68—Systems specially adapted for using specific information, e.g. geographical or meteorological information
    • H04H60/73—Systems specially adapted for using specific information, e.g. geographical or meteorological information using meta-information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4131—Peripherals receiving signals from specially adapted client devices home appliance, e.g. lighting, air conditioning system, metering devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42202—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • H04N21/8133—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme

Definitions

  • the present invention relates to a broadcast system, a transmission device, a transmission method, a reception device, a reception method, a presentation device, a presentation method, a program, and a recording medium.
  • content such as an image can be flexibly presented.
  • the present invention relates to a broadcasting system, a transmission device, a transmission method, a reception device, a reception method, a presentation device, a presentation method, a program, and a recording medium.
  • images and sound as material data are edited by a content creator who produces content as a program at a broadcasting station on the transmission side, and the edited contents are edited.
  • Content consisting of images and sound is broadcast as a program.
  • an image of content as a program from a broadcast station is displayed and a sound is output by a TV (television receiver) or the like.
  • the TV on the receiving side presents (displays, outputs, etc.) the edited content edited at the broadcasting station.
  • the user views (views) the content edited by the content creator with a certain intention in the broadcasting station.
  • the user will appreciate the object.
  • a virtual space such as Second Life
  • content objects constituting the virtual space
  • the user processes content (objects constituting the virtual space), that is, for example, a user's favorite house. It can be built.
  • each user can process the content, but the processed content is shared by all users.
  • the user can only view the contents (objects constituting the image and sound) that can be viewed visually and auditorily. It has been difficult to appreciate (sniff) content that can be appreciated with a sense other than hearing, for example, the smell of cooking.
  • the user appreciates content with two senses of sight and hearing, but is requested to present content that can be appreciated with three or more senses. Yes.
  • the present invention has been made in view of such a situation.
  • a user can view content specific to each user, or a user can view content with a sense other than visual and auditory senses.
  • the content can be flexibly presented so that the content can be viewed with three or more senses.
  • a broadcasting system is a broadcasting system including a transmitting device that transmits data and a receiving device that receives the data, and the transmitting device is an object for each object used for content.
  • the broadcast system includes a transmission unit that transmits information as broadcast data, and the reception device includes a generation unit that generates content based on the object information.
  • object information for each object used for content is transmitted as broadcast data in the transmission device, and content is generated based on the object information in the reception device.
  • a transmission apparatus or program according to the second aspect of the present invention is a transmission apparatus that transmits the data and that forms a broadcasting system together with a reception apparatus that receives data, and broadcasts object information for each object used for content. It is a program for causing a computer to function as a transmission device provided with a transmission means for transmitting data, or as a transmission device.
  • a transmission method is a transmission method of a transmission apparatus that transmits the data, which constitutes a broadcasting system together with a reception apparatus that receives data, and the transmission apparatus is configured for each object used for content.
  • a transmission method including a step of transmitting object information as broadcast data.
  • object information for each object used for content is transmitted as broadcast data.
  • the receiving device or program according to the third aspect of the present invention is a receiving device that receives the data and constitutes a broadcasting system together with a transmitting device that transmits data, and the transmitting device is for each object used for content.
  • the object information is transmitted as broadcast data, it is a receiving device including a generating unit that generates content based on the object information, or a program for causing a computer to function as the receiving device.
  • a receiving method is a receiving method for a receiving apparatus that receives the data, which constitutes a broadcasting system together with a transmitting apparatus that transmits data, and the transmitting apparatus is provided for each object used for content.
  • the receiving apparatus includes a step of generating content based on the object information.
  • content is generated based on the object information.
  • the presentation device or program according to the fourth aspect of the present invention presents content generated based on the data received by the receiving device that receives the data, which constitutes a broadcasting system together with the transmitting device that transmits data.
  • a presentation device comprising: a presentation unit that presents content generated based on the object information when the transmission device transmits object information for each object used for the content as broadcast data; Or it is a program for functioning a computer as a presentation apparatus.
  • the presentation method of the 4th side surface of this invention is a presentation apparatus which presents the content produced
  • program can be provided by being transmitted through a transmission medium or by being recorded on a recording medium.
  • the transmission device, the reception device, and the presentation device may be independent devices, or may be internal blocks constituting one device.
  • content can be presented flexibly.
  • FIG. 2 is a block diagram illustrating a configuration example of a sensing device 11.
  • FIG. It is a figure explaining the 1st form of sensing by sensing device. It is a figure explaining the 2nd form of sensing by sensing device. It is a figure explaining the 3rd form of sensing by sensing device 11.
  • 3 is a block diagram illustrating a configuration example of a transmission device 12.
  • FIG. 4 is a flowchart for explaining processing of a transmission device 12.
  • 3 is a block diagram illustrating a configuration example of an object information generation unit 31.
  • FIG. 10 is a flowchart illustrating processing of the reception device 13.
  • 3 is a block diagram illustrating a configuration example of a presentation device 14.
  • FIG. 3 is a block diagram illustrating a detailed configuration example of a presentation device 14.
  • FIG. 10 is a diagram schematically illustrating an example of arrangement of devices constituting a presentation unit 223. 10 is a flowchart for explaining processing of the presentation device 14.
  • It is a block diagram which shows the other detailed structural example of the presentation apparatus. It is a figure which shows the structural example of the presentation apparatus 14 typically. It is a perspective view which shows typically the structural example of a hemispherical display. It is a block diagram which shows the structural example of one Embodiment of the computer to which this invention is applied.
  • FIG. 1 shows a transmission / reception system to which the present invention is applied (a system is a logical collection of a plurality of devices, regardless of whether each configuration device is in the same housing). The structural example of embodiment is shown.
  • the transmission / reception system is, for example, a broadcasting system that broadcasts content as a program, and includes a sensing device 11 and a transmission device 12 as transmission-side devices, and a reception device 13 as a reception-side device, And the presentation device 14.
  • the sensing device 11 senses various objects constituting the content (used for the content), and supplies data obtained as a result of the sensing to the transmission device 12.
  • the transmission device 12 generates various data (information) necessary for content presentation from the data from the sensing device 11, and transmits it as broadcast data for broadcasting.
  • the transmission device 12 generates content reflecting the intention of the content creator, that is, data necessary for presentation of the edited content by the content creator, and transmits it as broadcast data.
  • Broadcast data is transmitted via transmission media such as terrestrial, satellite, Internet, and CATV (Cable Television) networks.
  • transmission media such as terrestrial, satellite, Internet, and CATV (Cable Television) networks.
  • the reception device 13 receives the data transmitted from the transmission device 12, generates presentation data (presentation data as content) for presentation by the presentation device 14 from the data, and supplies the presentation data to the presentation device 14. To do.
  • the receiving device 13 generates presentation data based on an instruction from the user as necessary.
  • the presentation device 14 presents content to the user according to the presentation data from the reception device 13.
  • FIG. 2 is a diagram illustrating the sensing device 11 of FIG.
  • various sensors that sense any physical quantity required for presentation of content by the presentation device 14 can be employed.
  • the sensing device 11 for example, a force sensor for sensing force, a speed sensor for sensing speed, a rotation number sensor for sensing (measuring) the number of revolutions, a liquid sensor for sensing liquid, a gas concentration, an ion concentration, or the like.
  • An optical sensor that senses light, a temperature sensor that senses temperature, and the like can be employed.
  • the optical sensor includes an imager (camera) that performs photographing and outputs an image in addition to a sensor that detects light and brightness.
  • imager camera
  • the sensing device 11 employs a sensor that senses time, electric field, current, radiation (X-ray), flow rate, humidity, vibration, smell, infrared light, sound (voice), and other physical quantities. can do.
  • the sensing device 11 senses various objects used for content, and there are first, second, and third forms of sensing.
  • FIG. 3 shows a first form of sensing by the sensing device 11.
  • an object that is a background and two objects that are objects other than the background for example, a person who is an actor, a car, other tangible objects, etc.
  • a And object sensing for the content consisting of B and B.
  • the entire background is a single object
  • the objects A and B are each one of the objects A and B.
  • a background sensor that is a sensor that senses an object that is a background and an object sensor that is a sensor that senses each of the objects A and B are separately prepared, The background of the state constituting the content and each of the two objects A and B are sensed simultaneously.
  • the background is sensed by a background sensor
  • the object A is sensed by an object sensor
  • the object B is sensed by another object sensor.
  • the background sensor includes a camera that senses light and outputs an image, a microphone that senses sound, a temperature sensor that senses temperature, a humidity sensor that senses humidity, an ion density sensor that senses ion density, and an odor
  • An odor sensor that senses a position, a position sensor that senses a position (for example, GPS (Global Positioning System)), a distance sensor that senses a distance, and the like are used.
  • the object sensor includes the above-mentioned camera, microphone, temperature sensor, odor sensor, position sensor, taste sensor for sensing taste, angular velocity sensor for sensing angular velocity, sensor for sensing X-rays, etc. Used.
  • a plurality of object sensors can be assigned to one object, and one object can be sensed by the plurality of object sensors. The same applies to the background sensor.
  • the arrangement of the plurality of object sensors or background sensors is, for example, the camera arrangement described in JP-A-2004-266511. Can be adopted.
  • FIG. 4 shows a second form of sensing by the sensing device 11.
  • a background sensor that is a sensor that senses a background object and an object that is a sensor that senses each of the objects A and B
  • the sensor for use is prepared separately, and the background used for the content and the two objects A and B are sensed simultaneously.
  • the sensing of the background and the objects A and B is not performed in a state where the background and the objects A and B constitute the content, but each of the background, the object A, and the object B It is done in a separate state.
  • two cars as objects A and B are used.
  • sensing such as shooting of the race track as the background and the two cars as the objects A and B is performed.
  • sensing such as shooting of a racetrack in a state where a car race is not performed is performed, and sensing of a car as an object A is performed at a place and time unrelated to the sensing of the racetrack. Is called.
  • the sensing of the vehicle as the object B is also performed independently of the sensing of the racetrack and the sensing of the object A.
  • FIG. 5 shows a third form of sensing by the sensing device 11.
  • a sensor for sensing the background a sensor for sensing the object A, and a sensor for sensing the object B are combined.
  • a sensor that combines a sensor that senses the background, a sensor that senses the object A, and a sensor that senses the object B is referred to as a dual-purpose sensor.
  • sensing is performed in a state where the physical quantities of the background, the object A, and the object B are combined.
  • sensing information data obtained as a result of sensing the physical quantity in the sensing device 11 is used to generate object information for each object in the transmission device 12, as will be described later.
  • the physical quantity sensed by the dual-purpose sensor can separate the sensing information output by the dual-purpose sensor into sensing information for each object.
  • an image as sensing information output by a camera can be separated into images for each object by an object image extraction technique.
  • sound as sensing information output from a microphone can be separated into sound for each object by a sound source separation technique. Therefore, the camera and the microphone can be used as a dual-purpose sensor.
  • FIG. 6 shows a configuration example of the transmission device 12 of FIG.
  • the transmission device 12 includes an object information generation unit 31, a content default information generation unit 32, and a transmission unit 33.
  • Sensing information is supplied from the sensing device 11 to the object information generation unit 31. Further, the object information generating unit 31 is also supplied with instruction information representing an instruction for editing performed by the content creator operating an operation unit (not shown).
  • the object information generation unit 31 generates object information for each object from the sensing information from the sensing device 11, for example, for an object that is instructed to be edited by the instruction information. Further, the object information generating unit 31 generates a unique object ID (idenfitication) for identifying the object, and supplies the generated object ID to the transmitting unit 33 together with the object information of the object represented by the object ID.
  • object ID is also supplied from the object information generation unit 31 to the content default information generation unit 32.
  • the object information generated by the object information generation unit 31 includes object information of a background object and object information of an object other than the background.
  • the background object information is also referred to as background object information
  • the object information of an object other than the background is also referred to as object object information.
  • the object information is also information for reproducing the feeling obtained when viewing the object.
  • the senses that can be obtained when appreciating an object are those that can be felt by the so-called five senses, that is, for example, an image of an object that is visually recognized (perceived) (light reflected by the object), an auditory recognition, There are sound (sound) generated by an object, smell of an object recognized by smell sense, taste of an object recognized by taste sense, temperature of an object recognized by touch sense, and the like.
  • the object information is information for reproducing the sense obtained when viewing the object, and thus the content configured using the object, and is information for each object.
  • information for reproducing a sense obtained when viewing an object is also referred to as sensory information.
  • a parameter (hereinafter also referred to as a model parameter) for identifying a physical model necessary for a physical simulation for reproducing a sense obtained when an object is viewed can be adopted.
  • the model parameters include, for example, the first parameter specific to the object for simulating the motion of the object as a rigid body, and when the object is viewed from various positions and directions under various lighting conditions. There is a second parameter specific to the object for simulating an image of the object that is visible.
  • Examples of the first parameter for simulating the motion when the object is a rigid body include the object boundary used in the object's mass, moment of inertia, and geometry (interaction between objects (collision, friction, etc.)) Shape), coefficient of restitution, coefficient of friction, size in the real world (for example, how much distance the unit scale of mesh coordinates corresponds to in the real world), and the like.
  • the first parameter for the background object includes, for example, the wind speed (in air), the flow velocity (in water), gravity, and the like.
  • the receiving device 13 uses the first parameter to perform a simulation for simulating the motion when the object is a rigid body.
  • ODE Open Dynamics Engine
  • a second parameter for simulating an image of an object that is visually seen when the object is viewed from various positions and directions under various lighting conditions for example, external shapes such as color information and mesh information Model, illumination information, and object material information (related to illumination reflectance and reflection characteristics).
  • intermediate data described in Japanese Patent Laid-Open No. 2004-015106 can be employed.
  • the intermediate data is, for example, data of an image that appears visually when the object is viewed from various directions, and is also referred to as a light space expression database as appropriate.
  • the receiving device 13 uses the second parameter to view the object from various positions and directions under various illumination conditions.
  • a simulation is performed to simulate an image of an object that is visually visible.
  • CG Computer Graphics
  • R Open Graphics Graphics Library
  • the ideal information as model parameters is information necessary for simulating all state changes that the object causes in the real world (real world).
  • the receiving device 13 uses the ideal information to, for example, rigid body simulation that simulates the motion of a rigid body, and object deformation such as fluid.
  • rigid body simulation that simulates the motion of a rigid body
  • object deformation such as fluid.
  • general-purpose dynamics simulation eg lattice method, particle method, etc.
  • simulation of light diffusion simulation of temperature change
  • simulation of temperature change simulation of temperature change
  • atomic A large number of simulators for performing such a simulation are required.
  • object specific information can be used as object information instead of model parameters.
  • the object specific information is, for example, sensing information obtained by sensing an object with the sensing device 11 and information for simulating a pseudo physical quantity of the object.
  • Sensing information as object-specific information includes, for example, information representing the sound (sound) emitted by the object, the temperature of the object, the internal structure of the object (photographed by X-rays), the odor of the object, the taste of the object, etc. is there.
  • information for simulating an object's pseudo physical quantity as object-specific information for example, information necessary for cross-creation that creates sound from the appearance of an object (for example, generated when objects collide) Template).
  • the object information can include information for improving the entertainment property of content (hereinafter also referred to as entertainment information), energy information, and trigger information.
  • the entertainment information is, for example, information for creating a change in the state of an object that cannot be performed by physical simulation, and is, for example, information on a sketch drawing (an image like a sketch drawing) (a sketch-like image). .
  • Energy information is information necessary to realize reality considering the viewing environment (for example, display size) on the receiving device 13 side, for example, the size of an object.
  • the object for example, a person
  • the object for example, a person
  • the object displayed on the display is displayed so that the object is not displayed larger than the size in the real world, that is, as if the user is looking at the object in the real world.
  • Reality processing is performed to adjust the size of.
  • Energy information is used in the receiver 13 for the reality processing as described above.
  • Trigger information is information necessary for the above-mentioned cross creation, for example.
  • the content default information generating unit 32 is supplied with instruction information indicating an instruction for editing performed by the content creator operating an operation unit (not shown).
  • the content default information generation unit 32 generates content default information including time-series data regarding the state of the object represented by the object ID supplied from the object information generation unit 31 according to the instruction information from the content creator. To the transmission unit 33.
  • the content creator Information related to the state of the object, such as the placement and orientation of each object in the content world, represented by the object ID from the information generation unit 31, the lighting conditions in the content world (the position of the light source that illuminates the object in the content world, Brightness, etc.) and information such as the position and orientation of the viewer (user) who appreciates the object in the content world are instructed by operating the operation unit.
  • the content default information generation unit 32 generates time-series data relating to the state of the object, time-series data on the lighting conditions, time-series data such as the position and orientation of the viewer, in accordance with the instruction information representing the instruction of the content creator. Is sent to the transmission unit 33 as content default information.
  • the arrangement (position) and orientation of each object in the content world when there is no specific instruction from the user, the arrangement (position) and orientation of each object in the content world, the content world, based on the content default information.
  • the lighting conditions, viewer position and orientation are set, and according to the settings, images viewed from the viewer are generated as contents as a program.
  • the transmission unit 33 broadcasts the object information (object object information and background object information) supplied from the object information generation unit 31, the object ID, and the content default information supplied from the content default information generation unit 32. Send (broadcast) as data.
  • the object information generated by the object information generation unit 31 may be sensory information for all five senses, or sensory information for a part of the five senses, for example, 1 of the five senses. , 2, 3 or 4 and sensory information about one or more of senses other than sight and hearing, that is, sense of smell, taste, or touch.
  • step S11 the object information generation unit 31 generates object information for each object from the sensing information supplied from the sensing device 11, and generates an object ID for identifying the object. Furthermore, the object information generation unit 31 supplies the object ID to the content default information generation unit 32, and associates the object ID with the object information of the object represented by the object ID and supplies the object ID to the transmission unit 33.
  • step S 11 the content default information generation unit 32 generates content default information for each object represented by the object ID from the object information generation unit 31 according to the instruction information from the content creator, and supplies the content default information to the transmission unit 33. To do.
  • step S11 the process proceeds from step S11 to step S12, and the transmission unit 33 broadcasts the object information associated with the object ID from the object information generation unit 31 and the content default information from the content default information generation unit 32. Send as data.
  • FIG. 8 shows a configuration example of the object information generation unit 31 of FIG.
  • the object information generation unit 31 includes a background object information generation unit 41, an object object information generation unit 42, and an ID provision unit 43.
  • Sensing information is supplied from the sensing device 11 (FIG. 1) to the background object information generating unit 41 and the object object information generating unit 42.
  • the background object information generation unit 41 generates background object information of a background object (an object other than an object object necessary for generating content) from the sensing information from the sensing device 11 (including background sensing information). , And supplied to the ID assigning unit 43.
  • the background object information includes model parameters for identifying a physical model necessary for a physical simulation for reproducing a sense obtained when an object is viewed.
  • model parameters for example, the first parameter for simulating the motion when the object is a rigid body, and when the object is viewed from various positions and directions under various lighting conditions
  • background object for example, wind speed, flow velocity, gravity, and the like at various positions of the background object are the first parameters as background object information.
  • a ray space expression database of a background object that can be created from an image of only the background object (hereinafter also referred to as a background image) obtained when the background object is viewed from a plurality of angles (directions).
  • the second parameter as background object information.
  • the image as sensing information output by the sensing device 11 includes: In addition to the background object, a physical object is also shown.
  • the background object information generating unit 41 extracts a background image, that is, an image showing only the background object, from the image showing both the background object and the object object using the background memory.
  • a method for extracting a background image using a background memory is described in, for example, Japanese Unexamined Patent Application Publication No. 1995-038858, Japanese Unexamined Patent Application Publication No. 1995-046533, Japanese Unexamined Patent Application Publication No. 1999-164264, and the like.
  • a ray space expression database of background objects from a plurality of angles of background images can be performed by, for example, the technique described in Japanese Patent Application Laid-Open No. 2004-015106.
  • the background object information generation unit 41 extracts color information of light rays emitted from the background object in a predetermined direction from the background images at a plurality of angles, and the color information is extracted from the background object from infinity in the predetermined direction.
  • the image data when viewed is held for each predetermined direction.
  • the image data held for each predetermined direction is a ray space expression database.
  • a pixel value interpolation method for example, an average value of pixel values used for interpolation is set as a pixel value of a pixel to be interpolated, or image data before interpolation is converted into image data after interpolation by class classification adaptive processing. There is a method of converting to image data.
  • the class classification adaptive processing is described in, for example, Japanese Patent Application Laid-Open No. 2007-235312.
  • sensing information (as object-specific information) can be adopted as background object information (object information) instead of model parameters.
  • the sensing information that becomes the background object information includes, for example, the sound emitted by the background object (sound of the background object), the temperature of the background object, the humidity of the background object, the smell of the background object, the size of the background object (in the real world) There is information indicating the size (space size, etc.).
  • the sound as sensing information output from the sensing device 11 is the sound of the background object. Therefore, the sound of only the background object can be obtained.
  • the sensing device 11 collects sound as sensing in the first or third form (FIG. 3 or 5), the sound as sensing information output from the sensing device 11 In addition to the sound of the background object, the sound of the object object may be included.
  • the background object information generation unit 41 extracts only the sound of the background object from the sound including the sound of both the background object and the object object by the sound source separation technique, and uses it as the background object information.
  • the object object information generating unit 42 generates object object information of the object object for each object object from the sensing information from the sensing device 11 (among the object sensing information), and supplies the object object information to the ID assigning unit 43.
  • model parameters for identifying a physical model necessary for a physical simulation for reproducing a sense obtained when an object is viewed For example, the first parameter for simulating the motion when the object is a rigid body, and when the object is viewed from various positions and directions under various lighting conditions.
  • a second parameter for simulating an image of an object that is visible to the eye There is a second parameter for simulating an image of an object that is visible to the eye.
  • the mass, size, geometry shape, moment of inertia, friction coefficient, etc. of the object object are the first parameters as the object object information.
  • sensing for measuring the weight of the object object is performed in the sensing device 11 (FIG. 1), and the object object information generation unit 42 calculates the object from the sensing information obtained as a result of the sensing. Get the mass of an object.
  • the sensing device 11 For the size and geometry shape of the object object, the sensing device 11 performs sensing for measuring the object object from each direction of 360 degrees using, for example, laser light, and the object object information generating unit 42 From the sensing information obtained as a result of sensing, the size of the object object and the geometry shape are calculated.
  • the object object information generation unit 42 calculates the moment of inertia using the geometry shape, size, and density of the object object.
  • values of the coefficient of restitution and the friction coefficient are prepared in advance for each type of object object, and values corresponding to the type of object object are selected from the values. .
  • the coefficient of restitution and the coefficient of friction may be measured if the sensing device 11 (FIG. 1) can perform measurement as sensing.
  • the second parameter as object object information is a parameter for simulating an image of an object object that is visually seen when the object object is viewed from various positions and directions under various illumination conditions.
  • the wire frame (three-dimensional shape) and texture information used for creating the image can be employed.
  • the object object information generation unit 42 obtains the first object from an image (for example, a stereo image) obtained by viewing the object object from two or more directions, which is obtained by performing imaging as sensing in the sensing device 11.
  • the wire frame and texture information are obtained as the second parameter.
  • a method for obtaining a wire frame and texture information from a stereo image and a method for creating an image (CG) from the wire frame and texture information are described in, for example, Japanese Patent Application Laid-Open No. 2005-165468.
  • sensing information (as object-specific information) can be adopted as object object information (object information) instead of model parameters.
  • Sensing information that is object object information includes, for example, sound emitted by the object object (sound of the object object), temperature of the object object, taste of the object object, smell of the object object, and object object (captured by X-ray) There is information representing the internal structure of the object and the size of the object.
  • the sound as sensing information output from the sensing device 11 is one object object. Therefore, the sound of only one object can be obtained.
  • the sensing device 11 collects sound as sensing in the first or third form (FIG. 3 or 5), the sound as sensing information output from the sensing device 11 In addition to the sound of one object object, the sound of other object objects and background objects may be included.
  • the object object information generation unit 42 extracts only the sound of one object object from the sound including the sound of one object object and the sounds of other object objects and background objects, using the sound source separation technique.
  • the object object information of the object object is used.
  • the ID assigning unit 43 adds an object ID for identifying the background object to the background object information of the background object from the background object information generating unit 41, supplies the object ID to the transmitting unit 33 (FIG. 6), and generates object object information.
  • An object ID for identifying the body object is added to the body object information of the body object from the unit 42 and supplied to the transmission unit 33.
  • the object object information generation unit 42 obtains the wire frame and texture information using the technique described in JP-A-2005-165468.
  • FIG. 9 shows a configuration example of an image processing apparatus that obtains a wire frame and texture information using the technique described in Japanese Patent Laid-Open No. 2005-165468, and further creates an image from the wire frame and texture information. ing.
  • the image processing apparatus in FIG. 9 calculates the three-dimensional shape of the measurement object 101 from two or more images taken at least two as a unit, and measures the three-dimensional shape of the measurement object 101.
  • a texture that expresses a stereoscopic effect is pasted on the 2D image of the measurement object 101.
  • the texture refers to a pattern attached to the surface of a graphic or a drawing for expressing the texture in graphics or the like.
  • a measurement object 101 is a tangible object that is a construction object or a production object, and corresponds to, for example, various works such as buildings, people, and landscapes.
  • the image photographing device 110 is a device in which, for example, a stereo camera or a general-purpose digital camera is combined with a device that compensates lens aberration for the left and right images of the measurement object 101 photographed by the digital camera.
  • the stereo image data storage unit 112 stores a stereo image of the measurement object 101. That is, the stereo image data storage unit 112 stores the left and right images of the measurement object 101 photographed by the image photographing device 110.
  • the three-dimensional position measuring device 114 is a device that automatically measures a target position to be surveyed using a laser or the like, and includes a total station, a three-dimensional scanner, and the like.
  • the measurement object position data storage unit 116 stores the position data of the measurement object 101 serving as three reference points required for absolute orientation. That is, the measurement object position data storage unit 116 stores the position data of the reference point measured by the three-dimensional position measurement device 114. Note that the object position data storage unit 116 does not need to be provided in applications where relative orientation using the model coordinate system is sufficient without performing absolute orientation.
  • Corresponding point instruction unit 122 instructs corresponding points used in the orientation unit 124.
  • the corresponding point instruction unit 122 is configured to indicate feature points in the vicinity of the indicated position with respect to the measurement object 101 displayed on the display device 40.
  • the feature points near the designated position include, for example, the center position, the center of gravity position, and the corner position of the measurement object 101.
  • the orientation unit 124 obtains the corresponding point relationship of the stereo image based on the shooting position and the inclination regarding the stereo image.
  • the display image forming unit 130 includes a three-dimensional coordinate data unit 131, a model forming unit 132, a model storage unit 133, an image corresponding unit 134, a model display unit 135, a posture instruction unit 136, and an image conversion unit 137.
  • the three-dimensional coordinate data unit 131 obtains the three-dimensional coordinate data of the corresponding point of the measurement object 101 from the corresponding point relationship obtained by the orientation unit 124.
  • the model forming unit 132 forms a model of the measurement object 101 from the three-dimensional coordinate data of the corresponding points.
  • the model storage unit 133 stores the model of the measurement object 101 formed by the model forming unit 132.
  • the image correspondence unit 134 associates the stereo image of the measurement object stored in the stereo image data storage unit 112 with the model formed by the model formation unit 132 using the corresponding point relationship obtained by the orientation unit 124.
  • the model display unit 135 displays the measurement object 101 on the display device 140 as a two-dimensional image having a stereoscopic effect using the stereo image associated with the model by the image corresponding unit 134.
  • the model display unit 135 can be a display unit for confirming an image of an object model on the broadcasting station side.
  • the orientation instruction unit 136 instructs the orientation of the model of the measurement object 101 displayed on the display device 140, for example, when the operator operates a cursor input device such as a mouse.
  • the image conversion unit 137 performs coordinate conversion of corresponding points in accordance with an orientation instruction for the model.
  • the display device 140 is an image display device such as a liquid crystal display device.
  • the image processing apparatus configured as described above, first, two or more images of the measurement object 101 captured using the image capturing apparatus 110 such as a digital camera are registered in the stereo image data storage unit 112.
  • a set of left and right images forming a stereo pair is set.
  • the processing can be performed in the relative coordinate system, and thus the three-dimensional coordinates of the measurement object 101 are used in the measurement object position data storage unit 116.
  • the above-described processing for reading the received position data is skipped.
  • the orientation unit 124 performs orientation work, performs relative orientation of the stereo image of the measurement object 101 stored in the stereo image data storage unit 112, and obtains the corresponding point relationship of the stereo image.
  • This orientation work has three modes: manual, semi-automatic, and fully automatic.
  • the orientation work is an operation in which corresponding points (same points) of two or more images are indicated on each image by the corresponding point instruction unit 122 with a mouse cursor or the like, and the image coordinates are read. Normally, 6 or more corresponding points are required for each image. If the reference point coordinates are stored in the measurement object position data storage unit 116, the absolute orientation is executed by associating the reference point coordinates with the image.
  • the orientation calculation process is performed by the orientation unit 124 using the coordinates of the corresponding points obtained by the orientation work.
  • the orientation calculation process By the orientation calculation process, the position and tilt of the photographed camera, the position of the corresponding point, and the measurement accuracy can be obtained. If there is a reference point coordinate, it is read from the measurement object position data storage unit 116 and an absolute orientation calculation process is also performed. If there are no reference point coordinates, the calculation is performed using relative (model) coordinates.
  • the orientation calculation processing is performed by mutual orientation for associating stereo models, and the orientation between all images is performed by bundle adjustment.
  • stereo measurement is performed in the three-dimensional coordinate data unit 131 to obtain the three-dimensional coordinates of the measurement object 101.
  • the three-dimensional coordinate data unit 131 stores the measurement object position data.
  • the three-dimensional coordinates of the corresponding points are read from the unit 116.
  • stereo measurement there are various modes of manual measurement, semi-automatic measurement, and automatic measurement, as described in, for example, Japanese Patent Laid-Open No. 2003-284098.
  • stereo measurement in stereo measurement, a stereo (right and left) image that can be viewed stereoscopically is created and displayed, and the three-dimensional coordinates of the corresponding points are obtained using the above-described mode.
  • automatic measurement is performed by specifying a measurement region for automatic measurement.
  • manual and semi-automatic measurement while observing the left and right stereo screens, the corresponding points on the left and right images are measured semi-automatically or while the points corresponding to the manual are determined with the mouse.
  • the model forming unit 132 creates a model of the measurement object 101, for example, a wire frame.
  • Texture mapping refers to pasting a texture expressing a three-dimensional effect on a two-dimensional image of the measurement object 101.
  • the image corresponding unit 134 texture-maps the stereo image of the measurement object 101 and the model formed by the model forming unit 132 using the corresponding point relationship obtained by the orientation unit 124.
  • the model display unit 135 displays the texture-mapped image on the model screen.
  • the model screen is a two-dimensional image of the measurement object 101 that represents a perspective state, such as an image with a three-dimensional texture formed from a stereo image or an image from which the texture is removed.
  • the image conversion unit 137 displays the measurement object 101 displayed on the display device 140.
  • the display direction is coordinate-converted to the direction instructed by the posture instruction unit 136.
  • FIG. 10 is a block diagram showing a configuration example of the receiving device 13 of FIG.
  • the receiving device 13 includes a transmission data receiving unit 201, an object information DB (Database) 202, a generating unit 203, a command receiving unit 206, and a processing information DB 207.
  • the transmission data reception unit 201 receives data transmitted from the transmission device 12 (the transmission unit 33 (FIG. 6)), and separates it into object information associated with an object ID and content default information. Further, the transmission data receiving unit 201 supplies object information associated with an object ID (hereinafter also referred to as object information with an object ID) to the object information DB 202, and supplies content default information to the generation unit 203. .
  • object information with an object ID hereinafter also referred to as object information with an object ID
  • the object information DB 202 stores object information with an object ID supplied from the transmission data receiving unit 201.
  • the generation unit 203 generates content based on the object information stored in the object information DB 202.
  • the generation unit 203 is also based on processing information that is information related to processing of content in accordance with an instruction from a user (a user of the receiving device 13) who views the content, supplied from the command reception unit 206 or the processing information DB 207. To generate content.
  • the generation unit 203 generates content based on the content default information supplied from the transmission data reception unit 201 as necessary.
  • the generation unit 203 includes a simulation unit 204 and a presentation data generation unit 205.
  • the simulation unit 204 generates a content (content world) by performing a physical simulation based on a model parameter as object information stored in the object information DB 202, and generates the content (data thereof) as presentation data generation. To the unit 205.
  • simulation unit 204 applies processing to the objects constituting the content based on the processing information supplied from the command receiving unit 206 and the processing information DB 207 and the content default information supplied from the transmission data receiving unit 201.
  • the simulation unit 204 applies processing to an object (and thus content configured from the object) based on the processing information.
  • the physical simulation unit 204 is based on the content default information supplied from the transmission data reception unit 201. Add processing to the object.
  • the presentation data generation unit 205 presents the presentation by the presentation device 14 that performs presentation for giving a sense corresponding to the sensory information based on the content generated based on the object information that is sensory information from the simulation unit 204. Presentation data to be performed is generated and supplied to the presentation device 14.
  • the presentation data generation unit 205 is supplied with the processing information from the command reception unit 206 and the processing information DB 207.
  • the presentation data generation unit 205 generates presentation data based on the processing information supplied from the command reception unit 206 and the processing information DB 207.
  • the command receiving unit 206 receives a command representing a user instruction transmitted from the remote commander 210.
  • the remote commander 210 transmits a command corresponding to the operation, that is, a command representing the user's instruction, by infrared rays or the like when instructing processing or the like on the object (content).
  • the command receiving unit 206 receives the command transmitted from the remote commander 210 as described above.
  • the command receiving unit 206 generates processing information that is information related to processing on the content in accordance with a user instruction represented by the command from the remote commander 210, and generates a simulation unit 204, a presentation data generation unit 205, and processing information. Supply to DB207.
  • the machining information DB 207 sequentially stores the machining information supplied from the command receiving unit 206.
  • the processing information DB 207 stores a history of user instructions.
  • the simulation unit 204 simulates the content world based on at least the object information and the content default information.
  • At least an object image (light reflected by the object) that is visually recognized (perceived) when the user appreciates the content world is set as a simulation target.
  • the content default information is time-series data related to the state of the object, time-series data of lighting conditions, time-series data such as the position and orientation of the viewer, etc., instructed by the content creator.
  • the simulation unit 204 performs settings such as the arrangement and orientation of objects, lighting conditions, the viewing position and orientation of the user (content viewer) based on the content default information, and generates content according to the settings.
  • content as intended by the content creator for example, a program similar to the current television broadcast program
  • content as intended by the content creator for example, a program similar to the current television broadcast program
  • the simulation unit 204 simulates a change in the state of the object if necessary.
  • the transmission device 12 does not include the data related to the object state in the content default information, that is, the time series related to the object state as the content default information. Instead of data, an initial value of the object state can be transmitted.
  • the simulation unit 204 generates content by performing a physical simulation that simulates a change in the state of the object from the initial value of the state of the object as content default information.
  • a specific example of the “change in state governed by a simulatable physical law” includes, for example, a change in position (speed) due to free fall of an object.
  • the physical simulator for example, a simulator capable of simulating collisions between objects
  • changes in position due to collisions between objects whose required parameters are already known are also “changes in the state governed by physics that can be simulated” Can be considered.
  • the simulation unit 204 can perform not only the motion simulation but also a physical simulation of a change in state caused by various physical phenomena such as heat conduction using the initial value of the object state.
  • the transmission device 12 transmits content default information that does not include data (time-series data) relating to the object state, and the reception device 13 simulates a change in the object state, thereby The amount of data transmitted from the transmission device 12 can be reduced as compared with the case of transmitting content default information including time-series data related to.
  • the simulation unit 204 simulates the content world based on the intervention information from the user.
  • the simulation unit 204 determines that the content default information Content is generated using processing information with priority.
  • the processing information is information regarding processing on the content, and here is information regarding processing for each object, for example, manipulating the position of the object.
  • the command receiving unit 206 when the user operates the remote commander 210 to indicate, for example, the position of the object object, the command receiving unit 206 generates processing information for operating the position of the object object according to the instruction, and the simulation unit 204. Based on the processing information from the command receiving unit 206, the simulation unit 204 operates (sets) the positions of the object objects constituting the content.
  • the user can operate the remote commander 210 to instruct, for example, deletion of an object object, replacement with another object, force (shooting force) action (tapping), temperature, and the like. Further, the user can operate the remote commander 210 to instruct, for example, the wind speed, flow velocity, gravity, temperature, and the like of the background object.
  • the command receiving unit 206 generates processing information related to processing of content (objects constituting the content) according to a user instruction, and supplies the processing information to the simulation unit 204.
  • the simulation unit 204 based on the processing information from the command receiving unit 206, the object object and the background object that constitute the content are processed.
  • the user can operate the remote commander 210 to instruct deletion of data relating to a predetermined sense of a predetermined object.
  • the command receiving unit 206 generates processing information for deleting data related to a predetermined sense of a predetermined object in accordance with a user instruction, and supplies the processing information to the simulation unit 204.
  • the simulation unit 204 based on the processing information from the command reception unit 206, data related to a predetermined sense of a predetermined object that constitutes the content is deleted.
  • the simulation unit 204 deletes the peanut taste data constituting the content. Is done. In this case, for example, it is possible to prevent the taste of peanuts from being presented to a peanut allergic user.
  • the user can operate the remote commander 210 to instruct to convert data related to a predetermined feeling of a predetermined object into data related to a predetermined feeling of another object.
  • the command receiving unit 206 generates processing information for converting data related to a predetermined sensation of a predetermined object into data related to a predetermined sensation of another object in accordance with a user instruction, and the simulation unit 204 To supply.
  • the simulation unit 204 based on the processing information from the command receiving unit 206, data related to a predetermined feeling of a predetermined object constituting the content is converted into data related to a predetermined feeling of another object.
  • the simulation unit In 204 the data relating to the taste of peanuts constituting the content is converted into data relating to the taste of strawberries.
  • peanuts that feel the taste of strawberries are presented.
  • the user can operate the remote commander 210 to instruct the position (viewing position) and direction of the user who is a viewer who appreciates the content world.
  • the command receiving unit 206 generates processing information for operating the viewing position and orientation of the viewer according to the user's instruction, and supplies the processing information to the simulation unit 204.
  • the simulation unit 204 generates content by performing a physical simulation in which the viewing position and orientation of the viewer are set based on the processing information from the command receiving unit 206.
  • the user can instruct to operate the remote commander 210 to perform processing for improving the entertainment property of the content (hereinafter referred to as entertainment processing).
  • the command receiving unit 206 generates processing information for requesting entertainment processing and supplies the processing information to the simulation unit 204.
  • the simulation unit 204 performs entertainment processing based on the processing information from the command reception unit 206.
  • the simulation unit 204 uses the entertainment information described with reference to FIG. 6 included in the object information stored in the object information DB 202, and uses the object image as a sketch drawing of the object.
  • the user can operate the remote commander 210 to instruct to perform reality processing.
  • the command receiving unit 206 generates processing information for requesting to perform the reality processing, and supplies the processing information to the simulation unit 204.
  • the simulation unit 204 performs reality processing based on the processing information from the command reception unit 206.
  • the simulation unit 204 generates content that does not impair the reality according to the size (size) of the display that presents (displays) the image in the presentation device 14.
  • the simulation unit 204 accesses the presentation device 14 and recognizes the size of the display in the presentation device 14. Further, the simulation unit 204 recognizes the size (size) of the objects constituting the content from the energy information described in FIG. 6 included in the object information stored in the object information DB 202.
  • the simulation unit 204 determines that the object displayed on the display is the size of the object in the real world.
  • the content is generated by adjusting (enlarging or reducing) the size of the object so that the size of the object is recognized when viewed in the world.
  • the command receiving unit 206 supplies the processing information to the simulation unit 204 and also supplies the processing information DB 207.
  • the processing information DB 207 When the processing information is supplied from the command receiving unit 206, the processing information DB 207 generates status information indicating the situation when the processing information is supplied from the command receiving unit 206, and stores the processing information in association with the processing information.
  • the machining information DB 207 supplies the machining information associated with the situation information to the simulation unit 204.
  • the simulation unit 204 performs the same processing as when processing information is supplied from the command receiving unit 206 based on the processing information.
  • the user when content including peanuts is presented on the presentation device 14, the user operates the remote commander 210 to instruct deletion of the data related to the taste of peanuts as described above, and the command receiving unit
  • the processing information is stored in the processing information DB 207 in association with situation information representing a situation in which content including peanuts is presented. .
  • the processing information DB 207 displays processing information associated with the situation information representing the situation, that is, processing information for deleting data related to the taste of peanuts,
  • the data is supplied to the simulation unit 204 (and the presentation data generation unit 205).
  • data related to the taste of peanuts constituting the content is deleted based on the processing information from the processing information DB 207 as in the case where the processing information is supplied from the command receiving unit 206.
  • the simulation unit 204 when processing information is not supplied from the command receiving unit 206 (or processing information DB 207), content is generated based on the content default information.
  • Content as intended by the creator for example, an image of the content world in which objects are arranged as intended by the content creator as viewed from the viewpoint as intended by the content creator.
  • the simulation unit 204 displays the processing information rather than the content default information.
  • the content is generated with priority.
  • the content unique to the user for example, the content intended by the content creator is the content that presents (feels) the peanut taste
  • the data related to the peanut taste is obtained.
  • processing for deleting the data related to the taste of peanut is performed, and the content without presentation of the taste of peanut is presented.
  • the simulation unit 204 gradually changes the state of the object to a state based on the content default information unless the user specifically instructs.
  • the simulation unit 204 simulates so that the content world gradually returns to a state based on the content default information.
  • the simulation unit 204 when the user operates the remote commander 210 to instruct the direction of a certain object to be a predetermined direction, in the simulation unit 204, the content in which the object object is facing the predetermined direction Is generated.
  • the object object that has faced the predetermined direction gradually changes in the direction specified by the content default information. Generated.
  • the content generated by the simulation unit 204 as described above is supplied to the presentation data generation unit 205.
  • the presentation data generation unit 205 presents the content based on the content from the simulation unit 204 as described above. Data is generated and supplied to the presentation device 14 (FIG. 1).
  • the presentation data generation unit 205 extracts information suitable for the configuration of the presentation device 14 from the content world information simulated by the simulation unit 204 and converts the information into a data configuration optimal for the configuration of the presentation device 14. To the presentation device 14.
  • the presentation device 14 presents (outputs) a speaker that presents (outputs) sound (speech), a display that presents (displays) an image (light), an odor generating device that presents (emits) an odor, and a temperature.
  • the presentation data generation unit 205 synthesizes the audio of each object constituting the content from the simulation unit 204 Audio information is generated as presentation data and supplied to the speaker of the presentation device 14.
  • the presentation data generation unit 205 generates image information of the content world of the content from the simulation unit 204 as presentation data and supplies it to the display of the presentation device 14.
  • the presentation data generation unit 205 generates scent information obtained by synthesizing the scents of the objects constituting the content from the simulation unit 204 as presentation data, and supplies the scent information to the odor generation device of the presentation device 14.
  • the presentation data generation unit 205 generates temperature information indicating the temperature of the background object constituting the content from the simulation unit 204 as presentation data, and supplies it to the air conditioner of the presentation device 14.
  • the presentation data generation unit 205 generates humidity information representing the humidity of the background object that constitutes the content from the simulation unit 204 as presentation data, and supplies it to the humidifier of the presentation device 14.
  • the presentation data generation unit 205 deletes the processing information supplied from the command reception unit 206 or the processing information DB 207 when it is information for deleting data related to a predetermined feeling of a predetermined object. Is included in the presentation data generated based on the content from the simulation unit 204, the deletion target data is deleted (not supplied to the presentation device 14).
  • step S61 the transmission data receiving unit 201 receives the data transmitted from the transmitting device 12 (FIG. 6), and separates it into object information with an object ID and content default information. Further, in step S61, the transmission data receiving unit 201 supplies the object information to the object information DB 202 for storage, supplies the content default information to the generation unit 203, and the process proceeds to step S62.
  • step S62 the generation unit 203 generates content based on the object information stored in the object information DB 202, necessary content default information, and processing information.
  • the simulation unit 204 generates content by performing a simulation based on the object information stored in the object information DB 202, necessary content default information, and processing information, and generates presentation data.
  • the presentation data generation unit 205 generates presentation data based on the content from the simulation unit 204 and supplies the presentation data to the presentation device 14.
  • FIG. 12 shows a configuration example of the presentation device 14 of FIG.
  • the presentation device 14 can be configured by a device that gives stimulation to the five senses and thereby reproduces a sense that would be obtained if the objects that make up the content were appreciated in the real world.
  • the presentation device 14 outputs (presents) sound, a speaker that stimulates hearing, a display that stimulates vision by displaying (presenting) an image, and a smell that emits (presents) odor. It can be composed of a stimulating odor generating device or the like.
  • the presentation device 14 is a negative ion generator that generates negative ions, a chair with an actuator that shakes a seated user, a taste reproduction device that reproduces taste, an air conditioner that adjusts temperature, and humidity. It can be comprised with the device which performs various presentations, such as a humidifier.
  • the transmission device 12 transmits object information for each object, and the reception device 13 generates content based on the object information from the transmission device 12.
  • Presenting sound output, image display, odor generation, taste reproduction, etc. can be performed flexibly.
  • the transmission device 12 transmits object information for each object, and the reception device 13 receives the content based on the object information.
  • the transmission device 12 transmits object information for each object, and the reception device 13 receives the content based on the object information.
  • the reception device 13 receives the content based on the object information.
  • information including the object information of a plurality of objects for example, an image showing a plurality of object objects or each of a plurality of object objects is emitted on the transmission side.
  • Object extraction for example, image extraction of individual object objects, separation of individual sounds (audio), etc., which is indispensable for content (object) processing on the receiving side when a mixed sound is transmitted) Etc.
  • Etc. is not necessary.
  • the transmission device 12 further transmits content default information
  • the reception device 13 generates content based on the object information and the content default information. Therefore, the intention of the content creator (for example, the content creator intended) Object placement etc.) can be added to the content.
  • the receiving device 13 since the receiving device 13 generates content based on the processing information according to the user's instructions, the user can appreciate the user-specific content as described above. In addition, the content processing by the user and the content creator's intention can be merged.
  • the object information is also sensory information for reproducing the senses obtained by the five senses when the object is viewed (the broadcasting performed in the transmission / reception system in FIG. 1 is the visual (image) of the five senses, and Because it is a multi-sensory broadcast that stimulates (and also) senses other than hearing (sound), the user recognizes by smell, taste, and touch in addition to visually recognized images and auditory recognized sounds. Therefore, the user can appreciate the content with a sense other than visual and auditory senses, appreciate the content with three or more senses, and the like.
  • sensory information for reproducing the senses obtained by the five senses when an object is viewed is passive (only images (including characters etc.) recognized visually and sounds recognized by auditory sense (passive).
  • the user likes and dislikes the content.
  • sensory information that reproduces the taste of peanuts is broadcast uniformly to all users and presents the taste of peanuts to the peanut allergic users.
  • the receiving device 13 generates content (generation of presentation data) based on the processing information, and deletes peanut taste data, for example, so that a peanut allergic user can receive peanuts. It is possible to prevent a taste from being presented.
  • the transmission device 12 transmits object information for each object
  • the reception device 13 processes data related to taste, smell, and touch (data that stimulates taste, hearing, and touch). Can be easily performed.
  • the transmitting device 12 transmits sensory information for reproducing the scent of a plurality of objects such as strawberries and peanuts
  • the receiver 13 receives the strawberry and peanuts from the sensory information. It is difficult to separate the olfactory data.
  • the transmission device 12 transmits object information for each object, such a separation of data is unnecessary, and thus there is a great merit.
  • FIG. 13 is a block diagram illustrating a detailed configuration example of the presentation device 14 of FIG.
  • the presentation device 14 includes a sensing unit 221, a presentation data reception unit 222, and a presentation unit 223.
  • the sensing unit 221 senses an environment in which the content presented on the presentation device 14 is viewed (hereinafter also referred to as an viewing environment), and outputs viewing environment information regarding the viewing environment obtained as a result of the sensing.
  • the viewing environment information output by the sensing unit 221 is supplied to the simulation unit 204 of the receiving device 13.
  • simulation unit 204 generation unit 203 of the receiving device 13
  • object information model parameter
  • processing information as well as necessary viewing environment information from the sensing unit 221 are displayed.
  • the content is generated based on the content, and the presentation device 14 presents the content thus generated.
  • the sensing unit 221 senses the positional relationship between the presentation device 14 itself and a user (viewer) who appreciates the content presented by the presentation device 14, for example, and displays information representing the positional relationship. Output as environment information.
  • the sensing unit 221 senses an object that exists in the viewing environment, and outputs information about the object as viewing environment information.
  • the sensing unit 221 senses lighting conditions (such as brightness) in the viewing environment, and outputs information representing the lighting conditions as viewing environment information.
  • the presentation data reception unit 222 receives the presentation data from the generation unit 203 (the presentation data generation unit 205) of the reception device 13 and supplies the presentation data to the presentation unit 223.
  • the presentation unit 223 stimulates the five senses, and thereby reproduces a sensation that would be obtained if the objects constituting the content were viewed in the real world, that is, for example, an audio output unit that outputs audio And various devices such as an image display unit that displays an image, and presents content according to the presentation data from the presentation data receiving unit 222.
  • FIG. 14 schematically shows an example of the arrangement of devices constituting the presentation unit 223 of FIG.
  • a speaker as an audio output unit for outputting audio
  • a display as an image display unit for displaying an image
  • an air conditioner for adjusting temperature
  • a humidifier for adjusting humidity
  • an odor generating device or the like for presenting is provided.
  • the sensing unit 221 senses the viewing environment, and supplies viewing environment information obtained as a result of the sensing to the simulation unit 204 of the receiving device 13.
  • step S71 the presentation data reception unit 222 waits for the presentation data to be supplied from the presentation data generation unit 205 of the reception device 13, receives the presentation data, supplies the presentation data to the presentation unit 223, and performs processing. Advances to step S72.
  • step S72 the presentation unit 223 presents content according to the presentation data from the presentation data reception unit 222.
  • FIG. 16 is a block diagram showing another detailed configuration example of the presentation device 14 of FIG.
  • the presentation device 14 in FIG. 16 is configured in the same manner as in FIG. 13 except that a moving mechanism 231 is newly provided.
  • each object constituting the content such as presentation data obtained by synthesizing the sound of each object constituting the content, or presentation data obtained by synthesizing the odor of each object, in the generation unit 203 of the reception device 13.
  • the presentation data obtained by synthesizing the information is generated and supplied to the presentation device 14.
  • the generation unit 203 of the reception device 13 generates the presentation data of the object for each object constituting the content. And is supplied to the presentation device 14.
  • the presentation device 14 is individually prepared for each object constituting the content. In FIG. 16, only one presentation device 14 is illustrated.
  • the sensing unit 221 supplies the viewing environment information to the moving mechanism 231 in addition to the simulation unit 204.
  • the presentation data generation unit 205 acquires the object information and the content default information from the simulation unit 204 and supplies them to the presentation data receiving unit 222 together with the presentation data.
  • the presentation data reception unit 222 receives the presentation data from the presentation data generation unit 205, the object information, and the content default information, and supplies the presentation data to the presentation unit 223, and also provides the object information and the content default. Information is supplied to the moving mechanism 231.
  • the movement mechanism 231 moves the presentation device 14 based on the viewing environment information from the sensing unit 221, the object information from the presentation data reception unit 222, and the content default information, and the presentation device 14 presents the object information. Control.
  • FIG. 17 is a diagram schematically illustrating a configuration example of the presentation device 14 of FIG.
  • the presentation device 14 is a device constituting the presentation unit 223, a speaker as an audio output unit that outputs audio, a display as an image display unit that displays an image, an air conditioner that adjusts temperature, and a humidity adjustment
  • a humidifier, an odor generating device for presenting an odor, and the like are mounted and moved by a moving mechanism 231.
  • the moving mechanism 231 includes wheels, and moves the presentation device 14 by rotationally driving the wheels.
  • moving mechanism 23 for example, a mechanism that moves the presentation device 14 along a movable rail (movable rail) can be employed.
  • an object presented by one of the presentation devices 14 individually prepared for each object constituting the content is also referred to as a target object.
  • the moving mechanism 231 presents the target object based on information representing the position of the target object on the content generated by the generation unit 203 of the reception device 13. The device 14 is moved.
  • the movement mechanism 231 uses the position information of the object of interest set by the content default information from the presentation data receiving unit 222 and the position information of the viewer (user), and Find the positional relationship on the content.
  • the movement mechanism 231 obtains the actual positional relationship between the presentation device 14 itself (own device) and the viewer, which is grasped by the viewing environment information from the sensing unit 221, from the content default information.
  • the presentation device 14 is moved so as to coincide with the positional relationship on the content (the movement of the presentation device 14 is controlled).
  • the viewer operates the remote commander 210 so as to give an instruction to change the viewpoint position, whereby processing information corresponding to the instruction to change the viewpoint position is generated from the command receiving unit 206 by the generating unit.
  • the presentation data generation unit 205 of the generation unit 203 supplies the processing information from the command reception unit 206 to the presentation data reception unit 222 of the presentation device 14.
  • the presentation data receiving unit 222 receives the processing information from the presentation data generation unit 205 and supplies it to the moving mechanism 231.
  • the moving mechanism 231 moves the presentation device 14 based on information representing the positional relationship between the attention object after the change of viewpoint and the viewer's content, which is grasped from the processing information from the presentation data receiving unit 222.
  • the device as the presentation unit 223 included in the presentation apparatus 14 is a device having directivity in presentation (hereinafter, also referred to as a directional device) such as a flat display or a directional speaker.
  • the moving mechanism 231 controls the orientation of the presentation device 14 (own device) so that the directivity of the directivity device matches the direction with respect to the viewer.
  • the sensing unit 221 senses information representing the relationship between the position and orientation of the presentation device 14 itself (own device) and the viewer as viewing environment information, and the simulation unit 204 of the reception device 13. To supply.
  • the simulation unit 204 simulates the presentation content of the target object to be viewed by a viewer who has a position and orientation relationship represented by the viewing environment information from the sensing unit 221 with the target object in the content world.
  • the presentation data corresponding to the simulation result is supplied from the presentation data generation unit 205 to the presentation device 14 (presentation device 14 that presents the attention object) corresponding to the attention object.
  • the presentation device 14 can perform an operation that emphasizes the interaction with the viewing environment (hereinafter also referred to as an interaction operation).
  • the sensing unit 221 of the presentation device 14 senses an object (including a viewer) existing in the vicinity (viewing environment), and information on the object is The viewing environment information is supplied to the simulation unit 204 of the receiving device 13.
  • the simulation unit 204 of the reception device 13 simulates the movement change or deformation of the object due to the collision, and the result is presented via the presentation data generation unit 205. 14.
  • the presentation device 14 can be caused to interact with an object (including a viewer) existing in the vicinity (such as a collision).
  • the sensing unit 221 of the presentation device 14 senses lighting conditions in the viewing environment, and supplies information representing the lighting conditions to the simulation unit 204 of the receiving device 13 as viewing environment information.
  • the simulation unit 204 of the reception device 13 Based on the viewing environment information from the sensing unit 221, the simulation unit 204 of the reception device 13 generates an image of the object under illumination conditions in the viewing environment and supplies the generated image to the presentation device 14 via the presentation data generation unit 205. .
  • the presentation device 14 can present the object under the illumination condition in the viewing environment.
  • the content is presented by the presentation device 14 prepared for each object, and the content is also based on the viewing environment information obtained by sensing the viewing environment by each presentation device 14.
  • the viewer can be physically interacted with the viewer who is in the viewing environment or the object (furniture, etc.) that is actually present in the viewing environment.
  • sensing unit 221 can be provided not in the presentation device 14 but in the reception device 13.
  • the receiving device 13 is provided separately from the presentation device 14, but the receiving device 13 is not provided separately from the presentation device 14, but is provided separately for each object.
  • Each of the devices 14 can incorporate the function of the receiving device 13.
  • a display as an image display unit for displaying the image of the presentation device 14, in addition to a display for displaying an image on a plane (substantially plane), a display for displaying an image on a hemispherical screen (hereinafter referred to as a hemispherical display). It is also possible to adopt.
  • the hemispherical display is described in, for example, JP-A-2005-070963.
  • FIG. 18 is a perspective view schematically showing a configuration example of a hemispherical display.
  • the hemispherical display includes a screen 251, an image generation unit 252, and a fisheye lens 253.
  • the image generator 252 generates light corresponding to the image.
  • the light generated by the image generation unit 252 is applied to the screen 251 through the fisheye lens 253.
  • the screen 251 is a large hemispherical screen and is installed so as to cover the image generation unit 252 and the fisheye lens 253.
  • the light irradiated through the fisheye lens 253 is scattered, and thereby, for example, an image that can be seen when a certain object is viewed from its periphery is displayed on the entire screen 251.
  • the display unit 261 which is a display area at a position facing the user (viewer) A at a position around the screen 251 among the display areas of the screen 251, An image that would be seen if the object was actually viewed from the position A is displayed.
  • the display unit 262 which is a part of the display area of the screen 251 facing the position of the user B on the opposite side of the position of the user A across the screen 251 has a position from the position of the user B. An image that would be seen if the object was actually viewed is displayed.
  • a plurality of viewers can easily view content.
  • FIG. 19 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.
  • the program can be recorded in advance in a hard disk 305 or ROM 303 as a recording medium built in the computer.
  • the program is stored temporarily on a removable recording medium 311 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a magnetic disc, or a semiconductor memory. It can be stored permanently (recorded).
  • a removable recording medium 311 can be provided as so-called package software.
  • the program can be wirelessly transferred from a download site to a computer via an artificial satellite for digital satellite broadcasting, or LAN (Local Area Network),
  • the program can be transferred to a computer via a network such as the Internet.
  • the computer can receive the program transferred in this way by the communication unit 308 and install it in the built-in hard disk 305.
  • the computer has a CPU (Central Processing Unit) 302 built-in.
  • An input / output interface 310 is connected to the CPU 302 via the bus 301, and the CPU 302 is operated by an input unit 307 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 310.
  • a program stored in a ROM (Read Only Memory) 303 is executed accordingly.
  • the CPU 302 also transfers a program stored in the hard disk 305, a program transferred from a satellite or a network, received by the communication unit 308 and installed in the hard disk 305, or a removable recording medium 311 attached to the drive 309.
  • the program read and installed in the hard disk 305 is loaded into a RAM (Random Access Memory) 304 and executed.
  • the CPU 302 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above.
  • the CPU 302 outputs the processing result from the output unit 306 including an LCD (Liquid Crystal Display), a speaker, or the like, for example, via the input / output interface 310, or from the communication unit 308 as necessary. Transmission and further recording on the hard disk 305 are performed.
  • processing steps for describing a program for causing a computer to perform various types of processing do not necessarily have to be processed in time series according to the order described in the flowchart, but in parallel or individually. This includes processing to be executed (for example, parallel processing or processing by an object).
  • the program may be processed by a single computer, or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.
  • the transmission / reception system of FIG. 1 is a broadcasting system, but the present invention is applicable to transmission / reception of data other than broadcasting.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Social Psychology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Security & Cryptography (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Ecology (AREA)
  • Emergency Management (AREA)
  • Environmental & Geological Engineering (AREA)
  • Environmental Sciences (AREA)
  • Remote Sensing (AREA)
  • Processing Or Creating Images (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
PCT/JP2009/052364 2008-02-14 2009-02-13 放送システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体 Ceased WO2009101998A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US12/866,684 US20100325678A1 (en) 2008-02-14 2009-02-13 Broadcast system, transmission device, transmission method, reception device, reception method, presentation device, presentation method, program, and recording medium
CN2009801072485A CN101960849A (zh) 2008-02-14 2009-02-13 广播系统、发射设备、发射方法、接收设备、接收方法、表现设备、表现方法、程序和存储介质
EP09710743A EP2242270A1 (en) 2008-02-14 2009-02-13 Broadcast system, transmission device, transmission method, reception device, reception method, presentation device, presentation method, program, and recording medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2008032674A JP2009194595A (ja) 2008-02-14 2008-02-14 放送システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体
JP2008-032674 2008-02-14

Publications (1)

Publication Number Publication Date
WO2009101998A1 true WO2009101998A1 (ja) 2009-08-20

Family

ID=40957029

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2009/052364 Ceased WO2009101998A1 (ja) 2008-02-14 2009-02-13 放送システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体

Country Status (5)

Country Link
US (1) US20100325678A1 (https=)
EP (1) EP2242270A1 (https=)
JP (1) JP2009194595A (https=)
CN (1) CN101960849A (https=)
WO (1) WO2009101998A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102307292A (zh) * 2011-09-01 2012-01-04 宇龙计算机通信科技(深圳)有限公司 一种可视通话方法及可视终端
KR20220149654A (ko) * 2020-01-06 2022-11-08 소스픽처, 인코포레이티드 컨텐츠 디스플레이를 위한 재생 장치 적응 시스템 및 방법

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000197030A (ja) * 1998-12-24 2000-07-14 Toshiba Corp 対話型視聴装置および方法および記憶媒体
JP2003060996A (ja) * 2001-08-17 2003-02-28 Hiroyuki Tarumi 放送装置、受信装置、および記録媒体
JP2003333563A (ja) * 2002-05-08 2003-11-21 Nippon Hoso Kyokai <Nhk> コンテンツ提供システム、コンテンツ受信装置、コンテンツ提供方法、コンテンツ受信方法、コンテンツ提供プログラムおよびコンテンツ受信プログラム
JP2005123824A (ja) * 2003-10-15 2005-05-12 Nippon Hoso Kyokai <Nhk> 映像オブジェクト軌跡合成装置、その方法及びそのプログラム
JP2005159592A (ja) * 2003-11-25 2005-06-16 Nippon Hoso Kyokai <Nhk> コンテンツ送信装置およびコンテンツ受信装置
WO2007069189A2 (en) * 2005-12-15 2007-06-21 Koninklijke Philips Electronics N.V. Method and processing unit for processing a video transport stream
JP2007178644A (ja) * 2005-12-27 2007-07-12 Pioneer Electronic Corp 表示装置、表示制御装置、表示方法、表示プログラム、および記録媒体

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9119964D0 (en) * 1991-09-18 1991-10-30 Sarnoff David Res Center Pattern-key video insertion
IL109487A (en) * 1994-04-29 1996-09-12 Orad Hi Tec Systems Ltd Chromakeying system
JP3569992B2 (ja) * 1995-02-17 2004-09-29 株式会社日立製作所 移動体検出・抽出装置、移動体検出・抽出方法及び移動体監視システム
DE19619090A1 (de) * 1996-04-30 1997-11-13 Cfb Gmbh Vorrichtung und Verfahren zur Erzeugung eines Kompositbildes
US5917553A (en) * 1996-10-22 1999-06-29 Fox Sports Productions Inc. Method and apparatus for enhancing the broadcast of a live event
US6100925A (en) * 1996-11-27 2000-08-08 Princeton Video Image, Inc. Image insertion in video streams using a combination of physical sensors and pattern recognition
US6754905B2 (en) * 1998-07-23 2004-06-22 Diva Systems Corporation Data structure and methods for providing an interactive program guide
US6396473B1 (en) * 1999-04-22 2002-05-28 Webtv Networks, Inc. Overlay graphics memory management method and apparatus
JP3802322B2 (ja) * 2000-07-26 2006-07-26 株式会社東芝 動画像内オブジェクト抽出方法及び装置
KR100372092B1 (ko) * 2001-02-06 2003-02-14 주식회사 우리기술 바탕화면 동영상 매체재생기 및 이를 이용한 바탕화면의동영상 처리방법과 상기 매체재생기 또는 방법을실행시키기 위한 프로그램을 기록한 컴퓨터로 읽을 수있는 기록매체
CN1666503A (zh) * 2002-07-01 2005-09-07 皇家飞利浦电子股份有限公司 用于处理视频信号的系统
US7872688B2 (en) * 2005-11-30 2011-01-18 Broadcom Corporation Selectively applying spotlight and other effects using video layering

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000197030A (ja) * 1998-12-24 2000-07-14 Toshiba Corp 対話型視聴装置および方法および記憶媒体
JP2003060996A (ja) * 2001-08-17 2003-02-28 Hiroyuki Tarumi 放送装置、受信装置、および記録媒体
JP2003333563A (ja) * 2002-05-08 2003-11-21 Nippon Hoso Kyokai <Nhk> コンテンツ提供システム、コンテンツ受信装置、コンテンツ提供方法、コンテンツ受信方法、コンテンツ提供プログラムおよびコンテンツ受信プログラム
JP2005123824A (ja) * 2003-10-15 2005-05-12 Nippon Hoso Kyokai <Nhk> 映像オブジェクト軌跡合成装置、その方法及びそのプログラム
JP2005159592A (ja) * 2003-11-25 2005-06-16 Nippon Hoso Kyokai <Nhk> コンテンツ送信装置およびコンテンツ受信装置
WO2007069189A2 (en) * 2005-12-15 2007-06-21 Koninklijke Philips Electronics N.V. Method and processing unit for processing a video transport stream
JP2007178644A (ja) * 2005-12-27 2007-07-12 Pioneer Electronic Corp 表示装置、表示制御装置、表示方法、表示プログラム、および記録媒体

Also Published As

Publication number Publication date
US20100325678A1 (en) 2010-12-23
CN101960849A (zh) 2011-01-26
EP2242270A1 (en) 2010-10-20
JP2009194595A (ja) 2009-08-27

Similar Documents

Publication Publication Date Title
JP2009194597A (ja) 送受信システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体
JP7578755B2 (ja) 複合現実デバイスにおける仮想および実オブジェクトの記録
US11450071B2 (en) Adapting acoustic rendering to image-based object
US11895483B2 (en) Mixed reality spatial audio
US11656682B2 (en) Methods and systems for providing an immersive virtual reality experience
CN107835971B (zh) 基于用户触觉空间（HapSpace）提供触觉反馈和交互的方法和装置
CN103258339A (zh) 基于实况记录和基于计算机图形的媒体流的实时合成
US20140176607A1 (en) Simulation system for mixed reality content
WO2018086224A1 (zh) 一种虚拟现实场景的生成方法、设备及虚拟现实系统
KR20130010424A (ko) 콘텐츠 재생 방법 및 장치
CN103258338A (zh) 利用真实数据来驱动仿真的虚拟环境的方法和系统
CN110058696A (zh) 一种虚拟实景实现方法及其使用方法和相关方法装置
JP7702675B2 (ja) 映像表示システム
US20060114251A1 (en) Methods for simulating movement of a computer user through a remote environment
WO2009101998A1 (ja) 放送システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体
JP2006040053A (ja) 画像処理方法及びプログラム
WO2023276252A1 (ja) 情報処理装置、情報処理方法及びプログラム
JP2009194596A (ja) 送受信システム、送信装置、送信方法、受信装置、受信方法、提示装置、提示方法、プログラム、及び記録媒体
US20240331317A1 (en) Information processing device, information processing system and method
KR20210056414A (ko) 혼합 현실 환경들에서 오디오-가능 접속된 디바이스들을 제어하기 위한 시스템
Dittrich et al. An Immersive and Collaborative Virtual Theater Experiences
EP4554229A1 (en) Production platform
US20240406666A1 (en) Sound field capture with headpose compensation
US20250046000A1 (en) Advanced multimedia system for analysis and accurate emulation of live events
WO2025041486A1 (ja) 表示制御装置、表示制御方法および表示制御プログラム

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200980107248.5

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09710743

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 12866684

Country of ref document: US

Ref document number: 2009710743

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE