JP2010130388A - Imaging apparatus, imaging method, reproduction device, and reproduction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus for recording bodily sensation information felt by a subject together with a video image and sound of a subject, and to provide a reproduction device for reproducing a bodily sensation information felt by the subject together with a video image and sound of the subject. <P>SOLUTION: The imaging apparatus includes a photographing unit for photographing the video image and sound of the subject and the bodily sensation measuring unit for measuring the bodily sensation. The photographing unit includes a bodily sensation stream generation means, and the bodily sensation stream generation means multiplexes the video image and sound stream with the measured bodily sensation data photographed and the bodily sensation data to coincidentally record the bodily sensation and photographed video image by generating a bodily sensation added AV stream. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The technical field relates to an imaging device that captures video and a playback device that reproduces the captured video.

  Conventionally, shooting conditions such as brightness, white balance, shutter speed, and the like are recorded along with the subject at the time of shooting the subject, and can be confirmed at the time of playback.

  Further, in Patent Document 1, “Proposed imaging device, video camera system, and playback device that can easily know environment information at the time of photographing” (see Patent Document 1 [0007]) is a problem to be solved. “Measurement environment measurement means measures a shooting environment at the time of shooting a subject as environment information” (see Patent Document 1 [0008]).

JP-A-10-56608

  In patent document 1, the temperature and humidity at the time of imaging | photography are measured with the temperature sensor and humidity sensor attached to the camera, and it is recorded as environmental information. However, the measured temperature and humidity do not always match the sensation experienced by the subject or the photographer. For this reason, the conventional technique cannot record and reproduce the sensation experienced by the subject or the photographer during photographing.

  In order to solve the above-described problem, an embodiment of the present invention converts light into a video signal, encodes the converted video signal, and multiplexes and records the encoded video signal and bodily experience information. .

  According to the above means, it is possible to provide an imaging device and a playback device that are convenient for the user. Specifically, an imaging device capable of recording the sensation information felt by the subject and the photographer at the time of photographing together with the photographed video, and the sensation felt by the subject and the photographer at the time of photographing can be reproduced together with the photographed video. A playback device can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram illustrating an example of an imaging apparatus according to the present embodiment. Reference numeral 100 denotes an imaging unit, and 110, 120, and 130 denote sensation measurement units. The imaging unit 100 includes a lens 101, an imaging element 102, a microphone 103, a signal processing unit 104, a sensation stream generation unit 105, a recording unit 106, a control unit 107, a communication I / F unit 108, and an output I / F 109.

  The sensation measurement unit (sensation information acquisition means) 110 includes sensors 111, 112, 113, a driver 114, and a communication I / F means 115. The driver 114 includes a sensation measurement unit management unit 117, a device information storage unit 118, and a bus 119. The sensation measurement units 120 and 130 have the same configuration as the sensation measurement unit 110.

  In the image pickup apparatus of FIG. 1, three sensation measurement units are connected to the image pickup unit 100, but this is an example and the number of sensation measurement units is not limited. Further, the sensation measurement unit may be included in the imaging device, but may be connected to the imaging device separately from the imaging device. The imaging unit 100 and the sensation measurement unit can be connected 1 to N via a communication I / F unit. Moreover, although the sensory measurement unit includes three sensors, this is also an example. There is no limit on the number of sensors included in the sensory measurement unit.

  In addition, the communication protocol used by the communication I / F means 108, 115, 125, and 135 is not limited including the physical layer. Either a wired LAN (Local Area Network), a wireless LAN, UWB (Ultra Wide Band), Bluetooth, IrDA (Infrared Data Association) or the like may be used, or an original protocol may be used.

  The signal processing unit 104 (also referred to as an encoding unit) is an AV stream obtained by encoding a video / audio signal from a video signal input from the lens 101, the image sensor 102, and the microphone 103 (a video stream obtained by encoding the video signal). , An encoded audio signal is also referred to as an audio stream) and is input to the sensation stream generation means 105. The bodily sensation stream generation unit 105 multiplexes the AV stream and the bodily sensation data generated by the bodily sensation measurement units 110, 120, and 130 to generate a bodily sensation AV stream 210 and records it in the recording unit 106. A control unit 107 is a control unit that controls the entire imaging apparatus. The control means acquires device information from, for example, a sensation measurement unit, generates a sensation data header and a sensation stream header, and controls the sensation stream generation means 105 to record the sensation information of the subject and the photographer along with the video and audio For example, an experienced AV stream 210 is generated.

  From the output I / F 109, the sensation-added AV stream 210 generated by the sensation stream generation unit 105 and the sensation-added AV stream 210 recorded in the recording unit 106 are output to the outside.

  FIG. 2 shows an example of the configuration of the sensation stream generation unit 105 and an example of the sensation-added AV stream generated by the sensation stream generation unit 105. A method for generating a sensation-added AV stream will be described with reference to FIG.

  The sensation stream generating unit 105 includes a multiplex unit 151 and a sensation information header adding unit 152. The sensation stream generation unit 105 receives the AV stream 201 generated by the signal processing unit 104 and the sensation data 21, 22, and 23 generated by the sensation measurement units 110, 120, and 130. The sensation information header adding unit 152 adds the sensation data headers 26, 27, and 28 to the input sensation data 21, 22, and 23. The bodily sensation data header is generated by using the device information acquired from the bodily sensation measurement units 110, 120, and 130 by the control means 107, and indicates the function of the bodily sensation measurement unit and what the bodily sensation the sensation measurement unit measures Show.

  The sensation data to which the sensation data header is added is multiplexed by the multiplexing means 151. Further, the sensation information header adding unit 152 adds the sensation stream header 25 to generate the sensation stream 211. The sensation stream header 25 is also generated using the device information acquired from the sensation measurement units 110, 120, and 130 by the control unit 107. Then, the multiplexing unit 151 multiplexes the video / audio stream 201 and the sensation stream 211 to generate the sensation-added AV stream 210.

  FIG. 4 is an example of device information of the sensation measurement unit. In this example, the device information structure is described in XML (Extensible Markup Language).

  <sensorUnit> is the root tag of the sensory measurement unit. It has an id attribute that identifies the sensory measurement unit and a version attribute that indicates the version of the sensory measurement unit. The <sensorUnit> tag has <vendor> tag, <unitType> tag, <playerUnitType> tag, <infomationLink> tag, <detailLink> tag, and <detail> tag as elements. The <vendor> tag is a tag indicating the manufacturer of the sensory measurement unit. <unitType> is a tag indicating the type of the sensory measurement unit. <playerUnitType> is a tag indicating the type of a playback unit that reproduces the sensation measured by the sensation measurement unit.

  When there is no tag, or when no value is described, it means that the reproduction means does not exist. If there are multiple playback units, list multiple <playerUnitType> tags. The <infomationLink> tag is a tag indicating a URL (Uniform Resource Locator) from which detailed information of the sensation measurement unit is acquired. Using this information, the user can know the details of the function of the sensation measurement unit and the corresponding status of the playback unit. The <detailLink> tag is a URL indicating the acquisition destination of the device information of the sensation measurement unit. Using this information, the latest device information can be acquired.

  The <detail> tag is a tag that provides sensor information of the sensory measurement unit. The <detail> tag has a <sensors> tag as an element tag. The <sensors> tag has as many <sensor> tags as the number of sensors included in the sensory measurement unit. The <sensor> tag is a tag that defines detailed information of each sensor of the sensory measurement unit, and has an id attribute that identifies the sensor.

  The <sensors> tag has tags indicating the type of sensor, the sensor measurement method, and the format of the measured data as element tags. The <class> tag is a tag that defines the type of sensor. In the figure, smell indicates an odor sensor. The <model> tag is a tag representing the sensor model. <mothod> indicates the sensor measurement method. The <format> tag indicates the format of the sensor measurement output. The <playerType> tag is a tag indicating the means for reproducing the sensation measured by the sensor. If there is no <playerType> tag, or no value is described, it means that there is no real means of reproduction.

  If there are a plurality of playback units, a plurality of tags are listed. <substitute> is a tag indicating a method of an alternative means of experiencing measured by the sensor. It has a <dictionary> tag that indicates conversion dictionary data of alternative expressions as an element. The type attribute of the <dictionary> tag indicates the type of alternative method. For example, text indicates that it is expressed by a word (character), and icon indicates that it is expressed by an icon image.

  FIG. 5 is an example of the sensation data header. 4 includes a tag portion that provides device information details 42, which are detailed sensor information of the device information sensory measurement unit of FIG.

  FIG. 6 is an example of the sensation stream header. The sensation stream header includes sensation measurement unit and sensation person correspondence information 61 and sensation measurement unit list information 62 included in the imaging apparatus 100. <senceInfoRoot> is the root tag of the sensation stream header.

  The correspondence information 61 between the sensation measurement unit and the sensation person lists a <target> tag representing the sensation person. The <target> tag has <sensorUnitId> tag and <image> tag as elements. The <sensorUnitId> tag is an id that uniquely identifies the sensation measurement unit and is acquired from the device information of the sensation measurement unit. The <image> tag is image information of the experienced person. The image information is obtained by a procedure for registering a sensation person (described with reference to FIG. 8) for registering the correspondence between the sensation measurement unit and the sensation person.

  The bodily sensation measurement unit list information 62 is generated by the control unit 107 acquiring device information from all the bodily sensation measurement units included in the imaging apparatus, and enumerating a part (portion 41 in FIG. 4) indicating the outline information of the sensation measurement unit. . <sensorUnit List> is a tag representing a bodily sensation measurement unit list.

  As described above, by defining device information of the sensation measurement unit in a common format and adding it to the sensation-added AV stream 210, it is possible to generate the sensation-added AV stream 210 to which all types of sensation information are added. become. Although the XML format is used in the examples of FIGS. 4, 5, and 6, any format may be used as long as it can express equivalent information.

  The sensation measurement units 110, 120, and 130 measure the sensation of a sensation person wearing the sensation measurement unit with sensors. The bodily sensation is a sensation felt when a bodily sensation performs some action or action. The bodily sensation measurement unit measures the so-called five senses of sight, hearing, touch, smell, and taste and biological information such as body temperature, pulse, respiration, and brain waves as bodily sensation information, and generates bodily sensation data.

  FIG. 3 shows an example of sensation data generated by the sensation measurement unit 110. In the figure, reference numeral 20 denotes sensation information data output from the sensors 111, 112, and 113 of the sensation measurement unit 110 in time series. The sampling period can be set from the driver 114.

  3 in FIG. 3 shows that sensory information data is measured at different periods for each sensor. The sampling period is 1 / T for the sensor 111, 1 / 2T for the sensor 112, and 1 / 3T for the sensor 113. Represents.

Reference numeral 21 in FIG. 3 shows an example of sensation data generated by the driver 114. In order not to lose the information of the sampling time interval, the sensation data 21 to which the time stamp (t0, t1, t2) is added is generated. The sensation data 21 is input to the imaging unit 100 via the communication I / F unit 115. The sensation measurement units 120 and 130 similarly generate sensation data and input them to the imaging unit 100.
FIG. 7 shows an image of using the photographing apparatus of the present invention. 100 is an imaging device, 110, 120 and 130 are sensation measurement units of the imaging device 100, 11 and 12 are subjects, and 13 is a photographer. Reference numerals 71, 72, and 73 are sensation unit identifiers for identifying individual sensation measurement units by overview. The sensation unit identifier is used to register the correspondence between the sensation person and the sensation measurement unit. Use user-friendly identification information such as letters, colors, and marks.

  FIG. 8 shows an example of the procedure for registering a sensation. Hereinafter, the procedure for registering the sensation will be described with reference to FIG.

  S801: When the sensation person registration process is started, the control means 107 displays the sensation unit identifier (75 in FIG. 7) on the viewfinder (74 in FIG. 7) or the like of the imaging apparatus 100, and registers the sensation person to the user. Select the sensory measurement unit. In FIG. 7, the sensation unit identifier is represented by a color, and the identifier color of the sensation measurement unit to be registered is displayed on the viewfinder 74 like a frame (75). By doing in this way, the user can easily recognize the combination of the sensation measurement unit and the sensation person.

  S802: When the user selects the sensation measurement unit, the process proceeds to 803.

  S803: The selection of the sensation person is selected by a method in which the control means recognizes the sensation measurement unit from the image captured at the time of selection and presents it to the user, or the marker 76 is displayed on the viewfinder 74 as shown in FIG. It is also possible to take a picture so that the experienced person can fit in the camera.

  S804: When recognizing the sensible person, the control means 107 changes the marker display as shown in 77 of FIG. If the user confirms the sensation determination, the process proceeds to S805.

  S805: The control means 107 records the image in the marker when the sensation person is determined, adds the sensation person image and the sensation unit information to the sensation stream header (FIG. 6), and ends the sensation person registration procedure.

  Note that the association between the sensation measurement unit and the sensation person is an example, and other methods may be used as long as the sensation measurement unit and the sensation person can be associated with each other.

  As described above, according to the present invention, it is possible to provide an imaging apparatus that records the sensation information felt by the subject and the photographer at the time of photographing together with the photographed video. When data in which sensation information is recorded is reproduced, in addition to the video and audio recorded by the imaging device, the sensation experienced by the subject and the photographer can also be reproduced.

  Next, reproduction of data in which sensation information is recorded will be described. FIG. 10 is a configuration diagram illustrating an example of a playback apparatus according to the present embodiment. In FIG. 10, 800 is a reproduction unit, and 810 and 820 are sensation reproduction units. The reproduction unit 800 includes a separation unit 801, a reproduction unit 802, an input I / F unit 803, a display unit 804, a speaker 805, a control unit 807, and a communication I / F unit 808. The sensation reproduction unit (sensation reproduction means) 810 includes sensation reproduction elements 811, 812, 813, a driver 814, and a communication I / F means 815. The driver 814 includes playback unit management means 817, device information storage means 818, and a bus 819. The sensation reproduction unit 820 has the same configuration as the sensation reproduction unit 810.

  The stream with sensation measurement information 210 input to the separation unit 801 via the input I / F unit 803 is separated into a general video signal and audio signal AV stream and a sensation stream 211.

  The AV stream is input to the reproduction means 802. The reproduction means 802 performs general signal processing for reproduction of an AV stream such as a composite of video and audio, and reproduces a video signal and an audio signal. The video signal reproduced by the reproducing unit 802 is displayed on the display unit 804, and the audio signal is output to the speaker 805.

  The sensation stream is separated into sensation data based on the sensation stream header. The control unit 807 determines a sensation reproduction unit to which sensation data should be transmitted from <playerUnitTyp> information included in the sensation stream header acquired from the separation unit 801 and device information of the sensation reproduction unit. The separating unit 801 outputs the sensation data to the sensation reproduction unit to be transmitted via the communication I / F unit 808 in accordance with an instruction from the control unit.

  The sensation reproduction unit is a means for reproducing the sensation from the sensation data. This is a means of reproducing the sensation experienced by the experienced person at the time of shooting. For example, an odor display that reproduces the odor and a tactile display that reproduces the touched feeling correspond to this.

  Although details of the device information of the sensation reproduction unit are not shown, the device has a configuration corresponding to the device information (FIG. 4) of the sensation measurement unit. Corresponds to the tag <playerUnitType> that indicates the type of the playback unit that reproduces the sensory experience measured by the sensory measurement unit in FIG. Has a <player> tag that provides element information.

  FIG. 13 is an image of using the playback apparatus of the present invention. In the figure, components having the same functions as those in other figures are denoted by the same reference numerals, and description thereof is omitted here. 81 is a remote controller used for the operation, and 82 and 83 are viewers. In this embodiment, the sensation reproduction units 810 and 820 are incorporated into a sofa-like chair.

  Reference numeral 14 denotes a marker for selecting the sensation to be reproduced by the sensation reproduction unit. A plurality of viewers can reproduce the same sensation, or can select and reproduce different sensations. When there is a sensation that can be reproduced or cannot be reproduced by the function provided by the sensation reproduction unit, the control means can indicate the optimum combination of the sensation to be reproduced and the sensation reproduction unit.

  By doing so, it is possible to provide a playback device that reproduces the sensation information felt by the subject and the photographer at the time of shooting together with the shot video.

  Next, another embodiment of the imaging apparatus will be described. FIG. 9 is a configuration diagram illustrating an example of an imaging apparatus according to the present embodiment. In the figure, components that provide the same functions as those of the first embodiment shown in FIG. 9, a network I / F 107 that communicates with an external network, the Internet 90, and a sensation measurement unit device information providing server 91 that provides device information of the sensation measurement unit are added to the configuration shown in FIG.

  The control means 107 may update the device information of the sensation measurement unit to the latest device information from the sensation measurement unit device information providing server 91 via the Internet 90 based on the information of the <infomationLink> tag included in the sensation stream header. it can. Further, when the bodily sensation stream header does not have the device information details 42, it can be acquired based on the information of the <detailLink> tag. The acquired device information and device information details are stored in the storage unit 903. In this embodiment, the communication I / F unit 108 and the network I / F 107 are provided separately, but there is no problem even if the same network I / F unit is used.

  With the above-described configuration, it is possible to provide an imaging apparatus that records the sensation information felt by the subject and the photographer at the time of photographing together with the photographed video. Furthermore, device information can be updated via the Internet. Thereby, for example, information on the sensation measurement unit for which the latest reproduction means did not exist can be acquired. Further, since the device information details 42 can be acquired, there is an advantage that the capacity of the device information storage means (118 in FIG. 1) of the sensation measurement unit can be reduced.

  Next, another embodiment of the playback apparatus will be described. FIG. 11 is a configuration diagram illustrating an example of a playback apparatus according to the present embodiment. In the figure, components that provide the same functions as those of the playback device shown in FIG. 10 are denoted by the same reference numerals and description thereof is omitted. In this embodiment, a description will be given of a playback device that expresses a sensation using alternative means when the sensation reproduction unit that reproduces the sensation associated with a recorded sensation stream is not provided.

  In FIG. 11, reference numeral 808 denotes communication I / F means for communicating with the connected sensation reproduction unit and communication with an external network, 90 is the Internet, 91 is the sensation measurement unit, and device information of the sensation reproduction unit. It is a device information providing server that provides

The control means 107 can update the device information of the sensation measurement unit and the sensation reproduction unit to the latest device information via the Internet 90. Further, the device information details 42 are acquired and stored in the storage unit 903. Reference numeral 902 denotes a switch. The output destination of the sensation data input from the separation unit 801 is switched to the alternative unit 910 or the communication I / F 808 (the sensation reproduction unit side). Switching is controlled by the control means 107. Reference numeral 901 denotes an alternative means for replacing the bodily sensation into a data format that can be output by the playback unit 800, such as characters, icon images, voice and music, and light emitting diode control signals. The control means 107 controls what data format is converted. Further, the control means 107 controls whether to superimpose on the video signal and the audio signal or to substitute with an output means other than the audio video signal such as a light emitting diode. Reference numeral 911 denotes a superimposing means. When the data format of the substitute signal is a video data format such as characters or icon images or a voice data format such as voice or music, the data is input to the superimposing means 911. The substitute signal in the video data format is superimposed on the video signal and output to the display means. The substitute signal in the audio data format is superimposed on the audio signal and output to the speaker.
Reference numeral 912 denotes a light emitting diode (LED). Reference numeral 912 denotes an example of output means other than the audio / video signal. Any output means provided in the reproduction unit 800 may be used, and a means for outputting a beep sound may be used.

FIG. 12 is a flowchart showing an example of processing relating to substitution of the sensation reproduction unit by the control means 107. Hereinafter, the procedure of the sensation substitution process will be described with reference to FIG.
Please replenish.

  S111: The control unit 107 determines whether or not there is a sensation reproduction unit to which sensation data should be sent from the <playerUnitTyp> information included in the sensation stream header acquired from the separation unit 801 and the device information of the sensation reproduction unit. If there is a sensation reproduction unit to which sensation data should be sent, S117. If not, S112.

  S112: If there is no sensation reproduction unit, switch 902 is switched to an alternative means. The process proceeds to S113.

  S113: The control unit 107 refers to the storage unit 903 and determines whether or not alternative information is held. If alternative information corresponding to the bodily sensation stream is held, the process proceeds to S115.

  S114: If there is no substitute information, it is acquired from the <playerUnitTyp> information included in the sensation stream header via the Internet 90 and stored in the storage means 903.

  S115: The control means 107 refers to the substitute information corresponding to the bodily sensation stream, and generates an alternative signal obtained by converting the bodily sensation data into a data format that can be output by the playback unit 800. When a plurality of alternative methods that can be output by the playback unit 800 are provided, the alternative information to be used is determined according to the priority order set in advance by the user or the system.

  S116: When the control unit 107 instructs the substitute unit 910 to generate a substitute signal in the video data format or the audio data format, the control unit 107 causes the superimposing unit 911 to output the substitute signal. In other cases, the output means corresponding to the substitute signal is output. For example, when an instruction is given to generate a substitute signal for the control signal of the light emitting diode, the light emitting diode (LED) 912 is output.

  S117: The switch 902 is switched to the communication I / F 808 (sensory reproduction unit), and the process is terminated.

  As shown in the flowchart of the alternative process in FIG. 12, the control means 107 determines whether or not it has a sensation reproduction unit that can reproduce the sensation of the sensation data 210. If not, the control means 107 switches the switching means 902 and uses the alternative means. Express the experience.

  FIG. 14 is an example of the reproduction of the bodily sensation by the alternative means. As shown in FIG. 14, for example, when there is no means for reproducing an odor, the sensation can be expressed by superimposing and displaying the text indicating the odor on the video information. In addition to substitution by characters, substitution by what can be superimposed on video information such as an icon image, and substitution by superimposing voice and sound effects indicating bodily sensation on voice information are also possible. It is also possible to convert the sensation into a control signal of the light emitting diode and substitute it with the luminescent color. For example, the odor can be replaced by converting the odor into a light emission color such as blue for the tide, green for the forest and grass, and pink for the flower.

  As described above, according to the present embodiment, it is possible to provide a reproducing apparatus that reproduces the sensation information felt by the subject and the photographer at the time of photographing together with the photographed video. Furthermore, it is possible to provide a playback device that can express the sensation using alternative means even when the sensation reproduction unit that reproduces the sensation is not provided.

  FIG. 15 is a configuration diagram showing an example of a photographing / reproducing apparatus having the functions of both the photographing apparatus and the reproducing apparatus. With such a configuration, it is possible to provide a photographing / reproducing apparatus that records and reproduces the sensation information felt by the subject and the photographer at the time of photographing together with the photographed video.

1 is a configuration diagram illustrating an example of an imaging apparatus according to a first embodiment. An example of an experienced AV stream Example of experience data generated by the experience measurement unit An example of device information of the sensory measurement unit Example of experience data header Example of experience stream header Example of imaging device usage image in the embodiment An example of the procedure for experiencing registration 1 is a configuration diagram illustrating an example of an imaging apparatus according to a second embodiment. 1 is a configuration diagram illustrating an example of a playback apparatus according to a first embodiment. Configuration diagram showing an example of a playback apparatus according to a third embodiment The flowchart which shows an example of the alternative process by a control means Example of playback device usage image in the embodiment An example of experience replay by alternative means Configuration diagram of photographing and reproducing apparatus according to embodiment 4

Explanation of symbols

11, 12 Subject 13 Photographer 14 Playback sensation selection marker 41 sensation unit information 42 Device information details 61 sensation measurement unit and correspondence information 62 sensation measurement unit list information 81 Remote control 82, 83 Viewers 71, 72, 73 sensation unit Identifier 74 Viewfinder 75 Display of sensory unit identifier 90 Internet 91 Device information providing server 100 Imaging unit 101 Lens 102 Image sensor 103 Microphone 104 Signal processing unit 105 Sensory stream generation unit 106 Recording unit 107 Control unit 108, 115, 125 Communication I / F means 109 Output I / F
110, 120, 130 Sensory measurement unit (sensory information acquisition means)
111, 112, 113, 121, 122, 123 Sensor 114, 124 Driver 117 Sensation measurement unit management means 118 Device information storage means 119 Bus 151 Sensation stream generation means 152 Sensation information header addition means
20, 21, 22 Sensory data 25 Sensory stream header 26, 27, 28 Sensory data header 201 Video / audio stream 202, 203 Sensory measurement information 210 Sensory-added AV stream 211 Sensory stream 2021, 2022, 2031, 2032 Sensory measurement data 800 Playback unit 801 Separating means 802 Reproducing means 803 Input I / F means 804 Display means 805 Speaker 807 Control means 808, 815, 825 Communication I / F means 810, 820 Sensation reproduction unit (sensation reproduction means)
814, 820 Drivers 811, 812, 813, 821, 822, 823 Sensation reproduction element 817 Playback unit management means 818 Device information storage means 819 Bus 901 Network I / F means 902 Switch 903 Storage means 910 Alternative means 911 Superimposition means 912 Light emitting means (LED)

Claims (17)

An image sensor that converts light into a video signal;
An encoding unit that encodes the video signal converted by the imaging device;
A sensation information acquisition unit for acquiring sensation information;
A multiplexing unit that multiplexes the video signal encoded by the encoding unit and the sensation information acquired by the sensation information acquisition unit;
An imaging apparatus comprising: a recording unit that records an output from the multiplexing unit. The imaging apparatus according to claim 1,
The multiplexing device generates an sensation stream in which a sensation information header indicating the function of the sensation information acquisition unit is added to the sensation information, and multiplexes the sensation stream and the encoded video signal. The imaging apparatus according to claim 2,
The bodily sensation information acquisition unit is an imaging device including a sensory sensor that quantitatively measures a sense of ecology. It is an imaging device of Claims 1-3,
An imaging apparatus that records the sensory information and a subject that is a part of the video signal in the recording unit in association with each other. An input unit to which a signal in which an encoded video signal and bodily sensation experience information are multiplexed is input;
A separation unit that separates the signal input to the input unit into an encoded video signal and sensation information;
A reproduction unit for reproducing the encoded video signal separated by the separation unit;
A reproduction apparatus including a sensation reproduction unit that reproduces the sensation indicated by the sensation information separated by the separation unit. An input unit to which a signal in which an encoded video signal and bodily sensation experience information are multiplexed is input;
A separation unit that separates the signal input to the input unit into an encoded video signal and sensation information;
A reproduction unit for reproducing the encoded video signal separated by the separation unit;
A reproducing apparatus comprising: an alternative unit that converts the bodily sensation indicated by the bodily sensation information separated by the separation unit into an alternative signal in a data format that can be output by the reproducing apparatus. An input unit to which a signal in which an encoded video signal and bodily sensation experience information are multiplexed is input;
A separation unit that separates the signal input to the input unit into an encoded video signal and sensation information;
A reproduction unit for reproducing the encoded video signal separated by the separation unit;
A sensation reproduction unit that reproduces the sensation shown by the sensation information separated by the separation unit;
A reproducing apparatus comprising: an alternative unit that converts the bodily sensation indicated by the bodily sensation information separated by the separation unit into an alternative signal in a data format that can be output by the reproducing apparatus. The playback device according to claim 5 or 7,
The bodily sensation information includes a bodily sensation information header indicating the function of the device that has obtained the bodily sensation information,
And a control unit that determines a sensation reproduction unit that reproduces the sensation indicated by the sensation information based on the sensation information header included in the sensation information. The playback device according to claim 6 or 7, wherein
A reproduction apparatus comprising: a superimposing unit that superimposes the substitute signal converted by the substitute unit on the encoded video signal. The playback device according to claim 6 or 7, wherein
When there are a plurality of alternative methods that can be output by the playback device, the playback device includes a control unit that determines a data format of an alternative signal generated with a priority set in advance by a user or a system. Convert light into video signal,
Encode the converted video signal,
An imaging method for multiplexing and recording the encoded video signal and sensation information relating to sensation. The imaging method according to claim 11, comprising:
The multiplexing of the encoded video signal and the sensation information related to the sensation generates a sensation stream in which a sensation information header indicating the function of the sensation information acquisition unit for obtaining the sensation information is added to the sensation information, An imaging method performed by multiplexing an encoded video signal. The imaging method according to claim 12, comprising:
The bodily sensation information acquisition means includes an sensation sensor that quantitatively measures an ecological sensation. It is an imaging method of Claims 11-13,
An imaging method for recording the sensation information and a subject that is a part of the video signal in association with each other. Receive a signal in which the encoded video signal and the sensation information about the sensation are multiplexed,
Separating the received signal into an encoded video signal and experience information;
Replaying the separated encoded video signal;
A reproduction method for reproducing a bodily sensation indicated by the separated bodily sensation information. The reproduction method according to claim 15, wherein
The bodily sensation information includes a bodily sensation information header indicating the function of the device that has obtained the bodily sensation information,
A reproduction method for determining a method for reproducing the sensation indicated by the sensation information based on the sensation information header included in the sensation information. The reproduction method according to claim 15 or 16, comprising:
A reproduction method of superimposing information indicating the sensation information on video information when the separated sensation information cannot be reproduced.

Images