JP2004180290A - Apparatus and method of transmission, apparatus and method of reproduction, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily reproduce a content generated with a changed frame rate at a variable speed. <P>SOLUTION: Transmission data DTz constituted by concatenating associated information including frame rate information and frame identification information of a frame contained in a reference frame period to main data indicating video and/or audio is generated and outputted from a transmission device 50. When a content reproducing apparatus 70 reproduces the main data by using this data DTz, a reproduction speed variable range is set on the basis of the frame rate information. According to the reproduction speed indicated within the reproduction speed variable range, thinning and repetition of the video and audio data are performed by utilizing the frame identification information, and reproduction speed of the main data is easily varied to generate a video signal Svz or an audio signal Saz. This signal Svz or Saz is supplied to a content presentation apparatus 80 to perform content presentation at a desired reproduction speed. <P>COPYRIGHT: (C)2004,JPO

Description

  The present invention relates to a transmission device, a transmission method, a reproduction device, a reproduction method, a program, and a recording medium. More specifically, transmission data is generated by linking auxiliary information including frame rate information of the main data to main data indicating an image and / or sound, and the transmission data is output via a transmission path. Further, based on the frame rate information, a reproduction variable range indicating a frame rate range of the main data to be reproduced is set, and the main data is reproduced at a frame rate within the reproduction variable range.

  2. Description of the Related Art Conventionally, in the generation of image and / or audio contents used for broadcasting, in order to produce an effect intended by a creator, it is often the case that contents whose movement speed of a subject is partially changed are created.

  In the generation of the content in which the movement speed of the subject is changed, for example, the content is generated by setting the frame rate higher than the reference frame rate, and the content is reproduced at the reference frame rate, so that the movement of the subject is slowed down. Generate the expressed content. In addition, content is generated by setting the frame rate lower than the reference frame rate, and the content is reproduced at the reference frame rate, thereby generating content that expresses the movement of the subject faster. Further, by adjusting the set frame rate and the frame rate at the time of reproduction, the moving speed of the subject can be freely varied.

  In this way, the creator can use the content generated at the reference frame rate as well as the content that changes the movement speed of the subject so that the intended effect can be obtained when played back at the reference frame rate. Generate content.

  A video camera capable of expanding and compressing the time axis so as to generate content with a variable frame rate as described above is disclosed in, for example, Patent Document 1.

JP-A-11-177930

  By the way, as a communication network has a wider bandwidth and a lower price, interactive transmission of contents via the communication network has been put to practical use. In the transmission of content via a communication network, the transmitted content is temporarily stored in a buffer and then reproduced, thereby absorbing fluctuations (variation in data arrival) occurring in the communication network and continuously reproducing the content. It has been made possible. In addition, it has become possible to transmit more data due to the broadening of communication networks.

  However, even in the transmission of the interactive content, the content to be transmitted is a content generated so that the intended effect can be obtained when the content is reproduced at the reference frame rate, as in the case of the broadcast. For this reason, even if an attempt is made to reproduce a content portion generated with a variable frame rate at a desired speed different from the speed intended by the creator, this portion cannot be reproduced at a desired speed.

  Therefore, according to the present invention, a transmission device, a transmission method, a reproduction device, a reproduction method, and a program for enabling a content portion generated by changing a frame rate to be easily reproduced at a speed different from a speed intended by a creator. And a recording medium.

  A transmission apparatus according to the present invention includes a transmission data generation unit that generates transmission data by connecting ancillary information including frame rate information of the main data to main data indicating an image and / or a sound, and a transmission path. Transmission processing means for performing a transmission data output process via the transmission processing means.

  A transmission device according to the present invention includes a transmission data generation step of generating transmission data by linking ancillary information including frame rate information of the main data to main data indicating an image and / or a sound, and And a transmission processing step of performing an output processing of transmission data via the transmission processing step.

  A reproducing apparatus according to the present invention reproduces a main data to be reproduced based on frame rate information of transmission data formed by connecting main data indicating an image and / or audio to ancillary information including frame rate information of the main data. It has a setting means for setting a reproduction variable range indicating a range of a data frame rate, and a reproduction means for reproducing main data at a frame rate within the reproduction variable range.

  According to the reproducing method of the present invention, the main data to be reproduced is formed based on the frame rate information of the transmission data which is formed by connecting the main data indicating the image and / or the sound to the additional information including the frame rate information of the main data. The method includes a setting step of setting a variable reproduction range indicating a range of a frame rate of data, and a reproducing step of reproducing main data at a frame rate within the variable reproduction range.

  A program according to the present invention includes a transmission data generating step of generating transmission data by connecting ancillary information including frame rate information of the main data to main data indicating an image and / or sound, and And a transmission processing step of performing transmission data output processing. Further, based on the frame rate information of the transmission data, which is formed by connecting the main data indicating the image and / or the sound to the auxiliary information including the frame rate information of the main data, the range of the frame rate of the main data to be reproduced is determined. And a playback step of playing back the main data at a frame rate within the playback variable range.

  A recording medium according to the present invention is a medium in which main data indicating an image and / or sound is linked to ancillary information including frame rate information of the main data.

  According to the present invention, the auxiliary data including the frame rate information of the main data is linked to the main data indicating the image and / or the sound and output as transmission data. Here, the main data is temporarily stored, for example, and the stored main data is read out according to the band of the transmission path, and the frame rate of the main data is adjusted. Accordingly, the frame rate information included in the attached information is corrected and connected. This additional information includes information indicating the recommended reproduction speed of the main data and information indicating the maximum reproducible speed of the main data. In addition, at least the frame rate information and the frame identification information of each frame included in the reference frame period are linked to the main data as auxiliary information, and the frame identification information is used in accordance with the notified band using the frame identification information. By controlling the reading of the main data, the frame rate of the main data is adjusted.

  When the main data is reproduced using the transmission data in which the auxiliary information is linked to the main data, a reproduction variable range indicating a range of the frame rate of the main data to be reproduced is set based on the frame rate information. The main data is reproduced at a frame rate within the reproduction variable range. Further, when the attached information includes information indicating the recommended reproduction speed of the main data, and the user does not give an instruction of the reproduction speed, the main data is reproduced at the recommended reproduction speed. Also, when the attached information includes information indicating the maximum reproducible speed of the main data, the reproduction speed variable range is set using the information indicating the maximum speed. In addition, the accessory information includes frame identification information of each frame included in the reference frame period, and the reproduction speed of the main data is changed by performing thinning and repetition of the main data using the frame identification information. .

  According to the present invention, the auxiliary data including the frame rate information of the main data is connected to the main data indicating the image and / or the sound and output as transmission data. When the main data is reproduced using the transmission data, a reproduction variable range indicating the range of the frame rate of the main data to be reproduced is set based on the frame rate information included in the accessory information. The main data is reproduced at a frame rate within the reproduction variable range. For this reason, by connecting the additional information to a portion that can be reproduced in a speed range different from the speed intended by the content creator, this portion is reproduced at a speed different from the speed intended by the content creator. be able to.

  Further, the main data is temporarily stored, and the stored main data is read out in accordance with the band of the transmission path, thereby adjusting the frame rate of the main data. Therefore, the frame rate of the main data can be easily adjusted. In addition, it is possible to prevent interruption of reproduction of images and sounds. Further, the frame rate information included in the additional information is corrected according to the adjustment of the frame rate, so that the additional information corresponding to the main data to be transmitted can be connected.

  In addition, the main data is reproduced at the recommended reproduction speed if the user does not instruct the reproduction speed at the time of reproducing the main data, including the information indicating the recommended reproduction speed of the main data in the attached information. Creators can specify the playback speed.

  In addition, when the main data is reproduced, the reproduction speed variable range is set using the information indicating the maximum speed at which the main data can be reproduced. Can regulate the reproduction speed variable range.

  Further, at least the frame rate information and the frame identification information of each frame included in the reference frame period are linked to the main data as auxiliary information, and the frame rate of the main data is adjusted using this frame identification information. The adjusted transmission data is generated. Further, when reproducing the main data using the transmission data, the main data is thinned or repeated using the frame identification information, and the reproduction speed of the main data is varied. Therefore, the main data can be reproduced at a desired speed with a simple configuration.

  Hereinafter, an embodiment of the present invention will be described. FIG. 1 shows an overall configuration of a content transmission system that transmits content, for example, image and / or audio content. The imaging device 10 generates video data with a variable frame rate, connects the attached information related to the video data, and supplies the resulting data to the editing device 30 as material data DTm. Further, when the audio input device 20 is provided in the imaging device 10, the audio data is generated, and this audio data is also supplied to the editing device 30 as the material data DTm. The material data DTm may be supplied not only from the imaging device 10 but also from other devices.

  The editing device 30 performs an editing process using the supplied material data DTm, and generates data indicating an image and / or a sound desired by the editor. In addition, the data indicating the image and / or the sound is used as main data, and the main data is linked with the accessory information to generate content data DC for transmission and supply the content data DC to the content transmission device 50.

  The editing device 30 generates a video signal Svm related to editing and supplies the video signal Svm to the edited image display device 40. Thereby, the user can confirm the editing progress and the editing result of the image on the display image on the edited image display device 40. Similarly, an audio signal Sam relating to editing is generated and supplied to the edited audio output device 41. Thereby, the user can confirm the editing progress of the audio, the editing result, and the like by the audio output from the editing audio output device 41.

  The content transmission device 50 stores the content data DC supplied from the editing device 30. Further, for example, when a request for content data is made from the content reproduction device 70, the frame rate of the content data is adjusted according to the band of the transmission path 60, and the transmission data DTc is generated based on the content data after the frame rate adjustment. The transmission data DTc is supplied to the content reproduction device 70 via a wired or wireless transmission path 60.

  The content reproduction device 70 generates a video signal Svz or an audio signal Saz of the content based on the transmission data DTc supplied via the transmission path, and supplies the content to the content presentation device 80. In addition, the content reproduction device 70 controls a content reproduction operation based on the attached information.

  The content presentation device 80 presents content by performing image display based on the video signal Svz and audio output based on the audio signal Saz.

  Here, the connection may be in a state where the main data and the additional information including the frame rate information regarding the main data are linked to each other. For example, even if the main data and the auxiliary information are transmitted as transmission data on different transmission paths, if the frame numbers corresponding to the auxiliary information are included, they can be associated with each other later. it can. In the present embodiment, it is said that the connection is made including such a case.

  FIG. 2 shows a configuration example of the imaging device 10. The light incident through the imaging lens system 11 is incident on the imaging unit 12, and a subject image is formed on an imaging surface of an imaging element such as a CCD (Charge Coupled Device) provided in the imaging unit 12. . The imaging element generates an imaging charge of a subject image by photoelectric conversion. Further, the imaging unit 12 reads out the generated imaging charge based on a driving signal CR from a timing generator 142 to be described later, generates an imaging signal Sp having a frame rate corresponding to the driving signal CR, and It is supplied to the processing circuit 131.

  The camera processing circuit 131 performs various signal processing at a timing synchronized with the imaging signal Sp based on the timing signal CT supplied from the timing generator 142. For example, the camera processing circuit 131 performs a process of removing noise components from the imaging signal Sp by performing correlated double sampling or the like, a process of converting the imaging signal Sp from which noise has been removed into digital video data, and a clamping process of video data. , Shading correction, image sensor defect correction, γ processing, contour compensation processing, knee correction processing, and the like. Further, the camera processing circuit 131 performs various signal processing under processing conditions and the like based on the operation control signal CS supplied from the imaging control circuit 141 of the control unit 14. As described above, the video data DV obtained by performing various kinds of signal processing by the camera processing circuit 131 is supplied to the output unit 15.

  The timing generator 142 of the control unit 14 generates a drive signal CR according to the operation control signal CS from the imaging control circuit 141 and supplies the driving signal CR to the imaging unit 12, thereby changing the readout cycle of the imaging charge in the imaging unit 12. Then, the frame rate of the imaging signal Sp is controlled to the set frame rate FRs based on the operation signal PSa from the user interface unit 16. The timing generator 142 uses, for example, the frame frequency of 59.94 Hz or 29.97 Hz in the NTSC system and the frame frequency of 50 Hz or 25 Hz in the PAL system as the frame frequency of the reference frame rate FRr, and sets the set frame rate FRs to k (k) of the reference frame rate FRr. When an operation of multiplying by a positive value that is not limited to an integer) is performed, control is performed such that the frame rate of the imaging signal Sp becomes k times the reference frame rate FRr. Note that, for example, by changing the period of a read pulse (sensor gate pulse) for moving the imaging charge accumulated in each pixel from each pixel of the imaging device such as a CCD to the transfer unit, the imaging charge is read. The cycle is varied, and the frame rate is varied. At this time, a CDR (Common Data Rate) method may be adopted. When the CDR method is used, the effective frame rate is variable, but the frame rate of the signal output from the CCD can be unchanged, and the processing rate of the camera processing circuit 131 and the like can be constant. This CDR system is disclosed in PCT application, application number PCT / JP03 / 00551, 2003/1/22 application.

  Further, the timing generator 142 generates a timing signal CT synchronized with the drive signal CR and supplies the timing signal CT to the camera processing circuit 131 and the audio processing circuit 132. Further, the timing generator 142 generates frame rate information DM-FRs indicating a set frame rate FRs which is a frame rate of the video data DV, and supplies the frame rate information DM-FRs to the output unit 15. The timing generator 142 also generates a subframe number BN. The sub-frame number BN is a number that enables each frame included in the frame period of the reference frame rate FRr to be identified when the set frame rate FRs is set higher than the reference frame rate FRr. This subframe number BN is supplied to the output unit 15 as frame identification information DM-BN.

  FIG. 3 is a flowchart showing an operation of adding a subframe number in the timing generator 142. The timing generator 142, for example, divides an oscillation signal of a predetermined frequency, sets the frame period of the reference frame rate FRr and the frame period of the set frame rate FRs in synchronization, and drives based on the frame period of the set frame rate FRs. The signal CR and the frame reference timing indicating the break of the frame period of the reference frame rate FRr are generated.

  The timing generator 142 identifies whether or not the frame reference timing has been detected in step ST1. Here, when the reference timing is detected, the process proceeds to step ST2. If the reference timing has not been detected, the process returns to step ST1.

  When the frame reference timing is detected in step ST1 and the process proceeds to step ST2, the timing generator 142 initializes the subframe number BN in step ST2, sets the subframe number BN to an initial value, for example, “0”, and proceeds to step ST2. Proceed to ST3.

  In step ST3, the timing generator 142 determines whether or not the frame reference timing has been detected from the detection of the frame reference timing to the lapse of one frame period of the set frame rate FRs. Here, when the frame reference timing has not been detected, the process proceeds to step ST4, where the timing generator 142 updates the subframe number BN by adding “1” to the subframe number BN, and returns to step ST3. As described above, when the frame reference timing is not detected before the elapse of one frame period of the set frame rate FRs, the subframe number BN is sequentially assigned for each one frame period of the set frame rate FRs.

  Thereafter, when the frame reference timing is detected before the elapse of one frame period of the set frame rate FRs, the process returns to step ST2, and the subframe number BN is initialized.

  Therefore, for each frame period of the reference frame rate FRr, the sub-frame number BN can be added to the frame image of the set frame rate FRs provided during this frame period.

  The user interface unit 16 is connected to the imaging control circuit 141 of the control unit 14 shown in FIG. When an operation switching operation or a variable frame rate operation is performed on the imaging device 10, the user interface unit 16 generates an operation signal PSa corresponding to the operation and supplies the operation signal PSa to the imaging control circuit 141. When the operation signal PSa is supplied from an external device (not shown) such as a remote controller, the user interface unit 16 supplies the operation signal PSa to the imaging control circuit 141.

  The imaging control circuit 141 generates an operation control signal CS based on the operation signal PSa from the user interface unit 16 so that the operation of the imaging device 10 becomes an operation corresponding to the operation signal PSa, Supply to generator 142.

  The audio processing circuit 132 is supplied with an analog audio signal Sin from the audio input device 20. The audio processing circuit 132 performs sampling processing of the audio signal Sin based on the timing signal CT supplied from the timing generator 142, generates digital audio data DA, and supplies the digital audio data DA to the output unit 15.

  The output unit 15 generates the additional information DM including the frame rate information DM-FRs and the frame identification information DM-BN, and connects the video data DV and the audio data DA to generate the material data DTm and supplies it to the editing device. I do. If the material data DTm or the recording signal generated based on the material data DTm is recorded on a recording medium, the editing device can use the editing device to record the material data DTm or the recording signal generated based on the material data DTm. By reproducing, the material data DTm can be supplied to the editing device via the recording medium. Further, the additional information DM may include not only the information of the set frame rate FRs and the sub-frame number BN but also information indicating an imaging date and time, an imaging condition, an imaging content, and the like.

  Here, as an example of a method of connecting the additional information DM to the video data DV and the audio data DA, when the video data DV and the audio data DA are compressed to generate the material data DTm as a data stream, the video data DV The additional information DM may be inserted into the header of the data stream, or the additional information DM may be inserted into the header of the data stream.

  In addition, in order to transmit uncompressed video data and audio data, it is standardized as SMPTE (Society of Motion Picture and Television Engineers) 259M “Television-10-Bit 4: 2: 2 Component and 4fsc Composite Digital Signals-Serial Digital Interface”. The SDTI format and the SDTI format standardized as SMPTE305M "Television-Serial Data Transport Interface (SDTI)" for transmitting compressed video data and audio data are further limited. When the SDTI-CP format standardized as SMPTE326M “Television-SDTI Content Package Format (SDTI-CP)” is used, the auxiliary information DM is a UMID standardized as SMPTE330M “Television-Unique Material Identifier (UMID)”. As data into the signal of each format. . Note that the method of connecting the additional information DM to the video data DV and the audio data DA is not limited to this, and various methods are conceivable, and are not limited to the above examples. In addition, the connection may be one in which the mutual relationship can be understood by some method, that is, a link can be obtained. For example, even in the case of being sent via different transmission paths, if they are assigned the same UMID, they can be associated, and this is also included in the connection.

  By the way, the above-described imaging device 10 generates material data DTm of a desired set frame rate FRs by changing the readout cycle of the imaging charge in the imaging unit 12, and continuously changes the set frame rate FRs. it can. However, when it is only necessary to change the set frame rate FRs in a stepwise manner, the frame data is thinned out, so that the material data DTm of the desired set frame rate FRs can be generated. That is, video data DVa having a higher frame rate than the set frame rate FRs and having a constant frame rate is generated, and video data is extracted from the video data DVa by the set frame rate FRs, whereby the set frame rate is obtained. Video data DV of FRs can be generated. FIG. 4 shows the configuration in this case. In FIG. 4, parts corresponding to those in FIG. 2 are denoted by the same reference numerals, and detailed description is omitted.

  The timing generator 182 of the control unit 18 generates a drive signal CRa corresponding to the maximum value of the set frame rate FRs set via the user interface unit 16 and supplies the drive signal CRa to the imaging unit 12. The imaging unit 12 generates an imaging signal based on the drive signal CRa, generates an imaging signal Spa having a fixed frame rate FRq whose frame rate is higher than the reference frame rate FRr, and sends the imaging signal Spa to the camera processing circuit 131 of the signal processing unit 17. Supply. For example, when the set frame rate FRs can be changed up to n times (n is a positive number) times the reference frame rate FRr, the imaging unit 12 generates the imaging signal Spa having a frame rate n times the reference frame rate FRr. And supplies it to the camera processing circuit 131. That is, the imaging unit 12 generates the imaging signal Spa having a fixed frame rate without being affected by the set frame rate FRs set via the user interface unit 16.

  Further, the timing generator 182 generates a timing signal CTa synchronized with the drive signal CRa and supplies the timing signal CTa to the camera processing circuit 131, the audio processing circuit 132, and the effective frame signal generation circuit 183 of the signal processing unit 17.

  The camera processing circuit 131 supplies the effective data selection circuit 171 with the video data DVa having the fixed frame rate FRq generated based on the imaging signal Spa. The audio processing circuit 132 supplies the valid data selection circuit 171 with audio data DAa generated by performing sampling based on the constant frequency timing signal CTa.

  The imaging control circuit 181 generates a setting information signal CF indicating the setting frame rate FRs based on the operation signal PSa from the user interface unit 16 and supplies the setting information signal CF to the effective frame signal generation circuit 183.

  The effective frame signal generation circuit 183 extracts data from the video data DVa on a frame basis based on the ratio between the frame rate FRq, which is a constant value of the video data DVa, and the set frame rate FRs indicated by the setting information signal CF. An extraction control signal CC for generating video data DV of the set frame rate FRs is generated. Further, the valid frame signal generation circuit 183 supplies the extraction control signal CC to the valid data selection circuit 171 in synchronization with the timing signal CTa. For example, when the frame rate FRq of the video data DVa is n times the reference frame rate FRr and the set frame rate FRs is (n / 2) times the reference frame rate FRr, the valid frame signal generation circuit 183 outputs An extraction control signal CC for extracting data every other frame from DVa in frame units is generated and supplied to the valid data selection circuit 171 in synchronization with the timing signal CTa.

  Further, the valid frame signal generation circuit 183 generates frame rate information DM-FRs indicating the set frame rate FRs based on the setting information signal CF, and supplies the frame rate information DM-FRs to the output unit 15. Further, since the number of frames during the frame period of the reference frame rate FRr can be identified by the extraction control signal CC, the valid frame signal generation circuit 183 sets the subframe number BN for the frame during each frame period of the reference frame rate FRr. Then, the subframe number BN is also supplied to the output unit 15 as frame identification information DM-BN.

  The valid data selection circuit 171 extracts the video data DVa and the audio data DAa of the frame indicated by the extraction control signal CC and supplies them to the output unit 15 as the video data DV and the audio data DA. Although not shown, the effective frame signal generation circuit 183 supplies the effective data selection circuit 171 with frame rate information DM-FR indicating the set frame rate FRs. The thinning of the audio data DAa may be performed in accordance with the ratio to the frame rate when the audio data DAa is generated. For example, when the frame rate FRq when the audio data DAa is generated is n times the reference frame rate FRr and the set frame rate FRs is (n / 2) times the reference frame rate FRr, the audio data DAa Thinning out every other sample. In this case, since the thinning interval can be made smaller than in the case where the audio data is thinned in frame units, the sound based on the audio data DA can have good sound quality.

  As described above, by making the frame frequency of the video data DVa constant, it is not necessary to change the operation frequency of the imaging unit 12 or the signal processing unit 17 in the camera processing circuit 131. The configuration can be simplified. Further, since the video data DV of the set frame rate FRs can be generated simply by performing data extraction on a frame basis from the video data DVa, the video data DV of the desired set frame rate FRs can be easily generated from the video data DVa.

  Further, the image data, the adder, and the divider may be provided in the imaging apparatus, and the video data may be added for each predetermined frame to generate the video data DV. In this case, the variable range of the frame rate of the imaging signal Sp can be narrowed. That is, if the image signal Sp for n frames is added and the signal level is multiplied by (1 / n), the frame rate of the image signal Sp can be increased to (1 / n) without increasing the frame rate of (1 / n). A doubled signal can be obtained.

  FIG. 5 and FIG. 6 are diagrams for explaining the relationship between the video data DV generated by the imaging devices 10 and 10a and the attached information DM. Assuming that the set frame rate FRs is, for example, one or two times the reference frame rate FRr as shown in FIG. 5A, the video data DV shown in FIG. 5B (a frame image based on the video data DV is shown in the figure) is shown in FIG. 5C indicating the set frame rate FRs and the associated information DM including the frame identification information DM-BN of FIG. 5D indicating the subframe number BN are connected. FIG. 5E shows the relationship between the passage of time and the frame image. Further, the frame rate information DM-FRs may indicate not only the set frame rate FRs but also a magnification of the set frame rate FRs with respect to the reference frame rate FRr. The magnification is shown in the frame rate information DM-FRs shown in FIG. 5C and the following figures.

  As shown in FIG. 6A, when the set frame rate FRs is, for example, 1 or 1/2 times the reference frame rate FRr, video data DV shown in FIG. 6B (a frame image based on the video data DV is shown in the figure). 6C is connected to the frame rate information DM-FRs of FIG. 6C indicating the set frame rate FRs and the frame identification information DM-BN of FIG. 6D indicating the subframe number BN. FIG. 6E shows the relationship between the passage of time and the frame image.

  Next, the editing device 30 will be described. FIG. 7 shows the configuration of the editing device 30. The material data DTm supplied to the editing device 30 is supplied to an information detection circuit 311 of the material capturing unit 31. The information detection circuit 311 detects the attached information DM from the material data DTm. The detected attached information DM is supplied to the database processing circuit 312. Further, the video data DV and the audio data DA included in the material data DTm are supplied to the database processing circuit 312.

  The database processing circuit 312 stores the video data DV, the audio data DA, and the attached information DM detected by the information detection circuit 311 in the data storage device 321 of the editing processing unit 32 in association with each other. Further, the database processing circuit 312 easily confirms the content of the material data based on the additional information DM stored in the data storage device 321 and the video data DV and the audio data DA related to the additional information DM. A possible database information DB is generated and supplied to the editing control unit 33. For example, the database information DB includes information (for example, thumbnails) for identifying the content of the material data, the time length of the material data, the set frame rate FRs, the subframe number BN, the storage position in the data storage device 321, and the like. Is what is done.

  The editing control unit 33 generates video data DVg for enabling editing in a GUI (Graphical User Interface) environment and video data DVi for displaying the contents of database information, and sends the video data DVi to the video output signal generation circuit 351. Supply. The video output signal generation circuit 351 generates a video signal Svm based on the supplied video data DVg and DVi, and outputs the video signal Svm to the edited image display device 40. In this manner, by supplying the video signal Svm to the edited image display device 40, what kind of material data is stored can be displayed on the screen of the edited image display device 40.

  The editing control unit 33 controls post-production processing. That is, the operation signal PSe using the display in the GUI environment is supplied from the user interface unit 34 connected to the editing control unit 33, and it is indicated that any material data is selected by the operation signal PSe. In some cases, the edit control unit 33 generates a read control signal RC corresponding to the operation signal PSe and supplies the read control signal RC to the write / read processing circuit 322 of the edit processing unit 32. When the operation signal PSe relates to an editing operation such as processing or combining read material data, the edit control unit 33 generates an edit control signal ET corresponding to the operation signal PSe, and outputs a signal from the edit processing unit 32. It is supplied to the editing circuit 323. Further, when the editing of the material data is completed and the content data is completed, and when the operation signal PSe indicates an operation of storing the content data in the data storage device 321, the editing control unit 33 responds to the operation signal PSe. The write control signal WC is generated and supplied to the write / read processing circuit 322. When the operation signal PSe indicates the output of the content data, the editing control unit 33 generates an output control signal RP corresponding to the operation signal PSe and supplies the output control signal RP to the write / read processing circuit 322. When the operation signal PSe specifies a playback speed range of the content data, the editing control unit 33 generates a speed range setting signal LP corresponding to the operation signal PSe and supplies the signal to the signal editing circuit 323.

  The write / read processing circuit 322 reads the requested material data from the data storage device 321 and supplies the read material data to the signal editing circuit 323 based on the read control signal RC. Further, the write / read processing circuit 322 stores the completed content data DC in the data storage device 321 based on the write control signal WC. The write / read processing circuit 322 reads the requested content data DC from the data storage device 321 based on the output control signal RP and outputs the read content data DC.

  The signal editing circuit 323 uses the video data DV and / or the audio data DA included in the material data read from the data storage device 321 to perform editing processing such as processing, combining, and deleting of images and sounds, as an edit control signal. Perform based on ET. Here, the signal editing circuit 323 supplies the video data DVe before, during or after editing to the video output signal generating circuit 351 and converts the audio data DAe before, during or after editing into the audio output signal generating circuit. 352. Further, when the frame rate of the video data DV or the audio data DA is changed by the editing processing, the signal editing circuit 323 also changes the attached information DM in accordance with the edited video data or audio data. Further, the signal editing circuit 323 includes frame rate information DM-FRs indicating the set video frame rate FRs corresponding to the edited video data DV and audio data DA and the edited video data DV and audio data DA, and frame identification information DM-. The content data DC is generated by connecting the attached information DMc including the BN. Further, when the speed range setting signal LP is supplied, the signal editing circuit 323 also connects speed range information indicating the playback speed range of the content data DC as the accessory information DMc based on the speed range setting signal LP. Further, when the title of the content and the recommended reproduction speed of the content are input from the user interface unit 34, the signal editing circuit 323 also concatenates such information as additional information DMc. Further, when the reproduction time length information of the content data is obtained by the editing process, this information may be connected as the additional information DMc. Further, when the maximum reproducible speed of the content data is input, this maximum speed is also linked as the additional information DMc. When the subframe number BN is not added to the material data, the signal editing circuit 323 or the editing control unit 33 sets the subframe number BN and sets the frame identification information DMc- BN.

  The video output signal generation circuit 351 of the edit output signal generation unit 35 generates the video signal Svm based on the video data DVg and DVi supplied from the edit control unit 33 and supplies the video signal Svm to the edited image display device 40, as described above. . For this reason, information on the material data can be displayed in the GUI environment. Further, the video output signal generation circuit 351 generates a video signal Svm based on the video data DVe supplied from the signal editing circuit 323. Thereby, the user can check the image before editing, during editing, or after editing on the screen of the edited image display device 40.

  The audio output signal generation circuit 352 converts the audio data DAe supplied from the signal editing circuit 323 into an analog audio signal Sam, and sets the desired signal level to an edited audio output device configured using, for example, a speaker or headphones. 41. For this reason, the user can confirm the sound before, during or after editing by the sound output from the edited sound output device 41.

  As described above, when the post-production process using the material data DTm is performed in the editing device 30 and the content data DC is completed, the completed content data DC is supplied to the content transmission device 50, and the content transmission device 50 Is supplied to the content reproducing apparatus 70 of the user.

  FIG. 8 shows the configuration of the content transmission device 50. The content data DC for transmission supplied from the editing device 30 is supplied to the writing processing unit 51. The write processing unit 51 is connected to the content storage device 521 of the transmission data generation unit 52, and stores the supplied content data for transmission in the content storage device 521. Note that the content data DC is not limited to the data supplied from the editing device 30, and material data or the like generated by the imaging device 10 may be used as the content data DC.

  The transmission data generation unit 52 generates transmission data DTz based on the content data DC, and a read processing circuit 522 is connected to the content storage device 521 of the transmission data generation unit 52. The read processing circuit 522 is supplied with the band information WB of the transmission path for transmitting the content data for transmission and a content request signal RQ from the content reproducing apparatus from a transmission processing unit 53 described later.

  The read processing circuit 522 controls the reading of the requested content data based on the band information WB and the attached information DMc of the requested content data stored in the content storage device 521, adjusts the frame rate, and adjusts the frame rate. The adjusted content data DCza is supplied to the information correction circuit 523.

  For example, when the encoding process is performed by the encoder 524 described later, the data amount of one frame is BDbit, and the set frame rate FRs indicated by the frame rate information DMc-FRs is n (n is a positive number) of the reference frame rate FRr. ) Times, the data amount BT transmitted in the unit time is “BT = BD × n × FRr + BH”. The data amount BH collectively indicates header information and the like added when the content data is packetized and transmitted.

  Here, when the transmittable data amount (bandwidth) BA indicated by the band information WB is not smaller than the data amount BT, the frame rate adjustment of the content data is not performed, and the content data is transferred from the content storage device 521 to the content storage device 521. The data is sequentially read and supplied to the information correction circuit 523. Further, when the bandwidth BA is smaller than the data amount BT, images and sounds may be interrupted during a streaming operation of performing frame rate adjustment on video data or the like of content data and reproducing while receiving transmission data. Reduce the amount of data so that it does not exist. For example, a multiple “m” of the set frame rate FRs with respect to the reference frame rate FRr is identified from the set frame rate FRs and the reference frame rate FRr indicated by the attached information DMc. Further, a divisor of the identified multiple “m” is obtained, and a maximum value of the divisor excluding “m” is multiplied by the reference frame rate FRr to obtain a set frame rate after adjustment. That is, when "m = 10", the maximum value of the divisor is "5", so that the frame rate is adjusted to "m = 5". In this frame rate adjustment, the content data of even subframe numbers “0, 2, 3, 6, 8” is read out every other frame, that is, by using the frame identification information DMc-BN, thereby obtaining the reference frame rate. Content data having a frame rate five times that of FRr is generated. When “m = 9”, the content data of the subframe number “0, 3, 6” is adjusted every two frames, that is, by using the frame identification information DMc-BN, by adjusting to “m = 3”. The content data after frame rate adjustment is generated by reading. When the adjusted data amount BT is larger than the bandwidth BA, the frame rate is further adjusted. As described above, if the adjusted frame rate is determined using the maximum value of the divisor excluding “m”, it is only necessary to use the frame identification information DMc-BN to perform thinning-out in units of frames when reading the content data. Thus, the content data after the frame rate adjustment can be easily generated.

  After that, even when “m = 1”, when the adjusted data amount BT is larger than the bandwidth BA, the data amount BT is further reduced by performing frame decimation so that “m = 1 / k” (k: natural number). Can be reduced. When the bandwidth BA changes, the frame rate is varied according to the change in the bandwidth BA.

  The audio data of the content data is sample-thinned in accordance with the frame rate adjustment for the video data, and the audio data after the frame rate adjustment is generated. For example, when video data is read every other frame, audio data is read every other sample. When video data is read out every two frames, audio data is read out every two samples to generate audio data after frame rate adjustment.

  When the frame rate is adjusted by the read processing circuit 522, the information correction circuit 523 corrects the additional information DMza of the content data DCza so as to correspond to the adjusted frame rate, and the additional information DMz that correctly indicates the frame rate. And Further, the content data DCz to which the attached information DMz is connected is supplied to the encoder 524. For example, when “m = 10” is adjusted to “m = 5”, a modification is made to change the set frame rate FRs from “× 10” to “× 5”, and the set frame rate FRs is “× 10”. Is changed to frame rate information DMz-FRs indicating that the set frame rate FRs is “× 5”. With the change of the set frame rate FRs, the frame identification information DMza-BN is also changed. That is, the frame identification information DMz-BN is obtained by replacing the subframe numbers BN “0 to 9” with the subframe numbers BN “0 to 4”. Further, the attached information DMc is changed to the attached information DMz using the frame rate information DMz-FRs and the frame identification information DMz-BN after this change.

  The encoder 524 encodes the supplied video data DVz and audio data DAz of the content data DCz into signals suitable for transmission, and generates encoded data DZ. For example, a signal suitable for streaming transmission is encoded using an encoding method standardized as Moving Picture Experts Group (MPEG) 4 or the like. The additional information DMz is linked to the coded data DZ obtained by this coding process, and is supplied to the transmission processing unit 53 as transmission data DTz. As described above, the content data can be efficiently transmitted by performing the encoding process.

  When a request for content data is made by the transmission signal TMrq supplied from the content reproduction device 70, the transmission processing unit 53 supplies a content request signal RQ indicating the requested content data to the read processing circuit 522. Further, the transmission processing unit 53 generates band information WB regarding the band of the transmission path 60 and supplies the band information WB to the read processing circuit 522. Further, based on the request for the content data, the transmission processing unit 53 uses the transmission data DTz supplied from the encoder 524 as a transmission signal of a predetermined protocol, and makes a request for the content data via the transmission path 60. To supply.

  The band information WB to be supplied to the read processing circuit 522 can be obtained from a management information base (MIB) of the network device by configuring the transmission processing unit 53 using a network device such as a router. Traffic information can be used as band information WB. In addition, it is also possible to transmit a measurement packet to the content reproduction device 70 and measure a response time from the content reproduction device 70 to identify a band, and use the identification result as the band information WB.

  Also, not only the frame rate is adjusted by the readout processing circuit 522 based on the band information WB, but also the data amount BT is changed according to the bandwidth BA by changing the data compression ratio by the encoder 524 based on the band information WB. It is good. In this case, since the control of the data amount can be controlled more finely, it is possible to reduce the deterioration of the quality of the transmitted image and sound even when the bandwidth BA is narrowed. Further, during the frame period in which the state in which the set frame rates FRs are equal to each other, the adjusted frame rate can be fixed regardless of the band information WB, and the data amount BT can be adjusted by the encoder 524. In this case, it is possible to prevent the content part set to a desired frame rate by the imaging device 10 or the editing device 30 from being adjusted to a different frame rate according to the bandwidth BA.

  Furthermore, when the recommended playback speed is set in the attached information DMc, it is necessary to adjust the frame rate within a range that allows playback at the recommended playback speed, and to reduce the number of frames as compared with playback at the recommended playback speed. When this occurs, the data amount BT may be adjusted by the encoder 524. In this case, the content can be reproduced at the recommended reproduction speed even if the band of the transmission path 60 is narrowed.

  By the way, the above-described content transmission processing of the content transmission apparatus 50 can be realized by software processing using a computer. FIG. 9 shows a configuration in the case where content transmission is performed by this software processing.

  The computer has a built-in CPU (Central Processing Unit) 551 as shown in FIG. 9. The CPU 551 uses a ROM 552, a RAM 553, a hard disk drive or the like having a large storage capacity via a bus 560. A data storage unit 554 and an input / output interface 555 configured as described above are connected. Further, a signal input unit 561, a communication unit 562, and a recording medium drive 563 are connected to the input / output interface 555.

  The CPU 551 performs a content transmission process by executing a program stored in the ROM 552, the RAM 553, or the data storage unit 554. The content data input to the signal input unit 561 is stored in the data storage unit 554 via the input / output interface 555 and the bus 560. Further, when the content request signal RQ is supplied via the communication unit 562, the CPU 551 reads the requested content data from the content data stored in the data storage unit 554, and controls the reading. The frame rate is adjusted so that the content data has a data amount corresponding to the transmission path 60. Further, the CPU 551 performs encoding suitable for transmission to generate transmission data DTz. The generated transmission data DTz is output via the communication unit 562.

  The program for performing the content transmission processing may be stored in the ROM 552 or the data storage unit 554 in advance, or the content transmission processing program may be recorded on the recording medium by the recording medium drive 563 or may be recorded on the recording medium. May be read and executed. Further, the program may be transmitted or received by the communication unit 562 via a wired or wireless transmission path, and the received program may be executed by a computer.

  FIG. 10 is a flowchart showing the content transmission processing operation. In step ST11, the CPU 551 captures the content data DC, and stores the content data DC input to the signal input unit 561 in the data storage unit 554. Note that the content data is not limited to the data supplied from the editing device 30, and material data or the like generated by the imaging device 10 may be stored in the data storage unit 554 as content data.

  In step ST12, the CPU 551 determines whether or not a request for content data has been made. Here, when the request for the content data is not made, the process returns to step ST12, and when the request for the content data is made, for example, via the communication unit 562, the process proceeds to step ST13.

  In step ST13, the CPU 551 reads the attached information of the requested content data, and proceeds to step ST14.

  In step ST14, the CPU 551 detects the band of the transmission path and adjusts the frame rate by controlling the content data read from the data storage unit 554 using the frame identification information according to the detected band.

  In step ST15, the CPU 551 modifies the attached information DMza of the read content data DCza so as to correspond to the adjusted frame rate, and sets it as the attached information DMz. In step ST16, the CPU 551 performs an encoding process according to the transmission path using the content data DCz in which the attached information has been corrected, and generates encoded data DZ. Further, using the generated coded data DZ and the modified additional information DMz, transmission data DTz is generated, and the process proceeds to step ST17.

  In step ST17, the CPU 551 outputs the generated transmission data DTz from the communication unit 562 to the destination of the content data request.

  Next, a content reproducing apparatus will be described. FIG. 11 shows a configuration of the content reproduction device 70. The transmission signal TMz supplied from the content transmission device 50 is supplied to the communication circuit 711 of the input unit 71. The input unit 71 captures content data, and the communication circuit 711 of the input unit 71 generates transmission data DTz from the supplied transmission signal TMz, and uses the transmission data DTz to generate encoded data DZ and additional information DMz. Is extracted. Further, the communication circuit 711 supplies the extracted additional information DMz to the information storage circuit 712, and supplies the encoded data DZ to the data holding circuit 713. In addition, the communication circuit 711 generates a transmission signal TMrq based on a content request signal RQ from a reproduction control unit 72 described later, and supplies the transmission signal TMrq to the content transmission device 50.

  The information storage circuit 712 stores the supplied additional information DMz. The data holding circuit 713 stores the supplied encoded data DZ.

  The reproduction control unit 72 is connected to a user interface unit 73. When the operation signal PSp from the user interface unit 73 requests content data, the reproduction control unit 72 generates a content request signal RQ based on the operation signal PSp and supplies it to the communication circuit 711. A content data transmission request is issued to the content transmission apparatus 50.

  When the operation signal PSp instructs the reproduction of the content data, the reproduction control unit 72 supplies the read control signal CN to the data holding circuit 713, and the encoded data of the content instructed to be reproduced is supplied. DZ is read from the data holding circuit 713 and supplied to the reproduction processing unit 74. Further, the reproduction control unit 72 reads the additional information DMz corresponding to the read encoded data DZ from the information storage circuit 712, and converts the additional information DMz into video data DVs for displaying information included in the additional information DMz, for example, the additional information DMz. When the video data indicating the reproducible speed range based on the included restriction information and the time information such as the time code are included in the accessory information DMz, the total time and the time of the reproduction position indicated by the time information are determined. The video data shown is generated and supplied to the reproduction processing unit 74. Thus, the reproducible speed range, the total time, the time of the reproduction position, and the like are displayed on the screen of the content presentation device 80 such as a television device or a monitor device. When the speed range information is not included in the attached information DMz, the reproducible speed range is set as shown in the editing device 30 described above. The set reproducible speed range is displayed on the screen of the content presentation device 80. When the operation signal PSp changes the content playback speed FP, the playback control unit 72 generates a presentation control signal CP for controlling the operation of the playback processing unit 74 based on the attached information DMz, and sends the presentation control signal CP to the playback processing unit 74. Supply.

  When the maximum speed at which the content can be played back is indicated by the accessory information DMz, the playback control unit 72 sets the maximum speed in the variable range of the playback speed FP to the maximum speed indicated by the accessory information DMz. Further, when the recommended reproduction speed is indicated by the accessory information DMz, and when the reproduction speed is not instructed by the operation signal PSp, the reproduction control unit 72 causes the presentation control signal CP to perform the reproduction operation at the recommended reproduction speed. Generate When the attached information DMz indicates the title and the time length of the content, the playback control unit 72 displays such information on the screen of the content presentation device 80.

  The playback processing unit 74 that performs variable-speed playback of the content performs a decoding process on the encoded data DZ supplied from the data holding circuit 713, and generates video data DVz and audio data DAz of the content. The reproduction processing unit 74 further performs a thinning-out or repetition process using the frame identification information DMz-BN on the generated video data DVz and the audio data DAz based on the presentation control signal CP, and performs a reproduction speed FP or a reproduction speed set by the user. The content presentation is performed by generating and supplying the video signal Svz and the audio signal Saz corresponding to the playback speed FP equal to the recommended playback speed to the content presentation device 80. Further, when the video data DVs indicating the variable range of the playback speed FP is supplied, the playback processing unit 74 generates the video signal Svz for displaying the variable range of the playback speed FP on the screen of the content presentation device 80. .

  When the encoded data DZ is composed of intra-frame encoded data, the reproduction processing unit 74 reads out the encoded data DZ by thinning out the encoded data DZ in frame units based on the presentation control signal CP from the data holding circuit 713. It is good. In this case, there is no need to decode the thinned video data, and the decoding process can be easily performed.

  Further, the content reproducing device 70 may use a recording medium on which content data is recorded. In this case, the additional information DMz and the encoded data DZ are separated from the reproduced signal obtained by reproducing the recording medium, the additional information DMz is stored in the information storage circuit 712, and the encoded data DZ is stored in the data holding circuit 713. , The same processing can be performed.

  By the way, the content reproduction process of the content reproduction device 70 described above can also be realized by performing software processing on a computer. FIG. 12 shows a configuration in the case where content reproduction is performed by this software processing.

  The computer has a built-in CPU 751 as shown in FIG. 12, and a ROM 752, a RAM 753, a data storage unit 754, and an input / output interface 755 are connected to the CPU 751 via a bus 760. Further, a communication unit 761, a user interface unit 762, a signal output unit 763, and a recording medium drive 764 are connected to the input / output interface 755.

  The CPU 751 executes a program stored in the ROM 752, the RAM 753, or the data storage unit 754, and performs a content transmission processing operation based on the operation signal PSp from the user interface unit 762. Here, when the transmission data DTz is supplied to the communication unit 761, the communication unit 761 extracts the encoded data DZ and the additional information DMz. The encoded data DZ and the attached information DMz extracted by the communication unit 761 are stored in the data storage unit 754. Further, the CPU 751 reads and decodes the encoded data DZ stored in the data storage unit 754 based on the operation signal PSp from the user interface unit 762, and generates video data DVz and audio data DAz. To the signal output unit 763. The signal output unit 763 generates and outputs a video signal Svz and an audio signal Saz corresponding to the content presentation device 80 based on the video data DVz and the audio data DAz.

  Note that the program for performing the content reproduction processing is stored in advance in the ROM 752 or the data storage unit 754, or the program for the content reproduction processing is recorded on the recording medium by the recording medium drive 764 or is recorded on the recording medium. May be read and executed. Further, the communication unit 761 may transmit or receive the program via a wired or wireless transmission path, and may execute the received program on a computer.

  FIG. 13 is a flowchart showing the content reproduction processing operation. When reproducing the content data, the CPU 751 displays an image for configuring the GUI environment on the content presentation device 80, and performs an operation corresponding to the displayed image on the user interface unit 762 to perform an operation input.

  FIG. 14 illustrates a display image of the content presentation device 80, and the content presentation device 80 displays an image for a GUI. On the screen of the content presentation device 80, a viewer unit 801 for displaying an image of the content, a speed variable console unit 802 which is an interface for changing the reproduction speed FP, a reproduction speed display unit 803 for displaying the reproduction speed FP, an operation An operation control unit 804 for switching the mode, volume, etc., a title display unit 805 showing the title of the content, a time display unit 806 showing the playback time of the content and the current time, and a playback position display showing the current playback position A portion 807 and the like are provided.

  In step ST21 of FIG. 13, the CPU 751 reads the attached information DMz of the content from the data storage unit 754, and generates the attached information DMz from the signal output unit 763 to the content presentation device 80 via the input / output interface 755 based on the attached information DMz. The video signal Svz and the audio signal Saz are output. Thus, the content presentation device 80 performs display according to the attached information DMz. For example, the title of the content and the time length of the content are displayed on the title display unit 805 and the time display unit 806. The minimum speed and the maximum speed are displayed on the speed variable console unit 802 based on the speed range information.

  In step ST22, the CPU 751 uses the operation control unit 804 to identify whether or not a content reproduction start operation has been performed based on the operation signal PSp. Here, when the reproduction start operation is not performed, the CPU 751 returns to step ST22, and when the reproduction start operation is performed, the process proceeds to step ST23.

  In step ST23, the CPU 751 sets the reproduction processing conditions according to the reproduction speed FP and the set frame rate FRs, that is, the video signal Svz and the audio signal Saz from the video data DVz and the audio data DAz obtained by decoding the encoded data DZ. Determine the data thinning interval and the number of data repetitions to be performed when generating.

  In step ST24, the CPU 751 reads out and decodes the encoded data DZ from the data storage unit 754 to generate video data DVz and audio data DAz, and generates the frame identification information DMz-BN based on the reproduction processing conditions determined in step ST23. Is used to thin out data and repeat data, thereby generating a video signal Svz and an audio signal Saz for content presentation. The CPU 751 supplies the generated video signal Svz and audio signal Saz to the content presentation device 80, so that the reproduced image of the reproduction speed FP indicated by the cursor position (shown by a thick line) of the speed variable console unit 802 is displayed in the content presentation mode. It is displayed on the viewer section 801 of the device 80. The playback speed FP at this time is displayed on the playback speed display unit 803, and the playback time and the playback position are displayed on the time display unit 806 and the playback position display unit 807. Further, the content presentation device 80 outputs a playback sound at the playback speed FP indicated by the cursor position of the speed variable console unit 802.

  In step ST25, the CPU 751 determines whether or not the cursor position of the variable speed console unit 802 has been moved and the reproduction speed FP has been changed. Here, the CPU 751 returns to step ST23 when identifying that the change of the reproduction speed FP has been performed, and proceeds to step ST26 when determining that the change of the reproduction speed FP has not been performed.

  In step ST26, the CPU 751 determines whether or not the reproduction operation has ended. Here, when the operation of stopping the reproduction operation is not performed, or when the reproduction position of the content is not the end position, the CPU 751 returns to step ST25. Further, when the stop operation is performed or when the reproduction position reaches the end position, the CPU 751 ends the variable speed reproduction operation.

  FIG. 15 is a flowchart showing an operation of setting a reproduction processing condition for an image. In step ST31, the CPU 751 identifies the reproduction speed FP based on the cursor position of the variable speed console unit 802, and proceeds to step ST32. Here, the reproduction speed FP at the start of the reproduction operation is set by setting the reference frame rate FRr to 1 × speed and setting the initial position of the cursor in the variable speed console unit 802 to, for example, a 1 × speed position. When the editing device 30 recommends the reproduction speed FP, the CPU 751 sets the position of the recommended reproduction speed FP as the initial position of the cursor and sets the recommended reproduction speed FP at the time of starting the reproduction operation. The reproduction speed is set to FP. Further, when the cursor position is being moved by the user, the CPU 751 sets the speed corresponding to the cursor position as the reproduction speed FP.

  In step ST32, the CPU 751 identifies the set frame rate FRs based on the frame rate information DMz-FRs included in the attached information DMz, and proceeds to step ST33. In step ST33, the CPU 751 calculates an identification value FD for determining a reproduction processing condition by multiplying the reproduction speed FP by the set frame rate FRs.

  In step ST34, the CPU 751 determines a reproduction processing condition based on the identification value FD. Here, when the identification value FD is 1 or more and does not include a value below the decimal point, the CPU 751 determines a reproduction processing condition such that images are thinned out at frame intervals according to the identification value FD and output. When the identification value FD is equal to or greater than 1 and includes a value below the decimal point, the CPU 751 performs image thinning using the frame identification information DMz-BN at frame intervals corresponding to the integer value part of the identification value FD, and performs desired reproduction. When an image having the number of frames corresponding to the speed is obtained, the reproduction processing conditions are determined so that the position of the image is moved to the next initial value of the subframe number BN. When the identification value FD is less than 1, the CPU 751 determines the reproduction processing condition so that the same image is repeatedly output until the number of frames reaches a desired reproduction speed. By performing the processing in step ST24 based on the playback processing conditions determined in this way, it is possible to correctly present the content image at the desired playback speed.

  FIG. 16 shows a reproducing operation when the identification value FD is 1 or more and does not include a value below the decimal point. FIG. 16A shows an image based on the video data DVz when the set frame rate FRs is set to be 10 times faster than the reference frame rate FRr. 16B shows frame rate information DMz-FRs indicating the set frame rate FRs of the frame image, FIG. 16C shows frame identification information DMz-BN indicating the subframe number BN of the frame image, and FIG. 16D shows the absolute frame number AN. I have.

  Here, when the reproduction speed FP is set to (１ ／) × speed, the identification value FD is “10 × (１ ／) = 2”. For this reason, as shown in FIGS. 16E to 16G, it is necessary to generate the video signal Svz by using the video data DVz every other frame using the “FD = 2” frame, that is, using the frame identification information DMz-BN. Thus, a reproduced image at (1/5) speed can be displayed on the content presentation device 80. 16E shows the frame identification information DMz-BN of the displayed image, FIG. 16F shows the absolute frame number AN of the displayed image, FIG. 16F shows the frame identification information DMz-BN of the displayed image, and FIG. This shows a frame image displayed in Svz.

  When the reproduction speed FP is 1, the identification value FD is “10 × 1 = 10”. For this reason, as shown in FIGS. 16H to 16J, the video signal Svz is generated for each “FD = 10” frame, that is, while skipping nine frames of the video data DVz using the frame identification information DMz-BN. As a result, a 1 × speed reproduced image can be displayed on the content presentation device 80. 16H shows the frame identification information DMz-BN of the displayed image, FIG. 16I shows the absolute frame number AN of the displayed image, and FIG. 16J shows the frame image displayed by the video signal Svz.

  When the reproduction speed FP is double, the identification value FD is “10 × 2 = 20”. For this reason, as shown in FIGS. 16K to 16M, the video signal Svz is generated for every “FD = 20” frame, that is, while skipping 19 frames of the video data DVz using the frame identification information DMz-BN. As a result, a double-speed playback image can be displayed on the content presentation device 80. 16K shows the frame identification information DMz-BN of the displayed image, FIG. 16L shows the absolute frame number AN of the displayed image, and FIG. 16M shows the frame image displayed by the video signal Svz.

  FIG. 17 shows a reproducing operation when the identification value FD is 1 or more and does not include a value below the decimal point. FIG. 17A shows a frame image when the set frame rate FRs is set to be 7 times faster than the reference frame rate FRr. 17B shows frame rate information DMz-FRs indicating the set frame rate FRs of the frame image, FIG. 17C shows frame identification information DMz-BN indicating the subframe number BN of the frame image, and FIG. 17D shows the absolute frame number AN. I have.

  Here, when the reproduction speed FP is (1/3) -times speed, the identification value FD is "7 * (1/3) = 2.33 ...". Therefore, as shown in FIGS. 17E to 17G, the video data DVz is used every second frame, that is, every other frame using the frame identification information DMz-BN according to the integer part of the identification value FD. Further, since the number of frames according to a desired reproduction speed, that is, (1/3) times speed, when an image of three frames is output during one frame period of the reference frame rate FRr, the image following the subframe number BN is output. The position of the video data DVz used up to the initial value is moved. In this case, the video signal Svz is sequentially generated using the video data DVz having the subframe numbers BN of “0”, “2”, and “4”, and a (１ ／)-fold speed reproduced image can be displayed on the content presentation device 80. . 17E shows the frame identification information DMz-BN of the displayed image, FIG. 17F shows the absolute frame number AN of the displayed image, and FIG. 17G shows the frame image displayed by the video signal Svz.

  FIG. 18 shows a reproducing operation when the identification value FD is less than 1. FIG. 18A shows a frame image when the set frame rate FRs is (（) times the reference frame rate FRr. 18B shows frame rate information DMz-FRs indicating a set frame rate FRs of a frame image, FIG. 18C shows frame identification information DMz-BN indicating a subframe number BN of the frame image, and FIG. 18D shows an absolute frame number AN. I have.

  Here, when the reproduction speed FP is set to 1 × speed, the identification value FD is “(１ ／) × 1 = 1/4”. For this reason, as shown in FIGS. 18E to 18G, the video signal Svz is generated by repeatedly using the number of frames according to the reproduction speed, that is, the video data DVz four times for each frame, so that the reproduced image at the 1 × speed can be reproduced. It can be displayed on the presentation device 80. 18E shows the frame identification information DMz-BN of the displayed image, FIG. 18F shows the absolute frame number AN of the displayed image, and FIG. 18G shows the frame image displayed by the video signal Svz.

  When the reproduction speed FP is a double speed, the identification value FD is “(１ ／) × 2 = １ ／”. For this reason, as shown in FIG. 18H to FIG. 18J, by generating the video signal Svz by repeatedly using the video data DVz twice for each frame, a reproduced image at 2 × speed can be displayed on the content presentation device 80. 18H shows the frame identification information DMz-BN of the displayed image, FIG. 18I shows the absolute frame number AN of the displayed image, and FIG. 18J shows the displayed frame image displayed by the video signal Svz. Is shown.

  When the reproduction speed FP is a quadruple speed, the identification value FD is “(１ ／) × 4 = 1”. Therefore, as shown in FIG. 18K to FIG. 18M, by generating the video signal Svz by sequentially using the video data DVz for each frame, a reproduced image at 4 × speed can be displayed on the content presentation device 80. 18K shows the frame identification information DMz-BN of the displayed image, FIG. 18L shows the absolute frame number AN of the displayed image, and FIG. 18M shows the displayed frame image displayed by the video signal Svz. Is shown.

  As described above, by reading out the image data at the readout interval based on the recording speed and the reproduction speed by using the frame identification information, the image at the desired reproduction speed can be easily displayed.

  Next, audio will be described. FIG. 19 is a flowchart showing an operation of setting a reproduction processing condition for audio. As for audio, when audio data DAz is used for each frame, the connection of sounds between frames is lost, and discontinuity of sounds occurs. For this reason, the audio is reproduced on a sample basis.

  In step ST41, the CPU 751 determines the reproduction speed in the same manner as in step ST31, and proceeds to step ST42. In step ST42, the CPU 751 reads out the set frame rate FRs as in step ST32, and proceeds to step ST43. In step ST43, the CPU 751 calculates the identification value FD as in step ST33, and proceeds to step ST44.

  In step ST44, the CPU 751 determines a reproduction processing condition based on the identification value FD. Here, when the discrimination value FD is 1 or more and does not include a value below the decimal point, the reproduction processing condition is determined so that the audio data is thinned out at sample intervals corresponding to the discrimination value FD. When the discrimination value FD is 1 or more and includes a value after the decimal point, the audio data is thinned out at a sample interval corresponding to the integer value part of the discrimination value FD from the frame which is a multiple of the set frame rate FRs with respect to the reference frame rate FRr. The reproduction processing conditions are determined so that the audio data corresponding to the reproduction speed is read. If the identification value FD is less than 1, the reproduction processing condition is determined so that the audio data is repeatedly used so that the number of samples is equal to the number of frames corresponding to the desired reproduction speed. By performing the processing in step ST24 based on the reproduction processing conditions determined in this way, it is possible to correctly present the audio of the content at a desired reproduction speed.

  FIG. 20 shows an audio reproducing operation when the identification value FD is 1 or more and does not include a value below the decimal point. 20A shows the absolute frame number AN, FIG. 20B shows the frame rate information DMz-FRs indicating the set frame rate FRs of the frame image, and FIG. 20C shows the frame identification information DMz-BN indicating the sub-frame number BN of the frame image. .

  Here, when the reproduction speed FP is set to (１ ／) × speed, since the set frame rate FRs is set to be 10 × speed with respect to the reference frame rate FRr, the identification value FD is “10 × (１ ／). ) = 2 ”. For this reason, by generating the audio signal Saz using the audio data DAz at every “FD = 2” sample, that is, every other sample, the reproduction audio at 1 × speed can be output from the content presentation device 80. FIG. 20D shows a frame used for generating the video signal Svz, and FIG. 20E shows audio data used for the audio signal Saz when the audio data DAz is 14 samples / frame.

  FIG. 21 shows a sound reproducing operation when the identification value FD is 1 or more and includes a value below the decimal point. 21A shows the absolute frame number AN, FIG. 21B shows the frame rate information DMz-FRs indicating the set frame rate FRs of the frame image, and FIG. 21C shows the frame identification information DMz-BN indicating the sub-frame number BN of the frame image. .

  Here, when the reproduction speed FP is set to (1/3) times speed, the set frame rate FRs is set to 7 times speed with respect to the reference frame rate FRr. ) = 2.3... Further, when the audio data DAz is 14 samples / frame, the number of samples in one frame at (1/3) speed is “14 × 3/7 = 6”. For this reason, when the audio data DAz is output every second sample, that is, every other sample according to the integer value part of the identification value FD, and when the audio data DAz of 6 samples corresponding to the number of samples of one frame is output, Is moved to the beginning of the next frame to output the audio data DAz every other sample. By selecting and outputting the audio data DAz in this way, it is possible to obtain a (1/3) times speed reproduction audio. In addition, if filter processing is performed when audio is output based on the audio signal Saz, the effect of thinning out the audio data DAz can be reduced and good reproduced audio can be output. Furthermore, since the sampling is fixedly skipped according to the set frame rate FRs and the reproduction speed FP and the number of samples is adjusted at the end of the frame, the output of the audio signal Saz according to the reproduction speed can be easily performed. it can. FIG. 21C shows a frame used for generating the video signal Svz, and FIG. 21D shows audio data used for the audio signal Saz when the audio data DAz is 14 samples / frame.

  When the audio signal Saz is generated by thinning out the audio data DAz, the audio signal Saz is used to generate the audio signal Saz so that the interval between the audio data DAz is not widened and the reproduced sound is not discontinuous. The thinning may be performed so that the data interval is substantially constant. For example, when the set frame rate FRs is set to KA times the reference frame rate FRr, and the reproduction speed FP is set to (1 / KB) times, the KB data is obtained at substantially equal intervals from the audio data DAz of the continuous KB samples. Is extracted, and an audio signal Saz is generated based on the extracted audio data. By doing so, the processing becomes more complicated than in the case shown in FIG. 21, but it is possible to output a reproduced sound with better sound quality.

  When the identification value FD is less than 1, the audio data DAz having a desired reproduction speed can be generated by repeatedly using each audio data by the number of image repetitions (not shown).

  Thus, on the content transmission side, the frame rate information and the additional information DMz including the frame identification information for identifying the frame included in the reference frame period are transmitted to the content data DCz linked to the main data indicating the image and / or audio. Is transmitted. On the content reproduction side, image and / or audio data is reproduced by changing the reproduction speed using the additional information DMz including the frame rate information and the frame identification information. Therefore, it is possible not only to view images and the like at a predetermined reproduction speed like a broadcast program, but also to view images and the like at a reproduction speed desired by the user. For example, if the set frame rate FRs is set to be higher than the reference frame rate FRr to generate content such as a sports broadcast, the user does not need to wait for the slow playback image to be supplied from the content provider side as in a conventional broadcast program. Can watch only a desired scene in slow speed, while usually watching at 1 × speed.

  On the content transmission side, the frame rate is adjusted in accordance with the band of the transmission path using the frame identification information, so that the frame rate can be easily adjusted. In addition, on the content reproduction side, data thinning or the like can be easily performed in frame units by using the frame identification information, so that the content reproduction speed can be easily varied.

As described above, the present invention is useful for transmitting and reproducing image content and the like, and is particularly suitable for controlling the frame rate during reproduction.

FIG. 1 is a diagram illustrating an overall configuration of a content transmission system. FIG. 2 is a diagram illustrating a configuration of an imaging device. FIG. 11 is a diagram illustrating another configuration of the imaging device. FIG. 11 is a diagram illustrating another configuration of the imaging device. FIG. 9 is a diagram showing a relationship (part 1) of additional information of video data. It is a figure which shows the relationship (2) of video data and attached information. FIG. 2 is a diagram illustrating a configuration of an editing device. FIG. 2 is a diagram illustrating a configuration of a content transmission device. FIG. 3 is a diagram illustrating a configuration when content transmission is performed by software. It is a flowchart which shows a content transmission processing operation. FIG. 2 is a diagram illustrating a configuration of a content reproduction device. FIG. 3 is a diagram illustrating a configuration in a case where content reproduction is performed by software. It is a flowchart which shows a content reproduction processing operation. It is a figure showing a display picture of a contents presentation device. 9 is a flowchart illustrating an operation of setting a reproduction processing condition for an image. It is a figure showing the image reproduction operation (the 1). It is a figure showing the image reproduction operation (the 2). It is a figure showing the image reproduction operation (the 3). 9 is a flowchart illustrating an operation of setting a reproduction processing condition for audio. It is a figure which shows the audio | voice reproduction operation | movement (the 1). It is a figure which shows the audio | voice reproduction | regeneration operation | movement (2).

Explanation of reference numerals

  10, 10a ... imaging device, 12 ... imaging unit, 13, 17 ... signal processing unit, 14, 18 ... control unit, 15 ... output unit, 16, 34, 73, 762 ... ..User interface unit, 20 voice input device, 30 editing device, 31 material acquisition unit, 32 editing processing unit, 33 editing control unit, 35 Edit output signal generation unit, 40: Edit image display device, 41: Edit audio output device, 50: Content transmission device, 51: Write processing unit, 52: Transmission data generation unit, 53: transmission processing unit, 70: content playback device, 71: input unit, 72: playback control unit, 74: playback processing unit, 80: content presentation device, 131 ...・ Camera processing circuit, 132 ・ ・ ・ Sound processing circuit, 141,181 ・ ・ ・ Imaging Control circuit, 142, 182 timing generator, 171 valid data selection circuit, 183 valid frame signal generation circuit, 311 information detection circuit, 312 database processing circuit, 321 ..Data storage devices, 322... Write / read processing circuits, 323... Signal editing circuits, 351... Video output signal generation circuits, 352... Audio output signal generation circuits, 521. Device, 522: Read processing circuit, 523: Information correction circuit, 524: Encoder, 554, 754: Data storage unit, 561: Signal input unit, 562: Communication unit, 711 ... Communication circuit, 712 ... Information storage circuit, 713 ... Data holding circuit, 763 ... Signal output unit

Claims (28)

Transmission data generating means for generating transmission data by linking auxiliary information including frame rate information of the main data to main data indicating an image and / or audio;
A transmission processing means for outputting the transmission data via a transmission path. The transmission data generation unit has a storage unit that temporarily stores the main data, and a read process control unit that controls a read process of the main data stored in the storage unit,
The transmission processing unit notifies the transmission data generation unit of a band of the transmission path,
2. The transmission apparatus according to claim 1, wherein the read processing control unit adjusts a frame rate of the main data by controlling reading of the main data according to the notified band. 3. The transmission apparatus according to claim 2, further comprising information correction means for correcting frame rate information included in said additional information in accordance with adjustment of a frame rate by said read processing means. The transmission device according to claim 1, wherein the additional information includes information indicating a recommended reproduction speed of the main data. The transmission device according to claim 1, wherein the additional information includes information indicating a maximum reproducible speed of the main data. 2. The transmission data generating unit according to claim 1, wherein, as the additional information, at least the frame rate information and frame identification information of each frame included in a reference frame period are connected to the main data. The transmission device as described. The transmission data generation unit has a storage unit that temporarily stores the main data, and a read processing unit that controls a read process of the main data stored in the storage unit,
The transmission processing unit notifies the transmission data generation unit of a band of the transmission path,
The read processing unit adjusts a frame rate of the main data by controlling reading of the main data according to the notified band using the frame identification information. 7. The transmission device according to 6. 8. The transmission according to claim 7, further comprising information correction means for correcting frame rate information and frame identification information included in said additional information in accordance with adjustment of a frame rate by said read processing means. apparatus. A transmission data generating step of generating transmission data by linking ancillary information including frame rate information of the main data to main data indicating an image and / or audio;
A transmission processing step of outputting the transmission data via a transmission path. The transmission data generation step includes a read processing control step of controlling a read processing of the main data temporarily stored in storage means,
10. The transmission method according to claim 9, wherein, in the reading processing control step, a frame rate of the main data is adjusted by controlling reading of the main data according to a band of the transmission path. 11. The transmission method according to claim 10, further comprising an information correction step of correcting frame rate information included in said additional information according to a frame rate adjustment in said reading processing step. The transmission method according to claim 9, wherein the additional information includes information indicating a recommended reproduction speed of the main data. The transmission method according to claim 9, wherein the additional information includes information indicating a maximum reproducible speed of the main data. 10. The transmission data generating step, wherein at least the frame rate information and frame identification information of a frame included in a reference frame period are connected to the main data as the additional information. The transmission method described. The transmission data generating step includes a reading processing step of controlling a reading process of main data stored in a storage unit that temporarily stores the main data,
In the reading processing step, the frame rate of the main data is adjusted by controlling the reading of the main data according to the band of the transmission path using the frame identification information. The transmission method according to claim 14. 16. The transmission according to claim 15, further comprising an information correction step of correcting the frame rate information and the frame identification information included in the additional information according to the adjustment of the frame rate in the reading processing step. Method. A transmission data generating step of generating transmission data by linking ancillary information including frame rate information of the main data to main data indicating an image and / or audio;
A transmission processing step of outputting the transmission data via a transmission path. The frame rate range of the main data to be reproduced is determined based on the frame rate information of the transmission data formed by linking the main data indicating the image and / or audio to the auxiliary information including the frame rate information of the main data. Setting means for setting the reproduction variable range shown,
A reproducing unit that reproduces the main data at a frame rate within the variable reproduction range. The reproducing means, wherein the attached information includes information indicating a recommended reproduction speed of the main data, and reproduces the main data at the recommended reproduction speed when an instruction of a reproduction speed is not given by a user. 19. The playback device according to claim 18, wherein: 19. The reproduction apparatus according to claim 18, wherein the setting unit sets the reproduction speed variable range using the information indicating the maximum speed when the accessory information includes information indicating a maximum reproduction speed of the main data. The playback device according to the above. The additional information includes frame identification information of each frame included in a reference frame period, and the reproducing unit performs thinning or repetition of the main data by using the frame identification information, thereby obtaining the main data. 19. The reproducing apparatus according to claim 18, wherein the reproducing speed is variable. The frame rate range of the main data to be reproduced is determined based on the frame rate information of the transmission data formed by linking the main data indicating the image and / or audio to the auxiliary information including the frame rate information of the main data. A setting step of setting a playback variable range shown,
A reproducing step of reproducing the main data at a frame rate within the variable reproduction range. In the reproducing step, the accessory information includes information indicating a recommended reproduction speed of the main data, and the main data is reproduced at the recommended reproduction speed when a reproduction speed is not instructed by a user. The reproducing method according to claim 22, characterized in that: In the setting step, when the attached information includes information indicating a maximum reproducible speed of the main data, the reproduction speed variable range is set using the information indicating the maximum speed. 23. The reproducing method according to 22. The auxiliary information includes frame identification information of each frame included in a reference frame period, and in the reproduction step, thinning and repetition of main data are performed using the frame identification information, so that the main data 23. The reproducing method according to claim 22, wherein the reproducing speed of the data is varied. The frame rate range of the main data to be reproduced is determined based on the frame rate information of the transmission data formed by linking the main data indicating the image and / or audio to the auxiliary information including the frame rate information of the main data. A setting step of setting a playback variable range shown,
A program for causing a computer to execute a reproducing method including a reproducing step of reproducing the main data at a frame rate within the variable reproduction range. A recording medium characterized by being recorded in a state where main data indicating an image and / or sound is linked to accessory information including frame rate information of the main data. 28. The recording medium according to claim 27, wherein the additional information includes frame identification information of each frame included in a reference frame period.

Images