JP2007267396A - Apparatus and method for data processing, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily perform edit processing, using a material generated at a standard frame rate and a material generated at a variable frame rate. <P>SOLUTION: When images, based on material data, are displayed on a material management browser, a stream board and a time line, display images are controlled so that material data connected with frame rate information as attached information, and material data that are not connected with frame rate information can be identified. If frame rate information is connected as attached information, a reproducible speed range that is determined, based on the frame rate information, is displayed. Reproduction speed display and speed variable console display to change a reproduction speed within the reproducible speed range are provided, and reproduced images are displayed on a material image display at a specified reproduction speed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an edited image display control device, an edited image display control method, and a program. Specifically, it is detected whether or not the frame rate information is connected to the material data as attached information, and the material data to which the frame rate information is connected as attached information and the frame rate information are not connected as attached information. The state of the display image for editing is controlled so that the material data is identified and displayed.

  Conventionally, in the generation of image or image / sound content used for distribution via broadcasting or communication networks or distribution as a recording medium, the content is generated at a reference frame rate such as 30 frames / second in order to produce the effect intended by the producer. Not only content, but also content with varying speed of movement is used.

  In the generation of content that changes the speed of movement, for example, the content is generated by setting the frame rate higher than the reference frame rate, and the content is played back at the reference frame rate, thereby expressing the movement slower than the actual frame rate. Generated content. Also, content is generated by setting the frame rate lower than the reference frame rate, and the content is reproduced at the reference frame rate, thereby generating content that expresses the motion faster than the actual frame rate. Furthermore, the speed of movement can be freely varied by adjusting the frame rate to be set and the frame rate at the time of reproduction.

  By performing editing processing using content that has changed the speed of movement and content that is generated at the reference frame rate, content that achieves the effects intended by the creator is generated.

  Further, for example, Patent Document 1 discloses a video camera capable of expanding and compressing a time axis so that content with a variable frame rate can be generated.

Japanese Patent Laid-Open No. 11-177930

  By the way, when the editing process is performed by using the content generated at the reference frame rate and the content whose movement speed is changed as the material as described above, what is the content of each content simply by displaying the content image? Whether it is generated at the frame rate cannot be easily identified during editing. Further, when the speed changing process is performed on the content whose movement speed is changed, it is not possible to identify whether or not the speed can be easily changed without degrading the image quality. For example, when a set frame rate (hereinafter referred to as “set frame rate”) is 10 times the reference frame rate, when the content generated at the set frame rate is played back at the reference frame rate, the motion is reduced to 1 / The content is expressed as 10 times speed. Here, when it is desired to express the motion as 1/5 times speed, since the set frame rate is 10 times the reference frame rate, the speed is easily reduced without degrading the image quality by thinning out every frame. It can be identified that changes can be made. However, it is impossible to identify whether or not the speed can be easily changed without degrading the image quality only by the displayed image.

Therefore, according to the present invention, an edit image display control device and an edit image display control device that control the state of an edit display image so that an edit process can be easily performed using a material generated at a reference frame rate and a material with a variable frame rate An image display control method and program are provided.

  The edited image display control device according to the present invention includes a detecting means for detecting whether or not frame rate information is linked to the material data as the attached information, material data in which the frame rate information is linked as the attached information, Control means for controlling the state of the display image for editing so that the material data not linked to the rate information as the attached information is displayed.

  The edited image display control method according to the present invention includes a detection step for detecting whether or not frame rate information is linked to the material data as attached information, and material data to which the frame rate information is linked as attached information, And a display control step for controlling the state of the display image for editing so that the frame rate information is identified and displayed with the material data not linked as the auxiliary information.

  Further, the program according to the present invention includes a step of detecting whether or not frame rate information is linked to the material data as attached information, material data in which the frame rate information is linked as attached information, and frame rate information. The computer is caused to execute a step of controlling the state of the display image for editing so that material data that is not linked as accessory information is identified and displayed.

  In the present invention, when material data is captured and editing processing is performed using the captured material data, it is detected whether or not frame rate information is linked to the material data as attached information, and the frame rate is included in the material data. The material data to which the information is linked and the material data to which the frame rate information is not linked are displayed so as to be distinguishable. For example, in the material management display showing a list of material data, the material data to which the attached information is linked and the material data to which the attached information is not linked are displayed so as to be distinguishable. Further, in the reproduction order display in which the material data is arranged in the reproduction order and content is generated, the material data to which the attached information is linked and the material data to which the attached information is not linked are displayed in an identifiable manner. Furthermore, in the playback time order display in which material data is pasted along the time axis and content is generated, the material data to which the attached information is linked and the material data to which the attached information is not linked are displayed in an identifiable manner. The

  Further, the reproducible speed range of the material data is determined based on the attached information, and the determined reproducible speed range is displayed. This reproducible speed range is displayed in a material management display showing a list of material data, a reproduction order display in which material data is arranged in the reproduction order and content is generated, and a reproduction image display of material data. A position display indicating the reproduction speed is provided in the display of the reproducible speed range.

  Furthermore, in the reproduction time order display in which material data is pasted along the time axis and content is generated, the display width of the material data is variably displayed according to the reproduction time calculated based on the reproduction speed. In addition, when an operation for changing the display width of the material data in the playback time order display is performed, the display position of the position display is moved according to the operation, and the operation for changing the display position of the position display is performed. When performed, the display width of the material data in the reproduction time order display is varied according to the operation.

  According to the present invention, when material data is captured and editing processing is performed using the captured material data, it is detected whether the frame rate information is linked to the material data as attached information, and the material data is framed. The material data to which the rate information is linked and the material data to which the frame rate information is not linked are displayed so as to be distinguishable. Therefore, it is possible to easily find a scene that can be played slowly using the frame rate information of the material data.

  In addition, since the reproducible speed range of the material data is determined based on the attached information and the determined reproducible speed range is displayed, it is possible to easily set the reproducible playback speed without noise. Also, in the playback time order display in which the material data is pasted along the time axis and the content is generated, the display width of the material data is variably displayed according to the playback time calculated based on the playback speed. You can visually grasp the expansion and contraction of the time axis. Furthermore, when an operation to change the display width of the material data in the playback time order display is performed, the position display of the playback speed in the display of the playable speed range is moved in accordance with the operation, and the position display is variable. When the operation is performed, the display width of the material data in the reproduction time order display is changed according to the operation, so that the relationship between the reproduction speed and the expansion / contraction of the time axis can be easily grasped.

  Embodiments of the present invention will be described below. FIG. 1 shows an example of the overall configuration of a content editing system for image and / or audio content. The imaging apparatus 10 generates video data with a variable frame rate, and links the attached information including the frame rate information indicating the frame rate of the video data to the video data to display the edited image as material data DTm. The data is supplied to the editing apparatus 30 that performs control. When the audio input device 20 is provided in the imaging device 10, audio data is generated and supplied to the editing device 30 as material data DTm together with attached information. The material data DTm may be supplied not only from the imaging device 10 but also from other devices. Note that the term “concatenation” as used herein may be a state in which video data and attached information including frame rate information are linked to each other. For example, even if they are recorded on different recording media or transmitted on different transmission paths, if the frame number corresponding to the attached information is included, etc. Can be attached. In the present embodiment, it is assumed that the connection is made including such a case.

  The editing device 30 performs an editing process using the supplied material data DTm, and generates and outputs content data DC. The editing device 30 generates a video signal Svm related to editing and supplies it to the edited image display device 40. Thereby, the user can check the editing progress of the image, the editing result, and the like by the display image on the edited image display device 40. Similarly, the editing device 30 generates an audio signal Sam related to editing and supplies it to the edited audio output device 41. As a result, the user can confirm the editing progress of the sound, the editing result, and the like by the sound output from the edited sound output device 41.

  FIG. 2 shows a configuration example of the imaging apparatus 10. The light incident through the imaging lens system 11 is incident on the imaging unit 12, and a subject image is formed on an imaging surface of an imaging element such as a CCD (Charge Coupled Device) provided in the imaging unit 12. . The imaging device generates imaging charges for the subject image by photoelectric conversion. Further, based on a driving signal CR from a timing generator 142 described later, the generated imaging charge is read, and an imaging signal Sp having a frame rate corresponding to the driving signal CR is generated to generate a camera processing circuit 131 of the signal processing unit 13. To supply.

  The camera processing circuit 131 performs various signal processing at a timing synchronized with the imaging signal Sp based on the timing signal CT supplied from the timing generator 142. For example, processing that removes noise components from the imaging signal Sp by performing correlated double sampling, processing that converts the imaging signal Sp from which noise has been removed, to digital video data, clamping processing of video data, shading correction, and imaging device Defect correction, γ processing, contour compensation, knee correction, and the like. Further, various signal processing is performed under processing conditions based on the operation control signal CS supplied from the imaging control circuit 141 of the control unit 14. Thus, the video data DV obtained by performing various signal processing in the camera processing circuit 131 is supplied to the output unit 15.

  The timing generator 142 of the control unit 14 generates a drive signal CR corresponding to the operation control signal CS from the imaging control circuit 141 and supplies the drive signal CR to the imaging unit 12, thereby changing the readout period of the imaging charge in the imaging unit 12. Thus, the frame rate of the imaging signal Sp is controlled to the set frame rate FRs based on the operation signal PSa from the user interface unit 16. For example, in the NTSC system, the frame frequency is 59.94 Hz or 29.97 Hz, and in the PAL system, the frame frequency is 50 Hz or 25 Hz as the frame frequency of the reference frame rate FRr, and the set frame rate FRs is the reference frame rate FRr k (k is limited to a positive number). When the operation of multiplying the image signal Sp is performed, control is performed so that the frame rate of the imaging signal Sp is k times the reference frame rate FRr. Note that, for example, by changing the period of a readout pulse (sensor gate pulse) for moving the imaging charge accumulated in each pixel from each pixel of the imaging element such as a CCD to the transfer unit, the imaging charge is read out. The period is varied and the frame rate is varied. At this time, a CDR (Common Data Rate) method may be adopted. When the CDR method is used, the effective frame rate is variable, but the frame rate of the signal output from the CCD can be kept unchanged, and the processing rate of the camera processing circuit 131 and the like can be made constant. This CDR system is disclosed in PCT application, application numbers PCT / JP03 / 0051 and 2003/01/22 applications.

  The timing generator 142 generates a timing signal CT synchronized with the drive signal CR and supplies the timing signal CT to the camera processing circuit 131 and the audio processing circuit 132. Further, the timing generator 142 uses the set frame rate FRs, which is the frame rate of the video data DV, as frame rate information, generates attached information DM including the frame rate information, and supplies the generated information DM to the output unit 15.

  A user interface unit 16 is connected to the imaging control circuit 141 of the control unit 14. When an operation switching operation or a variable frame rate operation is performed on the imaging apparatus 10, the user interface unit 16 generates an operation signal PSa corresponding to these operations and supplies the operation signal PSa to the imaging control circuit 141. Further, when an operation signal PSa is supplied from an external device such as a remote commander (not shown), the user interface unit 16 supplies the operation signal PSa to the imaging control circuit 141.

  The imaging control circuit 141 generates an operation control signal CS based on the operation signal PSa from the user interface unit 16 so that the operation of the imaging device 10 becomes an operation according to the operation signal PSa, and the camera processing circuit 131 and the timing. This is supplied to the generator 142.

  The audio processing circuit 132 is supplied with an analog audio signal Sin from the audio input device 20. The audio processing circuit 132 performs sampling processing of the audio signal Sin based on the timing signal CT supplied from the timing generator 142, generates digital audio data DA, and supplies the digital audio data DA to the output unit 15.

  The output unit 15 generates the material data DTm by connecting the auxiliary information DM to the video data DV and the audio data DA, and supplies the material data DTm to the editing device 30. If the material data DTm or the recording signal generated based on the material data DTm is recorded on the recording medium, the material data DTm or the material data DTm recorded on the recording medium is reproduced by the editing device 30, thereby recording the recording medium. The material data DTm can be supplied to the editing apparatus 30 via the.

  Here, as an example of a method for connecting the auxiliary information DM to the video data DV and the audio data DA, when generating the material data DTm as a data stream by compressing the video data DV and the audio data DA, It is conceivable to insert the accessory information DM into the header or insert the accessory information DM into the header of the data stream.

  In addition, SMPTE (Society of Motion Picture and Television Engineers) 259M “Television-10-Bit 4: 2: 2 Component and 4fsc Composite Digital Signals-Serial Digital Interface” is used to transmit uncompressed video and audio data. SDI format, SDTI format standardized as SMPTE305M “Television-Serial Data Transport Interface (SDTI)” for transmitting compressed video data and audio data, and SDTI format are further limited When the SDTI-CP format standardized as SMPTE 326M “Television-SDTI Content Package Format (SDTI-CP)” is used, the attached information DM is the UMID standardized as SMPTE 330M “Television-Unique Material Identifier (UMID)”. Data can be inserted into each format signal Erareru. Note that the method of connecting the auxiliary information DM to the video data DV and the audio data DA is not limited to this, and various methods are conceivable.

  By the way, the imaging device 10 described above changes the readout period of the imaging charge in the imaging unit 12 so that the imaging signal Sp at the set frame rate FRs can be obtained. However, the imaging signal Sp at the set frame rate FRs can be obtained without changing the readout period of the imaging charge in the imaging unit 12. That is, by generating video data DVa having a frame rate higher than the set frame rate FRs and having a constant frame rate, and extracting video data from the video data DVa by the set frame rate FRs, the set frame rate is set. FRs video data DV can be generated. The configuration in this case is shown in FIG. 3, parts corresponding to those in FIG. 2 are denoted by the same reference numerals and detailed description thereof is omitted.

  The timing generator 182 of the control unit 18 generates a drive signal CRa corresponding to the maximum value of the set frame rate FRs set via the user interface unit 16 and supplies it to the imaging unit 12. The imaging unit 12 generates an imaging signal based on the drive signal CRa, generates an imaging signal Spa having a fixed frame rate FRq whose frame rate is higher than the reference frame rate FRr, and sends it to the camera processing circuit 131 of the signal processing unit 17. Supply. For example, when the set frame rate FRs can be changed up to n (n is a natural number) times the reference frame rate FRr, an imaging signal Spa having a frame rate n times the reference frame rate FRr is generated, and the camera processing circuit 131 To supply. That is, the imaging unit 12 generates the imaging signal Spa having a fixed frame rate without being affected by the set frame rate FRs set via the user interface unit 16.

  The timing generator 182 generates a timing signal CTa synchronized with the drive signal CRa and supplies the timing signal CTa to the camera processing circuit 131, the audio processing circuit 132, and the effective frame signal generation circuit 183 of the signal processing unit 17.

  The camera processing circuit 131 supplies video data DVa having a fixed frame rate FRq generated based on the imaging signal Spa to the effective data selection circuit 171. The audio processing circuit 132 supplies the effective data selection circuit 171 with audio data DAa generated by performing sampling based on the timing signal CTa having a constant frequency.

  The imaging control circuit 181 generates a setting information signal CF indicating the setting frame rate FRs based on the operation signal PSa from the user interface unit 16 and supplies the setting information signal CF to the effective frame signal generation circuit 183.

  The valid frame signal generation circuit 183 extracts data from the video data DVa in units of frames based on the ratio between the frame rate FRq, which is a constant value of the video data DVa, and the set frame rate FRs indicated by the setting information signal CF. An extraction control signal CC for generating video data DV at a set frame rate FRs is generated. Further, the valid frame signal generation circuit 183 supplies the extraction control signal CC to the valid data selection circuit 171 in synchronization with the timing signal CTa. For example, when the frame rate FRq of the video data DVa is n times the reference frame rate FRr and the set frame rate FRs is (n / 2) times the reference frame rate FRr, every other frame from the video data DVa. Then, an extraction control signal CC for performing data extraction is generated and supplied to the valid data selection circuit 171 in synchronization with the timing signal CTa. Further, the valid frame signal generation circuit 183 generates auxiliary information DM having the set frame rate FRs as frame rate information based on the setting information signal CF and supplies it to the output unit 15.

  The valid data selection circuit 171 extracts the video data DVa and audio data DAa of the frame indicated by the extraction control signal CC, and supplies them to the output unit 15 as the video data DV and audio data DA. Further, although not shown, the effective frame signal generation circuit 183 supplies the attached information DM having the set frame rate FRs as the frame rate information to the effective data selection circuit 171, and the effective data selection circuit 171 sets the set frame rate FRs. The audio data DAa may be thinned out according to the ratio of the frame rate when the audio data DAa is generated. For example, when the frame rate FRq when the audio data DAa is generated is n times the reference frame rate FRr and the set frame rate FRs is (n / 2) times the reference frame rate FRr, the audio data DAa Thin out every other sample. In this case, since the thinning interval can be made smaller than in the case where the audio data is thinned out in units of frames, the sound based on the sound data DA can be improved in sound quality.

  Thus, by making the frame frequency of the video data DVa constant, it is not necessary to vary the operating frequency in the camera processing circuit 131 of the imaging unit 12 or the signal processing unit 17, and the imaging unit 12 or the camera processing circuit 131 The configuration can be simplified. Further, since the video data DV having the set frame rate FRs can be generated simply by extracting data from the video data DVa in units of frames, the video data DV having the desired set frame rate FRs can be easily generated from the video data DVa.

  Further, the image data DV may be generated by adding the video data for every predetermined (predetermined) frame, assuming that the imaging device is provided with an image memory, an adder, and a divider. In this case, the frame rate variable range of the imaging signal Sp can be narrowed. That is, if the image signal Sp for n frames is added and the signal level is multiplied by (1 / n), the frame rate is (1 / n) even if the frame rate of the image signal Sp is not (1 / n) times. A doubled signal can be obtained.

  4 and 5 are diagrams for explaining the relationship between the video data DV generated by the imaging devices 10 and 10a and the attached information DM. As shown in FIG. 4A, when the set frame rate FRs is, for example, 1 or 2 times the reference frame rate FRr, the video data DV shown in FIG. 4B (the frame image based on the video data DV is shown in the figure). Thus, the attached information DM including the frame rate information DM-FRs of FIG. 4C indicating the set frame rate FRs is connected. FIG. 4D shows the relationship between the passage of time and the frame image. Further, the frame rate information DM-FRs may indicate not only the set frame rate FRs but also the magnification of the set frame rate FRs with respect to the reference frame rate FRr. In the frame rate information DM-FRs shown in FIG. 4C and the following figures, the magnification is described.

  As shown in FIG. 5A, when the set frame rate FRs is, for example, 1 or 1/2 times the reference frame rate FRr, the video data DV shown in FIG. 5B (the frame image based on the video data DV is shown in the figure). In contrast, attached information DM including the frame rate information DM-FRs of FIG. 5C indicating the set frame rate FRs is connected. FIG. 5D shows the relationship between the passage of time and the frame image.

  Next, the editing device 30 will be described. FIG. 6 shows the configuration of the editing device 30. The material data DTm supplied to the editing device 30 is supplied to the information detection circuit 311 of the material take-in unit 31 serving as detection means. The information detection circuit 311 detects the attached information DM from the material data DTm. The detected accessory information DM is supplied to the database processing circuit 312. The video data DV and audio data DA included in the material data DTm are supplied to the database processing circuit 312.

  The database processing circuit 312 associates the video data DV, the audio data DA, and the auxiliary information DM detected by the information detection circuit 311 and stores them in the data storage device 321 of the editing processing unit 32. The database processing circuit 312 easily confirms the content of the material data based on the auxiliary information DM stored in the data storage device 321 and the video data DV and audio data DA related to the auxiliary information DM. The database information DB to be enabled is generated and supplied to the editing control unit 33. For example, the database information DB is composed of information (such as thumbnails) that makes it possible to identify the content of the material data, information such as the time length of the material data, the set frame rate FRs, and the storage location in the data storage device 321. .

  The editing control unit 33 generates video data DVg for enabling editing processing in a GUI (Graphical User Interface) environment and video data DVi for displaying the contents of database information, and supplies the video data DVi to the video output signal generation circuit 351. Supply. The video output signal generation circuit 351 generates a video signal Svm based on the supplied video data DVg and DVi and outputs it to the edited image display device 40. In this way, by supplying the video signal Svm to the edited image display device 40, what kind of material data is stored can be displayed on the screen of the edited image display device 40. This video signal Svm becomes a display image for editing.

  The edit control unit 33 controls post-production processing. That is, the user interface unit 34 connected to the editing control unit 33 is supplied with the operation signal PSe using display in the GUI environment, and indicates that any material data is selected by the operation signal PSe. Sometimes, a read control signal RC corresponding to the operation signal PSe is generated and supplied to the writing / reading processing circuit 322 of the editing processing unit 32. When the operation signal PSe relates to editing operations such as processing and combining of the read material data, an edit control signal ET corresponding to the operation signal PSe is generated and supplied to the signal editing circuit 323 of the edit processing unit 32. . Further, when the editing of the material data is completed and the content data is completed, when the operation signal PSe indicates an operation for storing the content data in the data storage device 321, the write control signal WC corresponding to the operation signal PSe is displayed. Is supplied to the writing / reading processing circuit 322. When the operation signal PSe indicates the output of content data, an output control signal RP corresponding to the operation signal PSe is generated and supplied to the writing / reading processing circuit 322. When the operation signal PSe defines the playback speed range of the content data, a speed range setting signal LP corresponding to the operation signal PSe is generated and supplied to the signal editing circuit 323.

  Based on the read control signal RC, the write / read processing circuit 322 reads the requested material data from the data storage device 321 and supplies it to the signal editing circuit 323. Further, the completed content data DC is stored in the data storage device 321 based on the write control signal WC. Further, based on the output control signal RP, the requested content data DC is read from the data storage device 321 and output.

  The signal editing circuit 323 uses the video data DV and audio data DA included in the material data read from the data storage device 321 to perform editing processing such as image and audio processing, combination, and deletion on the editing control signal ET. Based on. Here, the signal editing circuit 323 supplies the video data DVe before, during or after editing to the video output signal generation circuit 351, and the audio data DAe before, during or after editing as an audio output signal generation circuit. 352. Further, when the signal editing circuit 323 changes the frame rate of the video data DV or the audio data DA by the editing process, the signal editing circuit 323 also changes the attached information DM according to the edited video data or audio data. Further, the edited video data DV and audio data DA and the attached information DMc corresponding to the edited video data DV and audio data DA are connected to generate content data DC. When the speed range setting signal LP is supplied, speed range information indicating the reproduction speed range of the content data DC is also connected as the auxiliary information DMc based on the speed range setting signal LP. Further, when the content title and the recommended playback speed of the content are input from the user interface unit 34, these pieces of information are also linked as attached information DMc. Further, when the reproduction time length information of the content data is obtained by the editing process, this information may be linked as the attached information DMc. Furthermore, when the maximum speed at which the content data can be reproduced is input, this maximum speed is also linked as the attached information DMc.

  As described above, the video output signal generation circuit 351 of the editing output signal generation unit 35 generates the video signal Svm based on the video data DVg and DVi supplied from the editing control unit 33 and supplies the video signal Svm to the edited image display device 40. . For this reason, information regarding material data can be displayed in the GUI environment. Furthermore, by generating the video signal Svm based on the video data DVe supplied from the signal editing circuit 323, it is possible to check the image before editing, during editing, or after editing on the screen of the edited image display device 40.

  The audio output signal generation circuit 352 converts the audio data DAe supplied from the signal editing circuit 323 into an analog audio signal Sam and uses an edited audio output device configured as a desired signal level using, for example, a speaker or headphones. 41. For this reason, the voice before editing, during editing, or after editing can be confirmed by the voice output from the edited voice output device 41.

  In this way, when the editing apparatus 30 performs post-production processing using the material data DTm and the content data DC is completed, the completed content data DC is output.

  Next, the operation of the editing device 30 will be described. FIG. 7 shows a GUI screen for editing operation displayed on the edited image display device 40. In the GUI screen, a material management browser 401 that is a material management display showing a list of material data is provided at the upper left, and a storyboard 402 that is a reproduction order display that generates content by arranging the material data in the reproduction order at the lower left. Is provided. In the center of the screen, the monitor viewer 403 is a playback image display that displays the image of the material data before and after editing. In the lower center of the screen, the material data is pasted along the time axis to generate the content. A timeline 404 that is displayed in time order is provided. An operation control unit 405 is provided between the monitor viewer 403 and the timeline 404. Note that the arrangement and shape of the material management browser 401, the storyboard 402, the monitor viewer 403, the timeline 404, and the operation control unit 405 are illustrative and are not limited.

  The material management browser 401 shows a list of material data stored in the data storage device 321, a stamp image (thumbnail image) indicating the title, length, and content of the stored material data, noiseless and natural A reproducible speed range or the like where a reproducible image can be obtained is displayed for each material.

  The story board 402 is a place where content generation work is performed, and generates content by arranging material data in the order of reproduction. It also displays the reproducible speed range. The monitor viewer 403 not only displays an image based on the material data, but also displays a variable speed bar indicating the reproducible speed range and a position of the replay speed within the reproducible speed range.

  The timeline 404 is a place for performing content generation work in more detail by pasting material data along the time axis. Further, the display width of the material data is varied according to the reproduction time calculated based on the reproduction speed. The operation control unit 405 displays operation keys for reproducing material data and content connected on the timeline 404.

  Here, when the editing control unit 33 determines that a drag & drop operation, a key operation of the operation control unit 405, or the like has been performed based on the displayed GUI image and the operation signal PSe from the user interface unit 34, the operation is performed. Accordingly, the operation of the editing device 30 is controlled.

  FIG. 8 is a flowchart illustrating an example of display control during an editing operation. In step ST11, it is identified whether or not a material data processing request has been made. If no processing request is made, the process returns to step ST11. If a processing request is made, the process proceeds to step ST12.

  In step ST12, it is identified whether or not frame rate information DM-FRs indicating the set frame rate FRs is obtained from the material data to be edited. Here, referring to the database information DB, when the frame rate information DM-FRs is not obtained from the material data, the process proceeds to step ST13, and display control is performed assuming that the material data is 1 × speed data. If the frame rate information DM-FRs is obtained from the material data, the process proceeds to step ST14.

  In step ST14, the reproducible speed is set based on the set frame rate FRs indicated by the frame rate information DM-FRs. In the setting of the reproducible speed, the reproducible speed is set so that a natural reproduced image can be obtained without noise by performing thinning at a constant frame interval or by repeating the frame. For example, when a multiple of the set frame rate FRs with respect to the reference frame rate FRr is larger than 1, a value other than 1 is obtained as a divisor of this multiple, and the reproducible speed is set by calculating the reciprocal of the obtained divisor. it can. That is, when the set frame rate FRs is 10 times the reference frame rate, the divisor other than 1 is “2, 5, 10”, and the reciprocal of this divisor “1/10, 1/5, 1/2”. It becomes. By selecting the reciprocal of this divisor, a reproducible speed less than 1 × speed can be set. Further, by setting the reproducible speed of 1 × speed or more to a natural number multiple, the thinning intervals performed at each playback speed can be made equal. The upper limit of the reproducible speed is limited to a speed at which the content of the material data can be easily grasped, for example.

  When the multiple of the set frame rate FRs with respect to the reference frame rate FRr is smaller than 1, the reproducible speed can be set by calculating the reciprocal of this multiple and calculating the divisor other than 1 or the integral multiple of the reciprocal. . That is, when the set frame rate FRs is (1/6) times the reference frame rate, the divisor other than 1 is “2, 3, 6”, and the integral multiple of the reciprocal is “6, 12, 18, "..." If the reciprocal number other than 1 or an integral multiple of the reciprocal number is set as the reproducible speed, the thinning interval or the number of repetitions performed at each reproduction speed can be made equal. Further, even if the playback speed is set to (1 / FRs) or less when the multiple of the set frame rate FRs is smaller than 1, the playback image is a frame image repeated as many times as necessary. It is good also as 1 time speed.

  In step ST15, display control according to the frame rate information DM-FRs, that is, a display for identifying that the frame rate information DM-FRs indicating the set frame rate FRs is material data linked as the auxiliary information DM, The reproducible speed range set based on the set frame rate FRs is displayed. For example, in the material management browser 401 and the storyboard 402, the shape and color of the image frames of the stamp images 401a and 402a, etc. in order to make it easy to identify that the information of the set frame rate FRs is connected material data. Is different from the material to which the information of the set frame rate FRs is not linked. Further, when information display areas 401b and 402b for displaying information regarding the material are provided, the reproducible speed range set in step ST14 is displayed in this area. For example, in FIG. 7 and FIG. 9 to be described later, when the information of the set frame rate FRs is concatenated material data, the image frame is displayed with a wider width and the reproducible speed range is “× 0.1. ˜ × 2.0 ”.

  Similar to the material management browser 401 and the story board 402, the display of the timeline 404 can easily identify that the frame rate information DM-FRs indicating the set frame rate FRs is material data linked as the attached information DM. To do. For example, it is assumed that the shape and color of the image frame in the frame image display area 404a are different from the material not connected with the frame rate information DM-FRs of the set frame rate FRs. Further, in the timeline 404, the frame image display area 404a is adjusted in the horizontal direction, which is the time axis direction, according to the set frame rate FRs. For example, when the set frame rate FRs is larger than the reference frame rate FRr, the number of frame images per unit time increases. For this reason, the display width of the frame image display area 404a is increased. When the set frame rate FRs is smaller than the reference frame rate FRr, the number of frame images per unit time is reduced. For this reason, the frame image display area 404a is narrowed. Further, the monitor viewer 403 is provided not only with a material image display 403a based on the material data but also with a playback speed display 403b and a speed variable console display 403c corresponding to the set reproducible speed range so that a playback image and a playback image are displayed. Displays the speed and playable speed range. Also, the controllable direction can be identified by the cursor position indicated by the bold line on the variable speed console display 403c.

  In step ST16, it is identified whether or not the playback speed is instructed via the user interface unit 34. Here, when an operation for changing the display width of the frame image display area 404a in the timeline 404 is performed, or when the cursor position of the variable speed console display 403c displayed on the monitor viewer 403 is moved, the process proceeds to step ST17. If the playback speed is not instructed, the process returns to step ST16.

  In step ST17, the instructed playback speed is identified, and the identified playback speed is displayed. For example, when an operation for changing the display width of the frame image display area 404a in the timeline 404 is performed, the playback speed is sequentially selected from the playable speeds set in step ST14 according to the operation direction, and the speed at the end of the operation. Is the playback speed. When the cursor position of the variable speed console display 403c displayed on the monitor viewer 403 is moved, a speed corresponding to the cursor position is selected from the reproducible speeds set in step ST14 and set as a playback speed.

  When the playback speed is identified, the playback speed display 403b of the monitor viewer 403 is set as the instructed playback speed. When the playback speed is instructed by an operation of changing the display width of the frame image display area 404a by linking the cursor position of the speed variable console display 403c with the display width of the frame image display area 404a, the variable speed console display is displayed. By moving the cursor position of 403c to a position corresponding to the instructed playback speed, the playback speed is displayed, and when the playback speed is instructed by an operation for changing the cursor position of the speed variable console display 403c, a frame image is displayed. The display width of the display area 404a is changed to a width corresponding to the corresponding playback speed. In this way, by linking the cursor position of the speed variable console display 403c and the display width of the frame image display area 404a, the playback speed can be correctly varied regardless of which is used.

  In step ST18, the playback time when the material data is played back at the instructed playback speed is calculated from the instructed playback speed. For example, as described above, when the set frame rate FRs is 10 times the reference frame rate, the reproducible slow speed is any one of “1/10, 1/5, 1/2”. Here, when the playback time of the material is 30 seconds when the playback speed is 1 ×, the playback time when playback is 1/10 speed is 300 seconds, and when the playback speed is 1/5 ×, the playback time is 150 seconds. Thus, the reproduction time can be calculated by multiplying the reproduction time at the normal speed by the reciprocal of the double speed at the time of reproduction.

  In step ST19, display control according to the reproduction time is performed. For example, the display width of the frame image display area 404a on the timeline 404 is varied according to the reproduction time calculated in step ST18, and the process returns to step ST16. Further, during the reproduction operation of the material data, a reproduction image when the material data is reproduced at the designated reproduction speed is displayed as the material image display 403a of the monitor viewer 403.

  When changing the playback time of a part of scenes in the material data, the playback time can be varied by dividing the part of the scene and performing the above-described processing on the divided scene. .

  FIG. 9 shows an editing operation screen when using material data to which frame rate information DM-FRs indicating that the set frame rate FRs is 10 times the reference frame rate FRr, for example, is used. FIG. 9A is a GUI screen when the playback speed is 1 ×, FIG. 9B is (1/10) speed, and FIG. 9C is 2 × speed. The material data to which the information of the set frame rate FRs is linked is, for example, a storyboard In 402 or the timeline 404, an image frame display different from the material data not connected with the frame rate information DM-FRs indicating the set frame rate FRs is performed.

  Here, when the selected material data is reproduced, the set frame rate FRs is identified based on the frame rate information DM-FRs, and the identification value FD is calculated by multiplying the reproduction speed FP and the set frame rate FRs. A reproduction processing condition is determined based on the identification value FD. For example, when the playback speed is set to 1 × speed and the set frame rate FRs is set to 10 × speed with respect to the reference frame rate FRr as shown in the screen display of FIG. 9A described above, the identification value FD is “10”. × 1 = 10 ”. FIG. 10A shows an image based on the video data DV when the set frame rate FRs is 10 times the reference frame rate FRr. 10B shows frame rate information DM-FRs indicating a set frame rate FRs of the frame image, and FIG. 10C shows an absolute frame number AN.

  When the identification value FD is “FD = 10”, as shown in FIGS. 10D and 10E, the video signal Svm is generated using every “FD = 10” frame, that is, while skipping the video data DV by 9 frames. Thus, a 1 × speed playback image can be displayed on the monitor viewer 403 based on the video signal Svm. 10D shows the absolute frame number AN of the displayed image, and FIG. 10E shows the frame image displayed on the material image display 403a of the monitor viewer 403 by the video signal Svm.

  Next, when an operation for expanding the display width of the frame image display area 404a in the timeline 404 in the direction of arrow A or an operation for moving the cursor position of the speed variable console display 403c in the direction of arrow B is performed, the playback speed is decreased. Treat as the operation was performed. Here, for example, when (1/10) double speed is set, as shown in FIG. 9B, the playback speed display 403b of the monitor viewer 403 is changed to (1/10) double speed. In addition, the cursor position on the variable speed console display 403c is set to a position of (1/10) double speed. Furthermore, since the reproduction time becomes longer, the display width of the frame image display area 404a is increased.

  As shown in the screen display of FIG. 9B, in the reproduction at the reproduction speed of (1/10) times faster, the identification value FD becomes “10 × (1/10) = 1”. When the identification value FD is “FD = 1”, the video signal Svm is generated using every frame of the “FD = 1” frame, that is, the video data DV, so that the (1/10) times speed shown in FIG. The reproduced image can be displayed on the material image display 403 a of the monitor viewer 403.

  Further, when an operation for reducing the display width of the frame image display area 404a in the timeline 404 in the direction of arrow B or an operation for moving the cursor position of the speed variable console display 403c in the direction of arrow A is performed, an operation for increasing the playback speed. It is treated as having been made. Here, for example, when the double speed is set, as shown in FIG. 9C, the reproduction speed display 403b of the monitor viewer 403 is changed to the double speed. Further, the cursor position on the variable speed console display 403c is set to the double speed position. Further, since the reproduction time is shortened, the display width of the frame image display area 404a is narrowed.

  As shown in the screen display of FIG. 9C, the identification value FD is “10 × 2 = 20” in the reproduction at the reproduction speed of 2 ×. When the identification value FD is “FD = 20”, as shown in FIGS. 10F and 10G, the video signal Svm is generated using every “FD = 20” frame, that is, while skipping the video data DV by 19 frames. Thus, a double-speed playback image can be displayed on the material image display 403a of the monitor viewer 403 based on the video signal Svm. 10F shows the absolute frame number AN of the displayed image, and FIG. 10G shows the frame image displayed on the material image display 403a of the monitor viewer 403 by the video signal Svm.

  Thus, noiseless slow reproduction can be easily realized by performing thinning out in units of frames based on the frame rate information DM-FRs and the reproduction speed. Note that the playback speed can be changed during the playback operation, so that a playback image at a desired playback speed can be easily confirmed.

  In addition, since the reproduction speed and the display width are displayed in association with each other, the relation between the reproduction speed and the expansion / contraction of the time axis can be easily grasped, and the editing operation can be easily performed. It should be noted that if the frame image display area 404a is provided with a display indicating the direction in which the display width can be changed, that is, the direction in which the playback speed can be varied, for example, an arrow display indicated by a broken line, the direction in which the playback speed can be changed is easily Can be identified.

  FIG. 11 is a flowchart showing the editing processing operation using the editing screen displayed as shown in FIG. 9, for example, with the display control shown in FIG. In step ST21, material data is fetched, the fetched material data is stored in the data storage device 321 and a database information DB is generated.

  In step ST22, the contents of the fetched material data are displayed using the database information DB. The material data contents are displayed by, for example, displaying a stamp image or information of the captured material data on the material management browser 401. When any of the material data displayed on the material management browser 401 is selected, an image of the selected material data is displayed on the monitor viewer 403. Further, when the playback control key display is operated by the operation control unit 405 provided between the monitor viewer 403 and the timeline 404, playback and stop of the material data, variable speed playback, etc. are performed according to the key operation, and the playback image is displayed. Is displayed on the monitor viewer 403.

  In step ST23, it is determined whether or not material data has been selected. Here, when the material data is not selected, the process returns to step ST23, and when the material data is selected, the process proceeds to step ST24. For example, when dragging and dropping from the material management browser 401 to the timeline 404, the process proceeds to step ST24 assuming that the material data is selected.

  In step ST24, frame images based on the selected material data are displayed side by side in time order in the frame image display area 404a of the timeline 404.

  In step ST25, the material data is processed in accordance with an editing operation such as cutting, pasting, or rearranging the frame image displayed on the timeline 404, and desired video data or audio data is generated and used as content data. In addition, the video data and audio data are connected to the attached information DMc indicating the set frame rate and the like to obtain content data.

  In step ST26, content data completion processing is performed, and processing such as processing of the connecting portion of the material data, addition of a presentation effect, or synthesis of an image or sound is performed on the content data generated in step ST25. In step ST27, content data output processing is performed. That is, the completed content data is supplied to the program transmission apparatus as broadcast program data. Alternatively, it is supplied as content data for distribution to a content transmission apparatus. In addition, authoring such as adding content reproduction menu list or information for performing special reproduction according to the reproduction menu to the completed content data is performed, and the data that has been authored is recorded on the recording medium.

  Incidentally, the editing process of the editing device 30 described above can also be realized by software processing using a computer. FIG. 12 shows the configuration when content editing is performed with this software.

  The computer incorporates a CPU (Central Processing Unit) 381 as shown in FIG. 12, and this CPU 381 uses a ROM 382, RAM 383, a hard disk drive having a large storage capacity, etc. via a bus 390. The data storage unit 384 and the input / output interface 385 configured as described above are connected. Further, a signal input unit 391, a signal output unit 392, and a recording medium drive 393 are connected to the input / output interface 385.

  The CPU 381 executes the program stored in the ROM 382, the RAM 383, or the data storage unit 384, and performs the editing process shown in FIGS. The material data input to the signal input unit 391 is stored in the data storage unit 384 via the input / output interface 385 and the bus 390. Further, the material data used for editing is read from the material data stored in the data storage unit 384 and editing processing is performed, and the completed content data is stored in the data storage unit 384 again. The completed content data stored in the data storage unit 384 is read out and output via the signal output unit 392.

  The program for performing the editing process is stored in advance in the ROM 382 or the data storage unit 384, or the program for content transmission processing is recorded on the recording medium or recorded on the recording medium by the recording medium drive 393. The program may be read and executed. Furthermore, the program may be transmitted via a wired or wireless transmission path, and the received program may be executed by a computer.

  As described above, according to the above-described embodiment, since the material data that is not connected to the material data to which the frame rate information is connected as the attached information is displayed in an identifiable manner, the frame rate information of the material data is used. You can easily find a scene that can be played slowly. In addition, since the reproducible speed range is displayed based on the attached information, effective video expression can be easily performed by setting the speed within this range and performing variable speed playback.

  If the editing apparatus 30 generates content data without reducing the frame rate of the material data, the editing device 30 can provide content in which the material data is effectively used and the variable range of the reproduction speed is wide.

  As described above, the edited image display control apparatus, the edited image display control method, and the program according to the present invention are useful when editing using the image material data, and the material data whose frame rate is variable are used. It is suitable for use.

It is a figure which shows the whole structure of a content editing system. It is a figure which shows the structure of an imaging device. It is a figure which shows the other structure of an imaging device. It is a figure which shows the relationship (the 1) of the attached information of video data. It is a figure which shows the relationship (the 2) between video data and attached information. It is a figure which shows the structure of an editing apparatus. It is a figure which shows the GUI screen for editing operation. It is a flowchart which shows the display control at the time of editing operation. It is a figure which shows the GUI screen at the time of editing operation. It is a figure which shows image reproduction operation | movement. It is a flowchart which shows an edit process operation. It is a figure which shows the structure in the case of performing content editing with software.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10, 10a ... Imaging device, 12 ... Imaging part, 13, 17 ... Signal processing part, 14, 18 ... Control part, 15 ... Output part, 16, 34 ... User interface , 20 ... voice input device, 30 ... editing device, 31 ... material capture unit, 32 ... editing processing unit, 33 ... signal editing unit, 35 ... editing output signal generation 40: Editing image display device, 41 ... Editing audio output device, 131 ... Camera processing circuit, 132 ... Audio processing circuit, 141, 181 ... Imaging control circuit, 142, 18 ..Timing generator, 171... Valid data selection circuit, 183... Valid frame signal generation circuit, 311... Information detection circuit, 312 ... Database processing circuit, 321. ... Write / read processing Circuit 323 Signal editing circuit 351 Video output signal generation circuit 352 Audio output signal generation circuit 384 Data storage unit 391 Signal input unit 392 Signal output unit 401 ... Material management browser 402 ... Storyboard 403 ... Monitor viewer 404 ... Timeline 405 ... Operation control unit

Claims (25)

Detecting means for detecting whether or not frame rate information is linked to the material data as attached information;
Control the state of the display image for editing so that the material data to which the frame rate information is linked as auxiliary information and the material data to which the frame rate information is not linked as auxiliary information are identified and displayed And an editing image display control device. The control means, in the material management display showing a list of the material data, the material data to which the attached information is linked and the material data to which the attached information is not linked are displayed in an identifiable manner. 2. The edited image display control device according to claim 1, wherein the display image state is controlled. In the reproduction order display for generating the content by arranging the material data in the reproduction order, the control means can identify the material data to which the attached information is linked and the material data to which the attached information is not linked. The edited image display control apparatus according to claim 1, wherein the display image state is controlled so as to be displayed. In the display in order of reproduction time for creating the content by pasting the material data along the time axis, the control means includes the material data to which the attached information is connected and the material to which the attached information is not connected. The edited image display control apparatus according to claim 1, wherein the display image state is controlled such that the data is displayed in an identifiable manner.   The control means determines a reproducible speed range of the material data based on the attached information, and controls the state of the display image so that the determined reproducible speed range is displayed. Item 2. The edited image display control device according to Item 1. 6. The editing according to claim 5, wherein the control means controls the state of the display image so that the determined reproducible speed range is displayed in the material management display showing a list of the material data. Image display control device. The control means controls a state of the display image so that the determined reproducible speed range is displayed in a reproduction order display for generating the content by arranging the material data in the reproduction order. The edited image display control device according to claim 5. 6. The edited image display control according to claim 5, wherein the control means controls the state of the display image so that the determined reproducible speed range is displayed in the reproduction image display of the material data. apparatus. 9. The edited image display control apparatus according to claim 8, wherein a position display indicating a reproduction speed is included in the display of the reproducible speed range.   In the display in order of playback time for generating the content by pasting the material data along the time axis, the control means has a display width of the material data corresponding to the playback time calculated based on the playback speed. 6. The edited image display control apparatus according to claim 5, wherein the state of the display image is controlled so as to be variably displayed. In the reproduction image display of the material data, the control means displays the determined reproducible speed range, a position display indicating a replay speed is provided in the display of the reproducible speed range, and the control unit displays the position along the time axis. In the playback time order display for pasting material data and generating content, the display width of the material data is variably displayed according to the playback time calculated based on the playback speed, and the playback time order display The display position of the position display is moved in conjunction with the operation for changing the display width of the material data in the display, and the display width of the material data in the playback time order display according to the operation for changing the display position of the position display 6. The edited image display control apparatus according to claim 5, wherein the state of the display image is controlled so that the variable is variable. The control means is configured to display the material data so that the material data reproduced at the designated speed is displayed when a material data reproduction operation is designated at a reproduction speed within the reproducible speed variable range. 6. The edited image display control apparatus according to claim 5, wherein the state is controlled. A detection step for detecting whether or not frame rate information is linked to the material data as attached information;
Control the state of the display image for editing so that the material data to which the frame rate information is linked as auxiliary information and the material data to which the frame rate information is not linked as auxiliary information are identified and displayed An edited image display control method. In the display control step, in the material management display showing a list of the material data, the display is performed so that the material data to which the attached information is linked and the material data to which the attached information is not linked are displayed in an identifiable manner. 14. The edited image display control method according to claim 13, wherein the state of the image is controlled. In the display control step, in the reproduction order display for generating the content by arranging the material data in the reproduction order, the material data to which the attached information is linked and the material data to which the attached information is not linked can be identified. 14. The edited image display control method according to claim 13, wherein the state of the display image is controlled so as to be displayed on the screen. In the display control step, in the reproduction time order display for pasting the material data along the time axis to generate content, the material data to which the auxiliary information is connected and the auxiliary information are not connected. 14. The edited image display control method according to claim 13, wherein the state of the display image is controlled so that the material data is displayed in an identifiable manner. In the display control step, a reproducible speed range of the material data is determined based on the attached information, and the state of the display image is controlled so that the determined reproducible speed range is displayed. The edited image display control method according to claim 13. 18. The state of the display image is controlled in the display control step so that the determined reproducible speed range is displayed in a material management display showing a list of the material data. Edited image display control method. In the display control step, the state of the display image is controlled so that the determined reproducible speed range is displayed in a reproduction order display in which the material data is arranged in the reproduction order to generate content. The edited image display control method according to claim 17. 18. The edited image display according to claim 17, wherein in the display control step, the state of the display image is controlled so that the determined reproducible speed range is displayed in the reproduction image display of the material data. Control method. 21. The edited image display control method according to claim 20, wherein a position display indicating a reproduction speed is included in the display of the reproducible speed range. In the display control step, in the reproduction time order display for pasting the material data along the time axis to generate the content, the display width of the material data corresponds to the reproduction time calculated based on the reproduction speed. 18. The edited image display control method according to claim 17, wherein the state of the display image is controlled so as to be changed and displayed. In the display control step, in the reproduction image display of the material data, the determined reproducible speed range is displayed, and a position display indicating the replay speed is provided in the display of the reproducible speed range, along the time axis. In the playback time order display for pasting the material data and generating content, the display width of the material data is variably displayed according to the playback time calculated based on the playback speed, and the playback time order display The display position of the position display is moved in conjunction with the operation for changing the display width of the material data in the display, and the display width of the material data in the playback time order display according to the operation for changing the display position of the position display The edited image display control method according to claim 17, wherein the state of the display image is controlled so that the variable is variable. In the display control step, when a material data reproduction operation is instructed at a reproduction speed within the reproducible speed range, the state of the display image is set so that the material data reproduced at the instructed speed is displayed. 18. The edited image display control method according to claim 17, wherein the edited image display control method is controlled. Detecting whether or not frame rate information is linked to the material data as attached information;
Control the state of the display image for editing so that the material data to which the frame rate information is linked as auxiliary information and the material data to which the frame rate information is not linked as auxiliary information are identified and displayed A program that causes a computer to execute the step of

Images