CN100505858C - Image data processing apparatus and image data processing method - Google Patents

Abstract

The invention discloses a method for processing image data composing video materials, comprising: obtaining step, used to obtain image data at many time points from the video materials; generating step, used to generate image data and used to display the image data obtained in the obtaining step in packet state in a display region corresponding to the other display region arranged with plural material action plates.

Description

Image data processing system and image processing method

Technical field

The present invention relates to the display packing (method for displaying image) of the storyboard window of the user interface that a kind of conduct uses in editing device, and a kind of image processing apparatus that is used to implement this display packing.

In addition, the invention still further relates to a kind of program that is used to implement this display packing, and the recording medium that is used to store this program.

Background technology

Fig. 1 shows the embodiment of the screen of the graphic user interface (hereinafter referred to as " GUI ") that is used for video editing.On this editing screen, be provided with some window screens.For example, on this editing screen, be provided with monitor window 1, storyboard window 3 and timeline window 5.

Monitor window 1 is the viewing area that is used to grasp video material (fragment), preview, edit operation, inspection edited result etc.Monitor window 1 comprises source screen 1A and program screen 1B.

Source screen 1A is used to grasp fragment, preview etc.In addition, program screen 1B is used to edit the operation of fragment or program (for example, by the tactic a plurality of fragments of regeneration), inspection edited result etc.

In addition, according to user's selection, can only show in source screen and the program screen.

Storyboard window 3 is the viewing areas that are used to set up the plot of program.This viewing area is used to show the typical frame 3A (being generally the point frame that rises of fragment) of each fragment.

The user can be by selecting corresponding to the frame of each fragment and determine to put in order to create the plot of program.It is noted that each typical frame is all corresponding to a fragment.

Timeline window 5 is the viewing areas that are used for the editing saving operation.Equally, timeline window 5 can be used for also checking that the conversion effect of each fragment and the track in the whole program synthesize.

Timeline window 5 comprises timeline 5A, track of video 5B and audio track 5C.

Timeline window 5 is from the passing of screen left-to-right express time.Therefore, the fragment that is arranged in each track begins to regenerate successively from the left side.

In addition, the method for arrangement fragment comprises by drag-and-drop operation independent method of arranging the method for fragment and arranging whole fragments in storyboard window 3 in monitor window 1 in timeline window 5.

Timeline 5A is presented on the engineer's scale selected in the timeline window 5.Therefore, the length of the time that shows on each track of each fragment and engineer's scale is proportional.That is, large-sized fragment shows the relatively long time, and undersized fragment shows the relatively short time.

Fig. 2 A, 2B and 2C show the embodiment of the display format of each track of formulating in timeline window 5.

For example, when selecting the display format of Fig. 2 A, each fragment display file name.

For example, when selecting the display format of Fig. 2 B, each fragment shows typical frame (being generally start frame) and filename.

For example, when selecting the display format of Fig. 2 C, each fragment has shown point frame, ending frame and filename.

[patent documentation 1]

Japan's publication Hei11-146338 number

[patent documentation 2]

Japan's publication Hei11-284957 number

Summary of the invention

As mentioned above, in storyboard window 3, only arranged typical frame according to playing sequence.Therefore, storyboard window 3 is the user interfaces that can be used to grasp the plot summary.But in storyboard window 3, each fragment has only shown a typical frame.Therefore, when comprising a plurality of scene in the fragment, can produce difference between plot of grasping by typical frame and the actual reproduction content.As mentioned above, track of video 5B provides and can show a plurality of typical frame display formats in a fragment.Yet the demonstration size of typical frame depends on the engineer's scale of timeline 5A and the length of fragment.Therefore, under the situation of small scale or short-movie segment length, it is very little that the viewing area becomes, and be difficult to content is checked.

The present invention is devoted to above-mentioned technical problem, the technical method below having proposed.

According to the embodiment of the present invention, a kind of image processing method that the view data that constitutes video material is handled of being used for is provided, and this image processing method comprises: the obtaining step that obtains the view data of a plurality of time points that constitute video material from video material; And the generation step that generates the view data that is used to show, be used for the storyboard viewing area of having arranged a plurality of materials in the corresponding viewing area of video material in, the view data at a plurality of time points place that is presented in the obtaining step to be obtained with Packet State.

In addition, no matter the length of fragment how, each thumbnail all shows with identical size.

By using, can check the plot of whole program and the plot in each fragment simultaneously according to technology of the present invention.Thereby can grasp the plot of whole program more accurately.

Description of drawings

Fig. 1 shows the embodiment of the GUI screen that is used for video editing;

Fig. 2 A, 2B, 2C show the method that shows fragment in the timeline window;

Fig. 3 shows the demonstration embodiment of storyboard window;

Fig. 4 A and 4B show the demonstration embodiment when the thumbnail at two time point places that show random appointment;

Fig. 5 A, 5B have shown the thumbnail of point frame and any demonstration embodiment that is bonded to each other of the thumbnail of the time point of appointment;

Fig. 6 A and 6B have shown the thumbnail of point frame and ending frame and any demonstration embodiment that is bonded to each other of the thumbnail of fixed time point;

Fig. 7 A and 7B have shown the thumbnail of point frame and the demonstration embodiment that scene change thumbnail partly is bonded to each other;

Fig. 8 A and 8B show from fragment and begin the demonstration embodiment that is bonded to each other with the thumbnail that extracted at equal intervals goes out;

Fig. 9 A, 9B, 9C and 9D are the operating effect schematic diagram that aid illustration is edited the timeline window in storyboard window;

Figure 10 A and 10B have shown the demonstration embodiment that the thumbnail of the thumbnail of point frame and ending frame is bonded to each other;

Figure 11 A and 11B show the demonstration embodiment when the time difference between the thumbnail is shown as marginal live width;

Figure 12 A and 12B show the demonstration embodiment when the time difference between the thumbnail is shown as the width of clear area;

Figure 13 A and 13B show the demonstration embodiment when the time difference between the thumbnail is shown as marginal color;

Figure 14 A and 14B are the schematic diagram of the demonstration of aid illustration display format change;

Figure 15 A and 15B show the index that the number of the thumbnail that is about to be shown is shown and show embodiment when having selected detailed demonstration of segment contents;

Figure 16 A and 16B show the demonstration embodiment of the action button that is used for changing display format when only having thumbnail to show;

Figure 17 A shows index with 17B and shows the demonstration embodiment that is bonded to each other with the action button relevant with the instruction that changes display format;

Figure 18 A and 18B show the demonstration embodiment of the action button that is used for changing display format when showing a plurality of thumbnail;

Figure 19 A, 19B, 19C, 19D, 19E and 19F show the demonstration embodiment that is bonded to each other and creates new segment by two fragments;

Figure 20 shows the additional demonstration embodiment that shows the arrow of indication direction of regeneration;

Figure 21 shows the demonstration embodiment when showing fragment with the size of dwindling;

Figure 22 shows the demonstration embodiment when showing fragment with multirow;

Figure 23 shows the demonstration embodiment that uses scroll bar;

Figure 24 shows the fragment that is presented on the screen and is not presented at the schematic diagram of the relation between the fragment on this screen;

Figure 25 A and 25B show the demonstration embodiment when rearranging fragment;

Figure 26 A and 26B show the demonstration embodiment when inserting fragment;

Figure 27 shows the embodiment of the initial screen in the edit operation;

The embodiment of the screen when Figure 28 shows the selected target fragment;

Figure 29 shows the embodiment that conversion enters finishing pattern screen afterwards;

The embodiment of the screen when Figure 30 shows the select target fragment;

Figure 31 shows the embodiment that conversion enters finishing pattern screen afterwards;

Figure 32 shows the embodiment of the initial screen in the finishing pattern;

Figure 33 shows the embodiment that changes finishing target screen afterwards;

Figure 34 shows the embodiment that changes finishing target screen afterwards;

Figure 35 shows the embodiment of the screen when conversion effect is set;

Figure 36 shows the embodiment of the screen when setting is faded in;

Demonstration embodiment when Figure 37 A and 37B show before the edit validation and display format afterwards differs from one another;

Figure 38 shows the execution mode of editing device;

Figure 39 shows the execution mode of the editing device that comprises display control circuit;

Figure 40 shows the embodiment of the functional structure of editing device;

Figure 41 shows the embodiment of the processing procedure in the functional structure;

Figure 42 shows the embodiment of the functional structure of editing device; And

Figure 43 shows the embodiment of the processing procedure in the functional structure;

Embodiment

The preferred implementation of using according to the image processing apparatus of technical method of the present invention will be described below.

In addition, for the part that does not clearly illustrate or describe in this manual, use be well-known or technique known in the correlative technology field.

The execution mode that will describe is each execution mode of the present invention below, and the present invention is not limited to these execution modes.

(A) first embodiment of GUI screen

(1) is convenient to check the embodiment of the storyboard screen of plot

Below description is made the plot of the plot of whole program and sheet intersegmental part all be easy to the embodiment of the screen of the storyboard window checked.

Following execution mode all has the ability of a plurality of thumbnails that can show a fragment.

(1-1) first execution mode

Fig. 3 shows the demonstration embodiment of storyboard window 11.Among the demonstration embodiment shown in Fig. 3 corresponding to a plurality of viewing areas 13 of each fragment according to the regeneration sequence arrangement in storyboard window 11.In addition, direction of regeneration is for from left to right.

Three viewing areas 13 shown in Fig. 3 correspond respectively to Segment A, fragment B and fragment C.Each viewing area 13 all comprises title division (for example filename) 13A and sketch map part 13B.

Sketch map part 13B is the thumbnail that is used to show corresponding to a plurality of time points place of each fragment.In addition, the direction of regeneration is from top to bottom.For example, in the situation corresponding to the sketch map part 13B of Segment A, thumbnail A1, A2...AN regenerate with this order.

For each fragment, the number of the image that shows in can given sketch map part 13B.Under the situation of Fig. 3, order is N (N for more than or equal to 2 natural number), is M (M for more than or equal to 2 natural number), is L (L for more than or equal to 2 natural number) corresponding to the thumbnail number of fragment C corresponding to the thumbnail number of fragment B corresponding to the thumbnail number of Segment A.Certainly, the thumbnail number of all fragments can be identical.That is, can make N=M=L.For example, the number of thumbnail can unify to be set to 2 or 3.

No matter the size of corresponding fragment how, each thumbnail all shows with identical size.Therefore, can under identical condition, check the thumbnail of all fragments.That is, the thumbnail of small size fragment and large scale fragment has identical size.

Storyboard window that Here it is 11 can be grasped a reason of segment contents well.This storyboard window 11 makes the contents that not only are easy to grasp each fragment, can also grasp the plot of whole program at an easy rate.

In addition, when showing thumbnail on track of video 5B, the thumbnail of small size fragment may dwindle a lot, and therefore, the visuality of thumbnail will significantly reduce.

When by thumbnail of appointments such as pointer, expectation is combined in the function of this thumbnail regeneration certain hour of the front or rear preview of this thumbnail.The image of being regenerated shows on program screen 1B.Combine this function and can make the user can jump to the zone of wishing inspection immediately, come content is checked.

(1-2) second execution mode

As mentioned above, the thumbnail that constitutes a plurality of time points of each fragment is shown as corresponding to one group among the sketch map part 13B of each fragment.What in this embodiment, suppose demonstration is thumbnail by a plurality of time points of any appointment of user.

Fig. 4 A and 4B show demonstration embodiment in this case.The thumbnail number that is write down in the situation shown in Fig. 4 A and the 4B is 2.

Fig. 4 A shows the demonstration embodiment corresponding to the sketch map part 13B of Segment A.In addition, Fig. 4 B shows position (time point information) t1 and the t2 in thumbnail A1 and the pairing fragment of A2.

For example, the position among Fig. 4 B can be illustrated in the set frame position of index marker of record when editing.Perhaps, for example, the frame position that the index marker that the position among Fig. 4 B can be illustrated in when writing down to be write down is set up.

Such thumbnail can suitably be represented the content and the theme of fragment.Therefore, a plurality of thumbnails of being located by the arbitrfary point of user's appointment are shown that the content that can make fragment can be understood more accurately.In addition, can predict the development of the plot of whole program more accurately.

In addition, by using indicating equipment or other user interfaces to specify to thumbnail A1 and A2.For example, can use action button, keyboard, mouse or the pen that slides to specify thumbnail A1 and A2.

One in two thumbnails can be the thumbnail at set time point place.For example, as shown in Figure 5A and 5B, one in two thumbnails can be corresponding to the thumbnail that plays point frame.In addition, for example, one in two thumbnails also can be ending frame.

Fig. 5 A shows the demonstration embodiment of the sketch map part 13B of homologous segment A.In addition, Fig. 5 B shows the position (time point information) in thumbnail A1 and the pairing fragment of A2.

In addition, playing point frame and ending frame can combine.Fig. 6 A and 6B show the demonstration embodiment that shows under three thumbnail situations.Fig. 6 A shows the demonstration embodiment corresponding to the sketch map part 13B of Segment A.On the other hand, Fig. 6 B shows the position (time point information) in thumbnail A1, A2 and the pairing fragment of A3.

Shown in this embodiment,, when also having shown point frame and ending frame, can grasp at an easy rate and the getting in touch of other fragments when except optional thumbnail by the user.

(1-3) the 3rd execution mode

In this embodiment, go out scene change point as the situation that is used for thumbnail images shown with describing Automatic Extraction.

Fig. 7 A and 7B show demonstration embodiment in this case.A fragment shown in Fig. 7 A and the 7B does not have the successional sub-fragment of scene each other by three and constitutes.In image-editing operations, the short-movie section is connected to each other usually, handles as a fragment.In this case, be difficult to accurately grasp the content of fragment by single thumbnail.

Fig. 7 A shows the demonstration embodiment corresponding to the sketch map part 13B of Segment A.

On the other hand, Fig. 7 B shows the position (time point information) in thumbnail A1, A2 and the pairing fragment of A3.As shown in Fig. 7 B, thumbnail A1, A2 and A3 are corresponding to the starting point of each sub-fragment.

When selecting such display mode, can extract all scene changes in the fragment.In addition, the starting point of the first sub-fragment is consistent with the starting point of entire segment.Therefore, in Fig. 7 A, first thumbnail is shown, the thumbnail of second thumbnail and back is shown at the scene change point in starting point.

In addition, using prior art comes automatic checkout area scape to change.For example, under the compress coding data situation, can use such technology, wherein,, promptly determine in this frame scene change to have taken place when the data volume of frame during greater than designated value.In addition, also can use such technology, when the brightness of all pixels changes above designated value, can determine in this frame scene change to have taken place.

When all detected scene changes all showed, the number of detected scene change may be very big.In this case, expectation only shows the part of detected scene change.For example, can dwindle detected scene change discontinuously, thereby show with the ratio of a scene change and a plurality of scene changes.

(1-4) the 4th execution mode

In this embodiment, the original position of describing from fragment is begun equally spaced to extract the situation that is used for thumbnail images shown.

Fig. 8 A and 8B show demonstration embodiment in this case.Fig. 8 A and 8B show and extract four situations that are used for thumbnail images shown from fragments.

Fig. 8 A illustrates the demonstration embodiment corresponding to the sketch map part 13B of Segment A.In addition, Fig. 8 B shows the position (time point information) of pairing intersegmental part of thumbnail A1, A2, A3 and A4.

Equally, when selecting such display mode, can extract the content in the fragment all sidedly.As a result, can grasp the content of fragment more accurately.

In addition, used time interval t sets in advance during the Automatic Extraction thumbnail.Certainly, the user t that also can arbitrarily set interval.

When selecting this display mode, thumbnail can be to upgrade with the inter-related mode of the edit operation in timeline window 5 automatically.

Fig. 9 A, 9B, 9C and 9D show as the demonstration embodiment of thumbnail when upgrading automatically with the mode of edit operation interlocking in timeline window 5.In addition, the direction of fragment regeneration is for from left to right.Fig. 9 A shows the state before the operation of timeline window 5 ineditings.

Fig. 9 B shows the position reach of end point, thereby the state that reduces of entire segment length.Fig. 9 C moves after showing initial point position, thereby the state that reduces of entire segment length.Starting point is shown Fig. 9 D and the end point position all moves forward, and makes the state that entire segment length reduces.

Be appreciated that by changing the point frame that rises of fragment the position that begins the frame that goes out with extracted at equal intervals from the fragment section start also changes.Also can understand,, also change the number of thumbnail image that extracts by changing the length of fragment.

Therefore, the edited result in the timeline window 5 is reflected in the displaying contents of sketch map part 13B, the feasible displaying contents that can grasp each fragment exactly.

Certainly, this automatically updating function can be carried out or be stopped by user's selector switch.

(1-5) the 5th execution mode

In this embodiment, will the situation of the thumbnail that shows set time point place be described.

Figure 10 A and 10B show demonstration embodiment in this case.Figure 10 A and 10B show the situation that has shown point frame and these two time points of ending frame.

Figure 10 A shows the demonstration embodiment corresponding to the sketch map part 13B of Segment A.In addition, Figure 10 B shows the position (time point information) in thumbnail A1 and the pairing fragment of A2.

When selecting this display mode, not only can check the plot of whole program, can also check out the contact between the fragment simultaneously.

For example, when when sight moves corresponding to the thumbnail of the end point of Segment A with between corresponding to the thumbnail of the starting point of fragment B, the contact between Segment A and fragment B as can be seen.In addition, this display mode can combine with other display formats recited above.

(1-6) the 6th execution mode

In this embodiment, the situation that the corresponding live width of the time difference between two thumbnails describing the line of demarcation utilization and show in sketch map part 13B is shown.

Figure 11 A and 11B show demonstration embodiment in this case.In this embodiment, will three situations by the thumbnail of any appointment of user that show be described.

Figure 11 A shows the demonstration embodiment corresponding to the sketch map part 13B of Segment A.On the other hand, Figure 11 B shows the position (time point information) in thumbnail A1, A2 and the pairing fragment of A3.

In Figure 11 A, the line of demarcation 15A with live width W1 is presented between thumbnail A1 and the A2.Line of demarcation 15B with live width W2 is presented between thumbnail A2 and the A3.

Live width W1 is corresponding to the time difference T1 between thumbnail A1 and the A2.In addition, live width W2 is corresponding to the time difference T2 between thumbnail A2 and the A3.

In this case, the corresponding relation between live width W and the time difference T is proportional basically.That is, basically, time difference T is big more, and live width W is wide more.Also expectation has this proportional relation in the time difference process of estimating by the demonstration of sketch map part 13B between the thumbnail.

But, when using simple proportional relation, the live width non-constant width that can become.This shows live width with regard to the zone that means a non-constant width of needs.

Therefore, can give timing difference T with the logarithmic function variable, and live width W may be defined as the logarithmic transformation value.In this case, can improve the regional resolution of poorer T, and reduce regional resolution than large time difference T.

Conversely, when needs increase the regional resolution of large time difference T, can provide time difference T and change with exponential function, and with index conversion value definition live width W.

Certainly, be used for the function that time difference T and live width W connect each other is not limited to these functions.

By adopting such display format, not only can check the plot in the fragment, can also check the time difference between the thumbnail simultaneously.This display format can combine with above-described other display formats.

By as adjusting the live width corresponding in this embodiment, can check the time difference between the thumbnail simultaneously with time difference T.

(1-7) the 7th execution mode

In this embodiment, with describe between two thumbnails that show among the sketch map part 13B with and two thumbnails between corresponding length of time difference situation spaced apart from each other.That is, situation about providing between two thumbnails corresponding to the clear area of the time difference between the thumbnail images shown among the sketch map part 13B will be described in.

The 7th execution mode is with to have removed marginal the 6th execution mode identical.Figure 12 A and 12B show demonstration embodiment in this case.In this embodiment, same hypothesis shows the thumbnail of three width of cloth by any appointment of user.

Figure 12 A shows the demonstration embodiment corresponding to the sketch map part 13B of Segment A.On the other hand, Figure 12 B shows the position (time point information) in thumbnail A1, A2 and the pairing fragment of A3.

In Figure 12 A, width is that the clear area 17A of W1 is arranged between thumbnail A1 and the A2.Width is that the clear area 17B of W2 is arranged between thumbnail A2 and the A3.

Width W 1 is corresponding to the time difference T1 between thumbnail A1 and the A2.In addition, width W 2 is corresponding to the time difference T2 between thumbnail A2 and the A3.

Equally, in this case, the corresponding relation between width W and the time difference T is proportional relation basically.That is, basically, time difference T is big more, and width W is wide more.Also can adopt this proportional relation in the time difference process in estimating sketch map part 13B between the thumbnail images shown.

But, when adopting simple proportional relation, the clear area non-constant width that can become.In this case, the clear area non-constant width that becomes.

Therefore, the variable that time difference T can be used as logarithmic function provides, and can to define the clear area width W be the logarithm conversion value.In this case, hour regional resolution of difference T can be increased, and the regional resolution of large time difference T can be reduced.

Conversely, when needs increased the regional resolution of large time difference T, can make time difference T was the exponential function variable, and can to define the clear area width W be the index conversion value.

Certainly, the function that is used for getting in touch each other time difference T and clear area width W is not limited to these functions.

By using such display format, not only can check the plot in the fragment, can also check the time difference between the thumbnail simultaneously.This display format can combine with above-described other display formats.

By as the clear area width of adjusting in this embodiment according to time difference T, can check the time difference between the thumbnail simultaneously.

(1-8) the 8th execution mode

In this embodiment, will the situation that line of demarcation between two thumbnails that show among the sketch map part 13B utilizes the color corresponding to the time difference between two thumbnails to show be described.

Figure 13 A and 13B show demonstration embodiment in this case.Hypothesis shows four width of cloth thumbnails by any appointment of user in this execution mode.

Figure 13 A illustrates the demonstration embodiment corresponding to the sketch map part 13B of Segment A.On the other hand, Figure 13 B illustrates the position (time point information) in the pairing fragment of thumbnail A1, A2, A3 and A4.

In Figure 13 A, between thumbnail A1 and A2, provide red sub boundary line 19A.Green sub boundary line 19B is provided between thumbnail A2 and A3.Blue line of demarcation 19C is provided between thumbnail A3 and A4.

Red sub boundary line 19A is corresponding to the time difference T1 between thumbnail A1 and the A2.On the other hand, green sub boundary line 19B is corresponding to the time difference T2 between thumbnail A2 and the A3.Blue line of demarcation 19C is corresponding to the time difference T3 between thumbnail A3 and the A4.

Under the situation of this execution mode, the relation between the time difference is T1〉T2〉T3.In this case, the corresponding relation between marginal color and the time difference T depends on setting in advance.

In this embodiment, the time difference is divided into a plurality of grades, and concrete color is corresponding with each rank.In addition, by the actual measured value of time difference T is compared with corresponding each other threshold value of level, just be enough to determine the rank under the actual measured value of time difference T.

In addition, other number of level and the number of color corresponding with each rank preferably can be set independently by the user.This is because the occupation mode that meets the user can be increased work efficiency effectively.

When as in this embodiment, when representing time difference T between the thumbnail by marginal Show Color, the space of preparing in the 6th execution mode and the 7th execution mode can be distributed to thumbnail and show.That is effective use that, can the implementation space.

In addition, the display position of thumbnail can align between fragment, thereby has increased visuality.

(1-9) the 9th execution mode

In the present embodiment, will the situation of display format that can selectively changing sketch map part 13B be described.That is, display format and fragment that is described in aforesaid each execution mode only shown situation about can selectively switch between the display format of a width of cloth thumbnail.

Figure 14 A and 14B show demonstration embodiment in this case.Figure 14 A is corresponding to existing display format.That is, Figure 14 A only shows the demonstration embodiment of a width of cloth thumbnail corresponding to a fragment.In this embodiment, shown the thumbnail of representing three Segment A, B and C respectively.Usually, the thumbnail corresponding to the starting point of each fragment is used as such thumbnail.

On the other hand, Figure 14 B shows the demonstration embodiment that a fragment shows the thumbnail of a plurality of time points.

In addition, suppose and to import the instruction that changes display format by using icon and indicating equipment on the screen.

When combining the function of switching in this embodiment between two kinds of display format, the user can change display format as required.

For example, when the user wished to check the content of a certain fragment, the user can select the display format in the aforementioned embodiments, can the details of each fragment being checked.

For example, when the user wished to understand the plot outline of whole program, the user can select existing display format.

Therefore, can improve user's usability, and increase operating efficiency.

(1-10) the tenth execution mode

In the above-described 9th embodiment, can between display format, carry out simple the switching.In the tenth execution mode, will the variant of existing display format be described.

Figure 15 A and 15B show demonstration embodiment in this case.Figure 15 A is corresponding to the existing display format that only has a thumbnail to show in sketch map part 13B.In this embodiment, a width of cloth thumbnail A1 who has only shown the expression Segment A.

But, in Figure 15 A, provide the viewing area that is used for being presented at the non-existent information of existing display format.This viewing area is index viewing area 21, and when selecting to be used to illustrate the display format of fragment detailed content (, when showing a plurality of thumbnail), this index viewing area is used for showing will be at sketch map part 13B number of thumbnail images shown N.

Under the situation of Figure 15 A, index shows that 21 show that " 3 " are as number of thumbnail image.In addition, Figure 15 B shows and changes display format demonstration embodiment afterwards.Index shows that 21 value is consistent with number of thumbnail image.

Therefore, during the display packing represented in adopting Figure 15 A, the user can know that storyboard window 11 has been prepared two kinds of display formats.That is, the user can know, can switch between display format.

In addition, the user can know the number of thumbnail image of checking in advance after changing display format.This predictability has improved user's operating efficiency effectively.

In addition, in this embodiment, sum that will thumbnail images shown after display format changes is shown as the number N that index shows demonstration in 21.That is, shown the number of thumbnail image that is included in Figure 15 A thumbnail images shown A1.

Yet for showing the number N that shows in 21 at index, can only be presented at the display format change afterwards will be by the number from new thumbnail images shown.That is " 2 " that, can show expression thumbnail A2 and A3.

When using this display format, can predict the number of thumbnail image that the change by display format obtains.With reference to this number, the user can determine whether to change display format.

In addition, will can switch to the display format that the fragment detailed content is shown at another icon of describing subsequently by using.By showing that at index placing a pointer in 21 can carry out the screen switching.

(1-11) the 11 execution mode

The variant of existing display format will be described in this embodiment, equally.Figure 16 A and 16B show demonstration embodiment in this case.Figure 16 A is corresponding to the existing display format that only shows a thumbnail in sketch map part 3A.In this embodiment, the thumbnail A1 that only shows the expression Segment A.

Yet, in Figure 16 A, provide the action button 23 that does not have in the existing display format.This action button 23 is the icons that are used to provide the instruction that changes to the display format that shows the fragment detailed content.

Under the situation of Figure 16 A, on action button 23, show pull-down flags.In this embodiment, the user can change to the display format of Figure 16 B by assigned operation buttons 23 such as pointers.In addition, in Figure 16 B, three thumbnail A1, A2 and A3 have been shown.

Therefore, when the display packing that adopts Figure 16 A to represent, the user can know that storyboard window 11 has been prepared two kinds of display formats.That is, the user can know, can switch between display format.

In addition, the setting operation button 23 like this, make it approaching with the sketch map part 3A that is unfolded.Therefore, the user is ongoing operation effectively, and does not need mobile significantly sight.That is user's ongoing operation effectively.

In addition, the display format of the button shown in the display position of action button 23 and Figure 16 A is an embodiment, and the present invention is not restricted to this.

(1-12) the 12 execution mode

In this embodiment, will the demonstration embodiment of the tenth execution mode and the 11 execution mode combination be described.

Figure 17 A and 17B show demonstration embodiment in this case.Figure 17 A is corresponding to the existing display format that only shows a thumbnail in sketch map part 3A.

In this embodiment, index demonstration 21 and action button 23 in sketch map part 3A, have been shown.

Index shows 21 zones for describing in the tenth above-mentioned execution mode, when the display format of the detailed content of selecting to be used to show fragment, this zone is used for being presented at sketch map part 13B and is about to number of thumbnail images shown N.

On the other hand, action button 23 icon for describing in above-mentioned the 11 execution mode is used to provide instruction, changes to the display format of the detailed content that is used to show fragment.

This execution mode can be checked after display format changes will number of thumbnail images shown, and need not mobile sight promptly and change display format.

(1-13) the 13 execution mode

In above-mentioned the tenth to the 12 execution mode, the modification to existing display format has been described, in the 13 execution mode, with the modification of describing the display format that shows the fragment detailed content.

Figure 18 A and 18B show demonstration embodiment in this case.Figure 18 A shows the demonstration embodiment in the display format that is used for showing the fragment detailed content.In this embodiment, shown three thumbnail A1, A2 and the A3 that constitutes Segment A.

But, in Figure 18 A, provide non-existent action button 25 in previous other display formats of describing.This action button 25 is the icons that are used to provide the instruction that changes display format, from showing that state corresponding to three thumbnails of a fragment is to the state that only shows a thumbnail.

Under the situation of Figure 18 A, draw mark showing on the action button 25.In this embodiment, the user can change to the display format of Figure 18 B by assigned operation buttons 25 such as pointers.In addition, in Figure 18 B, only shown thumbnail A1.

Therefore, during the display packing represented in adopting Figure 18 A, the user can know that storyboard window 11 has been prepared two kinds of display formats.That is, the user can know, can switch between display format.

In addition, action button 25 is provided with to such an extent that approach the sketch map part 13B that will fold up.Therefore, the user is ongoing operation effectively, and does not need mobile significantly sight.That is user's ongoing operation effectively.

In addition, the display format of the button shown in the display position of action button 25 and Figure 18 A is an embodiment, and the present invention is not limited to this.

(1-14) the 14 execution mode

In the present embodiment, with describe by binding fragment A and fragment B create new Segment A ' situation under demonstration embodiment.

Figure 19 A, 19B, 19C, 19D, 19E and 19F show demonstration embodiment in this case.In addition, in Figure 19 A, 19B, 19C, 19D, 19E and 19F, employing be to show the marginal method have corresponding to the live width of the time difference between the thumbnail.

Figure 19 A shows the demonstration embodiment of Segment A.Suppose in this embodiment, show three thumbnail A1, A2 and A3.On the other hand, Figure 19 B shows the demonstration embodiment of fragment B.Suppose in this embodiment, show two thumbnail B1 and B2.

In this case, at the demonstration embodiment when fragment B is combined in the rear portion of Segment A shown in Figure 19 C.

Figure 19 C corresponding in new segment A ', remain in conjunction with before the display packing of display format.Since consistent with afterwards display format before combination, so this display format is suitable for the image of fragment combination.In addition, when with Segment A ' when being divided into Segment A and fragment B once more because before cutting apart and display format afterwards consistent, so the image that the suitable fragment of this display format is cut apart.

But, for the display packing of Figure 19 C, may be very big in conjunction with the back number of thumbnail images shown.

In this case, can adopt the display packing of the 12 execution mode, therefore, adopt the display packing of representing among Figure 19 D.Show among the embodiment at this, shown thumbnail number and drop-down mark.

Other be used for showing Segment A ' method be included in the display packing that Figure 19 E and Figure 19 F represent.

Figure 19 E corresponding to show to constitute Segment A ' five thumbnails in first thumbnail and the demonstration embodiment of last thumbnail.In addition, showing the line of demarcation that has corresponding to the live width of the time difference between thumbnail A1 and the thumbnail B2 between thumbnail A1 and the thumbnail B2.

Figure 19 F corresponding to show to constitute Segment A ' each sub-Segment A and first thumbnail of B and the demonstration embodiment of last thumbnail.Show among the embodiment at this, shown thumbnail A1, A3, B1 and B2.Showing the line of demarcation that has corresponding to the live width of the time difference between the thumbnail between the thumbnail.

By using these display packings, realized using effective edit operation of storyboard window 11.

It is noted that Segment A ' demonstration embodiment be not limited among those embodiment shown in Figure 19 C to Figure 19 F.When Segment A ' when arbitrarily boundary position is divided into two, Segment A ' display format can be stored in the sub-fragment after dividing.

(1-15) the 15 execution mode

Another display packing of storyboard window 11 below will be described.

Figure 20 shows demonstration embodiment in this case.Direction of regeneration embodiment among the embodiment shown in Figure 20 between the arrow 27 indication fragments.Arrow 27 is arranged between the demonstration corresponding to each fragment.By using this display packing, the display format that can provide a kind of Any user all to understand easily.That is, Any user can easily be known relation and the direction of regeneration between the arrangement of fragments.

(1-16) the 16 execution mode

Other display packings of storyboard window 11 below will be described.

Figure 21 shows demonstration embodiment in this case.When the number of fragments that constitutes program was big, Figure 21 was suitable for.

In the display packing of Figure 21 representative, the display size of thumbnail images shown reduces among the sketch map part 13B.

Display size among Figure 21 is less than the display size among Fig. 3.When having the fragment of a large amount of formation programs, what the frame that shows with the display size that dwindles made it possible to disposable all fragments of inspection plays point frame and ending frame.

In addition, preferably can not only from two kinds of display sizes, select, can also from three kinds or multiple display size, select display size.Along with the increase of selecting, can select more suitable display size according to the number of fragment.

(1-17) the 17 execution mode

The other display packing of storyboard window 11 below will be described.

Figure 22 shows demonstration embodiment in this case.Demonstration embodiment shown in Figure 22 is suitable for constituting the bigger situation embodiment of fragment number of program equally.

Among the demonstration embodiment shown in Figure 22, fragment is divided into the fragment of fixed number, and can't be arranged on the fragment in the delegation and show in next line.

In addition, Figure 22 shows the situation that shows fragment in two row.Certainly, fragment also can be divided and is presented at triplex row or more in the multirow.

In the situation of Figure 22, the direction of regeneration between the row is descending from going upward to.In this embodiment, regenerate, after the regeneration of finishing first row, regenerate from left end to the right-hand member of second row from left end to the right-hand member of first row.

The demonstration embodiment of Figure 22 is corresponding to this regeneration order.By this demonstration embodiment that collects seed, can disposable all fragments of inspection play point frame and ending frame.

(1-18) the 18 execution mode

Other display packings of storyboard window 11 below will be described.

Figure 23 shows demonstration embodiment in this case.Demonstration example shown in Figure 23 is equally applicable to constitute the number of fragment of the program situation when big.

Among the demonstration embodiment shown in Figure 23, adopted the display mode of using scroll bar 29.That is, only shown the fragment that can in storyboard window 11, show on the screen, can on screen, show other fragments by operation scroll bar 29.In addition, show that at this scrollable area 31 is arranged on bottom of screen among embodiment.

Figure 24 shows the concept map of embodiment for this.Figure 24 shows the situation that six fragments constitute a program, wherein can show three fragments in storyboard window 11.

It is the 3rd state to the 5th fragment from the outset that Figure 24 shows in the storyboard window 11 what show, and other three fragments do not show.

In addition, when showing that in being desirably in storyboard window 11 from the outset first is to second fragment, the user only need drag scroll bar 29 left.On the other hand, when showing the 6th fragment from the outset in being desirably in storyboard window 11, the user only need drag scroll bar 29 to the right.

In addition, the user can determine the position of fragment in program that shows in the storyboard window 11 from the relative position of the scroll bar scrollable area 31 29.

When adopting this display packing, can avoid the increase of thumbnail reduction rate and reducing of thumbnail resolution.In addition, owing to also shown scroll bar 29, can easily determine position relation from whole plot.

(1-19) the 19 execution mode

As another execution mode, show that at aforesaid each form that fragment selected or that be reproduced can be different from other fragments shows in example.

For example, fragment selected or regeneration can utilize thick frame around, can give prominence to outer rim or background, maybe can change the Show Color of outer rim or background.Certainly, those skilled in the art also can visualize other demonstration embodiment on the basis of aforesaid execution mode.

(B) second embodiment of GUI screen

(1) can carry out plot editor's storyboard screen embodiment

The embodiment of the editing screen that shows when the executive editor operates in the storyboard window of a plurality of thumbnails that can show a fragment will be described below.

At present, storyboard window 3 is limited to the framework that carries out plot basically.Therefore, after the plot framework, storyboard window 3 usefulness few.This is because after the plot structure, user's operation will enter into the edit operation that service time, linear window 5 carried out.

But it is a good user interface that storyboard window 3 develops for the grasp plot.Therefore can think that the use that can be used for the storyboard window 3 of part edit operation equally will further improve user's availability.

Therefore, the inventor has proposed an embodiment who realizes the screen of edit operation on the storyboard window 3 of a plurality of thumbnails that can show a fragment.

For example, proposed to have the technology of following process.

(a) the paired process that shows starting point thumbnail and end point thumbnail in corresponding to the viewing area of each fragment

(b) show the process of the viewing area of the thumbnail corresponding with the form that is different from other viewing areas with editing the position

Equally, for example, also proposed to have the technology of following process.

(a) the paired process that shows starting point thumbnail and end point thumbnail in corresponding to the viewing area of each fragment

(b) to show the process of the viewing area of the thumbnail corresponding corresponding to the display format of editing character with editing the position.

In addition, " editor " comprises the processing procedure of editing one or more fragments.In addition, " editor " need not carry out on entire segment, can carry out on the part of fragment.For example, can be only to around the part of starting point or only the part around end point is edited.

By adopting these treatment technologies, can check the plot of whole program and the content of the editor position (for example starting point or end point) in each fragment simultaneously.

In addition, can be at an easy rate in whole plot, check the editor position and by content edited.

As a result, can improve the efficient of edit operation.

(1-1) first example of editing screen

The embodiment of the screen when carrying out editor that fragment rearranges will be described with the embodiment of edit operation screen below.In addition, can in existing storyboard window, arrange fragment.

Demonstration embodiment when Figure 25 A and 25B show the fragment that shows in rearranging storyboard window 11.Figure 25 A and 25B move to the situation of fragment C back corresponding to fragment B.In this case, shown in Figure 25 A, the user is by using indicating device with the back of selected fragment B drag and drop to fragment C.

Figure 25 B shows the demonstration embodiment behind the mobile fragment B.That is, the broadcast of program order is changed into " Segment A → fragment C → fragment B " by " Segment A → fragment B → fragment C ".Certainly, when moving each fragment, play point frame 13B and ending frame 13C and be held in showing.

Figure 26 A and 26B show the demonstration embodiment when inserting new segment in storyboard window 11.This shows that embodiment is corresponding to the situation of inserting fragment C between Segment A and fragment B.In this case, as shown in Figure 26 A, the user by use indicating device with selected fragment C drag and drop to the position between Segment A and the fragment B.

Figure 26 B shows and inserts fragment C demonstration embodiment afterwards.That is, the broadcast of program order has changed over " Segment A → fragment C → fragment B " by " Segment A → fragment B ".Certainly, when inserting each fragment, play point frame 13B and ending frame 13C and be held in showing.

In addition, can in storyboard window 11, carry out deletion and edit and duplicate editor.

(1-2) second embodiment of editing screen

The finishing editor will be described as the embodiment of edit operation below.

The embodiment of four typical operations that use in the finishing editor will be described below.Wherein two operation embodiment are the operation embodiment when non-finishing pattern is converted to the finishing pattern.Two other operation embodiment is the operation embodiment when changing the finishing target in the finishing pattern.

In addition, in all operation embodiment, the consistent demonstration of each fragment in the description the carried out hypothesis, starting point thumbnail 13B1 and end point thumbnail 13B2 and storyboard window 11.Therefore, the finishing target can be in starting point and the end point one or all.

(a) the first operation embodiment

With the operation embodiment that at first describes when non-finishing pattern is converted to the finishing pattern.In this case, will the situation of the starting point of specified segment C for the finishing target be described.In addition, suppose on initial screen, to select Segment A.

(a1) initial screen

Figure 27 illustrates the embodiment that selects fragment C initial screen before.Figure 27 shows the demonstration embodiment of storyboard window 11 and timeline window 41.But timeline window 41 illustrates with simplified way.That is, only show timeline 41A and track of video 41B.

At this moment, storyboard window 11 shows the display format of the outer rim 51 of Segment A by changing viewing area 13 and selecteed state.In this embodiment, show the state of selecting frame 51A that the selection Segment A is shown by external margin in outer rim 51.In addition, use yellow demonstration look as selection frame 51A.

The state of selecteed fragment and editor position is shown by the display position of the broadcast line 43 on track of video 41B in addition.In Figure 27, play the place that line 43 is presented at the starting point of close Segment A.In addition, red as the demonstration look of playing line 43.

These demonstrations make the user determine that at an easy rate Segment A is in selected state.

(a2) fragment is selected screen

Figure 28 shows the embodiment that selects fragment C screen afterwards.There is several different methods to can be used for selecting fragment C.

For example, have with monitor pointer 53 on the screen move to corresponding to the viewing area 13 of fragment C on and confirm the method for move operation.

In this case, can use action button, keyboard, mouse or other pointing devices to come moving hand 53.For example, the operation that can be by pressing action button or the clicking operation of mouse are confirmed move operation.

At this moment, pointer 53 can be positioned at corresponding to the optional position on the viewing area 13 of fragment C.That is, pointer 53 does not need to be positioned on the thumbnail 13B1 corresponding to the starting point of fragment C, can be positioned on the thumbnail 13B2 corresponding to the end point of fragment C yet.

Certainly, play the display position of line 43 on track of video 41B with interrelated by pointer 53 appointed positions.

For example, when the thumbnail 13B1 that specifies by pointer 53 corresponding to the starting point of fragment C, play the starting point that line 43 moves to fragment C.On the other hand, when the thumbnail 13B2 that specifies by pointer 53 corresponding to the end point of fragment C, play the end point that line 43 moves to fragment C.

Therefore, play line 43 to move automatically with the selection mode that is mutually related in the fragment of storyboard window 11.

For example, there is another kind of hop button (button on the supervision screen or the button on guidance panel) to change the method for chosen position by operation storyboard window 11.

This hop button is known as " FS/BS button "." FS button " is corresponding to the redirect backward on the time shaft." BS button " is corresponding to the redirect forward on the time shaft.In this embodiment, by operation " FS button ", chosen position can change to fragment B then to fragment C from Segment A in turn.

Three kinds of redirect forms are arranged: the redirect between the starting point, the redirect between the end point and the alternately redirect from the end point to the starting point or from the starting point to the end point.

Which kind of redirect of manipulating for " FS/BS button " depends in advance setting.

In addition, with the mode of the mobile interlocking of selecting frame 51A, the broadcast line 43 on the track of video 41B will move to corresponding position.

In addition, the hop button (button on the supervision screen or the button on guidance panel) by operating time linear window 41 can change selected position.

This hop button is called " forward/backward button "." forwarding button " is corresponding to redirect backward on time shaft." back " is corresponding to redirect forward on time shaft.

In this embodiment, by operation " forwarding button ", chosen position will change to fragment B then to fragment C from Segment A in turn.

Six kinds of redirect forms are arranged: the redirect between the starting point, the redirect between the end point, alternately redirect, in the redirect between the effect original position, the redirect between the effect end position, alternately redirect from the effect original position to end position or from the effect end position to original position from end point to starting point or from starting point to end point.

In addition, the selection frame 51A in the storyboard window 11 with timeline window 41 in the mode of mobile interlocking of broadcast line 43 move.

Select frame 51A to move automatically by the signal processing of image processing apparatus.

In addition, for example, can change chosen position by the broadcast line 43 in the operating time linear window.

In this case, can use mouse, button (button on the supervision screen or the button on guidance panel) or other indicating equipments to move and play line 43.Confirm to select by the operation of the operation of clicking the mouse.

In this embodiment, the selection frame 51A in the storyboard window 11 with timeline window 41 in the mobile mode of being mutually related of broadcast line 43 move.

(a3) trim locations is specified screen

Figure 29 shows the embodiment that is transformed into finishing pattern screen afterwards.Be transformed into the finishing pattern by operation finishing button.

In this case, before operation finishing button, confirm to repair target according to the position of pointer 53 or the position of broadcast line 43.

In addition, as shown in figure 29, when playing line 43 and be positioned at the somewhere of Qian Ban district 45A of fragment C (for example, when playing line 43 and be positioned at the position of representing by a dotted line), the starting point of fragment C is confirmed as repairing target.

At this moment, outstanding outer rim 55 in storyboard window 11 corresponding to the thumbnail 13B1 that repairs target.

In this embodiment, on the outward flange of outer rim 55, show selection frame 55A.In addition, use red demonstration look as selection frame 55A.

The finishing mark 47 that in timeline window 41, shows expression finishing target therebetween.In addition, because the starting point of fragment C is for the finishing target, so the finishing mark 47 among Figure 29 is presented on the initial point position of fragment C.

(b) the second operation embodiment

Another operation embodiment in the time of will describing from non-finishing pattern to the finishing mode switch below.The end point that to describe fragment C below is designated as the situation of finishing target.

(b1) initial screen

Identical in initial screen and first operational example.That is, initial screen is identical with the screen embodiment of Figure 27.

(b2) fragment is selected screen

Figure 30 shows the screen embodiment behind the selection fragment C.Select the method for fragment C basic identical with in first operational example.Specifically, mainly contain two kinds of methods: in the method for selecting fragment C on the storyboard window 11 with on timeline window 41, select the method for fragment C.

The characteristic of the screen embodiment of Figure 30 below will be described.Figure 30 illustrates pointer 53 on the storyboard window 11 and has specified thumbnail 13B2 corresponding to the end point of fragment C.Therefore, the broadcast line 43 on the timeline window 41 is presented on the end point of fragment C.

Certainly, when the hop button by operation storyboard window 11 jumps to the end point of fragment C, similar demonstration appears.When jumping to the end point of fragment C by operating time linear window 41 hop button, similar demonstration also appears.Certainly, when broadcast line 43 is positioned at the end point of fragment C by the broadcast line 43 on the direct control timeline window 41, similar demonstration appears.

(b3) trim locations is specified screen

Figure 31 shows the embodiment that is transformed into finishing pattern screen afterwards.In this case, realize the conversion of finishing pattern by operation finishing button.

Before operation finishing button, determine the finishing target according to the position of pointer 53 or the position of broadcast line 43.

For example, as shown in Figure 31, when playing line 43 and be positioned at the somewhere of later half district 45B of fragment C (for example, when playing line 43 and be positioned at the position that dotted line represents), the end point of fragment C is confirmed as repairing target.

At this moment, outstanding outer rim 55 in storyboard window 11 corresponding to the thumbnail of repairing target.In this embodiment, show red selection frame 55A in the periphery of outer rim 55.

Meanwhile, the finishing mark 47 that has shown expression finishing target in the timeline window 41.In addition, because the end point of fragment C is for the finishing target, so the finishing mark 47 among Figure 31 is presented at the end point position of fragment C.

(c) the 3rd operation embodiment

Next will be described in the situation that changes the finishing target in the finishing pattern.The finishing target is transformed into the starting point of fragment D from the starting point of Segment A situation will be described below.

(c1) initial screen

The screen embodiment of the starting point that Figure 32 shows specified segment A during for the finishing target.Figure 32 shows the demonstration embodiment of storyboard window 11 and timeline window 41.

At this moment, in storyboard window 11, represent the state that Segment A is selecteed, simultaneously by selecting frame 55A to represent the selecteed state of starting point of Segment A by selecting frame 51A.

In this case, selecting frame 51A is the yellow frame that is presented at corresponding on the outward flange of the outer rim 51 of the viewing area 13 of Segment A.On the other hand, selecting frame 55A is the red frame that is presented at corresponding on the outward flange of the outer rim 55 of the thumbnail 13B1 of the starting point of Segment A.

On the other hand, play on the starting point of Segment A that line 43 and finishing mark 47 be presented at timeline window 41.

(c2) trim locations changes screen

Figure 33 shows the screen embodiment after trim locations changes.There is several different methods to can be used to change the finishing target.

For example, a kind of method of selecting frame 55A by the pointer 53 on the direct mobile monitoring screen is arranged.In this case, by pointer 53 being moved to corresponding on the thumbnail of the starting point of fragment D and confirm to select to move and select frame 55A.

Can use action button, keyboard, mouse or other indicating equipment moving hands 53.For example, select by the operation acknowledgement of clicking the mouse.

In addition, play line 43 and finishing mark 47 also with storyboard window 11 on the mode of mobile interlocking of selection frame 55A move.

In addition, a kind of method that moves selection frame 55A by the hop button on the operation storyboard window 11 is arranged.That is, move selection frame 55A by " FS button " or " BS button ".In this embodiment, because initial position is a Segment A, so can change chosen position by operation " FS button ".

Certainly, play line 43 and finishing mark 47 also with storyboard window 11 on the mode of mobile interlocking of selection frame 55A move.

In addition, a kind of method that directly moves selection frame 55A by operation hop button in timeline window 41 is arranged.That is, move selection frame 55A by " forwarding button " or " back ".In this embodiment, because Segment A is an initial position, so can change chosen position by operation " forwarding button ".

In this case, the selection frame 55A on the storyboard window 11 moves in the mode with the mobile interlocking of playing line 43 and finishing mark 47.

(d) the 4th operation embodiment

Next will be described in another operation embodiment of the situation that changes the finishing target in the finishing pattern.The finishing target changes to the end point of fragment D from the starting point of Segment A situation will be described below.

(d1) initial screen

Initial screen identical with in the 3rd operational example.That is, initial screen is identical with screen embodiment among Figure 32.

(d2) trim locations changes screen

Figure 34 shows the screen embodiment after the change trim locations.Change the method for finishing target and the basically identical in the 3rd operational example.Specifically, three kinds of methods are arranged: by pointer 53 in directly appointed method, the method in storyboard window 11, using the method for hop button and in timeline window 41, use hop button on the storyboard window 11.

In addition, owing to select frame 55A to move to end point, jump to end point or jump to starting point from end point so the setting of hop button becomes from starting point from starting point.

(e) operation after the finishing target selection

After determining the finishing target, use mouse, turning point logical (jog dial), cursor key to wait and carry out finishing operation by above-described four operation embodiment.That is, change starting point or end point.

In addition, behind the operation finishing button, determine the change of starting point or end point once more, withdraw from the finishing pattern subsequently.In addition, in changing starting point and end point one or all the time, the content of thumbnail images shown also changes to the content after the renewal on the storyboard window 11.

(f) effect

Shown respectively corresponding to the storyboard window 11 of the thumbnail of starting point and end point by such use and to have selected and to have checked the finishing target, can improve user's operability.

For example, can be directly by selecting thumbnail to specify the finishing target.Therefore, can easily specify the finishing target.

Even when on timeline window 41, specifying the finishing target, no matter the fragment size how, can be checked the position and the content of finishing target at an easy rate.

In addition, can in storyboard window 11, check the plot of finishing target and whole program simultaneously.Thereby can in brain, utilize the relation of specific fragment and whole plot to carry out finishing operation effectively.

Therefore, in the time also can using storyboard window 11 to be used for edit operation, can realize checking than more intuitive assigned operation and target location when service time only, linear window 41 carried out edit operation.

(3) another editing screen embodiment

Above-mentioned screen embodiment can be used for other to have as the starting point or the end point of compiling objective or comprises the edit operation of the starting point or the end point of compiling objective.For example, above-mentioned screen embodiment can be used for setting an effect or or the conversion effect process middle finger volume position of delimiting the organizational structure.

Figure 35 shows the screen embodiment when between fragment B and fragment C conversion effect being set.In this case, a kind of method of the editor position on the storyboard window 11 and method of the editor position on a kind of fixed time linear window 41 of specifying arranged.

Figure 35 shows as the starting point of the end point of specified segment B and the fragment C screen embodiment as the target of conversion effect.Certainly, end point that also can a specified segment B or the starting point of fragment C.The screen embodiment of screen embodiment in this case during with the finishing editor that describes in the above is identical.

In addition, conversion effect comprise wipe, weaken, mixing, page turning, slip, fractionation, decomposition etc.In addition, need not say that except conversion effect, above-mentioned screen embodiment also can be applied to mosaic processing, color correct, picture-in-picture etc. more.

Use the editor of conversion effect not only to comprise the monorail editor, and comprise A/B track editor.

In addition, above-mentioned screen embodiment can be used for checking the editor position and check content of edit (comprise and check editing mode).In this case " inspection " not only comprises the inspection edit pattern, and comprises under any edit pattern situation and check content of edit.

Figure 36 and Figure 37 A and B show the embodiment of the screen that is used to check.

Figure 36 is as the screen embodiment when the starting point setting of fragment B is faded in.In Figure 36, show edit box 55B at outward flange corresponding to the outer rim 55 of the thumbnail 13B1 of the starting point of fragment B.For example, use green demonstration look as edit box 55B.

In addition, when similar screen embodiment is shown when the end point setting is faded out.In this case, edit box 55B is presented on the outward flange corresponding to the outer rim 55 of the thumbnail 13B2 of end point.

In addition, can use the demonstration look of different colours as the edit box that fades in and fade out.According to different demonstration looks, can check the difference and the type of content of edit.

The demonstration embodiment that Figure 37 A and 37B differ from one another for the display format after the display format in the editing process is determined with editor.In Figure 37 A and 37B, the end point of fragment C is the editor position.Figure 37 A is corresponding to the demonstration in the edit operation process.What for example, the demonstration of edit box 55B was adopted is that blue flash of light shows.Figure 37 B is corresponding to the demonstration of editing after definite back, position or editing and processing are determined.For example, the demonstration of edit box 55B adopts blue illumination to show.

Demonstration corresponding to the thumbnail of starting point and end point is combined with the demonstration of edit box 55B according to editor position or type by this, can use storyboard window 11 to check compiling objective and editing type.

In addition, the edit operation below on storyboard window, can realizing.

For example, when starting point thumbnail, end point thumbnail and one or more typical thumbnail show, also can use storyboard window in the viewing area 13 corresponding to each fragment.For example, in this case, scene change point etc. can be applied to typical image.

For example, when expectation cut away from the starting point to the typical image scene the time, only use cursor positioning to typical image, then operation " excision front button ".In this case, the fragment of this typical image as new starting point will be obtained to have.

For example, when expectation cuts away scene from the typical image to the end point, only use with cursor positioning to typical image operation then " excision back button ".In this case, will obtain to be close to the fragment of this typical image point before as new end point.

Certainly, in this case, the broadcast line on the timeline window with storyboard window on the mode of selection interlocking of fragment move.Choice box on the storyboard window (being highlighted) also with the timeline window on the mode of mobile interlocking of timeline move.

(C) embodiment of the structure of editing device

(C-1) hardware configuration

Figure 38 shows the structure embodiment of the editing device when realizing above-mentioned Presentation Function by program.Figure 39 shows the structure embodiment of the editing device when realizing above-mentioned Presentation Function by particular processing plate (display control circuit).

In addition, this editing device not only comprises program generating system (PGS), film generation system and other business systems, but also comprises the domestic consumer system.

Editing device 61 shown in Figure 38 is used for implementing the video editing function by application software.Therefore, can use a computer as the basic structure of editing device 61.Therefore, the hardware configuration of editing device 61 is well-known computer configuation.

Specifically, editing device 61 comprises CPU 63, ROM (read-only memory) 65, RAM (random access memory) 67, hard disk drive 69 and external interface 71.

Editing device 61 is by the formation editing system that combines with monitor 73, guidance panel 75, video tape driver 77, video server 79 and other external equipments.

CPU 63 is key equipments, and it uses RAM67 to carry out various programs as the service area, thereby implements various functions.

ROM 65 is a memory device, be used to store from or export to the base program of the control input and output of external equipment.

RAM 67 is the memory device as the service area of executive operating system and application program (for example as above-described edit routine with the function that shows storyboard window).

Hard disk drive 69 is a memory device, is used for storage operating system and application program.To be known as main storage device corresponding with ROM 65 and RAM 67, and hard disk drive 69 is called as auxiliary storage device.Usually, procedure stores according to the embodiment of the present invention is on hard disk drive 69.Hard disk drive 69 also is used for being stored in the material data that video editing is used.

The equipment of external interface 71 for being used for editing device 61 is connected with external equipment.As external interface 71, use be and the corresponding equipment of external equipment that will be connected on the editing device 61.

For example, use serial data interface or parallel data grabbing card.In addition, used LAN (local area network (LAN)) interface.In addition, also can use radio communications interface.

Next monitor 73, guidance panel 75, video tape driver 77 and video server 79 embodiment as the embodiment of the external equipment that constitutes editing system will be described.

Monitor 73 has output equipment such as the user interface images of images such as button, menu for being used for display design.In addition, monitor 73 can be combined as a whole with the shell of editing device 61, perhaps can be in the outside of the shell of editing device 61.

Guidance panel 75 is that the user is used for instructing to the input equipment of computer input instruction and information.Except guidance panel 75, also can be used as input equipment such as the input equipment of action button, keyboard, mouse, slip pen and use.

Video tape driver 77 is to be used to regenerate and write down the equipment that the video image of material is edited in conduct.Suppose that in this embodiment editor's material is recorded on the video tape.But, when editor's material is recorded on CD or the semiconductor memory, should use driver corresponding to recording medium.

Video server 79 is to be used to store the high-capacity storage device of editing program.Usually, hard disk drive uses as video server 79.In addition, when the main purpose of video server 79 was programs stored, the tape drive of high power capacity or CD drive also can be used as video server 79 and use.Under any circumstance, use memory device can satisfy purpose or needs.

On the other hand, when realizing the video editing functions by special disposable plates (display control circuit) 83, use be the editing device 81 shown in Figure 39.Display control circuit 83 comprises the hardware that is used for video editing.

For example, display control circuit 83 comprises integrated circuit or the semiconductor memory that is specifically designed to video editing.When needs were handled high-quality (that is, high data volume) video image at short notice in a large number, this device structure was very suitable.

Save to Figure 38 in the description of the same other equipment of the parts represented of identical drawing reference numeral.

In addition, although in Figure 38 and Figure 39, do not illustrate especially, can come encoding device (encoder) and decoding device (decoder) as required in conjunction with video data.

(C-2) functional structure

Next the embodiment of the functional structure of editing device will be described.

(a) first embodiment of GUI screen

Figure 40 shows the functional structure of the editing device when showing the storyboard window of a plurality of thumbnails that can show a fragment.Realize this functional structure (Figure 39) by the routine processes of CPU 63 or the signal processing of display control circuit 83.

The major part of editing device 91 comprises that time point information acquiring unit 91A, thumbnail generation unit 91B, thumbnail storage unit 91C, layout determining unit 91D and layout change unit 91E.

Time point information acquiring unit 91A generates the processing capacity of position (time point information) corresponding to the thumbnail that obtains each fragment.Thumbnail generates the position and changes according to the display format (Fig. 4 A to 13B) of each fragment in storyboard window 11.During display format in can selecting storyboard window 11, time point information acquiring unit 91A can obtain each fragment and the corresponding time point information of the selected display format of edit operation device.

Thumbnail generation unit 91B has the time point information to obtain, and generates the processing capacity of the thumbnail of each fragment.Thumbnail generation unit 91B reads the view data of corresponding time point from RAM67 or hard disk drive 69, generates thumbnail (low-resolution image) according to generating according to display size then.

Thumbnail storage unit 91C is the memory device of thumbnail that is used for preserving a plurality of time points of each fragment that constitutes storyboard window.For example, RAM 67 and hard disk drive 69 are corresponding to thumbnail storage unit 91C.Thumbnail storage unit 91C preserves the thumbnail that is generated by thumbnail generation unit 91B, to be used for demonstration.Certainly, when be prepared in advance with each fragment association as the thumbnail of data (metadata) time, thumbnail storage unit 91C can be from having stored the thumbnail of reading a plurality of time points that constitute each fragment in the storyboard window in the zone corresponding to the material data of each fragment, and preserve these thumbnails.

Layout determining unit 91D has the processing capacity of the display layout of definite storyboard window 11.For example, layout determining unit 91D can determine such display layout, and wherein, the thumbnail that constitutes a plurality of time points place of each fragment is arranged in one group.Monitor 73 shows storyboard window 11 according to display layout.

Layout changes the processing capacity that unit 91E has the change processing of carrying out display layout.For example, layout converting unit 91E can switch between the second contact surface screen that first interface screen that each fragment is made of the thumbnail of a plurality of time points and each fragment are made of a typical thumbnail.In addition, for example, layout converting unit 91E can according to change, the change of time point information of the thumbnail number that constitutes each fragment, single file shows with the duplicate rows demonstration between the rolling of change, display position of switching, thumbnail dimensions wait and change layout.

Figure 41 representative shows the embodiment of the process of storyboard window 11 by the editing device with this functional structure.The embodiment that the usefulness that adopts when in addition, Figure 41 represents to show each time generates the processing procedure of the thumbnail that constitutes each fragment generates.Therefore, when using pre-prepd thumbnail, the thumbnail generative process can be read to handle and replace generating by thumbnail.

At first, time point information acquiring unit 91A obtains the time point information (P1) of the thumbnail that is used for showing each fragment that will show in storyboard window 11.This time point information is provided for thumbnail generation unit 91B.

The time point information of thumbnail generation unit 91B to be given generates the thumbnail (P2) that constitutes each fragment.The thumbnail that is generated is stored among the thumbnail storage unit 91C and generates.

After obtaining the thumbnail that will be presented in the storyboard window 11, layout determining unit 91D determines display layout (P3).In RAM (video-ram) 67, launch image corresponding to the storyboard window 11 of the display layout that determines.The image of storyboard window 11 shows (P4) with other editors on the screen of monitor 73.For example, this image shows with the form of the editing screen shown in Fig. 1.

Thereafter, layout change unit 91E determines whether the edit operation device has provided the instruction (P5) that changes display layout.

When the edit operation device provides the instruction that changes display layout (when obtaining positive result), layout changes unit 91E and determines whether this instruction relates to the change of the time point information of the thumbnail that constitutes each fragment (P6).

In addition, detect the instruction that changes display layout by operation input or the operation input on the screen via guidance panel 75.

When instruction does not relate to the change of time point information (when obtaining negative decision), layout changes unit 91E indicator layout determining unit 91D and changes display layout.On the other hand, when instruction comprises the change of time point information (when obtaining positive result), layout changes unit 91E instruction dot information acquiring unit 91A instruction time and changes time point information.

If the edit operation device does not provide the instruction (when obtaining negative decision) that changes display layout, layout changes unit 91E and has determined whether to provide the instruction (P7) that finishes to show storyboard window 11.Unless provide the instruction that finishes to show storyboard window 11, will repeat to determine whether the edit operation device has provided the instruction that changes display layout otherwise layout changes unit 91E.

By repeating top processing procedure, content displayed is upgraded sustainably in the storyboard window 11.

(b) second embodiment of GUI screen

Figure 42 shows the functional structure of the editing device when the executive editor operates on the storyboard window of a plurality of thumbnails that can show a fragment.Routine processes by CPU63 or the signal processing by display control circuit 83 also can realize this functional structure (Figure 39).

The critical piece of editing device 101 comprises time point information acquiring unit 101A, thumbnail generation unit 101B, thumbnail storage unit 101C, layout determining unit 101D and editor position indicating member 101E.In addition, when relating to the change of thumbnail display layout, need to use the processing procedure of the functional structure shown in Figure 40.Figure 42 mainly illustrates the function relevant with the indication of editing the position.

In addition, time point information acquiring unit 101A, thumbnail generation unit 101B, thumbnail storage unit 101C and layout determining unit 101D and the time point information acquiring unit 91A shown in Figure 40, thumbnail generation unit 91B, thumbnail storage unit 91C are identical with layout determining unit 91D.Therefore, the description of time point information acquiring unit 101A, thumbnail generation unit 101B, thumbnail storage unit 101C and layout determining unit 101D will be saved.

Editor position indicating member 101E is corresponding to changing fragment and corresponding to the processing capacity of the display format of the thumbnail of editor position, making fragment and thumbnail can be different from other zones.For example, editor position indicating member 101E can change the width of outer rim or the demonstration look of fragment and thumbnail.

Figure 43 representative shows the embodiment of the processing procedure of storyboard window 11 by the editing device with this functional structure.In this case, adopted the technology that generates the processing procedure of the thumbnail that constitutes each fragment when showing each time of utilizing.Therefore, when using pre-prepd thumbnail, the thumbnail generative process reads process by thumbnail and replaces.Among the embodiment that Figure 43 represents, suppose that the display layout that constitutes the thumbnail of each fragment in the edit operation process does not change.When the display layout of the thumbnail that constitutes each fragment does not change, carry out the processing procedure of representing among Figure 41.

At first, time point information acquiring unit 101A obtains the time point information (P11) of the thumbnail that is used for showing each fragment that will show in storyboard window 11.This time point information is offered thumbnail generation unit 101B.

Thumbnail generation unit 101B serves as that the basis generates the thumbnail (P12) that constitutes each fragment with the time point information of being given.The thumbnail that is generated is kept at thumbnail storage unit 101C.

After obtaining thumbnail images shown in storyboard window 11, layout determining unit 101D determines display layout (P13).In RAM (video-ram) 67, launch image corresponding to the storyboard window 11 of determined demonstration general layout.The image of storyboard window 11 shows (P14) with other editors on the screen of monitor 73.For example, these images show with the editing screen form shown in Fig. 1.

Thereafter, editor position indicating member 101E has determined whether to select the edit operation (P15) on the storyboard window 11.

When the edit operation device is not selected edit operation (when obtaining negative decision), editor position indicating member 101E repeats this and determines processing procedure.

When the edit operation device has been selected edit operation (when obtaining positive result), editor position indicating member 101E obtains editor's positional information, and the outer rim that will wait to edit fragment or thumbnail becomes and makes this wait to edit fragment or thumbnail can be different from other regional display format (P16).For example, shown in Figure 27 to 37B, indication editor position on storyboard window 11.

At this moment, editor position display unit 101E shows the editor position in the mode with 41 interlockings of timeline window on storyboard window 11.

(D) other execution modes

(a) in the above-described embodiment, the situation that shows the thumbnail at a plurality of time points place that constitutes each fragment on storyboard window 11 has been described.

In addition, when carrying out special operational such as preview by specified special thumbnail, (viewing area of the special thumbnail of appointment on storyboard window 11) can regenerate and show after a period of time before the thumbnail, the thumbnail a period of time or the thumbnail moving image of a period of time before and afterwards in the thumbnail viewing area.

This display packing can mobile sight, only carries out Content inspection and edit operation on storyboard window 11.

(b) though described the embodiment of the structure of editing device in the above-mentioned execution mode, as long as this editing device is an image processing apparatus that the function of editing device can be provided, editing device just can adopt any product form.

Such image processing apparatus comprises for example computer, video camera, digital camera, game machine, portable information terminal (portable computer, portable phone or portable game machine), memory device (such as compact disk equipment or home server) and disposable plates or combines transaction card according to function of the present invention.

In addition, by in conjunction with cabinet, the signal processing unit of common structure and have external interface, can constitute any image processing apparatus according to the external equipment of product form.

Except above-described structure, video camera or digital camera can also have camera unit and be used for the write circuit of the captured view data of storage on storage medium.

Portable phone or other have the electronic equipment of communication function except above-described structure, can also have transmitter/receiver circuit and antenna.

(c) on the basis that does not deviate from spirit of the present invention, can carry out various modifications to above-mentioned execution mode.In addition, can carry out on the basis that this specification is described, carrying out various modifications and application.

For a person skilled in the art, the present invention can have various changes and variation.Within the spirit and principles in the present invention all, any modification of being done, be equal to replacement, improvement etc., all should be included within the claim scope of the present invention.

Claims (15)

1. one kind is used for image processing method that the view data that constitutes video material is handled, and described image processing method comprises:

Obtaining step obtains the view data of a plurality of time points that constitute described video material from described video material; And

Generate step, the view data that generation is used to show, be used for the storyboard viewing area of having arranged a plurality of described materials in the corresponding viewing area of described video material in, be presented at the view data at the described a plurality of time points place that obtains in the described obtaining step with Packet State

Wherein, in described generation step, the described data presented that is used for generates like this: make when showing, on the time a plurality of view data adjacent one another are with and described a plurality of view data between corresponding distance of the time difference be separated from each other.

2. image processing method according to claim 1 further comprises the output step of the described view data that is used to show that output generates in described generation step.

3. image processing method according to claim 1,

Wherein, in described generation step, at least one in the view data at described a plurality of time points place is the view data by the user's place's of putting selection at any time.

4. image processing method according to claim 1,

Wherein, in described generation step, at least one in the view data at described a plurality of time points place is the scene change point.

5. image processing method according to claim 1,

Wherein, in described generation step, the view data at described a plurality of time points place is with the uniformly-spaced generation in the timing code of described video material.

6. image processing method according to claim 1,

Wherein, in described generation step, the described data presented that is used for generates like this: make between a plurality of view data adjacent one another are in time demonstration at interval with and described a plurality of view data between corresponding color of the time difference show.

7. image processing method according to claim 1,

Wherein, in described generation step, the described data presented that is used for generates like this: make the viewing area of the view data corresponding with the editor position be highlighted.

8. image processing method according to claim 1,

Wherein, in described generation step, the described data presented that is used for generates like this: make the corresponding color in viewing area utilization and described editor position of the view data corresponding with the editor position show.

9. image processing method according to claim 1,

Wherein, in described generation step, the described data presented that is used for generates in the mode with the described video material edit operation interlocking of timeline window.

10. image processing method according to claim 1,

Wherein, in described generation step, first data that are used to show and second data that are used to show generate selectively, described first data be used for described storyboard viewing area in the corresponding viewing area of described video material in be presented at the view data at the described a plurality of time points place that obtains in the described obtaining step with Packet State, and described second data are used for showing a described view data.

11. image processing method according to claim 10,

Wherein, in described generation step, the described data presented that is used for generates like this: make to demonstrate to be used to provide the action button district that changes to the instruction of described second data that are used to show from described first data that are used to show.

12. image processing method according to claim 10,

Wherein, in described generation step, the described data presented that is used for generates like this: make to demonstrate to be used to provide the action button district that changes to the instruction of described first data that are used to show from described second data that are used to show.

13. one kind is used for image data processing system that the view data that constitutes video material is handled, described image data processing system comprises:

Deriving means is used for obtaining from described video material the view data of a plurality of time points that constitute described video material; And

Generating apparatus is used to generate the view data that is used to show, be used on storyboard window with the corresponding viewing area of described video material in, show the view data at the described a plurality of time points place that obtains by described deriving means with Packet State,

Wherein, in described generating apparatus, the described data presented that is used for generates like this: make when showing, on the time a plurality of view data adjacent one another are with and described a plurality of view data between corresponding distance of the time difference be separated from each other.

14. one kind is used for image processing method that the view data that constitutes video material is handled, described image processing method comprises:

Obtaining step obtains the view data of a plurality of time points that constitute described video material from described video material; And

Generate step, the view data that generation is used to show, be used for the storyboard viewing area of having arranged a plurality of described materials in the corresponding viewing area of described video material in, be presented at the view data at the described a plurality of time points place that obtains in the described obtaining step with Packet State

Wherein, in described generation step, the described data presented that is used for generates like this: make between a plurality of view data adjacent one another are in time demonstration at interval with and described a plurality of view data between corresponding color of the time difference show.

15. one kind is used for image data processing system that the view data that constitutes video material is handled, described image data processing system comprises:

Deriving means is used for obtaining from described video material the view data of a plurality of time points that constitute described video material; And

Generating apparatus is used to generate the view data that is used to show, be used on storyboard window with the corresponding viewing area of described video material in, show the view data at the described a plurality of time points place that obtains by described deriving means with Packet State,

Wherein, in described generating apparatus, the described data presented that is used for generates like this: make between a plurality of view data adjacent one another are in time demonstration at interval with and described a plurality of view data between corresponding color of the time difference show.

Images