JP2011244362A - Content editing and generating system with automatic content arrangement function - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a content editing and generating system capable of detecting an optimal area in positioning source contents or changing the size, and arranging the contents in an optimal size in the relevant area.SOLUTION: A content editing and generating system 1 with an automatic content arrangement function for editing one or more source contents and generating multimedia contents comprises: a source content file 24 for storing source contents; a stage object 222 for storing attribute information of a stage window 32 to arrange source contents thereon; a view object 221 for storing the attribute information of the source contents on the stage window 32 in association with the stage object 221; and an authoring function 21 for arranging on the stage window arrangement target contents which are the source contents selected from the source content file 24.

Description

  The present invention relates to a content editing / generating system for generating a synchronized multimedia content by editing a moving image, a still image, or the like.

Synchronous multimedia that integrates media (time-based media) such as video and audio, etc. (time-based media) with media (non-time-based media) such as text (text) information and still images, etc. by editing. An authoring tool for generating content is known (for example, see Patent Document 1). In such an authoring tool, content such as a video to be edited (referred to as “source content”) is temporarily copied to a file area for editing (for example, an area referred to as “bin”), or the authoring tool Is copied and edited on the memory in which is executed.
JP-T-2004-532497

  However, with the conventional authoring tool, when the source content to be edited is placed on the stage, the editor has to select the area that can be placed according to the size of the source content, or the placement destination area is narrow However, there is a problem that it takes time for the editor to reduce the source content.

  The present invention has been made in view of such problems, and in the generation of synchronized multimedia content, various types of source content can be freely edited, and the source content arrangement area and the area size can be selected. Content editing that enables source content to be edited efficiently by automatically reducing (resizing) the source content in accordance with the content and arranging the source content in the optimal area in the optimal size An object is to provide a generation system.

  In order to solve the above-mentioned problem, a content editing / generating system according to the present invention is a content editing / generating system having a content automatic arrangement function for editing one or more source contents to generate multimedia contents. Source content storage means for storing content (for example, a source content file in the embodiment), and stage object storage means for storing attribute information including width and height information of a stage window in which the source content is arranged as a stage object (for example, , A data manager function in the embodiment), and a view object that stores attribute information including position information of the source content arranged in the stage window as a view object associated with the stage object A storage unit (for example, a data manager function in the embodiment), and a content arrangement unit (for example, an authoring function in the embodiment) that arranges an arrangement target content that is the source content selected from the source content storage unit on the stage window; The content arrangement means can take out the stage object from the stage object storage means, and further take out the view object associated with the stage object from the view object storage means to arrange the arrangement target content on the stage window. A possible arrangement area extraction step for extracting a candidate area as an arrangement candidate area, and when the image is resized while maintaining the aspect ratio of the source content among the extracted arrangement candidate areas. A placement area selection step for selecting a placement candidate area that can be placed, and placement candidate content is placed in the selected placement candidate area of the stage window, and attribute information of the placement candidate content is stored as a view object in the view object storage means. And a view object storing step.

  In addition, such a content editing / generating system according to the present invention has an arrangement target area management table for storing arrangementable areas on the stage window of the arrangement target contents, and in the arrangementable area extraction step, Setting a grid cell obtained by dividing the area into small areas having a predetermined width, storing attribute information for each grid cell in the arrangement target area management table, and among the grid cells, a view associated with the stage object A step of setting a placement flag in the attribute information of the grid cell in which the source content corresponding to the object is placed and storing the flag in the placement target region management table, and the placement region selection step includes: The grid cell stored in the allocation target area management table It is configured to extract a grid cell in which the placed flag is not set with respect to the attribute information as a placement candidate area and select an area in which the placement target content can be placed from the extracted grid cell. Is preferred.

  Also, in such a content editing / generating system according to the present invention, the content placement means can place the placement target content reduced to a predetermined minimum size while maintaining the aspect ratio in the placement region extraction step. It is preferable to be configured to extract cells as arrangement candidate regions.

  Further, in such a content editing / generating system according to the present invention, the content placement means further includes the placement target content with respect to the grid cell selected as the placement candidate region in the placement region selection step. When the reduced size is changed and arranged, the grid cell is preferably configured to select a grid cell in which the arrangement target content can be arranged at the maximum size.

  Further, in such a content editing / generating system according to the present invention, the content placement means reduces the placement target content to a predetermined minimum size while maintaining the aspect ratio in the placement region selection step, A step of extracting a grid cell in which the placed flag is not set in the attribute information stored in the target area management table, and for each of the extracted grid cells, the placement object of the minimum size with the grid cell as the origin When the placed flag is not set in any of the grid cells where the placement target object is placed, the step of extracting the origin grid cell as the origin grid cell and the origin grid cell With each as the origin, the placement target object Performing a step of calculating a reduced size that maximizes the placement target object that can be placed, and a step of selecting a grid cell including an origin grid cell in which the largest placement target object can be placed. preferable.

  When the content editing / generating system according to the present invention is configured as described above, the editor not only adjusts the arrangement position and size of the source content by itself, but also automatically selects the optimum region for arranging the source content. In addition, according to the size of the selected area, it is possible to automatically perform resizing such as reduction of the source content while maintaining the aspect ratio, and the source content can be arranged in the optimal area in the optimal size. Thus, efficient editing is possible without the need for the editor.

  In addition, multimedia content that is actually generated on the stage (synchronous multimedia content) can be edited while being recognized, and there are no restrictions on the types of content that can be placed on a track. Edit content easily. In addition, since source content is managed through view objects (logical view information) instead of being directly edited, the resource consumption of the computer on which this system is operated is reduced compared to direct editing. Can do.

It is a block diagram which shows the structure of the content edit production | generation system which concerns on this invention. It is explanatory drawing which shows the user interface of an authoring function. It is a block diagram which shows the relationship between a source content file, a view object, a display object, and a content clip. FIG. 4 is a data structure diagram showing the structure of a view object, where (a) is a data structure diagram for content with time, (b) is a data structure diagram for content without time, (c) Is a data structure diagram of a stage object. It is explanatory drawing for demonstrating the process of extracting the empty area | region which can arrange | position the still image (arrangement | positioning object object) of capture (arrangement) object, and selecting an optimal empty area. FIG. 6 is an explanatory diagram for explaining a state in which a still image is resized to a maximum area while maintaining the aspect ratio in the empty area selected in FIG. 5 and pasted in the empty area. FIG. 6 is a data structure of a table constituting a stage data management file, where (a) shows the data structure of an arrangement target area management table, (b) shows the arrangement of an arrangement candidate area management table, and (c) shows stage data management. The structure of a table is shown. It is explanatory drawing for demonstrating the state which divided | segmented the stage window into the grid cell. It is explanatory drawing for demonstrating the state which marked and mapped the arrangement | positioning area | region of the display object. It is explanatory drawing for demonstrating the process of searching the arrangement | positioning candidate area | region of an arrangement | positioning object object, (a) is the minimum grid cell range required for arrange | positioning in the state resized to the minimum size, maintaining an aspect ratio (B) shows the result of searching for a grid cell that can be the origin of the placement candidate region. It is explanatory drawing for demonstrating the process of selecting the area | region optimal for arrangement | positioning while showing the area | region which can arrange | position a placement object. It is explanatory drawing for demonstrating the process for determining the candidate area | region which can arrange | position an arrangement | positioning target object by the maximum size. It is explanatory drawing for demonstrating the state which has arrange | positioned the arrangement | positioning object to an arrangement | positioning area | region, maintaining the aspect ratio by making the selected grid cell into an origin. It is a block diagram for demonstrating the detailed function which comprises an authoring function. It is a flowchart for demonstrating the process of the content edit production | generation system of this embodiment. FIG. 16 is a flowchart showing a continuation of the flowchart of FIG. 15. FIG.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, the configuration of the content editing / generating system 1 according to the present invention will be described with reference to FIGS. As shown in FIG. 1, the content editing / generating system 1 is executed by a computer 2 having a display device 3, and using a mouse, a keyboard, or the like (not shown) connected to the computer 2, the display device 3. An authoring function 21 for editing synchronized multimedia content using the interface, a data manager function 22 for managing information on the edited content, and the content edited in this manner (the edited state is “edited content”). Is configured from a publisher function 23 that generates content that can be finally provided to the user (that is, the above-described synchronized multimedia content). It should be noted that data (moving image file, still image file, etc.) that is the basis for generating the synchronized multimedia content is stored in advance in the storage device 4 such as a hard disk of the computer 2 as the source content file 24. Whether to store in the hard disk or temporarily in the memory or the like is preferably selected as appropriate according to the use of the file or table.

  The user interface displayed on the display device 3 by the authoring function 21 includes a menu window 31, a stage window 32, a timeline window 33, a property window 34, and a scope window 35, as shown in FIG. The menu window 31 is for an editor to select an operation for editing and generating content, and has a role of controlling the entire operation of the content editing / generating system 1. The stage window 32 is used for selecting an operation for editing and generating the source content arranged by the editor. In addition, the timeline window 33 is configured to include a plurality of tracks 33a, and a content clip 331 is assigned to the track 33a for management for each of the plurality of display objects 321 pasted on the stage window 32, and the display thereof is performed. Execution time of the object 321 (display start time in the case of an image, reproduction start time in the case of an audio, and a relative start based on the start time of the edited content assigned to the timeline window 33 And end time).

  Here, a data management method in the content editing / generating system 1 according to this embodiment will be described with reference to FIG. In the content editing / generating system 1, the display object 321 arranged in the stage window 32 does not directly manage and manage the source content file 24, but via the view object 221 generated in the data manager function 22. Managed. That is, the data manager function 22 generates a stage object 222 for managing information on the stage window 32 for the stage window 32, and sets the attribute information of the display object 321 pasted on the stage window 32 as the stage object. The view object 221 corresponding to 222 is managed. Here, the data manager function 22 manages stage width and height information (e.g., expressed in resolution (pixels)) for each stage window 32, and uses this information, for example, This is used to calculate the size when the display object 321 is pasted on the stage window 32. In addition, the content edit generation system 1 manages the content clip 331 assigned to the track 33a of the timeline window 33 and the view object 221 in association with each other. In addition, the content edit generation system 1 manages the display object 321 arranged in the stage window 32 in association with a scope 223 described later.

  For example, when the display object 321 is a moving image file, the data structure of the view object 221 that manages the attribute information of the moving image file is an object ID for identifying the view object 221 as shown in FIG. Is stored in the object ID field 221a, the file name field 221b in which the storage location (eg, file name) of the source content file 24 is stored, and the display object 321 on the stage window 32 relative to the stage window 32 as a reference. An XY coordinate field 221c in which various XY coordinates are stored, a width / height field 221d in which the display size of the display object 321 in the stage window 32 is stored, and a display object 321 in the edited content A reproduction start time field 221e for storing a relative reproduction start time (a relative time from the start point of the edited content or a scope described later), and a reproduction end time field 221f for storing the reproduction end time; A video is reproduced from a file type field 221g in which the file type of the source content file 24 is stored, and from which time in the source content file 24 corresponding to the display object 321 (relative time from the first time of the source content file 24). Is stored in a file start time field 221h, a layer number field 221i in which a later-described layer number is stored, and a scope ID field 221j in which a scope ID indicating which scope 223 belongs is stored. .

  Further, the data manager function 22 manages the resolution (pixel) of the width and height of the stage for each stage window 32 by using the stage object 222 as shown in FIG. The stage data management object has a data structure as shown in FIG. 4C, is provided corresponding to the type of the stage window 32, the stage ID field 222a in which the stage ID is stored, and the screen of the stage window 32 The screen width field 222b stores the width (X direction) in pixel units, and the screen height field 222c stores the screen height (Y direction) in pixel units.

  In addition, in the content editing / generating system 1 according to the present embodiment, in addition to the moving image, content with time passage such as audio data, and data without time passage such as text character data, still image data, graphics, Content that can be handled in the same way as video data and has a time lapse has the same data structure as the above video (however, in the case of audio data, it does not have an XY coordinate field, width and height fields, etc.) Content that has not elapsed in time has the same data structure except the in-file start time field 221h. For example, when managing text character data, as shown in FIG. 4B, the text information is stored in the text information field 221b ′, and the font information used when the text information is displayed in the font type field 221g ′. Information is stored. At this time, the text may be configured to manage the information as the source content file 24. Further, instead of storing the start time and end time of reproduction as in the case of moving image content, a display start time field 221e ′ and a display time field 221f ′ are provided, and the display start time and the time during which the text character information is continuously displayed ( It is also possible to configure to manage the (length). Furthermore, when managing graphic data as the view object 221, a graphic having a predetermined shape is defined and registered in advance as the source content file 24, and this graphic can be selected by identification information (number or the like). It can also be configured to display.

  As described above, in the data manager function 22, since the display object 321 displayed in the stage window 32 is managed by the view object 221 corresponding to the source content file 24, for example, one source content file 24 (particularly, , A single view object 221 can be defined, or two view objects 221 can be defined. As described above, even if a plurality of view objects 221 are defined, the source content file 24 is commonly used, and therefore each display object 321 has an entity (has a copy of the source content file 24). Compared to the case, the consumption of storage areas such as a memory and a hard disk in the computer 2 can be reduced. Of course, when a plurality of view objects 221 are defined, the source content file 24 may be defined so that there is an overlap in time within each view object 221 file.

  Note that the view object 221 has an in-file start time field 221h for storing from which point in time the source content file 24 is to be played back with respect to the content such as a moving image as a reference (with the passage of time). Therefore, it is not necessary to execute the source content file 24 from the beginning, and the editor can freely set the playback start time for each view object 221. Further, since the source content file 24 is not directly edited as described above, the time in the above-described file in the source content file 24 of each view object can be freely set / changed in the timeline window 33 or the like. .

  The source content can be arranged on the stage window 32 by the editor by dragging and dropping the source content file 24 with a mouse or by selecting the source content file 24 from the menu window 31. . Text information and graphics can also be arranged by displaying predetermined candidates in a pop-up window and dragging and dropping them on the stage window 32 from there. Furthermore, when content (display object 321) is arranged in the stage window 32, a content clip 331 corresponding to the display object 321 is arranged on the currently selected track 33a of the timeline window 33. In the timeline window 33, as shown in FIG. 2, a current cursor 332 indicating a relative time in the currently edited synchronized multimedia content (edited content) is displayed. The content clip 331 is automatically arranged in the track 33a so that the display object 321 starts to be reproduced from the time indicated by 332. Here, the display on the track 33a is, for example, the entire time of the source content file 24 displayed as a white bar, and the playback portion defined in the view object 221 (file start time field 221h, playback start time field 221e). , And determined in the playback end time field 221f) are displayed with colored bars (this corresponds to the content clip 331).

  Here, the timeline window 33 is provided with a plurality of tracks 33a. The track 33a has no limitation on the type of content to be arranged, and includes video content, audio content, text information content, graphic content, Either still image content or interactive content requiring input can be arranged. At this time, since an icon indicating the type of the arranged content is displayed on the track 33a (not shown), the arranged content can be easily discriminated. Therefore, the editor can efficiently perform content editing work.

  In addition, the size and position of the display object 321 on the stage window 32 may be freely changed by the editor using a mouse or the like regardless of whether the editor itself has placed it or the automatic content capturing unit 213 has placed it automatically. it can. Furthermore, the position and size (playback time) of the content clip 331 on the timeline window 33 and the playback start position in the source content file 24 can be freely changed by the editor using a mouse or the like. . When the editor performs such arrangement, movement, size change of the source content file 24 on the stage window 32, movement of the content clip 331 on the timeline window 33, or change of the playback section, the authoring is performed. The function 21 sets the above-described attribute items of the display object 321 and the view object 221 corresponding to the content clip 331 according to the state of the stage window 32 and the timeline window 33 changed by the editor's operation. It is configured as follows. Further, such attribute items of the view object 221 can be displayed and corrected using the property window 34.

  By the way, in the above description, it is assumed that the content can be arranged on the stage window 32 by the editor himself by dragging and dropping, and in this case, for example, the resolution of the still image to be captured is high. If it is too high and too large to be pasted on the stage window 32, it may be time consuming for the editor to reduce the still image to an appropriate size and re-arrange it. In the present embodiment, such an operation can be automatically performed by the automatic content capturing unit 213 of the authoring function 21. That is, an arrangement target object selected by the editor from the menu window 31 or the like is arranged while maintaining an aspect ratio so that an automatic content capturing unit 213 (to be described later) selects an optimum empty area and becomes the maximum size in this area. The operation of reducing (resizing) and arranging the target object is automatically performed. Note that this automatic arrangement and automatic reduction are performed when, for example, a “automatically reduce content” check button is provided in the menu bar and this button is checked.

  Regarding the automatic selection, resizing, and arrangement mechanism of the area by the automatic content capturing unit 213, for example, as shown in FIG. 5, a still image content (arrangement target object 321a) is captured (arranged) on the stage window 32. ) Are already arranged on the stage window 32 when two display objects 321b and 321c are already arranged. When the editor manually arranges, the user selects an empty area by himself / herself, and when the resolution of the still image is high and cannot fit in the empty area, it takes time to reduce and paste. However, when the “automatically reduce content” check button is checked, the automatic content capturing unit 213 automatically extracts the empty areas 322a and 322b on the stage window 32. Among them, while maintaining the aspect ratio (aspect ratio) of the arrangement target object 321a, it is determined which of the empty areas 322a and 322b has the maximum area of the still image when it is captured, and the capture area is determined. . In the case of FIG. 5, the empty area 322a is selected as the optimum area. Next, as illustrated in FIG. 6, the capture target still image content (arrangement target object 321 a) is reduced so that the still image has the maximum area while the aspect ratio is maintained, and is pasted in the empty area 322 a.

  Now, the algorithm of the automatic content capturing unit 213 will be described with reference to the drawings. First, the automatic content capturing unit 213 creates the stage data management table 30 on the memory. As shown in FIG. 7A, the stage data management table 30 divides the stage window 32 into grid cells G having a predetermined pixel width, and for each grid cell G, whether or not the display object 321 has been arranged, The placement target area management table 301 that manages the position and size of the display object 321 that has been placed, and the position information of the grid cells G that are candidates for placing the placement target object, as shown in FIG. 7B. And the like are configured from an arrangement candidate area management table 302 in which are stored.

  The arrangement target area management table 301 is managed in a two-dimensional array (n, m) in units of grid cells G. That is, when the stage window 32 is divided into grid cells G, the upper left corner is the origin, n is the number of the grid cell G in the width direction (X direction) from the origin, and m is the height direction (Y Direction) grid cell G number. The arrangement target area management table 301 has a data structure as shown in FIG. 7A, and an arrangement determination flag field 301a in which a determination flag indicating whether or not the display object 321 has already been arranged in the grid cell G is stored. , The X coordinate field 301b in which the X coordinate of the grid cell G (in this embodiment, the grid number n when the upper left grid cell is defined as the origin 0) is stored in grid units, and the Y coordinate (in this embodiment, A Y coordinate field 301c in which the grid number m) in the case where the upper left grid cell G is the origin 0 is stored in grid units, and a width (grid) field 301d in which the widths of objects that can be arranged are stored in grid units. A height (grid) field 301e in which the height is stored in units of grids, and the width of the object that can be similarly arranged is a pixel. Width stored Le units (pixels) field 301f, and is constituted from a height (in pixels) field 301g is stored in pixels height. Note that the flag “off” (false value) is set as an initial value in the arranged determination flag field 301a, and the X and Y coordinates of the grid cell G are set in advance in the X coordinate field 301b and the Y coordinate field 301c.

  The placement candidate area management table 302 has a data structure as shown in FIG. 7B, and has already placed grid cells G (hereinafter referred to as origin grid cells G ′) that can be the origin of placement candidates for the placement target object. An arranged determination flag field 302a for storing a determination flag, an X coordinate field 302b for storing the X coordinate of the candidate grid cell G 'in grid units, and a Y coordinate field 302c for storing Y coordinate in grid units. It is composed of This set is a variable table that is stored each time a grid cell G ′ that is a candidate for placing the placement target object is searched.

  Next, an algorithm for arranging a still image as an arrangement target object on the stage window 32 on which a plurality of display objects 321 are already displayed will be described with reference to the flowcharts of FIGS. 15 and 16. The arrangement target object is a still image selected by the editor from the source content file 24 displayed in the menu window 31. First, as shown in the flowchart shown in FIG. 15, the content arrangement target area is divided into grid cells G (see FIG. 7), and a grid is created (step S1). Specifically, information on the size (pixel) of the width and height of the screen of the stage window 32 is acquired from the stage object 222 of the data manager function 22. In the example shown in FIG. 8, the size of the stage window 32 is horizontal × height = 640 × 480 (pixels). Then, the stage window 32 is divided into a plurality of grid cells G at a predetermined grid interval with the upper left corner of the stage window 32 as the origin. The interval between the grids may be determined in advance, but may be specified by an editor by inputting a numerical value as a parameter. The number n in the width direction of the grid is obtained by n = screen width (pixel unit) / grid interval (rounded down). Further, the number m in the height direction of the grid is obtained by m = screen height (pixel unit) / grid interval (rounded down). Using the calculated n and m, a grid (that is, an array for automatic arrangement target area management) is defined as shown in FIG.

  In this embodiment, the grid interval is 50 (pixels). Therefore, the number n in the width direction is n = 640/50 = 12.8, and the screen is divided into 12 in the width direction. Further, the number m in the height direction is m = 480/50 = 9.6, and the number is divided into nine in the height direction, and is divided into a total of 108 grid cells G. As shown in FIG. 8, the upper left corner of the stage window 32 is the origin (0, 0), the right direction on the paper is the X direction, the lower direction on the paper is the Y direction, and the (X , Y) represents coordinates. In the present embodiment, it is assumed that three display objects 321d, 321e, and 321f are arranged in advance on the stage window 32 as shown in FIG. The display objects 321d to 321e are managed in association with the stage object 222 by the data manager function 22. In the present embodiment, the right side and the lower side of the stage window 32 are blanks and are not subject to content placement. However, when it is not preferable that the content is biased to the upper left, the content may be divided into grid cells G starting from a place other than the upper left end of the screen when divided into grid cells G.

  Next, among the display objects 321 arranged on the stage window 32, the display object 321 that has not yet been mapped (painted) on the grid is selected as a processing target (step S2). This is because the area where the placement target object cannot be placed is confirmed and excluded. In the present embodiment, first, the display object 321d is selected as a processing target. If it is determined in step S2 that all the display objects 321 arranged on the stage window 32 have been mapped, the process moves to step S5 (step S3).

  In the present embodiment, since the display object 321d is selected as the processing target as described above, the process proceeds to the next step S4, and this display object 321d is mapped on the grid. First, the upper left coordinate (pixel unit) of the display object 321d is converted into the upper left coordinate (grid unit). As a calculation formula for conversion, X coordinate (grid unit) = X coordinate (pixel unit) / grid interval (rounded down) is used. Also, the conversion from the lower right coordinate (pixel unit) to the lower right coordinate (grid unit) is performed using the same calculation formula. Then, the grid cell G in the rectangle including the range from the calculated upper left coordinate (grid unit) to the lower right coordinate (grid unit) is marked and mapped (filled) (see FIG. 9). Specifically, the flag in the placement determination flag field 301a of the placement target area management table 301 is set to a true value (1: placed) for the grid cells G in the mapped range. The mapped state may be displayed on the stage window 32 so that the editor can visually recognize it.

  Then, returning to step S2, the next placed object (display object 321) that has not been mapped is selected, and if it is determined that it has been selected in step S3, mapping is performed in the same manner as in the above-described processing in step S4. The processes in steps S2 to S4 are repeated until there is no processed display object 321 to be processed. In the present embodiment, since the display objects 321e and 321f are arranged as the arranged objects in addition to the display object 321d, the processes in steps S2 to S4 are repeated two more times. As a result of the mapping in step S4, as shown in FIG. 9, a rectangular area of upper left coordinates (0, 2) -lower right coordinates (4, 5) corresponding to display objects 321d, 321e, 321f, upper left coordinates. The range of the grid cell G in the rectangular area of (7,1) -lower right coordinate (11,3), upper left coordinate (8,8) -lower right coordinate (10,8) is marked and mapped. (Painted).

  If it is determined in step S3 that the object to be processed is not selected, that is, if mapping is completed for the grid cells G of all the arranged display objects 321, the process proceeds to step S5 as described above, The minimum grid cell G range necessary for placing the placement target object selected by the editor is calculated. In this case, the minimum area of the arrangement target object in pixel units is calculated. First, the ratio of the width and height (aspect ratio) of the arrangement target object is calculated, and the short side of the arrangement target object is defined as a defined minimum value (in pixels). In the present embodiment, the minimum value of the short side is set to 100 pixels, and the long side (pixel unit) of the minimum region is calculated according to the aspect ratio. In the case of the present embodiment, as shown in FIG. 10A, when the aspect ratio is long side: short side = 4: 3 and converted into pixel units, the long side: short side = about 133: 100. . Next, based on these values, the minimum area of the arrangement target object in the grid unit is calculated, and the range of the grid cell G in which the rectangle with the calculated aspect ratio can be arranged is calculated. The calculation formula for calculating the width of the minimum arrangement area is the width of the object to be arranged (in pixels) / grid interval (rounded up). The calculation formula for calculating the height of the minimum arrangement area is the height (pixel unit) / grid interval (rounded up) of the arrangement target object. The reason for rounding up the fraction is to prevent the fractional part of the placement target object from protruding from the area where it is placed. Therefore, for example, when the aspect ratio is 4: 3 (133: 100 (pixels)), the necessary minimum grid G is width: height = 3: 2.

  When the process of step S5 ends, the process proceeds to the process of the flowchart of FIG. First, a grid cell G that has not been subjected to the investigation process as the arrangement area origin G ′ of the arrangement target object is selected as an investigation target (step S6). This selection is performed for all grid cells G set in the arrangement target area management table 301. First, the origin coordinate (0, 0) at the upper left corner is selected as the grid cell G to be investigated first. Is done.

  Next, it is determined whether or not the grid cell G to be investigated has been selected (step S7). Here, since the origin coordinate (0, 0) is selected, the process proceeds to the next step S8, and the arrangement determination flag field 301a corresponding to the (X, Y) coordinate of the grid cell G to be investigated is referred to. , Whether it is a true value (arranged) or not is investigated. From the result of this investigation, it is determined whether or not the arrangement determination flag of the grid cell G to be investigated is a true value (arranged) (step S9). Returning to step S6, the next grid cell G to be investigated is selected.

  On the other hand, if it is determined in step S9 that it is a false value (not placed), it is checked whether or not a mark already placed is included in the minimum grid cell G range with the selected grid cell G as the origin. (Step S10). Specifically, the presence or absence of a true value within the range from the coordinates (X, Y) of the grid cell G to (X + a−1, Y + b−1) is examined. In this coordinate, a indicates the width (grid unit) of the minimum arrangement area calculated in step S5, and b indicates the height (grid unit). Then, it is determined whether or not a true value (arranged) mark is included in the range (step S11). If it is determined that the mark is included, the process returns to step S6, and the next grid to be investigated Cell G is selected. On the other hand, when it is determined that the mark of the true value is not included, the grid cell G becomes the origin grid cell G ′ of the arrangement candidate area (see FIG. 10B), and is stored in the arrangement candidate area management table 302. The X and Y coordinates are set in grid units.

  Next, for the origin grid cell G ′ set in the arrangement candidate area management table 302, in order to try to expand the arrangement candidate area, the maximum area in pixel units of the arrangement target object in the arrangement candidate area is calculated. That is, when the grid cell range is expanded, a target side (width or height) is determined. Note that the width of the placement target object = the width of the placement target candidate area (in pixels), and the height of the placement target object = the width of the placement target candidate area (in pixels) × aspect ratio (step S12). Here, the calculated height of the placement target object is compared with the height of the placement target candidate area. If the height of the placement target object <the height of the placement target candidate region, the width is +1 grid. If the height of the placement target object> the height of the placement target candidate region, the height is +1 grid. If the height of the placement target object is equal to the placement target candidate region, both the width and height are +1 grid. Then, the presence or absence of a true value of the arranged flag 301a of the width or height portion (grid cell G) to be expanded is confirmed (step S13).

  Then, it is determined whether or not a true value (arranged) is included (step S14). If it is determined that a true value (allocated) is not included, the expanded range is set as an arrangement target area management table. In addition to recording in 301, the process returns to step S12 to try again. This recording is performed by recording the expanded value in the arrangement target area management table 301 in the width (unit: grid) and height (unit: grid). On the other hand, if it is determined in step S14 that a true value (arranged) is included, no further expansion is possible, so the size before expansion is recorded in the arrangement target area management table 301 (step S15). Further, the maximum area of the arrangement target object in the pixel unit within the arrangement candidate area is calculated. Record the calculated values in the width (unit: pixel) and height (unit: pixel).

  Next, returning to step S6, the grid cell G for which the next investigation process has not been performed is selected as an investigation target, and the processes of steps S7 to S15 are performed. These processes are repeated until it is determined in step S7 that the grid cell G to be surveyed has not been selected, that is, all grid cells G have been surveyed. Thereafter, the process proceeds to step S16. In this embodiment, as described above, the origin coordinate (0, 0) is selected first, and then the right next coordinate (1, 0) is selected as the order in which the grid cells G to be investigated are selected. Select in the order of ~ coordinate (8,0), then select in the order of lower coordinate (0,1) ~ coordinate (8,0), and finally select coordinate (11,8) In this way, selection is made in the X direction from the origin (0, 0), and then the Y coordinate is incremented by one, so that selection is made for every so-called row. However, the present application is limited to this. Instead, it may be selected in the Y direction from the origin (0, 0), that is, for each column.

  In the present embodiment, as a result of repeating the processing of steps S6 to S15 for all the grid cells G, as shown in FIG. 11, coordinates (0, 0) are set as the origin grid cell G 'of the minimum area of (i). , (1, 0), (2, 0), (3, 0), (4.0) information is recorded, and in the middle area of (ii), coordinates (0, 6), (1, 6) , (2,6), (3,6), (4,6) information is recorded, and in the maximum area of (iii), information of coordinates (5,4), (6,4) is recorded. It will be.

  In this way, when all the examination target grid cells G ′ are selected and the origin grid cell G ′ candidates are recorded in the arrangement target area management table 301, the process proceeds to step S16. In step S16, it is determined whether range information of the origin grid cell G ′ that is a candidate is recorded in the arrangement target area management table 301 and the arrangement candidate area management table 302. If it is not recorded, it is determined that the area where the placement target object is to be placed has not been found, and the process is terminated without placing the content, or the grid interval when dividing the grid is reduced, and the step is performed. The process from S1 is repeated.

  On the other hand, if it is recorded as in the present embodiment, the process proceeds to step S17, and the grid cell range that can be arranged is selected from among the grid cell ranges that can be arranged. In the present embodiment, as shown in FIG. 12, there are candidate areas as shown in (i), (ii), and (iii), respectively. In the present embodiment, the arrangement target object can be arranged at the maximum size when the coordinates (5, 4) and (6, 4) of (iii) are set as the origin grid cell G ′. As described above, when there are a plurality of candidates, it is preferable to determine in advance which one has priority. In this embodiment, the upper left is determined in advance as a priority, and coordinates (5, 4) are automatically selected. Then, as shown in FIG. 13, the coordinates (5, 4) (pixel: 300, 200) are set as the origin, and the placement target object is placed as the display object 321g on the stage window 32 (step S18). In this embodiment, the optimal size of width × height 267 × 200 pixels (aspect ratio 4: 3) can be arranged at the optimal position. The arrangement target object is managed by the data manager function 22 with the arrangement coordinates and the like set as the display object 321g in the view object 221.

  When the placement target object is smaller than the free area, the placement target object may be placed in the minimum area of the free area, or the maximum area of the free area may be detected, The arrangement target object may be configured to be enlarged and arranged on the stage window 32 so as to have the maximum size while maintaining the aspect ratio. Alternatively, the editor may change the position and size.

  The synchronized multimedia content (edited content) edited by using the authoring function 21 in this way has a predetermined start time and end time (relative time). In the system 1, the time defined by these times can be divided and managed in a plurality of scopes 223. Since content with a time course such as a moving image has a time axis, there is an essential problem that editing (moving, deleting, etc.) at a certain time gives side effects to other portions of the moving image. Therefore, in the present embodiment, a moving image content having a time axis is logically defined as a scope 223 (virtual definition) separately from physical information (content arrangement on the timeline window 33). It can be divided by setting multiple segments.

  The scope 223 data includes a scope ID for identifying a plurality of scopes, cover information displayed on the stage window 32 when each scope 223 is started, and relative contents in the edited content of the scope 223. The scope start time, which is the correct start time, and the scope end time, which is the relative end time within the edited content, are stored in the hard disk or the like. The information on the cover is composed of text information, for example, and is used to display the contents of each scope 223 on the scope window 35 'as indicated by the one-dot chain line in FIG. Then, the contents are displayed according to the order of the scopes.

  In the data manager function 22, as shown in FIG. 1, a view object 221 is managed for each scope 223, and an operation on a certain scope 223 does not affect data in other scopes 223 in the timeline window 33. . That is, for example, even if an operation of changing the order of the two scopes 223 is performed, the order and execution time of the view objects 221 in these scopes 223 are not affected. At this time, the content editing / generating system 1 does not directly edit the source content file 24 but manages it via the view object 221 as described above. Does not affect the source content file 24.

  By providing the scope 223 in this way, by specifying the display order of the scope 223, the physical information remains as it is (that is, the video content is not cut or rearranged at all), and the edited content It is possible to dynamically change the playback order of the moving image content within. In addition, the influence of editing operations within the scope 223 (for example, movement of the position of all elements including moving image content on the time axis, deletion, etc.) stays locally in the scope 223 and does not affect the other scopes 223 at all. Because it does not give the side effect of, you can do editing work with confidence.

  As described above, the authoring function 21 can automatically determine the arrangement position and change the size of still image content, moving image content, etc. with respect to the stage window 32, so that the editor does not need to do it directly. , Efficient operation becomes possible. In addition, editing can be performed while checking the edited content that is actually created. As a method for arranging the display objects 321 in the stage window 32, not only can the objects be selected one by one, but a plurality of objects can be selected (for example, a method of clicking with the mouse while pressing the shift key, or dragging the mouse). To determine an area and select all objects in the area).

  Since all the display objects 321 arranged in the stage window 32 are managed as view objects 221 in the data manager function 22, candidates for arrangement as text objects among the view objects 221 are displayed on the display device 3. It is also possible to make a list so that the editor can select from this list and arrange it as a new display object 321.

  Next, the publisher function 23 that converts the edited content created in this way into a data format that is finally provided to the user will be described. The publisher function 23 is the final content that is finally provided to the user from the stage object 222, the view object 221, the scope 223, the stage object 222, and the source content file 24 that are managed by the data manager function 22. A file 25 and a meta contents file 26 are generated.

  The final content file 25 basically corresponds to the source content file 24, and as described above, unnecessary portions (for example, the finally generated synchronized multimedia content of the source content file 24). This is a file obtained as a result of trimming a portion that is not reproduced or changing the compression rate in accordance with the size arranged on the stage window 32. Further, the meta content file 26 is a source content file 24 or final content such as execution (reproduction start) and end timing (time) of the final content file 25 corresponding to moving images, audio, and still images in the edited content. It defines the display contents and display timing (time) of information that is superimposed on the source content file 24 and the final content file 25 such as information for controlling reproduction of the file 25, text information, graphics, etc. It is managed as text format data. The meta content file 26 is also managed by the data manager function 22 as shown in FIG. 1 as a file for managing information related to the edited content edited by the authoring function 21.

  As described above, in the content editing / generating system 1 according to the present embodiment, it is not necessary for the editor to position or resize a still image or other content, and an optimum region in which the content can be arranged most automatically is automatically determined. Since the display content is automatically arranged in the area, the display content is automatically resized to the maximum size while maintaining the aspect ratio, efficient editing is possible. In addition, it is configured to edit and generate synchronized multimedia content (edited content) in two stages of the authoring function 21 and the publisher function 23. Therefore, at the time of editing, since the display information (start point and end point) of the moving image is managed by the view object 221, the information is retained as a logical view so that the trimmed section is not displayed. The display start / end time can be changed freely. On the other hand, since the source content file 24 is physically divided based on the logical view information (view object 221) at the time of generation, the capacity of the final content file 25 can be reduced without having extra data. Can do.

  Further, the final content file 25 generated by the publisher function 23 from each source content file 24 does not synthesize text information or the like (for example, the text information is managed by the meta content file 26). Therefore, the source content file 24 (final content file 25) is changed with the text information or the like (for example, a new source content file is generated by combining text information or the like with the source content file such as a moving image). However, even if the source content file 24 is compressed, it is possible to prevent the text or the like from being crushed (fogging or unclear on the screen).

  The detailed functional configuration of the authoring function 21 described above will be summarized with reference to FIG. The authoring function 21 includes a property editing unit 211. The property editing unit 211 further includes a time panel arrangement unit 212, an automatic content capturing unit 213, and a screen display unit 214. The property editing unit 211 provides a function of changing the property of the view object 221 by displaying the property window 34.

  In addition, the time panel arrangement unit 212 provides functions such as arrangement and deletion of the content clip 331 on each track 33a, changing the reproduction disclosure time and end time of the content clip 331 in the timeline window 33, and the like.

  On the other hand, the automatic content capturing unit 213 provides a function of automatically specifying the position of the display object 321 in the stage window 32 and the animation position. Further, as shown in FIG. 1, the automatic content capturing unit 213 provides a function of reading the size of the display screen from the stage data management table 30 and dividing the screen into grid cells, and also displays the display object 321 at the optimum position. In addition, the function to arrange in the optimal size is provided. The screen display unit 214 provides a function of displaying the display object 321 on the stage window 32.

  As described above, in the content distribution system 100 according to the present embodiment, various types of content can be freely edited, and content can be selected according to the selection of the content placement region and the size of the region as specified by the editor. Resizing such as reduction can be automatically performed, and content can be arranged in an optimal area in an optimal size while maintaining an aspect ratio.

  The editor can edit while recognizing the content actually generated on the stage (synchronous multimedia content), and there is a limit to the types of content that can be placed on the still image track. In addition, since the search for the arrangement position of the content, the resizing to the optimum size, and the arrangement are automatically performed, the editing of the content becomes easy. In addition, since the content is managed as a view object (logical view information) without being directly edited, it is possible to reduce the consumption of the resources of the computer on which this system is operated as compared with the case of direct editing.

1 Content Edit Generation System 21 Authoring Function (Content Placement Means: Automatic Reduction Capture Unit, etc.)
22 Data manager function (stage object storage means, view object means)
24 source content file (source content storage means)
30 stage data management table (arrangement target area management table, arrangement candidate area management table)
32 stage window 221 view object 222 stage object (stage object management means)

Claims (5)

A content editing generation system having a content automatic arrangement function for editing one or more source contents to generate multimedia contents,
Source content storage means for storing the source content;
Stage object storage means for storing attribute information including width and height information of a stage window in which the source content is arranged as a stage object;
View object storage means for storing attribute information including position information of the source content arranged in the stage window as a view object associated with the stage object;
Content placement means for placing placement target content that is the source content selected from the source content storage means on the stage window;
The content placement means includes:
The stage object is extracted from the stage object storage unit, and the view object associated with the stage object is extracted from the view object storage unit, and an area where the arrangement target content can be arranged on the stage window is arranged. A placeable area extraction step for extracting as a candidate area;
An arrangement area selection step for selecting the arrangement candidate area that can be arranged in a maximum size when resizing while maintaining the aspect ratio of the source content among the extracted arrangement candidate areas;
A view object storage step of placing the placement candidate content in the selected placement candidate area of the stage window and storing attribute information of the placement candidate content in the view object storage unit as the view object; Content editing generation system characterized by the above. The content placement means includes:
A placement target area management table for storing a placeable area on the stage window of the placement target content;
In the placeable area extracting step,
Setting a grid cell obtained by dividing an area on the stage window into small areas having a predetermined width, and storing attribute information for each grid cell in the arrangement target area management table;
Among the grid cells, a placed flag is set in the attribute information of the grid cell in which the source content corresponding to the view object associated with the stage object is placed in the placement target area management table. A step of storing, and
The arrangement area selection step extracts the grid cell in which the arrangement flag is not set as the arrangement candidate area for the attribute information of the grid cell stored in the arrangement target area management table, 2. The content editing / generating system according to claim 1, wherein the content editing / generating system is configured to select an area in which the placement target content can be placed from the extracted grid cells. The content placement means includes:
In the arrangement area extraction step, the grid cells that can arrange the arrangement target content reduced to a predetermined minimum size while maintaining the aspect ratio are extracted as the arrangement candidate areas. The content editing / generating system according to claim 2. The content placement means includes:
In the arrangement area selection step, when the reduced size of the arrangement target content is arranged with respect to the grid cell selected as the arrangement candidate area in the arrangement area selection step, the maximum size is selected. 4. The content editing / generating system according to claim 3, wherein the content editing / generating system is configured to select the grid cell in which the arrangement target content can be arranged. The content placement means includes:
In the arrangement area selection step,
Reducing the placement target content to a predetermined minimum size while maintaining an aspect ratio;
Extracting the grid cells for which the placed flag is not set in the attribute information stored in the placement target area management table;
For each of the extracted grid cells, when the placement target object of the minimum size is placed with the grid cell as the origin, the placed flag is placed in any of the grid cells in which the placement target object is placed. When not set, extracting the origin grid cell as the origin grid cell; and
Calculating a reduced size at which the placement target object can be placed at a maximum with each of the origin grid cells as the origin;
The content editing generation system according to claim 2, further comprising: selecting the grid cell including the origin grid cell in which the largest placement target object can be placed.

Images