JP2008154064A - Video creating device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To create video that a producer intends without burdening the producer. <P>SOLUTION: A video creating unit 34 creates video by adding effects to video raw materials according to parameters. The parameters are supplied from operators to a network synchronizing unit 36 and then supplied to the video creating unit 34. A recorder 48 stores the parameters from the beginning of supply of the parameters. Parameters of a specified area among the parameters stored in the recorder 48 are supplied to the video creating unit 34. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus for creating an image by adding an effect to an image.

  Conventionally, as an apparatus as described above, there is one disclosed in Patent Document 1, for example. In the apparatus of Patent Document 1, a photograph and BGM are reproduced according to a predetermined photograph and BGM, a video effect is given to the photograph according to meta information of the video, and when a series of videos is generated, it is stored.

JP 2006-279861 A

  However, in the technique of Patent Document 1, the effect to be given to the video is determined in advance according to the meta information, but the video to be added while changing the effect to be given according to the intention of the producer. May be created. In this case, it is necessary to save the video with the effect many times, which is a burden on the producer.

  An object of the present invention is to provide a video creation device that can easily obtain a video according to the intention of the producer.

  A video creation device according to an aspect of the present invention includes a video generation unit, which generates a video by applying an effect to a video material according to a parameter. The parameter supply unit supplies the parameter to the video generation unit with the passage of time. The storage means stores the parameters as time passes from the start of parameter supply. The parameter resupply unit reads parameters after the designated parameter among the parameters stored in order in the storage unit with the passage of time, and resupplys them to the video generation unit. In addition to the video generation means, there can also be provided sound effect giving means for giving an effect by giving a parameter to the audio signal, and the parameters for the sound effect giving means can also be stored in the storage means. As the parameter generation means, for example, a device that can be operated by a producer to generate parameters is used. In addition, the image generated by the video generation unit can be stored in a storage unit in a predetermined file format, and this storage can be started when the parameter supply unit starts supplying parameters. The parameter re-supply unit can specify a parameter at a position where supply is stopped, in addition to a parameter at a position where supply is started, among a series of parameters stored in the storage unit.

  With this configuration, since the producer sequentially stores parameters that change over time through various trials and errors, parameters for obtaining an image according to the manufacturer's intention among these parameters are again generated by the image generation means. By supplying to, it is possible to easily reproduce the video according to the intention of the producer, and it is easy to save the reproduced video.

  The storage means has a plurality of storage areas, stores the parameters from the top of the storage area, stores the parameters to the end of the storage area, and stores the parameters from the top of the storage area again. It can also be configured. With this configuration, since the parameters are stored in the storage means endlessly, even if a video that does not conform to the producer's intention is generated, the parameters are not changed until the video according to the manufacturer's intention is obtained. Even if the change is continued, the parameters for making the video according to the intention of the producer are always stored in the storage means, and the parameters that are stored in the storage means in the middle and become the video that does not conform to the intention of the producer are deleted. There is no need.

  As described above, according to the present invention, it is possible to produce an image according to the intention of the producer without imposing much burden on the producer.

  The video and music editing apparatus 2 according to one embodiment of the present invention edits and outputs audio material and video generation material, and as shown in FIG. used. Each editing device 2 is connected to a display means, for example, a liquid crystal display device 4, and these display devices 4 are used as a multi-screen. These editing apparatuses 2 are connected by a network and can transmit data to each other.

  Each editing device 2 has a bus 6 as shown in FIG. 1, for example, and a storage unit, for example, a storage 8 is connected to the bus 6. Each storage 8 stores a plurality of video generation materials and audio materials. An external memory 10 is also connected to the bus 6. The external memory 10 also stores a plurality of video generation materials and audio materials.

  The selected one of the plurality of video generation materials is stored in the scene temporary memory 12 via the bus 6. The video generation material of the scene temporary memory 12 is supplied to video processing means, for example, the GPU 14 via the bus 6. The GPU 14 processes the image generation material in the scene temporary memory 12 while using the storage means connected via the sub-bus 15, for example, the image temporary memory 16, and converts it into an analog signal by the D / A converter 18. To the corresponding display device 4.

  Similarly, the audio stream signal obtained by the A / D conversion converter 19 connected to the bus 6 and the audio material selected from the storage or the external memory are supplied to the audio processing means, for example, the DSP 20 via the bus 6. The The DSP 20 processes a sound material and the like using a storage means connected via the sub-bus 21, for example, a sound effect memory 22, converts the sound material into an analog signal by a D / A converter 23, and a sound system (not shown). To supply.

  Setting of parameters used in the processing in the GPU 14 and the DSP 20 and selection of image generation materials are performed based on instructions supplied to a control unit, for example, the CPU 25 from an operator (not shown) via the user interface 24. The parameters and the like are supplied from the CPU 25 to the GPU 14 and the DSP 20 via the bus 6.

  FIG. 3 is a functional block diagram showing the GPU 14 and the DSP 20 of the editing apparatus 2, and the audio material is reproduced by the audio reproduction unit 26. The reproduced audio signal is synthesized with the audio stream signal from the A / D converter 19 by the synthesizer 28, and supplied to the effect applying means, for example, the audio effect unit 30, where the audio effect is added and output. . Note that at the time of audio reproduction, the audio parameters set by the operation of the operator are supplied to the audio reproduction unit 26, and for example, a reproduction pitch is set. Also, an audio parameter is set in the audio effect unit 30 to determine an effect to be applied. Audio parameters are supplied from the audio LFO 32 to the audio effect unit 30 in order to periodically change the state of the effect applied by the audio effect unit 30. An audio parameter is given to the LFO 32 and BPM (beat per minute) data is supplied to the audio parameter, and the audio parameter determined by the LFO 32 is supplied to the audio effect unit 30, thereby changing the state of the audio effect periodically. It is done.

  In the editing apparatus 2, the video generation unit 34 generates a video output based on the video generation material. The video generation material is, for example, a still image, and an effect is applied to the still image to generate a moving image. For example, an image in which an object in a still image is changed every predetermined frame, for example, every frame is generated. Changes include, for example, an object that expands or contracts over time for each frame, an object that moves from a specific position to another position over time, or a planar object that is replaced with a three-dimensional object. There are various types such as those that are generated and the shape of the three-dimensional object changes over time. Drawing parameters for performing such changes are given to the network synchronization unit 36 as video settings, and are supplied from the network synchronization unit 36 to the video generation unit 34. Further, drawing parameters are supplied from the LFO 38 to the video generation unit 34 in order to periodically perform the above-described changes. BPM data is also supplied to the LFO 38.

  The drawing parameters set in the network synchronization unit 36 are supplied to the other editing apparatuses 2 via the network, and are also supplied to the video generation unit 34 of the other editing apparatuses 2.

  The video output from the video generation unit 34 can be synchronized with the audio output from the audio effect unit 30. For this purpose, the audio output is analyzed by the audio analysis unit 40, and the analysis result is supplied to the video generation unit 34. At this time, the video generation unit 34 generates a video output that changes in synchronization with the audio output.

  Contrary to the above, the audio effect can be changed in the same cycle, for example, in synchronization with the change of the video (change of the video effect) in the video generation unit 34. Therefore, a switch 42 is provided between the LFOs 32 and 38 so that the audio effect LFO 32 is synchronized with the video generation LFO 38. When the switch 42 is closed, the audio parameter from the LFO 32 changes, for example, in the same cycle in synchronization with the drawing parameter from the LFO 38.

  The editing devices 2 and 2 constitute a multi-screen using the two display devices 4 and 4. In this embodiment, the left half of the video to be displayed is displayed on one display device 4, and the right half of the video to be displayed is displayed on the other display device. Therefore, as shown in FIG. 2A, the video generation units 34 of the editing devices 2 and 2 each have a virtual screen area 44.

  Then, in one editing apparatus 2, the size of each virtual screen area 44, how it is divided, and which part of the divided area is output from which editing apparatus 2 is initialized. It is determined at the time of setting, and the determined value is transmitted to the other editing apparatus 2 via the network.

  For example, if the display areas of the display devices 4 and 4 have an aspect ratio of 4: 3, 1280 pixels in the horizontal direction, and 720 pixels in the vertical direction, the number of pixels in the horizontal direction of the virtual screen area 44 in any editing device 2 is The total value (2560 pixels) of the horizontal pixels of both display devices 4 and 4 and the number of pixels in the vertical direction are determined as the vertical pixels (720 pixels) common to both display devices 4 and 4. The virtual screen area 44 is determined as the number of divisions according to the number of display devices, in this case, two divisions. Since the display areas of both the display devices 4 and 4 are the same size, the division is divided into two equal parts, for example, the left area of (0, 0-1280, 720) and (1281, 0-2560, 720). And determine the right area. Then, the editing device 2 shown on the left side in FIG. 2A outputs the left side area of the virtual screen area 44 it has to the display device 4, and the editing device 2 shown on the right side in FIG. It is determined that the right area of the virtual screen area 44 it has is output to its display device 4.

  Alternatively, the display areas of both the display devices 4 and 4 have an aspect ratio of 4: 3, but the display area of one display device 4 is 1280 pixels in the horizontal direction and 720 pixels in the vertical direction, and the display area of the other display device 4 However, if the size of the display screen area is different from 800 pixels in the horizontal direction and 600 pixels in the vertical direction, the size of any virtual screen area 44 is the sum of the horizontal pixels of both display devices 4 and 4 in the horizontal direction ( 2080 pixels in the horizontal direction) and the larger value (720 pixels) of the vertical pixels in the display devices 4 and 4 in the vertical direction. Then, the number of divisions corresponding to the number of display devices in the virtual screen area 44 is determined, in this case, two divisions. Since the sizes of the display areas of the display devices 4 and 4 are different from each other, the division is determined to be cut out from the virtual screen area 44 according to the display areas of the display devices 4 and 4. For example, assuming that the display area of the display devices 4 and 4 that is in charge of the left area is 1280 * 720 pixels, and the display area of the display apparatus that is in charge of the right area is 800 * 600, the virtual screen area 44 is , (0, 0-1280, 720) is determined as the left region. Of the remaining (1281, 0-2080, 720), an area corresponding to a size of 800 * 600 is determined as the right area. For example, it is determined as (1281, 0-2080, 600), (1281, 60-2080, 660) or (1281, 120-2080, 720).

  These determinations are made in one editing device 2. Then, the determination is transmitted as a parameter from one editing device 2 to the other editing device. Each of the editing devices 2 and 2 acquires a virtual screen area 44 having a designated size based on this parameter. The virtual screen area acquired in each editing device 2, 2 has the same size and is larger than the display area of the display device 4, 4 of each editing device 2, 2.

  When the video material to be used in one editing device 2 is determined at the time of initial setting, data representing the video material to be used is transmitted to the other editing device 2 via the network. The editing apparatuses 2 and 2 generate an image having a size corresponding to the virtual screen area 44 based on the data, and the generated image is displayed on the virtual screen areas 44 and 44 as shown in FIG. Is written to. For example, if the data supplied via the network is data for each pixel such as bitmap data, the number of pixels in the horizontal and vertical directions and the aspect ratio correspond to the number of pixels in the virtual screen area and the aspect ratio. If they do not match, it is necessary to enlarge, reduce, or deform when writing to the virtual screen area 44, and the images displayed on the display devices 4 and 4 are degraded. However, in the editing apparatuses 2 and 2, the images created by the editing apparatuses 2 and 2 in a size corresponding to the virtual screen areas 44 and 44 based on the data representing the supplied video material are displayed on the virtual screen areas 44 and 44. The image is not deteriorated because the image is written on.

  Then, the areas designated by the editing devices 2 in the virtual screen areas 44 and 44 are output to the display devices 4 and 4 to which the editing devices 2 are respectively connected, as shown in FIG. . For example, as described above, when the display screens of the display devices 4 and 4 have a size of 1280 pixels in the horizontal direction and 720 pixels in the vertical direction, the multi-screen display is performed as a result of each of the editing devices 2 and 2. Is called. Even when the sizes of the display areas of the display devices 4 and 4 are different, the areas designated as the left area and the right area of the virtual screen area are cut out and displayed on the display apparatuses 4 and 4 as described above.

  Also, at the time of initial setting, a parameter for determining what kind of moving image is to be made based on the video material is determined by one editing device 2 and transmitted to the other editing device 2 via the network. Therefore, for example, as shown in FIG. 2C, when the character A is displayed as an object in the horizontal center of the virtual screen area 44, the object is moved rightward at a predetermined speed. Assuming that the parameter is set, in the next frame, the images of the virtual screen areas 44 and 44 of both editing apparatuses 2 are such that the object has moved to the right by the amount specified by the parameter on the right side. Rewritten and divided and output as described above. After a certain number of frames have occurred since the start of image output, both objects are positioned so that the object is located completely to the right of the center of the virtual screen area 44 as shown in FIG. Each is written in the virtual screen area 44. Accordingly, the object is not displayed on the left display device 4, and the entire object is displayed on the right display device 4.

  In order to give such an effect, parameters are supplied from the LFO 38 to the video generation unit 34 and are repeatedly performed. By synchronizing the LFO 38 and the audio effect LFO 32 by operating the switch 42, the audio effect applied to the audio output output from the audio effect unit 30 at that time is also synchronized with the effect applied to the video output. And repeated. Accordingly, even if, for example, the audio material supplied to the audio effect unit 30 at that time is non-periodic noise, an audio output having the same cycle as the effect applied to the video output can be obtained.

  The video output generated in this way is converted into a video file by the encoder 46 shown in FIG. 3, stored in the storage 8, and then read out from the storage 8 and reproduced. Thus, before storing as a video file, what effect is used and how to set parameters for the effect may be determined as a result of trial and error. In this case, how the parameters are changed is stored, the parameters when the desired video output is obtained, the video output is generated again from the video generation unit 34, and this is output by the encoder 46 to the video file. And Such processing is hereinafter referred to as skip back processing.

  Therefore, the parameters are supplied from the network synchronization unit 36 to which the parameters are supplied to the parameter storage means, for example, the recorder 48 and stored. In the recorder 48, a buffer having a plurality of storage areas is formed. Parameters are stored in order from the first storage area of the buffer. When parameters are stored up to the last storage area, the parameters are again stored from the first storage area. Remembered. The parameters selected and read from the recorder 44 are supplied to the video generation unit 34 of both editing apparatuses 2 through the network synchronization unit 36. These storage and readout instructions are given to the recorder 46 from the operator via the user interface 24 and the CPU 25.

  In the processing performed by the editing apparatus 2 for such multi-screen, video effect, audio effect assignment, parameter change storage, and readout, initialization is first performed as shown in FIG. 4 (step S2). Of the two editing devices 2, one of the editing devices 2 selects a video generation material and audio material, selects an effect to be applied to video and an effect to be applied to audio, and selects parameters for these effects during initialization. Is done. Furthermore, the size of the virtual screen area 44 and the display area to be output to the corresponding display device 4 in the virtual screen area 44 are set. The selected video generation material selection instruction, audio material selection instruction, effect instruction, parameter value, virtual screen area size and display area instruction are transmitted to the other editing apparatus 2 via the network. Is done. The other editing device 2 performs video generation material selection, audio material selection, effect selection, and virtual screen area setting in initialization based on data transmitted via the network.

  Following the initialization, each editing device 2 starts a drawing process (step S4). In the drawing process, as shown in FIG. 5, first, the set drawing parameters are acquired (step S40). Next, a drawing LFO process for the LFO 38 is performed based on the acquired drawing parameters and BPM data (step S42). Then, in accordance with the parameters from the LFO 38, an image obtained by applying an effect to the video generation material is drawn in the virtual screen area (step S44). A display area to be output to the corresponding display device 4 is acquired from the virtual screen area for which drawing has ended (step S46), and an image is output from the display area (step S46). . Thereafter, this process is repeated.

  Following this drawing processing, sound processing is performed (step S6). In the sound processing, as shown in FIG. 6, first, drawing parameters are acquired (step S60). Next, this drawing parameter is converted into a sound parameter based on the drawing parameter (step S61). Then, it is determined whether an instruction to link the audio LFO 32 and the drawing LFO 38 is given (step S62). If the answer to this determination is yes, a sound parameter based on the drawing parameter converted in step S61 is acquired (step S63). If the answer to the determination at step S62 is no, the parameter set for the LFO 32 is acquired (step S64). Sound LFO processing is performed based on the parameters acquired in step S63 or S64 (step S65). Next, an audio material and an audio stream signal to be effect-processed are input (step S66), an effect is applied to the sound based on the parameter subjected to the sound LFO process (step S67), and an audio output is output (step S68). Thereafter, these processes are repeated.

  Following the sound processing in step S6, it is determined whether the operation of the operator has been performed (step S8). If the answer is no, step S8 is repeated. If the answer is yes, skip back processing is performed. (Step S10).

  In the skip back process, as shown in FIG. 7, it is determined whether an operator that designates a skip back playback time (a skip back playback start time and a skip back playback end time) is operated among the controls (step S100). ). If the answer to this determination is no, skip back playback is not performed, so parameters are input (step S102), the time at that time is acquired as a reference time (step S104), and the recorder 48 prepared for skip back is provided. The value of the write pointer that specifies the write position (write point) of the buffer is advanced by one (step S106), and it is determined whether the buffer is full, that is, whether the write point is the end of the buffer (step S108). If the answer to this determination is yes, the value of the write pointer is returned to the beginning of the buffer (step S110). If the answer to the determination in step S108 is no, that is, when writing to the buffer is possible, or following step S110, the reference time and parameters are written to the write point specified by the write pointer of the buffer (step S112). ). Next, this parameter is passed to the LFO 38 and the video generation unit 34 of each editing apparatus 2 (step S114). Thereafter, such work is repeated every time a fixed time elapses, and the buffer stores the reference time for each elapse of the fixed time and the parameters at that time in order. The continuation is written from the beginning.

  If the answer to the determination in step S100 is yes, the operator that specifies the playback time for skip back is being operated. Therefore, the skip back playback start time and skip back playback end time specified by the operation of this operator are That is, the skip back time is input (step S116). Then, the current reference time is acquired (step S118), and the value of the read pointer to be read at present (initially corresponding to the skip back reproduction start time) is calculated from the skip back start time and the current reference time (step S118). S120). It is determined whether the value of the read pointer exceeds the end of the buffer (corresponding to the end point of skip back reproduction) (step S122). If the answer to this determination is no, the parameter is set from the position specified by the value of the read pointer. Reading (step S124) and passing the parameters to the LFO 38 and the video generation unit 34 of each editing device 2 (step S126). On the other hand, if the answer to the determination in step S122 is yes, the value of the read pointer is changed to a value corresponding to the point in time when the skip back reproduction starts (step S128), and steps S124 and S126 are executed. Subsequent to step S126, it is determined whether an instruction to end skip back playback is given (step S130). If the answer to the determination is yes, skip back playback ends, but the answer to the determination is no. , And again from step S118. At this time, the current reference time is acquired in step S118, and in step S120, a parameter corresponding to the next reference time after a predetermined time has elapsed from the skip back playback start time is read out.

  In the above embodiment, the video LFO and the audio LFO are synchronized in order to synchronize the video and audio. However, in addition to this, the effect parameter values for the video are directly set for audio. By using it as an effect parameter value, the video and audio can be synchronized.

  For example, when an effect such as an afterimage is given on the video side, an echo effect can be given to the sound in synchronization with this. In this case, for example, the number of afterimages, which is one of the afterimage parameters, is used as the number of echo delays, and the size or brightness of the afterimage, which is one of the afterimage parameters, is used as the echo delay level. The interval between afterimages, which is one of the afterimage parameters, is used as the delay time.

  Alternatively, when an effect that draws a shadow is given as an effect for video, a reverb effect can be given on the audio side. In this case, the shadow length for the effect of drawing the shadow is used as the reverb time of the reverb effect, and the shadow density for the effect of drawing the shadow is used as the reverb level of the reverb effect.

  Alternatively, when an effect that changes like a ripple is applied as an effect for video, the effect can be applied by changing the characteristics of the frequency characteristic filter on the audio side. In this case, the crest value of the ripple is used as the depth, the decay rate of the ripple is used as a speed at which the cutoff frequency of the filter changes to a low frequency, and the speed of the ripple is used to change the cutoff frequency of the filter. Use as

  Alternatively, when a color filter effect is used as an effect for video, a frequency characteristic filter can be applied on the sound side to provide the effect. In this case, a parameter for changing the brightness of the image in the color filter is used as a parameter for changing the brightness of the sound (the cutoff frequency of the frequency characteristic filter).

  In addition to this, when the effect of changing the position of the object is given on the video side, the localization of the sound image is moved in proportion to the change of the position of the object as the effect on the audio side, or the object drawn on the video side When the effect of changing the size of the object is given, the sound volume is changed in proportion to the change of the object size on the sound side, or the effect of changing the focus of the object drawn on the image side is given. On the other hand, the cutoff frequency of the frequency filter may be changed in proportion to the change in focus. In these cases, the position movement parameter is used as it is as a localization movement parameter, the detail change parameter as it is as a volume parameter, and the focus change parameter as it is as a cutoff frequency change parameter.

  The combination of the effect parameter for video and the effect parameter for audio can be arbitrarily changed. For example, in the above example, the image such as ripples and the echo to the audio can be synchronized, Conversely, it is possible to synchronize an effect such as an afterimage of video with a frequency characteristic filter for audio.

  In the above embodiment, two editing devices 2 and two display devices 4 are used, but the number of these devices can be further increased. However, when the number increases, the size of the virtual display screen is changed accordingly, and the position of the display area in charge of each display device on the virtual display screen also changes.

It is a block diagram of the editing apparatus which implemented the image | video production apparatus by this invention. FIG. 2 is a block diagram of an editing system in which two editing apparatuses in FIG. 1 are combined. It is a functional block diagram of the editing apparatus of FIG. It is a main flowchart of the process performed with the editing apparatus of FIG. 5 is a detailed flowchart of the drawing process in FIG. 4. It is a detailed flowchart of the sound process in FIG. 5 is a detailed flowchart of skip back processing in FIG. 4.

Explanation of symbols

34 Video generation unit (video generation means)
48 Recorder (storage means)

Claims (2)

Video generation means for generating video by applying effects to video material according to parameters;
Parameter supply means for supplying the parameter to the video generation means by changing it over time;
Storage means for sequentially storing the parameters that change over time from the start of supply of the parameters;
Among the parameters stored in order in the storage means, the parameters after the designated parameter are read in order with the passage of time, and the parameter resupply means for resupplying the video generation means,
A video creation device. 2. The video creation device according to claim 1, wherein the storage unit has a plurality of storage areas, stores the parameters from the top of the storage areas, and stores the parameters to the end of the storage areas, the storage again A video creation device that stores the parameters from the top of the area.

Images