KR20010108147A - Method and apparatus for editing a video recording with audio selections - Google Patents

Abstract

The present invention provides a video recording editing comprising the step of receiving a signal comprising video content (124) and analyzing (128) the video content of the received signal to identify a visual property indicative of the characteristic of the video content. It is about a method. Based at least in part on the identified visual properties of the video, the audio section 128 that synthesizes the received signal is identified in the available plurality of audio selections.

Description

METHOD AND APPARATUS FOR EDITING A VIDEO RECORDING WITH AUDIO SELECTIONS}

Recently, there have been many advances in the field of consumer electronics, in particular entertainment systems. Indeed, many homes in the United States now have televisions, video recorder / playback devices such as videocassette recorders, digital versatile disks (aka digital video disks or DVDs), laser disk players, and the like. In addition, families with video cameras, commonly referred to as "cam-corders," who can make their own movies documenting audio and video of children's birthdays, soccer games, vacations, etc. This is getting more and more. Similarly, although it is not a complete replacement for "still" photography, weddings are often housed in videotapes as well as photo albums.

A more advanced model of such a conventional video camera is characterized by the user being able to edit and manipulate recording. For example, some video cameras allow a user to manipulate the recording to add a title or credit to the beginning of the recording. Some other video cameras also allow the user to “fade in / fade out,” for example, a clear image in a “fuzzy” picture or a clear image in a completely black image. You can also slowly adjust the focus to reverse it. Similarly, some more advanced video recorder / playback devices have the feature that a user can edit a video record to add a title or credit to the record.

However, among the A / V devices described above, instead of repeating the original audio content present in the video recording, the user edits the recording so that audio content, such as sound tracks, Nothing can add poems, sonnets, or other enhanced audio. For example, if someone wants to put background music into their video recording of a sea using a conventional home A / V device, they can re-record the audio track of the video recording. Replace the sound of the sea with "background" music, which becomes the main audio content in the video record. One skilled in the art is, for example, a device that receives two signals and synthesizes them into a composite signal, and the editor synthesizes the audio into the video recording using an audio "mixer" corresponding to a general professional editing equipment. By augmenting this dilemma can be solved well. However, audio mixers, which are typical professional editing systems, are expensive and have difficulty in installation and use. That is, conventional conventional audio mixers require the user to select the audio content synthesized in the video recording, and the user must synchronize the audio selection with the main audio content and set various levels of audio (e.g., volume). do.

So even if the cost of such a professional mixer is not enormously expensive, a user of a typical home entertainment system finds out how difficult it is to use a professional audio mixer to edit his home movie.

In conclusion, although it is desirable to add background music or other sound effects to home movies, the consumer electronics industry to date does not meet this need.

What is needed is a method and apparatus for editing video recordings with audio selections that are not affected by the above deficiencies and limitations associated with the prior art.

FIELD OF THE INVENTION The present invention relates to the field of entertainment systems, and more particularly, to a method and apparatus for editing video recordings with audio selections.

Summary of the Invention

According to the gist of the present invention, there is provided a method and apparatus for editing a video record with audio selections. In a first embodiment of the invention, a method of editing a video record comprises receiving a signal comprising video content and analyzing the video content of the received signal to obtain a visual attribute representing the nature of the video content. Identifying. Based at least in part on the identified visual attributes of the video content, the audio selections synthesized to the received signal are identified from the plurality of available audio selections.

Although the present invention will be described using the embodiments illustrated with reference to the accompanying drawings in which like elements have the same reference numerals, the present invention is not limited to these embodiments.

Detailed Description of the Drawings

1 is a block diagram illustrating an entertainment system including the gist of the present invention.

2 is a block diagram of an A / V editing system according to an embodiment of the present invention.

3 is a flowchart showing an example of a method for automatically synthesizing audio selections into video recording, in accordance with the teachings of the present invention.

4 shows an example of a video channel of a received A / V signal depicting a quantization field used to characterize the visual properties of an A / V signal in accordance with an embodiment of the present invention. to be.

FIG. 5 is a diagram illustrating one embodiment of a method for automatically representing characteristics of primary audio content of a video recording according to an embodiment of the present invention.

6 is a diagram illustrating an example of an audio selection content database according to an embodiment of the present invention.

7 is a block diagram of an exemplary computer system suitable for use with an A / V editing system in accordance with an embodiment of the present invention.

8 is a block diagram illustrating an exemplary software architecture for implementing an A / V editing system in accordance with one embodiment of the present invention.

In the following detailed description, for purposes of explanation, specific numbers, materials and configurations are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that these details may not be required to practice the invention. In other instances, well-known features are omitted or simplified for ease of explanation. Also, for ease of understanding, the steps of a given method are described as each step. However, these separately described steps may not be arranged in order depending on performance as required.

1 shows a block diagram illustrating an example of an entertainment system incorporating the subject matter of the present invention. In accordance with the embodiment illustrated in FIG. 1, the entertainment system 100 includes an audio / video (A / V) editing system 128 that incorporates the subject matter of the present invention.

As described in more detail below, in accordance with one embodiment of the present invention, the A / V editing system 128 includes a video content within a video stream and a received signal (hereinafter generally referred to as an A / V signal). Receiving the signal from a video recording / reproducing apparatus including arbitrary audio content included in an audio stream. As such, the A / V editing system 128 analyzes the video content of the received A / V signal identifying the visual property that characterizes the video content and, at least in part, based on the identified visual property. Identifies the appropriate audio selection from the plurality of audio selections available for synthesis into the received A / V signal. Therefore, those skilled in the art will appreciate that the entertainment system 100, including the innovative A / V editing system 128, is an entertainment system with means for automatically editing and enhancing the quality of home movies and other video records with automatically selected audio selections. It will be provided to the user.

As described in the embodiment illustrated in FIG. 1, A / V editing system 128 may be used with a wide variety of A / V components. According to FIG. 1, the entertainment system 100 is a plurality of video recording / reproducing apparatus, routing multiplexers 108 and 112, each communicatively connected within the system 100 as shown in FIG. 1. For example, videocassette recording / players (VCRs) 116, digital versatile discs (aka digital video discs or DVDs) 118, laser discs 120, video cameras 122, television / monitors 126 And a plurality of audio components 132, commonly referred to as audio systems. Except for the A / V editing system which includes the gist of the present invention, each component of the system 100 is intended to represent a wide variety of commonly available A / V components, the function of each of these components As long as and features are known in the art, the detailed description thereof does not need to be described.

As shown in the entertainment system 100 of FIG. 1, the A / V signal may be output from any of a number of signal sources. In the illustrated embodiment of FIG. 1, the entertainment system 10 receives A / V signals from a wireless source and / or a wireline source. That is, the A / V editing system 128 may include many broadcast signal sources, including, for example, television broadcast 103 received at the antenna 102 and satellite broadcast 105 received at the satellite dish antenna 104. A / V signals may be received through any of the broadcast sources. Similarly, entertainment system 100 also receives A / V signals from wired signal sources, such as Internet resources, Intranet resources, cable television broadcasts, for example, over circuit 106. do. Thus, in accordance with the embodiment illustrated in FIG. 1, circuit 106 is a plain old telephone service (POTS) circuit that provides A / V signals from a variety of corresponding wired services, an integrated services digital network. It is intended to represent any of a variety of wired transmission media including, but not limited to, Digital Network, ISDN) lines, cable lines, Ethernet lines, TI / EI lines, and the like. Similarly, A / V editing system 128 may receive A / V signals from any of the plurality of video recording / playback devices 116-122 described above. In an alternative embodiment, television / monitor 126 and A / V editing system 128 may receive broadcast A / V signals directly from separate antenna / wired signal sources or MUX 108 or may disconnect line 110. This broadcast A / V signal can be received directly from the MUX 108. In conclusion, however, the system 100 is an example to illustrate other characteristics of the signal source available to the editing system 128, and systems with more or less performance may be substituted without departing from the spirit or scope of the present invention. It will be apparent to those skilled in the art that it can be.

In one embodiment of the present invention, A / V editing system 128 is a computer system that incorporates the subject matter of the present invention as described below with reference to FIG. In another embodiment, A / V editing system 128 may be a "set top box" in which required processing power is applied and incorporates the subject matter of the present invention. Alternatively, the A / V editing system 128 may be fully embedded within each component of the system 100 (eg, a television or videocassette recorder). Thus, in accordance with the embodiment shown in FIG. 1, the system 100 can receive A / V signals from any of a number of alternate sources and can be found in any of a number of entertainment systems found in many homes. It is to express things.

While introducing the concept of this innovative A / V editing system 128 into the context of the entertainment system 100, FIG. 2 is an A / V editing system suitable for use in an entertainment system that incorporates the subject matter of the present invention. A block diagram illustrating an example structure for 200 is shown. In the example structure shown in FIG. 2, the A / V editing system 200 includes a video analysis module 202, an audio analysis module 208, a controller 206, a display 216. ), A user input device 218, and an audio file unit 212, each of which is connected for communication, as shown. In the exemplary embodiment of FIG. 2, the audio file unit 212 is installed outside the controller 206, but the audio file unit 212 is sufficiently stored in a large storage device (not shown) in the controller 206. It will be apparent to those skilled in the art that it can. In alternative embodiments, the audio file portion 212 may be sufficiently located in a remote location accessible via the Internet and the line 106, or the audio file portion 212 may be located in an audio system (eg, audio system 132). It will be apparent to those skilled in the art that they can be sufficiently located within Here, the A / V editing system 200 and the audio system are interconnected using the line 214. Similarly, in an alternative embodiment, instead of the display device 216, a television (TV) / monitor 126 is used as the video display device for the A / V editing system 200.

If the structure for an exemplary A / V editing system is shown in FIG. 2, in accordance with the teachings of the present invention, an exemplary method for automatically synthesizing to an audio recording is made with reference to the flowchart shown in FIG. In particular, the operation of the editing system 200 for ease of explanation and not limitation, is described with reference to FIG. 3 and continues with reference to FIG. 2. That is, in one embodiment, the user of the A / V editing system 200 may enable the audio augmentation feature to be enabled using the user input device 218. If it is determined in step 302 that the audio synthesis characteristics of the A / V editing system 200 are not available, then the editing session continues without performing an automatic audio synthesis operation (step 304).

However, if the audio synthesis characteristics of the A / V editing system 200 are available, the A / V editing system 200 loads a predetermined amount of video recording into the A / V signal for analysis (step 306). . In one embodiment, A / V editing system 200 loads the entire video record into a buffer (not shown) in video analysis module 206 and audio analysis module 208 for analysis and audio synthesis. In an alternative embodiment, A / V editing system 200 loads a subset of the entire video recording for analysis and synthesis. More specifically, according to a later embodiment, the A / V editing system 200 loads a segment, or samples, of two to three minutes of video recording into a buffer for analysis and synthesis. The length of these samples corresponds to the average length of the audio file of the audio file portion 214. In another embodiment, A / V editing system 200 loads each scene of the video recording into a buffer. As described in more detail below, in one embodiment, the A / V editing system 200 performs initial analysis on the video record and synthesizes the audio to identify each of a plurality of scenes including the video record. Each of these scenes is incrementally loaded into the analysis buffer.

In step 306, if a predetermined amount of video recording is loaded into the A / V signal for analysis, in steps 308 and 310 the video analysis module 202 and the audio analysis module 208 simultaneously receive the received A / V signal. Analyze That is, according to the illustrated embodiment of FIG. 3, not only the audio analysis module 208 for analyzing the audio content of the A / V signal but also the video analysis module 202 for analyzing the video content of the A / V signal may be A / V simultaneously. Provide the V signal. In particular, video analysis module 202 analyzes the video content included in the video stream of the received A / V signal and features the video content with any of a number of visual attributes (step 308). In one embodiment, video analysis module 202 “splits” a video stream received in many quantization regions or quadrants, and includes video content contained within each quantization region of the video stream. Analyze An example of a video stream broken into this quantization region is shown in FIG. 4.

4, a video stream in a corresponding quantization region is shown. In particular, video stream 400 is shown to include many "frames" of video streams (e.g., predetermined amounts of video streams), denoted by reference numerals 402a, 402b through 402n, respectively. As shown in Fig. 4, the quantization region of the frame 402b consists of quantization region 1 (Q ₁ ) 404a to quantization region 9 (Q ₉ ) 404n. Thus, according to one embodiment of the present invention, video analysis module 202 analyzes the video content in the quantization region for each frame of the video stream to characterize the video content with its video attributes.

For example, in one embodiment, the video analysis module 202 is a color attribute with a suitable gradation, e.g., the video content of the received A / V signal is "cool" color (blue, white). , "Hot" (red, yellow) or "worn" or "earthy" (brown, orange), etc., 0 ["cold" (white)] to 10 [" Each quantization region 404a-404n of the video stream 400 is analyzed to see if it corresponds to an output having a scale of "hot" (red)]. In another embodiment, video analysis module 202 may use each quantization region 404a-404n of video stream 400 to reveal video attributes, for example, to reveal whether the video content is "bright" or "dark." Analyze In one embodiment, the video analysis module 202 is configured for content visual attributes and motion visual attributes, for example, whether the video includes urban or rural landscapes, humans, and humans. Each quantization region 404a-404n of the video stream 400 is analyzed for whether it is moving or sitting. An example of a system for quantizing motion / action in video content is US Patent Application No. 08 / 918,681, filed by Adnan Allatar, titled Bit-Rate Control of Video Data Compression). Also in another embodiment, the video analysis module 202 analyzes the quantization regions 404a through 404n of the video stream for each visual attribute already described.

Returning back to the example method shown in FIG. 2, in addition to analyzing the video at step 308, the video analysis module 208 also receives audio content, if any, received in the audio stream of the received A / V signal. , Referred to as primary audio content, and identifies audio attribute information that characterizes the primary audio content included in the audio stream (step 310). For the illustrated embodiment, the purpose of identifying the audio attribute information in step 310 is to set the level of the synthesized audio selection content, for example the volume, if there is a relevant level of the synthesized audio selection content, the main audio content. To completely "smother" or "drowned out" the main audio content. An example for analyzing (eg, step 310) the primary audio content of the received A / V signal is shown in FIG. 5.

5, an example of a method for analyzing main audio content of a received A / V signal is described according to an embodiment of the present invention. In one embodiment shown in FIG. 5, audio analysis operates the audio analysis module 208 to determine whether a received A / V signal includes an audio stream having audio content (ie, primary audio content). Start at step 502. If the audio analysis module 208 determines that the audio stream does not carry any audio content, the audio selection finally selected by the A / V editing system 200 provides only the audio content for recording and the audio analysis. Module 208 provides a level indication to controller 206 via line 214 (step 504). In this environment, this level indication sets the volume of the audio selection selected by the A / V editing system to a "high" level, as it provides only audio content corresponding to this part of the video recording. However, when the audio analysis module 208 determines the received A / V signal as containing audio content (step 502), the audio analysis module 208 determines whether the main audio content contains speech. Is determined next (step 506). In one embodiment, the audio analysis module 208 uses any of many available speech recognition devices to perform this task.

In step 506, when the audio analysis module 208 determines the main audio content as being composed of words, the audio analysis module 208 outputs a level indication to the controller via the line 214 so that words can be heard reliably. . Alternatively, when the audio analysis module 208 determines the main audio content to be nonsense at step 506, it is determined at step 510 whether the main audio content consists of music. In one embodiment, when making this determination, the audio analysis module 208 analyzes the level and width of the relevant frequency spectrum that characterizes the main audio content. For example, spectral analysis provides an indication that the main audio content consists of music, where the main audio content spans a wide frequency spectrum with a large slope in the spectrum that changes over time.

In an alternative embodiment, the audio analysis module 208 may be preprogrammed to have many music selections to compare the received main audio content.

Regardless of the analysis method, if it is already determined that the main audio content is nonsense, if the audio analysis module 208 determines that the main audio content is not music (step 510), the A / V editing system 200 The finally selected audio selection becomes background audio and as a result outputs a level indication to the controller 216 via line 214 so that the main audio content is not completely "buried" in the background sound selection (step) 508). However, if in step 510 the audio analysis module 208 determines the main audio content as being composed of music, the audio analysis module 208 provides this audio attribute information to the controller 206 via the line 214. The controller 206 then allows the user of the A / V editing system 200 to use the option to overwrite the main audio content (eg, music) via the display device 216. Although it will be apparent to those skilled in the art that this step 512 may be completed later via the method 300 shown in FIG. 3, for purposes of continuation and ease of explanation only, the controller 206 may include the A / V editing system 200. The ability to provide these options to the user of is provided in one embodiment as shown in FIG.

If the user chooses to overwrite the main audio content of the received A / V signal, the audio selection selected by the A / V editing system 200 is the primary of the synthesized signal generated by the A / V editing system 200. Since it is audio content, it is set to an appropriate level (step 504). However, in step 512, if the user of the A / V editing system 200 chooses not to overwrite the main audio content consisting of music, the editing operation for a predetermined amount of received A / V signals is continued without synthesizing the audio. do.

Therefore, in accordance with one embodiment of the present invention, the content of the audio stream is only analyzed for setting the level for combining to the received A / V signal. That is, in an exemplary embodiment, the functionality of the audio analysis module 208 is such that the recording level (eg, volume) of the synthesized audio set by the controller 206 does not completely "bury" the main audio content. To identify the audio attribute of the content. However, it will be apparent to those skilled in the art that, in another embodiment, analyzing the audio content is to perform additional functions without departing from the scope and spirit of the present invention.

Continuing to describe the example method shown in FIG. 3, upon receiving the visual attribute information from the video analysis module 202 in step 308, the controller 206 may at least in part based on the received visual attribute information. Appropriate audio selections are identified from the plurality of audio selections contained within 212 (step 312). In one embodiment, controller 206 relies on a database that references the appropriate audio selection based on any of many corresponding visual attributes. One example of a database suitable for use in the controller 206 is shown in FIG. 6.

In one embodiment, shown in FIG. 6, the database 600 is shown to contain many audio selections with many cross-references. As shown in the embodiment of FIG. 6, the database 600 references audio selection content corresponding to the corresponding visual attributes in the two-dimensional database. In one embodiment, the y-axis is characterized by identifying the type of audio selection content (genre 602). However, it will be apparent to those skilled in the art that this type information 602 is one of many alternative means by which type information can be organized within the database 600. Thus, there are many suitable alternative approaches for organizing this information in database 600 without departing from the spirit and scope of the present invention.

The x-axis of the database 600 refers to a corresponding visual attribute, such as, for example, a color attribute 606, a lighting attribute 608, a content / motion attribution 610. The characterization is indicated by the audio selection 604. According to one embodiment of the present invention, the A / V editing system 200 may be preloaded with many audio selections, and references these audio selections stored in the database 600 in accordance with various attribute information. In another embodiment, the A / V editing system 200 provides a user interface that can add or remove audio selections to or from the audio file portion 212, as appropriate to provide a controller ( 206 automatically updates the database 600 with add / delete. Also, although database 600 is shown as a two-dimensional database, it is apparent to those skilled in the art that this is for ease of explanation only. That is, the database 600 may preferably be replaced with a more complex or less complex database having a corresponding effect due to the many complex information contained within that database.

Returning to the embodiment shown in FIG. 3 again, in step 312 the controller 206 automatically selects the audio selections to synthesize to the video record based at least in part on the identified visual properties of the video record. The option of accepting the audio selections of 206 or rejecting the audio selections for another selection is made available to the user of the A / V editing system 200 (step 314). In step 314, if the user accepts the audio selections of the controller 206, the controller 206 synchronizes the audio selections to the video recording and, if possible, at a level automatically determined by the audio analysis module 208, if possible. The audio content and the audio selection are properly mixed to output a synthesized signal including the received A / V signal synthesized with the automatically identified audio selection. In one embodiment, while mixing the automatically identified audio selections with the main audio content, the controller 206 is responsible for the selection of the audio selections at the rate of movement identified in the video content or the tempo of the main audio content. Adjust the "tempo (eg speed)".

However, if the user rejects the audio selections of the controller 206 in step 314, the user provides an interface for the user to access the database 600 of available audio selections, which the user then synthesizes into the video record. Select the audio selection (step 316). At step 318, controller 206 determines whether the end of video recording has been reached. If so, exit this method. However, if the controller 206 determines that it has not yet reached the end of the video recording, the method proceeds to step 306 and then appropriately transfers the predetermined amount of video to the A / V editing system 200 for editing. Load.

Therefore, in accordance with the teachings of the present invention, the A / V editing system 200 analyzes the video content of the received A / V signal and indicates the nature of the video content with many visual attributes. The A / V editing system 200 also analyzes the audio content of the received A / V signal, for example the main audio content, to automatically identify the level for "mixing" the audio selection with the main audio content. Upon generating the visual attribute information, the A / V editing system 200 selects the audio selection contents from the plurality of available audio selection contents based at least in part on the identified visual attribute information. After identifying the audio selections to be synthesized to the main audio content, upon receiving user approval, the A / V editing system 200 automatically determines the main audio content and audio selection at a level determined automatically according to the determined audio attribute information of the audio stream. Mix it. According to one embodiment of the invention, the audio selection is a musical composition, for example a song. In an alternative embodiment, the audio selection content may be automatically selected by the A / V editing system 200 to inspire mood described by visual, sonnet or visual attributes of the received A / V signal. lyrical composition.

According to one embodiment of the invention, the A / V editing system 200 is a computer system suitably configured to analyze the video stream and to indicate in its visual attributes the characteristics of the video content of the received A / V signal, at least Based in part on these visual attributes, the computer system identifies the audio selections synthesized in the received A / V signals and generates for recording a combination of the received A / V signals and the automatically identified audio selections. As described in more detail below, the A / V editing system 200 is intended to represent a wide range of computer systems known in the art. One example of this computer system includes any of a number of audio and video input / output peripherals / interfaces for receiving, digitizing, compressing, and decompressing audio and video signals. generally manufactured by Intel Corporation, based in Santa Clare, California is a desktop system with high-performance microprocessors such as the Pentium processor ^{ⓡ, ⓡ} Pentium Pro processor, Pentium ^ⓡ III processor available from the Intel Corporation. The external size and design for the A / V editing system 200 may be replaced with a more visually appropriate fit within the entertainment system, for example the entertainment system 100. Thus, the A / V editing system 200 can be sufficiently installed in a "set top" box containing the subject matter of the present invention.

7 illustrates a block diagram of a computer system (eg, system 700) incorporating the subject matter of the present invention. In one embodiment, system 700 is A / V editing system 128 of FIG. 1. In the illustrated embodiment, the system 700 includes at least one processor (eg, processor 702) and cache memory 704 connected to one another as shown. System 700 also includes a high performance input / output (I / O) bus 706 and a standard I / O bus 708 as shown. The host bridge 710 connects the process 702 to the high performance I / O bus 706, while the I / O bus bridge 712 connects the high performance I / O bus 706 to the standard I / O bus. (708). The high performance I / O bus 706 connects the network / communication interface 724, system memory 714, audio / video interface board 730, A / V editor 732, and video memory 716. In addition, the display device 718 is connected to the video memory 716. A mass storage device 720, a keyboard and pointing device 722, and an I / O port 726 are connected to a standard I / O bus 708. In one embodiment, the keyboard and positioning device are connected to a standard I / O bus 708 using a serial communication interface cable, while in alternative embodiments, an infrared (IR) interface or a radio frequency (RF) interface. Can be connected for communication.

With continued reference to FIG. 7, components 702-730 perform their own unique functions that are already known in the art. In particular, network / communication interface 724 is used to effect communication between system 700 and any of conventional wide range networks, such as Ethernet, token ring, the Internet, and the like. Similarly, audio / video interface board 730 is used to receive broadcast communications from any of conventional wide area wired and wireless broadcast media, such as RF broadcasts, satellite broadcast cable broadcasts, and the like. Mass storage 720 is provided to permanently store data and programming instructions to perform the functions previously described, while system memory 714 is provided to temporarily store data or programming instructions when executed by process 702. ). I / O port 726 is one or more serial communication ports and / or parallel communication ports used to provide communication between additional peripheral devices that may be connected to system 700 (eg, stereo, speakers, etc.). to be. In conclusion, (collectively), connected components in the system 700 comprises a general purpose computer systems based on the Pentium ^ⓡ processor, Pentium ^ⓡ Pro processor, a Pentium ^ⓡ II processor capable of generally available from Intel Corporation, located in Santa Clare, California However, it is not intended to limit the scope of the hardware system.

In one embodiment, A / V editor 732 includes video analysis module 202 and audio analysis module 208 of A / V editing system 200, while A / V editor 732 includes A / V editor 208. The controller 206, the display device 216, and the user interface device 218 correspond to the processor 702, the display device 718, the keyboard, and the positioning device 722 of the system 700 shown in FIG. 7, respectively. do. In one embodiment, the audio file portion 216 is stored in or remotely located in the mass storage device 720 and is connected to the system 700 via a network / communication interface 724 for communication. In one embodiment, the system 700 receives A / V signals from the network / communication interface 724 and / or the audio / video tuner interface 730 in accordance with the foregoing subject matter, and provides video content for visual attribute information. Analyze and automatically identify audio selections synthesized on the received A / V signal. In an alternative embodiment, system 700 receives A / V signals through an antenna (not shown) connected to one of I / O ports 726 and selects the appropriate audio to be synthesized with the received A / V signals. Automatically identify the content.

Obviously, various components of the system 700 may be rearranged. For example, the cache 704 may be on-chip with a processor 702. Alternatively, cache 704 and processor 702 may be packaged together as a “processor module” with processor 702 referred to as a “processor core”. In addition, the mass storage device 720, the keyboard and the positioning device 722, and / or the display device 718 and the video memory 716 may not be embedded in the system 700. Additionally, the depicted peripherals connected to standard I / O bus 708 may be connected to high performance I / O bus 706 in alternative embodiments; Or in some implementations, only one bus may be present to connect with components of the system 700. In addition, additional components, such as additional processors, storage devices, or memory, may be included in the system 700.

In one embodiment, rather than including a separate A / V editor 732, the innovative features of the present invention described above may be implemented by executing a series of software routines with the system 700 shown in FIG. Such software routines execute a plurality of instructions or a series of instructions executed by a processor, such as processor 702 of system 700. First, a series of commands is stored in a storage device such as mass storage device 720. Stores a series of instructions in a conventional predetermined storage device such as a diskette, CD ROM, magnetic tape, digital general purpose disk (DVD) (also called digital video disk), laser disk, ROM (ROM), flash memory, etc. It is self-evident. A series of commands can be received from a remote storage device, such as a server on a network, via network / communication interface 724 without having to locally store a series of commands. The instructions are copied from a storage device such as mass storage device 720 to system memory 714 and then executed by the access of processor 702. In one embodiment, such software routines are written in the C ++ programming language. However, it is obvious that these software routines can be implemented in various kinds of programming languages. In alternative embodiments, the present invention may be implemented in separate hardware or firmware. For example, an application specific integrated circuit (ASIC) may be programmed to have the functionality of the present invention as described above.

8 is a block diagram illustrating software components having an exemplary software structure in accordance with one embodiment of the present invention. In particular, the example software structure 800 includes an A / V editor agent and driving device associated with the A / V editor application 802, the video analysis module 806 and the audio analysis module 808. And an operating system 810 associated with the BIOS 822. As shown in one embodiment illustrated in FIG. 8, the A / V Editor application 802 interfaces with the A / V Editor Agent 805 and provides the user interface with the A / V Editing System 128 of FIG. 1. do.

In one embodiment, the A / V Editor agent 804 may be connected to the database 812 and the audio file unit 814 to obtain information. In an alternative embodiment, the audio file portion 814 and / or database 812 is an integrated module of the A / V editor agent 804. As shown in FIG. 8, A / V editor agent 804 receives a video signal from a suitable drive within operating system 810 via a communication port. In one embodiment, in accordance with the presently described subject matter, video analysis module 806 performs the step of analyzing the video content of the received A / V signal for any of a number of visual attributes, while The audio analysis module 808, if possible, determines the audio content (eg, primary audio content) of the recorded A / V signal to determine the relevant level (eg, volume) at which the audio selection is recorded. Analyze In accordance with at least some of the output from the video analysis module 806, the A / V editor 804 accessing the database 812 may select audio corresponding substantially to the identified visual properties of the video content of the received signal. When identifying content and the user accesses the identified audio selection, the A / V Editor 802 combines the received A / V signal with the identified audio selection at the level that the audio analysis module 208 automatically selected. Thereby providing a combined A / V signal comprising the synthesized audio selections output through the appropriate drive of the operating system 810.

As suggested above, the BIOS 822 provides an interface between the operating system 810 and various I / O devices connected to the hardware system. This operating system 810, if possible, the BIOS 822 and A / V editor agent 804 as well as other software applications executed by the computer system (eg, system 700) on which the present invention is practiced. A software service that provides an interface between The operating system 810 provides an interface, such as a graphical user interface (GUI), between the user and the system controller. According to one embodiment of the invention, operating system 810 is a Windows ^TM 95 operating system available from Microsoft Corporation of Redmond, Washington. However, the present invention, for example, different versions of Microsoft WindowsTM (for ^{example, Windows TM 3.0, Windows TM 3.1} , Windows TM NT or ^{Windows TM CE), Microsoft DOS,} New York, available OS from International Business Machines Corporation of Armonk / 2, California, is that of the from Apple Computer Incorporated of Cupertino available Apple Macintosh operation system and NeXTSTEP ^ⓡ, California Santa Cruz conventional other operations, such as commercially available UNIX operating systems, from Santa Cruz operations system available to any Self-explanatory

Thus, in accordance with the teachings of the present invention, the A / V editing system analyzes the video content of the video record and overwhelms the main audio content of the video record if possible based at least in part on the visual properties of the video content. At the recording level automatically selected by the A / V editing system, the audio selection content synthesized in the video recording is automatically identified.

Although the apparatus and method of the present invention have been described in accordance with the embodiments described above, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments already described. Various modifications or changes may be made or modified from this specific embodiment without departing from the scope or spirit of the invention. For example, although depicted as separate components, the A / V editing system 128 may be fully embedded in any of the system components (eg, television / monitor / videocassette recording / playback apparatus) in the system 100. Can be. Further, editing system 128 may not include the component modes shown in FIGS. 2-7 or alternatively may include additional components without departing from the scope and spirit of the present invention. Accordingly, the detailed description of the invention is intended to be illustrative rather than restrictive.

Thus, a method and apparatus for editing a video record with audio selections is described.

Claims (37)

Receiving a signal comprising video content, Analyzing the video content of the received signal to identify a visual attribute of the video content, and Identifying an appropriate audio selection from a plurality of available audio selections to synthesize to the received signal based at least in part on the identified visual property of the video content. Mechanism implementation method comprising a. The method of claim 1, Synthesizing the received signal with the identified audio selection to form a composite audio / video (A / V) signal comprising at least one of the video content and the identified audio selection Mechanism implementation method further comprising. The method of claim 1, And wherein said analyzing video content characterizes said video content of said received signal with a color visual attribute. The method of claim 1, And wherein said analyzing video content characterizes the video content of said received signal with a lighting visual attribute. The method of claim 1, And the video content analyzing step characterizes the video content of the received signal with a motion visual attribute. The method of claim 1, The video content analysis step Characterizing the video content of the received signal with the color visual attribute; Characterizing the video content of the received signal with the brightness visual attribute, and Characterizing the video content of the received signal with the motion visual attribute Containing at least one of How to implement the mechanism. The method of claim 1, Identifying a recording level for synthesizing the received signal with the identified audio selection. The method of claim 7, wherein Synthesizing the received signal with the identified audio selection to form a composite audio / video (A / V) signal comprising at least one of the video content and the identified audio selection Mechanism implementation method further comprising. The method of claim 7, wherein The recording level identification step Determining whether the received signal includes audio content, If the received signal includes the audio content, identifying an audio attribute representing a characteristic of the audio content, and Selecting a recording level to synthesize audio content of the received signal with the identified audio selections; Containing How to implement the mechanism. An input port for receiving a signal having video content, A video analysis circuit coupled to the input port and analyzing video content of the received signal to identify a visual property of the video content, and Identify a suitable audio selection in a plurality of audio selections coupled to the video analysis circuitry and at least partially synthesized in the received signal based on the identified visual attributes of the video content of the received video signal; Controller to navigate, Device comprising a. The method of claim 10, And a mass storage device coupled to the controller for storing and retrieving each of the plurality of audio selections. The method of claim 10, And the video analysis circuitry is to determine the amount of color visual attributes of the video content. The method of claim 10, And the video analysis circuitry identifies brightness visual properties of the video content. The method of claim 10, The video analysis circuitry identifies a motion visual attribute of the video content. The method of claim 10, And the video analysis circuitry identifies color visual properties, brightness visual properties, and motion visual properties of the video content. The method of claim 10, The controller automatically mixes the identified audio selections with the main audio content of the received signal, if possible, at the identified recording level. The method of claim 16, And an audio analysis circuit coupled to the input port and a controller, the audio analysis circuitry analyzing the main audio content of the received signal, if possible, to identify audio attributes of the main audio content. The method of claim 17, And the audio analysis circuit selects a recording level for the identified audio selection content based at least in part on the identified audio attribute of the main audio content and provides the recording level to the controller. The method of claim 10, The plurality of audio selections are stored in a remote audio system communicatively coupled to the device via an audio interface, The controller identifies and retrieves the appropriate audio selections in the remote audio system via the audio interface. Device. The method of claim 10, The plurality of audio selections are stored in a network server communicatively connected to the device via a network connection, The controller may identify and retrieve appropriate audio selections from the network server via the network connection. Device. An input port for receiving a signal with video content, A video analysis circuit coupled to the input port and analyzing the video content of the received signal to identify a visual property of the video content; and A controller coupled to the video analysis circuitry for identifying and retrieving an appropriate audio selection from a plurality of audio selections to synthesize to the received signal based at least in part on the identified visual property of the video content Video editing system comprising a. The method of claim 21, And a mass storage device coupled to the controller for storing and retrieving each of the plurality of audio selections. The method of claim 21, And the video analysis circuitry determines the amount of color visual properties, brightness visual properties, and / or motion visual properties of the video content. The method of claim 21, The controller automatically synthesizes the identified audio selections with the main audio content of the received signal, if possible, at the identified recording level. The method of claim 21, And the audio analysis circuitry is coupled to the input port and a controller and analyzes the main audio content of the received signal, if possible, to identify audio attributes of the main audio content. The method of claim 25, The audio analysis circuit selects a recording level for the identified audio selection content based at least in part on the identified audio attribute of the main audio content and provides the recording level to the controller. The method of claim 21, The plurality of audio selections are stored in a remote audio system communicatively coupled to the video editing system via an audio interface, The controller identifies and retrieves the appropriate audio selections in the remote audio system via the audio interface. Video editing system. The method of claim 21, The plurality of audio selections are stored in a network server communicatively connected to the video editing system via a network connection, The controller identifies and retrieves appropriate audio selections from the network server via the network connection. Video editing system. A main audio / video (A / V) functional unit providing a signal comprising video content, and An input port coupled to the main A / V functional unit and receiving a signal having video content, a video coupled to the input port and analyzing the video content of the received signal to identify a visual property of the video content An appropriate audio selection in a plurality of audio selections coupled to the analysis circuitry and coupled to the video analysis circuitry and synthesized to the received signal based at least in part on the identified visual property of the video content of the received signal. Video editing system including a controller for identifying and retrieving a video signal and responding to the primary A / V functional unit Electronic device comprising a. The method of claim 29, The electronic device is a television. The method of claim 29, And the electronic device is a video recorder / playback device. The method of claim 29, The auxiliary video editing system further comprises audio analysis circuitry coupled to the input port and for analyzing the main audio content included in the received signal, if possible, to identify audio attributes of the main audio content. Electronic devices. 33. The method of claim 32, And the audio analysis circuit identifies a recording level suitable for synthesizing the received signal with the identified audio selection based at least in part on the identified visual property of the main audio content. The method of claim 29, The secondary video editing system synthesizes the received signal with the identified audio selection to generate a composite audio / video (A / V) signal comprising at least one of the video content and the identified audio selection. Device. In a medium readable by a mechanism in which a plurality of instructions for performing a video editing service are stored, The video editing service is a service that analyzes video content of a received signal to identify a visual property of video content and the received signal based at least in part on the identified visual property of the video content of the received signal. A service for identifying an appropriate audio selection from a plurality of available audio selections to be composited into the video content of the. Mechanism-readable medium. 36. The method of claim 35 wherein To identify a recording level suitable for the video editing service to identify audio properties of the main audio content and to synthesize the received video record with the identified audio selection based at least in part on the audio property. If possible, further comprising a service for analyzing the received primary audio content. 36. The method of claim 35 wherein A service for synthesizing the received signal with the identified audio selection to generate a composite audio / video (A / V) signal that includes the video editing service and at least one of the video content and the identified audio selection; Further comprising mechanism readable medium.