JP2018170678A - Live video processing system, live video processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically and properly switch cameras recording a live video in which music instruments or the like are played, in accordance with a music being played.SOLUTION: This system detects beats of a music from inputted audio signals, detects the feature amount of the audio signals of each music instrument or player, and determines a music instrument or a player whose detected feature amount has the strongest relation with a maximum feature amount. Then the video of a music instrument or a player whose detected feature amount is determined to have the strongest relation is selected for each interval corresponding to a beat.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a live video processing system, a live video processing method, and a program suitable for recording live video such as a concert, a debate, and a theater.

  Conventionally, when recording live video such as a concert, it is generally performed by arranging a plurality of cameras and shooting from various angles. For example, when recording live images being played by four performers on the stage, prepare four cameras to shoot each performer, and switch the operation of switching the images of the four cameras. By doing this, the live video is completed.

  Such switching operation of the camera image by the switching operator is required to switch to an appropriate camera at any time according to the music or musical instrument to be played. An operation by a person was necessary. In addition, a long operation is required from the start to the end of recording, and there is a problem that a burden on a switching operator who performs a camera switching operation is heavy.

Therefore, it is desired to develop a video switching device that can automatically switch the camera video without requiring an operation by a switching operator.
For example, Patent Document 1 describes a technique for analyzing a rhythm of music data and setting a video switching timing according to the analyzed rhythm when performing a slide show in which a display video is switched in synchronization with reproduction of music data. Has been.

JP 2007-25242 A

  By applying the technique described in Patent Document 1, when a rhythm of music data is analyzed and a slide show is reproduced, the video switching timing can be matched with the rhythm of the music data. However, when this video switching is performed, it is not easy to automatically determine which video is suitable for switching among a plurality of prepared videos.

  For example, as a method of switching images when four performers are playing different instruments (four kinds of instruments) on the stage, the sounds of the four kinds of instruments are recorded by individual microphones, and each time point is recorded. It is possible to switch to the image of the loudest musical instrument. However, some instruments produce relatively loud sounds such as drums and keyboard instruments, while others produce relatively small sounds such as stringed instruments. Since an image of a musical instrument that emits is often selected, it is not preferable as an actual live image.

  An object of the present invention is to provide a video processing system, a video processing method, and a program capable of automatically and appropriately performing camera switching of a live video of a musical instrument or the like according to the music being played. .

  The video processing system of the present invention includes a video switching unit that switches a plurality of video signals photographed by a plurality of cameras prepared for each instrument or player, a beat detection unit that detects the beat of the music from the input audio signal, and an input Based on the beat detected by the beat detection unit while recording the maximum feature amount for each instrument or player detected by the feature amount detection unit and the feature amount detection unit for detecting the feature amount for each instrument or player from the recorded audio signal For each set section, a feature quantity comparison unit for judging the instrument or player having the strongest correlation between the feature quantity detected by the feature quantity detection unit and the maximum feature quantity, and a musical instrument having the strongest correlation judged by the feature quantity comparison unit Alternatively, a camera selection unit that selects a player's video signal by a video switching unit for each set section is provided.

  The video processing method of the present invention includes a beat detection process for detecting a beat of a music piece from an input audio signal, a feature value detection process for detecting a feature value for each instrument or player from the input audio signal, and a feature value. Correlation between the feature quantity detected by the feature quantity detection process and the maximum feature quantity for each section set based on the beat detected by the beat detection process while recording the maximum feature quantity for each instrument or player detected by the detection process Of the musical instrument or player having the strongest correlation determined by the feature amount comparison processing from the plurality of video signals photographed by a plurality of cameras prepared for each instrument or player. Camera selection processing for selecting a video signal for each set section.

  The program of the present invention causes a computer to execute each process of the above video processing method as a procedure.

  According to the present invention, for each section based on the beat of the music, it is possible to switch to an image of a musical instrument or player having a high correlation of feature amounts. It will automatically switch to video, and appropriate live video can be recorded.

It is a figure which shows the structural example of the whole system by the example of 1 embodiment of this invention. It is a block diagram which shows the structural example of the control apparatus which comprises the video processing system by one example of this invention, a video switching apparatus, and a video processing apparatus. It is a frequency characteristic figure which shows the example of detection of the feature-value by one embodiment of this invention. It is a flowchart which shows the flow of the process by one embodiment of this invention. It is explanatory drawing which shows the example of the camera selection and effect state by the example of 1 embodiment of this invention.

  Hereinafter, an example of an embodiment of the present invention (hereinafter referred to as “this example”) will be described with reference to FIGS.

[1. Example of overall system configuration]
FIG. 1 shows an overview of the entire photographing system of this example.
In this example, a base 1, a drum 2, a keyboard 3, and a piano 4 are arranged on a stage 5, and different players play the musical instruments 1 to 4, respectively. And the microphones M1-M4 which collect the performance sound of each musical instrument 1-4 are arrange | positioned for every musical instrument 1-4. In addition, cameras C1 to C4 that photograph the musical instruments 1 to 4 and respective players are arranged near the stage 5 (or on the stage 5).
Each of the cameras C1 to C4 is a video camera that captures a performance state, and is a camera installed at a fixed position.
The arrangement state of the microphones M1 to M4 and the cameras C1 to C4 is an example, and in addition to the microphones M1 to M4 and the cameras C1 to C4, a microphone and a camera may be arranged on or near the stage 5. For example, a camera that captures the entire stage 5 or a microphone that collects the entire stage 5 may be disposed.

The audio signals obtained from the four microphones M1 to M4 are supplied to the audio mixer 6 and become a mixed audio signal in a preset mixed state or a mixed state adjusted at any time according to the performance. The audio signal mixed by the audio mixer 6 is supplied to and output from a speaker (not shown) via, for example, the output terminal 6a. The audio signal mixed by the audio mixer 6 is also supplied to a recording device 8 and a distribution device 9 described later.
Furthermore, the audio signals obtained from the four microphones M1 to M4 obtained by the audio mixer 6 are supplied to the control device 20 of the video processing system 10 without being mixed.

The video processing system 10 includes a control device 20, a video switching device 30, and a video processing device 40.
The control device 20 is configured by a computer, and controls selection of captured images and effects of the cameras C1 to C4 based on the result of analyzing the audio signals obtained from the four microphones M1 to M4. Details of the control performed by the control device 20 will be described later.

  The video switching device 30 is supplied with four video signals taken by the cameras C1 to C4, performs switching processing of the four video signals supplied, and outputs one processed video signal. In addition, the video processing device 40 performs effect processing such as zoom processing and color conversion on the video signal processed by the switching processing of the video switching device 30 and outputs the processed video signal.

  The video signal output from the video processing device 40 of the video processing system 10 is supplied to the projector device 7, the recording device 8, and the distribution device 9. The projector device 7 projects an image based on the supplied video signal. The recording device 8 records the supplied video signal. The distribution device 9 distributes the supplied video signal to the outside. Note that at the time of recording in the recording device 8 and distribution in the distribution device 9, the audio signal output from the audio mixer 6 is also recorded or distributed. Note that the projector device 7, the recording device 8, and the distribution device 9 are not necessarily required depending on the purpose and form of live video recording, and a configuration in which any one or a combination of the two is conceivable.

[2. Example of video processing system configuration]
FIG. 2 is a block diagram showing a functional configuration of the video processing system 10.
The control device 20 analyzes audio signals obtained from the four microphones M1 to M4, and performs processing for controlling the video switching device 30 based on the analysis result. The control device 20 is configured by a computer as described above, and is realized by executing a program for executing analysis processing or the like by an arithmetic processing function provided in the computer. Note that the control device 20 is configured by a computer, and may be configured by dedicated hardware.

As shown in FIG. 2, the control device 20 includes an audio input unit 21, and the audio input unit 21 is supplied with 4-channel audio signals obtained by collecting performance sounds of the musical instruments 1 to 4. The audio input unit 21 has a role of an analog-digital (AD) converter that converts an analog audio signal into a digital audio signal, for example.
Then, the audio input unit 21 mixes the converted digital audio signals and supplies them to the beat detection unit 22. The beat detection unit 22 detects the beat of the entire music from the periodic level change of the audio signal, and supplies the detected beat timing data to the feature amount comparison unit 25.

  The digital audio signal of each channel obtained in the audio input unit 21 is supplied to a fast Fourier transform (FFT) unit 23. The fast Fourier transform unit 23 analyzes each of the supplied audio signals of each channel by performing a fast Fourier transform individually, so that the low frequency (200 Hz band), the middle frequency (1200 Hz band), and the high frequency (8000 Hz band) are analyzed. The intensity (level) or amplitude of the three bands is detected.

FIG. 3 shows an example of a result obtained by analyzing the audio signal at a certain timing by the fast Fourier transform unit 23. FIG. 3A shows the frequency analysis characteristic f _A of the performance sound of the bass 1 collected by the microphone M1. Figure 3B shows the frequency analysis characteristic f _B performance sound of the drum 2 the microphone M2 is collected. FIG. 3C shows the frequency analysis characteristic f _C of the performance sound of the keyboard 3 collected by the microphone M3. 3D shows a frequency analysis characteristic f _D performance sound piano 4 the microphone M4 is collected. 3A to 3D, the vertical axis represents intensity (level), and the horizontal axis represents frequency.
3A to 3D, frequency bands f1, f2, and f3 indicate a low band (200 Hz band), a middle band (1200 Hz band), and a high band (8000 Hz band), respectively.
In this way, the fast Fourier transform unit 23 obtains intensity data for each band for each audio signal of each channel, and supplies the obtained intensity data to the feature amount detection unit 24.

The feature amount detection unit 24 acquires feature amounts indicating the intensities of the low frequency band f1, the middle frequency band f2, and the high frequency band f3 from the data of the intensity of the audio signal of each channel. As the feature amount indicating the intensity of the frequency here, for example, a vector value in which the intensity for each frequency band is arranged is used. The feature amount data of the three bands f1, f2, and f3 of each channel obtained by the feature amount detection unit 24 is supplied to the feature amount comparison unit 25.
The feature amount comparison unit 25 supplies the maximum value data among the feature amounts of the three bands f1, f2, and f3 of the supplied channels to the maximum feature amount recording unit 28, and records this. The recording of the maximum value of the feature value by the maximum feature value recording unit 28 is performed, for example, every half beat when the music starts, and is reset when the music ends.

The feature quantity comparison unit 25 detects the correlation between the maximum feature quantity of each channel recorded in the maximum feature quantity recording unit 28 and the real-time feature quantity supplied from the feature quantity detection unit 24. The correlation values of the two channels are compared, and the rank is determined in descending order of the correlation value. The determination of the ranking with the higher correlation value is made for each section based on the beat of the music being played detected by the beat detector 22. For example, the feature amount comparison unit 25 sets a section for each half beat of the music based on the beat detected by the beat detection section 22, and determines the rank with the highest correlation value for each half beat section. Here, for example, a feature amount is detected from an audio signal at one specific timing within a half-beat section, and the correlation is determined. Alternatively, the feature amount may be continuously detected from the audio signal in the half-beat section, and the correlation may be determined from the continuous feature amount.
Data having a higher correlation value determined by the feature amount comparison unit 25 is supplied to the camera selection unit 26. Further, the correlation value data detected by the feature amount comparison unit 25 is supplied to the effect selection unit 27.

The camera selection unit 26 performs a process of selecting the images of the four cameras C1 to C4 every time the correlation value rank data is supplied. For example, when the correlation value of the feature value obtained from the microphone M2 that collected the performance sound of the drum 2 is 1st in a certain half-beat section, the half-beat section is that of the camera C2 that captured the drum 2. Select a video.
However, the camera selection unit 26 monitors the selection state of the four cameras C1 to C4, and the feature value correlation value is first in order to avoid the selection of a video of a specific camera. In some cases, a camera image having a correlation value of second or lower is selected.
The camera selection data obtained by the camera selection unit 26 is supplied to the video switching unit 31 of the video switching device 30.

The effect selection unit 27 acquires the data of the camera selected by the camera selection unit 26 and the correlation value between the feature values of the audio signal of the musical instrument corresponding to the video captured by the camera. Then, an effect to be applied to the video is selected according to the acquired correlation value, and the obtained effect selection data is output. For example, various effect processes such as an effect process for zooming up the central portion of the video and an effect process for changing the color of the video to a color different from the normal are selected. When the effect selection unit 27 selects an effect to be applied to the video, the correlation value of the feature value of the audio signal is used as an example. The feature value obtained from the audio signal or the value obtained by image analysis of the video signal is used. Etc. may be used. Alternatively, the effect selection unit 27 may select an effect to be applied to the video by combining various values obtained from these audio signals and video signals. Furthermore, the effect selection unit 27 may randomly select an effect to be applied to the video.
The effect selection data obtained by the effect selection unit 27 is supplied to the video processing unit 41 of the video processing device 40.

The video switching device 30 includes a video switching unit 31 to which video signals captured by the four cameras C1 to C4 are supplied. The video switching unit 31 switches from the video signals captured by the four cameras C1 to C4 to the video signals instructed by the camera selection unit 26, and supplies the switched video signals to the video processing device 40.
The video processing device 40 includes a video processing unit 41 that performs effect processing on the video signal switched by the video switching unit 31 of the video switching device 30. The video processing unit 41 performs the effect processing instructed by the effect selection unit 27 on the supplied video signal, and outputs the video signal subjected to the effect processing. If there is an instruction not to perform effect processing, the video processing unit 41 outputs the video signal supplied from the video switching unit 31 as it is.
The video signal output from the video processing unit 41 is supplied to the projector device 7, the recording device 8, and the distribution device 9.

[3. Flow of video selection process]
FIG. 4 is a flowchart showing a flow in which video selection processing and effect selection processing are performed under the control of the control device 20 of the video processing system 10.
First, the beat detection unit 22 performs a beat detection process, and the feature amount comparison unit 25 performs the feature detection in the feature amount detection unit 24 every time the beat detection unit 22 detects a certain beat (in this case, a half beat). A feature amount indicating the intensity of each of the bands f1, f2, and f3 (FIG. 3) detected by the amount detection process is acquired (step S11). Here, the maximum feature amount recording unit 28 records the maximum feature amount of the audio signal of each musical instrument (step S12). The maximum feature amount recorded in the maximum feature amount recording unit 28 is updated and recorded as needed every time the performance progresses from the start of the music performance.

Then, the feature quantity comparison unit 25 calculates a correlation value for each audio signal of each instrument from the maximum feature quantity recorded in the maximum feature quantity recording unit 28 and the current feature quantity detected by the feature quantity detection unit 24. Calculate (step S13). The correlation value here is, for example, a high correlation value when the current feature value is close to the maximum feature value, and a low correlation value when the current feature value is a small value far from the maximum feature value.
Thereafter, the feature amount comparison unit 25 performs a feature amount comparison process for determining the rank from the highest correlation among the four channels of audio signals corresponding to the four types of musical instruments 1 to 4 (step S14). The determination of the ranking with the high correlation is performed for each half-beat section of the music based on the beat detected by the beat detection unit 22.

  Next, the camera selection unit 26 determines whether or not it is appropriate to select the camera image of the musical instrument that is ranked first in step S14 (step S15). Here, for example, when the state where the video of a specific camera continues to be selected continues to some extent, it is determined that the corresponding camera video is not appropriately selected. Even when the camera is frequently switched, it is determined that the selection of the corresponding camera video is not appropriate even when the video of a specific camera is relatively frequently selected. .

  In step S15, when it is determined that it is appropriate to select the camera image of the musical instrument with the first rank (YES in step S15), the camera selection unit 26 gives the rank to the video switching unit 31. An instruction to select an image of a camera that has photographed the instrument that has been ranked first is given (step S16). In step S15, when the camera selection unit 26 determines that it is not appropriate to select the camera image of the musical instrument that ranks first (NO in step S15), the camera selection unit 26 selects the video switching unit 31. In response to this, an instruction is given to select an appropriate camera image from among the images of the cameras ranked second or lower (step S17). In this step S17, for example, the video of the camera having the second highest correlation order is selected. Or you may make it select the image | video of the camera with the least frequency selected within the past fixed period irrespective of a correlation value.

  Then, after giving a video selection instruction in step S16 or step S17, the effect selection unit 27 instructs the video processing unit 41 about effect processing, and sets the video effect state (step S18). As for this effect processing, there are a case where an effect state selected by a correlation value is set and a case where an effect state is set randomly regardless of the correlation value.

[4. Actual video switching example]
FIG. 5 shows an example in which the video switching process is executed.
In FIG. 5, for each timing b1, b2, b3, b4,... For detecting a half-beat, a video selection state and an effect setting state of the cameras C1 to C4 are shown.
For example, the video of the camera C3 is selected in the section from the timing b1 to the timing b5 (two beat section), and no effect is set. Then, in the section from timing b5 to timing b7 (one beat section), the video of the camera C2 is selected, and as the effect state, processing for zooming up the center of the video shot by the camera C2 is performed. This zoom-up process is performed by a so-called digital zoom process. In addition, the video of the camera C1 is selected in the section from the timing b7 to the timing b8 (half-beat section), and no effect is set. Further, in the section from timing b8 to timing b10 (one beat section), the video of the camera C2 is selected, and effect processing for changing the color to sepia is set.

  As described above, by switching the video captured by the camera at every fixed beat based on the correlation with the maximum value of the feature value of the audio signal, the video captured by the plurality of cameras C1 to C4 is automatically displayed. Will be switched according to the performance. In this case, by detecting the beat of the performance sound and judging and switching for each section (here, the half-beat section) set based on the beat, switching is performed in synchronization with the performance of the music, Natural video switching without discomfort is executed.

In addition, the selected video is a state in which the sound of the musical instrument displayed in the video is relatively high, and a video of a performer to be noted among a plurality of performers is selected. Video switching is performed. That is, the magnitude of the performance sound produced by a plurality of musical instruments during performance varies from instrument to instrument. For example, in the example of FIG. 3, the frequency analysis characteristic f _B (FIG. 3B) of the performance sound of the drum tends to be relatively strong. If switching is determined based on sound intensity alone, only the drum image will be selected. Here, in this example, since the judgment is made based on the correlation with the maximum value of the feature value, it can be judged whether or not the feature value of the performance sound of each instrument is high, and appropriate video selection can be performed. become able to.
For example, when the performance sound shown in FIGS. 3A to 3D is obtained, it is determined that the performance sound of the drum is high at the absolute level of the feature amount, but the performance sound of other musical instruments such as a piano is most highly correlated. , Processing such as selecting the video of the instrument. Therefore, it is possible to realize appropriate video switching in accordance with the actual musical instrument performance state. In addition, as a feature-value, the vector value which arranged the intensity | strength for every frequency band, for example is applicable.

[5. Modified example]
In the above-described embodiment, an example in which video switching is performed based only on acoustic analysis has been described. On the other hand, the acoustic analysis of the present invention and real-time image analysis may be combined to further increase the switching accuracy. Specifically, in addition to the excitement of the performance sound of a specific performer, for example, the movement of fingers accelerates, the emotion is high, the facial expression of the performer changes greatly, or the body gesture becomes noticeable, for example, May be detected by image analysis and combined with acoustic analysis to execute video switching processing. For image analysis processing, existing technologies can be used.
Further, the execution of the effect process may be controlled by detecting a feature amount change from image analysis such as acceleration of finger movement and a large change in the player's facial expression.

  In the above-described embodiment, the video processing system 10 includes a control device 20 that performs video switching control based on an audio signal, and a video switching device 30 that performs video switching processing in response to an instruction from the control device 20. The video processing device 40 that performs effect processing is a separate device. On the other hand, one device may be provided with a function as the control device 20 and a function as the video switching device 30 and the video processing device 40. Alternatively, any two of these three devices 20, 30, 40 may be configured as one device. As described above, when the video processing system 10 is configured by one device, either the computer device (and its peripheral devices) or the dedicated hardware may be used.

  In the above-described embodiment, a microphone is arranged for each of the musical instruments 1 to 4, and the performance sound of each musical instrument is obtained as an audio signal of an individual channel. On the other hand, the number of channels of the audio signal may be smaller than the number of instruments, and the obtained audio signal may be analyzed to determine the intensity (feature value) of the performance sound of each instrument. .

  In the embodiment described above, the performance sound of the musical instrument is analyzed and the video switching process is performed. However, the sound other than the musical instrument performance sound is analyzed and the video switching process is performed. Also good. For example, at least one kind of sound collected by a microphone is used as a performer's singing voice, and a correlation value of the characteristic amount of the singing voice is obtained, and compared with a correlation value of the characteristic amount of the musical performance sound, A video switching process may be performed.

  Further, in the embodiment described above, after the video is switched by the video switching unit 31, the effect processing is performed by the effect processing unit 41. However, only the video switching process is performed and the effect processing is executed. You may make it not. Alternatively, the video processing system 10 may automatically perform only the video switching processing, and the effect processing unit 41 may perform the effect processing as needed by manual operation by the operator. As an example of the effect processing, processing other than zooming up the image or changing the color may be performed.

  Further, as the video switching process in the video switching unit 31, as described with reference to FIG. 5, any one video is selected in the section of each half beat. A plurality of videos may be synthesized based on the correlation value of the characteristic amount of the performance sound. For example, when there are two musical instruments with a high correlation value of the characteristic value of the performance sound in a certain section, a picture of two musical instrument players with a high correlation value divided into two in one screen May be arranged on each divided screen.

  Further, for at least one camera to be prepared, the entire stage may be photographed and switched to the image of the entire stage as needed. In this case, for example, when a state in which the correlation value of the feature values of performance sounds of a large number (three or four in this case) of musical instruments is higher than a threshold value is detected, the performance of almost all the players is swelled. As an example, a camera image that captures the entire stage may be inserted as appropriate.

Furthermore, in the above-described embodiment, the video is switched for each half beat based on the detection by the beat detection unit 22, but the setting of the section for each half beat is an example, and one beat or a plurality of beats are set. You may set the section which switches an image for every beat.
Further, the setting of the section for switching the video itself may be performed at random, and may be switched every half beat at a certain timing, and may be switched every one beat or a plurality of beats at another timing. In this case, for example, in a state where the correlation value of the feature value is relatively high (that is, a section where the performance sound is relatively high), the section to be switched is shortened and the correlation value of the feature value is relatively low (that is, the performance sound is relatively low) In (section), it may be variably set so that the section to be switched is lengthened.

In the above-described embodiment, the maximum value of the performance value of the performance sound of each musical instrument is sequentially updated from the start of the music. For example, the maximum value estimated in advance for each musical instrument is recorded in advance as the maximum characteristic value. You may make it set to the part 28. FIG.
Furthermore, although the feature amount of the sound of each channel is obtained from the result of frequency analysis by fast Fourier transform processing, the feature amount for each band may be obtained by filter processing other than fast Fourier transform. This is an example of selecting three bands of low, middle, and high as the bands, and selecting from the features in the three bands. The feature is detected using the number of other bands. You may do it.

  In the example of FIG. 5, when the correlation of performance sounds of a specific instrument continues to be high, the same video is selected continuously. For example, every predetermined period such as half beat or one beat. It is also possible to always switch to another video.

  Further, in the example of FIG. 1, the live video of the concert is shot (recorded), but the present invention is used when shooting (recording) the live video of various other live videos (discussion, theater, etc.). Is also applicable.

  DESCRIPTION OF SYMBOLS 1 ... Base, 2 ... Drum, 3 ... Keyboard, 4 ... Piano, 5 ... Stage, 6 ... Audio mixer, 7 ... Projector apparatus, 8 ... Recording apparatus, 9 ... Distribution apparatus, 10 ... Video processing system, 20 ... Control apparatus , 21 ... audio input unit, 22 ... beat detection unit, 23 ... fast Fourier transform unit (FFT unit), 24 ... feature amount detection unit, 25 ... feature amount comparison unit, 26 ... camera selection unit, 27 ... effect selection unit, 28 ... Maximum feature amount recording unit, 30 ... Video switching device, 31 ... Video switching unit, 40 ... Video processing unit, 41 ... Video processing unit, C1-C4 ... Camera, M1-M4 ... Microphone

Claims (8)

A video switching unit that receives a plurality of video signals captured by a plurality of cameras prepared for each instrument or player, and switches between the plurality of video signals;
A beat detector for detecting the beat of the music from the input audio signal;
A feature amount detection unit for detecting a feature amount for each instrument or player from the audio signal;
While recording the maximum feature amount for each instrument or player detected by the feature amount detection unit, the feature amount detected by the feature amount detection unit for each section set based on the beat detected by the beat detection unit and the feature amount A feature value comparison unit that determines the instrument or player who has the strongest correlation with the maximum feature value;
A live video processing system comprising: a camera selection unit that selects a video signal of a musical instrument or a player having the strongest correlation determined by the feature amount comparison unit by the video switching unit for each set section. The audio signal is a signal of a plurality of channels prepared for each instrument or player,
The live video processing system according to claim 1, wherein the feature amount detection unit detects the intensity of a specific frequency band from each of audio signals of a plurality of channels. When the frequency at which the camera selection unit most recently selected the video signal of the musical instrument or player having the strongest correlation is equal to or higher than a certain frequency, the video signal of the musical instrument or player whose correlation determined by the feature amount comparison unit is the second or later The live video processing system according to claim 1, wherein the live video processing system is selected. Furthermore, an effect selection unit that selects effect processing to be applied to the video signal;
The live video processing system according to claim 1, further comprising: a video processing unit that performs the effect processing selected by the effect selection unit on the video signal selected by the camera selection unit. Beat detection processing to detect the beat of the music from the input audio signal;
A feature amount detection process for detecting a feature amount for each instrument or player from the audio signal;
While recording the maximum feature quantity for each instrument or player detected by the feature quantity detection process, the feature quantity detected by the feature quantity detection process for each section set based on the beat detected by the beat detection process A feature value comparison process for determining the instrument or player who has the strongest correlation with the maximum feature value;
Camera selection for selecting a video signal of a musical instrument or player having the strongest correlation determined by the feature amount comparison processing for each set section from a plurality of video signals photographed by a plurality of cameras prepared for each instrument or player. And a live video processing method. Furthermore, an effect selection process for selecting an effect process to be applied to the video signal;
The live video processing method according to claim 5, further comprising: video processing for performing the effect processing selected by the effect selection processing on the video signal selected by the camera selection processing. A beat detection procedure for detecting the beat of the music from the input audio signal;
A feature amount detection procedure for detecting a feature amount for each instrument or player from the audio signal;
While recording the maximum feature amount for each instrument or player detected by the feature amount detection procedure, for each section set based on the beat detected by the beat detection procedure, the feature amount detected by the feature amount detection procedure and the A feature comparison procedure for determining the instrument or player with the strongest correlation with the maximum feature,
Camera selection for selecting a video signal of a musical instrument or player having the strongest correlation determined by the feature amount comparison procedure for each set section from a plurality of video signals photographed by a plurality of cameras prepared for each instrument or player. Procedure and
A program that causes a computer to execute. Furthermore, an effect selection procedure for selecting the effect processing to be applied to the video signal,
The program according to claim 7, wherein the computer executes a video processing procedure for performing the effect processing selected by the effect selection procedure on the video signal selected by the camera selection procedure.

Images