JP2012227712A - Audiovisual system, remote control terminal, hall apparatus controller, control method of audiovisual system, and control program of audiovisual system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an audiovisual system which livens up concert by user's instruction.SOLUTION: The audiovisual system includes a server 10 which controls a hall apparatus by the instruction information of a remote control microphone. The hall apparatus includes a sound collection microphone which acquires the voice in a hall, and a speaker which outputs the voice. The remote control microphone includes an instruction means for detecting the control instruction of the hall apparatus and transmits the control instruction as the instruction information. The server 10 includes a voice feedback unit 140 which feeds back and outputs the voice acquired by the sound collection microphone from the speaker, by the instruction information of the remote control microphone.

Description

  The present invention relates to an audiovisual system, a remote control terminal, a venue device control apparatus, an audiovisual system control method, and an audiovisual system control program, and in particular, an audiovisual system, a remote control terminal, and a venue device that control venue devices according to instructions from a user's remote control. The present invention relates to a control device, an audiovisual system control method, and an audiovisual system control program.

In modern times, concerts performed by artists (singer, performer) and the like are frequently held in venues such as concert halls.
In such concerts, a large number of parties including sound engineers need to operate audiovisual equipment and the like, and the scale is large.

Here, as a remote control method for audiovisual equipment used in a conventional venue, referring to Patent Document 1, a remote control terminal that outputs an infrared control signal, a light receiving unit that receives the infrared control signal, and a light receiving unit connected to the light receiving unit. A system controller, a switcher that is switched by a control signal from the system controller, a plurality of types of audiovisual devices connected to the switcher, and a power supply controller for each audiovisual device. There has been disclosed a remote control method for an audiovisual device configured to select an audiovisual device and to operate the selected audiovisual device with a remote control terminal (hereinafter referred to as Prior Art 1).
According to the prior art 1, since one remote control terminal has a function of selecting and operating a plurality of audiovisual devices, the presenter can easily operate the audiovisual devices according to the contents of the presentation. In addition, when an operation switch for venue environment maintenance equipment such as audio equipment and lighting equipment is incorporated in the same remote control terminal, the venue environment can be improved in accordance with the operation of the audiovisual equipment.

Japanese Patent Laid-Open No. 2005-5878

However, according to the prior art 1, it is not assumed that a user such as an artist operates the venue equipment during a concert.
For this reason, it was not possible to “enliven” the audience at the concert by operating the venue equipment as the concert progressed.

  This invention is made | formed in view of such a condition, and makes it a subject to eliminate the above-mentioned subject.

The audio-visual system of the present invention is an audio-visual system for controlling venue equipment, comprising: instruction means for receiving an instruction input for controlling the venue equipment and transmitting it as instruction information; a microphone unit for acquiring a user's voice; and the venue equipment. A venue equipment control device for controlling, the venue equipment comprises a sound collecting microphone unit for obtaining a venue audio signal, and a speaker unit for outputting sound, the venue equipment control device, according to the instruction information, An audio feedback unit is provided for audio feedback of the audio signal of the venue and outputting from the speaker unit.
In the audiovisual system of the present invention, the audio feedback unit includes delay means for performing the audio feedback so as not to cause howling due to delay, and volume changing means for changing the output of the audio feedback according to the magnitude of the audio feedback. It is characterized by providing.
The audiovisual system according to the present invention is characterized in that the sound volume changing means changes the output of the sound feedback so that a sound volume to be output is reduced according to the magnitude of the sound signal.
The audiovisual system according to the present invention is characterized in that the delay means changes the time of the delay according to the magnitude of the audio signal.
The audiovisual system of the present invention is characterized in that, when the audio signal is small, the delay means adds the recorded audio with random volume and panning.
In the audiovisual system of the present invention, the venue equipment control device includes an audience mood analysis unit that analyzes audience mood from the audio signal of the venue acquired by the sound collection microphone unit and calculates audience mood information. It is characterized by.
The audiovisual system of the present invention is characterized in that the venue device control device changes the output of the audio feedback or the delay according to the audience mood information.
The audiovisual system of the present invention is characterized in that the venue equipment control device includes a signal transmission / reception unit that transmits the instruction information and connects a voice call.
In the audiovisual system of the present invention, the venue equipment control device includes a sound effect control unit that outputs a sound effect from the speaker unit according to the instruction information, and a lighting / effect control that controls lighting and special effects according to the instruction information. And a background control unit that changes an image to be displayed on a background device according to the instruction information.
The audiovisual system of the present invention is characterized in that the venue equipment control device changes any one of the sound effect, the lighting and special effects, and the image displayed on the background equipment according to the audience mood information.
The audiovisual system of the present invention is characterized in that the instruction means includes a microphone unit that acquires a user's voice or a venue voice as an audio signal, and a transmission / reception unit that transmits the audio signal and the instruction information. .
In the audiovisual system of the present invention, the instruction means recognizes the voice of the user from the voice signal and acquires it as the instruction information, and a transmission / reception section that transmits the voice signal and the instruction information. It is characterized by providing.
The remote control terminal of the present invention is a remote control terminal which is an instruction means for transmitting instruction information for controlling the venue equipment of the audiovisual system. The microphone section for acquiring the voice of the user or the venue, and the instruction regarding the progress of the concert An input unit that detects as information, a voice recognition unit that performs voice recognition of the user's voice from the voice signal and acquires the voice as the instruction information, and a transmission / reception unit that transmits the voice signal and the instruction information. Features.
The venue equipment control apparatus of the present invention is a venue equipment control apparatus that controls venue equipment of an audiovisual system, and includes a voice feedback unit that voice-feeds a venue audio signal to a speaker unit.
An audiovisual system control method according to the present invention includes: an instruction unit that transmits instruction information for controlling a venue device; and a venue device control device that controls the venue device according to the instruction information of the instruction unit. In the method, the instruction means detects an instruction for controlling the venue equipment and transmits it as instruction information, the microphone unit obtains a venue audio signal, and the venue equipment control device voice-feeds back the audio signal. And outputting from the speaker unit.
An audiovisual system control program according to the present invention includes: an instruction unit that transmits instruction information for controlling a venue device; and a venue device control device that controls the venue device according to the instruction information of the instruction unit. In the control program for executing, the instruction means detects an instruction to control the venue equipment and transmits it as instruction information, the microphone section obtains a venue audio signal, and the venue equipment control device uses the voice. The signal is voice-feedback and output from a speaker unit.

  According to the present invention, there is provided an audiovisual system capable of exciting a concert or the like by performing audio feedback in which audio acquired by a sound collecting microphone unit is output as audio feedback from a speaker unit based on instruction information from a remote control terminal. be able to.

1 is a system configuration diagram of an audiovisual system X according to an embodiment of the present invention. It is a block diagram which shows the example of a control structure of the server 10 which concerns on embodiment of this invention. It is a conceptual diagram which shows the external appearance of the remote control microphone 20 which concerns on embodiment of this invention. It is a block diagram which shows the control structural example of the remote control microphone 20 which concerns on embodiment of this invention. It is a conceptual diagram which shows the example of the venue apparatus 30 which concerns on embodiment of this invention. It is a block diagram which shows the example of a control structure of the venue apparatus 30 which concerns on embodiment of this invention. It is a flowchart of the concert process which concerns on embodiment of this invention. It is a conceptual diagram which shows the example of a screen of the remote control microphone 20 which concerns on embodiment of this invention. It is a conceptual diagram which shows the content of the audio | voice feedback which concerns on embodiment of this invention.

<Embodiment>
[System configuration of audiovisual system X]
First, the system configuration of the audiovisual system X according to the embodiment of the present invention will be described with reference to FIG.
The audiovisual system X is a system for controlling venues including multipurpose halls such as concert halls, audiovisual rooms, karaoke rooms, amusement facilities, etc., and includes a server 10 (venue equipment control device) and a remote control microphone 20 (remote control terminal). ) And the venue equipment 30 are connected via the network 5.
More specifically, the network 5 includes a LAN, a wireless LAN, IEEE 1394, MIDI, mLAN, an optical fiber line, a dedicated line, or a mobile phone network, a PHS network, a wired telephone line, a power line LAN, cLink, etc. A network of control signals and audio / video signals can be used. Further, as the form of the network 5, an IP network or other star-shaped or ring-shaped network can be used.
The server 10 is a server using a PC such as a PC / AT compatible machine, various control devices, mixing control, and the like, and has a function of controlling the venue device 30 according to an instruction from the remote control microphone 20.
The remote control microphone 20 is used as a microphone by a user such as a singer, performer or presenter at a concert held at the venue, and used as a remote controller (remote controller) for giving instructions for controlling the venue equipment. Equipment.
The venue device 30 is a device that is used in the venue for performing audiovisual equipment, lighting equipment, stage equipment, special effects, and the like. The venue equipment 30 is controlled by the instruction information of the remote control microphone 20 and the instruction information of the administrator of the server 10 as the concert progresses.

[Control Configuration of Server 10]
The control configuration of the server 10 will be described in more detail with reference to FIG. The server 10 includes an audience mood analysis unit 110, a signal transmission / reception unit 120, a venue device control unit 130, a voice feedback unit 140, a sound effect control unit 150, a lighting / effect control unit 160, a background control unit 170, a database 180, and a transmission / reception unit 190. It is comprised including. Each unit is stored in a storage unit such as a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), or a flash memory. It can be realized using hardware resources by executing a control program and controlling various circuits.

More specifically, the audience mood analysis unit 110 is based on the audio signal from the microphone 240 (FIG. 4) of the remote control microphone 20 and the sound collection microphone unit 314 (FIGS. 5 and 6) of the venue equipment 30 ( This is a site for analyzing the “mood” of the audience, viewers, attendees at the venue, etc.).
As this “mood”, the frequency is analyzed by FFT (Fast Fourier Transform), the frequency of the voice of the audience of the concert is acquired, and the change of a predetermined frequency band, etc., HMM (Hidden Markov Model, Hidden Markov Model) etc. Judgment using various artificial intelligence and statistical methods.
Thereby, the audience mood analysis part 110 can discriminate | determine whether the audience is excited in a concert, ie, it is the mood which is "raising".

The signal transmission / reception unit 120 is a part that transmits and receives instruction information and is connected to perform voice communication such as a transceiver mainly using radio waves between devices.
The signal transmission / reception unit 120 can display information such as the state of the instruction information and the connection of the voice call on the monitor of the console unit 135.

The venue equipment control unit 130 is a part that includes various I / Os (Input / Output), switchers, control circuits, and the like for controlling the venue equipment 30.
The venue device control unit 130 can control various devices of the venue device 30 according to instructions from the remote control microphone 20 or the console unit 135, for example, change the output or the like, and turn on / off the power source or the like. .
In addition, the venue device control unit 130 can transmit a command or the like for controlling each device of the venue device 30 connected to the network 5 from the transmission / reception unit 190.

The console unit 135 includes an input unit including a pointing device for controlling the state of each device of the hall device 30 and a dedicated console having a slider, an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an organic This is a part including a display unit such as an EL (Electro Luminescence) or LED (Light Emitting Diode).
The console unit 135 is used by a dedicated administrator such as a recording engineer or a venue staff to manage the server 10 and to control it according to the progress of the concert.
Further, the console unit 135 can detect an instruction from the administrator to the user. The console unit 135 can convert this instruction into instruction information and transmit the instruction information from the transmission / reception unit 290 to the remote control microphone 20.
Further, the console unit 135 can transmit instruction information according to the progress of a concert or the like, and setting information such as voice feedback and sound effects to the remote control microphone 20.

The audio feedback unit 140 is a part that performs audio feedback processing that acquires and amplifies the audio of the audience. As a voice feedback process, the voice feedback unit 140 first acquires voice information that has been A / D converted to a predetermined volume level by a compressor or the like from the sound collecting microphone unit 314 (FIGS. 5 and 6). The audio feedback unit 140 extracts a predetermined frequency component from the audio information, applies a delay, spread spectrum, etc., and outputs it from the speaker unit 315 (FIGS. 5 and 6) without causing howling. Therefore, the audio feedback unit 140 includes a delay unit 142 (delay unit) that performs digital delay, and an output level adjustment unit 145 (volume change unit) that changes the level of the output audio.
As a result, the volume of cheering at the venue can be raised, and the mood of the audience at the concert can be raised. In addition, the recorded cheering voice can be output so that it can be heard naturally according to the acoustic characteristics of the venue.

The sound effect control unit 150 is a part such as a sampler or synthesizer that creates and outputs sound effects.
As the sound effect, for example, a predetermined drum phrase can be prepared and output separately from a concert drummer performing.
Further, a sound effect phrase can also be created based on information created by a holder of the remote control microphone 20 such as a singer and input through the input unit 260 (FIG. 6).

The lighting / effect control unit 160 is a part that creates and controls the lighting pattern of the lighting device 370 of the hall device 30, the operation pattern of the special effect device 350, and the like.
As an operation pattern or the like of the special effect device 350, ON / OFF of a plurality of devices, changes in time series, changes in combination with sound effects and background data, and the like can be used.

The background control unit 170 is a part that creates background data such as a background image of the background device 360 of the hall device 30 and controls display and the like.
As the background image, a moving image, a predetermined still image, or the like can be selected and converted into an image that matches the LED matrix or the like and output.

The database 180 is a database that stores original data of sound effects, backgrounds, lighting and special effect operation patterns, and stores various data.
Data in the database 180 is read and written by each unit as needed.

The transmission / reception unit 190 is a network card, router, hub, or the like for connecting to the network 5.
The transmission / reception unit 190 also includes functions such as load balancing and QoS (Quality of Service), and can transmit and receive information including audio, video, and various control signals in an integrated manner.

[Appearance and control configuration of remote control microphone 20]
First, referring to the external view of FIG. 3, the remote control microphone 20 includes, for example, a sensor unit 220 (instruction unit), a microphone 240, an input unit 260 (instruction unit), and a display unit 270.

  Next, the control configuration of the remote control microphone 20 will be described in detail with reference to the block diagram of FIG. The remote control microphone 20 includes a control unit 200, a storage unit 210, a voice recognition unit 230 (instruction means), and an output unit 250 in addition to the external configuration described above.

  The control unit 200 includes a CPU, a MPU (Micro Processing Unit), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), a dedicated processor, and the like. The control unit 200 executes a program stored in the storage unit 210, performs various calculations, and controls each unit.

  The storage unit 210 is a storage part such as a RAM, a ROM, a flash memory, and an HDD. The storage unit 210 includes a control program 215 executed by the control unit 200 and various data.

The sensor unit 220 (sensor) is a sensor for grasping the user's instruction and the state of the remote control microphone 20 using various sensors such as an acceleration sensor and a pressure sensor. As the acceleration sensor, a three-dimensional acceleration sensor using MEMS (Micro Electro Mechanical Systems) can be used. As the pressure sensor, a sensor using a piezoelectric element, a capacitance sensor, or the like can be used.
The information acquired by the sensor unit 220 is received as an instruction input for concert processing using the remote control microphone 20, used as instruction information, and controls the venue device 30 via the server 10.
The sensor unit 220 can also include a biosensor. As the biometric sensor, a biosensor such as a sweat sensor, an infrared pulse sensor, a blood flow sensor, a blood glucose sensor, or a temperature sensor that measures the tension of the body with an electrical resistance or an infrared transmission amount can be used. Helps you to know your tension and psychological state.

The speech recognition unit 230 (speech recognition means) is a part that includes a DSP, a dedicated processor for speech recognition, and a program executed by this processor.
The voice recognition unit 230 recognizes a word or the like from the voice signal using a known algorithm such as an HMM, and accepts it as an instruction input. At this time, the voice recognition unit 230 can recognize the voice of the user only by separating the voice of the venue and the voice of the user by a process such as a known voice filter. Thereby, the voice recognition unit 230 can also obtain instruction information for controlling the venue equipment 30.
In addition, the voice recognition unit 230 can recognize the emotional state of the user from the voice based on changes in the tone of the song or voice, changes in the speaking speed or pitch, and the like, thereby obtaining instruction information.

The microphone 240 is a part including a capacitor microphone and other ICs. The microphone unit 240 mainly inputs voices such as user voices and musical instrument performances. In addition, the microphone 240 may input the sound of the audience at the venue.
The audio signal input from the microphone 240 is converted into an electric signal, and further converted into a digital signal by an A / D converter and input.
Note that a plurality of microphones 240 can be provided, and a directional microphone that inputs a user's voice and an omnidirectional microphone that inputs a venue voice can be combined. At this time, the audio signals input from the respective microphones can be separated and transmitted separately.

The output unit 250 is a part including sound output means such as a dynamic speaker and a piezoelectric element, force feedback means such as a vibration motor (vibrator), an electromagnetic actuator, and a piezoelectric element.
The output unit 250 outputs the D / A converted electric signal as an audio signal and outputs vibration. Thereby, the user can obtain confirmation about each instruction without looking at the display unit 270.

The input unit 260 is an input part that includes input means such as buttons such as “up”, “down”, and “select”, numeric keys, cursor keys (cross keys), cancel buttons, various other buttons, a touch pad, and a touch panel. is there.
The input unit 260 also includes other input means related to the user interface.
For example, the input unit 260 detects a signal input by a user using a button or a touch panel of the input unit 260 and converts the signal into an electrical signal. Thereby, instruction input can be received and instruction information can be obtained.

The display unit 270 is a part including a liquid crystal display, an organic EL display, a plasma display, a projector, and the like. The display unit 270 can display an image drawn by the GPU of the control unit 200 or the like.
Moreover, the display part 270 and the input part 260 can also be comprised integrally as a display with a touch panel.
Note that the display unit 270 may be configured as an HMD (Head Mounted Display) so as to be connected separately by wire or wirelessly. In addition, a monitor that can be seen only by an artist, a screen of a projector, and the like can be prepared in a place that can be seen only from the stage of the venue, and can be configured similarly to the display unit 270.

The transmission / reception unit 290 is a part for connecting to the network 5. The transmission / reception unit 290 includes a DSP, an RF modulator, an antenna, and the like, and is connected to the network 5 mainly using a wireless connection to transmit / receive various signals including instruction information.
The transmission / reception unit 290 transmits instruction information and user voice information to the server 10. The transmission / reception unit 290 also receives instruction information from the administrator of the server 10. The instruction information from the administrator of the server 10 can be displayed on the display unit 270 or output by vibration or the like at the output unit 250.

[Appearance and control configuration of venue equipment 30]
Next, referring to the external view of FIG. 5 and the block diagram of FIG. 6, the venue device 30 includes, for example, a sound collecting microphone unit 314 and a speaker unit 315 that become the audio device 310, a stage control device 330, a special effect device 350, A background device 360 and a lighting device 370 are included.
Each device of the venue device 30 can control output, operation, operating range, speed, and the like from the server 10.

The acoustic device 310 is a device including a microphone (microphone), an amplifier, a speaker, a PA (Public Address, a sound loudspeaker) device, a mixing device that controls the volume and output of these devices, and the like.
The sound collection microphone unit 314 is a microphone for acquiring the sound of the audience at the venue. The sound collection microphone unit 314 may also be configured to acquire user voice and performance.
The speaker unit 315 is a speaker that outputs a sound effect, feedback sound, and the like, which will be described later, in addition to a user's voice and performance at a concert, performances of other performers, and recorded voice.

The stage control device 330 is a part of, for example, curtains such as a book or a dark curtain, a sliding stage, a rotating stage, a suspended object, or a stage control device of an amusement facility. The stage control device 330 can make the audience feel a change in appearance as the concert progresses.
The special effect device 350 is a device such as smoke, fog, fountain, shower, firecracker, flash (flash), rotating lamp, siren, discharge, odor substance release, audience seat drive, and the like. This special effect device 350 can produce effects for the audience and enhance the stage effect.
The background device 360 is a device that creates a background image and displays it on the back of the stage or on the stage surface. As the background device 360, an LED array, an LED moving image screen, a projector, or the like can be used. The background device 360 can show the moving image of the background or the like to the audience by changing the amount of light for each wavelength of the pixel light.
The lighting device 370 is a controllable lighting system device used in a venue. Use spotlights, ceiling lights, suspension lights, special effects devices, etc. The lighting device 370 can change various lighting and effects as the concert progresses.

[Stage equipment control processing by audio-visual system X]
Next, with reference to FIGS. 7 to 9, the stage equipment control process by the audiovisual system X according to the embodiment of the present invention will be described.
In this process, for example, at the concert venue, the server 10 analyzes the “rising” of the audience or controls the venue equipment 30.
At this time, according to the progress, the user who is an artist gives various instructions using the remote control microphone 20, and transmits this instruction information to the server 10. The server 10 controls each device of the venue device 30 based on the received instruction information. In other words, the venue device 30 can be controlled by the user to excite a concert or the like.
The details of each step of the stage equipment control process will be described below with reference to the flowchart of FIG.

(Step S101)
First, the audience mood analysis unit 110 of the server 10 performs audience mood analysis processing.
In this process, first, the spectator mood analysis unit 110 obtains voices near the seats of the spectators from the sound collection microphone unit 314 of the audio equipment 310 of the venue equipment 30, and performs D / A conversion to obtain a voice signal (audience). (Voice information).
The audience mood analysis unit 110 divides this audience voice information into each frequency band by FFT or the like, and calculates the magnitude of speech such as spoken voice and the size of whistle. The level of applause of the audience can also be calculated.
From the calculated information, the audience mood analysis unit 110 also considers the influence of the number of audiences and the sound of the venue, etc., and determines whether it is in a “swelling” state using an HMM or the like.
Specifically, the audience mood analysis unit 110 acquires a frequency band such as a concert audience's voice, and if the audience's cheering voice, song voice, whistle, applause, etc. are large, the audience is excited about the concert. It is determined that the person is in a “swelling” mood.
At this time, it may be configured to discriminate from the audience sound information even when the audience is “boost” feeling such as booing.
The audience mood analysis unit 110 classifies the determined audience mood information into several levels according to a predetermined unit threshold value, outputs the information as audience mood information, and transmits it from the transmission / reception unit 190 to the remote control microphone 20.

Referring to FIG. 8, when receiving the audience mood information from the transmission / reception unit 290, the control unit 200 of the remote control microphone 20 can display it on the display unit 270. In the example of FIG. 8, the control unit 200 can draw the audience mood information in the range of 0% to 100% as the “rising degree” as in the graph 500, for example.
In addition, about this audience mood analysis process, you may acquire information, such as background audio | voice acquired from the microphone 240 of the remote control microphone 20, as audience audio | voice information. Further, the control unit 200 of the remote control microphone 20 can be configured to calculate the mood information of the audience from the audience audio information.

(Step S102)
Next, the voice feedback unit 140 of the server 10 performs voice feedback processing.
Specifically, for example, the user instructs the input unit 260 of the remote control microphone 20 (FIG. 4) to execute audio feedback processing that acquires and amplifies the audience's audio.
This process will be described with reference to the screen example of FIG.
The control unit 200 moves the cursor 400 to the button 510 when the user touches the button 510 with the input unit 260, for example, once. At this time, the type and degree of processing can be selected using the buttons 610, 620, and 630. For example, in the case of audio feedback, how much the audience's voice is acquired can be set using the buttons 610 and 620 within a range such as “level 1 to level 5”. A select button 630 can be used to select an audio feedback method such as delay or spread spectrum.
In addition, when the cursor 400 is on the button 510, the control unit 200 detects that the user has touched twice or more, detects that an instruction of “voice feedback” is given, and accepts it as an instruction input.
When this acceptance is performed, for example, a pop-up such as “speech feedback execution” may be displayed. Further, it is possible to notify that the instruction has been detected by vibrating the output unit 250 or outputting an electronic sound or the like.

In addition, as an instruction of “voice feedback”, when the user calls the audience “No voice, sing!”, The voice recognition unit 230 recognizes the instruction from the voice signal of the user. Also good. Such a relation between the instruction and the voice can be set to recognize a predetermined word or phrase and stored in the storage unit 210.
Further, the control unit 200 may recognize as an instruction that the remote control microphone 20 is swung in a predetermined direction in a predetermined pattern by the acceleration sensor of the sensor unit 220. In addition, the control unit 200 may recognize an instruction using the pressure sensor of the sensor unit 220 as a switch. At this time, if the user is unconsciously excited and grasps the microphone strongly, it can be automatically detected as an instruction. Furthermore, when the sensor unit 220 is a biometric sensor, the control unit 200 can recognize a user's excitement state such as sweating, an increase in body temperature, and an increase in pulse, and can detect it as an instruction based on this. .
The user's instruction using each unit as described above can be similarly detected in the following steps and accepted as an instruction input.
In addition, the user instruction information can be displayed not only on the display unit 270 but also on a monitor or the like where only the voice instruction or the artist can be seen.
Further, the control unit 200 can notify the detection of an instruction by the output unit 250 even for an instruction other than the input unit 260.

When the control unit 200 detects a voice feedback execution instruction from the user, the control unit 200 transmits the instruction information from the transmission / reception unit 290.
When the transmitting / receiving unit 190 of the server 10 receives this instruction information, the audio feedback unit 140 performs audio feedback.
The user can also give a “standby” instruction for executing voice feedback. In the case of “standby”, audio feedback can be performed when the audience mood information analyzed by the audience mood analysis unit 110 is equal to or less than a predetermined value.

In this processing, first, the audio feedback unit 140 instructs to turn on the sound collecting microphone unit 314 of the auditorium.
Next, the audio feedback unit 140 acquires the audience's voice as audience audio information from the sound collection microphone unit 314 of the audio equipment 310 of the hall device 30.
The audio feedback unit 140 does not output the audience audio information as it is, but processes the delay unit 142 by performing a delay (delay) of about several milliseconds to several hundreds of milliseconds, spread spectrum processing, or the like.
As this delay, not only simply delaying the waveform of the spectator audio information, but also processing of shifting the panning (position) from side to side or up and down so as to correspond to the plurality of speaker units 315 is performed. Thereby, it is possible to prevent conspicuous cheers coming from the same direction and giving an unnatural feeling. It is also possible to perform processing such as a “chorus” effect in which audience audio information is superimposed with a plurality of delays.
As the spread spectrum, after the sound of the spectator sound information is decomposed into frequency bands, this frequency can be re-synthesized by randomly converting, for example, a predetermined sequence. Furthermore, it is possible to simply divide the spectator audio information into a window audio signal of several milliseconds, and add the audio signal with a random pan.
In addition, the voice feedback unit 140 may acquire waveform data such as cheers that have been recorded and stored in the database 180 in advance, and may add a recorded voice that is processed in addition to the audience voice information. At this time, waveform data such as cheers can also be added in an overlapping manner so that the panning can be heard from a plurality of positions by shifting the panning up and down and left and right.

Here, with reference to FIG. 9, the relationship between each method at the time of performing the audio feedback and the volume level of the audience audio information will be described.
When the user gives a “standby” instruction for voice feedback execution, the voice feedback unit 140 automatically performs voice feedback.
At this time, the audio feedback unit 140 uses the delay unit 142 and the output level adjustment unit 145 to adjust the output of the audio feedback.
For example, the voice feedback unit 140 can perform spectrum spreading and recording voice addition so that the shout of a specific small number of people does not stand out when the cheer is small. At this time, it is possible to increase the audience by increasing the level of the processed voice reproduced by voice feedback. For example, when the maximum volume is set to level 5, the audio feedback unit 140 reproduces the sound at a level of about 4 that is about 4/5.
In addition, when the size of the cheer is moderate, the audio feedback unit 140 designates the delay unit 142 so as to have a longer delay of about 50 to 500 ms, for example. On this basis, the output level adjustment unit 145 is instructed to perform audio feedback at an intermediate audio level, for example, at level 2 which is about 2/5 of the maximum volume. At this time, a recorded sound can be added. By configuring in this way, when the cheer is already increased, it is not disturbed that the cheer is further increased by the volume feedback. Therefore, when the audience starts to swell, it can be made to swell more.
Furthermore, the voice feedback unit 140 can shorten the delay time when the size of the cheer increases because the cheer is already close to noise. In such a case, for example, the audio feedback unit 140 designates a delay of about 2 to 49 ms to the delay unit 142. Then, the audio feedback unit 140 instructs the output level 145 to perform audio feedback at level 1, for example. Thereby, even when the audience is quite excited, it can be further increased.
In this way, by changing the level of audio feedback according to the size of the cheer of the audience, it is possible not only to prevent howling but also to make the audience more exciting without making the audience feel unnatural. .
In addition, by changing the delay time according to the volume of the audience audio information at the venue, more natural audio feedback is possible in order to increase the audience cheer.
Furthermore, by adding a recorded voice in addition to the acquired audience voice information, it is possible to excite the audience without feeling unnatural even when there is almost no cheering.

Note that the voice feedback unit 140 can also stop voice feedback when the size of the cheer is further increased and the user's performance or song is drowned out. In addition to this, it is also possible to perform a process of generating a voice in the reverse phase of the cheering of the audience voice information and suppressing the cheering sound volume.
Also, the amount of the audio feedback can be adjusted as appropriate by the user. Thereby, for example, when the size of the cheer is medium, the amount of audio feedback can be increased, and the adjustment can be made so as to increase quickly as the music of the concert progresses.
In addition, the voice feedback unit can not only control the voice feedback according to the size of the cheer but also perform voice feedback using the audience mood information analyzed by the audience mood analysis unit 110 described above. Thereby, for example, the voice feedback can be further enhanced when the spectator feels more “raised”. In addition, when it is “rising up”, the level of audio feedback can be lowered so that audio feedback deviating from the mood of the audience is not made. On the other hand, when it “swells”, it can be intended to be raised as a larger audio feedback level.

Thereafter, the audio feedback unit 140 converts the processed audience audio information into audio and outputs the audio from the speaker unit 315 of the acoustic device 310 of the hall device 30. At this time, sound may be output not only from a front speaker on the stage side but also from a surround speaker (not shown) arranged so as to go around the audience.
Thereby, the voice of the audience can be output from the speaker unit 315 so as not to perform howling, and a cheer or the like can be increased. With such a production, the mood of the audience can be raised.
In particular, when the volume of the audience audio information is small, unnaturalness can be reduced by outputting the sound from the surround side speaker.

(Step S103)
Next, the sound effect control unit 150 of the server 10 performs sound effect addition processing.
Specifically, when the user gives an instruction to add a sound effect at the input unit 260 or other part of the remote control microphone 20, the control unit 200 detects the instruction information and transmits it from the transmission / reception unit 290. In the example of FIG. 8, an instruction to add a sound effect is detected with the button 520, the type of the sound effect or the like is selected with the buttons 610 to 630, and the volume level or the like can be set.
When the instruction information is received by the transmission / reception unit 190 of the server 10, the sound effect control unit 150 performs a process of adding a sound effect to a sound such as a performance.
Specifically, the sound effect control unit 150 adds, for example, a pre-recorded drum sound and the like and outputs it from the speaker unit 315 of the hall device 30. At this time, the sound effect control unit 150 is synchronized with performance accompaniment, for example, by MIDI (Musical Instrument Digital Interface) or SMPTE (Society of Motion Picture and Television Engineers) time code. It can also be output to play a rhythm pattern. Further, the sound effect control unit 150 can output a whistle, a cymbal sound, a clap sound and the like that excite the mood as a sound effect. At this time, the user can select and instruct a plurality of types of sound effects, rhythms, and the like using the input unit 260 of the remote control microphone 20, and can switch to instruction information corresponding to the selected type.
By adding sound effects in this way, even if a drum or the like cannot be installed on the stage, the sound effects can be controlled and played by the user, and the concert can be further excited.
Further, the sound effect control unit 150 may be configured such that the audience mood information analyzed by the audience mood analysis unit 110 is not so much excited or “excited” and / or the volume level of the acquired audience audio information. When is low, the volume of the sound effect can be increased. Also, the sound effect control unit 150 can be used by changing to a more “flashy” sound effect at this time. For example, if it is a drum sound, it can be changed to a sound effect such as adding a sound of thunder, and the audience can be more excited.

(Step S104)
Next, the illumination / effect control unit 160 of the server 10 performs illumination effect processing.
Specifically, when the user gives an instruction about the lighting effect at the input unit 260 or other part of the remote control microphone 20, the control unit 200 detects the instruction information and transmits it from the transmission / reception unit 290. In the example of FIG. 8, an instruction about the lighting effect is detected by the button 530, and the type or the like of the lighting effect can be selected by the buttons 610 to 630.
When the instruction information is received by the transmission / reception unit 190 of the server 10, the illumination / effect control unit 160 performs processing for controlling illumination such as a stage.
Specifically, the lighting / effect control unit 160 controls the lighting device 370 of the hall device 30. Thereby, the illumination / effect control unit 160 can illuminate, for example, a predetermined position on the stage with a predetermined color or a predetermined pattern. Further, the illumination / effect control unit 160 can perform illumination like a mirror ball by a special effect device of the illumination device 370 or the like. Furthermore, the illumination / effect control unit 160 can perform control such as dimming or brightening the illumination of the audience seat.
The illumination / effect control unit 160 can also recognize the direction in which the user points the remote control microphone 20 from information such as an acceleration sensor of the sensor unit 220 and control the position to which the illumination is applied. At this time, even if the position indicated by the user is not on the stage such as a spectator seat, the lighting can be controlled.
In addition, the lighting / effect control unit 160 may increase the lighting to a predetermined level when the audience mood information is not raised or “swelled” and / or when the volume level of the audience audio information is low. Processing such as changing to a pattern or the like or increasing the amount of light can be performed. With such a process, the audience can be excited without feeling unnatural.

(Step S105)
Next, the background control unit 170 of the server 10 performs background effect processing.
Specifically, when the user gives an instruction about the background effect at the input unit 260 or other part of the remote control microphone 20, the control unit 200 detects this as instruction information and transmits it from the transmission / reception unit 290. In the example of FIG. 8, an instruction for a background image or the like is detected by the button 540, and the background type can be selected by the buttons 610 to 630.
When the instruction information is received by the transmission / reception unit 190 of the server 10, the background control unit 170 performs a background effect process for controlling the stage background to change.
Specifically, the background control unit 170 acquires image data of a background still image or moving image selected by the user from the database 180 and displays it on the background device 360 of the hall device 30. At this time, the background control unit 170 may control the lighting device 370 to be interlocked. The background control unit 170 can also control the performance and the background to be synchronized. Further, the background control unit 170 can detect emotions such as a user's mood from the input information of the sensor unit 220 and can select a background image based on this.
In addition, when the background control unit 170 receives a plurality of user instruction information, for example, the background control unit 170 creates an effect image that spreads an image effect such as a ripple in real time (real time) and displays it on the background device 360 as a background. You can also
Furthermore, the background control unit 170 can display an image obtained by capturing a spectator seat or the like on the background device 360 as a background by pointing a camera (not shown) in the direction in which the user faces the remote control microphone 20.
In addition, the background control unit 170 changes the background image to a more flashy one when the audience mood information is not raised or “swelled” and / or when the volume level of the audience audio information is low. Thus, it is possible to perform processing such as adding an effect. Thereby, an audience can be enlivened.

(Step S106)
Next, the illumination / effect control unit 160 of the server 10 performs special effect processing.
Specifically, when the user gives an instruction to execute a special effect at the input unit 260 or other part of the remote control microphone 20, the control unit 200 detects the instruction information and transmits it from the transmission / reception unit 290. In the example of FIG. 8, a special effect instruction is detected by a button 550 and a type or the like is selected by buttons 610 to 630.
When the transmission / reception unit 190 of the server 10 receives this instruction information, the illumination / effect control unit 160 performs a process of controlling the special effect device 350 of the hall device 30 and controlling the stage control device 330.
When controlling the special effect device 350, the illumination / effect control unit 160 operates, for example, a smoke machine of the special effect device 350 to generate dry smoke using dry ice or the like. Moreover, it is possible to blast firecrackers and fireworks, to generate flashes in a predetermined pattern, to discharge water by fountains and showers, to turn on a rotating lamp, and to sound a siren. In addition, it is possible to discharge, release odorous substances, and move the audience seats together with the background.
When controlling the stage control device 330, the lighting / effect control unit 160 performs control related to the stage. For example, the lighting / effect control unit 160 causes the stage control device 330 to make a user, a back dancer, etc. appear in a “seri”, open and close curtains of the stage background, and operate a stage, a rotating stage, and a suspended object. To embody a special effect.
By controlling such special effects at the initiative of the user, a concert or the like can be excited.
In addition, the lighting / effect control unit 160 increases the effect level of the special effect when the guest mood information is not raised or “swelled” and / or when the volume level of the audience audio information is low. It is also possible to perform control such as. For example, processing such as flashing smoke, enlarging a shower, or rotating a rotating lamp that does not normally rotate can be made more exciting.

(Step S107)
Next, the signal transmission / reception unit 120 of the server 10 performs signal transmission / reception processing.
Specifically, when the user wants to transmit / receive an instruction of “cue” relating to the progress of the concert to other performers, the administrator of the server 10 or the like with the remote control microphone 20, an instruction by the input unit 260 or another part I do. This instruction can be given by a switch or the like that specifies the transmission partner and transmission method. In the example of FIG. 8, selection can be made with the button 560, and the other party to be signaled with the buttons 610 to 630, the presence or absence of voice transmission / reception, and the like can be instructed.
The control unit 200 of the remote control microphone 20 that has detected the instruction information transmits the instruction information to the server 10 using the transmission / reception unit 290.
When the signal transmission / reception unit 190 of the server 10 receives the instruction information of the signal, the signal transmission / reception unit 120 transmits the signal to a partner device that the user wants to transmit. As the counterpart device, for example, the console unit 135 of the administrator of the server 10, a transceiver (not shown) of other performers, and a supporter terminal (not shown) for giving instructions such as applause and cheers to the audience Etc. can be specified.
In addition, it is also possible to connect to the counterpart device so that a voice call is made by the transceiver when the instruction information of the signal is transmitted.
The connection status, instruction content, and the like can also be confirmed on the console unit 135 of the server 10.
This completes the concert process.

By configuring as described above, the following effects can be obtained.
First, by using the audio-visual system X of the present invention, a user such as an artist can be excited by voice feedback, various sound effects, special effects, or the like when the excitement is low. At this time, the remote control microphone 20 can be used to give instructions such as voice feedback, generation of sound effects, lighting control, background change, special effects, etc. Can increase the sex.
Furthermore, the user can also lead and perform production and sound collection. In other words, it is possible to operate with a smaller number of people than before when a concert or the like is held. In addition, it is possible to reduce a load such as rehearsal. Thereby, cost can be reduced.
Furthermore, since it is also possible to transmit and receive transceivers, instruction signals, etc. mainly by radio waves, it is possible to advance the concert in cooperation with each other. This makes it possible to perform a concert or the like according to the artist's intention.
Also, in the event of a contingency, such as when a spectator goes up on the stage, a malfunction of some instruments or special effects equipment does not work, etc., the user immediately informs the administrator, etc. Can be handled to the user. For this reason, the safety of the concert can be enhanced.
Further, in the audiovisual system X of the present invention, audio signals such as cheering of the audience can be separately acquired and stored by the sound collecting microphone unit 314, and the feedbacked cheers and sound effects can be recorded separately. For this reason, even when a music DVD or CD is created, the sound separation performance is good and a high-quality recording can be provided.

A configuration in which each part of the venue equipment 30 is controlled only by the remote control microphone 20 without the server 10 is also possible.
In addition, the remote control microphone 20 may have a simple configuration including only an input unit 260 such as a switch and a transmission / reception unit 190 that transmits a radio signal or an optical signal. Further, the remote control microphone 20 and the input unit 260 can be separated from each other.

  Note that the configuration and operation of the above-described embodiment are examples, and it is needless to say that the configuration and operation can be appropriately changed and executed without departing from the gist of the present invention.

  According to the present invention, the audiovisual system that enables the venue equipment to be actively controlled by a user such as an artist can be used industrially.

5 Network 10 Server 20 Remote Control Microphone 30 Venue Equipment 110 Audience Mood Analysis Unit 120 Signal Transmitting / Receiving Unit 130 Venue Equipment Control Unit 135 Console Unit 140 Audio Feedback Unit 142 Delay Unit 145 Output Level Adjustment Unit 150 Sound Effect Control Unit 160 Illumination / Effect Control Unit 170 Background control unit 180 Database 190, 290 Transmission / reception unit 200 Control unit 210 Storage unit 215 Control program unit 220 Sensor unit 230 Voice recognition unit 240 Microphone 250 Output unit 260 Input unit 270 Display unit 310 Audio equipment 314 Sound collection microphone unit 315 Speaker unit 330 Stage Control Equipment 350 Special Effects Equipment 360 Background Equipment 370 Lighting Equipment 400 Cursor 500 Graph 510, 520, 530, 540, 550, 560, 610, 620, 630 Button X Audiovisual System

Claims (16)

In the audiovisual system that controls the venue equipment,
An instruction means for receiving an instruction input for controlling the venue equipment and transmitting it as instruction information;
A microphone unit for acquiring a user's voice;
And a venue equipment control device for controlling the venue equipment,
The venue equipment is
A sound collection microphone that acquires the audio signal of the venue,
A speaker unit for outputting sound;
The venue equipment control device is:
An audiovisual system, comprising: an audio feedback unit that audio-feeds back the audio signal of the venue according to the instruction information and outputs the audio signal from the speaker unit. The voice feedback unit
Delay means for performing the audio feedback so as not to cause howling due to the delay;
The audiovisual system according to claim 1, further comprising: a volume changing unit that changes an output of the voice feedback according to a magnitude of the voice feedback. The volume changing means is
The audiovisual system according to claim 2, wherein the output of the audio feedback is changed so that an output sound volume is reduced according to a size of the audio signal. The delay means is
The audiovisual system according to claim 2 or 3, wherein the delay time is changed in accordance with a magnitude of the audio signal. The delay means is
The audiovisual system according to any one of claims 2 to 4, wherein when the audio signal is small, the recorded audio is added by random volume and panning. The venue equipment control device is:
The audience mood analysis part which analyzes audience mood from the audio signal of the venue acquired by the sound collecting microphone part and calculates audience mood information is provided. The audiovisual system described in 1. The venue equipment control device uses the audience mood information,
The audiovisual system according to claim 6, wherein the output of the audio feedback or the delay is changed. The venue equipment control device is:
The audiovisual system according to any one of claims 1 to 7, further comprising a signal transmission / reception unit that transmits the instruction information and connects a voice call. The venue equipment control device is:
A sound effect control unit that outputs sound effects from the speaker unit according to the instruction information;
A lighting / effect control unit that controls lighting and special effects according to the instruction information;
The audiovisual system according to claim 1, further comprising: a background control unit that changes an image displayed on a background device according to the instruction information. The venue equipment control device uses the audience mood information,
The audiovisual system according to claim 9, wherein any one of the sound effect, the illumination or special effect, or an image displayed on the background device is changed. The instruction means includes
A microphone unit that acquires a user's voice or the voice of the venue as an audio signal;
The audiovisual system according to any one of claims 1 to 10, further comprising a transmission / reception unit that transmits the audio signal and the instruction information. The instruction means includes
A voice recognition unit that performs voice recognition of the user's voice from the voice signal and acquires the instruction information;
The audiovisual system according to any one of claims 1 to 11, further comprising a transmission / reception unit that transmits the audio signal and the instruction information. In the remote control terminal which is an instruction means for transmitting instruction information for controlling the venue equipment of the audiovisual system,
A microphone unit for acquiring a user's voice or a venue voice;
An input unit for detecting an instruction relating to the progress of the concert as the instruction information;
A voice recognition unit that performs voice recognition of the user's voice from the voice signal and acquires the instruction information;
A remote control terminal comprising: a transmission / reception unit that transmits the audio signal and the instruction information. In the venue equipment control device that controls the venue equipment of the audiovisual system,
A venue equipment control device comprising: a voice feedback unit for voice feedback of a venue signal to a speaker unit for output. Instruction means for transmitting instruction information for controlling the venue equipment;
In a control method of an audiovisual system comprising a venue equipment control device that controls the venue equipment according to the instruction information of the instruction means,
By the instruction means, an instruction to control the venue equipment is detected and transmitted as instruction information,
The microphone part acquires the audio signal of the venue,
The audiovisual system control method, characterized in that the audio signal is fed back and output from a speaker unit by the venue equipment control device. Instruction means for transmitting instruction information for controlling the venue equipment;
In a control program for executing control of an audiovisual system comprising a venue equipment control device that controls the venue equipment according to the instruction information of the instruction means,
By the instruction means, an instruction to control the venue equipment is detected and transmitted as instruction information,
The microphone part acquires the audio signal of the venue,
A control program for an audiovisual system, characterized in that the audio signal is fed back by voice and output from a speaker unit by the venue device control device.

Images