WO2015169124A1 - 终端混音系统和播放方法 - Google Patents

终端混音系统和播放方法 Download PDF

Info

Publication number
WO2015169124A1
WO2015169124A1 PCT/CN2015/074243 CN2015074243W WO2015169124A1 WO 2015169124 A1 WO2015169124 A1 WO 2015169124A1 CN 2015074243 W CN2015074243 W CN 2015074243W WO 2015169124 A1 WO2015169124 A1 WO 2015169124A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal
robot
speaker
sound
microphones
Prior art date
Application number
PCT/CN2015/074243
Other languages
English (en)
French (fr)
Inventor
黄伟明
Original Assignee
制约智能机械人(Sir)(香港)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 制约智能机械人(Sir)(香港)有限公司 filed Critical 制约智能机械人(Sir)(香港)有限公司
Priority to EP15789399.1A priority Critical patent/EP3142383B1/en
Priority to US15/306,998 priority patent/US9986364B2/en
Priority to DK15789399.1T priority patent/DK3142383T3/da
Priority to CN201580021969.XA priority patent/CN106465008B/zh
Priority to JP2016565188A priority patent/JP6285574B2/ja
Publication of WO2015169124A1 publication Critical patent/WO2015169124A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/305—Electronic adaptation of stereophonic audio signals to reverberation of the listening space
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/301—Automatic calibration of stereophonic sound system, e.g. with test microphone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
    • H04R1/026—Supports for loudspeaker casings
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
    • H04R1/028—Casings; Cabinets ; Supports therefor; Mountings therein associated with devices performing functions other than acoustics, e.g. electric candles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/02—Details casings, cabinets or mounting therein for transducers covered by H04R1/02 but not provided for in any of its subgroups
    • H04R2201/021—Transducers or their casings adapted for mounting in or to a wall or ceiling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/02—Details casings, cabinets or mounting therein for transducers covered by H04R1/02 but not provided for in any of its subgroups
    • H04R2201/023—Transducers incorporated in garment, rucksacks or the like
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R29/00—Monitoring arrangements; Testing arrangements
    • H04R29/004—Monitoring arrangements; Testing arrangements for microphones
    • H04R29/005—Microphone arrays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/02—Spatial or constructional arrangements of loudspeakers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S1/00—Two-channel systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic

Definitions

  • the invention relates to a terminal mixing system for capturing, transmitting, storing and reproducing sounds, and to a terminal mixing playing method.
  • the recording of the existing recorded concerts cannot achieve the stereo effect of the live concert, and the listener who listens to the recording can't feel the live concert.
  • the microphone used to record the concert could not completely record the sound details of all the sounds in the concert, and the concert recording could not show the full details of the single or many sounds of the live concert.
  • the present invention can not realize the stereo effect of the live concert for the recording of the existing recorded concert, and cannot completely detail the sound of the live concert, especially the sound source position and the motion track in the multi-source recording and playback process.
  • the fully demonstrated problem provides a terminal mixing system and a terminal mixing method that can overcome the above-mentioned shortcomings.
  • the present invention provides a method for playing a terminal mix, and the method for playing the terminal mix includes the following steps:
  • each of the acoustic devices is disposed in a terminal environment corresponding to a location of the sounding body corresponding to the acoustic device in the initial environment; and providing motion tracking for communication with a plurality of acoustic devices device;
  • the plurality of microphones respectively record the sounds of the corresponding plurality of sounding bodies as the sound track;
  • the motion tracking device synchronously records the motion states of the plurality of sounding bodies as the motion state file;
  • the plurality of acoustic devices respectively synchronously move the motion states of the corresponding sounding bodies recorded by the motion state file, and respectively play the audio tracks recorded by the corresponding microphones synchronously, thereby playing the terminal mixing.
  • the microphone is disposed opposite to the sounding body corresponding to the microphone, and the distance between the plurality of microphones and the corresponding sounding body is equal.
  • the acoustic device includes a speaker.
  • part or all of the acoustic device is a speaker robot;
  • the speaker robot includes a robot wheel disposed at the bottom of the speaker robot, and a robot arm disposed at the top of the speaker robot;
  • the speaker is disposed on the hand;
  • the step S2 further includes: the speaker robot moves with the motion trajectory of the corresponding sounding body recorded by the motion state file.
  • all of the acoustic devices are speaker robots;
  • the speaker robot includes a robot wheel disposed at the bottom of the speaker robot, and a robot arm disposed at the top of the speaker robot;
  • the speaker is disposed on the hand;
  • the step S0 further includes providing robot furniture;
  • the robot furniture includes a robot seat movably, for carrying a listener, and a robot stand with a display screen or a projection screen for playing video;
  • the step S2 further includes: synchronously moving the robot seat, the robot standing frame, and the speaker robot in the terminal environment, and maintaining the relative position between the robot seat, the robot standing frame, and the speaker robot in the terminal environment.
  • the speaker is slidably disposed on a guide rail controlled by the motor;
  • the step S2 further includes: the speaker moves on the guide rail by the motion track of the corresponding sounding body recorded by the motion state file.
  • the step S1 further includes: providing communication connection with some or all of the plurality of microphones, and corresponding to some or all of the plurality of microphones a sound modification device for communicating with the acoustic device; the sound modification device modifies a sound quality of a track recorded by each of the plurality of microphones or a recording of a part or all of the plurality of microphones
  • the soundtrack adds sound effects
  • the step S2 further includes: playing the corresponding audio track modified by the sound modification device synchronously with the acoustic device corresponding to some or all of the plurality of microphones.
  • the audio track recorded by the plurality of microphones is saved in an EMX file format.
  • the present invention also provides a terminal mixing system, the terminal mixing system comprising a plurality of microphones corresponding to a plurality of sounding bodies in an initial environment for simultaneously recording corresponding sounds of the sounding body into a soundtrack.
  • a motion tracking device that synchronously records the motion states of the plurality of sounding bodies as a motion state file, a terminal environment corresponding to the initial environment of the type and size, and a one-to-one correspondence with the plurality of microphones, and a corresponding microphone communication connection, And communicating with the motion tracking device, synchronously moving the motion state of the corresponding sound body recorded by the motion state file, and synchronously playing the corresponding audio track recorded by the microphone, thereby playing the plurality of terminal mixes.
  • An acoustic device each of the acoustic devices is disposed in a terminal environment corresponding to a location of the sounding body corresponding to the acoustic device in the initial environment.
  • the terminal mixing system and the playing method of the invention respectively record the sounds of the plurality of sounding bodies as sound tracks through a plurality of microphones, and play the corresponding sound tracks through a plurality of speakers corresponding to the positions of the sounding bodies, so that the sounding body can be replayed.
  • the sound has a very high sound quality.
  • FIG. 1 is a schematic diagram of a palm speaker in an embodiment of a terminal mixing system of the present invention
  • FIG. 2 is a schematic diagram of a main product of an integrated terminal mixing according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a first form of integrated terminal mixing product according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a ceiling bracket of the integrated terminal mixing product of the first form shown in FIG. 3;
  • FIG. 5 is a schematic diagram of a second form of an integrated terminal mixing product according to an embodiment of the present invention.
  • FIG. 6 is another schematic diagram of a second form of integrated terminal mixing product according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a third form of an integrated terminal mixing product according to an embodiment of the present invention.
  • AuditionPosition is a logical 3D coordinate used to set up a receiving device, such as a human ear.
  • the listener has one or more receiving devices and also has several neural network structures.
  • the sound signal captured by the receiving device is transmitted to the neural network structure.
  • the neural network structure is usually the brain of the creature, which can form cognition and memory.
  • the reading person has a process of cognition and memory, which is defined as a first sequential mixing process ( FirstOrderMixingProcess).
  • FirstOrderMixingProcess The process of the auditory position, the reflection of the sound, and other factors that add additional features to the finaling sounding while the first sequential mixing process occurs is defined as the second sequential mixing process (SecondOrderMixingProcess).
  • the final vocalization before the receiving device is captured and transmitted to the brain to create cognition and memory.
  • the microphone is a receiving device and is placed at the auditory position; thus, the sound signal can be captured by the microphone and converted into an electronic signal, which is then transmitted to the computer.
  • the present invention provides a terminal mixing system including, corresponding to a plurality of sounding bodies in an initial environment, for synchronously recording corresponding sounds of the sounding body as a plurality of microphones of the audio track, a motion tracking device for synchronously recording the motion states of the plurality of sounding bodies as a motion state file, a terminal environment corresponding to the initial environment of the type and size, and a one-to-one correspondence with the plurality of microphones And communicating with the corresponding microphone and communicating with the motion tracking device, synchronously moving the motion state of the corresponding sound body recorded by the motion state file, and synchronously playing the corresponding microphone recorded audio track, thereby A plurality of onomiar devices that play out the terminal mix; each of the onomatopoeic devices is disposed in a terminal location corresponding to a location of the sounding body corresponding to the pseudo sound device in the initial environment.
  • Microphones have two main uses: one for recording a single sounding body and the other for recording a specific environment.
  • the terminal mix For each track (AudioTrack), the terminal mix is used to record the sound of a single sounding body, then convert the electronic signal into digital audio and transmit the digital audio to a remote environment for playback; or The digital audio is saved in the computer for later playback.
  • stereo audio data can be naturally mapped onto the logical left speaker and the logical right speaker; and when there are more than two speakers, stereo audio data can be When divided into left side audio data (LeftSideAudioData) and right side audio data (RightSideAudioData), a preset needs to be made to decide which speaker is used to reproduce the left side audio data and which speaker is used to reproduce the right side audio data.
  • the arrangement of the speakers for playing back the surround sound data is determined by the surround sound technique.
  • the use of stereo recording and more than one speaker for reproducing the sounding body can greatly amplify the sound image of the sounding body.
  • the left channel is treated as one track and the right channel is treated as another track, and the left and right channels remain independent during transmission and storage of audio data.
  • a terminal refers to an environment for playing back a soundtrack.
  • EM introduces new features including the use of existing speaker technology.
  • Scale 1 The speaker changes from a high degree of generalization to a high degree of specialization to a certain extent;
  • Scale 2 The speaker is changed from a high-level generalization to a highly specialized one by simulating a specific sounding body.
  • the Hi-end HiFi System is highly generalized and can play a very wide range of sounds in a large number of high quality.
  • speaker units there are a large number of speaker units on the speakers to cover different ranges of the sound range.
  • the sound playback device (or speaker) mimics a specific sounding body, which is a new method introduced by EM.
  • This application is intended to create a sounding device that mimics a particular type of sounding body or a single sounding body. For example, this application suggests that the technology development direction is toward the simulation of the following sounding bodies:
  • Liu Ze 990 alto saxophone personal voice, such as Whitney Houston.
  • EM recording for a single target sounding body eliminating the previously mentioned second mixing process is a key point; the auditory position, the reflection of the sound, and other factors can make the recorded sound completely different from the sound of the target object.
  • EM recording for a single target sounding is focused on recording all the details of the initial sound at high resolution.
  • the recording process also digitizes the following information about the sounding body synchronizing with the audio capture activity at a reasonable frequency throughout the recording, including but not limited to:
  • the auditory position relative to a fixed reference point in 3D space the orientation of each sounding body.
  • the microphone and the sounding body corresponding to the microphone are oppositely disposed, and the distance between the plurality of microphones and the corresponding sounding body are equal.
  • the sounding body corresponding to the microphone and the microphone is not limited to the relative arrangement, and the orientation of the microphone may be formed at a certain angle with the orientation of the sounding body corresponding to the microphone.
  • time shifting For time shifting, some techniques use the concept of time shifting, including the use of computer files, storage and transfer, and on-demand playback. In this application, we use all of these techniques when using time shifts.
  • the first form of terminal mixing terminal mixing for multiple simultaneous sounding bodies that are all in a fixed position
  • terminal mixing is to create a terminal capable of simulating the initial environment and all sounds associated with the initial environment; in particular, the terminal mix focuses on accurately replaying the sound of all singers and instruments at the terminal.
  • the playback process can be real-time or time-shifted.
  • the terminal is a terminal environment corresponding to the initial environment of type and size;
  • the terminal includes an acoustic device for simulating the initial sounding body; for example, the terminal includes an advanced high-fidelity system and an advanced speaker, or the terminal includes a HiFi System and a professional speaker for a certain range of sound ranges;
  • Each of the acoustic devices is disposed in the terminal environment corresponding to the fixed position where the sounding body is located in the initial environment.
  • the sound body is a band, which includes multiple guitars, such as a bass guitar, a first electric guitar, a second electric guitar, an acoustic guitar, and the like.
  • the band also includes keyboard instruments, drums and singers.
  • the terminal environment and the initial environment are the same seashore, and the direction of the acoustic device relative to the sea is the same as the direction of the band relative to the sea;
  • the acoustic device includes a guitar speaker, a stereo speaker, a drum sound analog speaker, and a singing voice analog speaker;
  • the keyboard instrument Since the sound of the simulated keyboard instrument is usually mixed with noise, in the terminal environment, the keyboard instrument is simulated by the stereo speaker;
  • Each of the acoustic devices is disposed at the same terminal position as the fixed position of the sounding body in the terminal environment (ie, the initial environment).
  • the sounding body is a plurality of musical instruments
  • the terminal environment is an auditorium whose type and size correspond to the initial environment
  • the acoustic device includes a plurality of professional speakers (or advanced high-fidelity systems), and the plurality of professional speakers (or advanced high-fidelity systems) respectively simulate a plurality of instruments in a one-to-one correspondence;
  • Each professional speaker (or advanced high-fidelity system) is set in the terminal environment in the terminal environment corresponding to the fixed position where the plurality of instruments in the initial environment are located.
  • the show can be used for simultaneous playout in a different terminal environment than the initial environment, or for playback at any time in the same environment and after a live show.
  • the second form of terminal mixing terminal mixing for some or all of the synchronized sounding bodies in motion
  • the second form of terminal mixing uses robotic technology on existing speakers or slidably mounts existing speakers on rails controlled by the motor. In this way, the speaker can move on the guide rail by the motion track of the corresponding sound body recorded by the motion state file.
  • the acoustic device is a speaker robot; the speaker robot includes a robot wheel disposed at the bottom of the speaker robot, a robot arm disposed at the top of the speaker robot; and a speaker is disposed on the hand of the robot arm.
  • the speaker robot moves to a particular 3D position and adjusts the orientation of the speaker based on the information stored on the track.
  • the step S2 further includes: the speaker robot moves with the motion trajectory of the corresponding sounding body recorded by the motion state file.
  • the motion state file may be a video file, or may be a record of the coordinates of the sounding body in the initial environment.
  • the motion state file is recorded by a motion tracking device communicatively coupled to a plurality of acoustic devices;
  • a speaker that moves on a rail is a low-cost way to replay a recording, but the effect of replaying the recording is not satisfactory.
  • each speaker robot needs to cooperate during playback to avoid collisions. While considering how to avoid collisions with the speaker robot, each speaker robot should reduce its impact on the overall effect of the recording playback. Another idea is to make the engagement between the speaker robots so that the impact of the speaker robot collision on the effect of the recording playback is minimized.
  • the speaker robot can move on the stage like a singer or wave to the fans like a singer.
  • the speaker robot can have any shape, and the shape of the speaker robot can be a common speaker shape, an animal shape, a general humanoid robot shape or the like. Any combination of speaker shapes can also be applied to the shape design of the speaker robot.
  • the third form of terminal mixing terminal mixing for unsynchronized sounding bodies
  • the existing music product workshop converts the audio tracks into EMX files; the music product factory also sets the fictitious position information, and sends the fictitious position information to the terminal, the audio It is possible to play back in the terminal. Only time-shifted transmissions can occur in this form of terminal mix.
  • EMX is a file format that contains only terminal mix audio data.
  • the terminal is a terminal environment suitable for audio style
  • the terminal includes an acoustic device for simulating an initial sounding body; for example, the terminal includes an advanced high fidelity system and an advanced speaker, or the terminal includes a HiFi System and a professional speaker for a certain range of sound ranges;
  • Each of the acoustic devices is disposed in the terminal environment corresponding to the fixed position where the sounding body is located in the initial environment.
  • the fourth form of terminal mixing terminal mixing for multiple free sounding bodies
  • the fourth form of terminal mixing requires that the speaker have the following characteristics:
  • the speaker is capable of movement (including moving, fast moving, flying); the speaker will use safety precautions during exercise to prevent the speaker from harming or damaging any object, animal, plant or anyone.
  • the speaker can dance along with the beat. As long as the movement of the loudspeaker is safe, there is no limit to the speed at which the loudspeaker can move within the hearing range, and the time delay of the propagation of the acoustic wave in the air is compensated.
  • the speaker moves within a predetermined physical boundary. If the speaker robot used as a speaker is part of the terminal mixing system, the speaker robot will always return to its initial position of motion.
  • the range of physical boundaries of the terminal is not limited.
  • the volume of each track can be adjusted from 0 to the maximum volume.
  • the speaker's track configuration, speaker position, speaker orientation angle, speaker motion, speaker followed by music rhythm dance, speaker volume, and speaker sound modification are determined by the following factors:
  • Synchronous playback with other terminal mixing systems The simultaneous playback of the terminal mixing system and other terminal mixing systems is based on the transmission of information between the simultaneous server or the terminal mixing system connected via the computer network.
  • the terminal mixing system can calculate the volume level in the terminal. When the volume is too large, the terminal mixing system can issue a visual warning and automatically adjust the volume of all the speakers in an equalized manner. Low to safe volume level.
  • the speakers that respectively play the two or more audios are separated from each other.
  • the mixing system allows the listener to be anywhere inside or outside the speaker area.
  • the acoustic device is a speaker robot
  • the speaker robot itself can be debugged, so that the listener can hear the best sound, or the speaker robot has a wide viewing angle, so that the listener can sit and stand Or walk between the speakers.
  • the listener can also bring his ear closer to the speaker to hear a louder and clearer soundtrack, for example, to hear the details of the singing or violin track.
  • the listener can also be located far away from the speaker and hear high quality sound.
  • the design of the speaker caters to the position of the listener, giving the speaker a wide viewing angle, and the viewing angle of the speaker can be 360° or spherical.
  • This application does not limit how the auditory area (i.e., the area of the auditory location) should be established, but this application cites an example in which the auditory area is the public area or bedroom of the auditorium, and all the listeners are at In the middle of the auditory area, the reading angle of each speaker is 360°.
  • the speaker plays the recorded terminal mix, the sound heard by one person at different positions in the auditory area is different, which is the experience of mixing with the listening terminal and the listening person passing by the beach or busy
  • the experience of the business center is similar.
  • the terminal mix can also allow the listener to pass through the orchestra; or the terminal mix can also allow the listener to bring his ear closer to the singing analog speaker, thereby enabling the listener to Try to listen to the full details of the singer's voice.
  • the original version of the EMX file format is similar to the MIDI file format.
  • the main difference between the EMX file format and the MIDI file format is that the design purpose of the EMX file format has a wide range, which not only caters to the needs of music creators for recording, editing, reading, and the needs of the listeners, but also makes the listeners It also has the ability to record and edit.
  • Another major difference between the EMX file format and the MIDI file format is that the EMX file format allows anyone to modify one track while the other tracks remain the same.
  • EMVS is a file format that contains terminal mix audio data and video data.
  • the modified track result can be a read-only file or a rewritable file.
  • the initial music creator can protect some or all of the music data created by using the terminal mixing tool, the EMX file format, and the copyright protection features of the terminal mixing system, so that the music data cannot be modified after it is released.
  • terminal mixing enables the music production process to take advantage of the social networking and virtual team work characteristics of the Internet, enabling musicians with different talents to work together and create an EMX file from an international perspective.
  • the terminal mixing system further includes a communication connection with some or all of the plurality of microphones for modifying a recording of some or all of the plurality of microphones.
  • a sound modifying device of a sound quality of a soundtrack or a sound effect of increasing a sound of a track recorded by a part or all of the plurality of microphones; and an acoustic device corresponding to some or all of the plurality of microphones The sound modification device communication connection is used to synchronously play the corresponding audio track modified by the sound modification device.
  • any type of speaker can be used as a surround sound speaker to play surround sound (including 5.1 surround sound, 6.1 surround sound and 7.1) as long as the position of the speaker is set to match the position of the surround speaker. Surround sound).
  • the speakers used here are recommended to use universal speakers.
  • the dedicated speakers are not suitable for playing surround sound, and the speaker robots that can only read motion data cannot be used.
  • the terminal mixing system has a predefined surround sound playback mode that is used to make sound on each speaker based on the type of surround sound technology. Terminal Mixing utilizes existing surround sound technology to decode and reproduce surround sound audio data.
  • All speakers are preferably connected together via WiFi.
  • a terminal mixing system utilizes a simple speaker robot.
  • a button such as pressing the "Build a speaker in 5.1 surround mode” button, the speaker will automatically be based on the preferred surround position and the actual terminal structure. Make physical moves. When all the speakers are used, the speaker will return to the initial position.
  • a speaker robot with a robot wheel and a vertical track and a WiFi connection with the terminal mixing system, and a built-in soft robot musician software-speaker robot model A is a speaker robot suitable for surround sound use.
  • this application does not limit the use of such a loudspeaker robot model A to surround sound applications.
  • MIDI is built into EMX files. For example, a music producer or listener can map a generic MIDI instrument to a professional speaker. This logical decision is made by mapping the instrument to the speaker based on the instrument's use. Mapping a musical instrument to a professional speaker is a more appropriate method of imaging. For example, mapping a MIDI grand piano (#1) to an automatic piano is most appropriate.
  • the data in the EMX file about the audio track using motion data is in the existing MIDI file format, but not in the standard digital audio data format.
  • the initial audio data cannot be transmitted in a particular channel, but the operation at the input device can be captured and saved in a MIDI file format.
  • the playback of the terminal mix can be realized in two ways: one is to convert the MIDI data into audio data by using the MIDI rendering module of the terminal mixing system, and the audio data is played by the universal speaker; the other is The speaker robot provides a stream of MIDI data to allow the speaker robot to replay directly.
  • the use of an automatic piano is a good example of how the speaker robot receives MIDI motion data from the terminal mixing system and how the speaker robot converts the MIDI motion data into sounds played in the terminal.
  • MIDI instruments can support the EMX file format so that end users can use the MIDI instrument to create and view music.
  • WAM Wide Area Media
  • the primary purpose of wide area media playback is to selectively use sub-devices that actively replay terminal mixes.
  • WAA WideAreaA
  • the sound is played only by the speakers near the listener, and all of the speakers that play the sound play the same track, or the speakers that play the sound play different tracks associated with the orientation of the listener, respectively.
  • the terminal mixing system can play EMX files or existing stereos on these speakers.
  • the listener can use the terminal mix control tool to play the EMX file so that each track of the EMX file can be played back on one or more speakers.
  • WAV files are played in a similar manner.
  • Terminal mixing broadcast is a form of audio and video broadcasting:
  • the range of terminal mix broadcasts covers the Earth and other appropriate planets, such as Mars.
  • the maximum transmission lag time between any two speakers of the same terminal mixing system is 60 s, wherein the transmission lag time is the difference between the time when the electronic signal is generated on the recording device and the time when the speaker emits the sound wave.
  • Secure Broadcast Data modification is strictly prohibited during the transmission of data between the recording device in the terminal and all speakers. The only exception is the modification based on the wishes of the listener. For example, the listener decides to use the modified rented sound provided by the cloud server on the broadcast feed.
  • the secure broadcast request is digitally marked by the public key crypto module.
  • This application covers the basic elements of broadcasting, however the application is not limited to the broadcast features mentioned herein; a broadcast-related area will enhance existing broadcast technologies to provide terminal mixing audio, such as cable television networks.
  • the EMX file is used to satisfy the data stream. Therefore, the terminal mixing system can reproduce the sound while downloading the terminal mixing data body. This is similar to most existing Internet video streaming technologies.
  • the bandwidth of the terminal mixing data stream is lower than the bandwidth of the video data stream. Therefore, the playback of the audio data stream with the EMX file can be realized by the prior art.
  • the data stream of the EMVS file suitable for video broadcasting is played in the same manner as the data stream of the EMX file.
  • Audio and video broadcasting can be implemented by using a video server by replacing the video file with an EMX file/EMVS file, and adding a client software module to the terminal mixing system, so that the client software module can receive the terminal mixing data body.
  • the terminal mix data body is decoded, rendered, tracked and audio played back on the speaker.
  • All speakers can be connected to the terminal mixing system.
  • a speaker robot can be made in any form.
  • the speaker robot In order to avoid damage, abuse or misuse of the speaker robot, when used outdoors and when the speaker robot is in a dark environment, the speaker robot must emit a clear visual signal to identify the presence of the speaker robot, for example, the speaker robot exhibits The slogan "audio playback is in progress" or "fourth form of terminal mixing" to inform the people around the presence and location of the speaker robot and let them know where and why they can hear sound.
  • the slogan should be clear enough.
  • the slogan can maintain the same brightness as when the speaker robot starts to display the slogan, or the slogan can be dimmed, but the brightness of the slogan every at least 10 minutes will be Change to the initial brightness.
  • the terminal mixing system also includes robotic furniture.
  • ROBOCHAIR is a seat with a high-capacity battery and robot wheels on each leg; a high-capacity battery is used to power the movement of the robot seat; the robot seat and speaker robot Similarly; one or more listeners can sit on the robot seat, which can be moved according to commands from the terminal mixing system.
  • the robot stand (ROBOSTAND) is also a standing frame suitable for the general purpose of the robot.
  • the robot stand is mainly used to hold a display for playing video (such as a 55-inch LED TV display) or a projection screen. .
  • the terminal mixing system regards the robot seat as the center, and determines the command and control signals sent to the robot seat, the robot stand and the speaker robot through the relative positions between the robot seat, the robot stand, the terminal environment, and the speaker speakers. .
  • the relative positions between the robot seat, the robot stand, the terminal environment, and the speaker speaker need only determine the following three items:
  • a virtual virtual environment can be created by simultaneously moving the robot seat, the robot stand and the speaker robot in the terminal environment and calculating the relative position between the robot seat, the robot stand and the speaker robot in the terminal environment.
  • "House moving effect” The house's movement effect depends on the movement of the robot seat, the robot stand and the speaker robot, the floor type, the wind and the mechanical precision in the terminal environment. These factors cooperate to make the house move effect maximize. limit.
  • the same method is also used outdoors, for example, when the terminal mixing system slowly passes through the forest, the user can experience the effect of "forest movement”.
  • the robotic seat, the robotic stand, and the speaker robot in the terminal environment are free to move; the free movement must follow a basic principle: the robotic stand is not used, and the user wants to get the "house” Or terminal environment) mobile effects; robot seats and speaker robots must follow the speaker positioning and auditory rules of the same terminal mix.
  • the robot seat between the fixedly disposed speaker robots is moved by using a WalkingAudience Listening Technique (WalkingAudienceListeningTechnique), or to maintain a relative movement relationship between the listener and the speaker robot.
  • WalkingAudienceListeningTechnique a WalkingAudience Listening Technique
  • robotic movement and remote control capabilities can be extended to other furniture in a similar manner; these include, but are not limited to:
  • Palm speaker (PalmSpeaker)
  • the speakers can be placed on the clothes, and this setup has many craftsmanship and sleek designs.
  • the palm speaker is a wearable terminal mixing product that includes a flat circular bluetooth speaker placed on the palm of the glove, as shown in Figure 1.
  • the software version of JBM2 runs on the user's smartphone.
  • the JBM2 is a device that has a computing power and input and output hardware, such as an RJ45 LAN port and an audio output DAC module.
  • Each glove has a circular LED and a gyroscope inside that is used to detect whether the hand is raised or lowered, or to indicate the orientation of the palm.
  • the audio output of the JBM2 will be mixed in the user's voice, and the user's voice will be played in the palm speaker.
  • the main purpose of the integrated terminal mixing product is to realize all the functions of the terminal mixing of the present application.
  • the main product of the integrated terminal mixing is an electronic product, which has a hardware system with a CPU, a memory and a memory for controlling the mixing of the terminal; the hardware system is loaded with a Linux system and loaded with terminal mixing software to control the terminal. Mixing.
  • the integrated terminal mixing product also has a WiFi communication module for connecting to a local area network (LAN) WiFi communication.
  • the integrated terminal mixing main product is also internally provided with a compartment in which at least four speakers mounted on the track are disposed.
  • the integrated terminal mixing main product has the following main features:
  • the position between the speakers changes depending on the type of terminal mix audio being played.
  • the integrated terminal mixing main product looks like a protective fence to avoid a situation in which a person or an animal is injured during rapid movement of the speaker during the movement of the speaker, especially when the terminal is mixed.
  • the first form of integrated terminal mixing products is the first form of integrated terminal mixing products
  • the first form of integrated terminal mixing products has the following additional features:
  • Figure 3 shows the first form of integrated terminal mixing product.
  • the first form of integrated terminal mixing product 10 includes a ceiling bracket 1 and a robot.
  • the ceiling bracket 1 is fixedly mounted on the ceiling, and the first form of the integrated terminal mixing product 10 except the other part of the ceiling bracket 1 is a robot.
  • the robot is detachably disposed on the ceiling bracket 1.
  • the ceiling holder 1 can be extended to adjust the height of the robot.
  • the robot height ie the height from the floor to the robot
  • the robot height being between 1 m and the ceiling height. Therefore, the listener can debug the height of the robot to listen to the sound of the angle of the listener.
  • the robot When the robot is detached from the ceiling bracket 1, the robot removes the bottom cover and displays the robot wheel 2 at the bottom of the robot, which can be used indoors or outdoors.
  • the user can command the robot to play audio through the remote control software running on his mobile phone, or control the robot movement, or allow the robot to move freely, or keep the robot in compliance with the order of the listener.
  • the visual signal can be transmitted to the user's mobile phone and played on the mobile phone.
  • a plurality of light bulbs 3 are circumferentially disposed on the robot; the plurality of light bulbs 3 can be subjected to conventional lighting control by a common wall switch or a mobile phone (software running on the mobile phone). During audio playback, the user can also flash the plurality of light bulbs 3 of different colors for entertainment purposes.
  • the ceiling bracket 1 can operate like a conventional electric light and is controlled by a conventional wall lamp or a mobile phone (software running on the mobile phone).
  • the second form of integrated terminal mixing product has the following additional technical features:
  • One or more display screens 4 can be automatically turned down or raised according to the result of the collision detection; when the display screen 4 is in use, the display screen 4 is turned up, as shown in FIG.
  • the audible alarm and LED are placed on one or more of the display screens 4.
  • the display 4 is connected to the JBOX-VIDEO output, and the JBOX-VIDEO is only software running on the computer having the display 4.
  • a conventional display can be used in place of the transparent display 4.
  • the third form of integrated terminal mixing products has the following additional technical features:
  • the third form of the integrated terminal mixing product is a speaker robot having a robot wheel or other component that enables the robot to move;
  • the third form of the integrated terminal mixing product has a beautiful appearance, as shown in FIG. 7, the appearance of the third form of the integrated terminal mixing product is an octopus;
  • a third form of integrated terminal mixing product is equipped with a light bulb, LED or laser light;
  • the LED is widely installed on the third form of the integrated terminal mixing product
  • the JBOX-VIDEO product near the integrated terminal mixing product of the third form can be used to control the flat panel LED display
  • the third form of integrated terminal mixing product can be used to control the light bulb, LED or laser light and/or flat panel display on the third form of integrated terminal mixing product.
  • New terminal environment the terminal environment spans a large geographical area, for example, using 100,000 speakers in a garden of 50,000 square meters, one audio track per speaker;
  • New instruments - creating a new artistic experience for people through sounding and terminal mixing techniques.
  • 5,000 glass columns each glass column is 10 meters high and filled with water, and the top of each column is provided with a speaker; all the speakers are communicatively connected to a terminal mixing system; each column is responsible for the harp A unique string of sound.
  • the terminal environment is used to play back MIDI tracks of EMX/EMVS files or to an electronic harp; when the musician plays the harp, the new terminal environment will sound simultaneously.
  • the electronic harp is a conventional harp, and each string of the electronic harp is connected to a microphone.
  • New musical expressions Selectively use all possible and recognized sounding bodies in the terminal environment.
  • the listeners carry their wearable terminal mixing equipment (WEM), where regular speakers are placed on the stage of the concert; each conventional speaker has a flying robot for The conventional speakers are taken off; there are also speaker robots on all four sides of the concert; some of them are moving around the audience.
  • WEM wearable terminal mixing equipment
  • the musicians sing and play the music, the musicians interact with the listeners, give the instruments to the listeners, and let the listeners raise their hands and let their wearable terminal mixes Become part of the terminal mixing system and become part of a concert instrument that can be sung by a wearable terminal mixing device.
  • musicians are free to use all the resources to advance the concert and allow the listeners to participate in the concert in a terminal mix.
  • the time difference of playing a single track of any two different speakers is less than 10-100 milliseconds.
  • the present application provides a preferred method for solving the above problem by synchronizing each speaker of an embedded Linux device with the same Internet time server at least once a day, and all synchronization activities (such as synchronization at the beginning of the playback process) should be Based on two factors, one is the command from the terminal mixing system, the command contains the target running timestamp in the future time; the other is the embedded Linux clock time, the format of the embedded Linux clock time is the operating system epoch time .
  • the method of the present application reduces the time difference of playing a single track of any two different speakers to less than 50 milliseconds.
  • the enhancement of router technology and the advancement of cable replacement cables will further reduce the turnaround period, thus completely eliminating the problem of different track time differences. Setting up a miniature atomic clock in a terminal mixing system is a future solution.
  • play time is 2017-03-17_10:23:59.001 (operating system epoch time, precision is 1 millisecond);
  • the time in the message is obtained, and the local time on the JBM2 device is checked, and the action is taken when the local time reaches the "play time”.
  • Internet communication complies with the TCP/IP protocol, so that we can obtain high quality information transmission guarantee.
  • Method 1 Use the same specifications of hardware and operating system with the same resources, configuration, and running programs
  • the "lock-report-cancel lock" process includes the following steps:
  • step 3 Send the information of step 2 to all the devices in the speaker group. If the speaker does not meet the requirements, the speaker will issue a “cancel lock; at the limited start time of the audio processing module, redefine the audio processing module”, otherwise the speaker will emit “ success”;
  • a sufficient processing resource for synchronously transmitting and receiving a large amount of communication information in the terminal mixing system for example, the processing resource has 100,000 units.
  • the terminal mixing broadcast station Based on the RTMP protocol of Adobe Corporation, the terminal mixing broadcast station provides terminal mixing audio with RTMP protocol, and one RTMP data stream is correspondingly played on one audio track.
  • the local terminal mixing system uses streaming media to decode audio data and synchronizes the playback of all speakers in a synchronized manner.
  • the station master list file format (StationMasterListFileFormat) is the M3U file format.
  • the terminal mixing system downloads the M3U station list on the pre-configured central server; a selection interface is provided to the user to select M3U Station. After that, the terminal mixing system is connected to the M3U station and used.
  • the RTMP protocol begins to download the contents of all tracks synchronously.
  • the decoding, synchronization and playback are then performed on the speakers of the terminal mixing system.
  • the speaker robot Based on a universal speaker, the speaker robot also includes:
  • the substrate comprises a high-capacity battery that can be repeatedly charged through its docking station (DockingStation) or connected to a power source;
  • JBM2 is built in the base, the JBM2 is powered by a high-capacity battery, and the JBM2 is also connected to the terminal mixing system via WiFi;
  • the robot wheel is disposed at the bottom of the base body, the robot wheel has a high-capacity battery to provide power, and the control signal line of the robot wheel is disposed on the back of the JBM2;
  • the substrate further comprising a light sensor disposed at the bottom of the substrate for identifying the color of the track;
  • the substrate further comprises a speaker disposed inside the substrate, the speaker is connected to the JBM2 through an audio signal, and the mono speaker line is connected to the speaker;
  • the substrate further comprises a sensor for detecting a blocking object on four sides of the substrate.
  • a vertical robot arm is arranged on the base body, and a speaker is arranged on the top of the robot arm, and the servo mechanism is disposed at the rear of the JBM2.
  • the vertical robot arm can be a two-part robot arm with a moving platform, or it can be a simple vertical track.
  • An additional software module built into JBM2 is used to identify the track signal at the bottom of the speaker robot; and to determine which part of the speaker robot is moving, and the vertical height of the speaker, based on the decoded position and direct information from the EMX file.
  • the EMX file information is mapped to the robot pose to mimic the position and orientation of the initial sounding body.
  • the software module will also perform collision avoidance from time to time.
  • the docking station after the robot is used, the robot can be put back into the docking station; the docking station serves as the initial position of the robot.
  • the docking station acts as a battery charger that automatically charges the robot's high-capacity battery until it is full.
  • At least one reference MIDI track with a music time signature (eg, a 4/4 beat song) is available;
  • the user can selectively initialize a soft robot running in a virtual machine of the built-in Linux system for each JMB2.
  • the user can initialize one or more soft robots corresponding to one sounding body and send the one or more soft robots to the speaker, but in order to achieve maximum motion flexibility, only one soft robot will be assigned to one speaker.
  • the user can initialize or selectively use another soft robot based on the same soft robot with different parameters. For example, two soft robots of a Fender-Stratocaster sounding body are separately assigned to two speakers; one of the two speakers is used to play chords and the other is used to play solo. Assign an additional soft robot of the solitary vocal body of the junior chord to one of the speakers.
  • Each sound body adds reference pitch, time signature, beat, key, and existing chord to its corresponding artificial intelligence (AI) module and decides what sound to make for the existing chord.
  • the sounding body can emit the percussion sound, bird sound or modal expression of the available chords of the existing chords, as well as various factors such as previous play, next play, reference hit rhythm and use of artificial intelligence.
  • Watching the movement of the speaker robot does not make the audience happy, but adding optical equipment and an LCD display to each speaker robot makes the movement of the speaker more entertaining.
  • a simple volume level LED strip, or a simple level laser gun show can be added to a moving speaker robot.
  • the robot seat is used when the robot seat has the same features as the speaker robot A (having a robot wheel, a vertical track, and a universal speaker connected to the terminal mixing system via WiFi, and a soft robotic artist software is also built in) To replace the ordinary speaker.
  • the positioning of the robot seat can be carried out simply by means of a track or by a reference point of a certain height on the rear wall. From the perspective of safety factors, the robotic arm is not provided on the robot seat to lift the robot seat.
  • the robotic seat has one, two or more seats; the robotic seat can have different designs, materials and types.
  • the robot seat can also have a massage function. However, all factors must be balanced against servo torque and noise levels determined by moving parts, battery capacity, and battery life.
  • the robot stand is a standing frame suitable for general purpose, which is used to support the LED TV display; the difference between the robot stand and the robot seat is that the robot seat can be replaced by the robot stand and move smoothly.
  • the payload can be held securely and safely.
  • WAM Wide Area Media
  • Each speaker (speaker, effective mark and volume level) of the terminal mixing system is recorded on the user interface;
  • the user interface can be an APP, PC software or webpage of the iPad;
  • the communication between the terminal mixing system and each JBM2 must be based on the TCP/IP protocol, so that a connection has been established between the terminal mixing system and each JBM2, given that the terminal mixing system and all JBM2 are connected.
  • a virtual private network ie VPN
  • the EMS file contains the following information:
  • DRM Digital Rights Management
  • Track metadata - details about the track the category and detailed model of the instrument, the name of the artist, the name of the author, the name of the songwriter, and the name of the artist.
  • the present invention provides a method for playing a terminal mix, and the method for playing the terminal mix includes the following steps:
  • each of the acoustic devices is disposed in a terminal environment corresponding to a location of the sounding body corresponding to the acoustic device in the initial environment; and providing motion tracking for communication with a plurality of acoustic devices device;
  • the plurality of microphones respectively record the sounds of the corresponding plurality of sounding bodies as the sound track;
  • the motion tracking device synchronously records the motion states of the plurality of sounding bodies as the motion state file;
  • the plurality of acoustic devices respectively synchronously move the motion states of the corresponding sounding bodies recorded by the motion state file, and respectively play the audio tracks recorded by the corresponding microphones synchronously, thereby playing the terminal mixing.
  • the step S1 further includes: providing a sound modification device that is in communication connection with some or all of the plurality of microphones, and is connected to the acoustic device corresponding to some or all of the plurality of microphones
  • the sound modification device modifies a sound quality of a track recorded by each of the plurality of microphones or a sound effect of a track recorded by a part or all of the plurality of microphones;
  • the step S2 further includes: playing the corresponding audio track modified by the sound modification device synchronously with the acoustic device corresponding to some or all of the plurality of microphones.
  • the invention records the sounds of the plurality of sounding bodies as the soundtracks by the plurality of microphones, and plays the corresponding soundtracks through the plurality of speakers corresponding to the sounding body positions, thereby playing the terminal sound mixing, and can reproduce the sounding body live playing.
  • the sound has a very high sound quality.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Stereophonic System (AREA)
  • Stereophonic Arrangements (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Electrophonic Musical Instruments (AREA)

Abstract

提供了一种终端混音系统和播放方法，该终端混音的播放方法包括以下步骤：S0）提供与初始环境中多个发声体对应的多个麦克风；还提供类型和尺寸与初始环境对应的终端环境以及多个拟声设备；提供运动追踪设备；S1）多个麦克风分别同步地将对应的多个发声体的声音录制为音轨；运动追踪设备同步地将多个发声体的运动状态记录为运动状态文件；S2）多个拟声设备分别同步地以运动状态文件所记录的各自对应的发声体的运动状态进行运动，并分别同步地播放对应麦克风录制的音轨，从而播放出终端混音。可以再现发声体现场播放的声音，具有极高的音质效果。

Description

终端混音系统和播放方法 终端混音系统和播放方法
技术领域
本发明涉及一种用于捕捉、传输、储存和重现声音的终端混音系统，本发明还涉及一种终端混音播放方法。
背景技术
现有的录制音乐会的录音无法实现现场音乐会的立体声效果，聆听录音的聆听人也无法身临其境地享受现场音乐会的感觉。同时，录制音乐会所采用的麦克风也无法将音乐会中所有发声体的声音细节完全录制下来，音乐会的录音也并不能将现场音乐会的单一或众多声音的全部细节展现出来。
发明内容
本发明针对现有录制音乐会的录音无法实现现场音乐会的立体声效果，不能将现场音乐会的声音的全部细节尤其是发声源位置及运动轨迹在多声源记录和重放过程中呈现的细节充分展现出来的问题，提供了可以克服上述不足的一种终端混音系统及终端混音的播放方法。
本发明就其技术问题提供的技术方案如下：
本发明提供了一种终端混音的播放方法，该终端混音的播放方法包括以下步骤：
S0)、提供与初始环境中多个发声体对应的多个麦克风；还提供类型和尺寸与初始环境对应的终端环境以及与所述多个麦克风一一对应的、并与对应的麦克风通讯连接的多个拟声设备；每个拟声设备设置在终端环境中与初始环境中与该拟声设备对应的发声体所处位置对应的终端位置上；提供与多个拟声设备通讯连接的运动追踪设备；
S1)、多个麦克风分别同步地将对应的多个发声体的声音录制为音轨；运动追踪设备同步地将多个发声体的运动状态记录为运动状态文件；
S2)、多个拟声设备分别同步地以运动状态文件所记录的各自对应的发声体的运动状态进行运动，并分别同步地播放对应麦克风录制的音轨，从而播放出终端混音。
本发明上述的终端混音的播放方法中，麦克风与和该麦克风对应的发声体相对设置，且所述多个麦克风与对应的发声体之间的距离都相等。
本发明上述的终端混音的播放方法中，拟声设备包括扬声器。
本发明上述的终端混音的播放方法中，部分或全部拟声设备为扬声器机器人；该扬声器机器人包括设置在该扬声器机器人底部的机器人轮子、设置在该扬声器机器人顶部的机器人手臂；在该机器人手臂的手部上设置有所述扬声器；
所述步骤S2还包括：扬声器机器人以运动状态文件所记录的对应的发声体的运动轨迹进行运动。
本发明上述的终端混音的播放方法中，所有的拟声设备均为扬声器机器人；该扬声器机器人包括设置在该扬声器机器人底部的机器人轮子、设置在该扬声器机器人顶部的机器人手臂；在该机器人手臂的手部上设置有所述扬声器；
所述步骤S0还包括提供机器人家具；机器人家具包括可运动地、用于承载阅听人的机器人座椅和可运动地、托有用于播放视频的显示屏或投影屏幕的机器人站立架；
所述步骤S2还包括：同步地移动处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人，并保持处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人之间的相对位置。
本发明上述的终端混音的播放方法中，扬声器可滑动地设置于由电机控制的导轨上；
所述步骤S2还包括：扬声器以运动状态文件所记录的对应的发声体的运动轨迹在导轨上进行运动。
本发明上述的终端混音的播放方法中，所有扬声器通过WiFi连接在一起。
本发明上述的终端混音的播放方法中，所述步骤S1还包括：提供与所述多个麦克风中的部分或全部麦克风通讯连接，并与所述多个麦克风中的部分或全部麦克风对应的拟声装置通讯连接的声音修改设备；该声音修改设备修改所述多个麦克风中的部分或全部麦克风各自录制的音轨的声音质量或给所述多个麦克风中的部分或全部麦克风各自录制的音轨增加声音效果；
所述步骤S2还包括：与所述多个麦克风中的部分或全部麦克风对应的拟声装置同步地播放该声音修改设备修改过的对应的音轨。
本发明上述的终端混音的播放方法中，所述多个麦克风所录制的音轨以EMX文件格式保存。
本发明还提供了一种终端混音系统，该终端混音系统包括与处于初始环境中的多个发声体对应的、用于同步地将对应发声体声音录制为音轨的多个麦克风，用于同步地将多个发声体的运动状态记录为运动状态文件的运动追踪设备，类型和尺寸与初始环境对应的终端环境以及与所述多个麦克风一一对应的、与对应的麦克风通讯连接、并与运动追踪设备通讯连接、以同步地以运动状态文件所记录的对应的发声体的运动状态进行运动、并同步地播放该对应的麦克风录制的音轨，从而播放出终端混音的多个拟声设备；每个拟声设备设置在终端环境中与初始环境中与该拟声设备对应的发声体所处位置对应的终端位置上。
本发明终端混音系统和播放方法通过多个麦克风分别将多个发声体的声音录制为音轨，并通过多个与发声体位置对应的扬声器播放对应的音轨，可以重新再现发声体现场播放的声音，具有极高的音质效果。
附图说明
下面将结合附图及实施例对本发明作进一步说明，附图中：
图1为本发明终端混音系统实施例中的手掌扬声器的示意图；
图2为本发明实施例的集成终端混音主要产品的示意图；
图3为本发明实施例的第一种形式的集成终端混音产品的示意图；
图4为图3所示的第一种形式的集成终端混音产品的天花支架的示意图；
图5为本发明实施例的第二种形式的集成终端混音产品的示意图；
图6为本发明实施例的第二种形式的集成终端混音产品的另一示意图；
图7为本发明实施例的第三种形式的集成终端混音产品的示意图。
具体实施方式
定义：自然声音
上帝创造万物，很多物体或生物可以发出声音，每种声音在空间中都具有独特的3D位置。听觉位置(AuditionPosition)是一种用于设置接收装置(如人类的耳朵)的逻辑3D坐标。
阅听人具有一个或多个接收装置，还具有几种神经网络结构。被接收装置捕捉到的声音信号会被传输给神经网络结构。神经网络结构通常是生物的大脑，可形成认知和记忆。
假定存在一个阅听人，附近多个发声体的声音直接传输给该阅听人的接收装置，并同时使该阅听人具有认知和记忆的过程，被定义为第一顺序混音过程(FirstOrderMixingProcess)。听觉位置、声音的反射以及其他因素会在第一顺序混音过程发生的同时给最终成音(ResultingSound)添加额外的特征的过程，被定义为第二顺序混音过程(SecondOrderMixingProcess)。接收装置前的最终成音会被捕捉，并被传输给大脑，从而创建认知和记忆。
上述认知和记忆的形成过程可以概括为：
发声体发出的声波→混音过程(第一顺序混音过程和第二顺序混音过程)→接收装置前的最终成音→阅听人大脑中所形成的认知和记忆
定义：麦克风
麦克风是一种接收装置，并被设置在听觉位置处；这样，声音信号可被麦克风捕捉，并被转换成电子信号，然后被传送给计算机。
上述声音信号被麦克风捕捉，并被传送给计算机的过程可以概括为：
发声体发出的声波→混音过程(第一顺序混音过程和第二顺序混音过程)→接收装置前的最终成音→电子信号
根据上述自然声音和麦克风的原理，本发明提供了一种终端混音系统，该终端混音系统包括与处于初始环境中的多个发声体对应的、用于同步地将对应发声体声音录制为音轨的多个麦克风，用于同步地将多个发声体的运动状态记录为运动状态文件的运动追踪设备，类型和尺寸与初始环境对应的终端环境以及与所述多个麦克风一一对应的、与对应的麦克风通讯连接、并与运动追踪设备通讯连接、以同步地以运动状态文件所记录的对应的发声体的运动状态进行运动、并同步地播放该对应的麦克风录制的音轨，从而播放出终端混音的多个拟声设备；每个拟声设备设置在终端环境中与初始环境中与该拟声设备对应的发声体所处位置对应的终端位置上。
什么是终端混音(EndpointMixing,EM)
麦克风有两个主要用途：一个是用于录制单个发声体的声音；另一个是用于录制特定环境的声音。
对于每个音轨(AudioTrack)来说，终端混音用于录制单个发声体的声音，然后将电子信号转换成数字音频，并将该数字音频传输给远程环境，以便用于重放；或者将该数字音频保存在计算机中，以便在之后重放。
多个数字音轨能在一定环境中重放；原则上为了实现高保真度的声音重放，每个音轨只在一个扬声器中重放。
然而，现实中也存在以下的一些变形情况，如：
1、使用两个或两个以上的扬声器来播放一个音轨；
2、如果录制特定环境的声音或录制一个发声体的声音是立体声，或后期录制产品创建了立体声或环绕效果时，则需要使用两个或两个以上的扬声器来播放。当存在两个扬声器(即逻辑左扬声器和逻辑右扬声器)时，立体声音频数据能被自然地映像到逻辑左扬声器和逻辑右扬声器上；而当存在多于两个扬声器，且立体声音频数据可被分为左侧音频数据(LeftSideAudioData)和右侧音频数据(RightSideAudioData)时，则需要作出预设来决定哪个扬声器用来重放左侧音频数据，哪个扬声器用来重放右侧音频数据。重放环绕音数据的扬声器的安排由环绕音技术决定。
立体声录音的运用和用于重现发声体的多于一个扬声器能在很大程度上放大发声体的声像。在EM系统中，左声道被看作一个音轨，右声道被看作另一个音轨，在音频数据的传输和储存过程中，左声道和右声道保持独立。
终端指的是用于重放音轨的环境。
在终端处，EM引入了包括使用现有扬声器技术的新特点。
首先，我们介绍一下扬声器发展的频谱的两种不同尺度。
1、尺度一：扬声器在一定程度上从高度概括变化到高度专化；
2、尺度二：扬声器采用模拟特定发声体的方式从高度概括变化到高度专化。
我们现在使用的大多数扬声器都是通用扬声器。其中，高级高保真系统(Hi-endHiFiSystem)是高度概括的，能大量级高质量的播放一个非常宽的音域。另一方面，扬声器上存在大量的扬声器单元，来覆盖不同范围的音域。
然而，声音重放设备(或扬声器)模仿特定发声体，是EM介绍的一种新方法。
模仿发声体
我们不知道岩石自身是否能够发声，但我们知道自然中的多数客体能够发出声音，如鸟、叶子、风、水、雷等。我们人类自身也是发声体，并能创造乐器，并使用乐器来发出独特的声音。
贯穿人类历史，为了便于管理，发声体被进行分类。我们对每个类别的特征进行识别以命名，如铜管乐器、萨克斯管、中音萨克斯管、女歌手惠特尼•休斯顿、鸟、夜莺等。
本申请是为了制造一个发声设备，来模仿一类特定的发声体或单一的发声体。例如，本申请建议技术开发方向朝向模拟以下发声体：
鸟、夜莺、叶子、蜜蜂、鲸、瀑布、铜管乐器、弦乐器、钢琴、小提琴、电吉他、女声等。
进一步缩小技术开发方向，可以模拟以下发声体：
柳泽990型中音萨克斯管、个人声音，如惠特尼•休斯顿等。
本申请揭示了EM能实现的全部潜能，并指明其技术发展方向。
然而，本申请的范围也确定了EM系统和扬声器的分界。
录制单个发声体的声音
在录制以前或录制期间，捕捉以下实际(或虚拟)演出台的信息：
GPS位置；海拔高度；演出台朝向的罗盘方向和角度(该演出台的朝向是真实(或虚拟)阅听人朝向的反方向)。
在为单一目标发声体进行EM录音期间，消除之前提到的第二混音过程是关键点；听觉位置、声音的反射以及其他因素会使录制的声音完全区别于目标发生客体的声音。换句话说，为单一目标发声体进行EM录音是聚焦于以高分辨率地将初始声音所有细节录制下来。
现在的工作室的录音或者在使用个体舞台麦克风或电子乐器的线性信号的直播演出期间的多音轨录音能满足上述关键点。
除了声音之外，录制过程也将以下关于在整个录音期间发声体以合理的频率与音频捕捉活动同步的信息进行数据化，该数据包括但不限于：
3D空间中相对于固定参考点的听觉位置；每个发声体的朝向。
本实施例中，麦克风与该麦克风对应的发声体相对设置，且所述多个麦克风与对应的发声体之间的距离都相等。
可以理解，麦克风与该麦克风对应的发声体并不限于相对设置，还可以是麦克风的朝向与该麦克风对应的发声体朝向构成一定角度。
定义：实时vs时移
有以下两种主要方式将录制的音频数据传送给终端：
1、实时
2、时移
对于时移来说，有一些技术都运用了时移的概念，包括使用计算机文件、存储和转送和按需播放等。本申请中，我们在使用时移时，使用了所有这些技术。
四种不同形式的终端混音
第一种形式的终端混音：用于均处于固定位置的多个同步发声体的终端混音
假定在录音时间中，所有发声体在同一时间发出声音，每个发声体在3D空间中都具有固定位置；例如，在海滨举行的音乐会或在礼堂举行的管弦乐演出中，每个音乐人都处于固定位置上。这里，终端混音的目的是建立一个能够模拟初始环境和所有与该初始环境相关的声音的终端；具体地，终端混音重点放在在终端处精确地重放所有歌手和乐器的声音。重放过程可以是实时的，也可以是时移的。
第一种形式的终端有以下特征：
1、终端是一种类型和尺寸与初始环境对应的终端环境；
2、终端中包括有用于模拟初始发声体的拟声设备；例如，终端包括高级高保真系统和高级扬声器，或终端包括高保真系统(HiFiSystem)和用于适用于一定音域范围的专业扬声器；
3、每个拟声设备设置在终端环境中与初始环境中发声体所处固定位置对应的终端位置上。
例如，在海滨举行的现场音乐会中，发声体为乐队，该乐队包括多个吉他，如低音吉他、第一电吉他、第二电吉他、木吉他等。该乐队还包括键盘乐器、鼓和歌手。
用于模拟在海滨举行的现场音乐会的终端应该具有以下特征：
1、终端环境和初始环境是同一海滨，拟声设备相对于大海的方向与乐队相对于大海的方向相同；
2、拟声设备包括吉他音箱、立体声扬声器、鼓声模拟扬声器以及歌声模拟扬声器；
3、在终端环境中，通过多个吉他音箱一一对应地模拟多个吉他；
4、由于仿真键盘乐器的声音通常混杂有杂声，所以在终端环境中，通过立体声扬声器来仿真键盘乐器；
5、在终端环境中，通过鼓声模拟扬声器来模拟鼓；
6、在终端环境中，通过歌声模拟扬声器来模拟歌声；
7、每个拟声设备设置在与终端环境(即初始环境)中发声体所处固定位置相同的终端位置上。
另一实施例中，在礼堂举行的管弦乐演出中，发声体为多个乐器；
用于模拟在礼堂举行的管弦乐演出的终端应该具有以下特征：
1、终端环境是类型和尺寸与初始环境对应的礼堂；
2、拟声设备包括多个专业扬声器(或高级高保真系统)，该多个专业扬声器(或高级高保真系统)分别一一对应地模拟多个乐器；
3、每个专业扬声器(或高级高保真系统)设置在终端环境中与初始环境中多个乐器所处固定位置对应的终端位置上。
通过该第一种形式的终端混音，演出可用于在与初始环境不同的终端环境进行同步播出，或用于在相同环境中且在实时演出之后的任意时刻重放。
第二种形式的终端混音：用于部分或全部处于运动中的同步发声体的终端混音
基于上述第一种形式的终端混音，第二种形式的终端混音在现有的扬声器上使用了机器人技术，或者将现有扬声器可滑动地安装在由电机控制的导轨上。这样，扬声器可以运动状态文件所记录的对应的发声体的运动轨迹在导轨上进行运动。
例如，拟声设备为一种扬声器机器人；该扬声器机器人包括设置在该扬声器机器人底部的机器人轮子、设置在该扬声器机器人顶部的机器人手臂；在该机器人手臂的手部上设置有扬声器。在音频播放期间，该扬声器机器人向特定的3D位置移动，并根据存储有音轨的信息来调整扬声器的朝向。
所述步骤S2还包括：扬声器机器人以运动状态文件所记录的对应的发声体的运动轨迹进行运动。
这里，运动状态文件可以是视频文件，也可以是记载发声体在初始环境中的坐标。这里，运动状态文件由与多个拟声设备通讯连接的运动追踪设备记录下来；
采用导轨上运动的扬声器是一种低成本重放录音的方式，但重放录音的效果并不令人满意。
在重放过程中，这些扬声器机器人需要进行配合，以避免发生相互碰撞。而在考虑如何避免扬声器机器人发生碰撞时，每个扬声器机器人应当减小其对录音重放的整体效果的影响。另一种思路是使扬声器机器人之间发生啮合，以使扬声器机器人发生碰撞对录音重放的效果造成的影响减弱到最小。
在扬声器机器人的另一个实际运用中，扬声器机器人可以像歌手一样在舞台上运动，或者像歌手一样向粉丝挥手。
在扬声器机器人的另一个实际运用中，因为在音乐人表演时，音乐人通常会跳舞，或者轻微地摇动身体，扬声器机器人在录音过程中会跟着摇动，而在重放录音时，扬声器机器人也会做出相同的摇动。这种扬声器机器人也被称为「舞蹈扬声器机器人」(DancingRoboticSpeaker,DRS)。
扬声器机器人可以具有任何外形，扬声器机器人的外形可以是常用扬声器造型，也可以是动物造型，还可以是通用类人机器人造型等等。任何扬声器造型的组合也可以同时运用于该扬声器机器人的外形设计上。
第三种形式的终端混音：用于不同步发声体的终端混音
假定部分或全部发声体在录音过程的不同时间进行表演，现有的音乐产品工场将音轨转换成EMX文件；该音乐产品工场还设置虚构位置信息，并将该虚构位置信息发送给终端，音频就可以在终端中重放。只有时移传输才可能发生在这种形式的终端混音中。这里，EMX是一种仅包含终端混音音频数据的文件格式。
第三种形式的终端有以下特征：
1、终端是适于音频风格的终端环境；
2、终端包括用于模拟初始发声体的拟声设备；例如，终端包括高级高保真系统和高级扬声器，或终端包括高保真系统(HiFiSystem)和用于适用于一定音域范围的专业扬声器；
3、每个拟声设备设置在终端环境中与初始环境中发声体所处固定位置对应的终端位置上。
第四种形式的终端混音：用于多个自由的发声体的终端混音
基于上述第一种形式的终端混音、第二种形式的终端混音和第三种形式的终端混音，第四种形式的终端混音要求扬声器具有以下特征：
1、扬声器在能够运动(包括移动、快速移动、飞行)；扬声器在运动中会采用了安全预防措施，以防该扬声器伤害或损害任何物体、动物、植物或任何人。当音乐响起时，扬声器能跟着节拍进行舞蹈。只要扬声器的运动是安全的，扬声器在听觉范围内运动的速度没有限制，声波在空气中传播的时间延迟速度也会被补偿。
2、扬声器在预定物理边界内运动，如果作为扬声器使用的扬声器机器人是终端混音系统的一部分时，扬声器机器人总是会返回其运动的初始位置。这里，终端的物理边界的范围没有被限制。
3、重新配置终端混音系统，使一个扬声器中的音轨在另一个扬声器中进行重放。
4、每个音轨的音量大小可调，从0到最大音量。
5、采用终端混音系统或在线互联网服务来修改声音质量或增加声音效果，如在每个音轨的基础上进行混响和延迟。
6、扬声器的音轨配置、扬声器位置、扬声器朝向角度、扬声器的运动、扬声器跟着音乐节奏舞蹈、扬声器的音量以及扬声器的声音修改由下述因素决定：
a)物理限制--终端的类型、尺寸和空间；每个扬声器的类型和质量；
b)初始音乐的创建者的思考；
c)音乐风格和意境；
d)终端混音全球服务中心的推荐；
e)终端混音爱好者社交网络的推荐；
f)阅听人所处位置、朝向、心境、身体内部条件；
g)阅听人为立体声音轨和环绕音轨创建声像的欲望；
h)终端混音重放系统中软件的预定程序主题；
i)阅听人的深思或情绪化的决定。
7 、和其他终端混音系统同步重放--该终端混音系统和其他终端混音系统同步重放是基于同时服务器或通过计算机网络连接的终端混音系统之间的信息传输而实施的 。
关于终端混音的进一步讨论
智能音量控制
通过采用扬声器的嵌入式Linux计算机传感器，终端混音系统能计算终端中的音量大小，当该音量太大时，终端混音系统能发出视觉警告，并以均衡方式自动地将所有扬声器的音量调低到安全音量水平。
阅听人位置
终端混音使用的场所没有限制，阅听终端混音重放的人数也没有限制；但只要人数不是太多，就存在有指南，以使每个阅听人都能很好地听终端混音；阅听人不会使其身体或其他物体阻挡其他越听人收听终端混音。
当两个或两个以上的音频在一终端混音系统中为不同阅听人同时重放时，分别播放该两个或两个以上的音频的扬声器会彼此分开。
当前技术(如环绕音系统)会要求阅听人处于特定的区域内；高级高保真系统更是要求阅听人处于特定的位置(即国王的座椅，KingSeat)上；不像这些技术，终端混音系统允许阅听人处于扬声器区域内部或外部的任何位置。当拟声设备为扬声器机器人时，扬声器机器人自身可以做调试，使阅听人听到最佳的声音，或者使该扬声器机器人具有宽的阅听角度，这样，阅听人可以坐着、站着或在扬声器之间行走。阅听人也能将耳朵靠近扬声器，从而听到音量大且更清晰的音轨，例如，可以听到歌声或小提琴音轨的详细细节。阅听人也能处在距离扬声器很远的位置，并听到高质量的声音。扬声器的设计迎合阅听人位置，使扬声器具有宽的阅听角度，扬声器的阅听角度可以是360°或球形的。
本申请并没有对听觉区域(即听觉位置的区域)应当如何建立做出限制，但是本申请列举了一个例子，在礼堂中，听觉区域是礼堂的公共区域或卧室，所有的阅听人都处于听觉区域中部，每个扬声器的阅听角度都是360°。在这种设置下，当扬声器播放录制的终端混音时，一个人在听觉区域中不同位置所听到的声音是不同的，这和阅听终端混音的经历和阅听人路过海滨或繁忙的商业中心的经历是相似的。进一步地，当管弦乐团演奏古典音乐的时候，终端混音也能允许阅听人穿过管弦乐团；或者终端混音也能允许阅听人将其耳朵靠近歌声模拟扬声器，从而使阅听人能试着去听歌手发声的全部细节。
然而，上述设置须假定阅听人都是处于能够阅听到最佳阅听效果的阅听人朝向上。而阅听人也能够通过专业设备听到最强音质。
编辑
EMX文件格式的最初版本和MIDI文件格式相似。EMX文件格式和MIDI文件格式的主要区别在于：EMX文件格式的设计目的具有宽广的范围，不仅迎合音乐创作者录音、编辑、阅听的需要和阅听人阅听的需要，还使阅听人也能具有录音和编辑的能力。EMX文件格式和MIDI文件格式另一个主要区别在于：EMX文件格式允许任何人修改一音轨，而与此同时其他音轨保持不变。
任何人能够采用EMX文件或EMVS文件来修改任何音轨，并将修改后的音轨结果保存为另外的EMX文件或EMVS文件，或将该修改后的音轨结果以如WAV或MP3的现有文件格式保存。EMVS是一种包含终端混音音频数据和视频数据的文件格式。该修改后的音轨结果可以是只读文件或可擦写文件。通过这种保存设计，任何人能够容易地添加、删除和修改EMX文件的音轨。因此，终端混音通过将音频编辑功能赋予给一般大众的方式开辟了一个音乐制作的新纪元。理论上，一个EMX文件中具有的音轨数目是没有限制的。然而，非常大的EMX文件仅能在设置于终端中的非常大的终端混音系统中进行重放，或者该非常大的EMX文件还可以利用运行于终端的云服务器来进行重放。
初始音乐创作者能利用终端混音工具、EMX文件格式、终端混音系统的版权保护特点来保护创作的部分或所有的音乐数据，使这些音乐数据在其发布后不能被修改。
并且，终端混音使音乐制作过程能够利用互联网的社交网络和虚拟团队工作特点，使具有不同天赋的音乐人能在一起工作，并以国际视角创作一个EMX文件。
根据EMX文件格式的特点，本实施例中，终端混音系统还包括与所述多个麦克风中的部分或全部麦克风通讯连接，用于修改所述多个麦克风中的部分或全部麦克风各自录制的音轨的声音质量或增加所述多个麦克风中的部分或全部麦克风各自录制的音轨的声音效果的声音修改设备；与所述多个麦克风中的部分或全部麦克风对应的拟声装置与该声音修改设备通讯连接，用于同步地播放该声音修改设备修改过的对应的音轨。
和现有的环绕音技术的比较
基于终端混音，在终端混音系统中，只要扬声器的位置设置符合环绕音扬声器位置要求，任何种类的扬声器能够用作环绕音扬声器，来播放环绕音(包括5.1环绕音，6.1环绕音和7.1环绕音)。然而，这里所使用的扬声器推荐选用通用扬声器，专用扬声器并不适合播放环绕音，而只能读运动数据的扬声器机器人也不能被使用。
终端混音系统具有预定义的环绕音重放模式，该环绕音重放模式用来根据环绕音技术的类型来制作每个扬声器上的声音。终端混音利用现有的环绕音技术来对环绕音音频数据进行译码和重放。
所有扬声器优选地通过WiFi连接在一起。
一种终端混音系统是利用了简单的扬声器机器人，通过按下一个按钮，如按下「建立处于5.1环绕音模式的扬声器」按钮，扬声器将会基于优选环绕音位置和实际终端结构而自动地进行物理移动。当所有扬声器的使用过程结束后，扬声器会返回初始位置。这里，一种具有机器人轮子和垂直轨道，并与终端混音系统WiFi连接，还内置软机器人音乐人软件的扬声器机器人-扬声器机器人模型A是一种适用于环绕音用途的扬声器机器人。然而，本申请并不将这种扬声器机器人模型A的用途局限于环绕音用途。
终端混音和MIDI的关系
MIDI被内置于EMX文件中，例如，音乐制作人或阅听人能将一种通用MIDI乐器映射到一种专业扬声器上。这种逻辑决定是根据乐器的使用效果来将乐器映射到扬声器上而做出的。将乐器映像到专业扬声器上是较合适的映像方式，例如，将MIDI三角大钢琴(#1)映射到一个自动钢琴上是最合适的。
EMX文件中有关使用运动数据的音轨的数据采用现有MIDI文件格式，而没有采用标准数字音频数据格式。换句话说，初始音频数据不能够在特定的声道中传输，但在输入设备的操作能够被捕捉，并以MIDI文件格式保存。
终端混音的重放可以通过以下两种途径实现：一种是通过利用终端混音系统的MIDI渲染模块将MIDI数据转换成音频数据，并利用通用扬声器来播放该音频数据；另一种是向扬声器机器人提供MIDI数据流，以使该扬声器机器人直接重放。自动钢琴的使用是一个很好地阐明扬声器机器人如何接收终端混音系统的MIDI运动数据，以及扬声器机器人如何将该MIDI运动数据转换成终端中播放的声音的例子。
另外，现有的MIDI乐器能够支持EMX文件格式，这样，终端用户能够利用该MIDI乐器来制作和阅听音乐。
广域媒体(WideAreaMedia,WAM)重放
广域媒体重放的主要目的是有选择地用于生动地重放终端混音的子设备。
下面描述了一个广域音频(WideAreaAudio,WAA)重放的主要形式：通过选择终端混音系统中的部分或所有扬声器，使用者能通过以下方式在这些扬声器上进行重放音频：
1、所有扬声器播放相同的音轨，即单声道。
2、只有在阅听人附近的扬声器播放声音，而所有这些播放声音的扬声器播放相同的音轨，或者这些播放声音的扬声器分别播放与该阅听人朝向相关的不同音轨。通过这种方式，终端混音系统能够在这些扬声器上播放EMX文件或现有立体声。同时，阅听人能够使用终端混音控制工具来播放EMX文件，使该EMX文件的每个音轨能在一个或多个扬声器上进行重放。
WAV文件以相似的方式进行播放。
音频和视频广播
终端混音广播是一种音频和视频广播的形式：
1、终端混音广播的范围覆盖地球以及其他适当的行星，如火星。
2、同一终端混音系统的任意两个扬声器之间的最大传输滞后时间为60s，其中，传输滞后时间是电子信号在录音设备上产生的时间和扬声器发出声波的时间之差。
3、安全广播：数据在终端中的录音设备和所有扬声器之间传输的过程中，数据修改是严格禁止的。只有一个例外，就是基于阅听人意愿的修改。例如，阅听人决定在广播馈送上采用由云服务器提供的修改后的租来的声音。安全广播要求就会被公用密钥加密模块数字化的标记出来。
本申请覆盖了广播的基本要素，然而本申请并不限于这里提到的广播特征；一个与广播相关的区域将会强化现有广播技术来提供终端混音音频，例如有线电视网络。
基于音频数据是连续地打入到终端混音数据主体中的设计，EMX文件是满足数据流的使用方式。因此，终端混音系统能一边下载终端混音数据主体，一边重放声音。这和大多数现有的互联网视频数据流技术相似，终端混音数据流的带宽要低于视频数据流的带宽，因此，这种具有EMX文件的音频数据流的播放可通过现有技术实现。
适于视频广播的EMVS文件的数据流的播放方式是和EMX文件的数据流播放方式相同的。
音频和视频广播能通过EMX文件/EMVS文件来替代视频文件的方式利用视频服务器来实现，并将客户软件模块添加入终端混音系统中，这样，该客户软件模块可接收终端混音数据主体，并将该终端混音数据主体译码、渲染、音轨分配并在扬声器上进行音频重放。
常规扬声器、扬声器机器人或通用机器人的视觉效果和实体
所有的扬声器都能与终端混音系统连接。
然而，本申请介绍的扬声器机器人具有更多的特点，但这些特点必须遵守下面的规则：
1、具有扬声器机器人能制作成任何形式。
2、为了避免扬声器机器人损坏、滥用或误用，在室外使用时和当扬声器机器人处于黑暗环境中时，扬声器机器人必须发出明显的视觉信号来标识该扬声器机器人的存在，例如，该扬声器机器人展示出标语「音频重放正在进行中」或者「第四种形式的终端混音」，以此来将该扬声器机器人的存在和位置告知周围的人们，并让这些人知道在什么地方和为什么能听到声音。在该扬声器机器人开始展示标语时，该标语要足够清晰，之后，该标语可以保持与该扬声器机器人开始展示标语时相同的亮度，或者，该标语可以暗淡一点，但每隔至少10min标语的亮度会改变成初始时的亮度。
机器人家具
终端混音系统还包括机器人家具。机器人座椅(ROBOCHAIR)是一种具有高容量电池，并在每个腿部上设置有机器人轮子的座椅；高容量电池用于为机器人座椅的运动提供电能；该机器人座椅和扬声器机器人相似；一个或多个阅听人可以坐在该机器人座椅上，该机器人座椅能根据终端混音系统的命令进行移动。
相似地，机器人站立架(ROBOSTAND)也是一种适于机器人通用目的的站立框架，该机器人站立架主要用于托住用于播放视频的显示屏(如55英吋LED电视显示屏)或投影屏幕。
终端混音系统将机器人座椅看作为中心，通过机器人座椅、机器人站立架、终端环境以及扬声器扬声器之间的相对位置来确定发送给机器人座椅、机器人站立架以及扬声器机器人的命令和控制信号。
具体地，本实施例中，机器人座椅、机器人站立架、终端环境以及扬声器扬声器之间的相对位置仅需要确定下列三项：
a)机器人座椅和终端环境之间的3D相对位置；
b)机器人座椅和机器人站立架之间的3D相对位置；
c)机器人座椅和扬声器机器人之间的3D相对位置。
通过同步地移动处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人，并通过计算保持处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人之间的相对位置可以创建一种虚拟的「房子移动效果」。该房子移动效果取决运动中的处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人，地板类型，风以及机械精度等因素的稳定化；这些因素相互配合，使房子移动效果增加到最大的限度。
相同的方法也被采用于户外，例如，当终端混音系统缓慢地穿过森林时，用户可以体验到「森林移动」的效果。
在另一实施例中，处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人可以自由移动；该自由移动必须遵循一基本原则：机器人站立架没有被使用，而用户想要获得「房子(或终端环境)移动效果」；机器人座椅和扬声器机器人必须遵守同一终端混音的扬声器定位和听觉规则。
在又一实施例中，通过采用阅听人运动阅听技术(WalkingAudienceListeningTechnique)来移动在固定设置的扬声器机器人之间的机器人座椅，或来保持阅听人和扬声器机器人之间相对移动关系。
相似地，机器人移动方式和远程控制能力能以相似的方式扩展到其他家具上；这些家具包括但不限于：
桌子；灯等。
可穿戴式终端混音产品
手掌扬声器(PalmSpeaker)
扬声器可设置在衣服上，这种设置方式具有很多工艺和时尚的设计。
手掌扬声器就是一种可穿戴式终端混音产品，该手掌扬声器包括设置在手套掌部的扁平圆形蓝牙扬声器，如图1所示。而与此同时，用户的智能手机上运行有JBM2的软件版本。JBM2是一种设置在扬声器中，具有计算能力和输入输出硬件的设备，例如RJ45局域网端口，音频输出的DAC模块。
每个手套内部具有圆形的LED和陀螺仪，该陀螺仪用于探测手部是举起的还是放下的，或指示手掌的朝向。
如果用户具有蓝牙耳机时，JBM2的音频输出结果会混在用户的声音里，该用户的声音会在手掌扬声器中播放。
集成终端混音(IntegratedEM,IEM)产品
集成终端混音主要产品(IEMMainProduct)
该集成终端混音主要产品的目的在于能实现本申请的终端混音的所有功能。
下面介绍了一种推荐产品，但本申请的产品并不限于下述的产品；所有根据本申请的思想而进行的改进或变换都应属于本申请的保护范围。
该集成终端混音主要产品是一种电子产品，内置有具有CPU、内存以及存储器，用于控制终端混音的硬件系统；该硬件系统装载有Linux系统，并装载有终端混音软件来控制终端混音。该集成终端混音主要产品还具有WiFi通讯模块，用于与局域网(LAN)WiFi通讯连接。该集成终端混音主要产品内部还设置有隔间，该隔间中设置有安装在轨道上的至少四个扬声器。
该集成终端混音主要产品具有以下主要特点：
能够播放终端混音音频；
扬声器之间的位置会根据所播放的终端混音音频种类而发生改变。
参照图2，该集成终端混音主要产品看起来像保护围栏，以避免在扬声器运动过程中，尤其是当终端混音音频重放时，扬声器快速运动时，人或动物受伤的情况。
第一种形式的集成终端混音产品
基于集成终端混音主要产品，第一种形式的集成终端混音产品具有以下附加特征：
1)图3示出了第一种形式的集成终端混音产品。该第一种形式的集成终端混音产品10包括天花支架1和机器人。天花支架1固定安装于天花板上，该第一种形式的集成终端混音产品10除天花支架1的其他部分就是机器人。该机器人可拆卸地设置于天花支架1上。
2)当天花支架1被安装时，天花支架1能被伸长，从而调整机器人高度。该机器人高度(即从地板到该机器人的高度)能被自动地调整，该机器人高度处于1m和天花板高度之间。因此，阅听人能调试该机器人高度，来阅听与该阅听人水平的角度的声音。
3)当机器人从天花支架1上拆卸下来时，机器人拆下其底盖，并将该机器人底部的机器人轮子2显示出来，该机器人能在室内或室外使用。用户能通过其移动电话上运行的远程控制软件来命令机器人播放音频，或控制机器人运动，或使该机器人自由运动，或使该机器人一直遵守阅听人的命令。视觉信号能被传输给用户的移动电话，并在该移动电话上进行播放。
4)多个电灯泡3环绕地设置在机器人上；该多个电灯泡3可以通过普通的墙上开关或者移动电话(该移动电话上运行的软件)来进行常规照明控制。在音频重放过程中，用户也能为娱乐目的，使不同颜色的该多个电灯泡3进行闪烁。
5)天花支架1被拆下来时，该天花支架1如图4所示。该天花支架1能像常规电灯一样工作，由常规墙上灯或者移动电话(该移动电话上运行的软件)进行控制。
第二种形式的集成终端混音产品
基于第一种形式的集成终端混音产品，该第二种形式的集成终端混音产品具有以下附加技术特征：
1)机器人手臂上的一个或多个透明显示屏4被安装在天花支架上，如图5所示。
2)能根据碰撞检波的结果，自动调低或调高一个或多个显示屏4；当显示屏4在使用时，则该显示屏4被调高，如图6所示。有声警报器和LED设置在一个或多个显示屏4上。
3)显示屏4与JBOX-VIDEO输出连通，JBOX-VIDEO仅是运行在具有该显示屏4的计算机中的软件。
4)常规显示屏能替代该透明显示屏4来使用。
第三种形式的集成终端混音产品
基于集成终端混音主要产品，第三种形式的集成终端混音产品具有以下附加技术特征：
1)该第三种形式的集成终端混音产品为扬声器机器人，该扬声器机器人具有机器人轮子或其他能使该机器人运动的部件；
2)该第三种形式的集成终端混音产品具有可爱的外表，如图7所示，该第三种形式的集成终端混音产品的外表为章鱼；
3)扬声器都安装在机器人手臂的端部；
4)具有第一种形式的集成终端混音产品和第二种形式的集成终端混音产品中部分或全部特征。
为了使该第三种形式的集成终端混音产品具有一些视觉效果，可以采用以下手段：
1)第三种形式的集成终端混音产品上安装有电灯泡、LED或激光灯；
2)根据该第三种形式的集成终端混音产品的形状，将LED遍布地安装在该第三种形式的集成终端混音产品上；
3)在该第三种形式的集成终端混音产品上安装平板LED显示屏；
4)能使用该第三种形式的集成终端混音产品附近的JBOX-VIDEO产品，来控制平板LED显示屏；
5)能使用该第三种形式的集成终端混音产品附近的移动设备，来控制该第三种形式的集成终端混音产品上的电灯泡、LED或激光灯和/或平板LED显示屏。
终端混音音乐的新世界--新的终端环境、新的乐器以及新的音乐表现方式
这可能是人们在人类历史的第一次以一种新的使用终端混音的方式来创造终端混音音乐。人们可以创造新的、革新的、突破性的以及精心制作的新世界，该新世界包括：
1)新的终端环境--该终端环境跨越范围巨大的地理区域，例如，在50000平方米的花园内使用100000个扬声器，每个扬声器播放一个音轨；
2)新的乐器--通过发声体和终端混音技术为人们创建了一个新的艺术体验。例如，5000根玻璃柱子；每根玻璃柱子10米高，并填充有水，每根柱子的顶端设置有扬声器；所有的扬声器都通讯连接于一个终端混音系统中；每个柱子负责发出竖琴的一个独特的弦的声音。该终端环境用来重放EMX/EMVS文件的MIDI音轨，或与电子竖琴连接；当音乐人弹奏竖琴的时候，新的终端环境将会同步地发出声音。这里，电子竖琴是一种常规竖琴，该电子竖琴的每根弦都与麦克风连接。
3)新的音乐表现方式--将所有可能的和被认可的发声体有选择地用于终端环境中。例如，在一场音乐会上，阅听人携带着他们的可穿戴式终端混音设备(WEM)，在该音乐会的舞台上设置有常规扬声器；每个常规扬声器都具有飞行机器人，用于使常规扬声器起飞；在音乐会的四边还遍布有扬声器机器人；其中有些扬声器机器人围绕阅听人运动。在音乐会过程中，音乐人歌唱和演奏音乐，音乐人和阅听人进行互动，将乐器给阅听人，并让阅听人举起他们的手，并让其可穿戴式终端混音设备成为终端混音系统的一部分，并成为音乐会乐器的一部分，阅听人可以通过可穿戴式终端混音设备进行唱歌。总之，音乐人可以自由的利用所有的资源来推进音乐会，并让阅听人以终端混音方式参与到音乐会中。
技术细节
终端混音系统的主要功能
1)罗列所有的扬声器；
2)采集每个扬声器的注册信息，并导入到实时数据库中；
3)扬声器进行同步发声；
4)实现所有的JBM2设备的播放、停止、其他命令和控制；
5)提供如下信息，以响应来自身份认证过的客户端的查询信息：
a)所有扬声器的总清单，以及每个扬声器的任务；
b)单一扬声器的类型、音域范围、终端位置、状态以及其他信息。
将扬声器的声音同步-算法
为了弱化不同音轨的音频差异，任意两个不同扬声器播放一个单节不同音轨的时差要小于10-100毫秒。
解决上述问题的方法有多种，包括基于消息传递、轮询等同步方法。但这些方法使任意两个不同扬声器播放一个单节不同音轨的时差处于100-500毫秒之间。
本申请提供了一种解决上述问题的优选方法，该方法是使嵌入式Linux设备的每个扬声器每天至少一次与同一互联网时间服务器进行同步，所有的同步活动(如重放过程开头的同步)应当基于两个因素，一个是来自终端混音系统的命令，该命令包含有处于未来时间的目标运行时间戳；另一个是嵌入式Linux时钟时间，该嵌入式Linux时钟时间的格式为操作系统纪元时间。
假设用户之间互联网交流延迟时，本申请的这种方法将任意两个不同扬声器播放一个单节不同音轨的时差降低到50毫秒以下。嵌入式Linux设备和时间服务器之间具有非常小的周转期，在2014年世界上所有的互联网终端上，这个假设都是真实的。同时，在未来，路由器技术的强化和光缆取代电缆的推进将进一步地减小该周转期，从而完全消除音轨时差不同的问题。在终端混音系统中设置微型原子钟是未来的解决方法。
为了控制JBM2设备，采用以下步骤：
在终端混音系统中：
如果用户按下播放按钮，则可以得到：『播放时间』为2017-03-17_10：23：59.001(操作系统纪元时间，精度为1毫秒)；
然后将「在『播放时间』开始播放」的信息发送给该终端混音系统的所有扬声器；
在JBM2设备上：
基于接收到的「在『播放时间』开始播放」的信息后，获取该信息中的时间，并查看该JBM2设备上的本地时间，并在该本地时间到达『播放时间』时作出行动。
注意：
开始播放一个列表需要进行一个过程，如使用选择(Fork)过程；
互联网交流遵守TCP/IP协议，这样，我们可以获得高质量信息传输保障。
将扬声器的声音同步-操作系统(OS)和多任务的考虑
大部分现代计算器操作系统都是多任务系统，由于各种原因，目前扬声器的运行程序与其他程序相互独立，这样，各扬声器播放声音的开始时间就不确定。
任意两个扬声器进行同一终端混音音频重放的时差不多于20毫秒。但任意两个扬声器的同步时间(SyncTimePeriod)不得超过10s。
为了满足上述要求，本申请通过以下两个方法来解决：
方法1：使用具有相同资源、配置和运行程序的相同规格的硬件和操作系统；
方法2：采用「锁-报告-取消锁-闹钟软件-处理」(Lock-Report-Calloff-Atomic-Transaction)算法
评估：
1)同时购买有两个或两个以上的相同硬件的顾客可以采用方法1；
2)采用混合硬件(MixedHandware，例如iPhone和计算机的组合)的顾客会陷入同步问题之中。同样的同步问题也出现在下面的终端中：该终端中的不同客体试图播放相同的音乐；该不同客体包括冰箱、茶杯和移动电话。方法2能在这被采用；
3)将一个新硬件添加到一个旧有硬件中的顾客也会遭遇同步问题，因为尽管旧有硬件可能会相互进行识别，但新硬件可能会更加先进，因此，新硬件和旧有硬件在硬件规格和软件规格上就存在差异。方法2能在这被采用。
4)一体化系统不存在同步问题。
'锁-报告-取消锁'('Lock-Report-Calloff')处理过程-算法
对于负责同一EMX文件重放任务的JBM2设备来说，「锁-报告-取消锁」处理过程包括以下步骤：
1)将音量调整到0%；
2)将音频处理模块限定于唯一的用途；
3)为目标重放时间实时检测本地时钟；当到达目标重放时间时，将音频数据块导入到音频硬件中；
4)通过将音频数据块的实际重放时间发送给终端混音系统的方式，向终端混音系统确定并报告音频数据块的实际重放时间；
5)等待终端混音系统的结果响应；
6)如果该结果响应是「取消锁；在音频处理模块的限定开始时间上，重新限定音频处理模块」时，则停止重放，并返回步骤2；
7)在7s内直线地将音量调整到100%。
在终端混音系统中：
1)等待并收集扬声器组中每一个扬声器的所有报告；
2)将该所有报告进行比较，以查明扬声器组是否满足时差要求；
3)将步骤2的信息发送给扬声器组中所有设备，如果扬声器不满足要求，扬声器会发出「取消锁；在音频处理模块的限定开始时间上，重新限定音频处理模块」，否则扬声器会发出「成功」；
4)如果扬声器不满足要求，会回到步骤1。
算法的评估
1)在一个小型系统中，小于50单位的JBM2、基本硬件、网络和软件资源是充足的；
2)在一个大型系统中，100000单位的JBM2、网络和终端混音系统资源必须是：
a)充足的网络资源；
b)具有较低的响应时延的网络，这样，可以避免太长的「阅听人等待时间」；
c)终端混音系统中充足的用于同步地发送和接收大量的通信信息的处理资源，例如该处理资源有100000单位。
多个RTMP(实时消息传送协议)数据流的广播
基于Adobe公司的RTMP协议，终端混音广播站提供了具有RTMP协议的终端混音音频，一个RTMP数据流对应地在一个音轨上播放。
本地终端混音系统采用流媒体译码音频数据，并以一种同步方式将所有的扬声器的重放过程同步。
站长列表文件格式(StationMasterListFileFormat)是M3U文件格式。
终端混音系统会在预配置中心服务器上下载 M3U 站列表；一个选择界面提供给用户，以便于选择 M3U 站。之后，终端混音系统与 M3U站连接，并通过使用
RTMP协议开始同步地下载所有音轨的内容。然后，在终端混音系统的扬声器上进行译码、同步和重放。
一种扬声器机器人的细节设计--具有机器人轮子和垂直轨道，并通过WiFi与终端混音系统连接，并内置有软机器人音乐人软件的通用扬声器，即扬声器机器人A
基于通用扬声器，该扬声器机器人还包括：
1)基体：
a)该基体包括高容量电池，该高容量电池能通过其插接站(DockingStation)或接上电源进行反复充电；
b)该基体中内置有JBM2，该JBM2由高容量电池提供电源，该JBM2还通过WiFi与终端混音系统连接；
c)机器人轮子设置在该基体的底部，该机器人轮子有高容量电池提供电源，该机器人轮子的控制信号线设置在JBM2的背面；
d)该基体还包括设置在该基体底部的用于识别轨道颜色的光传感器；
e)该基体还包括设置在该基体内部的扬声器，该扬声器与JBM2通过音频信号连接，单声道扬声器线与扬声器连接；
f)基体还包括用于探测该基体四边上的阻挡物体的传感器。
2)基体上设置有垂直机器人手臂，该机器人手臂顶部设置有扬声器，伺服机构设置在JBM2后部。该垂直机器人手臂可以是具有一移动平台的由两部分构成的机器人手臂，也可以是简单的垂直轨道。
3)内置于JBM2的附加软件模块用来识别该扬声器机器人底部的轨道信号；并根据来自EMX文件的译码位置和直接信息来确定该扬声器机器人的哪个部分进行移动，以及扬声器的垂直高度。将EMX文件信息与机器人姿态相映射，来模仿初始发声体的位置和方向。
4)软件模块也会不时执行避碰。
相关配件
1)插接站，机器人在完成被使用后，该机器人能放回到该插接站中；该插接站作为机器人的初始位置。该插接站用作电池充电器，能自动地对机器人的高容量电池进行充电直到充满为止。
软机器人音乐人软件设计
软机器人音乐人软件具有以下特征：
1)所有音轨必须在相同拍子下进行录制；
2)至少一个具有音乐拍号(例如，4/4拍的歌曲)的参考MIDI音轨是可用的；
3)参考音高--精确的音高调谐数据是可用于软机器人音乐人软件的调谐；
4)在EMX文件中设置键和和弦。
当上述所有条件都具备时，用户能为每个JMB2，有选择地初始化一运行于内置Linux系统的虚拟机中的软机器人。
用户能初始化对应一个发声体的一个或多个的软机器人，并将该一个或多个软机器人发送给扬声器，但为了实现最大的运动弹性，一个扬声器上只会分配有一个软机器人。用户能根据具有不同参数的相同软机器人，来初始化或有选择地使用另一个软机器人。例如，将芬达吉他(即Fender-Stratocaster)发声体的两个软机器人分别分配给两个扬声器；该两个扬声器中的一个用于播放和弦，另一个用于播放独奏。将大三和弦的独鸟发声体的一个附加软机器人分配给其中一个扬声器。
每个发声体将参考音高、拍号、拍子、键和现有和弦加入到与其对应的人工智能(AI)模块，并决定为适于现有和弦而发出什么声音。发声体可以发出现有和弦的可用标记的打击乐器的敲击声、鸟声或情态表达，以及前一播放、下一播放、参考打击节奏和使用人工智能的多种因素。
娱乐
观看扬声器机器人的运动并不能令阅听人愉悦，但将光学设备和LCD显示器添加到每个扬声器机器人上，就可以使扬声器的运动更加具有娱乐性。例如，简单音量级别的LED条，或简单级别激光枪秀能够被添加到运动的扬声器机器人上。
机器人家具的细节设计
当机器人座椅具有与扬声器机器人A相同特征(具有机器人轮子、垂直轨道，并通过WiFi与终端混音系统连接的，还内置有软机器人音乐人软件的通用扬声器)时，该机器人座椅被使用来替代普通扬声器。该机器人座椅的定位可简单地通过轨道进行，也可通过后墙上一定高度的参考点进行。从安全因素的考虑，机器人座椅上并没有设置机器人手臂来以此将该机器人座椅抬起。机器人座椅上设置有两个扬声器，而非一个扬声器；两个扬声器中的一个设置在该机器人座椅的左边，另一个设置在该机器人座椅的右边；当阅听人坐在该机器人座椅上时，两个扬声器分别正对该阅听人的两个耳朵。
该机器人座椅具有一个、两个或多个座位；该机器人座椅能有不同的设计、材质和类型。该机器人座椅还能具有按摩功能。然而，所有因素必须与有运动部件、电池容量和电池使用时间决定的伺服力矩、噪声水平保持平衡。
机器人站立架是一种适于通用目的的站立框架，用于托住LED电视显示屏；机器人站立架和机器人座椅之间的不同在于：机器人座椅可被机器人站立架替代，并在平滑移动时，能够牢固和安全地托住有效荷载。
宽域媒体(WideAreaMedia，WAM)重放--算法
1、对局域网(LAN)中终端混音系统的所有扬声器进行登记，每个扬声器通过俯视角度投影到地板平面上，每个扬声器被标记出来；
2、终端混音系统的每一个扬声器(扬声器、有效标记和音量水平)被记录于用户界面上；用户界面可为iPad的APP、PC软件或网页；
3、在终端混音时，按要求提供所需的扬声器；
4、休眠2s；
5、回到步骤2。
注意：终端混音系统和每个JBM2之间的通信必须基于TCP/IP协议，这样，假设终端混音系统和每个JBM2之间都已建立了联系，鉴于终端混音系统和所有的JBM2都处于相同的局域网中，或被分隔于互联网之外，为建立终端混音系统和每个JBM2之间的联系，虚拟专网(即VPN)需要被建立，来符合TCP/IP协议。
EMX文件结构
EMS文件包含下列信息：
文件类别；
版本号；
数字权限管理(DigitalRightManagement，DRM)信息，所有者，版权信息；
音频数据；
定位信息；
软机器人音乐人专用信息；
音轨元数据--关于音轨的细节信息：乐器的类别和详细型号、音乐人的名字、词作者名字、曲作者名字以及歌手名字等。
音轨之间的立体声耦合关系。
根据如上所述的内容，本发明提供了一种终端混音的播放方法，该终端混音的播放方法包括以下步骤：
S0)、提供与初始环境中多个发声体对应的多个麦克风；还提供类型和尺寸与初始环境对应的终端环境以及与所述多个麦克风一一对应的、并与对应的麦克风通讯连接的多个拟声设备；每个拟声设备设置在终端环境中与初始环境中与该拟声设备对应的发声体所处位置对应的终端位置上；提供与多个拟声设备通讯连接的运动追踪设备；
S1)、多个麦克风分别同步地将对应的多个发声体的声音录制为音轨；运动追踪设备同步地将多个发声体的运动状态记录为运动状态文件；
S2)、多个拟声设备分别同步地以运动状态文件所记录的各自对应的发声体的运动状态进行运动，并分别同步地播放对应麦克风录制的音轨，从而播放出终端混音。
进一步地，所述步骤S1还包括：提供与所述多个麦克风中的部分或全部麦克风通讯连接，并与所述多个麦克风中的部分或全部麦克风对应的拟声装置通讯连接的声音修改设备；该声音修改设备修改所述多个麦克风中的部分或全部麦克风各自录制的音轨的声音质量或给所述多个麦克风中的部分或全部麦克风各自录制的音轨增加声音效果；
所述步骤S2还包括：与所述多个麦克风中的部分或全部麦克风对应的拟声装置同步地播放该声音修改设备修改过的对应的音轨。
本发明通过多个麦克风分别将多个发声体的声音录制为音轨，并通过多个与发声体位置对应的扬声器播放对应的音轨，从而播放出终端混音，可以重新再现发声体现场播放的声音，具有极高的音质效果。
应当理解的是，对本领域普通技术人员来说，可以根据上述说明加以改进或变换，而所有这些改进和变换都应属于本发明所附权利要求的保护范围。

Claims (10)

  1. 一种终端混音的播放方法，其特征在于，该终端混音的播放方法包括以下步骤：
    S0)、提供与初始环境中多个发声体对应的多个麦克风；还提供类型和尺寸与初始环境对应的终端环境以及与所述多个麦克风一一对应的、并与对应的麦克风通讯连接的多个拟声设备；每个拟声设备设置在终端环境中与初始环境中与该拟声设备对应的发声体所处位置对应的终端位置上；提供与多个拟声设备通讯连接的运动追踪设备；
    S1)、多个麦克风分别同步地将对应的多个发声体的声音录制为音轨；运动追踪设备同步地将多个发声体的运动状态记录为运动状态文件；
    S2)、多个拟声设备分别同步地以运动状态文件所记录的各自对应的发声体的运动状态进行运动，并分别同步地播放对应麦克风录制的音轨，从而播放出终端混音。
  2. 根据权利要求1所述的终端混音的播放方法，其特征在于，麦克风与和该麦克风对应的发声体相对设置，且所述多个麦克风与对应的发声体之间的距离都相等。
  3. 根据权利要求 2 所述的终端混音的播放方法，其特征在于，拟声设备包括扬声器。
  4. 根据权利要求3所述的终端混音的播放方法，其特征在于，拟声设备包括扬声器机器人；该扬声器机器人包括设置在该扬声器机器人底部的机器人轮子、设置在该扬声器机器人顶部的机器人手臂；在该机器人手臂的手部上设置有扬声器；
    所述步骤S2还包括：扬声器机器人按照运动状态文件所记录的对应的发声体的运动轨迹进行运动。
  5. 根据权利要求3所述的终端混音的播放方法，其特征在于，拟声设备包括扬声器机器人；该扬声器机器人包括设置在该扬声器机器人底部的机器人轮子、设置在该扬声器机器人顶部的机器人手臂；在该机器人手臂的手部上设置有扬声器；
    所述步骤S0还包括提供机器人家具；机器人家具包括可运动地、用于承载阅听人的机器人座椅和可运动地、托有用于播放视频的显示屏或投影屏幕的机器人站立架；
    所述步骤S2还包括：同步地移动处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人，并保持处于终端环境中的机器人座椅、机器人站立架以及扬声器机器人之间的相对位置。
  6. 根据权利要求5所述的终端混音的播放方法，其特征在于，扬声器可滑动地设置于由电机控制的导轨上；
    所述步骤S2还包括：扬声器以运动状态文件所记录的对应的发声体的运动轨迹在导轨上进行运动。
  7. 根据权利要求 3-6 任意一项所述的终端混音的播放方法，其特征在于，所有扬声器通过 WiFi 连接在一起。
  8. 根据权利要求7所述的终端混音的播放方法，其特征在于，所述步骤S1还包括：提供与所述多个麦克风中的部分或全部麦克风通讯连接，并与所述多个麦克风中的部分或全部麦克风对应的拟声装置通讯连接的声音修改设备；该声音修改设备修改所述多个麦克风中的部分或全部麦克风各自录制的音轨的声音质量或给所述多个麦克风中的部分或全部麦克风各自录制的音轨增加声音效果；
    所述步骤S2还包括：与所述多个麦克风中的部分或全部麦克风对应的拟声装置同步地播放该声音修改设备修改过的对应的音轨。
  9. 根据权利要求8 所述的终端混音的播放方法，其特征在于，所述多个麦克风所录制的音轨以 EMX 文件格式保存。
  10. 一种终端混音系统，其特征在于，该终端混音系统包括与处于初始环境中的多个发声体对应的、用于同步地将对应发声体声音录制为音轨的多个麦克风，用于同步地将多个发声体的运动状态记录为运动状态文件的运动追踪设备，类型和尺寸与初始环境对应的终端环境以及与所述多个麦克风一一对应的、与对应的麦克风通讯连接、并与运动追踪设备通讯连接、以同步地以运动状态文件所记录的对应的发声体的运动状态进行运动、并同步地播放该对应的麦克风录制的音轨，从而播放出终端混音的多个拟声设备；每个拟声设备设置在终端环境中与初始环境中与该拟声设备对应的发声体所处位置对应的终端位置上。
PCT/CN2015/074243 2014-05-08 2015-03-13 终端混音系统和播放方法 WO2015169124A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
EP15789399.1A EP3142383B1 (en) 2014-05-08 2015-03-13 Terminal sound mixing system and playing method
US15/306,998 US9986364B2 (en) 2014-05-08 2015-03-13 Endpoint mixing system and playing method thereof
DK15789399.1T DK3142383T3 (da) 2014-05-08 2015-03-13 Slutanordningslydmiksesystem og afspilningsfremgangsmåde
CN201580021969.XA CN106465008B (zh) 2014-05-08 2015-03-13 终端混音系统和播放方法
JP2016565188A JP6285574B2 (ja) 2014-05-08 2015-03-13 エンドポイントミキシングシステムおよびその再生方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
HK14104355.5 2014-05-08
HK14104355.5A HK1195445A2 (zh) 2014-05-08 2014-05-08 終端混音系統和播放方法

Publications (1)

Publication Number Publication Date
WO2015169124A1 true WO2015169124A1 (zh) 2015-11-12

Family

ID=51845045

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/074243 WO2015169124A1 (zh) 2014-05-08 2015-03-13 终端混音系统和播放方法

Country Status (7)

Country Link
US (1) US9986364B2 (zh)
EP (1) EP3142383B1 (zh)
JP (1) JP6285574B2 (zh)
CN (1) CN106465008B (zh)
DK (1) DK3142383T3 (zh)
HK (1) HK1195445A2 (zh)
WO (1) WO2015169124A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107566949A (zh) * 2016-06-30 2018-01-09 上海博泰悦臻网络技术服务有限公司 一种音效配置方法、系统、一种均衡器以及一种电子设备
WO2018020337A1 (en) 2016-07-28 2018-02-01 Siremix Gmbh Endpoint mixing product

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
HK1195445A2 (zh) * 2014-05-08 2014-11-07 黃偉明 終端混音系統和播放方法
USD841621S1 (en) * 2016-12-29 2019-02-26 Facebook, Inc. Electronic device
US11853076B1 (en) 2017-07-21 2023-12-26 AI Incorporated Virtual reality concert system
KR102224216B1 (ko) * 2017-12-22 2021-03-08 주식회사 오드아이앤씨 공연 음악 플랫폼 시스템
US10317505B1 (en) 2018-03-29 2019-06-11 Microsoft Technology Licensing, Llc Composite sound output for network connected devices
CN110534110B (zh) * 2018-05-25 2022-04-15 深圳市优必选科技有限公司 一种机器人及提高其语音交互识别率的方法、装置和电路
CA185622S (en) * 2018-10-10 2020-01-24 Xiaofeng Gu Leopard headphones
CN110392276B (zh) * 2019-07-29 2021-06-22 湖南卡罗德音乐集团有限公司 一种基于rtmp协议同步传输midi的直播录播方法
US11496854B2 (en) 2021-03-01 2022-11-08 International Business Machines Corporation Mobility based auditory resonance manipulation
CN114666721B (zh) * 2022-05-05 2024-02-06 深圳市丰禾原电子科技有限公司 一种具有终端跟踪模式的wifi音箱及其控制方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3705957A (en) * 1970-02-19 1972-12-12 David S Goldsmith Translational,rotational and vertical movement controlled sound source pick-up system
US20050259832A1 (en) * 2004-05-18 2005-11-24 Kenji Nakano Sound pickup method and apparatus, sound pickup and reproduction method, and sound reproduction apparatus
CN1741601A (zh) * 2004-08-27 2006-03-01 索尼株式会社 声音产生方法和设备、声音再现方法和设备
US20110268287A1 (en) * 2009-01-08 2011-11-03 Yamaha Corporation Loudspeaker system and sound emission and collection method
TW201225696A (en) * 2010-12-03 2012-06-16 Merry Electronics Co Ltd Interactive sound playback
CN102640517A (zh) * 2009-10-28 2012-08-15 阿尔卡特朗讯 自操控有向扬声装置及其操作方法
HK1195445A2 (zh) * 2014-05-08 2014-11-07 黃偉明 終端混音系統和播放方法

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0377500A (ja) * 1989-08-19 1991-04-03 Sanyo Electric Co Ltd 音場補正装置
JPH06318087A (ja) * 1993-05-07 1994-11-15 Mitsui Constr Co Ltd 舞台用音響制御方法と装置
JPH09205607A (ja) * 1996-01-25 1997-08-05 Sony Corp ビデオ記録装置および再生装置
JP4097726B2 (ja) * 1996-02-13 2008-06-11 常成 小島 電子音響装置
JP3921817B2 (ja) * 1998-06-18 2007-05-30 ヤマハ株式会社 自動演奏装置およびフィンガリング記録装置
JP4423790B2 (ja) 2001-01-11 2010-03-03 ソニー株式会社 実演システム、ネットワークを介した実演方法
JP4363004B2 (ja) * 2002-05-24 2009-11-11 ソニー株式会社 音響提示装置と音響提示方法
JP4391091B2 (ja) * 2003-01-17 2009-12-24 ソニー株式会社 情報伝送方法、情報伝送装置、情報記録方法、情報記録装置、情報再生方法、情報再生装置および記録媒体
US8483757B2 (en) * 2004-01-09 2013-07-09 Revo Labs, Inc. Wireless multi-user audio system
WO2005076661A1 (ja) * 2004-02-10 2005-08-18 Mitsubishi Denki Engineering Kabushiki Kaisha 超指向性スピーカ搭載型移動体
JP2006142407A (ja) * 2004-11-17 2006-06-08 Sanyo Electric Co Ltd ロボット装置およびロボット装置システム
JP3112258U (ja) * 2005-05-02 2005-08-04 株式会社アガツマ 発音玩具
US20120306907A1 (en) 2011-06-03 2012-12-06 Huston Charles D System and Method for Inserting and Enhancing Messages Displayed to a User When Viewing a Venue
JP4678505B2 (ja) * 2005-08-16 2011-04-27 ソニー株式会社 ２輪走行装置及び走行停止方法
US20080304677A1 (en) * 2007-06-08 2008-12-11 Sonitus Medical Inc. System and method for noise cancellation with motion tracking capability
US20090237564A1 (en) * 2008-03-18 2009-09-24 Invism, Inc. Interactive immersive virtual reality and simulation
US9037468B2 (en) * 2008-10-27 2015-05-19 Sony Computer Entertainment Inc. Sound localization for user in motion
CN101590323B (zh) * 2009-07-08 2012-10-31 北京工业大学 独轮机器人系统及其控制方法
US20120099594A1 (en) * 2010-10-22 2012-04-26 Phorus Llc Media distribution architecture
HUE054452T2 (hu) * 2011-07-01 2021-09-28 Dolby Laboratories Licensing Corp Rendszer és eljárás adaptív hangjel elõállítására, kódolására és renderelésére
WO2013186593A1 (en) * 2012-06-14 2013-12-19 Nokia Corporation Audio capture apparatus
CN103941223B (zh) * 2013-01-23 2017-11-28 Abb技术有限公司 声源定位系统及其方法
US10129658B2 (en) * 2013-07-22 2018-11-13 Massachusetts Institute Of Technology Method and apparatus for recovering audio signals from images

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3705957A (en) * 1970-02-19 1972-12-12 David S Goldsmith Translational,rotational and vertical movement controlled sound source pick-up system
US20050259832A1 (en) * 2004-05-18 2005-11-24 Kenji Nakano Sound pickup method and apparatus, sound pickup and reproduction method, and sound reproduction apparatus
CN1741601A (zh) * 2004-08-27 2006-03-01 索尼株式会社 声音产生方法和设备、声音再现方法和设备
US20110268287A1 (en) * 2009-01-08 2011-11-03 Yamaha Corporation Loudspeaker system and sound emission and collection method
CN102640517A (zh) * 2009-10-28 2012-08-15 阿尔卡特朗讯 自操控有向扬声装置及其操作方法
TW201225696A (en) * 2010-12-03 2012-06-16 Merry Electronics Co Ltd Interactive sound playback
HK1195445A2 (zh) * 2014-05-08 2014-11-07 黃偉明 終端混音系統和播放方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3142383A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107566949A (zh) * 2016-06-30 2018-01-09 上海博泰悦臻网络技术服务有限公司 一种音效配置方法、系统、一种均衡器以及一种电子设备
WO2018020337A1 (en) 2016-07-28 2018-02-01 Siremix Gmbh Endpoint mixing product
CN109417675A (zh) * 2016-07-28 2019-03-01 赛耳混音有限公司 终端混音产品
US10993059B2 (en) 2016-07-28 2021-04-27 Siremix Gmbh Endpoint mixing product

Also Published As

Publication number Publication date
EP3142383A4 (en) 2018-01-10
JP2017520139A (ja) 2017-07-20
JP6285574B2 (ja) 2018-02-28
HK1195445A2 (zh) 2014-11-07
DK3142383T3 (da) 2019-07-22
CN106465008A (zh) 2017-02-22
EP3142383A1 (en) 2017-03-15
US9986364B2 (en) 2018-05-29
US20170055100A1 (en) 2017-02-23
EP3142383B1 (en) 2019-04-17
CN106465008B (zh) 2018-04-17

Similar Documents

Publication Publication Date Title
WO2015169124A1 (zh) 终端混音系统和播放方法
US10034091B2 (en) Multi-channel audio vibratory entertainment system
US10957297B2 (en) Self-produced music apparatus and method
Owsinski The recording engineer's handbook
US9942681B2 (en) Appliance for receiving and reading audio signals and live sound system
US20080184864A1 (en) Electric guitar including a connection for a digital music player
JP7143632B2 (ja) 再生システムおよび再生方法
US20150356960A1 (en) System and method for sound augmentation of acoustic musical instruments
CN106205577A (zh) 一种具有空间音效感的可柔性配置扬声器阵列的电子乐器
WO2018122730A1 (en) System and method for reproducing the sound of an orchestra
Bartlett Stereo microphone techniques
US6881892B2 (en) Method of configurating acoustic correction filter for stringed instrument
WO2023090831A1 (ko) 사용자 입력에 기반하여 사운드를 제공하는 전자 장치 및 그 동작 방법
Mulder Live sound and the disappearing digital
Einbond Mapping the Klangdom Live: Cartographies for piano with two performers and electronics
WO2019242097A1 (zh) 一种演绎系统
Tremblay Tuning to trust: System calibration as creative enabler
Pinch The Art of a New Technology: Early Synthesizer Sounds
CN105744443B (zh) 用于弦乐器的数字音频处理系统
Krachunova-Popova Recording music on set–specifics and challenges
Filosa Electronics Integration and Sound Reinforcement in the 21st Century Marching Arts
WO2022196073A1 (ja) 情報処理装置、情報処理方法、およびプログラム
Baxter The Art and Science of Microphones and Other Transducers
Schlienger Acoustic Localisation for Spatial Reproduction of Moving Sound Source: Application Scenarios & Proof of Concept.
de Dios Cuartas et al. The incursion of stereo into Spanish popular music: the English influence in thedefinition of a local sound through the professional exchanges between Madrid, Barcelona and London

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15789399

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2015789399

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015789399

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2016565188

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 15306998

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE