WO2010092140A2 - Dispositif et procede de controle du defilement d'un fichier de signaux a reproduire - Google Patents
Dispositif et procede de controle du defilement d'un fichier de signaux a reproduire Download PDFInfo
- Publication number
- WO2010092140A2 WO2010092140A2 PCT/EP2010/051763 EP2010051763W WO2010092140A2 WO 2010092140 A2 WO2010092140 A2 WO 2010092140A2 EP 2010051763 W EP2010051763 W EP 2010051763W WO 2010092140 A2 WO2010092140 A2 WO 2010092140A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- module
- signals
- control
- keystrokes
- file
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000033001 locomotion Effects 0.000 claims abstract description 30
- 230000033764 rhythmic process Effects 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 7
- 230000001537 neural effect Effects 0.000 claims description 7
- 210000004556 brain Anatomy 0.000 claims description 6
- 238000012790 confirmation Methods 0.000 claims description 4
- 230000003247 decreasing effect Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000011282 treatment Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000001020 rhythmical effect Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 206010003830 Automatism Diseases 0.000 description 1
- 241000004441 Sorites Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/36—Accompaniment arrangements
- G10H1/40—Rhythm
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0033—Recording/reproducing or transmission of music for electrophonic musical instruments
- G10H1/0041—Recording/reproducing or transmission of music for electrophonic musical instruments in coded form
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/04—Time compression or expansion
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
- A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
- A63B2071/0625—Emitting sound, noise or music
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
- A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
- A63B71/0686—Timers, rhythm indicators or pacing apparatus using electric or electronic means
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/155—User input interfaces for electrophonic musical instruments
- G10H2220/201—User input interfaces for electrophonic musical instruments for movement interpretation, i.e. capturing and recognizing a gesture or a specific kind of movement, e.g. to control a musical instrument
- G10H2220/206—Conductor baton movement detection used to adjust rhythm, tempo or expressivity of, e.g. the playback of musical pieces
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/155—User input interfaces for electrophonic musical instruments
- G10H2220/395—Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
- G10H2240/171—Transmission of musical instrument data, control or status information; Transmission, remote access or control of music data for electrophonic musical instruments
- G10H2240/281—Protocol or standard connector for transmission of analog or digital data to or from an electrophonic musical instrument
- G10H2240/311—MIDI transmission
Definitions
- the invention relates to controlling the scrolling of an audio file in real time.
- Electronic musical synthesis devices allow to play one or more synthetic instruments (made from acoustic models or samples of piano sounds, guitar, other stringed instruments, saxophone or other wind instruments, etc.) using a note input interface.
- the notes entered are converted into signals by a synthesis device connected to the interface by a connector and a software interface using the MIDI standard (Musical Instrument Digital Interface).
- MIDI Musical Instrument Digital Interface
- Automatic programming of the instrument or instruments for generating sets of notes corresponding to a partition can be performed using software provided for this purpose.
- the MAX / MSP programming software is one of the most popular and allows you to create such an application for the interpretation of musical scores.
- Such an application includes a graphical programming interface which makes it possible to choose and control note sequences and to control the DSP (Digital Signal Processor) of musical synthesis.
- DSP Digital Signal Processor
- the existing devices do not allow this control of the scrolling rhythm of the different types of audio files used (MP3 - MPEG (Moving Picture Expert Group) 1/2 Layer 3, WAV - WAVeform audio format, WMA - Windows Media Audio .. .) to reproduce pre-recorded music on electronic equipment.
- No device of the prior art can perform this control in real time under musicality conditions that are acceptable.
- the PCT application WO98 / 19294 deals only with the control of the scrolling rhythm of MIDI files and not with substantially continuously encoded signal files, such as mp3 or wav files.
- the present invention provides an answer to these limitations of the prior art by using an algorithm for controlling the scrolling of the automatic score which makes it possible to ensure satisfactory musical rendering.
- the present invention discloses a device for controlling by a user the rate of movement of a pre-recorded file of signals to be reproduced and the intensity of said signals, said signals being encoded in said pre-recorded file in a substantially continuous manner, said device comprising a first control keys input interface module, a second input module of said signals to be reproduced, a third module for temporal control of said prerecorded signals and a device for reproducing the inputs of the first three modules, said control device being characterized in that said second module is able to be programmed to determine the times when are waited for control keystrokes of the file scroll rate, and in that said third module is able to calculate for a number of control strikes a corrected velocity factor (CSF) relative to strikes preprogrammed in the second module and keystrokes actually entered in the first module and a relative intensity factor of the velocities of said keystrokes actually entered and expected then to adjust the rate of movement of said second module to adjust said corrected speed factor (CSF) to subsequent keystroke
- the first module comprises a MIDI interface.
- the first module comprises a motion capture sub-module and a sub-module for analysis and interpretation of gestures receiving as input the outputs of the motion capture sub-module.
- the motion capture sub-module performs said motion capture according to at least a first and a second axis, the sub-module for analyzing and interpreting gestures comprises a filtering function, a significant gesture detection function.
- the first module comprises an interface for capturing neuronal signals from the user's brain and a sub-module for interpreting said neural signals.
- the velocity of the typing input is calculated from the deviation of the signal output of the second sensor.
- the first module further comprises a submodule capable of interpreting the user's gestures whose output is used by the third module to control a characteristic of the audio output chosen from the group constituted by the vibrato and the tremolo.
- the second module comprises a tag placement sub-module in the prerecorded signal file to be reproduced at the times when the file's scroll rhythm control keystrokes are expected, said tags being generated automatically according to the rhythm of the files.
- pre-recorded signals that can be moved by a MIDI interface.
- the value chosen in the third module for adjusting the running speed of the second module is equal to a value chosen from a set of calculated values, one of whose limits is calculated by applying a speed factor CSF equal to the ratio of the time interval between the next tag and the previous tag minus the time interval between the current tap and the previous tap at the time interval between the current tap and the previous tap and whose other values are calculated by interpolation linear between the current value and the value corresponding to that of the terminal used for applying the CSF speed factor.
- the value chosen in the third module to adjust the scrolling speed of the second module is equal to the value corresponding to that of the terminal used for the application of the CSF speed factor.
- the invention also discloses a method of controlling by a user the rate of movement of a prerecorded file of signals to be reproduced and the intensity of said signals, said signals being encoded in said prerecorded file substantially continuously, said method comprising a first control keys input interface step, a second input step of said signals to be reproduced, a third step of time control of said prerecorded signals and a step of reproducing the inputs of the first three steps, said method of control being characterized in that the said second step is able to be programmed to determine the times when are expected control keystrokes of the file scroll rate, and in that the said third step is able to calculate for a number of control strokes a corrected speed factor (CSF) relative to preprogr strikes ammées in the second step and keystrokes actually entered in the first step and a relative intensity factor velocities of said keystrokes actually entered and expected then adjust the scroll rate in said second step to adjust said corrected speed factor (CSF ) to the following keystrokes at a chosen value and the
- Another advantage of the invention is that it makes it possible to control the scrolling of the prerecorded audio files intuitively.
- New scroll control algorithms can also be easily integrated into the device of the invention.
- the control of the sound power of the prerecorded audio files can also be carried out in a simple manner by the device of the invention.
- FIGS. 1A, 1B and 1C show, in a simplified manner, a functional architecture of a device for controlling the running speed of a prerecorded audio file according to three embodiments of the invention
- FIG. 2 represents the flowchart of a low-pass filtering of the signals of a motion sensor in one of the embodiments of the invention as represented in FIG. 1B;
- FIGS. 3A and 3B show two cases of implementation of the invention in which respectively the typing speed is higher / lower than that of running the audio band;
- FIG. 4 represents a flowchart of the processes of the measurement function of the velocity of the striking in one embodiment of the invention
- FIG. 5 represents a general flowchart of the treatments in one embodiment of the invention.
- FIG. 6 represents a detail of FIG. 5 which shows the rhythmic control points desired by a user of a device according to one embodiment of the invention
- FIG. 7 represents an expanded flowchart of a time control method in one embodiment of the invention.
- FIGS. 1A, 1B and 1C show three embodiments of the invention which differ only in the control keys input interface module 10.
- the characteristics of the module 20, the input of the signals to be reproduced, the timing control module 30 and the audio output module 40 are described below.
- At least three input interface modules are possible. They are respectively represented in FIGS. 1A, 1B and 1C.
- Each input module comprises a submodule 1 which captures interaction commands with the device and a part that makes the input and the translation of these modules. commands in the device.
- MIDI controllers 1 10A are control surfaces that may have buttons, faders (linear faders for adjusting the level of sound sources, pads (touch sensitive surfaces) or rotary knobs.These controllers are not sound management or playback devices, they only produce MIDI data. Other types of control surfaces can be used, such as a virtual harp, guitar or saxophone. These controllers can have a display screen.Whatever the elements that make up the control surface, all buttons, sliders, faders, keys, pads are assignable to each element of the visual interface of the software through setups (configuration files) . The sound controls can also be coupled with lighting controls.
- a MIDI controller 1 1 OA is connected to the time control processor 30 through an interface whose hardware part is a 5-pin DIN connector. Several MIDI controllers can be connected to the same computer by being chained together.
- the communication link is set at 31,250 bauds.
- the coding uses 128 tone values (from 0 to 127), the notes messages ranging between 8,175 Hz and 12544 Hz with a half-tone resolution.
- FIG. 1B there is shown a motion capture assembly 10B comprising a Movea TM MotionPod TM motion sensor 1 10B and a motion analysis interface 120B.
- An AirMouse TM or GyroMouse TM can also be used in place of the MotionPod, as well as other motion sensors.
- a MotionPod comprises a tri-axis accelerometer, a tri-axis magnetometer, a pre-processing capability for preforming signals from the sensors, a radiofrequency transmission module of said signals to the processing module itself and a battery.
- This motion sensor is called "3A3M" (three axes of accelerometer and three axes of magnetometer).
- Accelerometers and magnetometers are small, low-power, low-cost commercial micro-sensors, such as a Kionix TM three-way accelerometer (KXPA4 3628) and HoneyWell TM magnetometers of the HMC1041 Z type (1). vertical track) and HMC1042L for the 2 horizontal tracks.
- the raw signals are transmitted by a radio frequency protocol in the Bluetooth TM band (2, 4GHz) optimized for consumption in this type of applications.
- the data is therefore raw to a controller that can receive data from a set of sensors. They are read by the controller and made available software.
- the sampling rate is adjustable. By default, it is set at 200 Hz. Higher values (up to 3000 Hz or more) can nevertheless be envisaged, allowing greater accuracy in the detection of shocks, for example.
- the radio frequency protocol of the MotionPod makes it possible to guarantee the provision of the data to the controller with a controlled delay, which should not exceed 10ms (at 200 Hz), which is important for the music.
- An accelerometer of the above type makes it possible to measure the longitudinal displacements along its three axes and, by transformation, angular displacements (except those resulting from a rotation around the direction of the terrestrial gravitational field) and orientations with respect to a Cartesian reference system in three dimensions.
- a set of magnetometers of the above type makes it possible to measure the orientation of the sensor to which it is fixed with respect to the terrestrial magnetic field and thus displacements and orientations with respect to the three axes of the reference frame (except around the direction of the magnetic field earthly).
- the 3A3M combination provides complementary and smooth motion information.
- the AirMouse includes two gyrometer type sensors, each with an axis of rotation.
- the gyrometers used are from the Epson reference XV3500. Their axes are orthogonal and deliver the pitch angles (yaw or rotation around the axis parallel to the horizontal axis of a plane facing the user of the AirMouse) and yaw (pitch or rotation around a axis parallel to the vertical axis of a plane facing the user of the AirMouse).
- the instantaneous pitch and yaw speeds measured by the two gyrometer axes are transmitted by radio frequency protocol to a controller moving a cursor in a screen facing the user.
- the gesture analysis and interpretation module 120B provides signals that are directly usable by the time control processor 30. For example, the signals along an axis of the accelerometer and the magnetometer of the MotionPod are combined according to the method described in FIG. patent application filed by the present applicants in the patent application entitled "DEVICE AND METHOD FOR INTERPRETATION OF MUSICAL GESTURES". The processes implemented in the module 120B are performed in software.
- the treatments firstly comprise a low-pass filtering of the outputs of the sensors of the two modalities (accelerometer and magnetometer) whose detailed operation is explained in FIG. 2.
- This filtering of the output signals of the motion sensor controller uses a recursive approach of order 1.
- the gain of the filter may for example be set at 0.3.
- the filter equation is given by the following formula:
- the treatment then comprises a low-pass filtering of the two modalities with a cut-off frequency lower than that of the first filter.
- This lower cut-off frequency results from the choice of a coefficient of the second filter which is lower than the gain of the first filter.
- the coefficient of the second filter can be set at 0.1.
- the processing comprises a detection of a zero of the derivative of the output signal of the accelerometer with the measurement of the output signal of the magnetometer.
- a negative sign of the product FDA (n) * FDA (n-1) indicates a zero of the derivative of the filtered signal of the accelerometer and thus detects a strike.
- the processing module checks the intensity of the deviation of the other modality at the filtered output of the magnetometer. If this value is too low, the strike is considered not as a primary strike but as a secondary or ternary strike and discarded.
- the threshold for discarding non-primary strikes depends on the expected amplitude of the deviation of the magnetometer. Typically, this value will be of the order of 5/1000 in the envisaged applications. This part of the treatment thus makes it possible to eliminate the insignificant strikes.
- Figure 1C has an interface 1 OC, 1 10C Brain -> Computer or "Brain Computer Interface". These interfaces are still in the advanced stage of research but offer promising possibilities, especially in the field of musical performance.
- the neural signals are provided to an interpretation interface 120C which converts these signals into commands for the time control processor 30.
- Such neural devices operate, for example, as follows.
- the person's scalp is provided with a network of sensors for electrical and / or magnetic activity resulting from the neuronal activity of the subject.
- scientific models have not yet been signals to find the intention of the subject, for example in our case to beat the measure in a musical context.
- a pre-recorded music file 20 to one of the standard formats (MP3, WAV, WMA, etc.) is taken from a storage unit by a reader.
- To this file is associated another file with time marks or "tags" at predetermined times; for example, the table below shows nine tags at times in milliseconds that are listed next to the index of the decimal point:
- the tags can advantageously be placed at beats of the same index in the piece that is played. There is no constraint on the number of tags.
- Several techniques are possible to place tags in a pre-recorded piece of music: - Manually, by searching on the musical wave the point corresponding to a rhythm where a tag must be placed; it is a possible but tedious process;
- the prerecorded signal input module 20 to be reproduced can process different types of audio files, in the MP3, WAV, WMA formats.
- the file may also include other multimedia content than a simple sound recording. It may be for example video content, with or without soundtrack, which will be tagged and whose scrolling can be controlled by the input module 10.
- the time control processor 30 synchronizes the signals received from the input module 10 and the pre-recorded music piece 20, in a manner commented on in FIGS. 3A and 3B.
- the audio output 40 reproduces the pre-recorded piece of music from the module 20 with the variations of rhythm introduced by the commands of the input module 10 interpreted by the control processor 30. Any sound reproduction device does the job, including headphones, speakers.
- FIGS. 3A and 3B show two cases of implementation of the invention in which respectively the typing speed is higher / lower than that of running the audio band.
- the audio player of the module 20 starts playing the pre-recorded piece of music at a rate given. This rhythm can for example be indicated by several small prior strikes. Whenever the time control processor receives a strike signal, the current playing speed of the user is calculated.
- SF (n) [T (n + 1) - T (n)] / [H (n + 1) - H (n)]
- the player accelerates and advances the pre-recorded track: a new keystroke is received by the processor before the audio player has reached the sample of the piece of music where the tag is placed corresponding to this hit.
- the speed factor SF is 4/3.
- the time control processor makes the reading of the file 20 jump to the sample containing the index mark corresponding to the keystroke. Part of the pre-recorded music is lost, but the quality of the musical rendering is not too disturbed because the attention of those who listen to a piece of music usually focuses on the elements of the main rhythm and the tags will normally be placed on these elements of the main rhythm.
- the listener who is waiting for that item will pay less attention to the absence of the part of the pre recorded song that has been skipped, this jump thus passing almost unnoticed.
- the quality of listening can be further enhanced by smoothing the transition. This smoothing can by example be operated by interpolating a few samples (a dozen) between before and after the tag to which the player is blown up to catch up with the player's typing speed. Playback of the pre-recorded song continues at the new speed resulting from that jump.
- FIG. 3B Three positions of the tags at time n + 2 (in the time scale of the audio file) before changing the speed of the reader are indicated in FIG. 3B:
- the first starting from the left T (n + 2) is that corresponding to the speed of movement prior to the slowdown of the player;
- the second, NT 1 (n + 2) is the result of the calculation consisting in adjusting the speed of the player's scrolling speed to the player's typing speed by using the speed factor SF; we see that in this case the tags remain ahead of the keystrokes;
- NT 2 (n + 2) is the result of a calculation using a corrected speed factor CSF; this corrected factor is calculated so that the dates of the following keystroke and tag are identical, which is seen in Figure 3B.
- Another improvement, applicable to the embodiment comprising one or more motion sensors, is to measure the player's striking energy or velocity to control the volume of the audio output.
- the manner in which velocity is measured is also disclosed in the patent application filed by the present applicants in the patent application entitled "DEVICE AND METHOD FOR INTERPRETATION OF
- This part of the processing carried out by the module 120B for analyzing and interpreting gestures is represented in FIG. 4.
- the processing module calculates a velocity signal (or volume) of the strike by using the deviation of the filtered signal at the output of the magnetometer.
- DELTAB (n) is introduced into the sample n which can be considered as the pre-filtered signal of the centered magnetometer and which is calculated as follows:
- VEL (n) Max ⁇ DELTAB (n), DELTAB (p) ⁇ - Min ⁇ DELTAB (n), DELTA (p) ⁇
- p is the index of the sample in which the previous primary strike was detected.
- the velocity is the race (difference Max-Min) of the derivative signal between two detected primary strikes, characteristics of musical gestures.
- this embodiment comprising several motion sensors, to control by other gestures other musical parameters such as the spatial origin of the sound (or panning), the vibrato or the tremolo.
- a sensor in one hand will detect the striking while another sensor held in the other hand will detect the spatial origin of sound or tremolo.
- Rotations of the hand can also be taken into account: when the palm of the hand is horizontal, one obtains a value of the spatial origin of the sound or the tremolo; when the palm is vertical, another value of the same parameter is obtained; in both cases, the movements of the hand in space provide the detection of the strikes.
- conventionally used controllers can also be used in this embodiment of the invention to control the spatial origin of sounds, tremolo or vibrato.
- the invention can be advantageously implemented by processing the keystrokes via a MAX / MSP program.
- Figure 5 shows the general flowchart of the treatments in such a program.
- the blue display shows the waveform associated with the audio track loaded into the system. There is a classical part to listen to the original piece.
- Figure 7 details the part of Figure 5 at the bottom right that represents the time control that is applied.
- the SF acceleration / deceleration coefficient is calculated by comparing the existing duration between two consecutive marks on the one hand in the original piece, on the other hand in the current game of the user. The formula for calculating this speed factor is given above in the description.
- a timeout is set to stop the scrolling of the audio if the user has no longer typed for a time depending on the current musical content.
- the "if" object in the left column detects whether it is slowing down or accelerating. In the event of a slowdown, the reading speed of the algorithm is changed, but no jump is made in the audio file.
- the new reading speed is not necessarily exactly the one calculated in the right column (SF), but can be corrected (CSF speed factor) to take into account that we have already exceeded in the audio the corresponding mark at the last action of the player;
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Electrophonic Musical Instruments (AREA)
- General Health & Medical Sciences (AREA)
- Physical Education & Sports Medicine (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
Claims
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201080011162.5A CN102598117B (zh) | 2009-02-13 | 2010-02-12 | 用于控制要再现的信号文件的回放的设备和方法 |
JP2011549574A JP5945815B2 (ja) | 2009-02-13 | 2010-02-12 | 再生対象信号のファイルの再生を制御するための装置と方法 |
US13/201,175 US8880208B2 (en) | 2009-02-13 | 2010-02-12 | Device and method for controlling the playback of a file of signals to be reproduced |
KR1020117021349A KR101682736B1 (ko) | 2009-02-13 | 2010-02-12 | 재생되어질 신호 파일의 녹음재생을 제어하기 위한 장치 및 방법 |
EP10706971A EP2396788A2 (fr) | 2009-02-13 | 2010-02-12 | Dispositif et procede de controle du defilement d'un fichier de signaux a reproduire |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0950919A FR2942344B1 (fr) | 2009-02-13 | 2009-02-13 | Dispositif et procede de controle du defilement d'un fichier de signaux a reproduire |
FR0950919 | 2009-02-13 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2010092140A2 true WO2010092140A2 (fr) | 2010-08-19 |
WO2010092140A3 WO2010092140A3 (fr) | 2011-02-10 |
Family
ID=41136768
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2010/051763 WO2010092140A2 (fr) | 2009-02-13 | 2010-02-12 | Dispositif et procede de controle du defilement d'un fichier de signaux a reproduire |
Country Status (7)
Country | Link |
---|---|
US (1) | US8880208B2 (fr) |
EP (1) | EP2396788A2 (fr) |
JP (1) | JP5945815B2 (fr) |
KR (1) | KR101682736B1 (fr) |
CN (1) | CN102598117B (fr) |
FR (1) | FR2942344B1 (fr) |
WO (1) | WO2010092140A2 (fr) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102592485A (zh) * | 2011-12-26 | 2012-07-18 | 中国科学院软件研究所 | 一种以动作方向变化控制音符播放的方法 |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2396711A2 (fr) * | 2009-02-13 | 2011-12-21 | Movea S.A | Dispositif et procede d'interpretation de gestes musicaux |
JP5902919B2 (ja) * | 2011-11-09 | 2016-04-13 | 任天堂株式会社 | 情報処理プログラム、情報処理装置、情報処理システム、情報処理方法 |
JP2013213946A (ja) * | 2012-04-02 | 2013-10-17 | Casio Comput Co Ltd | 演奏装置、方法及びプログラム |
JP2013213744A (ja) | 2012-04-02 | 2013-10-17 | Casio Comput Co Ltd | 姿勢検出装置、方法及びプログラム |
JP6044099B2 (ja) | 2012-04-02 | 2016-12-14 | カシオ計算機株式会社 | 姿勢検出装置、方法及びプログラム |
EP2835769A1 (fr) | 2013-08-05 | 2015-02-11 | Movea | Procédé, dispositif et système de capture annotée de données de capteurs et de modélisation de la foule d'activités |
US11688377B2 (en) | 2013-12-06 | 2023-06-27 | Intelliterran, Inc. | Synthesized percussion pedal and docking station |
US9568994B2 (en) | 2015-05-19 | 2017-02-14 | Spotify Ab | Cadence and media content phase alignment |
US9536560B2 (en) | 2015-05-19 | 2017-01-03 | Spotify Ab | Cadence determination and media content selection |
CN106847249B (zh) * | 2017-01-25 | 2020-10-27 | 得理电子(上海)有限公司 | 一种发音处理方法及系统 |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1998019294A2 (fr) | 1996-10-25 | 1998-05-07 | Weinstock Frank M | Procede et appareil de correlation en temps reel d'une performance avec une interpretation musicale |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5662117A (en) * | 1992-03-13 | 1997-09-02 | Mindscope Incorporated | Biofeedback methods and controls |
US5629491A (en) * | 1995-03-29 | 1997-05-13 | Yamaha Corporation | Tempo control apparatus |
JP3149736B2 (ja) * | 1995-06-12 | 2001-03-26 | ヤマハ株式会社 | 演奏ダイナミクス制御装置 |
US5663514A (en) * | 1995-05-02 | 1997-09-02 | Yamaha Corporation | Apparatus and method for controlling performance dynamics and tempo in response to player's gesture |
JP3307152B2 (ja) * | 1995-05-09 | 2002-07-24 | ヤマハ株式会社 | 自動演奏制御装置 |
JP3750699B2 (ja) * | 1996-08-12 | 2006-03-01 | ブラザー工業株式会社 | 楽音再生装置 |
US5792972A (en) * | 1996-10-25 | 1998-08-11 | Muse Technologies, Inc. | Method and apparatus for controlling the tempo and volume of a MIDI file during playback through a MIDI player device |
JP2001125568A (ja) * | 1999-10-28 | 2001-05-11 | Roland Corp | 電子楽器 |
EP1855267B1 (fr) * | 2000-01-11 | 2013-07-10 | Yamaha Corporation | Dispositif et méthode pour détecter le mouvement d'un artiste et commander une interprétation musicale de manière interactive |
JP3646600B2 (ja) * | 2000-01-11 | 2005-05-11 | ヤマハ株式会社 | 演奏インターフェイス |
JP4320766B2 (ja) * | 2000-05-19 | 2009-08-26 | ヤマハ株式会社 | 携帯電話機 |
DE20217751U1 (de) * | 2001-05-14 | 2003-04-17 | Schiller, Rolf, 88212 Ravensburg | System zur Aufzeichnung und Wiedergabe von Musik |
JP2003015648A (ja) * | 2001-06-28 | 2003-01-17 | Kawai Musical Instr Mfg Co Ltd | 電子楽音発生装置及び自動演奏方法 |
DE10222315A1 (de) * | 2002-05-18 | 2003-12-04 | Dieter Lueders | Elektronischer Midi-Taktstock |
DE10222355A1 (de) * | 2002-05-21 | 2003-12-18 | Dieter Lueders | Audio-Dynamik-Zusatzmodul |
JP2004302011A (ja) * | 2003-03-31 | 2004-10-28 | Toyota Motor Corp | 指揮棒の振りタイミングに同期させて演奏する装置 |
JP2005156641A (ja) * | 2003-11-20 | 2005-06-16 | Sony Corp | 再生態様制御装置及び再生態様制御方法 |
EP1550942A1 (fr) * | 2004-01-05 | 2005-07-06 | Thomson Licensing S.A. | Interface utilisateur pour un dispositif de reproduction de fichiers audio |
WO2006050512A2 (fr) * | 2004-11-03 | 2006-05-11 | Plain Sight Systems, Inc. | Entraineur musical personnel |
US7402743B2 (en) * | 2005-06-30 | 2008-07-22 | Body Harp Interactive Corporation | Free-space human interface for interactive music, full-body musical instrument, and immersive media controller |
-
2009
- 2009-02-13 FR FR0950919A patent/FR2942344B1/fr not_active Expired - Fee Related
-
2010
- 2010-02-12 US US13/201,175 patent/US8880208B2/en not_active Expired - Fee Related
- 2010-02-12 WO PCT/EP2010/051763 patent/WO2010092140A2/fr active Application Filing
- 2010-02-12 KR KR1020117021349A patent/KR101682736B1/ko active IP Right Grant
- 2010-02-12 JP JP2011549574A patent/JP5945815B2/ja not_active Expired - Fee Related
- 2010-02-12 CN CN201080011162.5A patent/CN102598117B/zh not_active Expired - Fee Related
- 2010-02-12 EP EP10706971A patent/EP2396788A2/fr not_active Withdrawn
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1998019294A2 (fr) | 1996-10-25 | 1998-05-07 | Weinstock Frank M | Procede et appareil de correlation en temps reel d'une performance avec une interpretation musicale |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102592485A (zh) * | 2011-12-26 | 2012-07-18 | 中国科学院软件研究所 | 一种以动作方向变化控制音符播放的方法 |
CN102592485B (zh) * | 2011-12-26 | 2014-04-30 | 中国科学院软件研究所 | 一种以动作方向变化控制音符播放的方法 |
Also Published As
Publication number | Publication date |
---|---|
JP5945815B2 (ja) | 2016-07-05 |
KR20110115174A (ko) | 2011-10-20 |
WO2010092140A3 (fr) | 2011-02-10 |
FR2942344A1 (fr) | 2010-08-20 |
JP2012518192A (ja) | 2012-08-09 |
CN102598117A (zh) | 2012-07-18 |
US20120059494A1 (en) | 2012-03-08 |
EP2396788A2 (fr) | 2011-12-21 |
KR101682736B1 (ko) | 2016-12-05 |
US8880208B2 (en) | 2014-11-04 |
CN102598117B (zh) | 2015-05-20 |
FR2942344B1 (fr) | 2018-06-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2010092140A2 (fr) | Dispositif et procede de controle du defilement d'un fichier de signaux a reproduire | |
EP2396711A2 (fr) | Dispositif et procede d'interpretation de gestes musicaux | |
TWI433027B (zh) | 適應性使用者介面 | |
KR101915120B1 (ko) | 인공신경망을 이용한 오디오의 이퀄라이징 시스템 | |
WO2020224322A1 (fr) | Procédé et dispositif de traitement de fichier de musique, terminal et support d'informations | |
US20130032023A1 (en) | Real time control of midi parameters for live performance of midi sequences using a natural interaction device | |
JP2006171664A (ja) | 携帯及び通信可能な装置における電子音楽 | |
WO2017028686A1 (fr) | Procédé de traitement d'informations, dispositif de terminal et support de stockage informatique | |
WO2011133398A2 (fr) | Commande en temps réel de paramètres midi pour la performance en direct de séquences midi | |
WO2019114015A1 (fr) | Procédé de commande de performance de robot, et robot | |
FR2974226A1 (fr) | Procede de generation d'effet sonore dans un logiciel de jeu, programme d'ordinateur associe et systeme informatique pour executer des instructions du programme d'ordinateur. | |
EP1652180B1 (fr) | Procede de reproduction de documents audio a l'aide d'une interface presentant des groupes de documents, et appareil de reproduction associe | |
US9734809B2 (en) | Digital control of the sound effects of a musical instrument | |
CN111105776A (zh) | 有声播放装置及其播放方法 | |
CN113674723A (zh) | 一种音频处理方法、计算机设备及可读存储介质 | |
Yoshii et al. | INTER: D: a drum sound equalizer for controlling volume and timbre of drums | |
EP0978116A1 (fr) | Procede et dispositif pour l'enregistrement en boucles cycliques de plusieurs sequences phoniques | |
Tanaka et al. | MubuFunkScatShare: gestural energy and shared interactive music | |
EP3889958A1 (fr) | Égalisation de lecture audio dynamique à l'aide de caractéristiques sémantiques | |
JP5338312B2 (ja) | 自動演奏同期装置、自動演奏鍵盤楽器およびプログラム | |
FR2942345A1 (fr) | Dispositif et procede d'interpretation de gestes musicaux | |
WO2024080009A1 (fr) | Dispositif acoustique, procédé de commande acoustique et programme de commande acoustique | |
JP2004126934A (ja) | 音楽選曲装置、音楽選曲方法、プログラム記録媒体及びプログラム | |
JP2024528894A (ja) | ビデオクリップに基づくオーディオビジュアルコンテンツの生成 | |
CN115278350A (zh) | 一种渲染方法及相关设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080011162.5 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10706971 Country of ref document: EP Kind code of ref document: A2 |
|
REEP | Request for entry into the european phase |
Ref document number: 2010706971 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010706971 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011549574 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 20117021349 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13201175 Country of ref document: US |