CN101524594A - Anthropomorphic robot autonomously dancing along with rhythm - Google Patents
Anthropomorphic robot autonomously dancing along with rhythm Download PDFInfo
- Publication number
- CN101524594A CN101524594A CN200910061626A CN200910061626A CN101524594A CN 101524594 A CN101524594 A CN 101524594A CN 200910061626 A CN200910061626 A CN 200910061626A CN 200910061626 A CN200910061626 A CN 200910061626A CN 101524594 A CN101524594 A CN 101524594A
- Authority
- CN
- China
- Prior art keywords
- rhythm
- music
- subsystem
- beat
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to an anthropomorphic robot autonomously dancing along with rhythm. The brain of the robot is provided with an embedded computer system with musical rhythm recognition capability. The embedded computer system mainly comprises a hearing subsystem, a rhythm recognition subsystem and a dancing movement combination subsystem; wherein, the hearing subsystem obtains external played music by a sensor and converts the music into digital music signals which serve as the input of the rhythm recognition subsystem; the rhythm recognition subsystem processes the digital music signals and obtains the rhythm of the music and the characters of beats which comprises beats per minute and the beginning time of beats; the dancing movement combination subsystem autonomously combines dances according to the rhythm characters obtained by the rhythm recognition subsystem; the three subsystems process in parallel in the mode of a production line to cause the robot to autonomously dance along with rhythm. The anthropomorphic robot can autonomously perform a complete set of graceful and integrated dances along with the rhythm of the music. In addition, the anthropomorphic robot has the application prospect of art, education and recreational activity.
Description
Technical field
The present invention relates to robot, be specifically related to a kind of anthropomorphic robot of dancing with rhythm.
Background technology
Robot is as an interesting and complicated popular artificial intelligence study field, and it has merged the new and high technology in a plurality of fields such as robotics, electromechanical integration, Multi-Agent Cooperation, real-time vision system, radio communication, artificial intelligence.
At present in the world, the research of anthropomorphic robot promotes with two big races that mainly promptly FIRA and ROBOCUP in these two races, have added the event of anthropomorphic robot dancing.Robot research just develops towards the depth direction both at home and abroad, the level of intelligence of robot always is the subject matter that the researcher pays close attention to: wish on the one hand to come the multi-disciplinary joint study of hoisting machine people by robot soccer game, robot is also just towards more fully intelligence and application facet development widely on the other hand.In recent years, having occurred with ASIMO, HONDA, HRP-2 etc. in the academic research field is the large-scale anthropomorphic robot of representative.On high-grade toy robot market, the robot that multiple dance movement is stablized and can be finished in a few money walkings has abroad also appearred.The level of intelligence of anthropomorphic robot all has a few thing also in comprehensive lifting at aspects such as vision, the sense of equilibrium, language identification.And in music recognition, particularly the aspect that combines of rhythm identification and dance movement yet there are no report.
Dancing is to bring the high art that people's aesthetic feeling is enjoyed, and rhythm identification is the forward position investigative technique of current artificial intelligence, and anthropomorphic robot is the research focus that integrates multiple hard-core technology.Therefore triplicity is undoubtedly challenging highly difficult project, it is the product that new and high technology combines with art.
The research of anthropomorphic robot dancing at present also rests on the stage of artificial choreography, its process is according to selected music by the people, the complete dance movement of a pre-designed cover, write the program that realizes dance movement, performed according to set instruction by robot, robot does not have autonomous music analysis and autonomous choreography ability again.Aspect music recognition, domestic relevant work report is less, and related work comprises the note identification of single-tone and discerns based on the digital music of midi format, does not see the note identification at multiple musical instrument and chord, does not also have the report of rhythm identification related work.
Summary of the invention
Technical problem to be solved by this invention is: a kind of anthropomorphic robot with universal musical rhythm recognition capability is provided, and this robot can discern different rhythm music automatically, independently performs the graceful complete dancing of a cover.
The technical scheme that the present invention solves its technical problem employing is: a kind of anthropomorphic robot with universal musical rhythm recognition capability is provided, its brain is provided with the embedded computer system of universal musical rhythm recognition capability, this system mainly is made up of hearing subsystem, rhythm recognition subsystem and dancing movement combination subsystem, wherein: hearing subsystem obtains the music of external played by sensor, and being converted to digital music signal, this digital music signal is as the input of rhythm recognition subsystem; The rhythm recognition subsystem is handled digital music signal, obtains the rhythm and the beat feature of music, comprises per minute beat number and beat time started; The rhythm characteristic that dancing movement combination subsystem obtains according to the rhythm recognition subsystem independently carries out the dancing combination; These three subsystems make this robot independently dance with rhythm with the form parallel processing of streamline.
The present invention relates to science and technology field quite widely, comprise robotics, electromechanical integration, music signal processing, artificial intelligence etc., have very strong practicality.After the robot auditory system was heard music, music rhythm and the beat characteristic that the robot brain obtains according to analysis also selected corresponding dancing strategy, and robot can be along with the graceful complete dancing of autonomous performance one cover of the rhythm of music like this.Here said wherein music rhythm identification has type does not have special requirement, only requires that music is that the ordinary people can hear.The present invention also has following application prospect:
(1) art is used: the rhythm recognition technology can be developed as music recognition software, and this will expand the performance of existing music software.Recreational and the sight that the robot dancing is had has further increased the high-tech antagonism, thereby has bigger glamour.
(2) education and entertainment activity: this robot is each science and technology center and Youth Activity Center's exhibition at home, improves the interest of numerous teenagers to science, also can be used as amusement robot simultaneously and is widely used in aspects such as man-machine performance.
Description of drawings
Fig. 1 is the recording schematic flow sheet.
Fig. 2 be can the perception music rhythm psychologic acoustics experiment flow figure.
Fig. 3 be can not the perception music rhythm psychologic acoustics experiment flow figure.
Fig. 4 is the flow chart of identification music rhythm and beat.
Fig. 5 is six rank elliptic filter performance plots.
Fig. 6 is half Hanning window functional image.
Fig. 7 is half-wave checking process figure.
Fig. 8 is rhythm analysis process figure.
Fig. 9 is the phase calculation flow chart.
The specific embodiment
Anthropomorphic robot of dancing provided by the invention with rhythm, its brain is provided with the embedded computer system of universal musical rhythm recognition capability, this system mainly is made up of hearing subsystem, rhythm recognition subsystem and dancing movement combination subsystem, wherein: hearing subsystem obtains the music of external played by sensor, and being converted to digital music signal, this digital music signal is as the input of rhythm recognition subsystem; The rhythm recognition subsystem is handled digital music signal, obtains the rhythm and the beat feature of music, comprises per minute beat number and beat time started; The rhythm characteristic that dancing movement combination subsystem obtains according to the rhythm recognition subsystem independently carries out the dancing combination; These three subsystems make this robot independently dance with rhythm with the form parallel processing of streamline.
The rhythm recognition subsystem mainly extracts parts by bandpass filter group, waveform correction module, rhythm and forms, wherein: the digital music signal of input is decomposed into the digital music of a plurality of frequency ranges through the comb filter group, the digital music component of each frequency range is connected to the waveform correction module as input then, this module is carried out the extraction of ripple bag, Difference Calculation and half-wave correct operation to signal sequence, the digital music after obtaining proofreading and correct; Digital music after the correction extracts the input of parts as rhythm, handle through one group of comb filter, carry out the energy summation again, according to resonance principle, the frequency of the pairing comb filter of energy maximum is the beat cycle of original music, thereby tries to achieve beat information.
Described bandpass filter is six rank elliptic filters, and its digital music signal with input is decomposed into the digital music of the frequency range that contains 6 octaves commonly used.
Described waveform correction module, be with elliptic filter output square as importing, and store in the state array, owe the sampling period (about 0.1 second) every one, state dope vector and half Hanning window are done the cyclic convolution computing, carry out waveform correction, then the digital music behind the output calibration (also being the ripple bag).Afterwards, the ripple bag is carried out difference, half-wave verification, as the input of next step resonant tank wave filter (comb filter).
Described rhythm extracts parts, adopts the beat of the resonance principle computational music of resonant tank.The resonant tank number is the delay period (the beat cycle with test music correspondence of minimum bpm) of ripple bag sample rate * maximum.Test music bpm scope is 60～180bpm in the experiment, and the beat cycle is 0.33～1s, and the delay period of these resonant tanks has covered the beat cycle of these music.All resonant tanks use half identical energy time.Ripple bag after the verification is input in the resonant tank wave filter and calculates.Every a frame time (desirable 0.15 second, must less than the minimum beat cycle), carry out the extraction of a resonant tank status information, calculate the energy of all resonant tanks, the delay period of the maximum is the cycle of music.
Described robot hearing subsystem mainly is made up of synchronously recording module, recording and processing.
Drive programming towards sound card in the recording module.The recording flow process is mainly divided following step: whether the test sound card operate as normal; Open sound card equipment, each parameter (sample frequency, channel number, quantization digit, coded system) of equipment is set by ioctl; From the dsp sound card equipment, read in the pcm data to application buffer; Closing device.
Recording and processing synchronization module: use DMA mechanism to carry out of data (the recording data A/D conversion after) transmission of UDA1341 sound card chip buffering area to internal memory.In application program, the recording part as an independent thread, is carried out synchronously with processing threads.For communicating by letter of record thread and processing threads, the definition shared buffer.So recording thread and processing threads have constituted the basic reader-person's of writing problem.For fear of reading while write the deadlock that may occur, adopt and writing preference strategy, also be processing threads after the recording thread starts, wait for a period of time (being advisable more than 50 milliseconds) carry out read operation again.
The dancing subsystem mainly is made up of action design module, motor unit storehouse, combination maneuver library, choreography module, wherein: analyze the rhythm characteristics and the emotion of different music, with music assorting, be the basic dance movement of the music layout unit of each class; Then the gesture data category of motor unit is stored as database; With the sequence of motor unit, its category is stored as database again as the combination action; According to the result of rhythm identification, robot uses the sequence of combination action to generate a cover dance movement that coincide with music rhythm by calculating.
Above-mentioned action design module is provided with all kinds of basic dance movement unit and has alternative, representativeness and completeness.Do not have the motor unit of repetition between all kinds of basic dance movements unit, the motor unit in each classification has been represented the rhythm characteristics and the emotion of such music, and robot can stably finish each motor unit.
Above-mentioned motor unit library module is provided with the parameter in each joint of robot and uses the program that generates action, and the gesture data category of motor unit is stored as database.
Combinations thereof maneuver library module, this module is by analyzing linking continuity and the aesthetic property between each motor unit, design the order of connection of each motor unit, from the action cell library, read corresponding gesture data, sequence as the combination action, its category is stored as database, and each classification designs the sequence of multiple combination action, guarantees the completeness of combination maneuver library.
Above-mentioned choreography module, rhythm value that obtains with the rhythm recognition subsystem and beat sequence information are as input, by the corresponding relation of built-in rhythm value with combination action sequence classification, obtain the classification of action sequence, make up at random according to certain rule in a plurality of action sequences from such then, the rule here is that two continuous actions can not repeat.Generate a cover dance movement that coincide with music rhythm.Different rhythm music, the dance movement difference of layout, the emotion of expression is also different; Same rhythm music, the dance movement difference of layout, but the emotion of expressing is identical.
The invention will be further described below in conjunction with accompanying drawing, but do not limit the present invention.
1. robot auditory system:
The robot auditory system realizes recording and storage, extraneous analog music signals is converted to digital music stores.
The implementation process of this part as shown in Figure 1.Microphone converts the music sound signal of outside the DATA0 pin reception data of the signal of telecommunication from UDA1341 to, becomes digital information through analog-to-digital conversion then, and is converted into pcm encoder, and these data are written in the buffering area of application program by DMA mechanism then.
Primary control program is set up a shared buffering area (buffer) in the internal memory of CPU, be used for depositing the recording data.This shared drive is visited by two Line Procedure Mutually-exclusive.The recording thread writes buffering area with data, and processing threads is sense data from buffering area.For fear of deadlock, 50 milliseconds of recording thread operations earlier, and then start processing threads read data from buffering area.
The main contents that relate in the realization comprise:
1) ARM recording playback sound: drive interface programming towards sound card;
2) recording is synchronous with processing: DMA mechanism, multi-thread mechanism, inter-thread communication, the reader-person's of writing problem, deadlock are avoided.
1.1ARM the recording playback sound is realized
1.1.1 recording hardware: adopt the wider UDA1341 sound card chip of application to record.This chip is supported PCM hardware encoding and playback and sound-recording function, can realize analog-to-digital conversion easily, obtains the PCM data.Its programmable gain amplifier (PGA) and digital Auto Gain controller (AGC) can be realized the automatic tuning to music.The also built-in simultaneously L3 EBI that is used for transmission of control signals, with convenient to the I/O audio signal bass and the control of volume etc.The interface function that these functions provide by routine call Linux is realized the support to the recording playback sound of particular demands.
1.1.2 software platform: it is the (SuSE) Linux OS of 2.6.13 that software platform is selected the kernel version for use.And customize at the ARM framework.Because (SuSE) Linux OS has been done good encapsulation to many bottom hardwares, and the upper strata call function directly is provided, and has greatly facilitated and quickened the realization of system.Simultaneously Linux series operating system has the platform compatibility, usually based on the X86 structure of ordinary PC and the program of LINUX operating system exploitation, can not make an amendment or only do minor modifications and just can test on the flush type LINUX platform based on ARM and pass through.What unique needs were done does a cross compile at the ARM hardware structure exactly, and downloads on the control panel and get final product.
1.1.3 sound card drives: the LINUX based on the motherboard customization of ARM selects for use the OSS sound card to drive.OSS (Open SoundSystem) is the unified COBBAIF of on the Linux platform, promptly as long as audio frequency is handled application program to be write according to the API of OSS, when being transplanted to other hardware platform, only needs to recompilate to get final product so.The major function of OSS be with sound card abstract be a unified equipment, make things convenient for the Linux programmer to use.In implementation procedure, trickle modification has been done in the general purpose O SS sound card driving of LINUX, so that the hardware requirement of the concrete ARM master control borad that support is adopted overcomes the recording playback sound UDA1341 is supported bad problem.
1.1.4 recording software flow process
The main flow process of software of recording is as follows:
Whether A test sound card equipment is normal;
B open sound card equipment (default directory for/dev/dsp), and relevant parameter is set;
C is provided with each parameter (sample frequency, channel number, quantization digit, coded system) of equipment by ioctl;
D reads in the pcm data to application buffer from the dsp sound card equipment;
The E closing device.
1.1.5 play the music software flow process:
The main flow process of playing music software is as follows:
A reads the header of wav file, and the each several part parameter is preserved;
B is provided with each parameter of equipment by ioctl;
C writes sound card equipment/dev/dsp with the PCM data;
The effect of playout software is the correctness and the recording effect of test recording data.This software comes out to allow people's ear hear the music of recording software typing, can more directly hear recording effect.
1.2. multithreading is realized recording and is handled synchronously:
The real-time of rhythm identification is the real key issue that realizes with music and dance of robot.In order to strengthen real-time, need reduce the time of entire process as far as possible.In native system, designed a kind of mechanism of multithreading, effectively realized recording and rhythm identification synchronous processing, process as shown in Figure 1, specific explanations is as follows:
1.2.1 the DMA mechanism of recording
At bottom, the IIS of embedded-type ARM computer (Inter-IC Sound) bus and audio control chip UDA1341 communicate.DATA0 pin from UDA1341 during recording receives data, and data transfer mode is a dma mode.(IIS is used for transmitting audio data, and IIC is used to control 1341 chips.)
The basic procedure of DMA transmission is: deposit the data after A/D changes in the buffering area, when its data reach maximum DMA counting, and verification is when errorless, and 1341 chips send interrupt signal, and operating system begins byte-by-byte and from the DMA buffering area data read in internal memory (or word).The process of reading in internal memory is: its first section byte copied to the memory address place, and the DMA counting deducts the byte number of transmission then; Memory address increases the byte number that transmits.This process repeats always, becomes 0 up to the DMA counting.This moment, 1341 chips sent interrupt signal, notifying operation system data end of transmission.
On macroscopic view, just the record flow process of data in the thread Recording Process of this process, the buffering area of the corresponding application program of memory headroom of DMA transmission.Also be that data in the buffering area can present DMA " characteristic ", at set intervals, have a batch data and write.
1.2.2 communicate by letter between recording thread and processing threads
The data that the recording thread adopts DMA mechanism 1341 chip A/D to be transformed and are encoded into PCM in 1.2.1 have been written in the buffering area of application program.In this step, need to solve the communication issue between recording thread and processing threads.
It is as follows that shared buffer is set up flow process:
1) definition shared buffer buffer in application program;
2) processing threads is monitored shared buffer, must know in read data promptly whether data have write buffering area.In the starting stage, empty buffering area earlier, specific practice is that all data of buffering area are set to be a particular values of recording PCM data, identification data does not read in as yet; When data are read in, will revise initialized special data, thereby whether processing threads can be modified judgment data whether to write by the test initialization data.
3) processing threads reads the recording data from buffer, begins to handle.
The main difficulty that this process faced is: when two thread synchronization are carried out, because speed difference separately, the result of randomness can occur.This solves in the next section.
1.2.3 the reader-person's of writing problem
Recording thread and processing threads have been set up a shared buffering area buffer in 1.2.2, are used for depositing the recording data, and by two Line Procedure Mutually-exclusive visits.This has just formed the basic reader-person's of writing problem.The reader here is a processing threads, and the person of writing is the recording thread.The reader writes buffering area with data, and the person of writing is sense data from buffering area.
General processing procedure is in the reader-person's of writing problem: when writing, other persons of writing and all readers must wait for as the person of writing; When reading, other readers can enter as the reader, but the person of writing must wait for.
1.2.4 deadlock is avoided
May be because the following situation of appearance cause deadlock in the communication pattern that 1.2.2 sets up: the reader know not have in the buffering area data to enter, and person to be written such as continues and writes data owing to obtaining data; Because the reader occupies resource, the person of writing can't write, and also enters in the unlimited wait, and deadlock takes place at last but meanwhile.Certainly deadlock is accidental (because the speed of thread operation is unpredictable), and can different results occur because of the difference of the hardware platform of operation.
The method that we avoid above deadlock is at first to allow the person of writing move one earlier regularly ask, allows the reader begin read data from buffering area more then.
Specific implementation process is for allowing recording thread operation a period of time t earlier, obtain by experiment t be set to 50 milliseconds proper.
2. rhythm identification:
In music, the length of sound, power in an organized way are called rhythm.Rhythm is music tissue in time.In music, the appearance that clocklike circulates of the strong beat of identical duration and weak beat is called beat.Rhythm and beat are always simultaneous.
Basic element as music---rhythm is one of the most important information that can catch immediately in the music.The present invention is based on the rhythm identification of real-time multi part note signal, realized the algorithm of the computer rhythm identification of anthropomorphic dummy's ear perception.By a large amount of experiment confirms the high efficiency and the feasibility of rhythm and beat recognizer.
2.1 psychoacoustic model
Rhythm recognizer among the present invention is based on the method for a kind of musical features identification of psychoacoustic model proposition.Psychologic acoustics research be to concern between the sound and the sense of hearing that causes by sound.In order to reach best auditory effect, can utilize this relation to set up psychoacoustic model, its basic thought is compressed information, and distortion is not perceiveed out as far as possible.People's ear has masking effect usually, and promptly more weak sound can be covered by simultaneous stronger sound, makes people's ear can't perceive more weak sound.
In the audio compression coding, utilize masking effect, just the method that can be equipped with different quantizing bit numbers by the component of signal branch to the different frequency place is controlled quantizing noise, make the energy of noise be lower than masking threshold, thereby make the existence of the imperceptible quantizing process of people's ear.MPEG layer2,3 and AAC standard and AC-3 standard in all adopted psychoacoustic model, in present high quality audio compressed encoding, psychoacoustic model is an algorithm model the most effective.
Shown in Figure 2 is the psychologic acoustics experiment of energy perception music rhythm after handling.Specific practice is to use identical wave filter respectively to music and noise signal filtering, forms six sub-bands; Six sub-bands of music signal extract the efferent echo bag respectively, with six the noise signal word wave bands addition after itself and the filtering, output audio file.Experimental result proves that to the music after handling with the method, the beat of people's ear perception is consistent with the beat of original music.Illustrate that music through this method processing, still can reflect the rhythm and the beat of original music, it is more obvious to handle the back information characteristics simultaneously.
Shown in Figure 3 be can not the perception music rhythm after handling the psychologic acoustics experiment.Experiment flow forms six sub-bands for using wave filter to carry out filtering to music signal; Extract the efferent echo bag respectively; The addition of gained information; With result and noise signal stack, output audio file.Audio file and original audio file to being generated carried out perception by people's ear then, and experimental result proves that the singer or composer's ear after the method is handled can not perceived tempo and beat.
The psychologic acoustics experiment of rhythm perception shows: the ripple bag of audio signal has comprised the important information of rhythm and beat.Signal is divided into a plurality of sub-bands, has kept the ripple bag independence of each sub-band, this accurate identification for rhythm and beat is quite important, and this is tested to the rhythm recognizer theoretical foundation is provided simultaneously.
2.2 beat recognizer design:, proposed to extract the rhythm of music and the algorithm of beat feature based on the psychologic acoustics experiment.The frequency of audio signal has reflected the rhythm of music, and phase place has reflected the concrete moment that each beat takes place.Rhythm and the beat feature of extracting music promptly are that note signal is carried out frequency analysis and phase analysis.
2.2.1 algorithm flow: the overall procedure of beat recognizer by the audio signal of recording input, at first uses bank of filters to be divided into six sub-bands as shown in Figure 4.Then, calculate the ripple bag of each sub-band, proofread and correct through difference and half-wave again.At last, each ripple bag is by a bank of filters that 100 tuning resonant tanks are arranged.Utilize the energy response of resonant tank bank of filters, carry out the phase place locking on the one hand, with the concrete moment (beat point) of prediction beat generation; Be complementary with decision rhythm (beat number) by the frequency of resonant tank and the periodic modulation frequency of ripple bag on the other hand.
2.2.2 filtering: in the extraneous music of typing, include interfere informations such as noise, and Algorithm Analysis is the characteristic of the rhythm of music, can be by the frequency filtering of filtering operation with the pairing specific band of interfere information in the signal.It is important measures that suppress and prevent to disturb.
Wave filter can be handled the harmonic components of CF in the signal.What native system used is six rank elliptic filters, and its amplitude-frequency response chi square function is:
|H(jΩ)|
2＝1/(1+ε
2Un
2(Ω/Ωc)) (2.2.2-1)
Wherein, ε is the positive number less than 1, is the parameter relevant with passband ripple; The ε value is big more, and the ripple of passband is big more.Un is JacobianN rank elliptic functions, and Ω c is the passband effective cut off frequency.
What Fig. 5 showed is six rank elliptic filter characteristics.Each wave band all has a cut-off frequency and contains the scope of an octave, and sub-band 1 is a low pass filter that cut-off frequency is 200hz, and sub-band 3 and sub-band 5 are the logical part of band, the frequency range of its wave filter is respectively [400,800], [1600,3200].For fear of repetition, in same figure, show a plurality of wave filters, other sub-bands do not draw.The frequency range of other sub-band 2,4,6 is respectively [200,400], and [800,1600] and cut-off frequency are the high-pass filtering of 3200HZ.
2.2.3 the ripple bag extracts: from the Bo Baotu of audio signal, can clearly see the concrete moment that beat takes place, therefore, the Bo Baotu that extracts audio signal discerns for rhythm and is significant.
Signal by verification and smoothing processing, extracts the ripple bag of each wave band through after the filtering respectively.Half Hanning window that the result of verification wave filter output and window length are 0.2 second is done convolution.
The windowing process is actually time slice of intercepting from audio signal.Signal cutout will cause the leakage of spectrum energy, reveal in order to reduce spectrum energy, adopt half Hanning window.
Window function is as follows:
The half Hanning window functional image that Fig. 6 shows.Reveal in order to reduce spectrum energy, secondary lobe is low more, and the effect of eliminating High-frequency Interference and leakage energy is good more.Increase window length and can improve spectral resolution; The window that use is successively decreased can reduce secondary lobe, reveals thereby reduce energy.Half Hanning window just in time meets these characteristics, and the minimizing energy of trying one's best is revealed.
2.2.4 half-wave verification: carry out first-order difference after extracting the ripple bag, do the half-wave verification then; Signal through verification is done periodic modulation again.The main effect of this process is the stress in the detection signal, and beat point is amplified, and becomes more clear.Show among Fig. 7 be the waveform that sends of beat beating device extract by the ripple bag and the half-wave verification after the result.
2.2.5 rhythm analysis:
According to resonance principle: physical system is tending towards absorbing more multipotency from surrounding environment under its natural vibration frequency (being resonant frequency).Use the comb filter group, when the frequency of audio signal near or when equaling frequency some in the comb filter group, energy increases, promptly amplitude is strengthened.In all wave filters, select the wave filter of energy value maximum, promptly can be used as the rhythm frequency of music.In order to make each comb filter have half equal energy time, to the given different gain of different wave filters.The low frequency part gain is big, and the HFS gain is little.
Fig. 8 is rhythm analysis process figure.Each sub-band all has the comb filter that one group of size is 150, frequency is different.
2.2.6 phase place locking
Fig. 9 is the phase calculation flow chart.Find after the comb filter with ceiling capacity, analyze the deferred message of this wave filter inside.If establish the delay period of n=ripple bag sample rate * ceiling capacity resonant tank, n has just reacted the cycle size of audio signal on algorithm so, is the equal of the phase angle of 2*pi.Necessarily there is certain single order k to react the moment (beat must have and only have a beat point in the cycle) that beat point occurs between 1～n.Determine k, just determined the position of beat point.With the deferred message addition of six wave bands in every rank, have peaked exponent number and promptly reflected phase place.In fact, ask beat to ask the clock pulse cycle exactly, asking phase place is exactly the moment point of asking pulse to occur in one-period.
After obtaining phase place, can calculate the concrete moment that next beat takes place.
Tnext=Tnow+ phase place/ripple bag sample rate (2.2.6-1)
In order accurately to locate the concrete moment that each beat takes place, every 0.05s carries out a phase place locking.Identical or within certain error range when the beat position of continuous several frames prediction, think that then this predicts the outcome and has higher confidence level.Reference data as the final prediction of beat.
2.3 algorithm optimization: the purpose of algorithm optimization is to shorten the music recognition time as far as possible, and the present invention is in the measure of having taked two aspects aspect the preliminary treatment of algorithm for this reason.Specific as follows:
The first adopts static number group coding mode stored audio signal.With the minimizing data volume, and shorten operation time.In order to reduce data volume, positional information and data message be incorporated in the double number store.Coding method is as follows:
The individual bit representation numerical symbol of the integer part of double-precision number (0 for just, and 2 for negative, and 1 is numerical value 1), the position of all the other bit representation data of integer part, fractional part is represented numerical value.
Example is as follows:
1. positive number: [50] [186]=0.1256 are stored as 501860.1256
2. negative: [50] [186]=-0.1257 are stored as 501862.1256
3.1:[50] [186]=1, be stored as 501861.0000
Can utilize corresponding algorithm to carry out Code And Decode then handles.
It two is suitably to reduce ripple bag sample rate.Theoretically, ripple bag sample rate is high more, and resolution ratio is also high more, thereby the result of beat identification is good more.But, experimental results show that suitable reduction sample rate, the result of beat identification is constant.The speed of service can be doubled by this way.
3. dancing strategy
Calculate after the rhythm of music, with rhythm---the rhythm pattern in the dancing storehouse mates, and obtains the optimum Match scheme, determines the dance movement that robot will perform; According to the beat prediction result, when the decision robot begins to dance again.
The purpose of robot dancing is will be according to the result of rhythm identification, and dancing is finished in the dance movement that autonomous combination is basic.
The basic way of combined type choreography is: music is classified according to speed (per second beat number), the corresponding basic dance movement collection of each class according to the result of beat identification, is answered with corresponding basic dance movement set pair, carry out permutation and combination, form various dancing; When beat changes, select the basic dance movement collection of new correspondence.
At the music of per minute beat number (bpm) from 60 to 180, behavior aggregate is carried out following grouping.
One, bpm=[150,180]
Musical feature: rhythm is ultrafast, majority is rap, the activity that each joint of robot can be independent, many joints based on head, hand and pin.
Elemental motion: shake the head, advance fast, retreat, clap hands, on tiptoe, left and right sides translation fast, mechanical type right-hand man alternately swing up and down, horizontally rotate arm.
Two, bpm=[130,149]
Musical feature: rhythm is comparatively cheerful and light-hearted, is fit to the gymnastics class, the action of traditional dancing class.
Elemental motion: gymnastic exhibition, japanese traditional dance movement.
Three, bpm=[120,129]:
Musical feature: speed is suitable machine people's activity just in time, moves comparatively flexibly, can represent the advantage in each joint of robot heartily, and embody the emotion of music by action.
Elemental motion: precedingly turn over, after turn over, agitate arm, big word to contact to earth, big word is squatted down, moving knee, Thousands Hands Guanyin.
Four, bpm=[100,119]
Musical feature: comparatively releive, can do a little highly difficult actions, make it the time of showing heartily.
Elemental motion: a great hawk spreads its wings, handstand, split, the rotation of single leg.
Five, bpm=[60,99]
Musical feature: very slow, robot can set foot on the beat perform their routines.
Elemental motion: arm bending, arm swing, shank are flexible, left and right sides translation, front and back walking are changed formation.
Claims (10)
1. anthropomorphic robot of dancing with rhythm, the brain of this robot is provided with music recognition system, it is characterized in that described music recognition system has the embedded computer system of universal musical rhythm recognition capability, this system mainly is made up of hearing subsystem, rhythm recognition subsystem and dancing movement combination subsystem, wherein: hearing subsystem obtains the music of external played by sensor, and being converted to digital music signal, this digital music signal is as the input of rhythm recognition subsystem; The rhythm recognition subsystem is handled digital music signal, obtains the rhythm and the beat feature of music, comprises per minute beat number and beat time started; The rhythm characteristic that dancing movement combination subsystem obtains according to the rhythm recognition subsystem independently carries out the dancing combination; These three subsystems make this robot independently dance with rhythm with the form parallel processing of streamline.
2. anthropomorphic robot according to claim 1, it is characterized in that the rhythm recognition subsystem mainly extracts parts by bandpass filter group, waveform correction module, rhythm and forms, wherein: the digital music signal of input is decomposed into the digital music of a plurality of frequency ranges through the comb filter group, the digital music component of each frequency range is connected to the waveform correction module as input then, this module is carried out the extraction of ripple bag, Difference Calculation and half-wave correct operation to signal sequence, the digital music after obtaining proofreading and correct; Digital music after the correction extracts the input of parts as rhythm, handle through one group of comb filter group, carry out the energy summation again, according to resonance principle, the frequency of the pairing comb filter of energy maximum is the beat cycle of original music, thereby tries to achieve beat information.
3. anthropomorphic robot according to claim 2 is characterized in that: bandpass filter is six rank elliptic filters, and its digital music signal with input is decomposed into the digital music of the frequency range that contains 6 octaves commonly used.
4. anthropomorphic robot according to claim 2, it is characterized in that: the waveform correction module be with bandpass filter output square as importing, and store in the state array, every a cycle of owing to sample, state dope vector and half Hanning window are done the cyclic convolution computing, carry out waveform correction, output wave bag then, this ripple bag is the digital music after proofreading and correct, afterwards, the ripple bag is carried out difference, half-wave verification, and as the input of next step resonant tank wave filter, this wave filter is a comb filter.
5. according to claim 2 or 3 described anthropomorphic robots, it is characterized in that: rhythm extracts the beat that parts adopt the resonance principle computational music of resonant tank, the resonant tank number is the delay period of ripple bag sample rate * maximum, and this delay period has the beat cycle of the test music correspondence of minimum bpm; Test music bpm scope is 60～180bpm in the experiment, and the beat cycle is 0.33～1s, and the delay period of these resonant tanks has covered the beat cycle of these music, and all resonant tanks use half identical energy time; Ripple bag after the verification is input in the resonant tank wave filter and calculates, and carries out once the energy of all resonant tanks is calculated in the extraction of resonant tank status information respectively every a frame time, and the delay period of the maximum is the cycle of music.
6. anthropomorphic robot according to claim 5 is characterized in that a described frame time must be less than the minimum beat cycle.
7. anthropomorphic robot according to claim 1, it is characterized in that the robot hearing subsystem mainly by recording module, the recording with the processing synchronization module form, wherein:
Drive programming towards sound card in the recording module, the recording flow process mainly comprises following step: whether the test sound card operate as normal; Open sound card equipment, each parameter of equipment is set, these parameters are sample frequency, channel number, quantization digit, coded system; From sound card equipment, read in the pcm coded data to application buffer; Closing device;
Recording and processing synchronization module: use DMA mechanism that the transfer of data of UDA1341 sound card chip buffering area is arrived the embedded computer internal memory, these data are the recording data after the A/D conversion; In application program, the recording part as an independent thread, is carried out synchronously with processing threads; Define a shared buffer, realization recording thread is communicated by letter with processing threads.
8. anthropomorphic robot according to claim 7 is characterized in that: for fear of reading while write the deadlock that shared buffer may occur, adopt and write look ahead strategy, promptly processing threads carries out read operation again after the recording thread starts 50 milliseconds.
9. anthropomorphic robot according to claim 1, it is characterized in that dancing movement combination subsystem adopts following method to carry out the dancing combination: rhythm characteristics and the emotion of analyzing different music, with music assorting, be the basic dance movement of the music layout unit of each class; Then the gesture data category of motor unit is stored as database; With the sequence of motor unit, its category is stored as database again as the combination action; According to the result of rhythm identification, robot uses the sequence of combination action to generate a whole set of dance movement that coincide with music rhythm by calculating.
10. anthropomorphic robot according to claim 9 is characterized in that dancing movement combination subsystem mainly is made up of action design module, motor unit storehouse, combination maneuver library, choreography module, wherein:
Action design module: be provided with all kinds of basic dance movement unit and have alternative, representativeness and completeness, the motor unit that does not have repetition between all kinds of basic dance movement unit, the rhythm characteristics and the emotion of such music represented in basic dance movement unit in each classification, and robot can stably finish each motor unit;
Motor unit library module: be provided with the parameter in each joint of robot and use the program that generates action, the gesture data category of above-mentioned motor unit is stored as database;
Combination maneuver library module: this module is by analyzing linking continuity and the aesthetic property between each motor unit, design the order of connection of each motor unit, from the action cell library, read corresponding gesture data, sequence as the combination action, its category is stored as database, each classification designs the sequence of multiple combination action, guarantees the completeness of combination maneuver library;
The choreography module: this module is passed through to calculate after the rhythm recognition subsystem obtains information, uses the sequence of the various combinations actions of same classification to generate a cover dance movement identical with music; Different rhythm music, the dance movement difference of layout, the emotion of expressing when motor unit does not belong to same classification is all different; Same rhythm music, the dance movement difference of layout, the emotion of expressing when motor unit belongs to same classification is all identical.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100616262A CN101524594B (en) | 2009-04-17 | 2009-04-17 | Anthropomorphic robot autonomously dancing along with rhythm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100616262A CN101524594B (en) | 2009-04-17 | 2009-04-17 | Anthropomorphic robot autonomously dancing along with rhythm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101524594A true CN101524594A (en) | 2009-09-09 |
| CN101524594B CN101524594B (en) | 2012-06-27 |
Family
ID=41092760
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009100616262A Expired - Fee Related CN101524594B (en) | 2009-04-17 | 2009-04-17 | Anthropomorphic robot autonomously dancing along with rhythm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101524594B (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102116672A (en) * | 2009-12-31 | 2011-07-06 | 陈新伟 | Rhythm sensing method, device and system |
| CN101693371B (en) * | 2009-09-30 | 2011-08-24 | 深圳先进技术研究院 | Robot capable of dancing by following music beats |
| CN102866645A (en) * | 2012-09-20 | 2013-01-09 | 胡云潇 | Movable furniture capable of controlling beat action based on music characteristic and controlling method thereof |
| CN104587671A (en) * | 2014-11-25 | 2015-05-06 | 百度在线网络技术（北京）有限公司 | Intelligent doll and action control method for intelligent doll |
| CN105701196A (en) * | 2016-01-11 | 2016-06-22 | 北京光年无限科技有限公司 | Intelligent robot oriented audio processing method and intelligent robot |
| CN105773612A (en) * | 2016-03-28 | 2016-07-20 | 深圳前海勇艺达机器人有限公司 | System and method for controlling dance of robot |
| CN105881535A (en) * | 2015-02-13 | 2016-08-24 | 鸿富锦精密工业（深圳）有限公司 | Robot capable of dancing with musical tempo |
| CN106041964A (en) * | 2016-08-16 | 2016-10-26 | 管存忠 | Dancing robot |
| CN106054756A (en) * | 2016-08-18 | 2016-10-26 | 成都英博格科技有限公司 | Robot control method and device |
| CN106096720A (en) * | 2016-06-12 | 2016-11-09 | 杭州如雷科技有限公司 | A kind of method that dance movement is automatically synthesized |
| CN106217384A (en) * | 2016-07-14 | 2016-12-14 | 歌尔股份有限公司 | A kind of method and apparatus controlling service robot dancing |
| CN106598062A (en) * | 2016-07-29 | 2017-04-26 | 深圳曼塔智能科技有限公司 | Flight motion control method and device for unmanned plane |
| CN107071652A (en) * | 2016-12-27 | 2017-08-18 | 深圳市优必选科技有限公司 | A kind of service robot |
| CN108393906A (en) * | 2018-03-01 | 2018-08-14 | 深圳普得技术有限公司 | A kind of control robot realizes mechanical regular movements method and robot |
| CN108527376A (en) * | 2018-02-27 | 2018-09-14 | 深圳狗尾草智能科技有限公司 | Control method, apparatus, equipment and the medium of robot dance movement |
| CN108744542A (en) * | 2018-06-08 | 2018-11-06 | 武汉蛋玩科技有限公司 | A kind of Robot dancing movements design method and a kind of robot |
| CN108989706A (en) * | 2017-06-02 | 2018-12-11 | 北京字节跳动网络技术有限公司 | The method and device of special efficacy is generated based on music rhythm |
| CN109176541A (en) * | 2018-09-06 | 2019-01-11 | 南京阿凡达机器人科技有限公司 | A kind of method, equipment and storage medium realizing robot and dancing |
| CN110297432A (en) * | 2018-03-21 | 2019-10-01 | 北京猎户星空科技有限公司 | Robot motion sequence generating method, device and system |
| CN110955786A (en) * | 2019-11-29 | 2020-04-03 | 网易（杭州）网络有限公司 | Dance action data generation method and device |
| CN111113418A (en) * | 2019-12-25 | 2020-05-08 | 深圳市优必选科技股份有限公司 | Dance control method and device for robot |
| CN112365568A (en) * | 2020-11-06 | 2021-02-12 | 广州小鹏汽车科技有限公司 | Audio processing method and device, electronic equipment and storage medium |
| CN112402952A (en) * | 2019-08-23 | 2021-02-26 | 福建凯米网络科技有限公司 | Interactive method and terminal based on audio and virtual image |
-
2009
- 2009-04-17 CN CN2009100616262A patent/CN101524594B/en not_active Expired - Fee Related
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101693371B (en) * | 2009-09-30 | 2011-08-24 | 深圳先进技术研究院 | Robot capable of dancing by following music beats |
| CN102116672A (en) * | 2009-12-31 | 2011-07-06 | 陈新伟 | Rhythm sensing method, device and system |
| CN102866645A (en) * | 2012-09-20 | 2013-01-09 | 胡云潇 | Movable furniture capable of controlling beat action based on music characteristic and controlling method thereof |
| CN104587671A (en) * | 2014-11-25 | 2015-05-06 | 百度在线网络技术（北京）有限公司 | Intelligent doll and action control method for intelligent doll |
| CN104587671B (en) * | 2014-11-25 | 2017-01-25 | 百度在线网络技术（北京）有限公司 | Intelligent doll and action control method for intelligent doll |
| CN105881535A (en) * | 2015-02-13 | 2016-08-24 | 鸿富锦精密工业（深圳）有限公司 | Robot capable of dancing with musical tempo |
| CN105701196A (en) * | 2016-01-11 | 2016-06-22 | 北京光年无限科技有限公司 | Intelligent robot oriented audio processing method and intelligent robot |
| CN105773612B (en) * | 2016-03-28 | 2018-04-17 | 深圳前海勇艺达机器人有限公司 | The control system and its method of Robot dancing |
| CN105773612A (en) * | 2016-03-28 | 2016-07-20 | 深圳前海勇艺达机器人有限公司 | System and method for controlling dance of robot |
| CN106096720A (en) * | 2016-06-12 | 2016-11-09 | 杭州如雷科技有限公司 | A kind of method that dance movement is automatically synthesized |
| US10828784B2 (en) | 2016-07-14 | 2020-11-10 | Goertek Inc. | Method and apparatus for controlling dancing of service robot |
| CN106217384A (en) * | 2016-07-14 | 2016-12-14 | 歌尔股份有限公司 | A kind of method and apparatus controlling service robot dancing |
| CN106217384B (en) * | 2016-07-14 | 2019-03-15 | 歌尔股份有限公司 | A kind of method and apparatus that control service robot is danced |
| CN106598062A (en) * | 2016-07-29 | 2017-04-26 | 深圳曼塔智能科技有限公司 | Flight motion control method and device for unmanned plane |
| CN106041964A (en) * | 2016-08-16 | 2016-10-26 | 管存忠 | Dancing robot |
| CN106054756A (en) * | 2016-08-18 | 2016-10-26 | 成都英博格科技有限公司 | Robot control method and device |
| CN107071652A (en) * | 2016-12-27 | 2017-08-18 | 深圳市优必选科技有限公司 | A kind of service robot |
| CN107071652B (en) * | 2016-12-27 | 2023-08-04 | 深圳市优必选科技有限公司 | Service robot |
| CN108989706A (en) * | 2017-06-02 | 2018-12-11 | 北京字节跳动网络技术有限公司 | The method and device of special efficacy is generated based on music rhythm |
| CN108527376A (en) * | 2018-02-27 | 2018-09-14 | 深圳狗尾草智能科技有限公司 | Control method, apparatus, equipment and the medium of robot dance movement |
| CN108393906B (en) * | 2018-03-01 | 2021-04-27 | 深圳小墨智能科技有限公司 | Method for controlling robot to realize mechanical rhythm and robot |
| CN108393906A (en) * | 2018-03-01 | 2018-08-14 | 深圳普得技术有限公司 | A kind of control robot realizes mechanical regular movements method and robot |
| CN110297432A (en) * | 2018-03-21 | 2019-10-01 | 北京猎户星空科技有限公司 | Robot motion sequence generating method, device and system |
| CN110297432B (en) * | 2018-03-21 | 2021-07-02 | 北京猎户星空科技有限公司 | Robot action sequence generation method, device and system |
| CN108744542A (en) * | 2018-06-08 | 2018-11-06 | 武汉蛋玩科技有限公司 | A kind of Robot dancing movements design method and a kind of robot |
| CN109176541A (en) * | 2018-09-06 | 2019-01-11 | 南京阿凡达机器人科技有限公司 | A kind of method, equipment and storage medium realizing robot and dancing |
| CN112402952A (en) * | 2019-08-23 | 2021-02-26 | 福建凯米网络科技有限公司 | Interactive method and terminal based on audio and virtual image |
| CN110955786A (en) * | 2019-11-29 | 2020-04-03 | 网易（杭州）网络有限公司 | Dance action data generation method and device |
| CN110955786B (en) * | 2019-11-29 | 2023-10-27 | 网易（杭州）网络有限公司 | Dance action data generation method and device |
| CN111113418A (en) * | 2019-12-25 | 2020-05-08 | 深圳市优必选科技股份有限公司 | Dance control method and device for robot |
| CN112365568A (en) * | 2020-11-06 | 2021-02-12 | 广州小鹏汽车科技有限公司 | Audio processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101524594B (en) | 2012-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101524594B (en) | Anthropomorphic robot autonomously dancing along with rhythm | |
| CN104834642B (en) | Change the method, device and equipment of music deduction style | |
| JP4430368B2 (en) | Method and apparatus for analyzing gestures made in free space | |
| CN101315770B (en) | System on speech recognition piece and voice recognition method using the same | |
| CN108717852B (en) | Intelligent robot semantic interaction system and method based on white light communication and brain-like cognition | |
| US9558727B2 (en) | Performance method of electronic musical instrument and music | |
| CN104035556A (en) | Automatic Fitting Of Haptic Effects | |
| CN105741639B (en) | A kind of micro- sense palm musical instrument for simulating bowstring kind musical instrument | |
| CN105956153B (en) | The search platform of music file is obtained at internet | |
| CN103189913A (en) | Method, apparatus and machine-readable storage medium for decomposing a multichannel audio signal | |
| US20120062718A1 (en) | Device and method for interpreting musical gestures | |
| CN112365882A (en) | Speech synthesis method, model training method, device, equipment and storage medium | |
| CN112184859B (en) | End-to-end virtual object animation generation method and device, storage medium and terminal | |
| CN110136689A (en) | Song synthetic method, device and storage medium based on transfer learning | |
| CN116394277B (en) | Robot is played to imitative people piano | |
| CN108829253B (en) | Simulated music command playing method and device | |
| CN110427459A (en) | Visualized generation method, system and the platform of speech recognition network | |
| CN104318950A (en) | Information processing method and electronic equipment | |
| CN113539232A (en) | Muslim class voice data set-based voice synthesis method | |
| CN113423005A (en) | Motion-driven intelligent music generation method and system | |
| Liu et al. | Design and implementation of human-computer interaction intelligent system based on speech control | |
| CN109637509A (en) | A kind of music automatic generation method, device and computer readable storage medium | |
| CN2862265Y (en) | Audio control MP3 player | |
| CN101789238A (en) | Music rhythm extracting system based on MCU hardware platform and method thereof | |
| CN1640522A (en) | Toy sound-control device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120627 Termination date: 20130417 |