CN110764520B - Aircraft control method, aircraft control device, aircraft and storage medium - Google Patents

Aircraft control method, aircraft control device, aircraft and storage medium Download PDF

Info

Publication number
CN110764520B
CN110764520B CN201810842962.XA CN201810842962A CN110764520B CN 110764520 B CN110764520 B CN 110764520B CN 201810842962 A CN201810842962 A CN 201810842962A CN 110764520 B CN110764520 B CN 110764520B
Authority
CN
China
Prior art keywords
aircraft
voice
determining
sound source
audio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810842962.XA
Other languages
Chinese (zh)
Other versions
CN110764520A (en
Inventor
钱能锋
陈扬坤
陈展
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201810842962.XA priority Critical patent/CN110764520B/en
Publication of CN110764520A publication Critical patent/CN110764520A/en
Application granted granted Critical
Publication of CN110764520B publication Critical patent/CN110764520B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/08Control of attitude, i.e. control of roll, pitch, or yaw
    • G05D1/0808Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10Simultaneous control of position or course in three dimensions
    • G05D1/101Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

The invention discloses an aircraft control method, an aircraft control device, an aircraft and a storage medium, and belongs to the technical field of flight control. The method is applied to an aircraft, the aircraft comprises a voice acquisition array, the voice acquisition array comprises a plurality of voice acquisition units, and the method comprises the following steps: respectively collecting voice signals through a plurality of voice collecting units to obtain a plurality of paths of audio signals, wherein one voice collecting unit collects one path of audio signals; determining the sound source direction of the voice signal according to the multi-channel audio signal and the position of each voice acquisition unit in the aircraft; according to the sound source direction, carrying out wave beam forming on the multi-channel audio signals to obtain a voice instruction; and converting the voice command into a control command, and controlling the aircraft according to the control command. According to the invention, a user can control the aircraft directly through voice without a control device, so that the practicability of the aircraft is improved.

Description

Aircraft control method, aircraft control device, aircraft and storage medium
Technical Field
The invention relates to the technical field of flight control, in particular to an aircraft control method, an aircraft control device, an aircraft and a storage medium.
Background
Along with the progress of flight control technology, the application of unmanned aerial vehicle is more and more extensive, and at present, unmanned aerial vehicle not only uses in the military affairs, also begins to use in commercial or civilian use. For example, the user can shoot by the drone to record nice scenery or the like. When using the drone, the user needs to send control instructions to the aircraft to control the drone to accomplish the tasks specified by the user. For example, when shooting with an unmanned aerial vehicle, a user needs to control a camera on the unmanned aerial vehicle to start shooting, stop shooting, or control a shooting angle of the camera.
At present, a user sends a control command to an aircraft through a control device such as a control station or a remote controller on the ground. And the unmanned aerial vehicle receives the control instruction sent by the control equipment and controls the unmanned aerial vehicle according to the control instruction.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems:
need dispose a controlgear for unmanned aerial vehicle among the above-mentioned technique, just can realize the control to unmanned aerial vehicle through this controlgear, lead to the practicality poor.
Disclosure of Invention
The invention provides an aircraft control method, an aircraft control device, an aircraft and a storage medium, which can solve the problem of poor practicability in the prior art. The technical scheme is as follows:
in one aspect, the present invention provides an aircraft control method, which is applied in an aircraft, where the aircraft includes a voice acquisition array, the voice acquisition array includes a plurality of voice acquisition units, and the method includes:
respectively collecting voice signals through the plurality of voice collecting units to obtain a plurality of paths of audio signals, wherein one voice collecting unit collects one path of audio signals;
determining the sound source direction of the voice signals according to the multi-channel audio signals and the position of each voice acquisition unit in the aircraft;
according to the sound source direction, carrying out wave beam forming on the multi-channel audio signals to obtain a voice instruction;
and converting the voice command into a control command, and controlling the aircraft according to the control command.
In one possible implementation manner, the determining the sound source direction of the voice signal according to the multiple audio signals and the position of each voice acquisition unit in the aircraft includes:
determining a unit vector function, wherein the unit vector function is a function comprising an azimuth angle parameter and a pitch angle parameter;
determining the theoretical time difference of audio signals acquired by any two voice acquisition units according to the unit vector function and the position of each voice acquisition unit in the aircraft;
determining the actual time difference of audio signals acquired by any two voice acquisition units;
and when the theoretical time difference and the actual time difference of the audio signals collected by any two voice collecting units are the same, determining the value of the azimuth angle parameter and the value of the pitch angle in the unit vector function to obtain the azimuth angle and the pitch angle of the sound source direction.
In one possible implementation manner, the determining a time difference between the speech signals acquired by any two speech acquisition units includes:
for a first audio signal and a second audio signal, determining a cross-correlation function between the first audio signal and the second audio signal;
determining a time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes a maximum value as a time difference between the first audio signal and the second audio signal.
In a possible implementation manner, the beamforming the multiple audio signals according to the sound source direction to obtain a voice command includes:
determining the audio component of each audio signal in the sound source direction according to the sound source direction to obtain multiple paths of audio components;
and carrying out beam forming on the multi-channel audio frequency components to obtain the voice command.
In a possible implementation manner, the determining, according to the sound source direction, an audio component of each audio signal in the sound source direction to obtain multiple audio components includes:
for any audio signal, converting the audio signal into a time-frequency signal;
determining a beam forming coefficient of each frequency point in the time-frequency signal in the direction of the sound source;
and multiplying the frequency spectrum value of each frequency point of any path of signal by a corresponding beam forming coefficient to obtain the audio frequency component of any path of signal in the sound source direction.
In one possible implementation, before converting the voice command into a control command, the method further includes:
determining a noise reduction method corresponding to the noise type according to the noise type of the aircraft;
and carrying out noise reduction processing on the voice instruction by the noise reduction method.
In one possible implementation manner, before determining the sound source direction of the voice signal according to the multiple audio signals and the position of each voice acquisition unit in the aircraft, the method further includes:
determining reference tone information of a user sending the voice signal according to the multi-channel audio signal;
determining whether the user has the authority to control the aircraft or not according to the reference tone information and reference tone information, wherein the reference tone information is the tone information of the user having the authority to control the aircraft;
and when the user has the right to control the aircraft, executing the step of determining the sound source direction of the voice signal according to the multi-channel audio signals and the position of each voice acquisition unit in the aircraft.
In another aspect, the present invention provides an aircraft control device, the device is applied in an aircraft, the aircraft includes a voice collecting array, the voice collecting array includes a plurality of voice collecting units, the device includes:
the acquisition module is used for acquiring voice signals through the plurality of voice acquisition units respectively to obtain a plurality of paths of audio signals, and one voice acquisition unit acquires one path of audio signals;
the determining module is used for determining the sound source direction of the voice signals according to the multi-channel audio signals and the position of each voice collecting unit in the aircraft;
the synthesis module is used for carrying out beam forming on the multi-channel audio signals according to the sound source direction to obtain a voice instruction;
and the control module is used for converting the voice command into a control command and controlling the aircraft according to the control command.
In one possible implementation manner, the determining module includes:
a first determining unit, configured to determine a unit vector function, where the unit vector function is a function including an azimuth angle parameter and a pitch angle parameter;
the second determining unit is used for determining the theoretical time difference of audio signals acquired by any two voice acquiring units according to the unit vector function and the position of each voice acquiring unit in the aircraft;
the third determining unit is used for determining the actual time difference of the audio signals acquired by any two voice acquiring units;
and the fourth determining unit is used for determining the value of the azimuth angle parameter and the value of the pitch angle in the unit vector function when the theoretical time difference and the actual time difference of the audio signals acquired by any two voice acquisition units are the same, so as to obtain the azimuth angle and the pitch angle of the sound source direction.
In a possible implementation manner, the third determining unit is further configured to determine, for a first audio signal and a second audio signal, a cross-correlation function between the first audio signal and the second audio signal; determining a time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes a maximum value as a time difference between the first audio signal and the second audio signal.
In one possible implementation, the synthesis module includes:
a fifth determining unit, configured to determine, according to the sound source direction, an audio component of each channel of audio signal in the sound source direction, so as to obtain multiple channels of audio components;
and the synthesis unit is used for carrying out beam synthesis on the multi-channel audio components to obtain the voice command.
In a possible implementation manner, the fifth determining unit is further configured to convert, for any one of the audio signals, the any one of the audio signals into a time-frequency signal; determining a beam forming coefficient of each frequency point in the time-frequency signal in the direction of the sound source; and multiplying the frequency spectrum value of each frequency point of any path of signal by a corresponding beam forming coefficient to obtain the audio component of any path of signal in the sound source direction.
In one possible implementation, the apparatus further includes:
the noise reduction module is used for determining a noise reduction method corresponding to the noise type according to the noise type of the aircraft; and carrying out noise reduction processing on the voice instruction by the noise reduction method.
In a possible implementation manner, the determining module is further configured to determine, according to the multiple audio signals, reference timbre information of a user who sends the speech signal; determining whether the user has the authority to control the aircraft or not according to the reference tone information and reference tone information, wherein the reference tone information is the tone information of the user having the authority to control the aircraft;
the determining module is further used for determining the sound source direction of the voice signal according to the multipath audio signals and the position of each voice collecting unit in the aircraft when the user has the right to control the aircraft.
In another aspect, the present invention provides an aircraft comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to carry out the operations performed in the aircraft control method described above.
In another aspect, the present invention provides a computer-readable storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by a processor to perform the operations performed in the above-described aircraft control method.
In the embodiment of the invention, the voice acquisition array is integrated in the aircraft, the voice instruction is acquired through the voice acquisition array, and the voice instruction is converted into the control instruction, so that a user can control the aircraft directly through voice without a control device, and the practicability of the aircraft is improved. And, the extra supporting controlgear of aircraft has been saved for unmanned aerial vehicle becomes more portable, and saved the cost.
Drawings
FIG. 1 is a schematic illustration of an implementation environment provided by an embodiment of the invention;
FIG. 2 is a flow chart of an aircraft control method provided by an embodiment of the invention;
FIG. 3 is a flow chart of an aircraft control method provided by an embodiment of the invention;
FIG. 4 is a schematic structural diagram of an aircraft control device provided by an embodiment of the invention;
fig. 5 is a schematic structural diagram of a determining module according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a synthesis module according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of another aircraft control device provided by an embodiment of the invention;
fig. 8 is a schematic structural diagram of an aircraft provided in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
An embodiment of the present disclosure provides a schematic diagram of an implementation environment, see fig. 1, including an aircraft and a user. The user can control the aircraft by voice. The aircraft may be an aerospace vehicle or a drone. The aircraft comprises a fuselage and a voice collection array, wherein the voice collection array is mounted on the fuselage. The voice acquisition array comprises a plurality of voice acquisition units, wherein each voice acquisition unit is dispersedly installed on the machine body, and each voice acquisition unit is not on the same straight line. In the embodiment of the present invention, the number of the voice collecting units included in the voice collecting array and the installation position of each voice collecting unit on the body are not particularly limited. For example, the voice capturing array includes 4 or 5 voice capturing units. And, each voice collecting unit may be uniformly installed on the body or non-uniformly installed on the body. The distance between two adjacent voice acquisition units is within the range of 5-10 cm, so that the accuracy of signal acquisition of each voice acquisition unit is improved.
It should be noted that, after each voice collecting unit is mounted on the fuselage, the coordinates of each voice collecting unit with respect to the rotation center of the aircraft are stored in the memory of the fuselage. The voice acquisition array is any array capable of acquiring voice signals. For example, the speech acquisition array may be a microphone array, and accordingly, the speech acquisition unit may be a microphone.
An embodiment of the present invention provides an aircraft control method, which is applied in an aircraft, and referring to fig. 2, the method includes:
step 201: the voice signals are respectively collected through a plurality of voice collecting units to obtain a plurality of paths of audio signals, and one voice collecting unit collects one path of audio signals.
Step 202: and determining the sound source direction of the voice signal according to the multi-channel audio signal and the position of each voice acquisition unit in the aircraft.
Step 203: and performing beam forming on the multi-channel audio signals according to the direction of the sound source to obtain a voice instruction.
Step 204: and converting the voice command into a control command, and controlling the aircraft according to the control command.
In one possible implementation manner, the determining the sound source direction of the voice signal according to the multiple audio signals and the position of each voice acquisition unit in the aircraft includes:
determining a unit vector function, wherein the unit vector function is a function comprising an azimuth angle parameter and a pitch angle parameter;
determining the theoretical time difference of audio signals acquired by any two voice acquisition units according to the unit vector function and the position of each voice acquisition unit in the aircraft;
determining the actual time difference of audio signals acquired by any two voice acquisition units;
and when the theoretical time difference and the actual time difference of the audio signals collected by any two voice collecting units are the same, determining the value of the azimuth angle parameter and the value of the pitch angle in the unit vector function to obtain the azimuth angle and the pitch angle of the sound source direction.
In one possible implementation manner, the determining a time difference between the speech signals acquired by any two speech acquisition units includes:
for a first audio signal and a second audio signal, determining a cross-correlation function between the first audio signal and the second audio signal;
the time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes the maximum value is determined as the time difference between the first audio signal and the second audio signal.
In a possible implementation manner, the beamforming the multiple audio signals according to the sound source direction to obtain a voice command includes:
determining the audio component of each path of audio signal in the sound source direction according to the sound source direction to obtain multiple paths of audio components;
and carrying out beam synthesis on the multi-channel audio components to obtain the voice command.
In one possible implementation manner, determining an audio component of each audio signal in the sound source direction according to the sound source direction to obtain multiple audio components, includes:
for any audio signal, converting the audio signal into a time-frequency signal;
determining a beam forming coefficient of each frequency point in the time-frequency signal in the direction of the sound source;
and multiplying the frequency spectrum value of each frequency point of any path of signal by a corresponding beam forming coefficient to obtain the audio frequency component of any path of signal in the sound source direction.
In one possible implementation, before converting the voice command into the control command, the method further includes:
determining a noise reduction method corresponding to the noise type according to the noise type of the aircraft;
and carrying out noise reduction processing on the voice command by the noise reduction method.
In one possible implementation manner, before determining the sound source direction of the voice signal according to the multiple audio signals and the position of each voice acquisition unit in the aircraft, the method further includes:
determining reference tone information of a user sending the voice signal according to the multi-channel audio signal;
determining whether the user has the authority to control the aircraft according to the reference tone information and reference tone information, wherein the reference tone information is the tone information of the user having the authority to control the aircraft;
when the user has the right to control the aircraft, the step of determining the sound source direction of the voice signal according to the multi-channel audio signal and the position of each voice acquisition unit in the aircraft is performed.
In the embodiment of the invention, the voice acquisition array is integrated in the aircraft, the voice instruction is acquired through the voice acquisition array, and the voice instruction is converted into the control instruction, so that a user can control the aircraft directly through voice without control equipment, and the practicability of the aircraft is improved. And, the extra supporting controlgear of aircraft has been saved for unmanned aerial vehicle becomes more portable, and saved the cost.
The embodiment of the invention provides an aircraft control method, which is applied to an aircraft, and the execution subject of the method can be a processor in a fuselage of the aircraft, the fuselage comprising the processor or the aircraft comprising the processor. In the embodiment of the present invention, an executing subject is taken as an aircraft for example. Referring to fig. 3, the method includes:
step 301: the aircraft acquires voice signals through a plurality of voice acquisition units respectively to obtain a plurality of paths of audio signals, one voice acquisition unit acquires one path of audio signals, and the voice signals are used for controlling the aircraft.
The aircraft is provided with a voice acquisition array, and voice signals can be acquired through the voice acquisition array. Thus, the user can control the aircraft by voice. When the user needs to control the aircraft, the user speaks into the aircraft. The aircraft collects the voice signals through a plurality of voice collecting units respectively to obtain a plurality of paths of audio signals. For example, when the aircraft is used for aerial photography, the user may speak "auto-wrap" into the aircraft when the user wants to control the aircraft to automatically wrap around a target object. For another example, when the user wants to control the aircraft to follow the target object, the user may say "auto follow" to the aircraft.
After the multi-channel audio signal is collected by the aircraft, step 302 may be directly performed to determine the sound source direction of the audio signal. To improve safety, the aircraft may be controlled only by the user having the authority to control the aircraft. Correspondingly, after the aircraft collects the multi-channel audio signals, the aircraft determines whether a user sending the audio signals has the authority to control the aircraft; step 302 is only performed when the user has the right to control the aircraft; when the user does not have the right to control the aircraft, the step 301 is continuously executed to collect the voice signal, and the step 302 is not executed until the voice signal sent by the user with the right to control the aircraft is collected.
Before this step, reference timbre information of a user having authority to control the aircraft is stored in the aircraft. Accordingly, the step of the aircraft determining whether the user has the right to control the aircraft may be: the aircraft determines the reference tone information of the user sending the voice signal according to the multi-channel audio signal, and determines whether the user has the authority to control the aircraft according to the reference tone information and the reference tone information. The reference tone color information is tone color information of a user having authority to control the aircraft. Correspondingly, the step of determining whether the user has the authority to control the aircraft by the aircraft according to the reference tone color information and the reference tone color information may be: the aircraft determines the similarity between the reference tone color information and the reference tone color information; and if the similarity is larger than a preset threshold value, determining that the user has the authority to control the aircraft. And if the similarity is not greater than the preset threshold value, determining that the user does not have the authority to control the aircraft. And the reference tone color information is tone color information of a user having the authority of controlling the aircraft.
It should be noted that the aircraft may be provided with one or more users having the authority to control the aircraft. When a plurality of users having authority to control the aircraft are set in advance, reference tone color information of each user is stored in the aircraft. Correspondingly, the step of determining whether the user has the authority to control the aircraft by the aircraft according to the reference tone color information and the reference tone color information may be: determining the similarity between the reference tone information and each piece of reference tone information to obtain a plurality of similarities during flight; if the similarity with the similarity larger than a preset threshold exists in the multiple similarities, determining that the sound source has the authority to control the aircraft; and if the similarity degrees are all smaller than a preset threshold value, determining that the sound source does not have the authority of controlling the aircraft. The preset threshold may be set and changed as needed, and in the embodiment of the present invention, the preset threshold is not specifically limited. For example, the preset threshold may be 80% or 85%.
In one implementation, when the aircraft determines the reference tone information of the user who sends the voice signal according to the multiple audio signals, the aircraft may select one audio signal from the multiple audio signals, extract the tone information of the voice signal, and use the tone information as the reference tone information of the sound source. When the aircraft selects one audio signal from the multiple audio signals, the aircraft can select randomly, or select the audio signal with the maximum intensity according to the intensity of each audio signal, or select the audio signal with the best quality according to the quality of each audio signal. In the embodiment of the present invention, the manner in which the aircraft selects the audio signal is not particularly limited. In the embodiment of the invention, the aircraft only extracts the tone information of one path of audio signal, thereby saving the computing resource and improving the computing efficiency.
In another implementation, when the aircraft determines the reference tone information of the user who sent the voice signal according to the multiple audio signals, the aircraft may also extract multiple pieces of tone information from the multiple audio signals, and determine average tone information of the multiple pieces of tone information as the reference tone information of the user. Wherein the spectral envelope is used to indicate the timbre information. Accordingly, the step of the aircraft determining the average timbre information of the plurality of timbre information as the reference timbre information of the user may be: the aircraft determines a spectral envelope of each speech signal, determines an average envelope of the spectral envelopes of each speech signal, and determines the average envelope as the reference timbre information for the user. In the embodiment of the invention, the accuracy of the determined reference tone information is improved because the reference tone information is extracted from the multi-channel audio signal.
Step 302: and the aircraft determines the sound source direction of the voice signal according to the multi-channel audio signal and the position of each voice acquisition unit in the aircraft.
The sound source direction includes an azimuth angle and a pitch angle of the sound source. For each voice capture unit, the location of the voice capture unit in the aircraft includes the coordinates of the voice capture unit in the set three-dimensional coordinate system of the aircraft. And, prior to this step, the aircraft stores the location of each voice capture unit in the aircraft. Accordingly, this step can be realized by the following steps (1) to (4), including:
(1): the aircraft determines a unit vector function, which is a function that includes an azimuth parameter and a pitch parameter.
For example, if the azimuth angle parameter and the pitch angle parameter are α and β, respectively, the unit vector function is a α,β =[cosα*sinβ,sinα*sinβ,cosβ] T
(2): and determining the theoretical time difference of the audio signals acquired by any two voice acquisition units by the aircraft according to the unit vector function and the position of each voice acquisition unit in the aircraft.
For convenience of description, any two voice acquisition units are called a first voice acquisition unit and a second voice acquisition unit, an audio signal acquired by the first voice acquisition unit is called a first audio signal, and a voice signal acquired by the second voice acquisition unit is called a second audio signal. Then, the step of determining the theoretical time difference between the first audio signal collected by the first voice collection unit and the second audio signal collected by the second voice collection unit by the aircraft according to the unit vector function and the positions of the first voice collection unit and the second voice collection unit in the aircraft may be: the aircraft determines a first coordinate vector of the first voice acquisition unit and a second coordinate vector of the second voice acquisition unit; and determining the distance between the first voice acquisition unit and the second voice acquisition unit in the direction of the sound source according to the first coordinate vector, the second coordinate vector and the unit vector function, and taking the ratio of the distance to the sound velocity in the air as the theoretical time difference.
In one possible implementation manner, the step of determining, by the aircraft, the distance between the first voice collecting unit and the second voice collecting unit in the direction of the sound source according to the first coordinate vector, the second coordinate vector and the unit vector function may be: the aircraft determines a transposition function of the unit vector function; determining the product of the transposition function and the first coordinate vector to obtain a first distance between the first voice acquisition unit and the center of the coordinate system, and determining the product of the transposition function and the second coordinate vector to obtain a second distance between the second voice acquisition unit and the center of the coordinate system; and determining the difference value of the first distance and the second distance to obtain the distance.
In another possible implementation manner, the step of determining, by the aircraft according to the first coordinate vector, the second coordinate vector and the unit vector function, the distance between the first voice collecting unit and the second voice collecting unit in the direction of the sound source may be: the aircraft determines a transposition function of the unit vector function, and determines the distance between the first voice acquisition unit and the second voice acquisition unit in the direction of the sound source according to the transposition function, the first coordinate vector and the second coordinate vector through the following formula I.
The formula I is as follows:
Figure GDA0001858806600000111
and i and j are respectively the serial number of the first voice acquisition unit and the serial number of the second voice acquisition unit. d ij The distance between the first voice acquisition unit and the second voice acquisition unit in the direction of the sound source.
Figure GDA0001858806600000112
Is a function of the unit vector. p is a radical of formula i Is a first coordinate vector, p j Is a second coordinate vector. />
It should be noted that the theoretical time difference between the audio signals collected by any two voice collecting units determined in this step is not a specific value, but a function including the azimuth angle and the pitch angle of the sound source direction.
(3): the aircraft determines the actual time difference between the audio signals acquired by any two voice acquisition units.
A timer is installed in each voice acquisition unit; when the voice signal is collected by the voice collecting unit, the time for collecting the voice signal is obtained through the timer. Correspondingly, the steps can be as follows: the step of the aircraft acquiring the actual time difference between the first audio signal acquired by the first audio acquisition unit and the second audio signal acquired by the second audio acquisition unit may be: the aircraft acquires the first time when the first voice acquisition unit acquires the first audio signal, acquires the second time when the second voice acquisition unit acquires the first audio signal, and takes the difference value between the first time and the second time as the actual time difference between the first audio signal acquired by the first voice acquisition unit and the second audio signal acquired by the second voice acquisition unit.
In the embodiment of the invention, a timer is not installed in the aircraft, and the actual time difference of the audio signals acquired by any two voice acquisition units is obtained by performing time-frequency analysis on the acquired audio signals. Correspondingly, the step of acquiring the actual time difference between the first audio signal acquired by the first audio acquisition unit and the second audio signal acquired by the second audio acquisition unit by the aircraft may be:
for the first audio signal and the second audio signal, the aircraft determines a cross-correlation function between the first audio signal and the second audio signal, and determines a time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes a maximum value as a time difference between the first audio signal and the second audio signal.
In one possible implementation, the aircraft may determine a cross-correlation function between the first audio signal and the second audio signal in the frequency domain. Accordingly, the step of the aircraft determining the cross-correlation function between the first audio signal and the second audio signal may be realized by the following steps (3-1) to (3-2), comprising:
(3-1): the aircraft respectively converts the first audio signal and the second audio signal into a first time-frequency signal and a second time-frequency signal, and the first audio signal and the second audio signal are any two paths of audio signals.
The aircraft respectively carries out framing processing on the first audio signal and the second audio signal to obtain a framed first audio signal and a framed second audio signal; windowing the first audio signal and the second audio signal after framing, and performing Fourier transform on the first audio signal and the second audio signal in the window to obtain a first time-frequency signal and a second time-frequency signal.
After the aircraft converts each audio signal into a time-frequency signal, the aircraft stores the corresponding relation between the audio signal and the time-frequency signal so as to directly acquire the stored time-frequency signal without converting each audio signal again when wave beam forming is carried out subsequently.
(3-2): the aircraft determines a cross-correlation function between the first audio signal and the second audio signal based on the first time-frequency signal and the second time-frequency signal.
And the aircraft multiplies the first time-frequency signal and the second time-frequency signal, and performs inverse Fourier transform on the multiplication result to obtain a cross-correlation function between the first audio signal and the second audio signal.
Because the waveforms of the first time-frequency signal and the second time-frequency signal are consistent, only one time delay relationship exists, and the maximum value is obtained when the time delay of the cross-correlation function is consistent with the time delays of the two paths of audio signals (the first audio signal and the second audio signal). Therefore, as long as the maximum value of the cross-correlation function is detected, the time difference of the first audio signal and the second audio signal reaching the first voice acquisition unit and the second voice acquisition unit can be obtained.
In another possible implementation, the aircraft may determine the cross-correlation function between the first audio signal and the second audio directly in the time domain. Accordingly, the aircraft determines the cross-correlation function by convolving the first audio signal and the second audio signal with each other.
(4): when the theoretical time difference and the actual time difference of the voice signals collected by any two voice collecting units of the aircraft are the same, the azimuth angle parameter value and the pitch angle value in the unit vector function are determined, and the azimuth angle and the pitch angle of the sound source direction are obtained.
In a possible implementation manner, the theoretical time difference between the voice signals collected by any two voice collecting units is a unit vector function including the azimuth angle parameter and the pitch angle parameter, so that the theoretical time difference between the voice signals collected by any two voice collecting units is the same as the actual time difference, and the value of the azimuth angle parameter and the value of the pitch angle parameter in the unit vector function are solved to obtain the azimuth angle and the pitch angle of the sound source direction.
In another possible implementation manner, the aircraft fits the theoretical time difference and the actual time difference of the voice signals acquired by any two voice acquisition units by using a least square method to obtain a fitting function shown in formula two. And when the value of the fitting function is zero, determining the value of the azimuth angle parameter and the value of the pitch angle in the unit vector function to obtain the azimuth angle and the pitch angle of the sound source direction.
The formula II is as follows:
Figure GDA0001858806600000131
wherein M is the number of voice acquisition units included in the acquisition voice acquisition array.
Figure GDA0001858806600000132
Figure GDA0001858806600000133
Is the actual time difference between the first speech signal and the second speech signal. Let J (α, β) be zero, a can be obtained α,β =A -1 b, wherein
Figure GDA0001858806600000134
Suppose a α,β =[a x ,.,a z ] T The azimuth angle and the pitch angle of the sound source direction are ^ and ^ respectively>
Figure GDA0001858806600000135
β=cos -1 a z
Step 303: and the aircraft determines the audio component of each audio signal in the sound source direction according to the sound source direction to obtain multiple paths of audio components.
The beam forming method can enable the voice collecting array to have certain directivity, namely, the voice signals in the sound source direction are enhanced, and the voice signals in other directions are restrained. Therefore, the aircraft synthesizes only the audio component of each audio signal in the direction of the sound source when beamforming multiple voice signals. For any audio signal, the step of the aircraft determining the audio component of the audio signal in the sound source direction according to the sound source direction can be realized by the following steps (1) to (3), including:
(1): the aircraft converts any one of the audio signals into a third time-frequency signal.
In one possible implementation manner, in step 302, the aircraft has converted each audio signal into a time-frequency signal, and in this step, the aircraft obtains a third time-frequency signal corresponding to the audio signal from a corresponding relationship between the audio signal and the time-frequency signal according to any one of the audio signals.
In another possible implementation manner, the terminal may not store the correspondence between the audio signal and the time-frequency signal. The aircraft directly performs time-frequency conversion on the audio signal in the step to obtain a third time-frequency signal. Correspondingly, the step of converting any one of the audio signals into the third time-frequency signal by the aircraft may be: the aircraft carries out framing processing on any one path of audio signal to obtain any one path of audio signal after framing; windowing any one channel of audio signals after framing, and carrying out Fourier transform on the audio signals in the window to obtain a third time-frequency signal.
(2): and the aircraft determines the beam forming coefficient of each frequency point in the third audio signal in the direction of the sound source.
For each frequency bin comprised by the third audio signal. The step that the aircraft determines the beam forming coefficient of the frequency point in the sound source direction may be: and the aircraft determines the guide vector of the frequency point in the sound source direction, and determines the beam forming coefficient of the frequency point in the sound source direction according to the guide vector of the frequency point in the sound source direction. In the embodiment of the present invention, the specific manner of determining the beam forming coefficient of the frequency point in the sound source direction according to the steering vector of the frequency point in the sound source direction is not specifically limited; for example, the steering vector of the frequency point in the sound source direction is directly used as the beam forming coefficient of the frequency point in the sound source direction.
For another example, in order to improve the algorithm stability, the aircraft may obtain the cross-power spectral density of the diagonal vector and the spatial noise, and determine the beam forming coefficient of the frequency point in the sound source direction according to the diagonal vector, the cross-power spectral density of the spatial noise, and the steering vector of the frequency point in the sound source direction by the following formula three.
The formula III is as follows:
Figure GDA0001858806600000141
wherein ε I is a diagonal loading to improve the stability of the algorithm; d (k) is a guide vector of the k-th frequency point in the sound source direction, and Q is the cross-power spectral density of the spatial noise. And ε I and Q are known quantities. Also, we generally assume that spatial noise is spherically and uniformly distributed noise, and the cross-power spectrum between the speech acquisition units can be approximated by a 0 th order bessel function. w (k) is a beamforming coefficient vector, and w (k) is a vector of M × 1, M being the number of voice acquisition units included in the voice acquisition array.
(3): and the aircraft multiplies the frequency spectrum value of each frequency point of any path of signal by the corresponding beam forming coefficient to obtain the audio component of any path of signal in the sound source direction.
Step 304: and the aircraft performs beam synthesis on the multi-channel audio components to obtain a voice instruction.
The aircraft superposes the multi-channel audio components to obtain the voice command. And the aircraft has received the voice command, the aircraft may proceed directly to step 305 to convert the voice command. As the aircraft may pick up the sound of the aircraft's own propeller and the ambient noise. Therefore, after the aircraft obtains the voice command, the aircraft may perform noise reduction processing on the voice command, and convert the noise-reduced voice command into a control command through step 305. The aircraft can perform noise reduction processing on the voice command by using any existing noise reduction method for performing noise reduction processing on the language command, and in the embodiment of the invention, the noise reduction method for performing noise reduction processing on the aircraft is not particularly limited.
In order to further improve the noise reduction effect, the aircraft can carry out noise reduction processing on the voice command in a targeted manner according to the noise characteristics of the aircraft, so that the noise reduction effect is improved. Correspondingly, the aircraft may process the speech signal by: the aircraft determines a noise reduction method corresponding to the noise type according to the noise type of the aircraft; and carrying out noise reduction processing on the voice command by the noise reduction method. The noise reduction method may be a spectral subtraction method, etc.
In the embodiment of the invention, the voice control distance can be increased by using the voice acquisition array technology, so that the voice control of the unmanned aerial vehicle becomes more practical. In addition, the voice acquisition array can measure the position of a user, and feasibility is provided for a self-photographing alignment function.
Step 305: the aircraft converts the voice command into a control command.
In one possible implementation, the correspondence between the voice commands and the control commands is stored in the aircraft. Correspondingly, the steps can be as follows: and the aircraft acquires the control instruction corresponding to the voice instruction from the corresponding relation between the voice instruction and the control instruction according to the voice instruction. In another possible implementation, the correspondence between the keywords and the control instructions is stored in the aircraft. Correspondingly, the steps can be as follows: and the aircraft extracts keywords from the voice command, and acquires a control command corresponding to the voice command from the corresponding relation between the keywords and the control command according to the keywords. Wherein the key may be a verb that controls the aircraft.
Step 306: and the aircraft controls the aircraft according to the control command.
In one possible implementation, after the aircraft converts the voice command into a control command, the aircraft is controlled directly according to the control command. In another possible implementation, the aircraft may be provided with a part of the control commands that can be controlled by the user by voice, and a part of the control commands that can be controlled by a control device such as a ground control station or a remote controller, for the purpose of improving safety. After the aircraft converts the voice command into a control command, determining whether the control command is a command capable of being controlled by voice; when the control command is a command capable of being controlled by voice, executing step 306; when the control command is not a command capable of being controlled through voice, prompt information is output, and the prompt information is used for prompting a user to control the aircraft through the control equipment.
Before the step, storing an instruction library in the aircraft, wherein at least one instruction capable of being controlled by voice is stored in the instruction library; accordingly, the step of the aircraft determining whether the control command is a command capable of being controlled by voice may be: the aircraft determines whether the control command is contained in the command library; when the instruction library contains the control instruction, determining that the control instruction is an instruction capable of being controlled by voice; when the control instruction is not contained in the instruction library, determining that the control instruction is not an instruction capable of being controlled by voice.
In the embodiment of the invention, the voice acquisition array is integrated in the aircraft, the voice instruction is acquired through the voice acquisition array, and the voice instruction is converted into the control instruction, so that a user can control the aircraft directly through voice without a control device, and the practicability of the aircraft is improved. And, the extra supporting controlgear of aircraft has been saved for unmanned aerial vehicle becomes more portable, and saved the cost.
The embodiment of the invention provides an aircraft control device, which is applied to an aircraft, wherein the aircraft comprises a voice acquisition array, the voice acquisition array comprises a plurality of voice acquisition units, and the device is used for executing the steps executed by the aircraft in the aircraft control method. Referring to fig. 4, the apparatus includes:
the acquisition module 401 is configured to acquire voice signals through the plurality of voice acquisition units, respectively, to obtain multiple channels of audio signals, and one voice acquisition unit acquires one channel of audio signals;
a determining module 402, configured to determine a sound source direction of the voice signal according to the multiple audio signals and a position of each voice collecting unit in the aircraft;
a synthesizing module 403, configured to perform beam forming on the multiple audio signals according to the sound source direction to obtain a voice instruction;
and the control module 404 is configured to convert the voice command into a control command, and control the aircraft according to the control command.
In one possible implementation, referring to fig. 5, the determining module 402 includes:
a first determining unit 4021, configured to determine a unit vector function, where the unit vector function is a function including an azimuth angle parameter and a pitch angle parameter;
the second determining unit 4022 is configured to determine a theoretical time difference between the audio signals acquired by any two voice acquiring units according to the unit vector function and the position of each voice acquiring unit in the aircraft;
a third determining unit 4023, configured to determine an actual time difference between audio signals acquired by any two voice acquiring units;
a fourth determining unit 4024, configured to determine a value of an azimuth angle parameter and a value of a pitch angle in the unit vector function when the theoretical time difference and the actual time difference of the audio signals acquired by any two voice acquisition units are the same, so as to obtain an azimuth angle and a pitch angle of the sound source direction.
In a possible implementation manner, the third determining unit 4023 is further configured to determine, for the first audio signal and the second audio signal, a cross-correlation function between the first audio signal and the second audio signal; the time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes the maximum value is determined as the time difference between the first audio signal and the second audio signal.
In one possible implementation, referring to fig. 6, the synthesis module 403 includes:
a fifth determining unit 4031, configured to determine, according to the sound source direction, an audio component of each channel of audio signal in the sound source direction, to obtain multiple channels of audio components;
a synthesizing unit 4032, configured to perform beamforming on the multiple audio components to obtain the voice command.
In a possible implementation manner, the fifth determining unit 4031 is further configured to convert, for any one of the audio signals, the any one of the audio signals into a third time-frequency signal; determining a beam forming coefficient of each frequency point in the third time-frequency signal in the direction of the sound source; and multiplying the frequency spectrum value of each frequency point of any path of signal by the corresponding beam forming coefficient to obtain the audio component of any path of signal in the sound source direction.
In one possible implementation, referring to fig. 7, the apparatus further includes:
a noise reduction module 405, configured to determine, according to a noise type of the aircraft, a noise reduction method corresponding to the noise type; and carrying out noise reduction processing on the voice command by the noise reduction method.
In a possible implementation manner, the determining module 402 is further configured to determine, according to the multiple audio signals, reference timbre information of a user who sent the speech signal; determining whether the user has the authority to control the aircraft according to the reference tone information and reference tone information, wherein the reference tone information is the tone information of the user having the authority to control the aircraft;
the determining module 402 is further configured to determine a sound source direction of the voice signal according to the multiple audio signals and the position of each voice collecting unit in the aircraft when the user has the right to control the aircraft.
In the embodiment of the invention, the voice acquisition array is integrated in the aircraft, the voice instruction is acquired through the voice acquisition array, and the voice instruction is converted into the control instruction, so that a user can control the aircraft directly through voice without a control device, and the practicability of the aircraft is improved. And, saved the extra supporting controlgear of aircraft for unmanned aerial vehicle becomes more portable, and saved the cost.
It should be noted that: the aircraft control device provided in the above embodiment is only illustrated by dividing the functional modules in the aircraft control, and in practical application, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the aircraft control device provided by the above embodiment and the aircraft control method embodiment belong to the same concept, and specific implementation processes thereof are described in the method embodiment and are not described herein again.
Fig. 8 is a schematic structural diagram of an aircraft 800 according to an embodiment of the present invention. For example, the aircraft 800 may be used to perform the aircraft control methods provided in the various embodiments described above. Referring to fig. 8, the aircraft 800 includes:
the aircraft 800 may include components such as RF (Radio Frequency) circuitry 810, memory 820 including one or more computer readable storage media, an input unit 830, a display unit 840, sensors 850, audio circuitry 860, a WiFi (Wireless Fidelity) module 870, a processor 880 including one or more processing cores, and a power supply 890. Those skilled in the art will appreciate that the aircraft structure illustrated in FIG. 8 does not constitute a limitation of aircraft and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. Wherein:
the RF circuit 810 may be used for receiving and transmitting signals during information transmission and reception or during a call, and in particular, for receiving downlink information from a base station and then processing the received downlink information by the one or more processors 880; in addition, data relating to uplink is transmitted to the base station. In general, RF circuit 810 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, and the like. In addition, the RF circuit 810 may also communicate with networks and other devices via wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communications), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.
The memory 820 may be used to store software programs and modules, and the processor 880 performs various functional applications and data processing by operating the software programs and modules stored in the memory 820. The memory 820 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the stored data area may store data created from use of the aircraft 800 (such as audio data, phone books, etc.), and the like. Further, the memory 820 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 820 may also include a memory controller to provide the processor 880 and the input unit 830 access to the memory 820.
The input unit 830 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, the input unit 830 may include a touch-sensitive surface 831 as well as other input devices 832. The touch-sensitive surface 831, also referred to as a touch display screen or a touch pad, may collect touch operations by a user on or near the touch-sensitive surface 831 (e.g., operations by a user on or near the touch-sensitive surface 831 using a finger, a stylus, or any other suitable object or attachment) and drive the corresponding connection device according to a predefined program. Alternatively, the touch-sensitive surface 831 can include two portions, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, and sends the touch point coordinates to the processor 880, and can receive and execute commands sent from the processor 880. In addition, the touch-sensitive surface 831 can be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. The input unit 830 may include other input devices 832 in addition to the touch-sensitive surface 831. In particular, other input devices 832 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
The display unit 840 may be used to display information entered by or provided to the user as well as various graphical user interfaces of the aircraft 800, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 840 may include a Display panel 841, and the Display panel 841 may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like, as an option. Further, touch-sensitive surface 831 can overlay display panel 841 and, when a touch operation is detected on or near touch-sensitive surface 831, communicate to processor 880 to determine the type of touch event, whereupon processor 880 provides a corresponding visual output on display panel 841 in accordance with the type of touch event. Although in FIG. 8 the touch-sensitive surface 831 and the display panel 841 are implemented as two separate components to implement input and output functions, in some embodiments the touch-sensitive surface 831 may be integrated with the display panel 841 to implement input and output functions.
The aircraft 800 may also include at least one sensor 850, such as light sensors, motion sensors, and other sensors. In particular, the light sensors may include an ambient light sensor that may adjust the brightness of the display panel 841 based on the brightness of ambient light, and a proximity sensor that may turn off the display panel 841 and/or backlight when the aircraft 800 is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the mobile phone is stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which may also be configured on the aircraft 800, further description is omitted here.
Audio circuitry 860, speakers 861, microphone 862 may provide an audio interface between the user and the aircraft 800. The audio circuit 860 can transmit the electrical signal converted from the received audio data to the speaker 861, and the electrical signal is converted into a sound signal by the speaker 861 and output; on the other hand, the microphone 862 converts collected sound signals into electrical signals, which are received by the audio circuit 860 and converted into audio data, which are then processed by the audio data output processor 880 and passed through the RF circuit 810 to be transmitted to, for example, another aircraft, or output to the memory 820 for further processing. The audio circuitry 860 may also include an ear-bud jack to provide communication of peripheral headphones with the aircraft 800.
WiFi belongs to short-range wireless transmission technology, and the aircraft 800 can help users send and receive e-mails, browse web pages, access streaming media and the like through the WiFi module 870, and provides wireless broadband internet access for users. Although fig. 8 shows WiFi module 870, it is understood that it does not belong to the essential constituents of aircraft 800 and may be omitted entirely as needed within the scope that does not alter the essence of the invention.
The processor 880 is the control center of the aircraft 800, connects the various parts of the entire handset using various interfaces and lines, and performs the various functions of the aircraft 800 and processes data by running or executing software programs and/or modules stored in the memory 820 and invoking data stored in the memory 820, thereby monitoring the handset as a whole. Optionally, processor 880 may include one or more processing cores; preferably, the processor 880 may integrate an application processor, which mainly handles operating systems, user interfaces, applications, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 880.
The aircraft 800 also includes a power supply 890 (e.g., a battery) for powering the various components, which may preferably be logically coupled to the processor 880 via a power management system to manage charging, discharging, and power consumption management functions via the power management system. Power supply 890 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
Although not shown, the aircraft 800 may also include a camera, a bluetooth module, etc., which are not described in detail herein. In particular, in this embodiment, the display unit of the aircraft is a touch screen display, the aircraft further includes a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors. The one or more programs include instructions for performing the methods described above in connection with the embodiments of fig. 2 or 3.
The embodiment of the present invention also provides a computer-readable storage medium, which is applied to a terminal, and in which at least one instruction, at least one program, a code set, or a set of instructions is stored, and the instruction, the program, the code set, or the set of instructions is loaded and executed by a processor to implement the operations performed by the aircraft in the aircraft control method according to the above-mentioned embodiment.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.

Claims (14)

1. An aircraft control method, wherein the method is applied to an aircraft, wherein the aircraft comprises a voice acquisition array, wherein the voice acquisition array comprises a plurality of voice acquisition units, and wherein the method comprises the following steps:
the voice signals are respectively collected through the voice collecting units to obtain a plurality of paths of audio signals, and one voice collecting unit collects one path of audio signals;
determining a unit vector function, wherein the unit vector function is a function comprising an azimuth angle parameter and a pitch angle parameter;
determining a transpose function of the unit vector function; determining a product of the transposition function and a first coordinate vector of a first voice acquisition unit of any two voice acquisition units to obtain a first distance between the first voice acquisition unit and the center of a coordinate system, and determining a product of the transposition function and a second coordinate vector of a second voice acquisition unit of any two voice acquisition units to obtain a second distance between the second voice acquisition unit and the center of the coordinate system; determining a difference value between the first distance and the second distance to obtain a distance between the first voice acquisition unit and the second voice acquisition unit in the direction of a sound source; taking the ratio of the distance to the sound velocity in the air as the theoretical time difference between any two voice acquisition units;
determining the actual time difference of the audio signals collected by any two voice collecting units;
when the theoretical time difference and the actual time difference of the audio signals collected by any two voice collecting units are the same, determining the value of the azimuth angle parameter and the value of the pitch angle in the unit vector function to obtain the azimuth angle and the pitch angle of the sound source direction;
according to the sound source direction, carrying out wave beam forming on the multi-channel audio signals to obtain a voice instruction;
and converting the voice command into a control command, and controlling the aircraft according to the control command.
2. The method of claim 1, wherein the determining the actual time difference between the audio signals acquired by any two voice acquisition units comprises:
for a first audio signal and a second audio signal, determining a cross-correlation function between the first audio signal and the second audio signal;
determining a time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes a maximum value as a time difference between the first audio signal and the second audio signal.
3. The method according to claim 1, wherein the beamforming the multiple audio signals according to the sound source direction to obtain a voice command comprises:
determining the audio component of each audio signal in the sound source direction according to the sound source direction to obtain multiple paths of audio components;
and carrying out beam forming on the multi-channel audio frequency components to obtain the voice command.
4. The method according to claim 3, wherein said determining the audio component of each audio signal in the sound source direction according to the sound source direction to obtain multiple audio components comprises:
for any audio signal, converting the audio signal into a time-frequency signal;
determining a beam forming coefficient of each frequency point in the time-frequency signal in the direction of the sound source;
and multiplying the frequency spectrum value of each frequency point of any path of signal with a corresponding beam forming coefficient to obtain the audio component of any path of signal in the sound source direction.
5. The method of claim 1, wherein prior to converting the voice command into a control command, the method further comprises:
determining a noise reduction method corresponding to the noise type according to the noise type of the aircraft;
and carrying out noise reduction processing on the voice instruction by the noise reduction method.
6. The method according to any one of claims 1-5, wherein before obtaining the azimuth angle and the pitch angle of the sound source direction, the method further comprises:
determining reference tone information of a user sending the voice signal according to the multi-channel audio signals;
determining whether the user has the authority to control the aircraft or not according to the reference tone information and reference tone information, wherein the reference tone information is the tone information of the user having the authority to control the aircraft;
and when the user has the right to control the aircraft, executing the step of determining the sound source direction of the voice signal according to the multi-channel audio signals and the position of each voice acquisition unit in the aircraft.
7. An aircraft control device, the device being used in an aircraft, the aircraft including a speech acquisition array comprising a plurality of speech acquisition units, the device comprising:
the acquisition module is used for acquiring voice signals through the plurality of voice acquisition units respectively to obtain a plurality of paths of audio signals, and one voice acquisition unit acquires one path of audio signals;
the determination module includes a first determination unit, a second determination unit, a third determination unit, and a fourth determination unit,
the first determining unit is configured to determine a unit vector function, where the unit vector function is a function including an azimuth angle parameter and a pitch angle parameter;
the second determination unit is configured to determine a transpose function of the unit vector function; determining a product of the transposition function and a first coordinate vector of a first voice acquisition unit in any two voice acquisition units to obtain a first distance between the first voice acquisition unit and the center of a coordinate system, and determining a product of the transposition function and a second coordinate vector of a second voice acquisition unit in any two voice acquisition units to obtain a second distance between the second voice acquisition unit and the center of the coordinate system; determining a difference value between the first distance and the second distance to obtain a distance between the first voice acquisition unit and the second voice acquisition unit in the direction of a sound source; taking the ratio of the distance to the sound velocity in the air as the theoretical time difference between any two voice acquisition units;
the third determining unit is used for determining the actual time difference of the audio signals acquired by any two voice acquiring units;
the fourth determining unit is configured to determine a value of an azimuth angle parameter and a value of a pitch angle in the unit vector function when the theoretical time difference and the actual time difference of the audio signals acquired by any two voice acquisition units are the same, so as to obtain an azimuth angle and a pitch angle of the sound source direction;
the synthesis module is used for carrying out beam forming on the multi-channel audio signals according to the sound source direction to obtain a voice instruction;
and the control module is used for converting the voice command into a control command and controlling the aircraft according to the control command.
8. The apparatus of claim 7,
the third determining unit is further configured to determine, for a first audio signal and a second audio signal, a cross-correlation function between the first audio signal and the second audio signal; determining a time difference in the time domain of the first audio signal and the second audio signal when the cross-correlation function takes a maximum value as a time difference between the first audio signal and the second audio signal.
9. The apparatus of claim 7, wherein the synthesis module comprises:
a fifth determining unit, configured to determine, according to the sound source direction, an audio component of each channel of audio signal in the sound source direction, so as to obtain multiple channels of audio components;
and the synthesis unit is used for carrying out beam synthesis on the multi-channel audio components to obtain the voice command.
10. The apparatus of claim 9,
the fifth determining unit is further configured to convert any one of the audio signals into a time-frequency signal; determining a beam forming coefficient of each frequency point in the time-frequency signal in the direction of the sound source; and multiplying the frequency spectrum value of each frequency point of any path of signal by a corresponding beam forming coefficient to obtain the audio frequency component of any path of signal in the sound source direction.
11. The apparatus of claim 7, further comprising:
the noise reduction module is used for determining a noise reduction method corresponding to the noise type according to the noise type of the aircraft; and carrying out noise reduction processing on the voice instruction by the noise reduction method.
12. The apparatus according to any one of claims 7 to 11,
the determining module is further configured to determine, according to the multiple audio signals, reference timbre information of a user who sent the voice signal; determining whether the user has the authority to control the aircraft or not according to the reference tone information and reference tone information, wherein the reference tone information is the tone information of the user having the authority to control the aircraft;
the determining module is further used for determining the sound source direction of the voice signal according to the multipath audio signals and the position of each voice collecting unit in the aircraft when the user has the right to control the aircraft.
13. An aircraft comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the instruction, the program, the set of codes, or the set of instructions being loaded and executed by the processor to carry out operations performed in the aircraft control method according to any one of claims 1 to 6.
14. A computer-readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to carry out the operations performed in the aircraft control method according to any one of claims 1 to 6.
CN201810842962.XA 2018-07-27 2018-07-27 Aircraft control method, aircraft control device, aircraft and storage medium Active CN110764520B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810842962.XA CN110764520B (en) 2018-07-27 2018-07-27 Aircraft control method, aircraft control device, aircraft and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810842962.XA CN110764520B (en) 2018-07-27 2018-07-27 Aircraft control method, aircraft control device, aircraft and storage medium

Publications (2)

Publication Number Publication Date
CN110764520A CN110764520A (en) 2020-02-07
CN110764520B true CN110764520B (en) 2023-03-24

Family

ID=69327625

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810842962.XA Active CN110764520B (en) 2018-07-27 2018-07-27 Aircraft control method, aircraft control device, aircraft and storage medium

Country Status (1)

Country Link
CN (1) CN110764520B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402863A (en) * 2020-03-23 2020-07-10 北京京东乾石科技有限公司 Control method, system, device, medium and equipment for formation unmanned aerial vehicle
CN112002344A (en) * 2020-08-07 2020-11-27 仿翼(北京)科技有限公司 Aircraft control method, aircraft, control device, and storage medium
CN113419557B (en) * 2021-06-17 2022-07-19 哈尔滨工业大学 Audio synthesis method for unmanned aerial vehicle

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5175262B2 (en) * 2009-12-02 2013-04-03 日本電信電話株式会社 Voice acquisition device
CN102447993A (en) * 2010-09-30 2012-05-09 Nxp股份有限公司 Sound scene manipulation
US9554208B1 (en) * 2014-03-28 2017-01-24 Marvell International Ltd. Concurrent sound source localization of multiple speakers
CN107534725B (en) * 2015-05-19 2020-06-16 华为技术有限公司 Voice signal processing method and device
CN105025182B (en) * 2015-06-11 2018-07-06 广东欧珀移动通信有限公司 A kind of noise-reduction method and terminal
CN107120791A (en) * 2017-04-27 2017-09-01 珠海格力电器股份有限公司 A kind of air conditioning control method, device and air conditioner
CN107680593A (en) * 2017-10-13 2018-02-09 歌尔股份有限公司 The sound enhancement method and device of a kind of smart machine

Also Published As

Publication number Publication date
CN110764520A (en) 2020-02-07

Similar Documents

Publication Publication Date Title
US11450337B2 (en) Multi-person speech separation method and apparatus using a generative adversarial network model
CN107464564B (en) Voice interaction method, device and equipment
CN110113459B (en) Antenna control method and folding screen terminal
US10264135B2 (en) Echo cancellation method and apparatus, and computer storage medium
CN107705778B (en) Audio processing method, device, storage medium and terminal
EP3614383A1 (en) Audio data processing method and apparatus, and storage medium
CN111029772B (en) Antenna circuit, electronic device and method for adjusting antenna performance
CN110764520B (en) Aircraft control method, aircraft control device, aircraft and storage medium
CN111835488B (en) Method for determining mapping of antenna port and terminal
CN108769893B (en) Terminal detection method and terminal
CN110602389B (en) Display method and electronic equipment
CN111445901B (en) Audio data acquisition method and device, electronic equipment and storage medium
CN111405416B (en) Stereo recording method, electronic device and storage medium
CN109739394B (en) SAR value processing method and mobile terminal
CN110456395B (en) Positioning method and terminal equipment
CN111370018A (en) Audio data processing method, electronic device and medium
CN110517677B (en) Speech processing system, method, apparatus, speech recognition system, and storage medium
CN110517702B (en) Signal generation method, and voice recognition method and device based on artificial intelligence
CN112233689A (en) Audio noise reduction method, device, equipment and medium
CN109451146B (en) Vibration processing method and terminal
CN112204943B (en) Photographing method, apparatus, system, and computer-readable storage medium
CN114255781A (en) Method, device and system for acquiring multi-channel audio signal
CN109450508B (en) Antenna determination method and device and mobile terminal
CN111399011A (en) Position information determining method and electronic equipment
CN109325219B (en) Method, device and system for generating record document

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant