JP2020006793A - Unmanned air vehicle with sound absorbing function - Google Patents

Abstract

To provide an unmanned air vehicle with a sound absorption function constantly hovering or patrolling/flying in a monitoring area, rapidly detecting a direction of a user requiring rescue or the like, appropriately receiving the needs of the user, performing a processing satisfying the needs and controlling the degradation of convenience for the user.SOLUTION: An unmanned air vehicle with a sound-absorbing function includes: a microphone array disposed on the bottom surface side of a housing and absorbing sound in a monitoring area; a voice processing part estimating a direction of a sound source generating in the monitoring area on the basis of data of the sound absorbed by the microphone array; a flying control part controlling hovering or patrolling/flying in the monitoring area, and controlling the flying/moving in a direction of the estimated sound source; a voice output part disposed on the bottom surface side of the housing and outputting stipulated voice from a speaker; and a control part receiving input of the processing satisfying the needs of a user and performing the processing after outputting the stipulated voice.SELECTED DRAWING: Figure 5

Description

  The present disclosure relates to an unmanned aerial vehicle with a sound collection function that hoveres or circulates around a monitored area.

  Patent Literature 1 discloses an unmanned aerial vehicle that controls the execution of a rescue operation based on the received input contents when an input of a rescue operation is received from a user in order to execute a rescue operation in an emergency such as a disaster. ing. Before performing rescue work, the unmanned aerial vehicle waits at a waiting place in the hospital where the user stays in peacetime, and after accepting the work selected by the user, exits the waiting place and moves along the travel route. , Head to the disaster site. This moving route is obtained by a calculation process based on the position of the disaster place and the position of the standby place received by the transmitting and receiving unit of the unmanned aerial vehicle.

JP 2017-213951 A

  The unmanned aerial vehicle disclosed in Patent Document 1 waits in a standby place in a hospital during an emergency when no emergency occurs, and after an emergency occurs, when the position of a disaster site where rescue work is performed is designated, the disaster occurs. Move towards the site location. In other words, the unmanned aerial vehicle disclosed in Patent Literature 1 does not constantly hover or circulate around an area to be monitored such as a disaster site. For this reason, it is not possible to determine where to move if the destination designation has not been input in advance, and it is not possible to quickly detect the voice of a victim who seeks rescue at a disaster site, so rescue is required. There was a problem that the work such as rescue could not be performed promptly afterwards, and the convenience was reduced.

  The present disclosure has been devised in view of the above-described conventional situation, constantly hovering or circling in the monitoring area, quickly detecting the direction of the user seeking rescue, etc., appropriately accepting the needs of the user, and An object of the present invention is to provide an unmanned aerial vehicle with a sound collection function that can execute processing that meets needs and suppresses a decrease in user convenience.

  The present disclosure is arranged on the bottom side of the housing, and estimates a direction of a sound source generated in the monitoring area based on sound data collected by the microphone array and sound collected by the microphone array. A voice processing unit that controls hovering or circulating flight of the monitoring area, and a flight control unit that controls flight movement in the direction of the estimated sound source. An unmanned aerial vehicle with a sound collection function, comprising: a sound output unit that outputs sound from a speaker; and a control unit that receives and executes an input of a process that satisfies the user's needs after outputting the predetermined sound.

  According to the present disclosure, it is possible to constantly hover or circulate around the monitoring area, quickly detect the direction of the user seeking rescue, etc., and appropriately execute the process that accepts the user's needs and meets the needs. A reduction in user convenience can be suppressed.

The figure which shows an example of the external appearance which looked at the housing | casing of the unmanned aerial vehicle which concerns on Embodiment 1 from the bottom side. The figure which shows an example of the external appearance which looked at the housing | casing of the unmanned aerial vehicle which concerns on Embodiment 1 from the side. FIG. 2 is a block diagram illustrating a hardware configuration example of the unmanned aerial vehicle according to the first embodiment. Explanatory drawing which shows the operation | movement outline example of the unmanned aerial vehicle which concerns on Embodiment 1. Flow chart showing an example of a first operation procedure of the unmanned aerial vehicle according to Embodiment 1 in chronological order Flow chart showing an example of a second operation procedure of the unmanned aerial vehicle according to Embodiment 1 in chronological order Block diagram showing an example of a hardware configuration of an unmanned aerial vehicle and a cloud server according to Embodiment 1.

  Hereinafter, an embodiment that specifically discloses an unmanned aerial vehicle with a sound collection function according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, an unnecessary detailed description may be omitted. For example, a detailed description of well-known matters and a repeated description of substantially the same configuration may be omitted. This is to prevent the following description from being unnecessarily redundant and to facilitate understanding of those skilled in the art. The accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

  Hereinafter, as an unmanned aerial vehicle with a sound collecting function according to the present disclosure, a monitoring area to be monitored (a place where a disaster, an incident or an accident is likely to occur or a place where it occurs) is constantly hovered (flighted) and stays or patrols. An example of an unmanned aerial vehicle that flies in the air will be described. The unmanned aerial vehicle has a function of collecting sound in a monitoring area and is configured using a UAV (Unmanned Aerial Vehicle) such as a drone. The present disclosure can also be defined as a processing method executed in an unmanned aerial vehicle with a sound collecting function, or a system including an unmanned aerial vehicle with a sound collecting function.

  In the first embodiment, in a monitoring area to be monitored by an unmanned aerial vehicle, for example, a rider who is in trouble due to a motorcycle accident seeks rescue or the like from an unmanned aerial vehicle hovering or circulating in the monitoring area, This will be described as an example. For convenience, the above-mentioned rider (that is, a person who seeks help from an unmanned aerial vehicle) is referred to as a “user”.

  The unmanned aerial vehicle 10 according to the first embodiment has a microphone array MA that is disposed on the bottom surface side of the housing 11 and that collects sound in the monitoring area 8. The unmanned aerial vehicle 10 includes an audio signal processing unit 26 that estimates a direction of a sound source generated in the monitoring area 8 based on sound data collected by the microphone array MA. The unmanned aerial vehicle 10 has a drone flight control unit 22 that controls hovering or round-trip flight of the monitoring area 8 and controls flight movement in the direction of the estimated sound source. The unmanned aerial vehicle 10 has sound output units 291 to 294 that are arranged on the bottom side of the housing 11 and output predetermined sounds (see below) from the speakers SP1 to SP4, respectively. The unmanned aerial vehicle 10 includes a drone control management unit 21 that receives and executes a process that satisfies the needs of the user after outputting a predetermined voice.

  FIG. 1 is a diagram illustrating an example of an appearance of a housing 11 of an unmanned aerial vehicle 10 according to Embodiment 1 as viewed from a bottom side. FIG. 2 is a diagram illustrating an example of an appearance of the housing 11 of the unmanned aerial vehicle 10 according to Embodiment 1 as viewed from a side.

  The unmanned aerial vehicle 10 is a multi-copter type UAV (Unmanned Aerial Vehicle), and is also called a drone. Specifically, the unmanned aerial vehicle 10 includes a housing 11 including a rotor blade mechanism 24 (see below) having a plurality of (for example, four) rotor blades PR1, PR2, PR3, and PR4, a camera CA, and a microphone array MA. And a plurality of (for example, four) speakers SP1 to SP4.

  Accordingly, the unmanned aerial vehicle 10 autonomously ascends, descends, turns left, moves to the left, moves to the right, turns to the right, or uses a combination thereof by the rotation of the four rotors PR1 to PR4. Flying movement is possible by performing an action having a plurality of degrees of freedom. The unmanned aerial vehicle 10 can acquire its own three-dimensional spatial position information using a GPS (Global Positioning System) receiver (not shown) built in the housing 11.

  As described above, the unmanned aerial vehicle 10 hoveres (that is, levitates and stays in the air) over the monitoring area 8 or flies around to monitor the monitoring area 8. In addition, the unmanned aerial vehicle 10 may, for example, take an aerial image of a destination or a target object, spray a loaded medicine, or carry goods.

  As shown in FIG. 1, the monitoring area 8 is imaged (that is, the sky) on the bottom surface side of the housing 11 of the unmanned aerial vehicle 10 as an example of the unmanned aerial vehicle with a sound collection function (that is, the vertically downward side when flying over the sky). The housing of the camera CA for shooting (photographing) and the housing BD of the microphone array MA for collecting the sound of the monitoring area 8 are coaxially and integrally formed and arranged. The housing of the camera CA and the housing BD of the microphone array MA may be supported via a gimbal (not shown). Thus, the unmanned aerial vehicle 10 can collect sound generated in the monitoring area 8 while flying in the monitoring area 8 and capture an image (for example, a still image or a moving image) of the monitoring area 8 as a subject. It is possible. Although the details will be described later, the optical axis direction J1 of the camera CA coincides with the center axis of the housing BD of the microphone array MA.

  Although not shown in FIGS. 1 and 2, the unmanned aerial vehicle 10 may include a plurality of imaging devices exposed from the housing 11 separately from the camera CA. The plurality of (for example, four) imaging devices are, for example, sensing cameras that image the periphery of the unmanned aerial vehicle 10 in order to control the flight of the unmanned aerial vehicle 10. Some (for example, two) imaging devices may be provided on the front of the unmanned aerial vehicle 10, which is the nose. Further, the remaining (for example, two) imaging devices may be provided on the bottom surface of the unmanned aerial vehicle 10. The two imaging devices on the front side may be paired and function as a so-called stereo camera. The two imaging devices on the bottom side may also be paired and function as a stereo camera. Three-dimensional spatial data around the unmanned aerial vehicle 10 may be generated based on images captured by a plurality of imaging devices. Note that the number of imaging devices separately provided in the unmanned aerial vehicle 10 is not limited to four.

  Further, the unmanned aerial vehicle 10 is located on the opposite side of the arms AM1, AM2, AM3, and AM4 corresponding to the rotary wings PR1, PR2, PR3, and PR4 (that is, on the bottom side of each arm AM1, AM2, AM3, and AM4). Speakers SP1 to SP4 are provided. Note that the speakers SP1 to SP4 are located at substantially the center of the corresponding arms AM1 to AM4 (that is, at the substantially middle position between the positions on the opposite sides of the rotary blades PR1 to PR4 and the center of the housing 11). It may be arranged. As will be described later, when the unmanned aerial vehicle 10 sufficiently approaches the user HM1 seeking rescue, the speakers SP1 to SP4 provide the user HM1 with an instruction from the drone control management unit 21 built in the housing 11. A default sound for inquiring whether there is a task such as rescue is output.

  FIG. 3 is a block diagram illustrating a hardware configuration example of the unmanned aerial vehicle 10 according to the first embodiment. The unmanned aerial vehicle 10 includes a drone control management unit 21, a drone flight control unit 22, a memory 23, a rotary wing mechanism 24 including a motor group 241, a wireless communication unit 25, a voice signal processing unit 26, and a voice recognition unit. 26, a camera control unit 27, an image processing unit 28, an image analysis unit 28a, and a plurality (for example, four) of audio output units 291 to 294. The unmanned aerial vehicle 10 is configured to include a microphone array MA, a camera CA, and a plurality (for example, four) of speakers SP1 to SP4. Each of the audio output units 291 to 294 is provided corresponding to each of the speakers SP1 to SP4.

  A drone control management unit 21, a drone flight control unit 22, a wireless communication unit 25, a voice signal processing unit 26, a voice recognition unit 26a, a camera control unit 27, an image processing unit 28, an image analysis unit 28a Is configured using the processor PRC1. The processor PRC1 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA). The processor PRC1 reads and executes a program and data stored in the memory 23 in advance, thereby controlling the drone control management unit 21, the drone flight control unit 22, the wireless communication unit 25, the audio signal processing unit 26, The recognition unit 26a, the camera control unit 27, the image processing unit 28, and the image analysis unit 28a can be functionally realized.

  The drone control management unit 21 performs control processing for controlling the operation of each unit of the unmanned aerial vehicle 10 in a unified manner, data input / output processing with other units, data calculation (calculation) processing, and data storage processing. I do.

  The drone control management unit 21 may acquire date and time information indicating the current date and time from a timer (not shown) built in the housing 11. The drone control management unit 21 acquires date and time information indicating the current date and time. The drone flight control unit 22 may acquire date and time information indicating the current date and time, and position information indicating the latitude, longitude and altitude at which the unmanned aerial vehicle 10 is located, from a GPS receiver (not shown) built in the housing 11. . The drone control management unit 21 acquires direction information indicating the direction of the unmanned aerial vehicle 10 from a magnetic compass (not shown) built in the housing 11. The direction information indicates, for example, a direction corresponding to the direction of the nose of the unmanned aerial vehicle 10.

  The drone flight control unit 22 controls the flight of the unmanned aerial vehicle 10 according to a program stored in the memory 23. The drone flight control unit 22 may control the flight of the unmanned aerial vehicle 10 according to a command received from a remote operation terminal (not shown) via the wireless communication unit 25.

  The drone flight control unit 22 controls the flight of the unmanned aerial vehicle 10 by controlling the motor group 241 mounted on the rotary wing mechanism 24. That is, the drone flight control unit 22 controls the position including the latitude, longitude, and altitude of the unmanned aerial vehicle 10 by controlling the rotary wing mechanism 24. The drone flight control unit 22 may control the imaging range of the camera CA by controlling the flight of the unmanned aerial vehicle 10. The drone flight control unit 22 may control the angle of view and the zoom magnification of the camera CA, for example, by controlling a pan / tilt mechanism and a zoom lens (not shown) included in the camera CA.

  The drone flight control unit 22 receives the image analysis result of the image analysis unit 28a (for example, the environment around the unmanned aerial vehicle 10) and controls the flight while avoiding obstacles. The drone flight control unit 22 may generate three-dimensional space data around the unmanned aerial vehicle 10 based on a plurality of images captured by the camera CA, and control flight based on the three-dimensional space data.

  The memory 23 stores programs and data necessary for controlling execution of operations (processing) of each unit realized by the processor PRC1. The memory 23 may be a computer-readable recording medium, such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), and It may include at least one of a flash memory such as a USB memory. The memory 23 may be provided inside the housing 11 or may be provided detachably from the housing 11.

  The memory 23 holds a first threshold value, a second threshold value, and a third threshold value that are compared with the sound pressure levels of the blocks constituting the omnidirectional captured image data calculated by the audio signal processing unit 26. Here, the sound pressure level is used as an example of a sound parameter indicating a sound volume generated in the monitoring area 8 picked up by the microphone array MA, and indicates a sound volume output from the speakers SP1 to SP4. This is a different concept from volume. The first threshold, the second threshold, and the third threshold are thresholds that are compared with the sound pressure level of the sound generated in the monitoring area 8. In addition, a plurality of thresholds can be set other than the above-described first threshold, second threshold, and third threshold, and only the first threshold may be used. Here, for simplicity, for example, three values are set: a first threshold value, a second threshold value smaller than the first threshold value, and a third threshold value smaller than the first threshold value (first threshold value> second threshold value> Third threshold).

  The memory 23 has a pattern memory in which sound patterns unique to each of the rotor blades PR1 to PR4 of the unmanned aerial vehicle 10 are registered. The audio signal processing unit 26 excludes (i.e., removes) a sound signal having a sound pattern unique to each of the rotary blades PR1 to PR4 registered in the memory 23 in advance from the sound data collected by the microphone array MA. Above, the position of the sound source is estimated.

  The rotary wing mechanism 24 includes a plurality (for example, four) of rotary blades PR1, PR2, PR3, and PR4, and a motor group 241 including a plurality of drive motors for rotating each of the rotary blades PR1 to PR4. Including.

  Here, in a multi-copter type drone such as the unmanned aerial vehicle 10, when the number of wings in each of the rotary wings PR1 to PR4 is generally two, a harmonic having a frequency twice as high as a specific frequency, and further a multiplication of the harmonic. Harmonics of frequency are generated. Similarly, when the number of blades in each of the rotary blades PR1 to PR4 is three, a harmonic having a frequency three times the specific frequency and a harmonic having a frequency multiplied by three times the specific frequency are generated. The same applies to the case where the number of blades in each of the rotary blades PR1 to PR4 is four or more.

  The wireless communication unit 25 communicates with an external device (for example, an operation terminal for remotely operating the unmanned aerial vehicle 10) according to a wireless communication standard of a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark). Perform wireless communication between them. The wireless communication unit 25 receives various commands (commands) related to the flight of the unmanned aerial vehicle 10 transmitted from the above-described external device, and sends the received commands to the drone control management unit 21.

  The audio signal processing unit 26 as an example of the audio processing unit estimates the direction of the sound source generated in the monitoring area 8 based on the sound data of the monitoring area 8 collected by the microphone array MA. The audio signal processing unit 26 uses the sound data collected by the microphone array MA in accordance with, for example, a known whitening cross-correlation method (CSP (Cross-power Spectrum Phase analysis) method) to generate a sound source position in the monitoring area 8. Is estimated. In the CSP method, the audio signal processing unit 26 divides the monitoring area 8 into a plurality of blocks (not shown), and when a sound is collected by the microphone array MA, a sound parameter (a sound parameter indicating a loudness of each block) For example, whether a normalized output value of a cross-correlation value between sound data picked up by two microphones of a plurality of sets among a plurality of microphones constituting the microphone array MA exceeds a threshold value (predetermined value). By determining whether the sound source is located, the sound source position in the monitoring area 8 can be roughly estimated.

  The audio signal processing unit 26 converts the omnidirectional captured image data of the monitoring area 8 based on the omnidirectional captured image data captured by the camera CA (for example, an omnidirectional camera) and the sound data collected by the microphone array MA. For each of the constituent blocks (for example, a pixel set composed of a predetermined number of pixels such as “2 * 2”, “4 * 4”, etc .; the same applies hereinafter), the loudness at the position corresponding to the block Is calculated (for example, the normalized output value of the cross-correlation value or the sound pressure level described above). The sound signal processing unit 26 uses the calculation result of the sound parameter for each block constituting the omnidirectional captured image data, and assigns, for each block, the calculated sound pressure level to the position of the corresponding block. Generate a map. Further, the audio signal processing unit 26 includes, for example, for each corresponding block of the generated sound pressure map, so that the sound pressure map is constituted by a matrix in which numerical values are arranged, so that the user can easily recognize the sound pressure map visually. A sound pressure heat map is generated by performing color conversion processing on a colored image (an example of a visual image) corresponding to the sound pressure level of the block and assigning it. The sound parameter calculation processing described above is a known technique, and a detailed description of the processing will be omitted.

  The audio signal processing unit 26 performs directivity forming processing of the sound data collected by the microphone array MA using the sound data collected by the microphone array MA and the information indicating the estimated direction of the sound source. Then, the sound data in the sound source direction is emphasized. The processing for forming the directivity of sound data is a known technique described in, for example, JP-A-2015-029241.

  Note that the audio signal processing unit 26 has been described to generate the sound pressure heat map in which the sound pressure level calculated for each block described above is assigned to the position of the corresponding block. May be calculated, and a sound pressure heat map in which the sound pressure level of each pixel is assigned to the position of the corresponding pixel may be generated.

  In the sound pressure heat map generated by the sound signal processing unit 26, for example, a red image is assigned to a pixel region where a sound pressure level higher than the first threshold is obtained. For example, a pink image is assigned to a pixel region where a sound pressure level greater than the second threshold and equal to or less than the first threshold is obtained. For example, a blue image is assigned to a pixel region where a sound pressure level greater than the third threshold and equal to or less than the second threshold is obtained. For example, a colorless image is assigned to a region having a sound pressure level of a pixel equal to or less than the third threshold value, and thus is not different from the display color of the omnidirectional captured image data.

  Note that, as shown in FIGS. 1 and 2, the camera CA and the microphone array MA are respectively arranged such that the optical axis direction J1 of the camera CA and the center axis of the housing of the microphone array MA are coaxial. For this reason, the direction of the sound source estimated by the audio signal processing unit 26 can be used as a parameter indicating the direction when cutting out the image of the direction of the sound source from the omnidirectional image data captured by the omnidirectional camera. Therefore, when the direction of the sound source is estimated by the audio signal processing unit 26, the image analysis unit 28a uses the information indicating the direction of the sound source to generate an omnidirectional image captured by the camera CA (for example, an omnidirectional camera). It is possible to cut out an image in the direction of the estimated sound source from the data.

  However, when the camera CA and the microphone array MA are not coaxially arranged, the camera control unit 27 is estimated by the audio signal processing unit 26 according to, for example, a method described in JP-A-2005-029241. The direction of the sound source may be geometrically corrected, and an image in the corrected direction may be cut out.

  The voice recognition unit 26a performs voice recognition on the sound data that has been subjected to the emphasis processing by the voice signal processing unit 26, for example, using a voice recognition dictionary (not shown) registered in the memory 23 in advance. The voice recognition unit 26a sends the voice recognition result to the drone control management unit 21 or temporarily stores the result in the memory 23.

  The camera control unit 27 controls the operation (processing) of the camera CA (for example, an omnidirectional camera). The camera control unit 27 acquires an image captured by the camera CA and sends the image to the audio signal processing unit 26. When the camera CA is, for example, a PTZ (Pan Tilt Zoom) camera, the camera control unit 27 sends a pan / tilt control instruction to the camera CA to control the camera CA to change the optical axis direction and the zoom magnification. Good.

  The image processing unit 28 receives an image captured by the camera CA, and performs a predetermined image process on the image suitable for an analysis process in the image analysis unit 28a. The image processing unit 28 sends the image data after the image processing to the image analysis unit 28a or temporarily stores the data in the memory 23.

  The image analysis unit 28a analyzes the image generated by the image processing unit 28 (in other words, the image captured by the camera CA), and specifies the environment around the unmanned aerial vehicle 10. Upon receiving the information indicating the direction of the sound source estimated by the audio signal processing unit 26, the image analysis unit 28a obtains an image indicating the direction of the sound source from the omnidirectional image data captured by the camera CA (for example, an omnidirectional camera). (For example, an image in the direction where the user HM1 who has caused the motorcycle accident exists) may be cut out.

  A plurality of (for example, four) audio output units 291 to 294 are provided corresponding to the speakers SP1 to SP4, respectively. When a predetermined audio output instruction is received from the audio signal processing unit 26, a predetermined audio output unit is set according to the output instruction. Is output from each of the corresponding speakers SP1 to SP4.

  The microphone array MA is disposed on the bottom surface side of the housing 11 of the unmanned aerial vehicle 10, and collects sound in all directions (that is, 360 degrees) in the monitoring area 8 in an omnidirectional state. The microphone array MA has a housing BD (see FIGS. 1 and 2) in which a circular opening having a predetermined width is formed in the center. The sounds picked up by the microphone array MA include, for example, mechanical operation sounds such as the unmanned aerial vehicle 10, sounds emitted by human beings, and other sounds, and sound in the audible frequency (ie, 20 Hz to 20 kHz) range. The present invention is not limited thereto, and a low-frequency sound lower than an audio frequency or an ultrasonic sound higher than the audio frequency may be included.

  Microphone array MA includes a plurality of omnidirectional microphones. The microphones are arranged concentrically at predetermined intervals (for example, uniform intervals) around a circular opening provided in the housing BD along the circumferential direction. As each microphone, for example, an electret condenser microphone (ECM) is used. The microphone array MA sends the sound data signal obtained by the sound pickup of each microphone to the sound signal processing unit 26. Note that the arrangement of the microphones is merely an example, and other arrangements (for example, a square arrangement or a rectangular arrangement) may be used, but it is preferable that the microphones are arranged at equal intervals.

  The microphone array MA has at least a plurality of microphones, a plurality of amplifiers for respectively amplifying output signals of the plurality of microphones, and a plurality of A / D converters. An analog signal output from each amplifier is converted into a digital signal by a corresponding A / D converter. The number of microphones in the microphone array MA may be, for example, 16, 32, 64, or 128.

  The camera CA is, for example, an omnidirectional camera, and is accommodated inside the circular opening formed in the center of the housing BD (see FIG. 1) of the microphone array MA so as to substantially match the volume of the circular opening. . That is, the microphone array MA and the camera CA are integrated and arranged so that their respective housing centers are coaxial with each other (see FIG. 1). The camera CA is a camera equipped with a fisheye lens capable of capturing images in all directions (that is, 360 degrees) of the monitoring area 8 as an imaging area of the camera CA. The camera CA functions as, for example, a monitoring camera capable of capturing an image of the monitoring area 8.

  By fitting the camera CA inside the circular opening of the housing BD, the camera CA and the microphone array MA are coaxially arranged. As described above, since the optical axis direction J1 of the camera CA coincides with the central axis of the housing of the microphone array MA, the imaging area and the sound collection area in the axial circumferential direction (that is, the horizontal direction) become substantially the same, The position of the subject in the omnidirectional image captured by the camera CA (in other words, the direction indicating the position of the subject viewed from the camera CA) and the position of the sound source to be picked up by the microphone array MA (in other words, from the microphone array MA) This can be expressed in the same coordinate system (for example, coordinates indicated by (horizontal angle, vertical angle)).

  The speakers SP1 to SP4 acoustically output predetermined sounds (see below) output from the corresponding sound output units 291 to 294, respectively.

  Next, an example of an outline of the operation of the unmanned aerial vehicle 10 according to Embodiment 1 will be described with reference to FIG.

  FIG. 4 is an explanatory diagram illustrating an example of an operation outline of the unmanned aerial vehicle 10 according to the first embodiment. As shown in FIG. 4, in the monitoring area 8 to be monitored by the unmanned aerial vehicle 10, for example, the user HM1 who is driving the motorcycle BK1 is in trouble due to an accident, and hovering the monitoring area 8 or The rescue or the like is requested from the unmanned aerial vehicle 10 that is making a round flight. As will be described later with reference to FIG. 5 and FIG. 6, when the user HM1 finds the unmanned aerial vehicle 10 hovering or circulating in the monitoring area 8, it asks for a rescue or the like by calling out "Oh!"

  The unmanned aerial vehicle 10 estimates the direction of the sound source (that is, the direction in which the user HM1 is located in the example of FIG. 4) by using the sound signal processing unit 26 using the sound data collected by the microphone array MA, and changes the direction to DR1. Along the route so as to approach the user HM1. When the unmanned aerial vehicle 10 is sufficiently close to the user HM1 (for example, when the area occupied by the user HM1 in the image captured by the camera CA exceeds a predetermined ratio), the input command requested by the user HM1 (see below) And executes processing for the received input command.

  Next, an example of the first operation procedure and the second operation procedure of the unmanned aerial vehicle 10 according to Embodiment 1 will be described with reference to FIGS. 5 and 6.

  FIG. 5 is a flowchart showing an example of a first operation procedure of the unmanned aerial vehicle according to Embodiment 1 in chronological order. FIG. 6 is a flowchart showing an example of a second operation procedure of the unmanned aerial vehicle according to Embodiment 1 in a time series. In the description of FIG. 6, the same processes as those in the description of FIG. 5 will be assigned the same step numbers, and the description will be simplified or omitted, and different contents will be described.

  In FIG. 5, the unmanned aerial vehicle 10 is hovering or circling above the monitoring area 8 (St1). In Step St1, the unmanned aerial vehicle 10 is always flying above the monitoring area 8, and watches (ie, monitors) whether there is a sound source in the monitoring area 8. Therefore, the unmanned aerial vehicle 10 can be positioned as a monitoring drone that monitors the monitoring area 8.

  The unmanned aerial vehicle 10 picks up a voice calling the unmanned aerial vehicle 10 (for example, the shout “Oh!” Or “Help!” Given by the user HM1 shown in FIG. 4) in the microphone array MA (St2). It is preferable that such a voice calling the unmanned aerial vehicle 10 is registered in advance in, for example, the memory 23 of the unmanned aerial vehicle 10, and when such a voice is recognized by the voice signal processing unit 26 or the voice recognition unit 26a, The aircraft 10 may proceed to the next process (that is, the process of Step St3).

  When analyzing whether or not the voice calling the unmanned aerial vehicle 10 is included in the voice signal processing unit 26 or the voice recognition unit 26a, a learning obtained by machine learning of past sample data using artificial intelligence is used. The determination may be made according to a model. The analysis may be performed by the analysis cloud server 40 connected to the unmanned aerial vehicle 10 via a network such as wireless communication (see FIG. 7). Learning artificial intelligence may be performed using one or more statistical classification techniques. Statistical classification techniques include linear classifiers, support vector machines, quadratic classifiers, kernel density estimation, decision trees, artificial neural networks. Networks (artificial neural networks), Bayesian techniques and / or networks, hidden Markov models, binary classifiers, multi-class classifiers, Examples include a clustering technique, a random forest technique, a logistic regression technique, a linear regression technique, and a gradient boosting technique. However, the statistical classification technique used is not limited to these.

  The unmanned aerial vehicle 10 detects (estimates) the sound source direction of the sound (that is, the direction in which the user HM1 is located) in the sound signal processing unit 26 using the sound data of the sound collected in Step St2. The directivity of the sound data is formed in the direction of the detected sound source (St3).

  When the directions of a plurality of sound sources are detected (estimated) in Step St3, the unmanned aerial vehicle 10 may form the directivity of the sound data in the directions of the respective sound sources. Further, when the directions of the plurality of sound sources are detected (estimated) in step St3, the unmanned aerial vehicle 10 may fly to an intermediate position (for example, an intermediate or centroid position) of the plurality of sound sources. Alternatively, one or more other unmanned aerial vehicles 10 hovering or circulating around the monitoring area 8 may be called by wireless communication of a predetermined emergency signal.

  After the process of step St3, the unmanned aerial vehicle 10 starts flying in the direction of the sound source detected in step St3 and continues flying until it approaches the position of the sound source (that is, the user HM1) (St4).

  When the unmanned aerial vehicle 10 is sufficiently close to the user HM1 (for example, when the area occupied by the user HM1 in the image captured by the camera CA exceeds a predetermined ratio), the unmanned aerial vehicle 10 outputs a default (for example, a response request announcement) sound. Output is made from each of the speakers SP1 to SP4 (St5). In addition, the unmanned aerial vehicle 10 may output the predetermined sound from some of the speakers including at least one of the speakers SP1 to SP4 in step St5. The voice of the response request announcement is, for example, a voice saying, "If there is a business, please say" come here. "

  Here, in response to the voice of the response request announcement output from the unmanned aerial vehicle 10 in step St5, the user HM1 emits the voice of "come here" specified in the response request announcement. The unmanned aerial vehicle 10 picks up the user's voice "come here" at the microphone array MA (St6).

  After step St6, the unmanned aerial vehicle 10 analyzes whether or not the sound data collected by the microphone array MA includes “come here” in the voice recognition unit 26a (St7). When the voice recognition unit 26a determines that the sound data collected by the microphone array MA does not include "come here" (St7, NO), the unmanned aerial vehicle 10 leaves the place (St7, NO). St9). After that, the process of the unmanned aerial vehicle 10 returns to Step St1. When analyzing whether or not "come here" is included in the voice recognition unit 26a, it may be determined according to a learning model obtained by machine learning of past sample data using artificial intelligence. Good. The analysis may be performed by the analysis cloud server 40 connected to the unmanned aerial vehicle 10 via a network such as wireless communication (see FIG. 7). Learning artificial intelligence may be performed using one or more statistical classification techniques. Statistical classification techniques include linear classifiers, support vector machines, quadratic classifiers, kernel density estimation, decision trees, artificial neural networks. Networks (artificial neural networks), Bayesian techniques and / or networks, hidden Markov models, binary classifiers, multi-class classifiers, Examples include a clustering technique, a random forest technique, a logistic regression technique, a linear regression technique, and a gradient boosting technique. However, the statistical classification technique used is not limited to these.

  On the other hand, when the unmanned aerial vehicle 10 determines that the sound data collected by the microphone array MA includes “come here” (St7, YES), the unmanned aerial vehicle 10 receives the input command requested by the user HM1 (St7, YES). St8) If necessary, a process corresponding to the input command (for example, calling another unmanned aerial vehicle 10, outputting a predetermined alarm sound acoustically, turning on a light such as an LED (Light Emission Diode)). Execute Thus, the user HM1 can notify, via the unmanned aerial vehicle 10, that the user HM1 is in a state of emergency to a third party such as the police, thereby improving the convenience.

  Here, the input command may be, for example, a command based on a gesture (action), a command based on a facial expression, a command based on a voice, or the like. When analyzing whether or not the command is an input command, the command may be determined according to a learning model obtained by machine learning of past sample data using artificial intelligence. The analysis is performed by the image analysis unit 28a when using an image such as a command by a gesture or a command by a facial expression, and is performed by the voice signal processing unit 26 or the voice recognition This may be performed in the unit 26a. Further, the analysis may be performed by an analysis cloud server 40 connected to the unmanned aerial vehicle 10 via a network such as wireless communication (see FIG. 7). Learning artificial intelligence may be performed using one or more statistical classification techniques. Statistical classification techniques include linear classifiers, support vector machines, quadratic classifiers, kernel density estimation, decision trees, artificial neural networks. Networks (artificial neural networks), Bayesian techniques and / or networks, hidden Markov models, binary classifiers, multi-class classifiers, Examples include a clustering technique, a random forest technique, a logistic regression technique, a linear regression technique, and a gradient boosting technique. However, the statistical classification technique used is not limited to these.

  The command by the gesture (action) is a command to request the unmanned aerial vehicle 10 to rescue or the like by the action of the user HM1 waving, for example. This is effective in that the user HM1 can request rescue or the like to the unmanned aerial vehicle 10 even when the user HM1 is injured at his mouth and cannot speak or is difficult to speak.

  The command based on the facial expression is rescued to the unmanned aerial vehicle 10 when, for example, only the face of the user HM1 is seen from the unmanned aerial vehicle 10 (for example, when the user HM1 falls down and the camera CA cannot capture an image other than the user's face). Is a command that asks for The facial expression corresponds to, for example, closing one eye, laughing, angry, sad, closing the eyelids, etc., but is not limited thereto.

  The command by voice is a command to request the unmanned aerial vehicle 10 for rescue or the like according to, for example, a voice emitted by the user HM1. The voice used for the voice command is, for example, a word uttered by the user HM1, a communication by a sound, or a combination thereof, but is not limited thereto. It should be noted that the communication by sound can be such that the user hits once when the content is positive, and hits twice when the content is negative.

  In FIG. 6, when the unmanned aerial vehicle 10 is sufficiently close to the user HM1 (for example, when the area occupied by the user HM1 in the image captured by the camera CA exceeds a predetermined ratio), the unmanned aerial vehicle 10 defaults (for example, a response request announcement). ) Is output from each of the speakers SP1 to SP4 (St5A). In addition, in step St5A, the unmanned aerial vehicle 10 may output the predetermined sound from some of the speakers including at least one, but not all of the speakers SP1 to SP4. The sound of the response request announcement is, for example, a sound "Wave your hand if there is a business requirement."

  Here, the user HM1 performs a motion (action) of “Please wave your hand” specified in the response request announcement with respect to the voice of the response request announcement output from the unmanned aerial vehicle 10 in Step St5A. The unmanned aerial vehicle 10 analyzes the image captured by the camera CA in the image analysis unit 28a, and analyzes whether the user HM1 has waved his / her hand in response to "Wave your hand" (St7A). When it is determined that the user HM1 has not waved by analyzing the image captured by the camera CA (St7A, NO), the unmanned aerial vehicle 10 leaves the place (St9). After that, the process of the unmanned aerial vehicle 10 returns to Step St1.

  On the other hand, when the unmanned aerial vehicle 10 determines that the user HM1 has waved his hand based on the analysis of the image captured by the camera CA (St7A, YES), the unmanned aerial vehicle 10 further approaches the direction of the sound source where the user HM1 is located (St10). . After Step St10, the process of the unmanned aerial vehicle 10 proceeds to the process of Step St5 shown in FIG.

  As described above, the unmanned aerial vehicle 10 according to the first embodiment has the microphone array MA arranged on the bottom surface side of the housing 11 and collecting the sound of the monitoring area 8. The unmanned aerial vehicle 10 includes an audio signal processing unit 26 (an example of an audio processing unit) that estimates a direction of a sound source generated in the monitoring area 8 based on sound data collected by the microphone array MA. The unmanned aerial vehicle 10 has a drone flight control unit 22 (an example of a flight control unit) that controls hovering or round flight in the monitoring area 8 and controls flight movement in the direction of the estimated sound source. The unmanned aerial vehicle 10 has sound output units 291 to 294 that are arranged on the bottom side of the housing 11 and output predetermined sounds (see below) from the speakers SP1 to SP4, respectively. The unmanned aerial vehicle 10 includes a drone control management unit 21 (an example of a control unit) that receives and executes a process that satisfies the needs of the user after outputting a predetermined voice.

  Thereby, the unmanned aerial vehicle 10 can constantly hover or circulate around the monitoring area 8 and quickly detect the direction of the user HM1 seeking rescue or the like. Accordingly, the unmanned aerial vehicle 10 flies and moves so as to approach the direction of the user HM1, and after sufficiently approaching the user HM1, executes a process of appropriately accepting the needs of the user HM1 and adapting to the needs. Therefore, it is possible to suppress a decrease in user convenience.

  In addition, the unmanned aerial vehicle 10 outputs a predetermined voice (for example, a voice saying “Please come here when there is a business”) to the output default voice by the user HM1. When the voice signal processing unit 26 recognizes (for example, the voice “come here”), the input command of the process from the user HM1 is received. Thereby, the unmanned aerial vehicle 10 can properly authenticate that it is the user HM1 requesting rescue or the like by using the voice uttered by the user HM1 with respect to the predetermined voice, and accepts the necessary processing of the user HM1. And user convenience can be improved.

  Further, the unmanned aerial vehicle 10 is disposed on the bottom side of the housing 11 and has a camera CA (for example, an omnidirectional camera) that captures an image of the monitoring area 8, an image analysis unit 28 a that analyzes the captured image of the monitoring area 8, Has further. The unmanned aerial vehicle 10 performs a predetermined action (for example, an action of waving a hand) performed by the user HM1 with respect to the output default voice (for example, a voice of “please wave your hand when there is a requirement”). ) Is recognized by the image analysis unit 28a, the flight control unit 22 instructs the drone flight control unit 22 to perform a flight movement to further approach the sound source where the user HM1 is located. Accordingly, the unmanned aerial vehicle 10 can perform the behavior performed by the user HM1 even when the user HM1 is injured at his mouth and cannot speak or is difficult to speak, or is not sufficiently close to the user HM1 ( By the action, the user HM1 requesting rescue or the like can be properly authenticated. Accordingly, the unmanned aerial vehicle 10 can further approach the user HM1 in order to receive the necessary processing of the user HM1, and for example, can rescue the unmanned aerial vehicle 10 even if the user does not make a loud voice because the unmanned aerial vehicle 10 approaches the user HM1. And other necessary processes can be easily received.

  In addition, when the unmanned aerial vehicle 10 approaches the user HM1 located at the sound source, the unmanned aerial vehicle 10 generates a predetermined second voice (for example, a voice saying “If there is a business, please say“ come here ””). The output from the speakers SP1 to SP4 is instructed to the corresponding audio output units 291 to 294, and a predetermined audio (for example, "come here") issued by the user with respect to the predetermined second audio output in accordance with the instruction. Is recognized by the audio signal processing unit 26, the input of the process is accepted. Thereby, the unmanned aerial vehicle 10 is strictly authenticated as the user HM1 requiring rescue or the like by using not only the user's action for the predetermined second voice but also the voice of the user HM1 for the predetermined voice. It is possible to receive necessary processing of the user HM1 and to improve user convenience.

  In addition, the unmanned aerial vehicle 10 emphasizes the sound data in the estimated direction of the sound source in the sound signal processing unit 26 by, for example, forming the directivity of the sound data. Thereby, the unmanned aerial vehicle 10 can emphasize the voice emitted by the user HM1 existing in the direction of the sound source, so that the unmanned aerial vehicle 10 suppresses the omission of the voice emitted by the user HM1 requiring rescue or the like, or improves the listening accuracy of the voice. Thus, the voice recognition processing can be properly performed.

  The unmanned aerial vehicle 10 further includes a rotary wing mechanism 24 for performing flight movement. The unmanned aerial vehicle 10 suppresses the rotating sound generated when the rotating wing rotates from the rotating wing mechanism 24, and estimates the direction of the sound source in the audio signal processing unit 26. Thereby, the unmanned aerial vehicle 10 can properly detect (estimate) the direction of the sound source in the monitoring area 8 by eliminating the influence of the rotating sound having a large sound pressure level generated from the rotors PR1 to PR4 such as the rotor.

  Although various embodiments have been described with reference to the drawings, it is needless to say that the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive various changes, modifications, substitutions, additions, deletions, and equivalents within the scope of the claims. Naturally, it is understood that they belong to the technical scope of the present disclosure. Further, the components in the above-described various embodiments may be arbitrarily combined without departing from the spirit of the invention.

  In FIG. 3, an operation terminal for operating the unmanned aerial vehicle 10 is described as an example of a wireless communication partner (that is, a connection destination) of the unmanned aerial vehicle 10. It may be connected to a server device (not shown) arranged in a monitoring room where it is located. The unmanned aerial vehicle 10 may send the sound pressure heat map data generated by the audio signal processing unit 26 to the server device and display the data on a display (not shown) connected to the server device. Thereby, the observer in the monitoring room can accurately and visually grasp the state of the sound source in the monitoring area 8 where the unmanned aerial vehicle 10 is flying, and can support necessary measures and the like. Furthermore, the unmanned aerial vehicle with a sound collection function according to the present disclosure flies round the monitoring area to search for the presence or absence of a home delivery order, and picks up and detects a voice for a home delivery order instead of detecting a voice for a rescue order. May be used.

  FIG. 7 is a block diagram illustrating a hardware configuration example of the unmanned aerial vehicle and the cloud server according to the first embodiment. As described above, the unmanned aerial vehicle 10 may be connected to the analysis cloud server 40 via a network such as wireless communication. The cloud server 40 receives the sound data or the image data transmitted from the unmanned aerial vehicle 10 and performs various analyzes.

  The cloud server 40 has a configuration including a processor PRC2, a memory 43, and a communication unit 45. The processor PRC2 is configured to have higher performance than the processor PRC1 of the unmanned aerial vehicle 10 by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). . The processor PRC2 can functionally realize the voice recognition unit 46 and the image analysis unit 48 by reading and executing programs and data stored in the memory 43 in advance.

  The voice recognition unit 46 uses the voice recognition dictionary (not shown) registered in the memory 43 in advance, for example, to voice-recognize the sound data of the monitoring area 8 transmitted after being subjected to the emphasis processing by the unmanned aerial vehicle 10. . The voice recognition accuracy of the voice recognition unit 46 in the cloud server 40 is superior to the voice recognition accuracy of the voice recognition unit 26a in the unmanned aerial vehicle 10. The voice recognition unit 46 sends the voice recognition result to the unmanned aerial vehicle 10 via the communication unit 45, or temporarily stores the result in the memory 43.

  The image analysis unit 48 analyzes an image captured by the camera CA mounted on the unmanned aerial vehicle 10, and analyzes and specifies a command by a gesture of a user such as a rider and a command by a facial expression. In addition, the image analysis unit 48 sends the analysis result to the unmanned aerial vehicle 10 via the communication unit 45, or temporarily stores the analysis result in the memory 43.

  The memory 43 stores programs and data necessary for controlling execution of operations (processing) of each unit realized by the processor PRC2. The memory 43 may be a computer-readable recording medium, such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), and It may include at least one of a flash memory such as a USB memory.

  The communication unit 45 communicates with the unmanned aerial vehicle 10 wirelessly connected to the network NW via the network NW according to a wireless communication standard of a wireless LAN (Local Area Network) such as Wi-fi (registered trademark). Do. The connection between the communication unit 45 and the network NW may be wired or wireless. Alternatively, the communication unit 45 may perform wireless communication directly with the unmanned aerial vehicle 10 to perform communication. The communication unit 45 receives various data (sound data and image data) used for the processing of the processor PRC2 from the unmanned aerial vehicle 10, and transmits data of the processing result of the processor PRC2 to the unmanned aerial vehicle 10.

  In this way, for example, processes that are expected to have a processing load on the unmanned aerial vehicle 10, such as a voice recognition process for sound data and a command identification process based on image data analysis, do not cause the unmanned aerial vehicle 10 to execute all processes. By partially performing distributed processing in the cloud server 40, the processing load on the unmanned aerial vehicle 10 can be reduced. In addition, since the cloud server 40 can more accurately perform voice recognition and specify a command based on image analysis, it is expected that convenience for a user such as a rider who requests the unmanned aerial vehicle 10 for rescue can be improved.

  INDUSTRIAL APPLICABILITY The present disclosure can constantly hover or circulate around a monitoring area, quickly detect the direction of a user seeking rescue or the like, appropriately receive a user's needs, and execute a process adapted to the needs, thereby improving the user's convenience. It is useful as an unmanned aerial vehicle with a sound collection function that suppresses the deterioration of performance.

DESCRIPTION OF SYMBOLS 10 Unmanned aerial vehicle 11 Housing 21 Drone control management part 22 Drone flight control part 24 Rotor wing mechanism 25 Wireless communication part 26 Audio signal processing part 26a, 46 Speech recognition part 27 Camera control part 28 Image processing part 28a, 48 Image analysis part 241 Motor group 291, 294 Audio output unit CA Camera MA Microphone array SP1, SP4 Speaker

Claims (6)

A microphone array arranged on the bottom side of the housing and collecting the sound of the monitoring area,
A sound processing unit that estimates a direction of a sound source generated in the monitoring area based on sound data collected by the microphone array;
A flight control unit that controls hovering or patrol flight of the monitoring area and controls flight movement in the direction of the estimated sound source,
An audio output unit disposed on the bottom side of the housing and outputting a predetermined audio from a speaker;
A control unit that receives and executes an input of a process that satisfies a user's needs after the output of the predetermined voice,
Unmanned aerial vehicle with sound collection function. The control unit, when the voice processing unit recognizes a predetermined voice issued by the user with respect to the output default voice, accepts an input of the process,
An unmanned aerial vehicle with a sound collection function according to claim 1. A camera arranged on the bottom side of the housing and imaging the monitoring area,
An image analysis unit that analyzes the captured image of the monitoring area, further comprising:
The control unit, when recognizing a predetermined action performed by the user with respect to the output default sound by the image analysis unit, a flight movement for further approaching the direction of the sound source where the user is located. To the flight control unit,
An unmanned aerial vehicle with a sound collection function according to claim 1. The control unit, when approaching a user located at the sound source, instructs the audio output unit to output a predetermined second audio from the speaker, and outputs the predetermined second audio in accordance with the instruction. When a predetermined voice issued by the user is recognized by the voice processing unit, an input of the process is accepted.
An unmanned aerial vehicle with a sound collection function according to claim 3. The sound processing unit emphasizes sound data in the direction of the estimated sound source,
The unmanned aerial vehicle with a sound collection function according to any one of claims 1 to 4. A rotary wing mechanism for performing the flying movement,
The sound processing unit estimates the direction of the sound source by suppressing the rotation sound generated when the rotor blades rotate from the rotor blade mechanism,
An unmanned aerial vehicle with a sound collection function according to any one of claims 1 to 5.

Images