JP2017083216A - Sound operation system, operation method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To operate an object device by utilizing a simple operation sound.SOLUTION: A plurality of object devices 13a to 13e mounted with microphones 201a to 201e respectively are arranged in a space. A user 10 holds a reference device 15 mounted with a microphone 301. The user causes his/her fingers to sound at a first operation position 18a in order to operate the object device 13a, and then, generates operation sounds by causing his/her fingers to sound at a second operation position 18b heading to the object device 13d from the operation position. A network server 11 calculates a sound source distance from the operation positions 18a, 18b to the object device with respect to each object device on the basis of a difference between the propagation time to the reference device of the operation sound and the propagation time to the object device so that the object device 13d is specified. The network server transmits an operation command to the object device 13d.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technology for controlling a target device with a simple operation sound, and further to a technology for specifying a target device intended by a user with a non-directional operation sound.

  Home appliances such as televisions, air conditioners, and luminaires can be controlled by dedicated or integrated infrared remote controllers. In recent years, systems that individually operate devices such as IoT (Internet of Things) and M2M (Machine to Machine) are connected via a network, and data generated by each device is integrated in real time and used for device control. Has been developing. The home appliance can also be controlled through a portable electronic device built into the IoT / M2M system.

  Patent Document 1 discloses a method for recognizing and operating a voice command input to a mobile terminal device. Patent Document 2 discloses a voice command remote control device that operates a VTR, a television receiver, or the like with a voice command. According to this document, when a user inputs a command for an electronic device by voice after the operation unit is directed to a desired electronic device and the user presses the push button switch, the audio signal is wirelessly transmitted to the base unit and is transmitted to the VTR. The start of the playback operation is described.

JP 2006-221270 A Japanese Patent Laid-Open No. 11-120647

  One problem in the operation of the home appliance is that the user who intends to operate it can operate immediately on the spot. The dedicated remote controller cannot be said to be within the reach of the user's hand at the moment of operation, and the button operation cannot sufficiently meet the demand for prompt operation. The same applies to the method of sending an operation command from the portable electronic device constituting the IoT / M2M system.

  One method for responding to such a problem is a method of photographing and manipulating a user's gesture with a camera. There is also a method in which a home appliance or an IoT / M2M system equipped with a voice recognition device recognizes a language command such as “open” or “close” uttered by a user and performs a corresponding operation. However, the image recognition apparatus and the voice recognition apparatus have problems such as the large scale of the system and the recognition accuracy.

  Therefore, an object of the present invention is to provide a method for operating a target apparatus using simple operation sounds. A further object of the present invention is to provide a method capable of operating a target device immediately when the user intends. It is another object of the present invention to provide an acoustic operation system, a control device, a network server, a portable electronic device, and a computer program that realize such a method.

  The present invention provides an acoustic operation system for operating a target target device selected by acoustic operation from among a plurality of target devices. The acoustic operation system receives operation sound data using a plurality of target microphones that detect operation sounds generated by the sound source at the first operation position and the second operation position, and receives operation sound data from the sound source to the target microphone. A sound source distance calculation unit that calculates a first sound source distance from the first operation position to the target microphone and a second sound source distance from the second operation position to the target microphone based on the propagation time; and a first sound source A target device specifying unit that specifies a target device related to the target microphone selected based on the distance, the second sound source distance, and the moving distance from the first operation position to the second operation position;

  Although the operation sound does not have directivity in space, the acoustic operation system calculates the propagation time and movement distance of the operation sound generated at the first operation position and the second operation position, so that the user can obtain the operation sound. Extract the intended direction information. In the present invention, since it is not necessary to perform voice recognition, it can be realized only by generating an operation sound by a simple user's movement of the user, such as ringing a finger, hitting a palm, or hitting a casing of a portable electronic device. Therefore, it is convenient for the user to operate immediately on the spot. In the present invention, the voice generated by the user can be used as a simple operation sound. The target microphone may be mounted on the target device.

  The second operation position can be set so as to exist in a direction from the target microphone associated with the target target device toward the first operation position. In this case, after the user performs an acoustic operation at the first operation position, the user can easily select the second operation position using the target microphone associated with the target device as a guide, so that the user can easily perform the target operation on the acoustic operation system. Can communicate the intention of selecting a device.

  The sound source distance calculation unit can calculate the first sound source distance and the second sound source distance from the time difference between the operation sound generation time and the operation sound detection time detected by the target microphone. The sound source distance calculation unit receives operation sound data using a reference microphone disposed at a known reference distance from the sound source at the first operation position and the second operation position, and the operation sound detected by the target microphone. The first sound source distance and the second sound source distance can be calculated using the detected time, the detection time of the operation sound detected by the reference microphone, and the reference distance. In this case, since it is not necessary to acquire the generation time of the operation sound generated by the sound source, a human body part such as a fingertip or a palm can be used as the sound source.

  The reference microphone can be mounted on a wearable device worn by the user, and the sound source can be the fingertip or palm of the user. In this case, the sound source can be easily moved to the second operation position while maintaining the reference distance between the first operation position and the second operation position. Furthermore, the operation sound generated by the fingertip or palm enables an intuitive and quick operation. The reference microphone can be mounted on a portable electronic device that is held and used by the user, and the sound source can be a part of the casing of the portable electronic device.

  Providing an acoustic operation recognition unit that determines the effectiveness of an acoustic operation based on a temporal generation pattern of an operation sound from detection sounds including noise detected by the target microphone and the reference microphone can prevent erroneous operation due to noise. . The acoustic operation recognizing unit generates a pair of detection sounds detected by the target microphone and the reference microphone for the same generated sound based on a difference between the detection time of the detection sound detected by the target microphone and the detection time of the detection sound detected by the reference microphone. When it is determined not to form a pair of operation sounds, the acoustic operation can be invalidated. In this case, it is possible to prevent erroneous operation when noise is mistakenly recognized as an operation sound or when the operation sound cannot be recognized from the acoustic data of the target microphone.

  When the time difference between the detection time of the preceding detection sound detected by the reference microphone or the target microphone and the detection time of the subsequent detection sound is equal to or greater than a predetermined value, the acoustic operation recognition unit discards the detection sound and generates noise from the detection sound. Can be removed. The acoustic operation system may include an operation command generation unit that outputs operation commands corresponding to a plurality of operation sounds respectively generated at the first operation position and the second operation position to the target device.

  A second aspect of the present invention provides a method in which an acoustic operation system operates a target target device selected from a plurality of target devices according to an acoustic operation. The plurality of target devices detect the operation sound generated by the sound source at the first operation position and the operation sound generated at the second operation position. Subsequently, based on the propagation time of the operation sound from the sound source to the target device, the first sound source distance from the first operation position to the target device and the second sound source distance from the second operation position to the target device are calculated. To do. The movement distance between the first operation position and the second operation position is acquired. Subsequently, the target device is specified based on the first sound source distance, the second sound source distance, and the movement distance. Subsequently, an operation command is sent to the target device.

  When the second operation position is present in the direction from the first operation position toward the target device, an evaluation parameter obtained by subtracting the reference distance from the difference between the first sound source distance and the second sound source distance is set to a predetermined threshold value. The target device can be specified by comparing with. Alternatively, the target target device can be specified by comparing the evaluation parameters calculated for each of the plurality of target devices.

  The first detection time at which the operation sound is detected at a known reference distance from the sound source is acquired, the second detection time at which the plurality of target devices detect the operation sound is acquired, the second detection time and the first detection time By calculating the time difference, the first sound source distance and the second sound source distance can be calculated. A first number of operation sounds are detected at the first operation position, a second number of operation sounds are detected at the second operation position, and a plurality of operation commands are combined by combining the first number and the second number. And the acoustic operation can be disabled when the first number or the second number is equal to or greater than a predetermined value.

  According to the present invention, it is possible to provide a method for operating a target apparatus using simple operation sounds. Furthermore, according to the present invention, it is possible to provide a method that allows the target device to be operated immediately when the user intends. Furthermore, according to the present invention, an acoustic operation system, a control device, a network server, and a portable electronic device that realize such a method can be provided.

It is a figure for demonstrating the outline | summary of the acoustic operation system 100 concerning this Embodiment. FIG. 3 is a diagram for explaining an example of a reference device 15 constituting the acoustic operation system 100. It is a figure for demonstrating the method to detect the operation sound and to calculate the sound source distance d from the sound source 17 to the target apparatus 13. FIG. It is a figure for demonstrating the method to specify the target apparatus 13d which intends operation from the sound source distance d and the movement distance d0. 1 is a schematic functional block diagram for explaining an example of a configuration of an acoustic system 100. FIG. It is the figure which showed typically an example of the waveform of the detection sound 30 which the reference device 15 and the target apparatus 13 detected. FIG. 6 is a diagram for explaining a method by which the acoustic operation recognition unit 403 determines the effectiveness of the acoustic operation for each target device 13. It is a flowchart for demonstrating the method by which the acoustic operation recognition part 403 judges the effectiveness of acoustic operation. 3 is a flowchart for explaining the operation of the acoustic operation system 100. 3 is a schematic functional block diagram for explaining the configuration of an acoustic operation system 600. FIG. 4 is a schematic functional block diagram for explaining the configuration of an acoustic operation system 700. FIG.

[Definition]
The meanings of terms used in this specification will be described. The operation sound means a discrete sound on the time axis that is generated by a sound operation by the user. In the acoustic operation, the user generates one or a plurality of operation sounds at each of the first operation position and the second operation position, and selects a target target device intended for the operation from the plurality of target devices. Means the operation to be controlled. The generated sound means all sounds generated in the acoustic operation execution environment including the operation sound and noise. The detected sound means a generated sound detected by the acoustic operation system.

[Sound operation system]
FIG. 1 is a diagram for explaining the outline of the acoustic operation system 100 according to the present embodiment. FIG. 2 is a diagram for explaining an example of the reference device 15 constituting the acoustic operation system 100. In the three-dimensional space in which the user 10 exists, there are target devices 13a to 13e, a network server 11, and a reference device 15 to be operated. The target devices 13a to 13e are not particularly limited, but may be home appliances such as an air conditioner, a lighting device, a television, a door locking device, and a curtain as an example. The target devices 13a to 13e are equipped with microphones 201a to 201e, respectively.

  The microphones 201a to 201e do not necessarily have to be mounted on the target devices 13a to 13e, but only need to be connected to the corresponding target devices 13a to 13e so that the user can understand the relationship between them. In the following, when it is not necessary to individually distinguish the target devices 13a to 13e and the microphones 201a to 201e, they will be collectively referred to as the target device 13 and the microphone 201. When the target device 13 is described, a single target device may be indicated and a plurality of target devices may be indicated.

  The user 10 is about to send an operation command such as on / off or volume adjustment to the target device 13d (for example, a television) selected from the plurality of target devices 13 through the network server 11. The target apparatus 13 may use an operation with an existing remote controller or network device. The user 10 performs selection of the target device 13d and transmission of an operation command by acoustic operation at the first operation position 18a and the second operation position 18b.

  A virtual directional straight line 19 is defined between the user 10 and the target device 13d for determining the second operation position 18b for performing the acoustic operation after performing the acoustic operation at the first operation position 18a. The user 10 can determine the second operation position 18b while paying attention to the direction straight line 19 after performing the acoustic operation at the first operation position 18a. The distance between the first operation position 18a and the second operation position 18b is referred to as a movement distance d0.

  The reference device 15 is present in the hand 10a of the user 10 in a manner such as wearing or holding. The reference device 15 includes a microphone 301 and provides reference information for measuring the distance from the sound source 17 that generated the operation sound 40 to each target device 13. The reference device 15 can be a portable electronic device that is convenient for using the hand 10a of the user 10 as a sound source and that can be easily moved from the first operation location 18a to the second operation location 18b. As an example of a portable electronic device, a wearable device 15a attached to an arm like a wristwatch can be employed (FIG. 2A).

  In FIG. 2, the wearable device 15a includes a microphone 301a. When the wearable device 15a is employed as the reference device 15, the user can configure the sound source 17a that generates the operation sound 40a with the fingertip by repelling the middle finger pressed against the thumb. One or a plurality of operation sounds 40 a generated by the sound source 17 propagates in the air in a spherical shape and reaches the microphones 301 and 201.

  The microphone 301 may be present at a known reference distance r from the sound source 17 at least when the user generates the operation sound 40. The microphone 201 exists at an unknown distance from the sound source 17 during acoustic operation. In the case of the wearable device 15a, the reference distance r between the fingertip serving as the sound source 17a and the microphone 301a can be assumed in advance as a general human palm size. Also, the wearable device 15a is convenient because the reference distance r does not change even when the user 10 moves in the space or changes the posture. The sound source 17a may be configured using an instrument or percussion instrument worn by the user on the fingertip.

  The wearable device as the reference device 15 may be worn on the face like glasses or on the waist like a belt. The sound source 17a need not be limited to the fingertip as long as it exists at a known reference distance r from the microphone 301a during the acoustic operation, and may be a voice of the user 10 or a palm that is clapped. Such an acoustic operation can be performed intuitively immediately when the user intends, rather than operating from a remote controller or network device. The reference device 15 need not be limited to the wearable device 15a as long as the microphone 301 exists at a known reference distance r from the sound source 17 when generating the operation sound 40 at least.

  FIG. 2B shows a portable electronic device 15b that is held and used by the user 10 as another example of the reference device 15. The portable electronic device 15b can be a smartphone or a tablet terminal equipped with the microphone 301b. The user generates an operation sound 40b by hitting a predetermined part of the housing at a known reference distance from the microphone 301b with a finger or a pen. A predetermined part of the case to be hit is the sound source 17b. The operation sound 40b propagates through the air and reaches the microphone 201, and propagates through the air and the casing to reach the microphone 301.

  The sound sources 17a and 17b can generate the operation sounds 40a and 40b with a simple movement of the human body. Also, the wearable device 15a and the portable electronic device 15b are always close to the user. Therefore, it is convenient to perform an acoustic operation at the first operation position 18a and the second operation position 18b on the direction straight line 19 immediately when the user intends. Further, the reference distance r from the sound sources 17a and 17b to the microphones 301a and 301b can be assumed in advance, and it is convenient to maintain the reference distance r at the first operation position 18a and the second operation position 18b. However, the present invention does not limit the reference device 15 and the sound source 17 to the ranges exemplified here, and various reference devices 15 and the sound sources 17 that can be easily conceived by those skilled in the art without departing from the spirit of the present invention. can do.

  Returning to FIG. 1, the acoustic operation system 100 is based on the premise that a plurality of target devices 13 do not exist on the direction line 19 in principle. The reference device 15 and the target device 13 are connected to the network server 11 via a wireless network. The standard of the wireless network is not particularly limited, and a wireless LAN, Bluetooth (registered trademark), or the like can be adopted. The present invention does not exclude the application of a wired network. In particular, since the network server 11 and the target device 13 do not need to change their positions, even if they are connected via a wired network, there is no problem in acoustic operation.

  The network server 11 exchanges data necessary for processing of the operation sound 40 with the reference device 15 and the target device 13 through the network in real time. In one example, the network server 11 processes the operation sound 40 and generates an operation command for operating the target device 13. The reference device 15, the target device 13, and the network server 11 can be configured as an IoT / M2M system in an example. In another example, without using the network server 11, the data required for processing the operation sound 40 can be processed by either the reference device 15 or the target device 13, or can be distributed and processed by both. .

  The operation sound 40 generated by the sound source 17 propagates to the target device 13 as a substantially spherical wave. The sound source 17 can be an omnidirectional point sound source. Since the acoustic operation system 100 uses the operation sound 40 as a physical sound that exists discretely on the time axis, the operation sound 40 needs to have information on a direction for associating the user 10 with the target device 13. Absent. Therefore, the target device 13d that the user 10 intends to operate cannot be specified only by detecting the operation sound 40 by the microphone 201 of each target device 13. Here, attention is paid to the fact that, for example, the microphones 201c and 201d that detect the same operation sound 40 have different propagation times because the distances from the sound source 17 are different. In this embodiment, as will be described below, the target device 13d intended for the operation is specified by distinguishing the target device 13d existing on the direction straight line 19 from the other target devices using the difference in propagation time. .

  The acoustic operation generates an operation sound 40 at the first operation position 18a and the second operation position. The user 10 generates the first operation sound 40 at the first operation position 18a and performs the second operation at the second operation position 18b on the directional line 19 connecting the first operation position 18a and the microphone 201d of the target device 13d. The operation sound is generated. At this time, both the microphone 301 and the sound source 17 of the reference device 15 move from the first operation position to the second operation position while maintaining a known reference distance r.

  The reference device 15 and the target device 13 detect the same operation sound 40 at different detection times. The reference device 15 and each target device 13 instantaneously convert the operation sound 40 into data and send it to the network server 11. The network server 11 is configured based on the difference in the propagation time of the operation sound from the sound source 17 to the microphone 201 mounted on each target device 13 at the first operation position 18a and the second operation position 18b, and the reference distance r. The distance between the microphone 201 and the sound source 17 can be calculated. The distance between each microphone 201 and the sound source 17 is referred to as a sound source distance d (FIG. 3). When the size of the target device 13 is sufficiently small with respect to the sound source distance d, the sound source distance d can be handled as the distance between the target device 13 and the sound source 17.

  The sound source distance d calculated at each of the two operation positions 18a and 18b gives information for distinguishing the target device 13d that the user intends to operate from other target devices. The network server 11 sends a predefined operation command to the identified target device 13d. The acoustic operation may generate one operation sound 40 at each of the first operation position 18a and the second operation position 18b.

  In this case, one piece of information for changing the current state can be sent to the target device 13 by an operation command. Specifically, it is possible to send an operation command for stopping the target device that is operating or operating the target device that is stopped. In order to further increase the number of operation commands, a plurality of operation sounds 40 can be generated continuously at the first operation position 18a and the second operation position 18b, respectively. A plurality of operation commands can be associated with a plurality of combinations of the number of operation sounds 40 at each of the first operation position 18a and the second operation position 18b.

[Calculation of sound source distance]
Next, a method for detecting the operation sound 40 and calculating the sound source distance d from the sound source 17 to the target device 13 will be described with reference to FIG. The generation time of the operation sound 40 is t0, the reference distance between the sound source 17 and the microphone 301 is r, and the sound source distance between the sound source 17 and the microphone 201 is d. 3A is a diagram corresponding to the spatial positional relationship among the microphone 301, the sound source 17, and the microphone 201, and FIG. 3B is a diagram in which they are arranged on the time axis. The microphone 301 exists on a circle 16 having the sound source 17 as a center and a reference distance r as a radius when performing acoustic operation at least.

The user 10 performs an acoustic operation at an arbitrary occurrence time t0. The operation sound 40 generated by the sound source 17 propagates through the space at the sound speed s. The microphone 301 detects the operation sound 40 at the detection time t1, and the microphone 201 detects the same operation sound 40 at the detection time t2. The time tt1 for the operation sound 40 to propagate to the microphone 301 is tt1 = t1−t0 = r / s, where s is the speed of sound in the air. Accordingly, when the detection sound of the microphone 301 is used, the network server 11 can calculate the generation time t0 of the operation sound 40 by t0 = t1−r / s. The time tt2 for the operation sound 40 to propagate to the microphone 201 is tt2 = t2−t0 = d / s. Therefore, the network server 11 sets the sound source distance d to d = (t2−t0) s (1)
Can be calculated with

The network server 11 that calculates the sound source distance d using Equation (1) needs to electrically acquire the generation time t0 when the sound source 17 generates the operation sound 40. For example, when the reference device 15 is the portable electronic device 15b, a sensor that detects operation sound, vibration, pressure, or the like is provided immediately below the sound source 17b, and the time when the sensor is operated may be set as the generation time t0. it can. Here, the expression (1) is obtained by using the detection time t1 detected by the microphone 301.
d = (t2-t1) s + r (2)
Can be calculated with

  t2−t1 corresponds to a difference in propagation time of the operation sound 40 from the sound source 17 to the microphone 301 and the microphone 201. Expression (2) indicates that the sound source distance d can be calculated by obtaining the detection time t2 of the operation sound 40 at the microphone 201, the detection time t1 at the microphone 301, and the reference distance r. Expression (2) can be applied even when the sound source 17 that is difficult to measure the occurrence time t0 such as a fingertip or a palm is employed.

In the case where the reference device 15 is the portable electronic device 13b, the operation sound 40b propagates in the air and the case when a predetermined position of the case at the reference distance r is hit from the microphone 301b. When the operation sound 40 detected by the microphone 301b propagating through the casing is detected, if the sound speed of the medium of the casing is m, t0 = t1-r / m, and the network server 11 determines the sound source distance d,
d = (t2-t1) s + rs / m (3)
Can be calculated with

[Identification of target device intended for user operation]
FIG. 4 is a diagram for explaining a method of specifying the target device 13d intended for the operation from the sound source distance d and the movement distance d0. Here, a case where the target device 13d intended to be operated is distinguished from the target device 13c will be described as an example. However, the principle of distinguishing the target device 13d from other target devices can be understood in the same manner. The user 10 first generates the operation sound 40 by positioning the sound source 17 at an arbitrary operation position 18a.

  The arbitrary operation position 18a can be freely selected as a place where the user intends to operate. The network server 11 calculates sound source distances d1 and d3 from the operation position 18a to the microphones 201c and 201d by the method described with reference to FIG. Next, the user 10 generates the operation sound 40 by positioning the sound source 17 at the second operation position 18 b close to the target device 13 d on the directional line 19. The second operation position 18b can be a position farther from the target device 13d than the first operation position 18a. The movement distance d0 may be determined in advance, but in the present embodiment, the network server 11 measures an arbitrary movement distance d using the reference device 15 as described later or by another method.

  The second operating position 18b does not have to be strictly on the direction line 19. The allowable value when the second operation position 18b moves away from the directional line 17 depends on the resolution of the acoustic operation system 100 with respect to the target device 13 or the distance between the target devices 13. The network server 11 calculates the sound source distances d2 and d4 from the second operation position 18b to the microphones 201c and 201d by the method described in FIG. The reference distance r at the first operation position 18a and the reference distance r at the second operation position 18b may be different values as long as the network server 11 recognizes the distance.

  The network server 11 calculates the difference between the sound source distances d1 and d2 and the difference between the sound source distances d3 and d4. Here, the absolute value of the difference between the sound source distances d1 and d2 and the absolute value of the difference between the sound source distances d3 and d4 are Δ12 and Δ34, respectively. If the second operation position 18b is substantially on the directional line 19, Δ12 related to the target device 13d is Δ12 = abs (d1−d2) ≈d0. Further, Δ34 related to the target device 13c is smaller than the moving distance d0 because of the size relationship of the three sides of the triangle, and Δ34 = abs (d3−d4) ≠ d0.

  Here, the difference between Δ12 and Δ34 and the movement distance d0 is referred to as an evaluation parameter Z. The evaluation parameter Z of the target device 13d that the user intends to operate is smaller than the evaluation parameter Z of the other target device 13c. Using this property, the network server 11 can specify the target device 13d from the evaluation parameter Z by the following method.

  In the first method, the network server 11 calculates the evaluation parameter Z for all the target devices 13 with the sound source distance at the first operation position 18a as dp and the sound source distance at the second operation position 18b as dq. . The network server 11 sets a predetermined threshold Th for the evaluation parameter Z, and can recognize that the target device 13 that satisfies Expression (4) is the target device 13d that the user intends to operate.

Z = abs (dp−dq) −d0 <Th (4)
In Expression (4), since the absolute value of the evaluation parameter Z is evaluated with the threshold Th, it is possible to prevent any target device 13 from reacting to the acoustic operation performed carelessly by the user. On the other hand, it is necessary to set the threshold Th for each operation environment, and depending on the degree to which the second operation position 18b deviates from the direction straight line 19, the target device 13d may not be specified.

In the second method, among the evaluation parameters Z calculated for all the target devices 13, the target device 13d whose evaluation parameter Z indicates the minimum so as to satisfy Expression (5) can be specified as the operation target.
Z = abs (dp−dq) −d0 = min (5)
Since Expression (5) evaluates the evaluation parameter Z as a relative value, even if the second operation position 18b deviates from the directional line 19, any target device 13 that is closest to the operation line 19 is specified. However, any one of the target devices 13 reacts to an acoustic operation performed carelessly by the user. Which specific method is adopted can be selected based on the environment in which the acoustic operation system 100 is introduced.

[Sound operation system]
FIG. 5 is a schematic functional block diagram for explaining an example of the configuration of the acoustic operation system 100. In one example, the acoustic operation system 100 includes a plurality of target devices 13, a reference device 15, and a network server 11. The acoustic operation system 100 can also be configured without using the network server 11 as described with reference to FIGS. 10 and 11 in another example. When the pairing is completed, the target device 13, the reference device 15, and the network server 11 exchange data through the network and perform a coherent operation as the acoustic operation system 100. The speed of data exchange through the network is sufficiently shorter than the time for sound to propagate through space and solid media.

  Each of the plurality of target devices 13 includes a microphone 201 that detects the operation sound 40, an A / D conversion unit 203, a target device function 205, a target device operation unit 207, and a network interface 209. The target device function 205 corresponds to an original function of each home appliance such as a television and an air conditioner. The target device operation unit 207 receives an operation command from the network server 11 and controls the target device function 205. The target device operation unit 207 may control the target device function 205 by receiving an operation signal from an infrared remote controller.

  The reference device 15 includes a microphone 301, an A / D conversion unit 303, a motion sensor 305, an A / D conversion unit 307, and a network interface 309. The motion sensor 305 detects acceleration, angular velocity, and geomagnetism on three axes. The motion sensor 305 outputs a state signal for measuring the movement distance d0 from the first operation position 18a to the second operation position 18b. The A / D conversion units 203, 303, and 307 perform sampling processing and quantization processing on the analog signals received from the microphones 201 and 301 or the motion sensor 305 to convert them into digital data, and then convert them to the network server 11. Output to.

  The network server 11 includes a network interface 401, an acoustic operation recognition unit 403, a movement distance calculation unit 405, a timing unit 407, a sound source distance calculation unit 409, a target device identification unit 411, and an operation command generation unit 413. These elements can be configured by cooperation of software resources such as an OS and application programs and hardware resources such as a CPU, a chip set, and a system memory that execute the software resources. Software resources can be stored in advance in a non-volatile memory of the network server 11 or downloaded through a network.

  The acoustic operation recognition unit 403 recognizes an effective acoustic operation from the acoustic data of the detected sound received from the plurality of target devices 13 and the reference device 15. When the acoustic operation recognition unit 403 recognizes an effective acoustic operation, the acoustic operation recognition unit 403 sends the detection time t2 of each target device 13 and the detection time t1 of the reference device 15 to the sound source distance calculation unit 409 and the movement distance calculation unit 405. The acoustic operation recognition unit 403 sends the number of operation sounds 40 performed at the first operation position 18a and the second operation position 18b to the operation command generation unit 413.

  The timer 407 measures the time and provides it to the acoustic operation recognition unit 403. The sound source distance calculation unit 409 obtains the sound speeds s and m and the reference distance r in advance. The sound source distance calculation unit 409 calculates the sound source distance d at the first operation position 18a and the second operation position 18b for each target device 13 by the method described with reference to FIG.

  The movement distance calculation unit 405 sends the movement distance d0 calculated from the state signal of the motion sensor 305 at the detection times t1 and t2 to the target device identification unit 411. The target device specifying unit 411 specifies the target device 13d that the user intends to operate by absolute value evaluation or relative value evaluation using the evaluation parameter Z, and outputs the identifier. The operation command generation unit 413 receives the number of operation sounds 40 at each of the first operation position 18a and the second operation position 18b received from the acoustic operation recognition unit 403 and the target device 13d received from the target device specifying unit 411. An operation command is generated from the identifier and output to the target device operation unit 207 through the network interface 401.

  For example, when one or two operation sounds are generated at the first operation position 18a and the second operation position 18b, respectively, one acoustic operation performed at the two operation positions 18a and 18b is performed by (one, Four pieces of operation information of (1 piece), (1 piece, 2 pieces), (2 pieces, 1 piece), and (2 pieces, 2 pieces) can be generated. The operation command generation unit 413 can associate operation information and operation commands for each target device 13 in advance.

[Sound operation recognition]
Next, an example of the operation of the acoustic operation recognition unit 403 will be described with reference to FIGS. FIG. 6 is a diagram schematically illustrating an example of the waveform of the detection sound 30 detected by the reference device 15 and the target device 13. FIG. 7 is a diagram for explaining a method in which the acoustic operation recognition unit 403 determines the effectiveness of the acoustic operation for each target device 13. FIG. 8 is a flowchart for explaining a procedure by which the acoustic operation recognizing unit 403 determines the effectiveness of the acoustic operation.

  The detection sound 30 detected by the microphones 201 and 301 includes noise in addition to the operation sound 40. If the sound pressure level of the detected sound 30 is low, the acoustic operation recognition unit 403 cannot recognize the operation sound 40. Accordingly, it is necessary to distinguish between the noise and the operation sound 40 in order to prevent erroneous operation. In addition, when a plurality of operation commands are generated using a plurality of operation sounds 40, it is necessary to distinguish between the subsequent operation sounds 40 generated continuously and the operation sounds 40 generated at the second operation position 18b. is there. The acoustic operation recognition unit 403 recognizes an effective operation sound 40 from the detection sound 30 including noise received from the target device 13 and the reference device 15 and extracts the detection time and the number of operation sounds 40.

  FIG. 6 is a diagram for explaining an example of a method for determining the effectiveness of the acoustic operation from the detection sound 30 by exemplifying an acoustic operation composed of a maximum of three operation sounds at the operation positions 18a and 18b. . FIG. 6 shows detected sounds 31a and 32a detected by the reference device 15 and detected sounds 31b and 32b detected by the target device 13 for two consecutive generated sounds generated at the generation time t0 at the first operation position 18a. Is shown. FIG. 6 also shows detection sounds 33a to 35a detected by the reference device 15 and detection sounds 33b to 33b detected by the target device 13 for three consecutive generation sounds generated at the generation time t00 at the second operation position 18b. 35b is shown.

  At this time, the acoustic operation recognition unit 403 does not recognize the occurrence times t0 and t00, and does not recognize the relationship between the detected sound 30 and the operation locations 18a and 18b. The acoustic operation recognizing unit 403 detects the detected sound 30 at the elapsed time from the detection times t0 and t00 of the detected sounds 31a and 33a first detected at the first operation position 18a and the second operation position 18b. A threshold value Th1 until the acoustic operation is completed at the operation positions 18a and 18b is set. A method for recognizing the first operation position 18a and the second operation position 18b will be described later.

  In addition, the acoustic operation recognition unit 403 performs an acoustic operation at the second operation position 18b on the detection sound 30 during the elapsed time from the detection time t0 of the detection sound 31a first detected at the first operation position 18a. Threshold values Th2 and Th3 are set as time until start and end. The minimum threshold Th2 can be set based on the minimum value of the movement time until the sound source 17 moves from the first operation position 18a to the second operation position 18b. The maximum threshold Th3 is an allowable time until the acoustic operation system 100 resets the current acoustic operation when the acoustic operation is not performed at the second operation position 18b after the acoustic operation is performed at the first operation position 18a. Can be set based on.

  In block 501 of FIG. 8, the acoustic operation recognition unit 403 is reset. The reset acoustic operation recognizing unit 403 treats the detection sounds 31a and 31b acquired first after the reset as being generated at the first operation position 18a. The acoustic operation recognition unit 403 receives acoustic data from each target device 13 and the reference device 15 and starts analyzing the detected sound 30. In block 503, the acoustic operation recognizing unit 403 sets a predetermined amplitude level 36 for the amplitude in one example in order to specify the generation time and the disappearance time of each detection sound 30 that exists discretely on the time axis. When the sound source distance d from the sound source 17 to the target device 13 is long, the acoustic operation recognition unit 403 can perform waveform processing to correct the attenuation of the sound pressure level as necessary.

The acoustic operation recognition unit 403 calculates the duration w by specifying the detection time and disappearance time of each detected sound 30 at the amplitude level 36. The acoustic operation recognition unit 403 registers feature information such as the duration and frequency component of the operation sound 40 generated by the sound source 17 in advance. The acoustic operation recognition unit 403 sets a lower limit threshold Th4 and an upper limit threshold Th5 for the duration w of the detection sound 30,
Th4 <w ≦ Th5 (6)
Detected sound that does not satisfy can be regarded as noise. The acoustic operation recognition unit 403 compares the registered frequency component with the frequency component of the detection sound 30 to recognize whether or not the operation sound is 40. The acoustic operation recognizing unit 403 can discard the detected sound 30 that cannot be recognized as the operation sound 40 from the duration w or frequency component in block 551 as noise.

  In block 505, the acoustic operation recognition unit 403 calculates a time difference “a” between the preceding detection sound and the subsequent detection sound based on the detection time of each detection sound 30. FIG. 6 shows a time difference a1 between the detection sounds 31a and 32a, a time difference a2 between the detection sounds 33a and 34a, and a time difference a3 between the detection sounds 34a and 35a. The time difference a for the detected sound detected by the target device 13 can be calculated in the same manner. The time difference between the detected sound 32a detected last at the first operation position 18a and the detected sound 33a detected first at the second operation position 18b is excluded from the time difference a.

The acoustic operation recognition unit 403 sets a minimum threshold Th6 and a maximum threshold Th7 for the time difference a. The minimum threshold Th6 can be set based on the minimum time difference that allows the user to continuously generate the operation sound 40, and the maximum threshold Th7 can be set from the threshold Th1 and the number of operation sounds. The acoustic operation recognition unit 403
Th6 <a ≦ Th7 (7)
The subsequent detection sound 30 that does not satisfy the above condition can be regarded as noise and discarded at block 551.

  The detected sound 30 processed in the procedure so far may be discarded as noise because there is not a sufficient sound pressure level even if it is the operation sound 40, or noise that cannot be distinguished in the blocks 503 and 505 may be mixed. There is sex. In the present embodiment, the acoustic operation recognition unit 403 can further determine the effectiveness of the acoustic operation based on the temporal generation pattern of the detected sound 30 by executing the following procedure. The acoustic operation system 100 measures the sound source distance d as described with reference to FIG. 3, and the operation sound 40 detected by the reference device 15 and the operation detected by each target device 13 for the same operation sound 40. The sound 40 needs to form a pair of operation sounds 40.

  In block 507, the acoustic operation recognizing unit 403 determines whether or not the pair candidate of the detection sound 30 for the same generated sound including noise forms a pair of the operation sound 40 in order to determine the effectiveness of the acoustic operation. . The acoustic operation recognizing unit 403 selects, for example, the detected sound 31b detected by the target device 13 as the closest candidate detected sound (in this case, the detected sound 31a) detected by the reference device 15 as a pair candidate of the detected sound 30. be able to. FIG. 6 shows pair candidates (31a, 31b), (32a, 32b), (33a, 33b), (34a, 34b), and (35a, 35b) of the detected sound 30.

  The acoustic operation system 100 can assume in advance the maximum value of the sound source distance d in the environment in which acoustic operation is used. The acoustic operation recognizing unit 403 sets a minimum threshold Th8 and a maximum threshold Th9 for the time difference (t2−t1) of the detection time in order to determine the validity of the pair candidate of the detection sound 30. The minimum threshold Th8 can be set based on the difference between the assumed minimum sound source distance d and the reference distance r. The maximum threshold value Th9 can be set based on the difference between the assumed maximum sound source distance d and the reference distance r.

The acoustic operation recognizing unit 403 determines whether the detection sound 30 pair candidate is
Th8 <(t2-t1) ≦ Th9 (8)
Since the pair of operation sounds 40 cannot be formed for the target device 13 when the condition is not satisfied, the acoustic operation is treated as invalid in block 553. For the target device 13 that satisfies Expression (8), the process proceeds to block 509. FIG. 7A shows that since the target device 13 does not detect the detection sound 31b, the acoustic operation recognition unit 403 indicates that the pair candidate of the detection sound 30 by the subsequent detection sound 32b and the preceding detection sound 31a is not valid. Show.

  The acoustic operation recognizing unit 403 detects, for each target device 13, first detected sounds 31a, 31b detected after reset and subsequent detected sounds 32a, 32b that satisfy Expression (7) at the first operation position 18a. Consider 30. The acoustic operation recognizing unit 403 generates the detected sounds 33a and 33b that are detected first after the threshold value Th1 and the subsequent detected sounds 34a, 35a, 34b, and 35b that satisfy Expression (7) at the second operation position 18b. The detection sound 30 is considered. In block 509, the acoustic operation recognizing unit 403 calculates a time difference b between the detected sound 31a first detected at the first operation position 18a and the detected sound 33a first detected at the second operation position 18b. The same calculation can be performed for the target device 13.

In the acoustic operation recognition unit 403, the time difference b is Th2 <b ≦ Th3 (9)
When the target device 13 is not satisfied, the acoustic operation is treated as invalid in block 553. For the target device 13 that satisfies Equation (9), the process proceeds to block 511. FIG. 7B shows a state in which the time difference b until the detection sound 33a is detected is too long and the acoustic operation recognition unit 403 is reset for the target device 13.

  In block 511, the acoustic operation recognition unit 403 calculates the number of detection sounds 30 detected between the detection time t1 and the threshold Th1, and the number of detection sounds 30 detected between the detection time t00 and the threshold Th1. When the number of detected sounds 30 exceeds the set maximum number (in this case, three), the acoustic operation recognizing unit 403 considers that noise is mixed, and the target device 13 performs acoustic processing in block 553. Treat the operation as invalid. For the target devices 13 having the maximum number or less, the process proceeds to block 513. FIG. 7C shows a state in which the acoustic operation is disabled by detecting four detection sounds 31a, 32a, 36a, and 37a during the threshold Th1.

  In block 513, the acoustic operation recognition unit 403 proceeds to block 607 in FIG. 9 for the target device 13 that has recognized an effective acoustic operation. In block 553, the acoustic operation recognizing unit 403 resets the acoustic operation recognizing unit 403 for the target device 13 that does not recognize an effective acoustic operation, and proceeds to block 603 in FIG. 9 to newly perform the acoustic operation at the first operation position 18a. Wait until is done. The procedures of blocks 503 to 511 can be appropriately selected and adopted according to the acoustic operation environment. Further, the order of processing does not have to be limited to that illustrated in FIG. 8. For example, the order of the blocks 503 and 505 may be changed, and the order of the blocks 507 to 511 may also be changed.

[Operation of acoustic operation system]
FIG. 9 is a flowchart for explaining the operation of the acoustic operation system 100. In block 601, the target device 13, the network server 11, and the reference device 15 acquire the other party's network address, complete mutual authentication, and construct a network. In block 603, the acoustic operation recognition unit 403 receives the detection sound 30 at the occurrence times t0 and t00.

  In block 605, the acoustic operation recognizing unit 403 determines the effectiveness of the acoustic operation for each target device 13 in the procedure of FIG. In block 607, the acoustic operation recognition unit 403 recognizes, for example, the operation sound 40 corresponding to the detection sound 30 illustrated in FIG. 6 for the target device 13 that has recognized that the acoustic operation is valid. The acoustic operation recognizing unit 403 sends detection times t1, t2, t11, and t12 related to the first operation position 18a and the second operation position 18b to the sound source distance calculation unit 409 and the movement distance calculation unit 405.

  In block 609, the acoustic operation recognizing unit 403 informs the operation command generation unit 413 of the number (two) of operation sounds 40 at the first operation position 18a and the number (three) of operation sounds 40 at the second operation position 18b. send. In block 611, the movement distance calculation unit 405 calculates the movement distance d0 from the detection times t1 and t22 and the state data output from the motion sensor 305, and sends the movement distance d0 to the target device identification unit 411.

  In block 613, the sound source distance calculation unit 409 uses any one of the formulas (1) to (3) for each target device 13, and at each of the target devices 13 at the first operation position 18a and the second operation position 18b. The sound source distance d is calculated. In block 615, the target device identification unit 411 applies either Equation (4) or Equation (5) to the target device for which the acoustic operation recognition unit 403 has recognized an effective acoustic operation, and intends the operation. 13d is specified. In block 617, the operation command generation unit 413 outputs an operation command corresponding to the combination of the number of operation sounds to the target device operation unit 207 of the specified target device 13d, and resets the acoustic operation system 100.

  The acoustic operation system 100 can distribute the functions of the network server 11 to the target device 13 and the reference device 15. For example, the acoustic operation recognizing unit 403 and the time measuring unit 407 can be distributed and mounted on the target device 13 and the reference device 15. When distributing the time measuring unit 407 to the target device 13 and the reference device 15, it is possible to synchronize the RTC (Real Time Clock) of the distributed time measuring unit 407 using an NTP server or a synchronization command as necessary. When distributing the time measuring unit 407, the acoustic operation recognizing unit 403 may receive the detection times t1 and t2 from the target device 13 and the reference device 15. The movement distance calculation unit 405 can also be mounted on the reference device 15.

  The microphone 301 and the device for measuring the moving distance d0 can be separated from the reference device 15. At this time, the movement distance d0 may be measured by a stereo camera instead of the motion sensor 305. The resolution of the acoustic operation system 100 depends on the distance between the target devices 13. The measurement accuracy of the sound source distance d can be made appropriate to the required resolution. In this case, when the operation sound 40a is generated by ringing a finger as shown in FIG. 2A, the first operation position 18a is set near the chest, and the position where the forearm is extended is the second operation. The position may be set to the position 18b, and the standard upper arm length may be set to the movement distance d0.

[Other examples of acoustic operation system]
10 and 11 are schematic functional block diagrams for explaining the configuration of the acoustic operation systems 600 and 700. FIG. 10 and 11, the same reference numerals are given to the same elements as those in FIG. 5 or elements whose functions can be easily understood from FIG. The acoustic operation systems 600 and 700 are configured without using the network server 11. The acoustic operation system 600 includes a reference device 605 and a target device 603 connected via a network. In the acoustic operation system 600, the function of the network server is incorporated in the reference device 605. A plurality of target devices 603 are connected to the network interface 607.

  The acoustic operation system 700 includes a target device 703 and a reference device 705. In the acoustic operation system 700, the function of the network server is incorporated in the target device 703. A plurality of target devices 703 are connected to the network interface 309. Also in the acoustic operation systems 600 and 700, the functions installed in the reference device 605 and the target device 703 can be distributed. The present invention is not limited to the acoustic operation systems 100, 600, and 700 described with reference to FIGS. 5, 10, and 11, and a predetermined function may be distributed by another method or a similar function may be adopted. Anything that can be easily conceived by those skilled in the art including the concept of the present invention is included in the scope of the present invention.

  Although the present invention has been described with the specific embodiments shown in the drawings, the present invention is not limited to the embodiments shown in the drawings, and is known so far as long as the effects of the present invention are achieved. It goes without saying that any configuration can be adopted.

10 User 11 Network server 100, 600, 700 Acoustic operation system 13, 13a-13e, 603, 703 Target device 15, 605, 705 Reference device 15a Wearable device (reference device)
15b Portable electronic device (reference device)
17, 17a, 17b Sound source 18a First operation position 18b Second operation position 19 Directional straight line 30, 31a-35a, 31b-35b Detection sound 40, 41a-45a, 41b-45b Operation sound 201, 201a-201e, 301 , 301a, 301b Microphone d Sound source distance r Reference distance

Claims (20)

An acoustic operation system for operating a target target device selected by acoustic operation from a plurality of target devices,
The operation sound data is received using a plurality of target microphones that detect operation sounds generated by the sound source at the first operation position and the second operation position, and the operation sounds from the sound source to the target microphone are received. A sound source distance calculator that calculates a first sound source distance from the first operation position to the target microphone and a second sound source distance from the second operation position to the target microphone based on a propagation time;
The target device associated with the target microphone selected based on the first sound source distance, the second sound source distance, and the movement distance from the first operation position to the second operation position is specified. An acoustic operation system having a target device specifying unit.   The acoustic operation system according to claim 1, wherein the second operation position is present in a direction from the target microphone associated with the target device toward the first operation position.   The sound source distance calculation unit calculates the first sound source distance and the second sound source distance from a time difference between a generation time of the operation sound and a detection time of the operation sound detected by the target microphone. Acoustic operation system.   The sound source distance calculation unit receives the operation sound data using a reference microphone disposed at a known reference distance from the sound source at the first operation position and the second operation position, and the target microphone The first sound source distance and the second sound source distance are calculated using the detected time of the detected operation sound, the detected time of the operation sound detected by the reference microphone, and the reference distance. Acoustic operation system.   The acoustic operation system according to claim 4, wherein the reference microphone is mounted on a portable electronic device, and the sound source is a user's fingertip or palm.   The acoustic operation system according to claim 5, wherein the portable electronic device is equipped with a motion sensor that outputs state information for acquiring the moving distance.   The acoustic operation system according to claim 4, wherein the reference microphone is mounted on a portable electronic device, and the sound source is a part of a casing of the portable electronic device.   The acoustic operation system according to claim 4, further comprising: an acoustic operation recognition unit that determines the effectiveness of the acoustic operation based on a temporal generation pattern of detection sound including noise detected by the target microphone and the reference microphone.   The acoustic operation recognition unit detects the same generated sound by the target microphone and the reference microphone based on a difference between a detection time of the detection sound detected by the target microphone and a detection time of the detection sound detected by the reference microphone. The acoustic operation system according to claim 4, wherein the acoustic operation is invalidated when it is determined that the detected sound pair does not form the operation sound pair.   The acoustic operation recognition unit discards the detection sound when a time difference between a detection time of a preceding detection sound detected by the reference microphone or the target microphone and a detection time of a subsequent detection sound is a predetermined value or more. 4. The acoustic operation system according to 4.   The acoustic operation system according to claim 1, further comprising an operation command generation unit that outputs operation commands corresponding to a plurality of operation sounds respectively generated at the first operation position and the second operation position to the target device. . A network server capable of configuring an acoustic operation system for operating a target target device selected by acoustic operation from a plurality of target devices,
A first operation sound data is received from a reference device having a reference microphone disposed at a known reference distance from the sound source at the first operation position and the second operation position so that the operation sound generated by the sound source can be detected. And the interface
A second interface for receiving the operation sound data from a plurality of target devices each including a target microphone for detecting the operation sound generated by the sound source at the first operation position and the second operation position;
A first sound source between the target microphone and the first operation position based on a difference between a propagation time of the operation sound from the sound source to the target microphone, a propagation time from the sound source to the reference microphone, and the reference distance A sound source distance calculating unit for calculating a distance and a second sound source distance between the target microphone and the second operation position;
A target device that identifies the target target device on which the reference microphone selected based on the first sound source distance, the second sound source distance, and the moving distance from the first operation position to the second operation position is mounted A specific part,
A network server including an operation command generation unit that transmits an operation command to the target device through the second interface; A control device mounted on a target device that can be operated by acoustic operation,
An interface for receiving the operation sound data from a reference device having a reference microphone disposed at a known reference distance from the sound source at the first operation position and the second operation position so that the operation sound generated by the sound source can be detected; ,
A target microphone for detecting the operation sound;
A first sound source between the target microphone and the first operation position based on a difference between a propagation time of the operation sound from the sound source to the target microphone, a propagation time from the sound source to the reference microphone, and the reference distance A sound source distance calculation unit for calculating a distance and a second sound source distance between the target microphone and the second operation position;
A target device specifying unit that recognizes the target device based on the first sound source distance, the second sound source distance, and a moving distance from the first operation position to the second operation position. Control device. A portable electric device capable of operating a target target device selected by acoustic operation from a plurality of target devices,
A reference microphone disposed at a known reference distance from the sound source at the first operation position and the second operation position so that the operation sound generated by the sound source can be detected;
An interface for receiving the operation sound data from the plurality of target devices each having a target microphone;
A first sound source between the target microphone and the first operation position based on a difference between a propagation time of the operation sound from the sound source to the target microphone, a propagation time from the sound source to the reference microphone, and the reference distance A sound source distance calculation unit for calculating a distance and a second sound source distance between the target microphone and the second operation position;
A target device specifying unit for specifying the target target device selected based on the first sound source distance, the second sound source distance, and the movement distance from the first operation position to the second operation position;
A portable electronic device comprising: an operation command generation unit that transmits an operation command to the target device through the interface. A method in which an acoustic operation system operates a target target device selected from a plurality of target devices according to an acoustic operation,
The plurality of target devices detecting an operation sound generated by the sound source at the first operation position and an operation sound generated at the second operation position;
Based on the propagation time of the operation sound from the sound source to the target device, a first sound source distance from the first operation position to the target device and a second distance from the second operation position to the target device. Calculating the sound source distance;
Obtaining a moving distance between the first operation position and the second operation position;
A method comprising: specifying the target device based on the first sound source distance, the second sound source distance, and the moving distance; and sending an operation command to the target device. The second operation position exists in a direction from the first operation position toward the target device;
16. The step of identifying the target device includes comparing an evaluation parameter obtained by subtracting the reference distance from a difference between the first sound source distance and the second sound source distance with a predetermined threshold value. Method. The second operation position exists in a direction from the first operation position toward the target device;
The method according to claim 16, wherein identifying the target device includes comparing the evaluation parameters calculated for the plurality of target devices with each other. Calculating the first sound source distance and the second sound source distance;
Obtaining a first detection time at which the operation sound is detected at a known reference distance from the sound source;
Obtaining a second detection time at which each of the plurality of target devices detects the operation sound;
16. The method of claim 15, comprising calculating a difference between the second detection time and the first detection time. Detecting a first number of operation sounds at the first operation position;
Detecting a second number of operation sounds at the second operation position;
Generating a plurality of the operation commands by a combination of the first number and the second number;
The method according to claim 15, further comprising disabling the acoustic operation when the first number or the second number is equal to or greater than a predetermined value. In an acoustic operation system for operating a target target device selected from a plurality of target devices in accordance with an acoustic operation,
A function of the plurality of target devices to detect an operation sound generated by the sound source at the first operation position and an operation sound generated at the second operation position;
Based on the propagation time of the operation sound from the sound source to the target device, a first sound source distance from the first operation position to the target device and a second distance from the second operation position to the target device. A function to calculate the sound source distance,
A function of acquiring a movement distance between the first operation position and the second operation position;
A computer program for realizing a function of specifying the target device based on the first sound source distance, the second sound source distance, and the moving distance.

Images