JP2024018748A - Information processing device, program, information processing method, and information processing system - Google Patents

Abstract

To provide an information processing device, program, information processing method, and information processing system for improving convenience of a user.SOLUTION: An information processing system includes a sound collection device having two microphone arrays and an information processing device. A sound processing application 231 to be operated in the information processing device includes: a device detection section 241 which functions as a detection section for detecting a connection state with the sound collection device; a sound processing section 243 which functions as a processing section for processing a sound signal obtained by the sound collection device whose connection with the information processing device has been detected by the device detection section 241; and a virtual speaker section 251 which functions as a virtual sound output device and receives an input of a post-processing sound signal being the sound signal after processing by the sound processing section 243.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device, a program, an information processing method, and an information processing system.

In recent years, due to advances in technology, information processing devices such as PCs (Personal Computers), smartphones, and smart watches that can be obtained and used by general users have diversified. It has become common for these various information processing devices to be equipped with a sound collection device such as a sensor or a microphone, and to acquire and utilize the user's voice or other sounds. Furthermore, the information processing device performs processing related to emphasizing the sound picked up by the sound pickup device.

For example, Patent Document 1 and Patent Document 2 disclose techniques for emphasizing and acquiring sound from a certain sound source while reducing sound from a noise source.

Japanese Unexamined Patent Publication No. 4-021000 International Publication No. 2004/034734

However, it is desired to provide a technology that can further improve convenience for users of information processing devices.

Therefore, a technique is provided that can further improve convenience for users of information processing devices.

In order to solve the above problems, according to one aspect of the present invention, an information processing device includes a detection unit that detects a connection state with a sound collection device, and a detection unit that detects a connection state with the information processing device by the detection unit. a processing unit that processes the sound signal obtained by the detected sound collection device; and a virtual speaker unit that functions as a virtual sound output device and receives a processed sound signal that is the sound signal processed by the processing unit. An information processing device is provided.

The information processing device functions as a sound processing application including the detection unit and the processing unit, the virtual speaker unit, and a virtual sound input device, and receives the processed sound signal output from the virtual speaker unit as an input. a virtual tool having a virtual microphone section, the virtual tool is set in advance by a provider of the sound processing application as an output destination of the processed sound signal from the sound processing application. Good too.

The virtual microphone unit may have a name corresponding to the sound processing application as a name recognized by the information processing device.

The detection unit acquires hardware identification information of the sound collection device whose connection has been detected, and checks whether the hardware identification information matches target device identification information held in a storage unit, The processing unit starts processing the sound signal obtained by the sound collection device when the hardware identification information matches the target device identification information, and when the hardware identification information matches the target device identification information. If not, the sound signal may be output to the virtual speaker section without executing the processing of the sound signal.

The processing unit may stop processing the sound signal obtained by the sound collection device when the detection unit detects that the connection of the sound collection device whose connection has been detected has been released. .

The screen generation unit further includes a screen generation unit that generates notifications and screens to be displayed on the display unit, and a display control unit that controls display on the display unit, and the screen generation unit is configured to detect the connection of the sound collection device by the detection unit. When detected, a notification may be generated to the effect that a target device for processing the sound signal by the processing unit has been detected, and the display control unit may display the notification on the display unit.

When the detection unit detects that the sound collection device is disconnected, the screen generation unit generates a notification that the connection of the target device for processing the sound signal by the processing unit is disconnected. However, the display control section may display the notification on the display section.

The screen generator further includes a parameter setting unit configured to set a parameter related to the processing of the sound signal by the processing unit in response to a parameter change operation by the user, and the screen generation unit is configured to set a parameter that accepts the parameter change operation by the user. A setting screen may be generated, and the display control section may display the parameter setting screen on the display section.

The processing unit may perform area sound collection processing, which is sound signal processing for emphasizing sound components in a target area, on the sound signal obtained by the sound collection device.

The processing unit may perform the area sound collection process by performing processing on the sound signal obtained by the sound collection device to weaken ambient sound components other than sound components in the target area.

The processing unit may perform the area sound collection process by performing processing on the sound signal obtained by the sound collection device to amplify a sound component in the target area.

The output destination of the processed sound signal may be a usage application that uses the processed sound signal.

The usage application may be a state detection application that detects a state abnormality of a source of the processed sound signal by machine learning based on the processed sound signal.

According to another aspect of the present invention to solve the above problems, the computer includes a detection section that detects a connection state between an information processing device and a sound collection device; A processing unit that processes the sound signal obtained by the sound collection device whose connection has been detected, and a virtual device that functions as a virtual sound output device and receives a processed sound signal that is the sound signal processed by the processing unit. A program is provided to make it function as a speaker section.

According to another aspect of the present invention, in order to solve the above problems, a connection state between an information processing device and a sound collection device is detected, and a sound collection device whose connection with the information processing device is detected is provided. and inputting the processed sound signal, which is the processed sound signal, to a virtual speaker unit functioning as a virtual sound output device. A processing method is provided.

According to another aspect of the present invention, in order to solve the above problems, a sound collection device, a detection unit that detects a connection state between an information processing device and the sound collection device, and a detection unit configured to perform the information processing by the detection unit. A processing unit that processes the sound signal obtained by the sound collection device whose connection with the device has been detected, and a processed sound signal that functions as a virtual sound output device and receives the processed sound signal that is the sound signal processed by the processing unit. An information processing system is provided, including a virtual speaker section that is configured to perform a virtual speaker section.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram for explaining an overview and functional configuration example of an information processing system according to the present invention. FIG. 2 is a block diagram illustrating an example of the functional configuration of an information processing device 20 according to the present embodiment. FIG. 2 is a block diagram for explaining an example of a functional configuration of a sound processing application 231 according to the present embodiment. 3 is an explanatory diagram for explaining an example of a parameter setting screen generated by a screen generation unit 247. FIG. FIG. 7 is an explanatory diagram for explaining another example of a parameter setting screen generated by the screen generation unit 247. FIG. 7 is an explanatory diagram for explaining another example of a parameter setting screen generated by the screen generation unit 247. 12 is an explanatory diagram showing an example of a selection screen in which the virtual microphone unit 253 is selected as a sound input source device in the conference application 233. FIG. FIG. 2 is a flowchart diagram for explaining an example of operational processing of the information processing system according to the present embodiment. 1 is a diagram showing an example of a hardware configuration of an information processing device 20 according to an embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals and redundant explanation will be omitted.

Further, in this specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by using different numbers or alphabets after the same reference numeral. However, if there is no particular need to distinguish each of the plurality of components having substantially the same functional configuration, only the same reference numerals are given to each of the plurality of components.

<1. Overview of information processing system according to an embodiment of the present invention>
The present invention relates to a technology that can further improve convenience for users of information processing devices. More specifically, according to the present invention, a user of an information processing device inputs desired sound into the information processing device using a sound pickup device connected to the information processing device, and transmits the sound to another device. It is possible to improve the convenience for users when doing so. More specifically, the present invention performs processing to further emphasize the user's voice on a sound signal obtained by collecting the user's voice from a microphone attached to a headset, for example. This method is applicable to cases where a processed sound signal is sent to the other end of a remote conference system.

FIG. 1 is an explanatory diagram for explaining an overview and functional configuration example of an information processing system according to the present invention. As shown in FIG. 1, the information processing system according to this embodiment includes a sound collection device 10 and an information processing device 20. The sound collection device 10 and the information processing device 20 are connected by wire. Note that the sound collection device 10 and the information processing device 20 may be configured to be able to communicate via wireless connection.

(Sound collection device 10)
The sound collection device 10 is a sound collection device including two microphone arrays 110. The microphone array 110 is a microphone array of 2 channels (channels) or more each including at least two or more microphones. Each microphone may be a general omnidirectional microphone. The microphone array 110A and the microphone array 110B each have two or more microphones, so that they can have directivity in the sound collection range.

Furthermore, in the microphone array 110A and the microphone array 110B, a sound source (for example, the face or mouth of the user of the information processing device 20) that inputs sound to the sound collection device 10 is included in an area where the sound collection ranges of the microphone arrays 110A and 110B overlap. placed in such a position. In this way, the area where the sound collection ranges of microphone array 110A and microphone array 110B overlap is hereinafter also referred to as a target area.

Note that the positions of the microphone array 110A and the microphone array 110B are not particularly limited as long as they are placed in a positional relationship that creates a target area with respect to the position of the sound source that inputs sound to the sound collection device 10. . For example, the microphone array 110A and the microphone array 110B may be arranged so that their directivity intersects with respect to a target area that includes the position of a sound source that inputs sound to the sound collection device 10. For example, microphone array 110A and microphone array 110B may be arranged to face each other with the target area in between.

The sound collection device 10 may be realized by, for example, a headset including two microphone arrays. Alternatively, the sound collection device 10 may be a stereo speaker including two or more microphone arrays, a keyboard, a neck speaker, a directional speaker built into the upper part of a monitor, or the like. Alternatively, the sound collection device 10 may be a combination of two or more individual microphone arrays 110.

The sound collection device 10 has a function of collecting a user's voice (not shown in FIG. 1) and other sounds occurring within a sound collection range. The sound collection device 10 outputs a collected sound signal (sound signal) to the information processing device 20 .

(Information processing device 20)
The information processing device 20 is an information processing device that performs sound processing on the sound signal obtained by the sound collection device 10. The information processing device 20 is realized by, for example, a PC (Personal Computer). Alternatively, the information processing device 20 may be a server, a PDA (Personal Digital Assistant), a digital camera, an audio player, a landline telephone, a mobile phone/smartphone, a smart watch, a tablet terminal, a printer, a scanner, a multifunction device, or a vending machine. , automatic ticket vending machine, kiosk terminal, cash processing machine, ATM (Automated/Automatic Teller Machine) (Cash Machine), measuring equipment, measuring equipment, medical equipment, car navigation system (Automotive Navigation System), automobile, radio equipment for amateur stations, broadcasting Office radios, commercial radios (e.g. radio equipment for transmitting information for commercial use as specified in Article 3, Paragraph 1 (Classification and Definition of Business) of the Radio Law Enforcement Regulations of the Ministry of Internal Affairs and Communications), game consoles, televisions, It may be realized by an air conditioner, an air purifier, a refrigerator, a microwave oven, or the like.

The information processing device 20 has a function as a sound processing application that performs sound processing on the sound signal obtained by the sound collection device 10. The sound processing application includes, for example, the following functions.

As a first function, the information processing device 20 performs processing on the sound signal obtained by the sound collection device 10 to emphasize sounds in the target area or to weaken ambient sounds outside the target area.

More specifically, the sound component in the target area is the sound component in the area where the sound collection ranges of microphone array 110A and microphone array 110B overlap. Therefore, each sound signal obtained by each of the microphone array 110A and the microphone array 110B can contain the sound components of the target area simultaneously (in the same phase) and with the same magnitude (the same intensity). Using this fact, the information processing device 20 analyzes the sound signals obtained by the microphone array 110A and the microphone array 110B in the frequency domain, and extracts the sound component of the target area. The information processing device 20 can perform a process of emphasizing the extracted sound components of the target area or weakening sound components other than the sound components of the target area.

In addition, as described above, the information processing device 20 extracts the sound components of the target area by a method using two microphone arrays 110 each having two or more microphones, thereby detecting the target area as seen from the sound collection device 10 side. It is possible to extract only the sounds in the target area even if there is disturbing sound behind the device.

Furthermore, as a second function, the information processing device 20 has a function of identifying, as a target device, a sound collection device from which a sound signal to be subjected to the above-mentioned sound processing can be acquired. The information processing device 20 detects the connection state between the target device and the information processing device 20. Further, when the target device is connected, the information processing apparatus 20 automatically starts the above-mentioned processing on the sound signal obtained by the target device. This saves the user of the information processing device 20 the trouble of manually starting the process, improving convenience for the user.

In addition, as a third function, the information processing device 20 has a virtual speaker and a Has virtual microphone function. The virtual speaker functions as a virtual sound output device in the information processing device 20. The sound signal after the above processing is input to the virtual speaker. Furthermore, the sound signal input to the virtual speaker is input to a virtual microphone that functions as a virtual sound input device in the information processing apparatus 20. In the present invention, a virtual speaker and a virtual microphone are also collectively referred to as a virtual tool.

Further, the virtual tool is set in advance at the output destination of the processed sound signal from the sound processing application by the provider of the sound processing application.

When the user of the information processing device 20 wants to input the processed sound signal to the conference application on the information processing device 20, the user can select the virtual microphone as the sound input device. Further, the name of the virtual microphone according to the present invention presented to the user is set to a name corresponding to a sound processing application that processes the sound. Therefore, the user of the information processing device 20 can easily recognize which input device to use among the devices that can be selected as the sound input device.

The outline of the information processing system according to this embodiment has been described above with reference to FIG. Next, an example of the functional configuration of the information processing device 20 according to this embodiment will be described with reference to FIGS. 2 and 3.

<2. Functional configuration example>
<2-1. Information processing device 20>
FIG. 2 is a block diagram illustrating an example of the functional configuration of the information processing device 20 according to this embodiment. As shown in FIG. 2, the information processing device 20 includes a communication section 210, a storage section 220, a control section 230, an operation display section 270, and a sound output section 280.

(Communication department 210)
The communication unit 210 has a function of performing various communications with external devices under the control of the control unit 230. For example, the communication unit 210 transmits the sound signal obtained by the sound collection device 10 and processed by the control unit 230 to the information processing terminal of the other party in the conference application to which the sound signal is input. You may.

(Storage unit 220)
Storage unit 220 holds software and various data. For example, the storage unit 220 holds an OS (Operating System) and a sound processing application program to be described later. Furthermore, the storage unit 220 holds target device identification information of a target device whose sound is to be processed by the sound processing application.

Furthermore, the storage unit 220 stores setting values of parameters related to area sound collection processing, which will be described later. For example, a configuration file (for example, an ini format file) in which parameter setting values are described may be held in the storage unit 220.

(Control unit 230)
The control unit 230 includes an arithmetic unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and stores programs stored in a ROM (Read Only Memory) in a RAM (Random Access Memory). ) and executed This function can be realized by At this time, a computer-readable recording medium on which the program is recorded may also be provided. Alternatively, these blocks may be configured by dedicated hardware or may be realized by a combination of multiple pieces of hardware.

Data necessary for calculation by the calculation device is appropriately stored in the storage unit 220. The storage unit 220 may be configured with a memory such as a RAM, a hard disk drive, or a flash memory.

Such a control unit 230 controls the overall operation of the information processing device 20. For example, the control unit 230 controls the communication unit 210 to transmit the processed sound signal to an external device.

Further, the control unit 230 has functions as a sound processing application 231, a conference application 233, and a display control unit 235. The functions of the main body section 240 and the virtual tool section 250 included in the sound processing application 231 will be explained in detail later with reference to FIG. 3.

The conference application 233 is an application that allows a user of the information processing device 20 to interact or share information with other users using external devices via a network using audio and video. The conference application 233 has a function as a sound input section 260. A sound signal output from a sound output device selected by the user or by the initial settings of the conference application 233 is input to the sound input unit 260 . Conference application 233 is not limited to a specific conference application. Conference application 233 may be a known conference application.

The display control unit 235 has a function of controlling the operation display unit 270 to display various screens.

(Operation display section 270)
The operation display section 270 has a function as an operation section that can accept various operations by the user of the information processing device 20. For example, the user of the information processing device 20 may operate the operation display unit 270 to select an input source device for sound input to the remote conference system application.

Further, the operation display section 270 has a function as a display section that displays various screens under the control of the display control section 235.

The function of the operation display section 270 as an operation section can be realized by, for example, a touch panel or a keyboard. Further, the function as a display unit can be realized by a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD), or an OLED (Organic Light Emitting Diode) device. Note that the function as a display section and the function as an operation section may be configured separately.

(Sound output section 280)
The sound output unit 280 has a function of outputting sound under the control of the control unit 230. The sound output unit 280 is realized by, for example, a sound output device such as a speaker and headphones. The sound output unit 280 converts a sound signal included in audio data or the like into sound and outputs the sound.

The functional configuration example of the information processing device 20 according to this embodiment has been described above with reference to FIG. 2. Next, with reference to FIG. 3, the functions of the above-mentioned sound processing application 231 will be explained in detail.

FIG. 3 is a block diagram for explaining an example of the functional configuration of the sound processing application 231 according to this embodiment. As shown in FIG. 3, the sound processing application 231 has functions as a main body section 240 and a virtual tool section 250. The main body section 240 and the virtual tool section 250 are integrally installed in the information processing apparatus 20 by the same installer.

The sound processing application 231 may, for example, automatically start up when the information processing device 20 is started up and reside as a background application. The sound processing application 231 may wait in the background until the connection of the sound collection device 10, which is a target device to be described later, is detected. At this time, the sound processing application 231 does not process sound signals in the standby state. Further, the user of the information processing device 20 may be able to manually start the sound processing application 231 by operating the operation display unit 270. For example, the user of the information processing device 20 may start the sound processing application 231 by directly executing an executable file (for example, an exe file) of the sound processing application 231.

Further, the sound processing application 231 may be automatically terminated when the information processing device 20 is powered off. Furthermore, the user of the information processing device 20 may be able to manually terminate the sound processing application 231 by operating the operation display unit 270. For example, the user of the information processing device 20 may be able to open a menu from a task tray or the like displayed at the bottom right of the desktop screen of a typical PC and select to terminate the sound processing application 231.

(Main body part 240)
The main body section 240 has a function of processing sound signals obtained by the sound collection device 10 and output to the information processing device 20. More specifically, the functions of the main body section 240 are realized by functions as a device detection section 241, a sound processing section 243, a parameter setting section 245, and a screen generation section 247.

The device detection unit 241 has a function of detecting the connection state between the information processing device 20 and the sound collection device 10. The device detection unit 241 is an example of a detection unit of the information processing device 20.

The device detection unit 241 acquires hardware identification information of the sound collection device 10 that allows the sound collection device 10 whose connection with the information processing device 20 has been detected to be uniquely identified. Further, the device detection unit 241 checks whether the acquired hardware identification information matches the target device identification information held in the storage unit 220. Thereby, the device detection unit 241 can detect whether the sound collection device 10 whose connection to the information processing device 20 has been detected is the target device. Note that there may be a plurality of pieces of target device identification information held in the storage unit 220.

The sound processing unit 243 has a function of processing sound signals obtained by the sound collection device 10. More specifically, the sound processing unit 243 performs area sound collection processing on the sound signal obtained by the sound collection device 10, which is sound signal processing for emphasizing sound components in the target area. Such a sound processing section 243 is an example of a processing section of the information processing device 20.

If the hardware identification information of the sound collection device 10 detected to be connected to the information processing device 20 matches the target device identification information as a result of the above verification by the device detection unit 241, the sound processing unit 243 detects the sound collection device 10. automatically starts processing the sound signal obtained by. In addition, if the hardware identification information does not match the target device identification information, the sound processing unit 243 does not process the sound signal obtained by the sound collection device 10, and transmits the sound signal to a virtual speaker (described later). It is output to section 251. In this way, when a device whose sound signals are to be processed by the sound processing section 243 is connected to the information processing apparatus 20, processing of the sound signals by the sound processing section 243 is automatically started. Moreover, when the connection between the information processing apparatus 20 and the target device is released, the processing of the sound signal by the sound processing unit 243 is automatically stopped.

Further, the sound processing unit 243 may perform the following processing as area sound collection processing. For example, the sound processing unit 243 may perform area sound collection processing on the sound signal obtained by the sound collection device 10 by performing processing to weaken ambient sound components other than the sound components of the target area.

Alternatively, the sound processing unit 243 may perform area sound collection processing by performing processing on the sound signal obtained by the sound collection device 10 to amplify sound components in the target area.

Furthermore, the sound processing unit 243 performs processing on the sound signal obtained by the sound collection device 10 to weaken ambient sound components other than the sound components in the target area, and processing to amplify the sound components in the target area. Area sound collection processing may be performed by performing both processing.

As a result, for example, among the sound signals collected by the sound collection device 10, the voice of the user speaking toward the sound collection device 10 is emphasized, and noise other than the user's voice is relatively weakened. obtain.

The parameter setting unit 245 has a function of setting parameters related to sound signal processing by the sound processing unit 243 in response to a parameter changing operation by a user of the information processing device 20.

More specifically, the parameter setting section 245 has a function of reading a parameter setting file stored in the storage section 220. The configuration file may be, for example, an ini format file. Furthermore, the storage unit 220 may store setting files having a plurality of patterns in which parameter setting values are different from each other.

The parameter setting section 245 sets parameters based on the above-mentioned setting file read with reference to the storage section 220. Further, the parameter setting unit 245 may adjust the value of each parameter based on the read configuration file, or may set the parameter with the adjusted value. Further, the parameter setting section 245 may cause the storage section 220 to store the adjusted values of each parameter as a new pattern setting file. The sound processing section 243 performs area sound collection processing based on the parameters set by the parameter setting section 245.

The screen generation unit 247 has a function of generating a parameter setting screen that accepts a parameter change operation by a user of the information processing device 20. The parameter setting screen generated by the screen generation section 247 is displayed on the operation display section 270 under the control of the display control section 235.

Here, the parameter setting screen generated by the screen generation unit 247 and the parameters that can be changed by the user of the information processing device 20 will be described with reference to FIGS. 4 to 6.

FIG. 4 is an explanatory diagram for explaining an example of a parameter setting screen generated by the screen generation unit 247. As shown in FIG. 4, the screen generation unit 247 may generate a GUI (Graphical User Interface) with a name such as "Settings Panel", which can accept the user's operations to change various parameters. .

In the example shown in FIG. 4, the parameter setting screen (setting panel) generated by the screen generation unit 247 includes the microphone gain (for example, setting range 0 to 50, unit dB) and the degree of ambient sound capture (for example, 0 to 50). 100 range) as changeable parameters. Further, as shown in FIG. 4, the parameter setting screen adjusts either or both of the degree of emphasis of target area sound and the degree of attenuation of ambient sound in area sound collection processing by the sound processing unit 243. It may also include parameters for effectiveness (for example, 21 levels from 0 to 20).

Furthermore, as shown in FIG. 4, the parameter setting screen generated by the screen generation unit 247 includes a default value button that can be pressed to set the value of each parameter to a predetermined initial value. Good too. The predetermined initial value may be a value set in advance by the provider of the sound processing application 231. Alternatively, the predetermined initial value may be updated by a change operation by the user. Alternatively, the predetermined initial value may be automatically updated. For example, the predetermined initial value may be automatically updated based on a learning model that is machine-learned using a user's change operation history as learning data.

FIG. 5 is an explanatory diagram for explaining another example of the parameter setting screen generated by the screen generation unit 247. As shown in FIG. 5, the screen generation unit 247 may generate a GUI that can accept more detailed operations for changing various parameters by the user, with a name such as "detailed settings", for example.

In the example shown in FIG. 5, the parameter setting screen generated by the screen generation unit 247 includes MIC_INTERVAL (microphone interval, in mm), MIC_GAIN_12 (interval MIC_GAIN_34 (mic gain ratio of ch1 and ch2 of the sound device 10) and MIC_GAIN_34 (microphone gain ratio of ch3 and ch4 of the sound pickup device 10).

In the example shown in FIG. 5, the parameter setting screen generated by the screen generation unit 247 includes INPUT_GAIN (input gain, unit dB) and , OUTPUT_GAIN (output gain, unit dB).

FIG. 6 is an explanatory diagram for explaining another example of the parameter setting screen generated by the screen generation unit 247. In the example shown in FIG. 6, the parameter setting screen generated by the screen generation unit 247 includes a radio button for setting the output sound when an area sound collection error occurs. The setting of the output sound when an area sound collection error occurs is a setting that allows specifying the sound to be output from the sound processing application 231 when an error occurs in the area sound collection processing by the sound processing unit 243. When through is selected in setting the output sound when an area sound collection error occurs, the sound processing unit 243 may output the sound signal obtained by the sound collection device 10 without being processed. Alternatively, if silence is selected, the sound processing unit 243 does not need to output a sound signal.

The example of the parameter setting screen generated by the screen generation unit 247 and each parameter have been described above with reference to FIGS. 4 to 6.

In addition to the parameters described above with reference to FIGS. 4 to 6, the parameters may include other parameters that can be used for area sound collection processing by the sound processing unit 243.

Returning to FIG. 3, the description of the functional configuration example of the sound processing application 231 will be continued. As shown in FIG. 3, the sound processing application 231 further has the functions of the virtual tool section 250. More specifically, the virtual tool section 250 is configured by the functions of a virtual speaker section 251 and a virtual microphone section 253.

The virtual speaker unit 251 functions as a virtual sound output device on the information processing device 20. For example, the virtual speaker unit 251 can be recognized as a sound output device by various applications running on the information processing device 20. A sound signal processed by the sound processing section 243 is input to the virtual speaker section 251 .

The virtual microphone unit 253 functions as a virtual sound input device on the information processing device 20. For example, the virtual microphone unit 253 can be recognized as a sound input device by various applications running on the information processing device 20. A sound signal output from the virtual speaker section 251 is input to the virtual microphone section 253 . Thereby, the user of the information processing device 20 can select the virtual microphone unit 253 as a sound input source device in various applications running on the information processing device 20. For example, the user of the information processing device 20 can specify the sound signal output from the virtual microphone section 253 as the sound signal input to the sound input section 260 of the conference application 233.

Further, the virtual microphone unit 253 has a name that corresponds to the sound processing application 231 as a name recognized by the information processing device 20.

FIG. 7 is an explanatory diagram showing an example of a selection screen in which the virtual microphone unit 253 is selected as a sound input source device in the conference application 233. The conference application screen D1 in FIG. 7 is an example of a screen of the conference application 233 displayed on the operation display section 270. As shown in FIG. 7, the conference application screen D1, like a general conference application, displays an image of the user of the information processing device 20 acquired by a camera (not shown) included in the information processing device 20, Contains video of the other party. Further, as shown in FIG. 7, the conference application screen D1 includes a sound input device specification field SB1 and a cursor C1.

As shown in FIG. 7, on the screen of the conference application 233, the user of the information processing device 20 operates the operation display unit 270 to move the cursor C1 to select the conference application in the sound input device specification field SB1. 233 can be set.

Further, the sound processing application 231 may function as two or more virtual sound output devices using the virtual speaker section 251. At this time, the same sound may be output to two or more sound output devices, or one may output sound with a certain delay. By having this function, even if the software connected to one of the audio outputs involves a delay, audio processing can be performed appropriately. For example, one virtual speaker can be set as the sound input source device for the conference application 233, and the other virtual speaker can be set as the sound input source device for the recording application.

Furthermore, as shown in FIG. 7, it is understood that the name "sound processing application microphone" is displayed in the sound processing application 231 as the name of the virtual microphone section 253 recognized by the information processing device 20. Ru. In the example shown in FIG. 7, the user of the information processing device 20 operates the operation display unit 270 to select the sound processing application microphone as the sound input source from among the plurality of sound input devices. This is an example of a screen in which the name "sound processing application microphone" is displayed in the device specification field SB1. An example of the selection screen in which the virtual microphone unit 253 is selected as a sound input source device in the conference application 233 has been described above with reference to FIG. 7 .

The functional configuration example of the information processing device 20 according to the present embodiment has been described above with reference to FIGS. 2 and 3. Next, an example of the operation processing of the information processing system according to this embodiment will be described with reference to FIG.

<3. Operation processing example>
FIG. 8 is a flowchart diagram for explaining an example of operational processing of the information processing system according to this embodiment. First, the device detection unit 241 of the information processing device 20 detects the connection state between the information processing device 20 and the sound collection device 10. Further, the device detection unit 241 detects whether a compatible device is connected to the information processing apparatus 20.

If the connection between the compatible device and the information processing device 20 is not detected (S101/NO), the sound processing unit 243 does not process the sound signal obtained by the sound collection device 10. At this time, the screen generation unit 247 generates a notification to the effect that the connection of the compatible device has been canceled and to the effect that the sound signal processing by the sound processing unit 243 has stopped. The display control unit 235 displays the generated notification on the operation display unit 270 (S117).

If the connection between the compatible device and the information processing apparatus 20 is detected (S101/YES), the screen generation unit 247 displays a message indicating that the connection of the compatible device has been detected, and the sound processing unit 243 starts processing the sound signal. Generates a notification to that effect. The display control unit 235 displays the generated notification on the operation display unit 270 (S103).

Next, the parameter setting unit 245 performs parameter setting processing (S105). At this time, the parameter setting unit 245 sets the parameters using preset initial values.

Subsequently, the sound processing unit 243 processes the sound signal obtained by the sound collection device 10, which is the target device (S107). The sound signal processed by the sound processing section 243 is input to the virtual speaker section 251 (S109). Further, the virtual speaker unit 251 outputs the input processed sound signal to the virtual microphone unit 253, and inputs the processed sound signal to the virtual microphone unit (S111).

Next, the parameter setting unit 245 recognizes whether the user of the information processing device 20 performs a parameter changing operation on the parameter setting screen (S113).

If a parameter changing operation is performed (S113/YES), the processes of S105 to S111 are repeated. Thereby, when a parameter changing operation is performed, the changed parameter can be immediately reflected in the sound signal processing by the sound processing unit 243.

If the parameter changing operation is not performed (S113/NO), the processes of S101 to S113 and S117 are repeated. In this way, the information processing system according to the present embodiment, when the connection of the compatible device is detected, processes the sound signal collected by the compatible device until the disconnection of the compatible device is detected. Repeat output of the processed sound signal to the virtual speaker and virtual microphone.

The operational processing example of the information processing system according to this embodiment has been described above with reference to FIG.

<4. Hardware configuration example>
An embodiment of the present invention has been described above. As described above, the information processing device 20 detects the connection state between the information processing device 20 and the sound collection device 10, verifies whether the sound collection device 10 is the target device, and performs the following operations on the sound signal obtained by the target device. The area sound collection processing and the processing of outputting the processed sound signal to the virtual speaker and the virtual microphone are realized by cooperation between software and the hardware of the information processing device 20. An example of the hardware configuration of the information processing device 20 according to an embodiment of the present invention will be described below.

Note that the hardware configuration example of the information processing device 20 described below is only an example of the hardware configuration of the information processing device 20. Therefore, with respect to the hardware configuration of the information processing device 20, unnecessary configurations may be deleted from the hardware configuration of the information processing device 20 described below, or new configurations may be added.

FIG. 9 is a diagram showing an example of the hardware configuration of the information processing device 20 according to an embodiment of the present invention. The information processing device 20 includes a CPU 1001, a ROM 1002, a RAM 1003, an internal bus 1004, an input/output interface 1010, a display device 1011, an input device 1012, an audio output unit 1013, a storage device 1014, and a drive 1015. , a network interface 1016, and an external interface 1017.

The CPU 1001 functions as an arithmetic processing device and a control device, and controls overall operations within the information processing device 20 according to various programs. For example, the functions of the control unit 230 can be realized by the CPU 1001 working together with a ROM 1002, a RAM 1003, and software, which will be described later.

The ROM 1002 stores programs used by the CPU 1001, calculation parameters, and the like. The RAM 1003 temporarily stores programs used in the execution of the CPU 1001 and parameters that change as appropriate during the execution.

The CPU 1001, ROM 1002, and RAM 1003 are interconnected by an internal bus 1004, and are further connected to a display device 1011, an input device 1012, an audio output unit 1013, a storage device 1014, a drive 1015, a network interface 1016, and It is connected to external interface 1017.

The display device 1011 is, for example, a display device such as a CRT display device, a liquid crystal display (LCD), or an OLED device, and converts video data into video and outputs the video. Further, the input device 1012 may include a mouse, a keyboard, a touch panel, a button, a microphone, a sensor, a switch, a control circuit, and the like. Furthermore, the audio output unit 1013 is an audio output device such as a speaker or headphones, and converts audio data into audio and outputs the audio.

The storage device 1014 is a data storage device configured as an example of the storage unit 220 according to this embodiment. The storage device 1014 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 1014 is configured with, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a memory having an equivalent function. This storage device 1014 drives storage and stores programs executed by the CPU 1001 or various data.

The drive 1015 is a reader/writer for a storage medium, and is built into the information processing device 20 or attached externally. The drive 1015 reads information stored in a removable storage medium such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs it to the RAM 1003. Drive 1015 can also write information to removable storage media.

The network interface 1016 is, for example, a communication interface configured with a device for connecting to a communication network such as the Internet. Further, the network interface 1016 may be a communication device compatible with a wireless LAN (Local Area Network), or may be a wire communication device that performs communication via a wired LAN.

The external interface 1017 includes a connection port for connecting an external connection device, such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, or an optical audio terminal. connection interface. Note that the external interface 1017 may be a connection port that supports short-range wireless communication such as Bluetooth or IrDA (Infrared Data Association) and connects an externally connected device via wireless communication.

<5. Supplement>
Although preferred embodiments of the present invention have been described above in detail with reference to the accompanying drawings, the technical scope of the present invention is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present invention can come up with various changes or modifications within the scope of the technical idea described in the claims. It is understood that these also naturally fall within the technical scope of the present invention.

For example, in the embodiment described above, only an example of the operation of the information processing device 20 when the sound processing application 231 of the information processing device 20 operates normally has been described. However, the present invention is not limited to such an example. For example, if an error occurs during execution of the sound processing application 231, the information processing device 20 may generate the following error notification and display it on the operation display unit 270.

For example, the information processing apparatus 20 may display a device error notification on the operation display unit 270. The device error may be, for example, an error in which area sound collection processing by the sound processing unit 243 is not started even though the sound processing application 231 is being executed. In this case, the screen generation unit 247 of the information processing device 20 may generate a notification saying "An area sound collection microphone was detected, but the area sound collection process could not be started." Furthermore, the screen generation unit 247 sends a notification that a problem has occurred in acquiring the audio device for some reason, a notification of an instruction to unplug the target device and plug it in again, and if the problem is still not resolved, information processing A notification of instructions to restart the device 20 may be generated.

Further, the device error may be that the sound processing application 231 is not installed on the information processing device 20. In this case, the screen generation unit 247 may generate a notification that "area sound collection processing cannot be started unless the sound processing application 231 is installed." Furthermore, if the area sound collection process is not started, the screen generation unit 247 may generate a notification that the sound processing application 231 is not installed and a notification of an instruction to perform the installation work according to the manual.

Further, the information processing device 20 may display a system error notification on the operation display unit 270. The system error may be that the device detection unit 241 detected the target device but could not start area sound collection processing. In this case, the screen generation unit 247 may generate a notification such as "An area sound collection microphone was detected, but the process could not be started." In addition, if the area sound collection process does not start, the screen generation unit 247 notifies you that a problem has occurred in acquiring the input device for some reason, and also provides an instruction to remove the compatible device from the information processing apparatus 20 and then insert it again. and, if the problem is still not resolved, a notification of an instruction to restart the information processing device 20.

Further, the information processing device 20 may cause the operation display unit 270 to display a notification of an error in continuous detection of silence by the sound collection device 10. In this case, the screen generation unit 247 may generate a notification saying "The area sound collection microphone is connected, but the audio is interrupted." In addition, if a notification saying "The area sound collection microphone is connected, but the audio is interrupted" is displayed, the screen generation unit 247 generates a notification that silence is being continuously input from the compatible device. Then, a notification prompting the user to confirm whether the sound collection device 10 and the wiring of the main body base of the information processing device 20 are securely connected may be generated. Further, after displaying such an error notification once, the operation display unit 270 may not display the error notification again until the target device is once removed from the information processing apparatus 20.

Further, the sound processing application 231 may have a howling detection function, and may display a notification of an error in howling detection by the sound collection device 10 on the operation display unit 270. In this case, the screen generation unit 247 may generate a notification that "howling is occurring." Further, the control unit 230 may set the output of the virtual microphone unit 253 to be silent for a certain period of time after howling is detected.

Further, the steps in the processing of the operations of the sound collection device 10 and the information processing device 20 according to the present embodiment do not necessarily need to be processed in chronological order in the order described as the explanatory diagram. For example, each step in processing the operations of the sound collection device 10 and the information processing device 20 may be processed in a different order from the order described as the explanatory diagram, or may be processed in parallel.

Furthermore, it is also possible to create one or more computer programs for causing the hardware such as the CPU, ROM, and RAM built in the above-mentioned sound collection device 10 and information processing device 20 to exhibit the functions of the information processing system according to this embodiment. It is. Also provided is a computer readable storage medium storing the one or more computer programs.

Further, the effects described in this specification are merely explanatory or illustrative, and are not limiting. In other words, the technology according to the present invention can have other effects that are obvious to those skilled in the art from the description of this specification, in addition to or in place of the above effects.

<6. Others>
<6.1. Formation of sound processing application 1>
In the above, when the information processing device 20 is started, the control unit 230 executes the sound processing application program (main unit 240 and virtual tool unit 250 installed by the installer) stored in the storage unit 220 to collect area sound. It has been explained that the sound processing application 231 that can perform processing is formed. Here, the virtual tool unit 250 may be a driver program that enables mutual exchange between the OS and a device (for example, the sound collection device 10, etc.).

For example, when the information processing device 20 is started, the virtual tool unit 250 is dispatched by a system program such as a service control program or an audio management program in the OS of the information processing device 20 (the hardware is represented based on the virtual tool unit 250 which is a driver). The sound processing application 231 generates a device object, initializes the device, makes the device name available to the application, and puts it in a state where it can process requests for the device from the application).Then, the main unit 240 is loaded and executed. is formed.

In this case, the main unit 240 may be configured as an executable file program (for example, a program whose entry point is the “WinMain()” function), and the virtual tool unit 250 may be configured as a driver called from another program. (for example, a program whose entry point is the "DriverEntry()" function).

Furthermore, in the file system of the OS of the information processing device 20, the main unit 240 may be configured as a program with an extension ".exe" indicating an executable file, such as the file name "Hontai.exe". Often, the virtual tool unit 250 may be configured as a program with an extension ".sysy" representing a driver, such as a file name "KasouTool.sys".

Note that system programs such as the service control program and the audio management program search for the program of the virtual tool section 250, which is a driver, in a predetermined area of the file system of the OS of the information processing device 20 (for example, the file name of the virtual tool section 250 is searched for). and the "DriverEntry()" function in the program) and dispatch. Here, the virtual tool unit 250 may be installed by an installer so as to be placed in a predetermined area of the OS file system (for example, "%SystemRoot%\System32\Drivers").

Note that if the virtual tool section 250 is a driver program, when the user of the information processing device 20 manually terminates the sound processing application 231, the main body section 240 will terminate, even if the virtual tool section 250 is not terminated. good. In other words, the sound processing application 231 by the virtual tool unit 250 may be maintained.

<6.2. Formation of sound processing application 2>
In the above, when the information processing device 20 is started, the control unit 230 executes the sound processing application program (main unit 240 and virtual tool unit 250 installed by the installer) stored in the storage unit 220 to collect area sound. It has been explained that the sound processing application 231 that can perform processing is formed. However, the sound processing application 231 may be created when the connection of the sound collection device 10, which is the target device, is detected.

<6.2.1. Activation of sound processing unit 243>
For example, when the information processing device 20 is started, the main unit 240 that does not include the sound processing unit 243 and the virtual tool unit 250 are loaded and executed, so that the sound processing application 231 by the main unit 240 that does not include the sound processing unit 243 is formed. In this state, the sound processing application 231 by the main body section 240 that does not include the sound processing section 243 can perform the functions of the device detection section 241, parameter setting section 245, and screen generation section 247 of the main body section 240. After that, the information processing device 20 and the sound collection device 10 are connected, and when the sound processing application 231 by the main body 240 that does not include the sound processing section 243 detects the connection of the sound collection device 10, which is the target device, the main body 240, the sound processing section 243 is loaded and executed, and a sound processing application 231 is formed by the main body section 240 including the sound processing section 243 and the virtual tool section 250. In this state, the sound processing application 231 by the main body section 240 and the virtual tool section 250 has the functions of the device detection section 241, sound processing section 243, parameter setting section 245, and screen generation section 247 of the main body section 240, as well as the functions of the virtual tool section 240 and the virtual tool section 250. The functions of the virtual speaker section 251 and the virtual microphone section 253 of the section 250 can be exhibited. That is, when the connection of the sound collection device 10, which is the target device, is detected, the sound processing application 231 having the area sound collection processing function (first function) is formed.

In this case, the main unit 240 may be configured as an executable file program (for example, a program whose entry point is the “WinMain()” function), and the sound processing unit 243 may be configured as a library called from other programs (Dynamic Linking Library). ) (for example, a program whose entry point is the "DllEntryPoint( )" function).

Furthermore, in the file system of the OS of the information processing device 20, the main unit 240 may be configured as a program with an extension ".exe" indicating an executable file, such as a file name "Hontai.exe". Often, the sound processing unit 243 may be configured as a program with an extension ".dll" indicating a library, such as a file name "OtoSyori.dll".

Note that the main body section 240 includes a calling function (for example, "LoadLibrary()" function) for dynamically calling a program of the sound processing section 243 that is a library, and the device detecting section 241 detects the sound collecting device 10 that is a target device. When a connection is detected, the program name of the sound processing unit 243, which is a library, is specified and the call function is executed (for example, the file name "OtoSyori.dll" on the file system of the OS of the information processing device 20 is used as a parameter). The program of the sound processing unit 243, which is a library, may be loaded and executed by executing the function "LoadLibrary("OtoSyori.dll")".

<6.2.2. Activation of virtual tool section 250>
For example, when the information processing device 20 is started, the main body section 240 is loaded and executed, thereby forming the sound processing application 231 by the main body section 240. In this state, the sound processing application 231 by the main body section 240 can perform the functions of the device detection section 241, the sound processing section 243, the parameter setting section 245, and the screen generation section 247 included in the main body section 240. After that, the information processing device 20 and the sound collection device 10 are connected, and when the sound processing application 231 of the main body section 240 detects the connection of the sound collection device 10, which is the target device, the virtual tool section 250 is activated by the main body section 240. The program is loaded and executed, and a sound processing application 231 is formed by the main body section 240 and the virtual tool section 250. In this state, the sound processing application 231 by the main body section 240 and the virtual tool section 250 has the functions of the device detection section 241, sound processing section 243, parameter setting section 245, and screen generation section 247 of the main body section 240, as well as the functions of the virtual tool section 240 and the virtual tool section 250. The functions of the virtual speaker section 251 and the virtual microphone section 253 of the section 250 can be exhibited. That is, when the connection of the sound collection device 10, which is the target device, is detected, the sound processing application 231 having the functions of a virtual speaker and a virtual microphone (third function) is formed.

In this case, the main unit 240 may be configured as an executable file program (for example, a program whose entry point is the “WinMain()” function), and the virtual tool unit 250 may be configured as a library program called from another program (for example, It may be configured as a program whose entry point is the "DllEntryPoint( )" function.

Furthermore, in the file system of the OS of the information processing device 20, the main unit 240 may be configured as a program with an extension ".exe" indicating an executable file, such as the file name "Hontai.exe". Often, the virtual tool unit 250 may be configured as a program with an extension ".dll" indicating a library, such as a file name "KasouTool.dll".

Note that the main body section 240 includes a calling function (for example, "LoadLibrary()" function) for dynamically calling a program of the virtual tool section 250 that is a library, and the device detecting section 241 detects the sound collection device 10 that is a target device. When a connection is detected, the program name of the virtual tool unit 250, which is a library, is specified and the call function is executed (for example, the file name "KasouTool.dll" on the file system of the OS of the information processing device 20 is used as a parameter). The program of the virtual tool section 250, which is a library, may be loaded and executed by executing the function "LoadLibrary("KasouTool.dll")".

<6.2.3. Activation of sound processing unit 243 and virtual tool unit 250>
For example, when the information processing device 20 is started, the main body section 240 that does not include the sound processing section 243 is loaded and executed, thereby forming the sound processing application 231 by the main body section 240 that does not include the sound processing section 243. In this state, the sound processing application 231 by the main body section 240 that does not include the sound processing section 243 can perform the functions of the device detection section 241, parameter setting section 245, and screen generation section 247 of the main body section 240. Thereafter, when the information processing device 20 and the sound collection device 10 are connected, and the sound processing application 231 by the main body 240 that does not include the sound processing section 243 detects the connection of the sound collection device 10, which is the target device, the main body 240, the sound processing section 243 and the virtual tool section 250 are loaded and executed, and a sound processing application 231 is formed by the main body section 240 including the sound processing section 243 and the virtual tool section 250. In this state, the sound processing application 231 by the main body section 240 and the virtual tool section 250 has the functions of the device detection section 241, sound processing section 243, parameter setting section 245, and screen generation section 247 of the main body section 240, as well as the functions of the virtual tool section 240 and the virtual tool section 250. The functions of the virtual speaker section 251 and the virtual microphone section 253 of the section 250 can be exhibited. That is, when the connection of the sound collection device 10, which is the target device, is detected, the sound processing application 231 having the function of area sound collection processing (first function), the function of virtual speaker and virtual microphone (third function) is formed.

In this case, the main unit 240 may be configured as an executable file program (for example, a program whose entry point is the “WinMain()” function), and the sound processing unit 243 and the virtual tool unit 250 may be configured as The program may be configured as a library program (for example, a program whose entry point is the "DllEntryPoint()" function).

Furthermore, in the file system of the OS of the information processing device 20, the main unit 240 may be configured as a program with an extension ".exe" indicating an executable file, such as the file name "Hontai.exe". Often, the sound processing section 243 may be configured as a program with an extension ".dll" representing a library, such as the file name "OtoSyori.dll", and the virtual tool section 250 may be configured as a program with the file name "KasouTool", for example. The program may be configured as a program with an extension ".dll" indicating a library, such as ".dll".

The main unit 240 includes a calling function (for example, a “LoadLibrary()” function) for dynamically calling programs of the sound processing unit 243 and the virtual tool unit 250, which are libraries, and the device detecting unit 241 detects the target device. When the connection of the sound collection device 10 is detected, the name of the program of the sound processing unit 243, which is a library, is specified and the calling function is executed (for example, the file name "OtoSyori. Execute the function "LoadLibrary ("OtoSyori.dll")" with "OtoSyori.dll" as a parameter), specify the name of the program of the virtual tool unit 250, and execute the calling function (for example, By executing the function "LoadLibrary ("KasouTool.dll")" with the file name "KasouTool.dll" as a parameter, the programs of the sound processing section 243 and virtual tool section 250, which are libraries, are loaded and executed. good.

<6.2.4. Others＞
Further, the main body section 240 may be implemented on a driver that is loaded by the OS of the information processing device 20 at startup, or automatically loaded when connection of the sound collection device 10 is detected, for example. good.

<6.3. Obtaining hardware identification information>
In the above, it has been explained that the information processing device 20 acquires the hardware identification information of the sound collection device 10 that allows the detected sound collection device 10 connected to the external interface 1017 to be uniquely identified. Here, when acquiring the hardware identification information of the sound collection device 10, the device detection unit 241 may acquire the hardware identification information of the sound collection device 10 actively or passively.

<6.3.1. Actively obtaining hardware identification information>
When the hardware identification information of the sound collection device 10 is actively acquired, the device detection unit 241 refers to the management information held by the OS, and detects the hardware identification information of the sound collection device 10 included in the referenced management information. get.

In this case, the device detection unit 241 has an interface for accessing the management information and a keyword used when referring to the management information. This keyword is a search character related to the hardware identification information of the sound collection device 10.

Here, this keyword has a format corresponding to each character code such as EUC (Extended Unix Code)-JPP (Japan Extended Unix Code) and Shift JIS. The keyword allows management information to be referred to using a character code in the same format as the character code supported by the OS of the information processing device 20. That is, when the OS of the information processing apparatus 20 supports the Shift JIS character code, the device detection unit 241 refers to the management information using a keyword in the Shift JIS character code format.

Here, the management information is a registry, an initial setting file, a system setting file, etc. held by the OS, and is information for managing the status and setting information of the OS and programs. For example, when new hardware is added to the information processing device 20, the hardware identification information of the added new hardware is added to the management information.

<6.3.2. Passive hardware identification information >
When the hardware identification information of the sound collection device 10 is passively acquired, the device detection unit 241 acquires the hardware identification information of the sound collection device 10 based on a message sent from the OS.

In this case, the device detection unit 241 has an interface that receives messages sent from the OS.

Here, the message sent from the OS is an event-driven message sent when new hardware is added to the information processing device 20. For example, when the sound collection device 10 is added as new hardware to the information processing device 20, the OS of the information processing device 20 sends a message containing hardware identification information of the sound collection device 10.

<6.3.3. Example of hardware identification information>
The hardware identification information acquired by the device detection unit 241 includes, for example, a vendor ID (VID), a product ID (PID), a serial number (Serial Number), and a combination of these related to the sound collection device 10. It may be something.

Here, for example, the vendor ID is an ID assigned to a device manufacturer, the product ID is an ID assigned to a product by the manufacturer, and the serial number is an ID assigned to a product by the manufacturer. It is a serial number.

<6.4. Detecting target devices>
In the above, the device detection unit 241 acquires hardware identification information of the sound collection device 10 that makes it possible to uniquely identify the sound collection device 10 whose connection with the information processing device 20 has been detected, and acquires the acquired hardware identification information. It has been explained that whether or not the device is a target device is detected based on the information. However, it is also possible to detect whether the target device is the target device based on information related to the target device other than the hardware identification information of the sound collection device 10 (for example, vendor ID, product ID, serial number, or a combination thereof).

For example, when detecting whether the sound collection device 10 is a target device based on the device name, the storage unit 220 stores the target device name, and the device detection unit 241 detects whether the device is connected to the information processing device 20 or not. The device name of the sound collection device 10 that allows the detected sound collection device 10 to be uniquely identified is acquired. Then, the device detection unit 241 checks whether the acquired device name matches the target device name held in the storage unit 220. Thereby, the device detection unit 241 detects whether or not the sound collection device 10 whose connection to the information processing device 20 has been detected is the target device, based on the device name of the sound collection device 10 . Note that there may be a plurality of target device names held in the storage unit 220.

For example, when detecting whether the sound collection device 10 is a target device based on the device name and hardware identification information, the storage unit 220 stores the hardware identification information and the target device name as a pair; The device detection unit 241 acquires the device name and hardware identification information of the sound collection device 10 that allows the sound collection device 10 whose connection with the information processing device 20 has been detected to be uniquely identified. Then, the device detection unit 241 checks whether the acquired device name and hardware identification information match the device name and hardware identification information held in the storage unit 220. Thereby, the device detection unit 241 detects whether or not the sound collection device 10 whose connection to the information processing device 20 has been detected is the target device, based on the device name and hardware identification information of the sound collection device 10. . Note that there may be a plurality of hardware identification information and target device names held in the storage unit 220.

In this case, when comparing the acquired device name and hardware identification information, the device detection unit 241 may perform the comparison based on the device name or the hardware identification information. Alternatively, the hardware identification information may be checked based on the device name, and the hardware identification information may not be checked at the time it does not match the target device name held in the storage unit 220.

Note that, as described above, the device name of the sound collection device 10 may be acquired based on management information held by the OS or a message sent from the OS.

<6.5. Use of sound signals processed by area sound collection in thin client systems>
In the above description, in the information processing device 20 that can be started and operated standalone, the program of the sound processing application is installed, so that the information processing device 20 can perform the functions (the first function, the second function) by the sound processing application. The third function) was explained. That is, in the above description, the information processing device 20 is a fat client type device. However, the information processing device 20 may be a thin client type device.

<6.5.1. Download method (net boot method) thin client>
For example, if the information processing device 20 is a device compatible with a download-based thin client, the information processing device 20 receives a terminal OS and a terminal OS from a server that supports the download-based thin client arranged on the network. Download and run various programs that can be run on the computer. Note that the sound processing application program (main unit 240 and virtual tool unit 250) is included in various programs executable on the terminal OS.

In this case, the information processing device 20 has a download type thin client function unit that requests download via the network to a server corresponding to the download type thin client arranged on the network. In addition, in response to a download request from the information processing device 20, a server arranged on the network that supports the download-type thin client downloads the terminal OS and various programs executable on the terminal OS via the network. It has a download-type thin client server function unit for distribution. Requests and downloads are realized through interaction between the download method thin client function section and the download method thin client server function section.

Here, for example, the download method thin client function section and the download method thin client server function section may be function sections compatible with PXE (Preboot eXecution Environment), and the download method thin client function section is compliant with PXE. , download method Obtains the path name of NBP (Network Bootstrap Program) from the thin client server function section, downloads NBP (a program for obtaining and starting an OS image) based on the path name, and executes NBP. This downloads an OS image (an image file containing a terminal OS and various programs executable on the terminal OS), and starts the information processing device 20 by executing the OS image. Note that the download method thin client function unit acquires NBP based on the TFTP (Trivial File Transfer Protocol) communication protocol, and executes the NBP to transfer to HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol). r Protocol Secure) and FTP ( The OS image is acquired based on a communication protocol such as File Transfer Protocol (NFS) or Network File System (NFS).

When the information processing device 20 compatible with the download-based thin client downloads the terminal OS and various programs executable on the terminal OS, it executes the terminal OS and also downloads the sound included in the various programs. The processing application program (main unit 240 and virtual tool unit 250) is executed. By doing so, the sound processing application is activated, and the sound processing application 231 is formed on the information processing device 20.

By doing so, the information processing device 20, which is a download type thin client, can perform the functions (first function, second function, and third function) by the sound processing application.

Note that the sound processing application 231 is formed in the information processing device 20, which is a download-based thin client, as described above, when the connection of the sound collection device 10, which is the target device, is detected. There may be.

<6.5.2. Screen transfer method thin client>
For example, if the information processing device 20 is a device that supports a thin client using the screen transfer method, the information processing device 20 and a server that supports the thin client using the screen transfer method arranged on the network communicate with each other, and the server's memory The terminal OS and various programs executable on the terminal OS are executed, and the execution results are displayed on the information processing device 20. Note that the sound processing application program (main unit 240 and virtual tool unit 250) is included in various programs executable on the terminal OS.

In this case, the information processing device 20 has a screen transfer type thin client function unit that requests control via the network from a server corresponding to a screen transfer type thin client arranged on the network and receives the control result. . In addition, in response to a control request from the information processing device 20, a server that supports a screen transfer type thin client arranged on the network runs an OS for the terminal and various programs executable on the OS for the terminal in its memory. It has a screen transfer type thin client server function unit that executes the execution and provides the execution result (control result) to the information processing device 20. The exchange of control requests and control results is realized by the interaction between the screen transfer method thin client function section and the screen transfer method thin client server function section.

Here, for example, the screen transfer method thin client function section and the screen transfer method thin client server function section may be function sections compatible with RDP (Remote Desktop Protocol). The screen transfer type thin client function unit complies with RDP and transfers keyboard operations, mouse operations, voice inputs, etc. in the information processing device 20 to the screen transfer type thin client server function unit as control requests. The screen transfer method thin client server function section complies with RDP and receives keyboard operations, mouse operations, voice input, etc. on the information processing device 20 as control requests from the screen transfer method thin client function section, and stores them in the memory of the server. It is reflected in the control of the terminal OS to be executed and various programs executed on the terminal OS.

The information processing device 20 that supports the screen transfer type thin client requests the server that supports the screen transfer type thin client to control the startup of the terminal OS and various programs executable on the terminal OS. Then, the server corresponding to the screen transfer type thin client executes the OS for the terminal and also executes the sound processing application program (main unit 240 and virtual tool unit 250) included in various programs. By doing so, the sound processing application is started on the server that supports the screen transfer type thin client, and the sound processing application 231 becomes available on the information processing device 20.

By doing so, the functions (first function, second function, and third function) provided by the sound processing application can be used in the information processing device 20, which is a thin client using the screen transfer method.

Note that the formation of a sound processing application in the information processing apparatus 20, which is a thin client of the screen transfer method (formation of a sound processing application in the server corresponding to the thin client of the screen transfer method), is performed on the sound processing device 20, which is the target device, as described above. The sound processing application 231 may be created when the connection of the device 10 is detected.

<6.6. Connection between sound collection device 10 and information processing device 20>
In the above, it has been described that the sound collection device 10 and the information processing device 20 are configured to be able to communicate through a wired connection or a wireless connection. However, the connection between the sound collection device 10 and the information processing device 20 may be realized by a pass-through function provided by various virtualization software. More specifically, part of the connection between the sound collection device 10 and the information processing device 20 may be performed in software using a USB pass-through function or a PCIe pass-through function.

<6.7. Example of an application that uses sound signals subjected to area sound collection processing>
In the above, the sound signal collected by the sound collection device 10 is subjected to area sound collection processing on the information processing device 20 and is passed to the conference application 233, so that the sound signal is uttered toward the sound collection device 10. It has been explained that the user's voice can be emphasized and noise other than the user's voice can be relatively weakened. However, the application (utilization application) that uses the sound signal subjected to area sound collection processing may be, for example, an application such as a telephone call, recording, voice distribution, voice recognition, state detection, measurement/measurement, or the like.

For example, when a phone call application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as the device from which the sound is input in the phone call application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to the area sound collection processing is input to the call application, and the area sound collection is sent to the other party via the network. A processed sound signal can be transmitted (for example, a sound signal can be compressed and encoded using an audio codec such as G.711, G.723, G.729, or G.729a, and then transmitted).

For example, when a recording application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as a sound input source device in the recording application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to area sound collection processing is input to a recording application, and the sound signal subjected to area sound collection processing is recorded (e.g. Audio signals can be converted into MP3 (MPEG-1 Audio Layer-3), WAV (RIFF waveform Audio Format), WMA (Windows Media Audio), AAC (Advanced Audio Coding), or AC3 (Audio Code number 3) and FLAC (Free Lossless Audio Codec ) can be saved as a file in a format such as ).

For example, when the audio distribution application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as the device from which the sound is input in the audio distribution application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to area sound collection processing is input to an audio distribution application, and the sound signal subjected to area sound collection processing is audio distributed. (For example, a sound signal can be uploaded to a server on a network as a file in an MP3, WAV, WMA, AAC, AC3, FLAC, etc. format and distributed from the server on the network).

For example, when a voice recognition application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as a sound input source device in the voice recognition application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to area sound collection processing is inputted to a speech recognition application, and audio is generated based on the sound signal subjected to area sound collection processing. can be recognized. Note that as a method of speech recognition, for example, a hidden Markov model, which is a probabilistic model that receives a sound signal subjected to area sound collection processing as input and outputs a speech recognition result, may be used. Furthermore, the speech recognition method is not limited to the hidden Markov model, and machine learning methods such as deep learning, artificial neural networks, random forests, and gradient boosting may also be used.

For example, when the state detection application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as the device from which the sound is input in the state detection application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to the area sound collection processing is input to the state detection application, and the state detection application learns the normal sound signal. Input the sound signal that has undergone area sound collection processing into a learning model that has been subjected to machine learning as data, and determine whether the sound signal that has undergone area sound collection processing is an abnormal sound or the sound signal that has undergone area sound collection processing. By outputting the probability that the sound is an abnormal sound, the learning model makes it possible to detect an abnormal state of the source of the sound signal subjected to area sound collection processing. Note that machine learning methods include, but are not limited to, deep learning, artificial neural networks, random forests, gradient boosting, and the like. In addition, the sources of sound signals subjected to area sound collection processing include machine operating sounds, product operating sounds, vehicle running sounds, animal footsteps, animal voices, animal heart sounds, and animal breathing sounds. However, it is not limited to these.

For example, when the fish finder application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as the device from which the sound is input in the fish finder application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to the area sound collection processing is inputted to the fish finder application, and the sound signal subjected to the area sound collection processing is used to detect fish schools. Detection (for example, collecting the sound signals from the ultrasonic waves output from the transmitter reflected by schools of fish, converting the sound signals into digital signals, analyzing them, and creating an echogram image) is possible.

For example, when an underwater anomaly detection application that detects poaching uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as a sound input source device in the underwater anomaly detection application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to area sound collection processing is input to an underwater abnormality detection application, and the sound signal subjected to area sound collection processing is Anomaly detection (for example, detecting sounds with abnormal frequencies from sound signals, such as the sound of a ship's propeller or the sound of air passing through a regulator when a diver draws air from a cylinder) )can do.

For example, when a sound visualization application uses a sound signal subjected to area sound collection processing, the virtual microphone unit 253 is designated as a sound input source device in the sound visualization application. Then, area sound collection processing is applied to the sound signal obtained by the sound collection device 10, and the sound signal subjected to the area sound collection processing is input to a sound visualization application, and the sound pressure is calculated from the sound signal subjected to the area sound collection processing. The distribution can be acquired and converted into a color map (the sound pressure is measured from the phase difference information from the sound source to the sound collection device 10 and converted into a heat map).

10 sound collection device 110 microphone array 20 information processing device 210 communication unit 220 storage unit 230 control unit 231 sound processing application 233 conference application 235 display control unit 240 main unit 241 device detection unit 243 sound processing unit 245 parameter setting unit 247 screen generation unit 250 Virtual tool section 251 Virtual speaker section 253 Virtual microphone section 260 Sound input section 270 Operation display section 280 Sound output section

Claims (16)

An information processing device,
a detection unit that detects a connection state with the sound collection device;
a processing unit that processes a sound signal obtained by a sound collection device whose connection with the information processing device has been detected by the detection unit;
a virtual speaker section that functions as a virtual sound output device and receives a processed sound signal that is a sound signal processed by the processing section;
An information processing device comprising: The information processing device includes:
a sound processing application including the detection unit and the processing unit;
a virtual tool having the virtual speaker section and a virtual microphone section that functions as a virtual sound input device and into which the processed sound signal output from the virtual speaker section is input;
Equipped with
The information processing apparatus according to claim 1, wherein the virtual tool is set in advance by a provider of the sound processing application to an output destination of the processed sound signal from the sound processing application. The information processing device according to claim 2, wherein the virtual microphone unit has a name corresponding to the sound processing application as a name recognized by the information processing device. The detection unit includes:
obtaining hardware identification information of the sound collection device with which connection has been detected;
Verifying whether the hardware identification information matches target device identification information held in a storage unit;
The processing unit starts processing the sound signal obtained by the sound collection device when the hardware identification information matches the target device identification information,
If the hardware identification information does not match the target device identification information, the sound signal is not processed and the sound signal is output to the virtual speaker section.
The information processing device according to claim 3. The processing unit includes:
When the detection unit detects that the connection of the sound collection device whose connection has been detected has been released, processing of the sound signal obtained by the sound collection device is stopped;
The information processing device according to claim 4. a screen generation unit that generates notifications and screens displayed on the display unit;
further comprising a display control unit that controls display on the display unit,
When the detection unit detects the connection of the sound collection device, the screen generation unit generates a notification that a device for processing the sound signal by the processing unit has been detected,
The display control unit displays the notification on the display unit,
The information processing device according to claim 5. When the detection unit detects that the sound collection device is disconnected, the screen generation unit generates a notification that the connection of the target device for processing the sound signal by the processing unit is disconnected. death,
The display control unit displays the notification on the display unit,
The information processing device according to claim 6. further comprising a parameter setting unit that sets parameters related to processing of the sound signal by the processing unit in response to a parameter change operation by a user;
The screen generation unit generates a parameter setting screen that accepts an operation to change the parameter by the user,
the display control section causes the display section to display the parameter setting screen;
The information processing device according to claim 7. The processing unit includes:
9. The information processing apparatus according to claim 8, wherein the sound signal obtained by the sound collection device is subjected to area sound collection processing, which is sound signal processing for emphasizing sound components in a target area. The processing unit includes:
performing the area sound collection processing by performing processing on the sound signal obtained by the sound collection device to weaken ambient sound components other than the sound components of the target area;
The information processing device according to claim 9. The processing unit includes:
performing the area sound collection processing by performing processing to amplify the sound component of the target area on the sound signal obtained by the sound collection device;
The information processing device according to claim 9 or 10. The information processing apparatus according to claim 2, wherein the output destination of the processed sound signal is a usage application that uses the processed sound signal. The information processing apparatus according to claim 12, wherein the usage application is a state detection application that detects a state abnormality of a source of the processed sound signal by machine learning based on the processed sound signal. computer,
a detection unit that detects a connection state between the information processing device and the sound collection device;
a processing unit that processes a sound signal obtained by a sound collection device whose connection with the information processing device has been detected by the detection unit;
a virtual speaker section that functions as a virtual sound output device and receives a processed sound signal that is a sound signal processed by the processing section;
A program to make it work. detecting the connection state between the information processing device and the sound collection device;
processing a sound signal obtained by a sound collection device whose connection with the information processing device has been detected;
Inputting a processed sound signal, which is a processed sound signal, into a virtual speaker unit functioning as a virtual sound output device;
An information processing method performed by a computer, including: a sound collection device;
a detection unit that detects a connection state between the information processing device and the sound collection device;
a processing unit that processes a sound signal obtained by a sound collection device whose connection with the information processing device has been detected by the detection unit;
a virtual speaker section that functions as a virtual sound output device and receives a processed sound signal that is a sound signal processed by the processing section;
information processing systems, including

Images