JP2000209689A - Sound processor, its control method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a sound processing unit where sound pickup directivity is set independently by a plurality of clients and remotely controlled and to obtain its control method. SOLUTION: A sound pickup section 102 has a microphone array and a memory storing respectively sound information picked up by each microphone, reads sound information from the memory on the basis of a result of calculating sound pickup timing of each microphone depending on a control instruction received from each client, synthesizes the read sound information and transmits the result to each client. In the case that a plurality of clients make a transmission request of sound information, the above processing is conducted according to the control instruction received from each client and the sound information is transmitted to the clients issuing the control instruction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound processing apparatus and a control method therefor, for example, a sound processing apparatus capable of remotely controlling sound collecting directivity and a control method thereof, and a sound processing apparatus for remotely controlling such an apparatus. The present invention relates to an apparatus and a control method thereof.

[0002]

2. Description of the Related Art Using a microphone array capable of changing directionality, a device that focuses on sound including sound coming from a specific direction, and a gaze position autofocus (AF) function of a video camera device A line-of-sight position sound pickup device that changes the directivity of the microphone in conjunction with it, a device that changes the sound pickup direction by operating the camera direction by using a directional microphone fixed to a remotely controllable camera, etc. is there.

[0003]

However, in the above sound pickup device, the output of the obtained sound signal and the control of the directivity are performed by only one system, and the output of the sound signal from a large number of clients connected to the network. It does not respond to the case where directivity control is required.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and has a sound processing apparatus capable of remotely controlling sound collecting directivity, in which a plurality of clients can independently set sound collecting directivity, and a sound processing apparatus therefor. It is an object to provide a control method.

It is another object of the present invention to provide a sound processing apparatus which easily displays the sound collecting directivity of a sound collecting apparatus capable of remotely controlling the sound collecting directivity, and provides a sound processing apparatus and a control method therefor. For other purposes.

[0006]

The present invention has the following configuration as one means for achieving the above object.

A sound processing device according to the present invention is a sound processing device capable of remotely controlling sound collecting directivity, and includes a sound collecting array having a plurality of sound collecting devices, and sound collecting by each of the plurality of sound collecting devices. A plurality of storage means for respectively storing the obtained sound information, and a control means for calculating the sound collection timing in each sound collection device based on the received control command, and reading out the sound information from the plurality of storage means based on the calculation result. And transmission means for transmitting a result of synthesizing the sound information read from the plurality of storage means, and when there are transmission requests for sound information from a plurality of clients, the control means The plurality of storage units based on sound collection timing in each sound collection device calculated according to the received control command It reads Luo sound information, the transmission unit and transmitting to the client the result of combining the sound information corresponding to the control command, which has issued the control command.

A sound processing device for remotely controlling a sound collecting device capable of controlling sound collecting directivity, a display means for displaying an image indicating a sound collecting area sent from the sound collecting device; Directional display means for superimposing on the image displayed on the display means to display the sound collecting directivity of the sound collecting device, and input means for inputting a command for controlling the directivity of the sound collecting device. It is characterized by having.

A sound processing apparatus control method according to the present invention is directed to a sound collecting array having a plurality of sound collecting devices, and a plurality of storage means for storing sound information collected by each of the plurality of sound collecting devices. A method of controlling a sound processing apparatus capable of remotely controlling sound collecting directivity, wherein the sound collecting timing in each sound collecting device is calculated based on a received control command, and the plurality of sound collecting timings are calculated based on the calculation result. The sound information is read from the storage means, a result of synthesizing the sound information read from the plurality of storage means is transmitted, and when there is a transmission request for the sound information from a plurality of clients, the control received from each of the clients is performed. Calculating a sound pickup timing in each sound pickup device according to a command; It reads Luo sound information, the result of combining the sound information corresponding to the control command,
The control command is transmitted to the issuing client.

[0010] A method of controlling a sound processing apparatus for remotely controlling a sound collection device capable of controlling sound collection directivity, the method comprising displaying an image indicating a sound collection area sent from the sound collection device. , Superimposed on the image showing the area where several sounds are possible,
A sound collection directivity of the sound pickup device is displayed, and a command for controlling the directivity of the sound pickup device is input.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a sound collecting device and a sound reproducing device according to an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing a configuration example of a sound collecting device and a sound reproducing device of the present embodiment.

In FIG. 1, reference numeral 100 denotes a sound pickup device, which is mainly constituted by blocks 101 to 106 described below. Reference numeral 200 denotes a sound reproducing device, which is mainly constituted by blocks 201 to 209 described below.

[Sound Pickup Apparatus] The imaging unit 101 includes a lens unit, a charge-coupled device (CCD) that converts an optical image incident through the lens unit into an electric signal, and an electric signal obtained from the CCD. It consists of an analog-to-digital (A / D) converter for converting to a digital image signal.

The plurality of sound pickup units 102 include a microphone, which is a sound pickup device that converts sound including sound into an electric signal,
And an A / D converter for converting an electric signal obtained from the microphone into a digital sound signal.

The network interface (I / F) 103 is
The video and sound data obtained from the imaging unit 101 and the sound collection unit 102 are sent to the sound reproduction device 200, and control information necessary for directivity control of the sound collection unit 102 is received from the sound reproduction device 200.

The CPU 104 controls the entire sound pickup apparatus 100. The CPU 104 reads a program stored in the ROM 105 into the RAM 106, and performs appropriate processing for each unit based on the program and control information received from the sound reproduction apparatus 200. Or let the action take place.

The ROM 105 stores programs for various processes executed by the CPU 104, and the RAM 106 provides a work area for the CPU 104 to execute processes and a storage area for sound data obtained from the sound pickup unit 102. .

[Sound playback device] Network I / F 201
Receives video and sound data from the sound pickup device 100 and sends control information to the sound pickup device 100.

The external storage device 202 is a floppy disk drive (FDD), a hard disk drive (HDD), or
An optical disk drive such as a CD-ROM or the like can store the processing program of the sound reproducing device 200 in these media. CPU 204 of sound playback device 200
Is a processing for reproducing video and sound by reading the processing program, and
Is executed to control the.

The input unit 203 includes a keyboard and / or a mouse that functions as a pointing device, and is used by a user to input a control command.

The CPU 204 controls the whole of the sound reproducing apparatus 200, and includes the ROM 205 and / or the external storage device 2.
The program read from 02 is stored in the RAM 206, and various operations and processes are executed based on the program.

The ROM 205 stores various programs for processing data input from the input unit 203 and processing executed by the CPU 204. In addition, the RAM 206 provides a work area when the CPU 204 executes a process and a temporary storage area for data input from the input unit 203 and the like.

The amplifier 207 performs digital-to-analog (D / A) conversion of a digital sound signal received from the sound pickup device 100, and outputs the obtained analog sound signal to the speaker 209.

Display consisting of CRT or LCD
Reference numeral 208 denotes a video and / or image captured by the imaging unit 101 of the sound collection device 100 based on the video data received from the sound collection device 100.

FIG. 5 is a view showing a display screen on the display 208. The photographed image 501 indicates a range in which sound can be collected, and the directivity display 502 indicates a sound collecting direction. This directivity display 502
The graphic user interface (GUI) (directivity operation unit) for controlling the directivity of the input unit 102 is operated, and the position of the directivity display 502 is operated by a pointing device or the like of the input unit 203 to the sound collection unit 102. The control command is transmitted, and sound can be picked up with the desired directivity.

The sound pickup device 100 and the sound reproduction device 200
The network connecting to is not particularly limited, but Ethe
LAN-type networks such as rnet and networks such as the Internet can be used. In addition, the camera / microphone device 100 and the video / sound reproduction device 200 may be connected by a telephone line or a dedicated line without being limited to a network, and information, data, and signals exchanged between the two devices are transmitted at a required transfer speed. There is no limitation on the form as long as it can be performed.

[Directivity Control] FIG. 2 is a diagram showing a configuration example of a microphone array in the sound collection device 100 of the present embodiment.

The microphone 301 converts sound into an electric analog signal. A / D converter 302 converts an analog sound signal into a digital sound signal. The audio data storage unit 303 is allocated to a predetermined area of the RAM 106. In order to provide such a microphone array with directivity in the direction of the angle θ, a delay corresponding to the difference in distance until a sound wave reaches two microphones from a sound source is given by:
After providing the sound data collected by one of the microphones, the sound data collected by the two microphones may be added. The delay time D is calculated from the following equation. D = B · sin θ / M… (1) where B: distance between microphones M: sound velocity θ: sound pickup direction

FIG. 3 shows a sound pickup device 100 according to this embodiment.
FIG. 3 is a diagram illustrating a method of generating a plurality of sound outputs having different directivities.

The adder 401 extracts sound data with a delay time (delay) of zero and a delay time taking into account the delay time D1 from the sound data storage unit 303 in order to provide a directivity of θ1 in the sound pickup direction. These data are added. Similarly, the adder 402 also receives a delay time from the sound data storage unit 303 in order to provide a directivity of θ2 in the sound pickup direction.
(delay) takes out sound data at a timing considering zero and the delay time D2, and adds these data. By performing such an operation, a set of sound data having a predetermined time difference is taken out from the shared sound data storage unit 303 and added, so that a single sound collection device 10
At 0, it is possible to obtain a sound output with any directivity.

FIG. 4 shows a sound pickup apparatus 100 according to this embodiment.
FIG. 9 is a diagram illustrating a method of selectively collecting sound in consideration of not only a sound collection direction but also a distance to a sound collection target. in this case,
Using multiple microphones, make the sound wavefront spherical,
A delay corresponding to a difference in distance until a sound wave reaches a plurality of microphones from a sound source is obtained. Then, by taking out a set of sound data in consideration of the delay time from the memory and adding them, the single sound pickup device 100 can selectively collect sound in consideration of the sound pickup direction and the distance to the sound pickup target. It can be carried out.

[GUI] FIG. 5 is a view showing an example of a screen display on the display 208 in the sound reproducing apparatus 200 of the present embodiment.

In FIG. 5, reference numeral 501 denotes a picked-up image, and the area shown in the picked-up image 501 is substantially equal to the area in which the sound pickup device 100 can pick up sound. A pointer 502 for displaying the sound pickup direction is displayed so as to be superimposed on the picked-up image 501, and provides a means for indicating the sound pickup direction in the picked-up image 501 and changing the sound pickup direction. That is, by operating the pointer 502 with the pointing device of the input unit 230 or the like, a command for controlling the sound pickup direction (directivity control command) is generated.
Pointer 5 for width 503 and height 504 of captured image 501
The offsets 505 and 506 from the center of the screen of 02 become the angle 508 of the sound collection direction with respect to the imaging angle of view 507.

Further, in the case of the sound pickup device 100 having selectivity regarding the sound pickup distance as shown in FIG. 4, the sound pickup distance can be adjusted by the user operating the slider 509.

[Operation] Next, the sound pickup apparatus 100 of the present embodiment will be described.
Will be described. FIG. 6 is a flowchart illustrating the operation of the sound pickup device 100, and is a process executed by the CPU 104.

In step S101, the network I / F1
Determine whether or not the directivity control command has been received by 03,
If the control command has not been received, the process proceeds to step S103.

If it is determined that the directivity control command has been received, the process proceeds to step S102, where the delay time D corresponding to each microphone is calculated based on the directivity control command, and the sound pickup directivity is changed.

Subsequently, in step S103, an addition process (synthesis process) of the sound data is performed. In step S104, the added audio data is coded and compressed. In step S105, the video obtained from the imaging unit 101 is converted into a digital image. Convert, and if necessary, encode and compress. Next, in step S106, the obtained sound and image digital data are transmitted to the sound reproduction device 200 via the network I / F 103,
It returns to step S101. Note that, for encoding and compression of images and sounds, an encoding and compression method defined in MPEG4 is adopted.

Next, the sound reproducing apparatus 200 of this embodiment
Will be described. FIG. 7 is a flowchart illustrating the operation of the sound reproducing device 200, and is a process executed by the CPU 204.

In step S201, the network I / F2
01 to determine whether or not video and / or sound data has been received, and if received, decodes image data in step S202, decodes sound data in step S203, and decodes the decoded video in step S204. Is displayed on the display 208, and the decoded sound is sent to the amplifier 207 for reproduction. Also,
If there is no received data, the processing from steps S202 to S204 is not performed.

Next, in step S205, it is determined whether or not an operation input for controlling the sound pickup directivity has been made by the input unit 203. If an operation input has been made, in step S206, the pointer
The sound collection direction is determined using offsets 505 and 506 of 502, a directivity control command is generated, and transmitted to the sound collection device 100 via the network I / F 201. Further, step S207
Then, the pointer 502 indicating the sound collection direction relative to the captured image 501 is redrawn. After that, the process returns to step S201.

When the sound collecting device 100 and the sound reproducing device 200 correspond one-to-one, it is possible to output a sound signal obtained by the above-described configuration and procedure and to accept a request for directivity control.

When a plurality of sound reproducing devices 200 exist as clients for one sound collecting device 100, the CPU 104 of the sound collecting device 100 transmits a TCP (Transfer Control Protocol) / IP (IP) received by the network I / F 103. Internet P
network address (eg, Internet address) of the client that requested the sound pickup and / or directivity control by a packet according to a protocol such as rotocol).
Can be recognized. Therefore, the data transmission destination in step S106 shown in FIG. 6 can be set to the device having the network address. That is, when sound collection is requested from a plurality of clients, the processing shown in FIG. 6 is executed in parallel or in series for each client, so that a packet of sound data synthesized based on the directivity direction requested by the client ( (Including video data) may be transmitted to the client.

[0045]

[Other Embodiments] The present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), and can be applied to a single device (for example, a copying machine). Machine, facsimile machine, etc.).

An object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (or a CPU or M
Needless to say, this can also be achieved by the PU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also the OS running on the computer based on the instructions of the program code.
It goes without saying that an (operating system) performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, based on the instructions of the program code, It goes without saying that the CPU included in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0048]

As described above, according to the present invention,
It is possible to provide a sound processing apparatus capable of remotely controlling the sound collecting directivity, in which a plurality of clients can independently set the sound collecting directivity, and a control method thereof.

It is also an object of the present invention to provide a sound processing device and a control method thereof, in which the sound collecting directivity of a sound collecting device capable of remotely controlling the sound collecting directivity is easily displayed and the sound collecting directivity is easily operated. it can.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration example of a sound collection device and a sound reproduction device according to an embodiment of the present invention;

FIG. 2 is a diagram showing a configuration example of a microphone array in the sound collection device;

FIG. 3 is a diagram showing a method of generating sound outputs having a plurality of different directivities in the sound pickup device;

FIG. 4 is a diagram for explaining a method of selectively collecting sound in the sound collecting device in consideration of not only a sound collecting direction but also a distance to a sound collecting target;

FIG. 5 is an exemplary diagram showing a screen display example of a display in the sound reproducing device.

FIG. 6 is a flowchart illustrating an operation of the sound collection device;

FIG. 7 is a flowchart illustrating an operation of the sound reproducing device.

Claims (10)

[Claims] 1. A sound processing apparatus capable of remotely controlling sound collecting directivity, comprising: a sound collecting array having a plurality of sound collecting devices; and sound information collected by each of the plurality of sound collecting devices. A plurality of storage means for calculating sound collection timing in each sound collection device based on a received control command, and reading out sound information from the plurality of storage means based on the calculation result; and a plurality of storage means. Transmission means for transmitting the result of synthesizing the sound information read from the client, and when there are transmission requests for sound information from a plurality of clients, the control means calculates according to a control command received from each of the clients. The sound information is read out from the plurality of storage units based on the sound collection timing in each of the sound collection devices. The sound processing apparatus transmitting means for and transmitting to the client the result of combining the sound information corresponding to the control command, which has issued the control command. 2. The sound processing apparatus according to claim 1, wherein the control unit controls the sound collection directivity by reading sound information having different sound collection timings from the plurality of storage units. apparatus. 3. The sound pickup directivity and the sound pickup distance are controlled by reading out sound information having different sound pickup timings from the plurality of storage means. Sound processing equipment. 4. The image processing apparatus according to claim 1, further comprising an imaging unit configured to acquire an image of an area where sound can be collected, wherein the acquired image is transmitted together with the sound information by the transmission unit. Item 4. The sound processing device according to any one of Items 3. 5. A sound pickup array having a plurality of sound pickup devices, and a plurality of storage means for storing sound information picked up by each of the plurality of sound pickup devices. A control method of a controllable sound processing apparatus, comprising: calculating sound collection timing in each sound collection device based on a received control command; reading sound information from the plurality of storage units based on the calculation result; When a result of synthesizing the sound information read from the storage means is transmitted, and a plurality of clients request transmission of the sound information, the sound collection timing in each sound collection device according to the control command received from each of the clients. And reads out the sound information from the plurality of storage means based on the calculation result. The result of combining the corresponding sound information, control method and transmits to the client that issued the control command. 6. A sound processing device for remotely controlling a sound pickup device capable of controlling sound pickup directivity, comprising: display means for displaying an image indicating a sound pickup area sent from the sound pickup device. A directivity display unit that superimposes an image displayed on the display unit to display the sound collection directivity of the sound collection device; and an input unit that inputs a command to control the directionality of the sound collection device. A sound processing device comprising: 7. The sound processing apparatus according to claim 6, wherein the image indicating the sound collecting directivity also serves as an input unit for inputting a command for controlling a sound collecting direction. 8. A method for controlling a sound processing device for remotely controlling a sound pickup device capable of controlling sound pickup directivity, comprising displaying an image indicating a sound pickup area sent from the sound pickup device. A control method, wherein a command for controlling the directivity of the sound collecting device is input, the sound collecting directivity of the sound collecting device being displayed so as to be superimposed on an image indicating the area where several sounds can be obtained. . 9. A sound pickup array having a plurality of sound pickup devices, and a plurality of storage means for storing sound information picked up by the plurality of sound pickup devices, respectively, to remotely control sound pickup directivity. A recording medium on which a control program code of a controllable sound processing device is recorded, wherein the program code includes at least a code of a step of calculating a sound pickup timing in each sound pickup device based on a received control instruction; A code for reading sound information from the plurality of storage units based on the calculation result; anda code for transmitting a result of synthesizing the sound information read from the plurality of storage units. If there is a request to send sound information, control commands received from each of those clients Accordingly, the sound collection timing in each sound collection device is calculated, the sound information is read from the plurality of storage units based on the calculation result, and the result of synthesizing the sound information corresponding to the control instruction is converted into a client that issues the control instruction. A recording medium, which is transmitted to a recording medium. 10. A recording medium in which a control program of a sound processing device for remotely controlling a sound pickup device capable of controlling sound pickup directivity is recorded, wherein a sound pickup area sent from the sound pickup device is recorded. A code of a step of displaying an image indicating the sound collecting device; a code of a step of displaying the sound collecting directivity of the sound collecting device so as to be superimposed on the image indicating the area where several sounds can be obtained; and And a code for a step in which a control command is input.