WO2017143910A1

WO2017143910A1 - Acquisition processing method, device and system, and computer storage medium

Info

Publication number: WO2017143910A1
Application number: PCT/CN2017/073176
Authority: WO
Inventors: 胡钦扬; 李星
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-02-25
Filing date: 2017-02-09
Publication date: 2017-08-31
Also published as: CN107124540A

Abstract

The present invention provides an acquisition processing method, device and system, wherein the method comprises: determining current position information of a speaker by means of an annular microphone array group; determining relative position information of the speaker with respect to an image acquisition device according to the current position information, wherein the image acquisition device is used for acquiring an image of the speaker; and adjusting the acquisition angle of the image acquisition device for the speaker according to the relative position information. Embodiments of the present invention also provide a computer storage medium.

Description

Acquisition processing method, device, system and computer storage medium

Technical field

The present invention relates to the field of communications, and in particular to an acquisition processing method, apparatus, system, and computer storage medium.

Background technique

There are two main types of existing video tracking systems: (1) Adopting a high-definition wide-angle camera to shoot the entire scene. The limitation of this method is that if you want to present the local details of the speaker, you need to manually adjust the camera. This kind of processing is very cumbersome and the user experience is very low. (2) Combine the linear microphone array with the camera to locate the speaker. The technical solution, although the camera can rotate according to the positioning result, but the positioning still has the following problems: the microphone array is only used for positioning, and the function is single, which is equivalent to an additional hardware cost; the above two ways of positioning the speaker mode There is a problem that the positioning accuracy is not high.

In view of the related art, the technical solution for locating the speaker has a problem of low precision, and an effective solution has not been proposed.

Summary of the invention

Embodiments of the present invention provide an acquisition processing method, apparatus, system, and computer storage medium.

An aspect of an embodiment of the present invention provides an acquisition processing method, including: determining, by a ring microphone array group, current location information of a sound issuer; and determining, according to the current location information, the sound issuer relative to an image capture device Relative position information, wherein the image acquisition device is configured to acquire an image of the sound issuer; and adjust the location according to the relative position information The angle at which the image capture device collects the sound issuer.

According to another aspect of the present invention, there is also provided an acquisition processing apparatus, comprising: a first determining module configured to determine current location information of a sound issuer through a ring microphone array group; and a second determining module configured to The current location information determines relative position information of the sound issuer relative to the image capture device, wherein the image capture device is configured to acquire an image of the sound issuer; and the adjustment module is configured to adjust the image according to the relative position information The angle at which the image capture device collects the sound issuer.

Another aspect of the embodiments of the present invention further provides an acquisition processing system, including: an image collection device, a sound issuer, and a ring microphone array group, wherein the ring microphone array group is configured to determine a current sound issuer Location information, and determining relative position information of the sound issuer relative to the image capture device based on the current location information, wherein the image capture device is configured to acquire an image of the sound issuer; the image capture device, After the relative position information is acquired, the collection angle of the sound emitting device by the image capturing device is adjusted according to the relative position information.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used to execute the foregoing collection processing method.

The technical solution provided by the embodiment of the present invention can determine the current location information of the sound issuer through the ring microphone array group; and determine the relative position information of the sound issuer relative to the image capture device according to the current location information, where the image The collecting device is configured to collect an image of the sound emitting person, and adjust an acquisition angle of the sound emitting device by the image collecting device according to the relative position information, and solve the related art, the positioning speaker (ie, the sound emitting person) The technical solution has the problem of low precision, thereby improving the positioning accuracy of the positioner of the positioning sound.

DRAWINGS

1 is a schematic flowchart of a first collection processing method according to an embodiment of the present invention;

2 is a schematic diagram of a first ring microphone array according to an embodiment of the present invention;

3 is a schematic diagram of a first microphone array group according to an embodiment of the present invention;

4 is another schematic diagram of a second microphone array group according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a second microphone array position according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic diagram of positioning of a first sound issuer according to an embodiment of the present invention; FIG.

FIG. 7 is a schematic structural diagram of a first type of collection processing apparatus according to an embodiment of the present invention; FIG.

FIG. 8 is a schematic structural diagram of a first determining module 70 of a second type of collection processing apparatus according to an embodiment of the present invention;

FIG. 9 is another schematic structural diagram of a third collection processing device according to an embodiment of the present invention; FIG.

FIG. 10 is a schematic structural diagram of an acquisition processing system according to an embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

Other features and advantages of the invention will be set forth in the description which follows, The objectives and other advantages of the invention may be realized and obtained by means of the structure particularly pointed in the appended claims.

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is apparent that the described embodiments are merely a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

In the embodiment of the present invention, an acquisition processing method is also provided. FIG. 1 is a flowchart of an acquisition processing method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:

Step S102, determining current location information of the sound issuer through the ring microphone array group;

Step S104, determining relative position information of the sound issuer relative to the image capture device according to the current position information, wherein the image capture device is configured to collect an image of the sound issuer;

Step S106, adjusting an acquisition angle of the sound emitting device by the image capturing device according to the relative position information.

Through the steps provided by the embodiments of the present invention, the current location information of the sound issuer can be determined by the ring microphone array group; and the relative position information of the sound issuer relative to the image capture device is determined according to the current location information, where The image capture device is configured to collect an image of the sound issuer; adjust the collection angle of the sound imager by the image capture device according to the relative position information, and solve the related art, locate the speaker (ie, the sound issuer) The technical solution has a problem of low precision, thereby improving the positioning accuracy of the positioner of the positioning sound.

In determining the step S104, before determining the relative position information of the sound issuer relative to the image capture device, knowing the position information determined by the ring microphone array group in advance, and the relative relationship with the image capture device, in step S104, According to this relative relationship, combined with the current position information acquired by the ring microphone array and the relative relationship, the relative position information of the sound issuer relative to the image capture device is obtained by calculation. In some embodiments, the step S104 may further include: transmitting current location information determined by the ring microphone array group to other devices on the network side, for example, a positioning server, according to the current location information by the positioning server. And the relative relationship, the relative position information of the sound issuer relative to the image capture device is calculated. The relative relationship herein may include a first relative position of the annular microphone array relative to the image capture device. The current position information may be a second relative position of the sound generator relative to a specific position in the annular array. The first relative position, the second relative position, and three-dimensional modeling may determine the sound sender relative to the image collection. Relative position information of the device. Here, specific information on how to position the sound issuer relative to the image capture device is provided, and is not limited thereto. The plurality of microphones in the circular microphone array can collect sounds generated by the generator from different angles or collect sounds generated by the generators at different positions, and then collect sound and sound signals based on multi-angle or multi-position transmission, which can be precisely determined. The current position information is obtained, so that the accuracy of the position information can be improved, thereby achieving precise adjustment of the acquisition angle.

Optionally, the number of the wake-up microphone arrays included in the ring microphone array group is preferably the following number of two, three, or four. The following embodiments of the present invention are only two and three, for four or more. The implementation of the technical solution is not described here because the similar solution is adopted.

Optionally, when the ring microphone array group includes two ring microphone arrays, determining the current location information of the sound issuer by the ring microphone array group may be implemented by: adopting the first ring microphone in the ring microphone array group Determining, by the array, a first relative angle of the sound emitter; determining, by the second annular microphone array in the annular microphone array group, a second relative angle of the sound emitter; determining the above according to the first relative angle and the second relative angle Current location information.

Specifically, the method further includes: determining the first relative angle according to the plurality of microphones of the first ring microphone array; and determining the second relative angle according to the plurality of microphones of the second ring microphone array.

Optionally, determining, according to the current location information, that the sound emitter is collected relative to the image Before the relative position information of the device, the method further includes: acquiring first position information of the image capturing device relative to each of the ring microphone arrays in the ring microphone array group, and between the two ring microphone arrays in each of the ring microphone arrays Second location information.

In the embodiment of the present invention, the method further includes: determining the relative location information according to the current location information, the first location information, and the second location information.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

The above technical solutions are described in detail below with reference to an example, but are not intended to limit the scope of protection of the embodiments of the present invention.

2 is a schematic diagram of a ring microphone array according to an embodiment of the present invention. As shown in FIG. 2, the four microphones in FIG. 2 are annularly distributed, and the microphone faces outward, at 0°, 90°, 180°, and 270°, respectively. A ring microphone array with different numbers of microphones and spacings may be used as an alternative to the embodiment of the present invention; the number of arrays of the present invention may be limited to two, and more than two ring microphone array groups are within the protection scope of the patent. See Figure 3 and Figure 4.

FIG. 5 is a schematic diagram of a position of a microphone array according to an embodiment of the present invention, based on FIG. 5:

Step 1. Referring to Figure 5, the two ring microphone arrays are fixed. The preferred position is to align the 0° of the two microphone array positioning algorithms onto the same line. The distance between the two microphone arrays is adjusted according to the actual situation (conference room, conference table size);

Step 2, according to the actual situation, the microphone array group and the camera (corresponding to the image acquisition device of the above embodiment) are arranged in the use environment, when the camera and the microphone array group are in the same When it is flat, the practical effect is the best;

FIG. 6 is a schematic diagram of positioning of a sound issuer according to an embodiment of the present invention. Based on the schematic diagram of FIG. 6, the specific operation flow includes the following processing steps:

Step A, the camera, the microphone array and the sound source shown in FIG. 6 are projections of the corresponding objects on the ground. This step is necessary data measurement, and the part of the data is input into the sound source localization algorithm, and the data to be measured includes:

The distance L3 between the two array projections;

The distance L1 projected by the camera to the array 1 is projected;

The distance L2 projected by the camera to the array 2;

The angle between the camera projection 0° line and the projection of the microphone array 1 is α;

Step B: According to the positioning result β of the microphone array 1, the positioning result γ of the microphone array 2, and the actual data measured in the step A, the position of the sound source can be calculated by the sine and cosine theorem (the distance from the camera to the sound source L5) , the angle between the camera 0 ° and the sound source ε);

Microphone array sound source localization method:

1) Calculate the collected energy of each microphone according to the number of ring microphones;

2) Selecting the normal speech stage through VAD to perform energy judgment of each microphone;

3) Select the microphone with the largest energy, and then combine the two microphones, the three microphones form a microphone sub-array, and the circumferential angle formed is used as the primary sound source orientation;

4) According to MVDR (Minimum Variance Distortion Less Response, which can obtain higher azimuth resolution and noise interference suppression performance), beamforming is performed on the sub-array to find the beam direction of the sound source. ;

5) Through the multi-frame statistics, find a stable sound source orientation, and β and γ in this embodiment.

Sound source precise position calculation method:

Through the sound source projection, the microphone array 1 projection, the microphone array 2 projection formed by the triangle, through the sine theorem, L4 can be calculated;

It is known that the lengths of the three sides of the triangle composed of L1, L2, and L3 can be calculated by the cosine theorem;

Based on the triangle formed by camera projection, microphone array 1 projection, and microphone array 2 projection, L5 can be calculated by the cosine theorem;

Through the sine theorem, the angle ε can be calculated;

Step C: Rotate the camera to the designated position according to the final calculation result of the step B, the relative angle ε of the camera and the sound source.

In the embodiment, an acquisition processing device is also provided for implementing the above-mentioned embodiments and preferred embodiments. The descriptions of the modules and the preferred embodiments have been omitted. The modules involved in the devices are described below. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated. FIG. 7 is a structural block diagram of an acquisition processing apparatus according to an embodiment of the present invention. As shown in Figure 7, the device includes:

The first determining module 70 is configured to determine the sound issuer through the ring microphone array group Front position information;

The second determining module 72 is configured to determine relative position information of the sound issuer relative to the image capture device according to the current position information, wherein the image capture device is configured to collect an image of the sound issuer;

The adjustment module 74 is configured to adjust an acquisition angle of the sound emitting device by the image capturing device according to the relative position information.

The current position information of the sound issuer can be determined by the ring microphone array group through the combined action of the above modules; the relative position information of the sound issuer relative to the image capture device is determined according to the current position information, wherein the image is collected. The device is configured to collect an image of the sound issuer; adjust an acquisition angle of the sound issuer by the image capture device according to the relative position information, and solve the related art, the locate speaker (ie, the sound issuer) The technical solution has a problem of low precision, thereby improving the positioning accuracy of the positioner of the positioning sound.

The number of the wake-up microphone arrays included in the ring microphone array group in the first determining module 70 includes: 2, 3, and 4.

FIG. 8 is a structural block diagram of a first determining module 70 of the collection processing device according to an embodiment of the present invention. As shown in FIG. 8, when the ring microphone array group includes two ring microphone arrays, the first determining module 70 includes:

The first determining unit 700 is configured to determine, by using the first ring microphone array in the ring microphone array group, a first relative angle of the sound issuer;

The second determining unit 702 is configured to determine a second relative angle of the sound issuer by using the second ring microphone array in the ring microphone array group;

The third determining unit 704 is configured to determine the current location information according to the first relative angle and the second relative angle.

In the embodiment of the present invention, FIG. 9 is another structural block diagram of an acquisition processing apparatus according to an embodiment of the present invention. As shown in FIG. 9, the apparatus further includes:

The obtaining module 76 is configured to acquire first position information of the image capturing device relative to each ring microphone array in the ring microphone array group, and second position information between the two ring microphone arrays in each of the ring microphone arrays.

According to another aspect of the present invention, an acquisition processing system is further provided. FIG. 10 is a structural block diagram of an acquisition processing system according to an embodiment of the present invention. As shown in FIG. 10, the image collection device 100 includes a sound issuer 102. , ring microphone array group 104.

The ring microphone array group 104 is configured to determine current location information of the sound issuer, and determine relative position information of the sound issuer relative to the image capture device 100 according to the current location information, wherein the image capture device 100 is configured to collect The image capturing device 100 is configured to adjust the collection angle of the sound emitting device by the image capturing device based on the relative position information after acquiring the relative position information.

In order to better understand the execution method of the above operational events, the following detailed description will be given in conjunction with the preferred embodiments.

The trackable speaker system of the preferred embodiment of the present invention includes the following components: a ring microphone array set; a camera.

The sound source positioning designed by the preferred embodiment of the present invention uses a plurality of ring microphone arrays, and the two arrays are arranged in a line of absolute 0 degree according to the positioning algorithm, and the ring microphone array group and the camera position can be flexibly placed:

The sound source localization method of the preferred embodiment of the present invention comprises the following steps:

Step 1. Install hardware devices, including placement of the microphone array and fixing of the camera device;

Step 2. Measure the necessary data and input it to the calculation module. The data to be measured includes but not limited to The distance between the microphone arrays, the distance from the camera to the microphone array, and the angle of rotation between the camera and each microphone array.

Step 3: Calculate the precise position of the sound source by using a positioning algorithm according to the data measured in step 2;

Step 4. Rotate the camera to the sound source orientation according to the positioning result.

The technical solution of the preferred embodiment of the present invention improves the accuracy of sound source localization. Secondly, the microphone array of the method of the present invention can be used not only as a sound source for positioning but also as a sound collecting device. Therefore, while reducing the hardware cost, it also greatly reduces the difficulty of implementing the solution. Can be widely used in video conferencing, security and various real-time monitoring areas.

In another embodiment, a computer storage medium is provided, the computer storage medium having computer executable instructions for performing any one or more of the collection processing methods described above, for example, executable The method shown in Figure 1. The computer storage medium includes, but is not limited to, an optical disk, a floppy disk, a hard disk, a rewritable memory, etc., optionally a non-transitory storage medium.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order. It is to be understood that the objects so used are interchangeable, where appropriate, so that the embodiments of the invention described herein can be carried out in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be executed by computing devices The program code of the lines is implemented so that they can be stored in the storage device by the computing device, and in some cases, the steps shown or described can be performed in a different order than here, or they can be Each of the integrated circuit modules is fabricated separately, or a plurality of modules or steps thereof are fabricated into a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the present invention, and those skilled in the art should understand that modifications made in accordance with the principles of the present invention are intended to fall within the scope of the present invention.

Industrial applicability

In the embodiment of the present invention, the ring microphone array group is used to collect the sound of the generator, and the sound is collected from different angles by the plurality of microphones in the ring microphone array group, so that the current position information of the sound sender can be accurately determined and reused. Determining current position information, determining relative position information of the occurrence of the occurrence relative to the image acquisition device, wherein the image acquisition device is configured to acquire an image of the sound issuer; and finally, adjusting the image according to the relative position information The collecting angle of the collecting device to the sound emitting person is simple in the process, high in reproducibility and high in obtaining position information, thereby providing a more accurate acquisition angle adjustment of the image capturing device.

Claims

An acquisition processing method includes:

Determining current location information of the sound issuer through the ring microphone array group;

Determining relative position information of the sound issuer relative to the image capture device according to the current position information, wherein the image capture device is configured to collect an image of the sound issuer;

And adjusting an acquisition angle of the sound emitting device by the image capturing device according to the relative position information.
The method according to claim 1, wherein the number of annular microphone arrays included in the annular microphone array group comprises: 2, 3, and 4.
The method according to claim 2, wherein when the ring microphone array group comprises two ring microphone arrays, determining the current location information of the sound issuer through the ring microphone array group comprises:

Determining, by the first ring microphone array in the ring microphone array group, a first relative angle of the sound issuer;

Determining, by the second ring microphone array in the ring microphone array group, a second relative angle of the sound issuer;

The current location information is determined according to the first relative angle and the second relative angle.
The method of claim 3, wherein the method further comprises:

Determining the first relative angle according to the plurality of microphones of the first ring microphone array;

The second relative angle is determined according to a plurality of microphones of the second circular microphone array.
The method of claim 1, wherein the method further comprises: prior to determining the relative position information of the sound issuer relative to the image capture device based on the current location information, the method further comprising:

Acquiring first position information of the image acquisition device relative to each ring microphone array in the ring microphone array group, and two or two ring microphones in each ring microphone array Second position information between the wind arrays.
The method of claim 5, wherein the method further comprises determining the relative location information based on the current location information, the first location information, and the second location information.
An acquisition processing device, comprising:

a first determining module configured to determine current location information of the sound issuer through the ring microphone array group;

a second determining module, configured to determine relative position information of the sound issuer relative to the image capture device according to the current location information, wherein the image capture device is configured to collect an image of the sound issuer;

And an adjusting module configured to adjust an acquisition angle of the sound emitting device by the image capturing device according to the relative position information.
The apparatus according to claim 7, wherein the number of annular microphone arrays included in the ring microphone array group in the first determining module comprises: 2, 3, and 4.
The apparatus according to claim 8, wherein when the ring microphone array group comprises two ring microphone arrays, the first determining module comprises:

a first determining unit, configured to determine a first relative angle of the sound issuer through the first one of the ring microphone arrays;

a second determining unit, configured to determine a second relative angle of the sound issuer through the second annular microphone array in the annular microphone array group;

And a third determining unit configured to determine the current location information according to the first relative angle and the second relative angle.
The apparatus of claim 7 wherein said apparatus further comprises:

An acquiring module, configured to acquire first position information of the image capturing device relative to each ring microphone array in the ring microphone array group, and the respective ring microphone arrays The second position information between the two pairs of ring microphone arrays.
An acquisition processing system, comprising: an image acquisition device, a sound issuer, and a ring microphone array group, wherein

The ring microphone array group is configured to determine current location information of the sound issuer, and determine relative position information of the sound issuer relative to the image capture device according to the current location information, wherein the image capture device is configured to collect An image of the sound sender;

The image capturing device is configured to, after acquiring the relative position information, adjust an acquisition angle of the sound emitting device by the image capturing device according to the relative position information.
A computer storage medium having stored therein computer executable instructions for performing the method of any one of claims 1 to 6.