JPH0942961A - Position measuring device for voice input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a position measuring device for voice input device, which can find the information on position and direction of a microphone from the image. SOLUTION: This position measuring device for voice input device is provided with an acoustic/electrical signal converting device 1 held by a supporting member, of which one end side is provided at a constant position, in the condition that a neck thereof can be freely oscillated and provided with a mark in the side peripheral surface near the tip thereof, an image pickup device 2 for photographing the mark of the acoustic/electrical signal converting device 1 from the a constant position, and a computing device 3 for obtaining the position of the mark from the image, which is photographed by the image pickup device, and for obtaining the direction of the acoustic/electrical signal converting device on the basis of this position of the mark.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice input device, for example, a position measuring device for a voice input device capable of acquiring information on the position and direction of a microphone from an image.

[0002]

2. Description of the Related Art In recent years, attention has been paid to multimedia and various application attempts have been made. Although multimedia can handle various information such as image information such as still images and moving images, data, audio, etc., there is a growing demand for inputting images and audio into computers as multimedia information and using them. ing. In particular, next-generation instruction input means that can give an instruction by gesturing or gesturing a person in a moving image and instruction input means that can give an instruction by a voice based on the captured voice are the next generation. This is a man-machine interface technology required for information devices of

Here, as a voice instruction input technique, research and development have been carried out for practical use as an application of voice recognition technology, and it has been effective in recognizing voices of specific speakers and unspecified speakers. Next-generation information equipment uses man-machine interface technology that introduces such speech recognition technology, and obtains functions such as recording of comments and input of instructions, making it more advanced and more automated. It is expected to be a machine that is compatible with.

Further, when a speech is made at a conference or a purchase is made by a vending machine, the direction of the microphone is controlled so that the microphone is automatically pointed toward the speaker (speaker) or the purchaser. It is also considered to be used for control such as to efficiently capture the voice of the user and the voice signal of the purchaser.

By the way, conventionally, as a voice input means,
A microphone built in the device and a microphone connected by a cable have been widely used. However, even if you try to extract the voice of the speaker, if you also collect the voices of other people in the vicinity of the installed microphone and background sounds such as environmental noise, these will become noise, and these will be the target of recording. It becomes difficult to extract the voice of the speaker or the voice of the speaker giving an instruction.

Therefore, in such a case, it is necessary to suppress noise. The simplest method for suppressing noise is to use a microphone with high directivity as the voice input means and direct it toward the mouth of the speaker. Then, by extracting the voice output during the utterance period of the speaker from the voices obtained by the microphone, the necessary voice can be selected and extracted relatively easily.

On the other hand, in the case where there are many microphones and many participants, such as a conference, if the approximate position of the speaker is known, the input from each microphone is
Apply the utterance period of the speaker in the direction in which the microphone is facing, selectively extract the voice obtained within this period, or look at the mouth of the speaker in the direction in which the microphone is facing and input. Research is being made into technologies that utilize information about the position and direction of a microphone, such as extracting audio data by timing the data. However, this work requires manual labor. Since it is complicated to manually measure the position and direction of the microphone, it imposes a burden on the user.

[0008]

When an instruction is given by voice or a voice of a speaker is recorded, a microphone is generally used to input the voice. Then, in order to suppress the surrounding noise and obtain the desired sound,
A microphone with high directivity is used as the voice input means.

However, even if a microphone having high directivity is used, the background sound is not sufficiently removed, and it is necessary to use a soundproof room or the like to suppress the background sound when collecting the audio data. However, since it is not realistic to use the information equipment in the soundproof room, a method that can remove the background sound and extract the voice data of the speaker by a simpler method is required.

[0010] As an example thereof, such as a conference,
When the approximate position of the speaker is known and there are many microphones, the utterance period of the speaker in the direction facing the microphone is applied to the input from each microphone, and the voice obtained within this period is selectively selected. Information on the position and direction of the microphone and the trend of the speaker, such as extracting the voice data by extracting the voice data by extracting the voice data by timing the input data and the voice of the speaker who is in the direction in which the microphone is facing. It has been attempted to utilize and. However, this work requires manual labor. Since it is complicated to perform such work manually, it imposes a burden on the user.

As a countermeasure for this, a video of a conference participant is obtained, the movement of the speaker's mouth is analyzed from this video to detect whether or not the speaker is speaking, and if the speaker is speaking, turn to that direction. If it is possible to extract the output of the existing microphone, it is possible to automatically select and extract the utterance content of the speaker.

Alternatively, in the case of a conference or the like using a microphone, it is natural for the speaker to bring his or her mouth close to the microphone and to orient the microphone in his or her own direction. It is possible to automatically select and extract the utterance content of the speaker by associating and extracting the movement of the mouth and the output of the microphone oriented in that direction.

For that purpose, it is necessary to develop a technique for automatically measuring the position and direction of the microphone. Further, a technique for automatically measuring the position and direction of the microphone is also required to perform control for automatically pointing the microphone toward the speaker.

Therefore, the object of the present invention is to
It is an object of the present invention to provide a position measuring device for a voice input device that can automatically measure the direction and position of a microphone.

[0015]

In order to achieve the above object, the present invention is configured as follows. That is, one end side is held by a supporting member so that it can be swung freely, and an acoustic-electrical signal conversion device in which a mark is provided on at least a side circumferential surface near the tip side, and at least the mark of the acoustical-electrical signal conversion device is photographed. And an arithmetic unit for determining the position of the mark from the image captured by the image capturing apparatus and for determining the orientation of the acoustic-electrical signal converter from the determined mark position.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION In the present invention, a microphone whose one end side is held by a support member installed at a fixed position so that it can be swung freely, and a mark is provided on at least a side peripheral surface near the tip side, and at least this microphone is provided. An image pickup device that takes an image of the mark from a fixed position, an area of the mark is obtained from an image taken by the image pickup device, the center of gravity of the region is obtained, and the position of the center of gravity is set as the position of the mark in the taken image screen. , And an arithmetic unit for obtaining the direction of the microphone from the position information and the installation position of the support member.

Then, the mark of the microphone, which is an acoustic-electrical signal conversion device, is photographed by the image pickup device, the area of the mark is obtained from the picked-up image of the image pickup device by the arithmetic device, the center of gravity of the region is obtained, and its center of gravity is obtained. The position is defined as the position of this mark in the photographed image screen, and a computing device for determining the direction of the microphone from this position information and the installation position of the support member is provided.

As described above, since the position and direction of the acoustic-electrical signal converter can be obtained from the image, no mechanical element for measurement is required, and therefore the structure is simple and the cost is low. It is possible to realize a position measuring device for various voice input devices. Further, since the orientation and position of the acoustic-electrical signal converting device can be measured, the present invention can be applied to a measuring device in the case of automatically controlling the acoustic-electrical signal converting device (that is, a microphone or the like) so as to be directed to the speaker's mouth.

Furthermore, if the speaker and the acoustic-electrical signal converting device are imaged, even if a plurality of persons are shown in the image, the acoustic-electrical signal converting device is suitable for the mouth of any person. It is possible to easily identify the speaker when the voice extraction control is performed by linking the movement of the speaker's mouth with the output of the acoustic-electrical signal conversion device. It becomes possible to easily realize the extraction control. In particular, when the speaker speaks spontaneously, he or she spontaneously takes actions such as moving the tip of the microphone to adjust it so that it points toward the user's mouth. The ability to measure the position and orientation of a person can identify who the speaker is, even when a large number of people are displayed on the screen, thus opening the way for new applications in multimedia. Become.

According to the present invention, if only the position of the microphone is required, it is sufficient to obtain only the position.
Further, if it is desired to know only the direction of the microphone, it may be configured to output only the direction information, and if it is desired to know both the direction and the position, all the information may be output.

[More Specific Embodiment 1] An example of a more specific embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the main configuration of this apparatus. The device of the present invention captures a voice to obtain a voice signal, for example, an acoustic-electrical signal converter 1 using a microphone.
And an image pickup device 2 for picking up an image of the acoustic-electrical signal converting device 1, converting it into an image signal, and outputting the image signal, and at least one of the position and direction of the microphone is calculated from a picked-up image obtained by the image pickup device 2. It is composed of an arithmetic processing unit 3.

In the case of the above-mentioned example, the acoustic-electrical signal converting device 1 has a structure using a microphone, and is a device for taking in a sound as a sound signal with this microphone. In the case of this example, the acoustic-electrical signal converter 1 is a rod-shaped or columnar microphone, and in the following, the acoustic-electrical signal converter 1 is referred to as a microphone body 32.

As shown in FIG. 3, a rod-shaped microphone main body 32 is rotatably held by a swinging instruction mechanism 33 on a base 31, and the microphone main body 32 has its side peripheral surface at an appropriate position. A mark 34 is attached as shown in FIG. The microphone main body 32 is mounted on the base 31 which serves as a support base, so that the base 31
If you place it on the desk or podium, the microphone can be placed at the speaker's mouth.

The image pickup device 2 is a device for picking up an image of the microphone and sequentially outputting it as an image signal, for example, a TV camera or a CCD.
An image pickup apparatus such as an electronic still camera using (solid-state image pickup device) is used.

The image pickup device 2 is an element designated by the reference numeral 35 shown in FIG. 3, and as shown in FIG.
Is mounted on the base 31, and the microphone body 32 on the base 31 is imaged and the image is output as an image signal.

Further, the arithmetic processing section 3 has a function of calculating the position and direction of the microphone from the image signal which the image pickup device 2 (corresponding to the image pickup device 35 in FIG. 3) picks up and outputs. Specifically, from the pixel data of the image obtained by the image signal, the area of the mark 34 is obtained based on the information on the brightness and color in each pixel data, and the center of gravity thereof is obtained to determine the mark position in the image. To do. Then, assuming that the installation position of the microphone is known, the direction is obtained from the position and the mark position.

FIG. 2 shows the overall processing flow of the device of the present invention. The image pickup device 35 (corresponding to the image pickup device 2 in FIG. 1) provided on the stand 31 photographs the microphone body 32 on the stand 31 (step a-1 in FIG. 2). The image signal of the image pickup device 35 is input to the arithmetic processing unit 3 (step a in FIG. 2).
-2), the arithmetic processing unit 3 detects an image of the mark 34 provided on the side peripheral surface of the microphone body 32 by performing image processing from the input captured image (step a-3 in FIG. 2). Then, the arithmetic processing unit 3 calculates the direction and position of the microphone body 32 from the detected image of the mark 34 (step a-4 in FIG. 2 (microphone attribute calculation process)).

As shown in FIG. 3, a mark 34 is provided on the microphone body 32 in an example in which a rod-shaped microphone is installed so that the direction can be changed on the present apparatus or the information equipment including the apparatus. The method and the processing method for detecting the image of the mark 34 from the captured image by the arithmetic processing unit 3 will be described.

In the present invention, for example, L is used as the mark 34.
A light emitting body such as an ED (light emitting element) is used. The mounting position is on the side peripheral surface near the tip of the microphone body 32, and the microphone body 3 is attached to the microphone body 3 by the imaging device 35.
It is a position where an image can be picked up together with the tip of 2.

As a result, as shown in FIG. 4, a mark 34 made of a light emitting body such as an LED can be provided on the side peripheral surface near the tip of the microphone body 32.
Is one or more. Then, at the time of use, the mark 34 of the microphone body 32 is caused to emit light. As a result, since the mark 34 emits light, this portion is obtained as a high-intensity point or area in the captured image.

Therefore, the mark 34 can be detected and the position of the mark 34 in the image can be recognized by extracting the pixel of the high-intensity point or region from the photographed image.

(Method of Detecting Mark Position when Image is Monochrome Image) FIG. 5 shows an example of a method of extracting pixels of a high brightness point or area when the image is a grayscale image.

If the image pickup device 35 is a monochrome device, a monochrome photographed image is input to the arithmetic processing section 3 from the image pickup device 35 on the table 31. When the captured image is input to the arithmetic processing unit 3 (step b-
1), the arithmetic processing unit 3 obtains a luminance value for each pixel of the input captured image, and obtains a histogram of the luminance value for each pixel (step b-2 in FIG. 5). A schematic diagram of the histogram thus obtained is shown in FIG. In FIG. 6, the horizontal axis represents the pixel value (luminance value), and the vertical axis represents the cumulative number of pixels for each luminance value.

When the histogram as shown in FIG. 6 is obtained, the arithmetic processing section 3 then carries out a smoothing process on this histogram (step b-3). as a result,
The histogram as shown in FIG. 6 is as shown in FIG. 7, which is a schematic diagram. Then, the minimum brightness value of the histogram is obtained from the smoothing processing result of the histogram as shown in FIG. 7 (step b-4).

Next, the arithmetic processing unit 3 defines a region of the histogram with each minimum brightness value as a boundary, and labels each region (step b-5). Then, the arithmetic processing unit 3 extracts pixels corresponding to the area of maximum brightness from the image (step b-6). Next, the arithmetic processing unit 3 collects pixels adjacent to each other among these extracted pixels and extracts a small area having the same label in the image (step b-7). Then, the area of each small region is obtained from this (step b-
8).

After the area of each small area is obtained, the arithmetic processing unit 3 then selects a small area having an area within a predetermined area value range from these small areas and sets it as a mark. (Step b-9). Now that the mark is set,
Next, the arithmetic processing unit 3 obtains the center of gravity of the small area, and sets the position of this center of gravity as the position of the mark (step b-10).

As a result of such processing, the arithmetic processing section 3 can obtain the position of the mark of the microphone body 32 from the image. The above is the method for obtaining the position of the mark of the microphone body 32 from the monochrome image. However, when the image is a color image, the following is performed.

(Method of Detecting Mark Position when Image is Color Image) When the photographed image is a color image and the light emitter has a color, a method of detecting marks from the photographed image is shown in FIG. Describe according to the flow. In addition,
In this embodiment, hue, saturation, and lightness are used to represent the color of each pixel.

If the image pickup device 35 is a color device, a color photographed image is input from the image pickup device 35 on the base 31 to the arithmetic processing section 3. When the color photographed image is input to the arithmetic processing unit 3 (step c in FIG. 8).
1), the arithmetic processing unit 3 obtains a hue histogram of each pixel of the captured image (step c-2 in FIG. 8). When the histogram is obtained, the arithmetic processing unit 3 then smooths the histogram (step c-3 in FIG. 8).

Then, the arithmetic processing section 3 obtains the minimum luminance value of the histogram from the smoothing processing result of the histogram (step c-4 in FIG. 8). Next, the arithmetic processing unit 3 defines an area of the histogram with each minimum brightness value as a boundary, and selects an area including the hue of the light emitter (step c-5 in FIG. 8). Hereinafter, this selected area is referred to as a selected area.

When the selected area is obtained, the arithmetic processing unit 3 then extracts the pixels corresponding to the selected area from the image (step c-6 in FIG. 8). Then, these extracted pixels are combined with each other to form a small area (step c-7 in FIG. 8). When this is finished, the arithmetic processing unit 3 then obtains the area of each small region (step c- in FIG. 8).
8). Then, from these small areas, a small area having an area within a predetermined area value range is selected and used as a mark (step c-9 in FIG. 8). Then, the arithmetic processing unit 3 obtains the center of gravity of the small area and sets this center of gravity as the position of the mark (step c-10 in FIG. 8).

In the processing procedure, after the processing in step c-5 of FIG. 8, a step of extracting only pixels having a preset threshold value or more from the pixels corresponding to the selected area may be added. . Further, when the captured image is represented by RGB values, the hue may be calculated from the RGB values and used. In the case of handling a color image, the mark is not a light-emitting body, but dots or regions may be formed in a color different from that of the microphone body, and this may be used as the mark.

An example of processing when dots or areas are formed in a color different from that of the microphone body and are used as marks will be described below. In this case, when the color captured image from the imaging device 35 is input to the arithmetic processing unit 3, the arithmetic processing unit 3
Is the R of the pixel whose luminance value is at the same position in the captured image.
Three color image with GB value (R (red) image, G (green)
Image, B (blue) image) (step d- in FIG. 9)
1).

Next, the arithmetic processing unit 3 selects an image having a component closest to the color of the mark 34 from the generated three-color image (hereinafter referred to as a target image), and determines the brightness value of the target image. Obtain a histogram (step d- in FIG. 9)
2). Next, the arithmetic processing unit 3 performs smoothing processing on the obtained histogram (step d- in FIG. 9).
3).

When the histogram smoothing process is completed, the arithmetic processing unit 3 then obtains the minimum luminance value of the histogram from the result of this smoothing process (step d-4 in FIG. 9). Then, the area of the histogram is defined with each minimum brightness value as a boundary, and each area is labeled (step d-5 in FIG. 9).

When this is finished, the arithmetic processing unit 3 then
Pixels corresponding to the area of maximum brightness are extracted from the image (step d-6 in FIG. 9). Next, the arithmetic processing unit 3 collects pixels that are adjacent to each other among the extracted pixels, and extracts a small area having the same label in the image (step d-7 in FIG. 9). When this is finished, the arithmetic processing unit 3 then obtains the area of each small region (step d-8 in FIG. 9). Then, the arithmetic processing unit 3 selects a small area having an area within a predetermined area value range from these small areas, and sets this as a mark (step d-9 in FIG. 9). Then, the center of gravity of the small area is obtained and set as the position of the mark (step d-1 in FIG. 9).
0).

As a result, the position of the mark can be obtained from the color image. Although the RGB values are used in the above description, the HSV value and Y
Since there are IQ values and the like, these may be used. Further, instead of the light emitting body, a region colored in a color different from that of the microphone or a point may be added as a mark.

In this way, the image of the microphone on the same table was obtained from the image pickup device on the table, and the position of the microphone could be obtained from this image. Since the position of the connecting portion at the base of the microphone on the table is known, the arithmetic processing unit 3 uses the position of the connecting portion at the base of the microphone and the detected mark after detecting the mark position. Find the direction of the microphone on the shooting surface. Then, by this, the position and direction of the microphone on the table can be obtained.

Since the position and direction of the microphone are obtained from the image in this device, no mechanical element for measurement is required. Therefore, the position measuring device of the voice input device is simple and inexpensive. realizable.

The image pickup device 35 on the table has a structure in which the focus can be automatically changed, and the focus of the image pickup device 35 is changed to photograph the microphone on the same table, which corresponds to the mark detected from this image. It is also possible to use a focus position that minimizes the area of the small region to find the distance to the mark and to find the position of the mark in the three-dimensional space.

[More Specific Embodiment 2] The above example shows how to determine the direction and position of the microphone when a rod-shaped (or columnar) microphone is installed on the table. However, microphones are not limited to rod-shaped (or columnar) microphones, and there are structures in which a small microphone body is attached to the tip of a flexible arm. Therefore, this will be described next.

(How to Obtain Direction and Position of Microphone of Flexible Arm Tip Supporting Structure) As shown in FIG. 10, the flexible arm 52 fixed at one end on the base 51 has the tip of the free end shown in FIG. A method of obtaining the direction and position of the microphone will be described with respect to an example in which the microphone body 53 corresponding to the acoustic-electrical signal converter 1 is attached.

The flexible arm 52 is a pipe-shaped arm made of metal or the like having rigidity and flexibility, and is a member that can be freely deformed by hand, and has been used for a long time such as a microphone stand or a lighting desk lamp. Is a support member used in.

In the present invention, the mark is used even when the support portion of the microphone body 53 is the flexible arm 52. That is, as shown in FIG. 11, LEDs on the peripheral surface of the microphone main body 53 near the tip side and the peripheral surface of the base side, respectively.
Two or more illuminants such as the above are provided, and these are used as marks 54.

The marks 54 are provided on the same surface side of the microphone body 53, and the marks 5 are provided on the base 51.
The image pickup device 35 corresponding to the image pickup device 2 in FIG. 1 has a positional relationship such that the microphone body 53 including 4 can be picked up.
Is attached as shown in FIG.

In such a structure, each mark 54 is caused to emit light by energizing it, and in this state, the image pickup device 35 photographs the microphone body 53 on the base 51. The image signal of the imaging device 35 is input to the arithmetic processing unit 3,
The arithmetic processing unit 3 performs image processing on a captured image based on the input image signal to generate a microphone main body 5
The image of the mark 54 provided on the side peripheral surface of No. 3 is detected.

The processing up to this point is the processing method used in the example of the above-mentioned rod-shaped microphone ([more specific embodiment 1]).
Processing method). When the mark 54 is detected from the photographed image, the arithmetic processing unit 3 then finds the center of gravity of the mark 54, and connects the points of the center of gravity of two or more marks 54 detected on the image in order to obtain the photographed surface. The direction and position of the microphone body 53 above are obtained.

As in the case of the rod-shaped microphone, it is also possible to change the focus of the image pickup device (camera) and take an image to obtain the position of each mark in the three-dimensional space and the direction of the microphone. . It is also possible to set the brightness value or color of each mark to a different value or color, and use the order to determine the direction of the microphone.

In the above, a mark is attached to the microphone body supported by the flexible arm, the image of the microphone including the image of the mark is picked up by one image pickup device, and the position and direction of the microphone are determined from the image. It was something that I asked for.

However, when the image of the microphone including the image of the mark is picked up by one image pickup device, there may be a problem in the accuracy of the obtained position and direction.
In such a case, if the position and orientation of the microphone are determined based on the images obtained by the two image capturing devices, the position and orientation based on the stereoscopic image (three-dimensional image) can be obtained. Since it can be decided, accuracy can be secured. An example thereof will be described below as a more specific embodiment 3.

[More Specific Embodiment 3] (Example of Obtaining Position and Direction of Microphone Based on Stereoscopic Image) As shown in FIG. 12, a case where a plurality of cameras are arranged will be described. A method of obtaining the position and direction in the three-dimensional space will be described.

As in the example using the above-mentioned rod-shaped microphone or flexible arm, the microphone bodies 32 and 53 provided with the marks 34 and 54 are mounted on the bases 31 and 51, and these are also mounted on the bases 31 and 51. Moreover, a plurality of image pickup devices 35a, 35 whose mutual positions and directions are known
b is installed and the microphone bodies 32 and 53 are imaged by the plurality of imaging devices 35a and 35b.

Then, after the marks 34 and 54 on the microphone bodies 32 and 53 are detected from the images obtained by the video signals from the image pickup devices 35a and 35b, respectively, the marks in the images taken by the image pickup devices 35a and 35b are detected. The marks having similar brightness values and colors are associated with each other, and the position of each mark in the three-dimensional space is obtained by the stereo method. Then, using the position of the obtained mark, the direction and position of the microphone body are obtained as follows.

[0065]

[Equation 1]

This makes it possible to specify the direction and position of the microphone body in the three-dimensional space.
The shapes of the marks 34 and 54 such as light emitters, color points, and color regions provided on the microphone bodies 32 and 53 corresponding to the acoustic-electrical signal converter 1 in FIG. 1 are other than those shown in FIGS. 4 and 11. The configuration may be elongated as in the configuration shown in FIG. 13, or may be shaped to indicate the direction as in the configuration shown in FIGS. 14 and 15.

Further, the arrangement of the image pickup device 35 corresponding to the image pickup device 1 in FIG. 1 is not limited to the arrangement examples shown in FIGS. 3 and 10, and as shown in FIG.
3 may be arranged so as to be photographed from above, or as shown in FIG. 17, the microphone bodies 32 and 53 may be arranged so that a speaker who is near the microphone can be photographed at the same time.

Even when a plurality of image pickup devices 35a, 35b, ... Are used, as shown in FIG. 18, in addition to the configuration shown in FIG. 12, the microphone main bodies 32, 53 are arranged so as to photograph from the lateral and upward directions. Good.

In the above example, the position of the microphone is determined from the position of the center of gravity of the mark area. However, when the mark is long and narrow, the width of the mark is reduced.
It is also possible to form a line, obtain the end points thereof, and then obtain the position and direction.

As described above, the present system uses an acoustic-electrical signal converter such as a microphone which is installed at a fixed position and can be freely swung to input a voice, and the side surface of the acoustic-electrical signal converter is used. While giving a mark to
An image pickup device for picking up this acoustic-electrical signal conversion device together with the mark is provided, and the area of the mark is detected from the image picked up by the image pickup device. For example, the center of gravity of the area is determined and the position of the center of gravity is detected. Position on the screen, or by reducing the width of the area to make a line, knowing its end points, and knowing the position from this, the sound whose installation position is known-
It is composed of an arithmetic unit for obtaining the orientation of the acoustic-electrical signal converter from the position information of the electric signal converter and the position of this mark on the screen.

Then, the acoustic-electric signal converter is photographed together with the mark by the image pickup device, and the area of the mark is detected from the photographed image of the image pickup device by the arithmetic device,
From this, knowing the position of this mark in the photographing screen, the orientation of the acoustic-electrical signal converter is determined from the position information of the acoustic-electrical signal converter whose installation position is known and the position of this mark in the screen. I tried to ask.

As described above, since the position and direction of the acoustic-electrical signal converter can be obtained from the image, no mechanical element for measurement is required, and therefore the structure is simple and the cost is low. It is possible to realize a position measuring device for various voice input devices. Further, since the orientation and position of the acoustic-electrical signal converting device can be measured, the present invention can be applied to a measuring device in the case of automatically controlling the acoustic-electrical signal converting device (that is, a microphone or the like) so as to be directed to the speaker's mouth.

Furthermore, if the speaker and the acoustic-electrical signal converting device are imaged, even if a plurality of persons are shown in the image, the acoustic-electrical signal converting device is suitable for the mouth of any person. It is possible to easily identify the speaker when the voice extraction control is performed by linking the movement of the speaker's mouth with the output of the acoustic-electrical signal conversion device. It becomes possible to easily realize the extraction control. In particular, when the speaker speaks spontaneously, he or she spontaneously takes actions such as moving the tip of the microphone to adjust it so that it points toward the user's mouth. The ability to measure the position and orientation of a person can identify who the speaker is, even when a large number of people are displayed on the screen, thus opening the way for new applications in multimedia. Become.

[0074]

As described above, according to the present invention, the position and direction of the voice input means such as a microphone can be easily obtained, and the microphone or the like can be automatically controlled so as to be directed to the speaker or the speaker. There is an effect that it is possible to obtain information useful for realizing the function for identifying the.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of the overall configuration of a system of the present invention, which is a diagram for explaining an example of an embodiment of the present invention.

FIG. 2 is a diagram for explaining an example of an embodiment of the present invention, and is a flowchart for explaining an example of a concrete embodiment 1 of the present invention.

FIG. 3 is a diagram for explaining an example of an embodiment of the present invention and is a perspective view of a device configuration for explaining an example of a specific first embodiment of the present invention.

FIG. 4 is a diagram for explaining an example of an embodiment of the present invention, and is a perspective view of a device configuration for explaining an example of a specific first embodiment of the present invention.

FIG. 5 is a diagram for explaining an example of an embodiment of the present invention, and a flowchart for explaining an example of mark extraction processing in a concrete first embodiment of the present invention.

FIG. 6 is a diagram for explaining an example of an embodiment of the present invention and is a schematic diagram showing an example of a luminance value histogram in a specific embodiment 1 of the present invention.

FIG. 7 is a diagram for explaining an example of an embodiment of the present invention, and a schematic diagram showing an example of a smoothing processing result of the example histogram of FIG. 6;

FIG. 8 is a diagram for explaining an example of an embodiment of the present invention, and a flowchart for explaining an example of mark extraction processing in a concrete embodiment 1 of the present invention.

FIG. 9 is a diagram for explaining an example of the embodiment of the present invention, and is a flowchart for explaining an example of mark extraction processing in the first specific embodiment of the present invention.

FIG. 10 is a diagram for explaining an example of an embodiment of the present invention, and is a perspective view of a device configuration example in a specific second embodiment of the present invention.

FIG. 11 is a diagram for explaining an example of the embodiment of the present invention, and is a diagram for explaining an example of mark addition in the second specific embodiment of the present invention.

FIG. 12 is a diagram for explaining an example of an embodiment of the present invention, and is a perspective view of a device configuration example in a concrete third embodiment of the present invention.

FIG. 13 is a diagram for explaining an example of an embodiment of the present invention, and is a diagram for explaining an example of marking given in a concrete third embodiment of the present invention.

FIG. 14 is a diagram for explaining an example of an embodiment of the present invention, and is a diagram for explaining an example of marking given in a concrete third embodiment of the present invention.

FIG. 15 is a diagram for explaining an example of an embodiment of the present invention, and is a diagram for explaining an example of marking given in a concrete third embodiment of the present invention.

FIG. 16 is a diagram for explaining an example of an embodiment of the present invention, and is a diagram showing an example of an image pickup device installation position with respect to a microphone body.

FIG. 17 is a diagram for explaining the example of the embodiment of the present invention, and is a diagram showing an example of the installation position of the imaging device with respect to the microphone body.

FIG. 18 is a diagram for explaining the example of the embodiment of the present invention, and is a diagram showing an example of the installation position of the imaging device with respect to the microphone body.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Acoustic-electrical signal converter 2, 35, 35a, 35b ... Imaging device 3 ... Arithmetic processing part 31 ... Stand 32, 53 ... Microphone main body (acoustic-electrical signal converter 1) 33 ... Pivoting instruction mechanism 34, 54 … Mark 52… Flexible arm.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hisashi Kazama 1-30, Oyodo Naka, Kita-ku, Osaka City, Osaka Prefecture Toshiba Kansai Branch Office

Claims (8)

[Claims] 1. An acoustic-electrical signal conversion device, one end side of which is swingably held by a support member installed at a fixed position, and a mark is attached to at least a side peripheral surface near the tip side, and at least this. An image pickup device for picking up the mark given to the acoustic-electrical signal conversion device from a fixed position, a position of the mark is obtained from a picked-up image of the image pickup device, and the acoustic-electrical signal conversion is performed from the obtained position of the mark. A position measuring device for a voice input device, comprising: a computing device for determining the orientation of the device. 2. A support member, one end of which is installed in a fixed position, is swingably supported, and at least a luminous body or a mark formed by a color point or a color region is provided on a side peripheral surface near the tip side. A sound-electric signal converter, an image pickup device for photographing at least the mark given to the sound-electric signal converter from a fixed position, and an area of the mark from a photographed image of the image pickup device,
From this, it is possible to obtain a position of the mark in the photographed image screen, and a computing device for obtaining the orientation of the acoustic-electrical signal converter from the position information and the installation position of the support member. Position measuring device for voice input device. 3. A support member, one end side of which is installed in a fixed position, is swingably held, and at least a luminous body or a color point or a mark formed by a color region is provided on a side peripheral surface near the tip side. An acoustic-electrical signal converter, and a single imager that captures at least the mark attached to the acoustical-electrical signal converter from a fixed position, and obtains the area of the mark from the image captured by the imager,
From this, the position of the mark in the photographed image screen is provided, and an arithmetic device for determining the orientation of the acoustic-electrical signal converter from the position information and the installation position of the support member is provided. Position measuring device for voice input device. 4. An acoustic-electrical signal conversion device, one end side of which is swingably held by a support member installed at a fixed position, and a mark is attached to at least a side peripheral surface near the tip side, A plurality of image capturing devices that capture the marks provided to the acoustic-electrical signal conversion device from different fixed positions, and the area of the mark in the three-dimensional space is obtained from the captured images of these image capturing devices. A voice input device, comprising: a position of the mark in the photographing screen, and an arithmetic unit that determines the orientation of the acoustic-electrical signal converter from the position information and the installation position of the support member. Position measuring device. 5. An acoustic-electrical signal converting device, one end side of which is swingably held by a supporting member, and a mark is provided on at least a side circumferential surface near the tip side, and at least this acoustic-electrical signal converting device. An image pickup device for picking up the mark given to the mark, and obtaining the area of the mark from a picked-up image of the image pickup device,
A position measuring device for a voice input device, comprising: an arithmetic device for obtaining the position information of the acoustic-electrical signal converting device by obtaining the position of the mark in the photographed image screen. 6. An acoustic-electrical signal converter for voice input, one end of which is held by a supporting member so that it can be swung freely, and a mark is provided on at least a side circumferential surface near the tip side, and at least this acoustic- A plurality of image pickup devices for picking up the marks given to the electric signal conversion device from different positions respectively, and a region of the mark in a three-dimensional space is obtained from picked-up images of these image pickup devices, and from this, Obtaining the position in the captured image screen, the sound in the three-dimensional space-
A position measuring device for a voice input device, comprising: an arithmetic device for obtaining position information of an electric signal converter. 7. A support member, one end side of which is installed in a fixed position, is swingably supported, and at least a side peripheral surface near the front end side is provided with a light emitter or a mark formed of a color point or a color region. Acoustic-electric signal converting device, a plurality of image capturing devices for capturing at least the marks given to the acoustic-electric signal converting device from different fixed positions, and the captured images of these image capturing devices in a three-dimensional space. In the three-dimensional space from the position information and the installation position of the support member while obtaining the position of the mark in the photographed image screen from this
A position measuring device for a voice input device, comprising: an arithmetic device for determining a direction of an electric signal converter. 8. A microphone whose one end side is held by a support member installed at a fixed position so as to swing freely, and at least a mark is provided on a side circumferential surface near the tip side, and at least this microphone is provided. An image pickup device that photographs the mark from a fixed position, and obtains the area of the mark from a photographed image of the image pickup device,
A voice input device comprising: a calculation device for obtaining the position of the mark in the photographed image screen from this, and for obtaining the direction of the microphone from the position information and the installation position of the support member. Position measuring device.