JP2003331266A - Interactive agent apparatus, interactive agent method, and interactive agent program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interactive agent apparatus, an interactive agent method, and an interactive agent program that can identify a direction of a user's face at a high recognition rate. <P>SOLUTION: A first camera 1 takes a picture of a user. A computer 9 for image processing detects the distance between eyebrows of the user on the image taken by the first camera 1. A computer 10 for voice processing makes a speaker 7 perform speaking processing. A computer 9 for total control determines that the user faces the front when the distance between the eyebrows detected by the computer 9 for image processing according to the speaking processing. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dialog agent device, a dialog agent method, and a dialog agent program used for taking a picture in cooperation with a user, for example.

[0002]

2. Description of the Related Art Conventionally, photographs, video tapes, etc. have been used as media for recording experiences. Further, due to the widespread use of digital cameras, images taken by users are also being edited based on their own intentions.

On the other hand, a photography device called a print club is installed in entertainment facilities, large-scale stores, etc., and is widely used by relatively young people. With this photography device,
The user selects a desired background from among various backgrounds prepared in advance, photographs the background together with the background, and prints the photograph taken on the spot.

[0004]

However, in the above-mentioned photo-taking device, the user takes a photo by operating the operation button in accordance with the operation description by voice or display. Since the photography device is just a machine for taking a photograph according to the operation of the user, if the photographer is repeatedly used, the interest is diminished.

On the other hand, research on agents that achieve predetermined goals while forming interactions with humans autonomously and cooperatively is underway. It is conceivable that if such an agent's technique is applied to the above-described photography device, it becomes possible to take a photograph in accordance with the user's intention while understanding the user's intention. Thereby, it is possible to attract the interest of a wide age group. Further, it is expected that the application range of the photography device is not limited to the case of simply taking a picture of play.

In order to take a picture in cooperation with the user, it is first important to direct the user's face toward the camera. Image processing techniques can be used to identify the orientation of the user's face. However, the conventional image processing technology cannot identify the direction of the user's face with a high recognition rate.

Next, it is important to identify the number and location of users. Image processing techniques can also be used to identify the number and location of users. However, there are cases where it is difficult to distinguish humans from other objects such as luggage by image processing. In addition, the recognition rate of the number of people decreases when another person partially or wholly hides behind one person.

Further, it is desired that the photograph is taken in a layout according to the user's intention, without the user's operation.

An object of the present invention is to provide a dialogue agent device capable of identifying the direction of a user's face with a high recognition rate,
An object is to provide a dialog agent method and a dialog agent program.

Another object of the present invention is to provide a dialog agent device, a dialog agent method, and a dialog agent program capable of identifying the number and positions of users with a high recognition rate.

Still another object of the present invention is to provide a dialogue agent device, a dialogue agent method, and a dialogue agent program which can easily take a photograph with a layout according to a user's intention.

[0012]

Means for Solving the Problems and Effects of the Invention A dialogue agent apparatus according to the first invention is a dialogue agent apparatus for identifying the orientation of a user's face, and an image pickup means for picking up an image of the user and an image pickup means. In the image obtained by the means, a detecting means for detecting the distance between the eyes of the user, an utterance means for uttering the user, and a change in the distance between the eyes detected by the detecting means along with the utterance of the utterance means. And a determination unit that determines the orientation of the user's face based on the determination result.

In the dialog agent apparatus according to the present invention, the user is imaged and the distance between the two eyes of the user is detected in the image obtained by the imaging. The user is uttered, and the orientation of the user's face is determined based on the change in the distance between the eyes detected with the utterance.

By combining the detection of the distance between the eyes of the user on the image and the utterance in this manner, the face of the user is turned to the front based on the change in the distance between the eyes accompanying the utterance. It is possible to identify whether or not there is a high recognition rate.

A dialog agent method according to a second aspect of the present invention is a dialog agent method for identifying the orientation of a user's face, which comprises the step of capturing an image of the user and the image of the image captured by the image of the user's eyes. It is provided with a step of detecting the interval, a step of speaking to the user, and a step of determining the orientation of the user's face based on the change in the interval between both eyes detected with the speech.

In the dialog agent method according to the present invention, the user is imaged and the distance between the two eyes of the user is detected in the image obtained by the imaging. The user is uttered, and the orientation of the user's face is determined based on the change in the distance between the eyes detected with the utterance.

As described above, by detecting the distance between the eyes of the user and the utterance on the image, the face of the user is turned to the front based on the change in the distance between the eyes accompanying the utterance. It is possible to identify whether or not there is a high recognition rate.

A dialogue agent program according to a third aspect of the present invention is a computer-readable dialogue agent program for identifying the orientation of a user's face, and is used in a process of picking up an image of a user and an image obtained by the image pickup. A process of detecting the distance between both eyes of a person, a process of uttering a user, and a process of determining the orientation of the user's face based on a change in the distance between both eyes detected with utterance. It is what makes a computer execute.

In the dialogue agent program according to the present invention, the user is imaged and the distance between the two eyes of the user is detected in the image obtained by the image pickup. The user is uttered, and the orientation of the user's face is determined based on the change in the distance between the eyes detected with the utterance.

By combining the detection of the distance between the eyes of the user and the utterance in this manner, the face of the user is turned to the front based on the change in the distance between the eyes accompanying the utterance. It is possible to identify whether or not there is a high recognition rate.

A dialogue agent device according to a fourth aspect of the present invention is a dialogue agent device for identifying the number and position of users, which comprises image pickup means for picking up images of users and image processing based on an image obtained by the image pickup means. Recognizing a guessing unit that estimates the number of users based on a predetermined determination condition, a uttering unit that speaks to the user based on the number of users estimated by the estimating unit, and a user's response to the utterance of the uttering unit The recognition means and the judgment condition of the estimation means are switched based on the response of the user recognized by the recognition means, and the estimation means estimates the number of users, the utterance by the utterance means, and the response of the user by the recognition means. It is provided with a determination means for determining the number and position of the users by causing the recognition and the image pickup by the image pickup means.

In the dialogue agent device according to the present invention, the number of users is estimated under a predetermined determination condition by image-taking the user and performing image processing based on the image obtained by the image-taking. The user is uttered based on the estimated number of users, and the user's response to the utterance is recognized. The determination conditions are switched based on the recognized user response, and the number and position of the users are determined by estimating the number of users, utterance, recognizing the user's response, and imaging. .

As described above, by combining the image recognition process and the interactive process using different determination conditions, it is possible to identify the number and position of users with a high recognition rate.

A dialogue agent method according to a fifth aspect of the present invention is a dialogue agent method for identifying the number and position of users, the method including a step of picking up an image of the user and a predetermined step by image processing based on the image obtained by the image pickup. A step of estimating the number of users in the determination condition, a step of uttering the user based on the estimated number of users,
By recognizing the response of the user to the utterance and switching the judgment condition based on the recognized response of the user, by estimating the number of users, utterances, recognizing the user's response and imaging. , And the step of determining the number and position of users.

In the dialog agent method according to the present invention, the number of users is estimated under a predetermined determination condition by image-capturing the user and performing image processing based on the image obtained by the image-capturing. The user is uttered based on the estimated number of users, and the user's response to the utterance is recognized. The determination conditions are switched based on the recognized user response, and the number and position of the users are determined by estimating the number of users, utterance, recognizing the user's response, and imaging. .

As described above, by combining the image recognition processing and the interactive processing using different determination conditions, it is possible to identify the number and position of users with a high recognition rate.

A dialogue agent program according to a sixth aspect of the present invention is a computer-readable dialogue agent program for identifying the number and position of users, which is a process of picking up an image of a user and an image based on an image obtained by the image pickup. The process of estimating the number of users under a predetermined determination condition by the process, the process of uttering the user based on the estimated number of users, the process of recognizing the user's reply to the utterance, and the recognition The judgment conditions are switched based on the user's reply, and the number of users is estimated,
By recognizing utterances and responses and capturing images,
The computer is caused to execute the process of determining the number and position of users.

In the dialogue agent program according to the present invention, the number of users is estimated under a predetermined determination condition by image-capturing the user and performing image processing based on the image obtained by the image-capturing. The user is uttered based on the estimated number of users, and the user's response to the utterance is recognized. Judgment conditions are switched based on the recognized user's response, and the number of users is estimated, utterance,
By recognizing the user's reply and imaging,
The number and location of users are determined.

As described above, by combining the image recognition processing and the interactive processing using different determination conditions, it is possible to identify the number and position of users with a high recognition rate.

A dialogue agent apparatus according to a seventh aspect of the present invention is a dialogue agent apparatus for taking a picture of a user, which is a first image pickup means for picking up an image of the user and a second image pickup means for picking up an image of the user. And controlling the imaging direction of the first imaging means so that the image of the user is located at a fixed position of the image obtained by the first imaging means, and interlocking with the control of the imaging direction of the first imaging means. Then, the first control means for controlling the image pickup direction of the second image pickup means, the recognition means for recognizing the voice instruction from the user, and the layout of the layout when the image layout instruction is recognized by the recognition means. Controlling the second imaging means so as to obtain an image according to the instruction,
If the recognition means recognizes the instruction to take the photograph, the second
And second control means for controlling the second imaging means so that a still image is captured by the imaging means.

In the dialogue agent device according to the present invention, the user is imaged by using the first imaging means,
The user is imaged by using the image pickup means. Further, the image pickup direction of the first image pickup means is controlled so that the image of the user is located at a fixed position of the image obtained by the first image pickup means, and the image pickup direction of the first image pickup means is controlled. The imaging direction of the second imaging means is controlled in conjunction with each other. The second imaging means is controlled so that an image in accordance with the layout instruction is obtained when the voice instruction from the user is recognized, and when the image layout instruction is recognized, the photography instruction is recognized. In such a case, the second image pickup means is controlled so that a still image is taken by the second image pickup means.

As described above, by combining the image recognition processing and the interactive processing, it is possible to easily take a photograph in a layout intended by the user in cooperation with the user. Also, the user is tracked by the first imaging means, and the second
By interlocking the image pickup means of 1. with the first image pickup means and controlling the second image pickup means according to the layout instruction by the voice of the user, even if the user moves, it is possible to take a photograph with the layout intended by the user. It can be carried out.

An interactive agent method according to an eighth aspect of the present invention is an interactive agent method for taking a picture of a user, which comprises a step of imaging the user using the first imaging means and a second imaging means. A step of capturing an image of the user by using the first image capturing means while controlling the image capturing direction of the first image capturing means so that the image of the user is located at a fixed position of the image obtained by the first image capturing means. The step of controlling the image pickup direction of the second image pickup means in conjunction with the control of the image pickup direction of the means, the step of recognizing a voice instruction from the user, and the case where the image layout instruction is recognized by the recognition. The second image pickup means is controlled so as to obtain an image in accordance with the layout instruction, and the second image pickup means captures a still image when the instruction for photography is recognized by recognition. Shooting Those with and controlling means.

In the dialog agent method according to the present invention, the user is imaged using the first imaging means and the second
The user is imaged by using the image pickup means. Further, the image pickup direction of the first image pickup means is controlled so that the image of the user is located at a fixed position of the image obtained by the first image pickup means, and the image pickup direction of the first image pickup means is controlled. The imaging direction of the second imaging means is controlled in conjunction with each other. The second imaging means is controlled so that an image in accordance with the layout instruction is obtained when the voice instruction from the user is recognized, and when the image layout instruction is recognized, the photography instruction is recognized. In such a case, the second image pickup means is controlled so that a still image is taken by the second image pickup means.

As described above, by combining the image recognition processing and the interactive processing, it is possible to easily take a photograph in a layout intended by the user in cooperation with the user. Also, the user is tracked by the first imaging means, and the second
By interlocking the image pickup means of 1. with the first image pickup means and controlling the second image pickup means according to the layout instruction by the voice of the user, even if the user moves, it is possible to take a photograph with the layout intended by the user. It can be carried out.

A dialogue agent program according to a ninth aspect of the present invention is a computer-readable dialogue agent program for taking a picture of a user, which comprises a process of picking up an image of the user using the first image pickup means, and a second step. A process of capturing an image of the user by using the image capturing means, and controlling the image capturing direction of the first image capturing means so that the image of the user is located at a fixed position of the image obtained by the first image capturing means. A process of controlling the image pickup direction of the second image pickup unit in association with the control of the image pickup direction of the first image pickup unit, a process of recognizing a voice instruction from a user, and a recognition of an image layout instruction. The second image pickup means is controlled so that an image according to the layout instruction is obtained when the image pickup instruction is received, and the still image is taken by the second image pickup means when the photograph instruction is recognized by the recognition. And a process of controlling the second imaging means to be, those to be executed by a computer.

In the dialogue agent program according to the present invention, the user is imaged by using the first image pickup means and the user is imaged by using the second image pickup means. Further, the image pickup direction of the first image pickup means is controlled so that the image of the user is located at a fixed position of the image obtained by the first image pickup means, and the image pickup direction of the first image pickup means is controlled. The imaging direction of the second imaging means is controlled in conjunction with each other. The second imaging means is controlled so that an image in accordance with the layout instruction is obtained when the voice instruction from the user is recognized, and when the image layout instruction is recognized, the photography instruction is recognized. In such a case, the second image pickup means is controlled so that a still image is taken by the second image pickup means.

As described above, by combining the image recognition processing and the interactive processing, it is possible to easily take a picture with the layout intended by the user in cooperation with the user. Also, the user is tracked by the first imaging means, and the second
By interlocking the image pickup means of 1. with the first image pickup means and controlling the second image pickup means according to the layout instruction by the voice of the user, even if the user moves, it is possible to take a photograph with the layout intended by the user. It can be carried out.

[0039]

1 is a block diagram showing the configuration of a dialogue agent apparatus according to an embodiment of the present invention. FIG. 2 is a schematic diagram for explaining the operation of the dialogue agent device of FIG. The dialogue agent device of FIG. 1 is a photo-taking device used for taking a photo in cooperation with a user.

The dialogue agent apparatus of FIG. 1 comprises a first camera 1, a second camera 2 and a third camera 3. The first to third cameras 1 to 3 are video cameras.
The first camera 1 is swingably provided to the left and right and up and down by the drive device 4. Similarly, the second camera 2
Are provided so as to be swingable left and right and up and down by the drive device 5.

As shown in FIG. 2, the first camera 1 and the second camera 2 are arranged so as to image the user substantially from the front. The third camera 3 is arranged so as to capture an image of the user from above.

Further, the dialogue agent device of FIG. 1 comprises a monitor device 6, a speaker 7 and a microphone 8. The monitor device 6 is arranged almost in front of the user so that the user can visually recognize it.

Further, the dialogue agent device of FIG.
Image processing computer 9, audio processing computer 1
0 and a computer 11 for overall control. These computers 9, 10, 11 are, for example, personal computers, and perform various operations according to an operating system (OS) and a dialogue agent program described later.

The images obtained by the first to third cameras 1 to 3 are given to the image processing computer 9 as image data. The image processing computer 9 performs image processing on the image data given from the first to third cameras 1 to 3 according to the image processing program. The image processing includes image recognition processing and eyebrow extraction processing, which will be described later. The images obtained by the first to third cameras 1 to 3 are
It is appropriately displayed on the screen of the monitor device 6.

The voice processing computer 10 performs voice processing, outputs voice from the speaker 7 and inputs the voice of the user as voice data through the microphone 8.
This voice processing includes voice recognition processing, voice synthesis processing, speech processing, reception processing, and the like.

The overall control computer 11 is the image processing computer 9 according to the dialogue agent program.
And the audio processing computer 10 as a whole, and also controls the first to third cameras 1 to 3 and the driving devices 4 and 5.
To control.

In the present embodiment, the image processing computer 9, the voice processing computer 10 and the overall control computer 11 execute the dialog agent method according to the dialog agent program. The interactive agent method according to claim 1 may be implemented.

The operation of the dialog agent device of FIG. 1 according to the dialog agent program will be described below.

In taking a picture by the dialog agent apparatus of FIG. 1, first, it is important that the user's face is directed to the front (direction of the first and second cameras 1 and 2). In this case, it is necessary to identify the orientation of the user's face.

Now, the process of identifying the face orientation of the user by the dialog agent apparatus of FIG. 1 will be described. 3 and 4 are flowcharts showing the identification processing of the face orientation of the user by the dialogue agent device. Further, FIG. 5 is a schematic diagram for explaining an example of face orientation identification processing.

First, the overall control computer 11 causes the first camera 1 to start imaging the user (step S).
1). The image obtained by the first camera 1 is given to the image processing computer 9 as image data and is displayed on the screen of the monitor device 6.

The image processing computer 9 detects the distance between the eyebrows (between the eyes of the users) on the image by the eyebrow extraction processing based on the image data (step S).
2). In this case, for example, Shinjiro Kawato, Shinji Tetsuya: “Eye Detection and Tracking for the purpose of eye position output to eye camera”, IEICE Technical Report, PRMU 2001-153, pp. 1
-6 (2001), the eyebrow extraction technique proposed can be used. For example, as shown in the upper diagram of FIG. 5, the distance L1 between the eyebrows of the user is detected on the image VD1 obtained by the first camera 1.

The overall control computer 11 is
The distance between the eyebrows detected by the image processing computer 9 is stored as the maximum value (step S3).

Thereafter, the voice processing computer 10 performs speech processing (step S4). Than that, speaker 7
The sound is output from. For example, a utterance such as "Hye, look at me" is performed. In this case, simply the utterance may be performed, such as "Hello".

The image processing computer 9 detects the distance between the eyebrows of the user on the image by the eyebrow extraction processing based on the image data obtained by the first camera 1 (step S5).

Next, the overall control computer 11 determines whether or not the distance between the eyebrows detected by the image processing computer 9 has become longer than the maximum value (step S).
6). When the distance between the eyebrows detected by the image processing computer 9 becomes longer than the maximum value, the overall control computer 11 stores the detected distance between the eyebrows as a new maximum value (step S7). When the distance between the eyebrows detected by the image processing computer 9 is less than or equal to the maximum value, the process proceeds to step S8.

Here, the fact that the distance between the eyebrows on the image is longer than before can be considered to mean that the user is facing more toward the front. For example, as shown in the lower diagram of FIG. 5, an image VD obtained by the first camera 1 in association with the utterance process.
When the distance L1 between the eyebrows on 1 is long, it can be determined that the user is facing more frontward. On the other hand, the first
When the distance between the user's eyebrows is shortened on the image obtained by the camera 1 of 1, it can be determined that the user has turned sideways.

Next, the overall control computer 11 determines whether or not the user has spoken a predetermined number of times (step S8).
If the user has not uttered a predetermined number of times, the process returns to step S4 and the cycle of steps S4 to S8 is repeated. The content of the utterance processing in step S4 is the same as in step S of the previous cycle.
It depends on the judgment result of 6. For example, if it is determined in step S6 of the previous cycle that the distance between the eyebrows has become longer, the utterance processing of step S4 of the next cycle becomes "Look at me more", When it is determined in step S6 that the distance between the eyebrows has become shorter, the speech process of step S4 in the next cycle is like "Because this is what you are looking for".

Since the distance between the eyebrows on the screen when the user completely faces the front is unknown, after the utterance processing is performed a predetermined number of times, the overall control computer 11 uses the stored maximum value. It is determined that the distance is between the eyebrows when the person faces the front (step S9).

After that, the voice processing computer 10 performs speech processing (step S10). As a result, sound is output from the speaker 7. For example, an utterance such as "Thank you" is performed.

The image processing computer 9 detects the distance between the eyebrows of the user on the image by the eyebrow extraction processing based on the image data obtained by the first camera 1 (step S11).

Next, the overall control computer 11 determines whether or not the distance between the eyebrows detected by the image processing computer 9 is greater than or equal to the maximum value (step S1).
2). When the distance between the eyebrows detected by the image processing computer 9 is greater than or equal to the maximum value, the overall control computer 11 determines that the face of the user is currently facing the front (step S13).

When the distance between the eyebrows detected by the image processing computer 9 is smaller than the maximum value, the process proceeds to step S10 until the distance between the eyebrows detected by the image processing computer 9 becomes equal to or larger than the maximum value. Step S1
The processing from 0 to S12 is repeated. In this case, the user is made to face the front by utterance processing.

In this way, by combining the detection of the distance between the eyebrows of the user on the image and the utterance processing, it is possible to determine whether or not the face of the user is facing forward based on the change in the distance between the eyebrows. It can be identified with a high recognition rate.

Next, the process of identifying the number of users and the positions of the users by the dialogue agent device of FIG. 1 will be described. 6 and 7 are flowcharts showing the identification processing of the number of users and the positions of the users by the dialogue agent device of FIG. FIG. 8 is a schematic diagram for explaining an example of the number of users and the position identification process.

First, the overall control computer 11 starts image pickup by the first and third cameras 1 and 3 (step S21). The images obtained by the first and third cameras 1 and 3 are given to the image processing computer 9 as image data. The image obtained by the first camera 1 is displayed on the screen of the monitor device 6.

The image processing computer 9 estimates the number of users by image recognition processing based on the image data given by the first camera 1 (step S22).
Here, the image processing computer 9 estimates the number of users based on the first determination condition. The first determination condition is, for example, the presence / absence of a substantially circular flesh-colored area included in an image obtained by imaging the user from the front.

As shown in the upper diagram of FIG. 7, the first camera 1
The image VD1 obtained by
When 2, 53, 54 are included, it is estimated that there are four users. In this case, an object such as a bag having a color close to skin color may be erroneously identified as a human face. For example, region 54 is a beige bag.

Next, the voice processing computer 10 performs a dialogue process based on the result of the estimation of the number of persons by the image processing computer 9 (step S23). As a result, sound is output from the speaker 7. For example, an utterance such as "Thank you for coming four people today" is performed. In this case, a warning such as "How many people did you come to?" 1 Do not ask directly the number of people, but indirectly confirm whether or not the estimation result of the number of people by image recognition processing is correct. Utter. In order to attract the interest of the user, it is preferable to utter a content that gives the impression that the photographing apparatus has human-like eyes and thinking power. The user responds to this utterance. The user's reply is through the microphone 8 to the voice processing computer 1
Given to 0.

The voice processing computer 10 recognizes the content of the user's reply by the voice recognition processing and determines whether the content of the user's reply is negative (step S2).
4). In response to the above utterance, the user replies, for example, "There are no four."

If the user's reply is negative,
The image processing computer 9 estimates the number of users by image recognition processing based on the image data provided by the third camera 3 (step S25). Here, the image processing computer 9 estimates the number of users based on the second determination condition. The second determination condition is, for example, the presence / absence of a substantially circular black region included in the image obtained by imaging the user from above.

As shown in the lower diagram of FIG. 7, the third camera 3
The image VD3 obtained by
When 2, 63 are included, it is estimated that there are three users. In this case, the flesh-colored area 54 in the upper diagram of FIG. 7 is not determined as a human.

Next, the voice processing computer 10 carries out an interactive process based on the result of estimating the number of people by the image processing computer 9 (step S26). As a result, sound is output from the speaker 7. For example, "Sorry, 3
It was a person ". The user responds to this utterance. The user's reply is given to the voice processing computer 10 through the microphone 8.

The voice processing computer 10 recognizes the content of the user's reply by the voice recognition processing, and determines whether or not the content of the user's reply is negative (step S2).
7).

If the user's reply is negative,
Returning to step S22, the processing of steps S22 to S27 is repeated until the content of the user's reply becomes affirmative.

In step S24 or step S27, if the user's reply is positive, the image processing computer 9 gives the image data given by the first camera 1 or the image data given by the third camera 3. The number and position of users are determined by image recognition processing based on the obtained image data (step S28).

In the dialog processing of step S26, the user replies, for example, "Yes." In that case,
As shown in the lower diagram of FIG. 7, black regions 61, 62, 63 in the image VD3 obtained by the third camera 3
Can be determined to be the user's head. Thereby, the number and position of users can be identified.

In this way, by combining the image recognition processing and the dialogue processing using different determination conditions, it is possible to identify the number and position of users with a high recognition rate.

In this example, different judgment conditions are as follows:
The presence or absence of a substantially circular flesh-colored area included in the image obtained by capturing the user from the front and the presence or absence of the substantially circular black area included in the image obtained by capturing the user from above are determined. Although used, other determination conditions may be used.

Next, the photographing process by the dialogue agent device of FIG. 1 will be described. FIG. 8 is a flowchart showing a photograph taking process by the dialogue agent device of FIG. Further, FIG. 9 is a schematic diagram for explaining an example of the photography process.

First, the computer 11 for overall control controls the image pickup areas (image areas) of the first and second cameras 1 and 2.
Drive devices 4, 5 so that the user's face is located in the center of the
Thus, the imaging directions of the first and second cameras 1 and 2 are initialized (step S31). As a result, FIG. 9 (a)
, The image VD obtained by the first camera 1
Image V obtained by the central part of 1 and the second camera 2
The face of the user is located at the center of D2.

In this case, the image processing computer 9 detects the position of the user's eyebrow on the image by the eyebrow extraction process based on the image data, and performs the general control based on the detected position of the user's eyebrow. The computer 11 adjusts the imaging directions of the first and second cameras 1 and 2. here,
For example, as mentioned above, Shinjiro Kawato, Shinji Tetsuya: "Eye detection and tracking for eye position output to eye camera",
IEICE Technical Report, PRMU2001-153, pp, 1-6
The eyebrow extraction technique proposed in (2001) can be used.

The image processing computer 9 determines whether or not the user has moved based on the image data provided from the first camera 1 (step S32). When the user moves, the position of the user moves on the image VD1 obtained by the first camera 1 and the image VD2 obtained by the second camera 2 as shown in FIG. 9B. To do.

When the user moves, the integrated control computer 11 causes the drive device 4 to move the image pickup direction of the first camera 1 so that the user is positioned in the center of the image pickup area of the first camera 1. Is moved (step S33). Further, the overall control computer 11 causes the driving device 5 to move the imaging direction of the second camera 2 in association with the first camera 1 (step S34). As a result, FIG.
As shown in (c), the face of the user is located at the center of the image VD1 obtained by the first camera 1 and the center of the image VD2 obtained by the second camera 2.

In this way, the face of the user is detected by the eyebrow extraction processing, and thereafter the face of the user is continuously tracked. As a result, the first camera 1 is controlled so that the user is always located in the center of the imaging area even if the user moves, and the second camera 2 is linked with the first camera 1.
Also moves. Therefore, if there is no layout instruction to be described later from the user, the second camera 2 is also controlled so that the user is always positioned in the center of the image pickup area.

Next, the voice processing computer 10 determines whether or not there is a voice instruction from the user (step S35). If there is no voice instruction from the user, the process returns to step S32, and the processes of steps S32 to S35 are repeated until there is a voice instruction from the user.

The voice instruction from the user is given by the microphone 8
Through the audio processing computer 10. The voice processing computer 10 recognizes the content of the voice instruction from the user through voice recognition processing.

When there is a voice instruction from the user, the voice processing computer 10 determines whether or not the instruction is a layout instruction (step S3).
6). When the voice instruction from the user is a layout instruction, the overall control computer 11 controls the second camera 2 or the driving device 5 according to the layout instruction (step S37). Here, the layout instruction is an instruction of the arrangement or size of the user on the image. For example, layout instructions include "to the right", "to the left", "to the top", "to the bottom", "to the middle", "zoom in", "zoom out", and the like.

For example, when the voice instruction from the user is “to the right”, the overall control computer 11 causes the drive device 5 to move the image pickup direction of the second camera 2 to the right. As a result, the user moves to the right on the image VD2 obtained by the second camera 2 as shown in FIG. 9 (d). On the image VD1 obtained by the first camera 1, the user is always located in the center.

When the voice instruction from the user is "zoom in a little", the overall control computer 11 increases the image pickup magnification of the second camera 2. Thereby, as shown in FIG. 9E, the image VD2 obtained by the second camera 2 is enlarged.

The user can temporarily freeze the image displayed on the screen of the monitor device 6 by speaking "freeze" as a layout instruction.

Then, the process returns to step S32 and the processes of steps S32 to S37 are repeated. If the instruction from the user is not the layout instruction in step S36, the audio processing computer 10 determines whether the instruction from the user is a shooting instruction (step S3).
8). The shooting instruction is an instruction to shoot a still image (photograph). For example, the user speaks "take a picture" as a shooting instruction.

When the instruction from the user is a photographing instruction, the overall control computer 11 photographs the still image of the user with the second camera 2 (step S39).
The still image of the user captured by the second camera 2 is
It is displayed on the screen of the monitor device 6. After that, step S
Returning to 32, the processes of steps S32 to S37 are repeated. In this way, the user can continue to take a still image while being satisfied while looking at the still image displayed on the screen of the monitor device 6.

When the user selects a still image displayed on the screen of the monitor device 6, the selected still image is printed out.

In step S38, if the user's instruction is not a photographing instruction, the voice processing computer 10 performs an interactive process (step S40) and returns to step S32. Here, the dialogue processing is, for example, a greeting or the like.

In this way, by combining the eyebrow extraction process and the interactive process, it is possible to easily take a picture in the layout intended by the user in cooperation with the user. In addition, the first camera 1 detects and tracks the user, the second camera 2 is linked to the first camera 1, and the user is moved by controlling according to the layout instruction by the user's voice. However, it is possible to take a picture with the layout intended by the user.

In the present embodiment, the first camera 1
The second camera 2 corresponds to the image pickup means, the first camera 1 corresponds to the first image pickup means, and the third camera 3 is the second image pickup means.
Corresponds to the image pickup means. Further, the image processing computer 9 corresponds to the detecting means and the estimating means, the voice processing computer 10 and the speaker 7 correspond to the uttering means, and the voice processing computer 10 and the microphone 8 correspond to the recognizing means. The computer 11 corresponds to the determination means. Further, the overall control computer 11 and the driving devices 4 and 5 correspond to the first control means and the second control means.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a dialogue agent device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram for explaining the operation of the dialogue agent device of FIG.

FIG. 3 is a flowchart showing a process of identifying a user's face orientation by the dialogue agent device of FIG.

FIG. 4 is a flowchart showing a process of identifying a user's face orientation by the dialog agent device of FIG.

FIG. 5 is a schematic diagram for explaining an example of face orientation identification processing.

FIG. 6 is a flowchart showing a process of identifying the number of users and the positions of users by the dialog agent device of FIG.

FIG. 7 is a flowchart showing a process of identifying the number of users and the positions of users by the dialogue agent device of FIG.

FIG. 8 is a schematic diagram for explaining an example of identification processing of the number of people and positions of users.

FIG. 9 is a schematic diagram for explaining an example of a photography process.

[Explanation of symbols]

1st camera 2 second camera 3rd camera 4,5 drive 6 Monitor device 7 speakers 8 microphone 9 Image processing computer 10 Speech processing computer 11 Integrated control computer

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kenji Mase             2-2 Kodai, Seika-cho, Soraku-gun, Kyoto             International Telecommunications Basic Technology Research Institute Co., Ltd. F term (reference) 5B057 BA02 DA06 DA13 DB02 DB06                       DC03 DC05 DC08 DC09 DC25                       DC32                 5C022 AA13 AB61 AB62 AC69 AC71                       AC72 AC74

Claims (9)

[Claims] 1. An interactive agent device for identifying the orientation of a user's face, wherein the imaging means captures an image of the user, and the distance between the eyes of the user is detected in an image obtained by the imaging means. Detection means, utterance means for uttering the user, determination for determining the orientation of the face of the user based on a change in the distance between the eyes detected by the detection means in association with the utterance of the utterance means And a dialogue agent device. 2. An interactive agent method for identifying the orientation of a user's face, the step of capturing an image of the user, and the step of detecting the distance between the eyes of the user in the image obtained by the image capturing. , A dialogue comprising a step of uttering the user, and a step of determining a direction of the user's face based on a change in the detected distance between both eyes accompanying the utterance. Agent way. 3. A computer-readable dialogue agent program for identifying the orientation of a user's face, the process of capturing an image of the user, and the distance between the eyes of the user in the image obtained by the image capturing. A process of detecting a user's face, a process of uttering the user, and a process of determining the orientation of the user's face based on a change in the detected binocular distance associated with the utterance, A dialogue agent program characterized by being executed by a user. 4. A dialogue agent device for identifying the number and position of users, comprising: imaging means for imaging the users; and image processing based on an image obtained by the imaging means, wherein the usage is performed under a predetermined determination condition. Estimating means for estimating the number of users, uttering means for uttering the user based on the number of users estimated by the estimating means, and recognizing the response of the user to the utterance of the uttering means Recognizing means and switching the judgment condition of the estimating means based on the response of the user recognized by the recognizing means, estimating the number of users by the estimating means, utterance by the uttering means, and the recognizing means Determination means for determining the number and position of the users by recognizing the response of the user by the Dialog agent apparatus characterized by comprising a. 5. A dialogue agent method for identifying the number and position of users, comprising the steps of capturing an image of the user, and performing image processing based on the image obtained by the image capturing of the user under a predetermined determination condition. A step of estimating the number of people, a step of uttering the user based on the estimated number of the users, a step of recognizing a response of the user to the utterance, and the recognized users A step of determining the number and position of the user by switching the determination condition based on the response of the user, estimating the number of the user, uttering the speech, recognizing the response of the user, and performing the imaging. And a dialog agent method comprising: 6. A computer-readable dialog agent program for identifying the number and position of users, wherein a predetermined determination condition is satisfied by a process of capturing an image of the user and an image process based on an image obtained by the image capturing. A process of estimating the number of the users, a process of uttering the user based on the estimated number of the users, a process of recognizing a response of the user to the utterance, the recognized The number of users and the position are determined by switching the determination condition based on the response of the user and estimating the number of users, uttering, recognizing the response, and imaging. A dialogue agent program, characterized in that the computer is caused to execute processing. 7. An interactive agent device for taking a picture of a user, comprising: first imaging means for imaging the user, second imaging means for imaging the user, and the first imaging. The imaging direction of the first imaging means is controlled so that the image of the user is located at a fixed position of the image obtained by the means, and the imaging direction of the first imaging means is linked to the control of the imaging direction of the first imaging means. A first control means for controlling the image pickup direction of the second image pickup means; a recognition means for recognizing a voice instruction from the user; and a layout control unit for recognizing an image layout instruction by the recognition means. The second image pickup means is controlled so that an image in accordance with the instruction is obtained, and the still image is taken by the second image pickup means when the photograph instruction is recognized by the recognition means. Second imaging Dialog agent apparatus characterized by comprising a second control means for controlling the stage. 8. An interactive agent method for taking a picture of a user, comprising the step of taking an image of the user using a first imaging means, and the step of taking an image of the user using a second imaging means. Step, controlling the imaging direction of the first imaging device so that the image of the user is located at a fixed position of the image obtained by the first imaging device, and imaging the first imaging device. Controlling the imaging direction of the second imaging means in conjunction with the control of the direction; recognizing a voice instruction from the user; and recognizing an image layout instruction by the recognition. The second image pickup means is controlled so that an image according to the layout instruction is obtained, and when the image pickup instruction is recognized by the recognition, a still image is photographed by the second image pickup means. Interactive Agents method characterized by comprising the step of controlling said second image pickup means so. 9. A computer-readable dialog agent program for taking a picture of a user, the process comprising: taking a picture of the user using a first imaging means; and using the second imaging means. A process of capturing an image of a person, controlling the image capturing direction of the first image capturing device so that the image of the user is located at a fixed position of the image obtained by the first image capturing device, and A process of controlling the image pickup direction of the second image pickup unit in association with the control of the image pickup direction of the image pickup unit, a process of recognizing a voice instruction from the user, and a recognition of an image layout instruction by the recognition. The second image pickup means is controlled so that an image in accordance with the layout instruction is obtained in the case where the instruction is given, and the second image pickup means is operated when the instruction for photography is recognized by the recognition. And a process for controlling the second image pickup means so that the image is captured, conversation agent program for causing the computer to perform.