JPH05176325A - Information supplying system - Google Patents

Abstract

PURPOSE:To improve a service formation rate by automatically measuring and supplying a user's request purpose to an operator and forming an environment allowing the operator to easily communicate with the user. CONSTITUTION:A human monitoring device 13 detects the approach of a user to a terminal through a wide field camera 10, a sensor driving control device 14 drives a status sensor 41 and collects the movement and sound of the user. The collected result is processed by a purpose analyzer 15. The obtained data are collated with the data of a purpose data base(DB) 16. The quantity of a feature is extracted from a detected result of a status sensor 41 and the detected quantity is compared with feature quantity stored in the OB 16 to judge whether the extracted feature quantity exceeds a prescribed value or not. In a state that the user observes a screen for instance, the movement of an observing point is recorded by the sensor 41. When the user requires the operator, the analyzer 15 compressses the approached state of the user to the terminal with the lapse of time and informs the compressed result to the operator.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention utilizes video communication to
The present invention relates to an information providing system in which a specialized operator provides information to a large number of users, and particularly to an information providing system suitable for improving the service establishment rate.

[0002]

2. Description of the Related Art Conventionally, as a service in which a specialized operator provides information to an unspecified user by using video communication, a remote education system using a computer (a teacher is an operator, a student is a user (Corresponding), telewind shopping (a clerk corresponds to an operator, a customer corresponds to a user), and the like. In such a system, as shown in FIG. 2, a plurality of terminals are installed on the user side, and an operator sequentially communicates with the user of this terminal to provide information. The contents of information are diverse, such as education, various guides, sales, and so on. In FIG. 2, 21a to 21c are terminal-side display devices, and 22a to 21c.
22c is a user monitor camera, 30 is a wide-field camera, 2
3a to 23c are users, 24 is a switching device, 25 is a video providing device (video DB), 26 is an operator monitor camera,
27 is a display device on the operator side for appropriately switching and displaying information around and in front of the terminal and information necessary for the user, 28 is a device for controlling exchange switching for this purpose, and 29 is an operator. is there. By the way, in such a system, normally, the operator 2
9 In other words, the number of responders is reduced and efficient operation is attempted. Therefore, the efficiency of this system depends on how appropriate information can be provided to the terminal user who needs the help of the operator 29. To the user who does not need the help of the operator 29, the terminal may actively provide the service to some extent. If the operator does not respond when really needed, the users 23a to 23c immediately give up and leave the terminal, and the service cannot be established. This means losing the business opportunity for the information provider. For this reason, a wide-field camera 30 is installed near the terminal to capture the situation around the terminal,
A method has been adopted in which the operator 29 is sent to assist the operator 29 in selecting a terminal with which to communicate.
Regarding this type of device, for example, "NTT PL
AZA, published by Nippon Telegraph and Telephone Corporation, pp. 14-15 "
Are described in.

[0003]

As shown in FIG. 2, the above-mentioned conventional technique can cope with the case where the system is small in size, but when the system becomes large, it becomes difficult for the operator to make a connection switching decision, and the operator cannot make a decision. The burden is increased and efficiency is reduced. Therefore, it is necessary to support the operator's decision to switch by automatically measuring the user's request intention by some method and notifying the operator. An object of the present invention is to improve such problems, and in a service in which a specialized operator provides information to a large number of users by using video communication, the operator's request intention is automatically measured. By providing an environment in which the operator can easily communicate with the user, an information providing system that improves the service establishment rate is provided.

[0004]

In order to achieve the above object, the information providing system of the present invention is used when an operator operates a plurality of terminals and provides information to a terminal user using video communication. Means for monitoring human movements (wide-field camera), means for processing the images of the wide-field camera and detecting human movements (person group monitoring device), and monitoring the state of the user for information provided by the system Means (state sensor), means for driving and controlling the state sensor based on the monitoring result of the person group monitoring means (sensor drive control device), and analysis of the detection result of the state sensor to determine the time change of the user's state. It is characterized in that it is provided with means for intentionally compressing and transmitting it to the operator (intention analysis device). Furthermore, the intention analysis device is provided with a means (intention database) for storing data obtained by statistically arranging typical operations of the user with respect to the information provided by the above system, and the intention analysis device intends the detection result of the state sensor. A means to check whether the user needs an operator by comparing with the data in the database, and to warn the operator when necessary (for example, blinking a part of the display screen, the user on the screen Corresponding to the function to display the urgency)) is characterized.

[0005]

In the present invention, the wide-field camera and the group monitoring device capture the movement, locus and speed of a person around the terminal, and judge whether the person is heading to the terminal or trying to pass. When it is determined that a person approaches the terminal, the operator is notified of the information and the state sensor is driven. The state sensor applies near-infrared light or ultrasonic waves to the user to detect the direction of the face, the line of sight, the blink, etc., and also captures the sound or voice. With the information thus obtained,
The intent analysis device extracts the trajectory of the face direction, the movement of the line of sight, the change of the gaze target, the gaze time, the distribution of the blinks, and the like, and organizes these items so as to compress them temporally, and informs the operator. As a result, the operator can efficiently grasp the states of a plurality of users who are not actually responding, and can take appropriate actions. Further, the intention analysis device actively determines whether or not the user needs the operator and informs the operator, so that the operator can appropriately respond even if there are many users. Therefore,
It is possible to improve the service establishment rate when the operator provides information to a large number of users.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an information providing system in an embodiment of the present invention, FIG. 3 is a display example diagram for an operator in an embodiment of the present invention, and FIG. 4 is a configuration of a user terminal in an embodiment of the present embodiment. It is a figure. In FIG. 1, 1a to 1
c is a display device on the user side, 2a to 2c are user monitor cameras that capture near-infrared light, 3a and 3b are users, 4 is an exchange device, 5 is an image providing device (image DB), and 6 is , A camera on the operator side, 8 is an exchange control device, 9 is an operator, 10 is a wide-field camera, 17a to 17c are near-infrared light sources, 13 is a group monitoring device, and 14 is a state of the users 3a and 3b. A sensor drive control device, 15 is an intention analysis device, and 16 is an intention that statistically organizes and stores typical user's line-of-sight movements, blink frequency, etc. during menu selection, device operation, etc. It is a database (intention DB). Next, the function and operation of each unit will be described. The wide-field camera 10 shoots a wide range so that the movement of a person around the terminal can be seen. While this image is transmitted to the operator 9, it is processed by the group monitoring device 13. The person group monitoring device 13 is composed of an image processing device, a sound analysis device, and the like (not shown), and extracts a person, detects an operation speed, and detects a locus. In order to extract a moving object, a background image with no moving object is obtained in advance,
Take the difference from the image in which the moving object appears. Further, when obtaining the background image, the luminance average of the image at a certain time T and the image one frame before is calculated, this processed image is replaced with the image of T, and the image luminance of the image at the next time and the image of T is calculated. Take the average. When this process is executed in real time, the moving part in the image is erased and the background image is generated in a few seconds to a few tens of seconds. If the difference between the image created in this way and the image at the current time is taken, only the moving object is extracted, so if the position of this object is obtained for each image, the locus and speed are obtained. Further, it is possible to easily determine whether the user is approaching the terminal or passing through the terminal based on the position of the terminal and the movement of the user. Further, when the person group monitoring device 13 detects the approach of a person to the terminal, the person group monitoring device 13 commands the sensor drive control device 14 to drive the state sensor, and the user 3a uses a path (not shown). , 3b are transmitted from the video DB 5 to the display devices 1a, 1b. In addition,
The state sensor is a near infrared camera (user monitor camera 2a
˜2c, near-infrared light sources 17a to 17c), a microphone, a terminal input device (not shown), etc., and mainly the orientation of the faces of the users 3a, 3b, line of sight, blink, sound, operation of the terminal input device. To detect. On the other hand, the sensor drive controller 14
By moving the state sensor (camera, microphone, etc.) according to the movement of the user's head, minute changes can be accurately obtained. This result is sent to the intention analysis device 15.

The intention analysis device 15 compresses the state after the users 3a and 3b come in front of the terminal in time and informs the operator 9. This state is a reaction to the image provided by the image DB 5, and specifically, a face direction trajectory, a line of sight trajectory, a change in a gaze target, a blink distribution, and the like. These are displayed on the display device 7 of the operator as a graph so that the operator 9 can easily judge, but since the psychological changes of the psychology of the users 3a and 3b are shown, the operator 9 displays from this display screen. It is possible to know the user's situation and judge the timing of reception. An example of this display screen is shown in FIG. In FIG. 3, 31
Is a terminal A (near infrared light source 17a, user monitor camera 2
a, a screen showing the state of the display device 1a, etc., 32 is a screen showing the state of the terminal B (near infrared light source 17b, user monitor camera 2b, display device 1b etc.), 33 is the terminal C (near red) External light source 17c, user monitor camera 2c, display device 1c
Etc.), 34 is a portion showing a change in the gazing point of the user 3b of the terminal B, 35 is a portion showing the screen of the display device 1b, 36 is an image of the user monitor camera 2b,
37 is a portion showing a change in blinks, 38 is a portion showing a change in the orientation of the face, 39 is a portion showing the state of key operation, and 40a to 40c are commands. In this example,
The terminals A to C are provided on a part of the screen of the operator side display device 7.
Display the status of. For example, when the operator 9 is responding to the user 3b of the terminal B, the screen 35 of the display device 1b and the image 3 of the user 3b taken by the monitor camera 2b.
6 and various reactions 34, 37 to 39 of the user 3b,
As shown at a glance, they are shown in chronological order. While observing this movement, the operator 9 communicates with the user 3b and provides the user 3b with appropriate information. In addition,
Since the information of the terminals A and C is also displayed at the same time, the operator 9 can monitor the display of the information so that the operator 9 can roughly understand the situation of the user who does not actually correspond.

Further, the intention analyzer 15 determines whether the user needs the operator 9 and actively announces to the operator 9. At this time, the data detected by the state sensor is collated with the intention DB 16 to determine the state of the user. Here, the intention analysis device 1
A method of collating the intention DB 16 according to No. 5 will be described. Figure 4
FIG. 9 is a diagram showing a method of collating an intention DB according to an embodiment of the present invention. In FIG. 4, reference numeral 41 is a state sensor including a near-infrared light source, a camera, a microphone, a terminal input device, and the like, and shows the configuration of FIG. 1 in a simplified manner.
In this embodiment, when the group monitoring device 13 detects that the user is near the terminal, the sensor drive control device 14
Activates the state sensor 41 to collect user movements and sounds. The result is processed by the intention analyzer 15. When collating the data thus obtained with the data of the intention DB 16, a feature amount is extracted from the detection result of the state sensor 41, and this is compared with the feature amount in the intention DB 16 to determine whether or not a predetermined value is exceeded. To judge. For example, when the user is looking at the screen, the movement of the gazing point is detected by the state sensor 4
Record at 1. At this time, the frequency with which the movement of the gazing point at a predetermined angle or more occurs within a unit time is obtained. If this number exceeds a predetermined value, the intention analysis device 15 causes the user to be confused about the menu selection. Determine that Regarding eye movements, blinking behaviors and psychological changes, for example, "A study on the development of a pupil area measuring device and psychological changes, Yasutaka Shimizu, Shunichi Kondo, Takanori Maesako, Ryu Kumagai, Nihon Education Engineering Co., Ltd. Magazine 11 (1987), pp. 25-33 "," Blinking Psychology, edited by Hideoki Tata, Tomio Yamada, Kyosuke Fukuda, published by Kitaoji Shobo ". Further, as a method of announcing to the operator, a part of the display screen shown in FIG. 3 may be blinked, a chime or the like may be sounded, and the urgency of dealing with the user may be displayed on the screen.

In this embodiment, the case where the information about all the user terminals can be displayed in multiple windows has been described. However, due to the large scale of the system (when the number of users is excessive with respect to the number of operators), It is possible that the windows of all terminals cannot be displayed on the operator's display device. In this case, the group control device and the state sensor detect whether or not the terminal is used, and the exchange control device controls so that only the window of the used terminal is displayed on the display device of the operator. Further, when the number of terminals increases, the intention analysis device determines the operator's need degree of the user, and controls to display the terminal information on the operator side according to the need degree. By controlling in this way, a system in which a small number of operators respond to many terminals (for example, distance education service, mail order service,
It can be widely applied to public information guidance services.

[0010]

According to the present invention, the operator can efficiently monitor the situations of a plurality of terminal users at the same time through the display device, and can switch the line with the terminal to respond if necessary. it can. Therefore, even a system in which a small number of operators support a large number of terminals can be efficiently operated. In addition, since it is possible to automatically provide an image to the terminal and detect the approach of the user to the terminal and start the service, the user does not miss the opportunity to use the terminal and the service establishment rate is improved. be able to.

[0011]

[Brief description of drawings]

FIG. 1 is a configuration diagram of an information providing system according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a conventional information providing system.

FIG. 3 is a diagram showing a screen display example for an operator in one embodiment of the present invention.

FIG. 4 is a diagram showing a method of collating an intention DB according to an embodiment of the present invention.

[Explanation of symbols]

 1a display device 1b display device 1c display device 2a monitor camera 2b monitor camera 2c monitor camera 3a user 3b user 4 exchange device 5 image DB 6 monitor camera 7 display device 8 exchange control device 9 operator 10 wide-field camera 13 human group monitoring device 14 sensor drive control device 15 intention analysis device 16 intention DB 17a near infrared light source 17b near infrared light source 17c near infrared light source 21a display device 21b display device 21c display device 22a monitor camera 22b monitor camera 22c monitor camera 23a user 23b use Person 23c User 24 Exchange device 25 Image DB 26 Monitor camera 27 Display device 28 Exchange control device 29 Operator 30 Wide-field camera 41 State sensor

Claims (2)

[Claims] 1. An information providing system in which an operator operates a plurality of terminals to automatically provide information to a terminal user by using video communication, and means for monitoring the operation of the user, and the operation monitoring means. Means for processing the image of the person and detecting the movement of the person, means for monitoring the state of the terminal user with respect to the information provided by the system, and means for driving and controlling the state monitoring means based on the detection result of the movement detecting means. An information providing system comprising: and a means for analyzing a detection result of the operation detecting means, temporally compressing a state change of the user, and transmitting the compressed state change to the operator. 2. A means for storing data obtained by statistically processing a typical operation of a user with respect to information provided by the system, wherein the analyzing means stores the monitoring result of the state monitoring means. The information providing apparatus according to claim 1, further comprising means for checking whether or not the user needs an operator by collating with the data in the storage means, and warning the operator when the user needs the operator. system.