JPH09251441A - Cooperative work control device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To surely and easily control even a time schedule without compelling each member of the cooperative work to do a special job. SOLUTION: This device is provided with an image input part 10b which photographs an ambient situation including the persons related to the group work and inputs these photographed images, a situation information conversion part 10c which generates the situation information showing the states of persons included in the images based on the images inputted from the part 10c, a situation information communication part 12 which transmits and receives the situation information, and an applied service offer part 14 which decides the persons who can join in the group work among those persons related to the work based on the situation information generated by the part 10c and produces the messages of the participant persons of the work to offer these messages to one of information processors via a communication part.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a collaborative work adjusting device and a collaborative work adjusting method which are used in an office environment or the like in which a computer network is provided and which support a collaborative work, for example, a conference operation.

[0002]

2. Description of the Related Art In recent years, a computer network environment has been established in an office environment, and it has become commonplace for one person to have one computer system. Also, the multimedia environment is developing rapidly. On the other hand, various types of systems are becoming more and more complicated and increasing, and the number of situations requiring collaborative work rather than individual work is increasing more and more when developing such systems.

The importance of collaborative work, especially conferences, etc. is increasing more and more, and since the network environment and the multimedia environment are being developed, multimedia conferences via networks (where remote locations are Meetings such as exchanging images and sounds) are now being held.

However, at the stage of deciding the schedule and time of the conference, the promoter said that each member (participant of the collaborative work) who should attend the conference should hear and adjust the schedule of each person by telephone or letter. , Old-fashioned working methods are often used. In order to convene all the people who should attend the meeting by this work, make sure to hear the schedules of all the members once, decide the schedule and time, and surely convey the communication items (schedule, time, etc.) to everyone again. I have to. That is, since it is necessary to exchange information with all the members at least twice, it is a heavy work load, and especially when the number of attendees of the conference is large, the work amount becomes enormous.

As a method of surely transmitting the transmitted matter to all members, it is only necessary to send a message to a communication device carried by each member such as a pager, so that the work load is relatively light. Therefore, in order to reduce the work load, it depends on how to grasp the schedules of all members and to appropriately set the schedule and time for holding the meeting.

Conventionally, as a method of grasping the schedules of all the attendees of a conference and appropriately setting the schedule and time for holding the conference,
In an office environment with a computer network,
A method has been proposed in which each member inputs his or her own schedule into a computer, and an automatic schedule adjustment device determines the holding time of the meeting based on the data (eg: "Matsushita Atsushi,
Kenichi Okada: Collaboration and Communication (Kyoritsu Shuppan) (1995) ”Group Electronic Notebook (Team)
Desk)). With this method, it is not necessary to collect the schedule of the attendees of the conference by using a telephone, a letter, or the like, and the work load is reduced.

However, each person inputs his or her own schedule into the computer only up to the registration of the schedule at the date level, and it is (1) laborious to input a detailed time schedule of the day in advance. It is cumbersome, (2) it is impossible to enter a true schedule because the schedule is frequently changed depending on the progress of work, and (3) it is too busy to correct the schedule entered in the computer. However, there are problems such as being forced to receive calls.

However, in the case of adjusting the schedule on the day when the cooperative work is performed, that is, in order to determine the time (particularly the start time) for performing the cooperative work, if the temporal work status of each member cannot be grasped, the adjustment is made. Is impossible.

[0009]

As described above, conventionally,
When adjusting the time for conducting collaborative work such as meetings, the schedule of participants in the collaborative work is collected using telephone, letters, etc., and the schedule and time are decided based on that.
Since the participants were notified of the matters to be communicated, the work load was heavy.

Further, in an office environment or the like where a computer network is set up, each participant of the collaborative work is allowed to input his or her own schedule, and the schedule automatic adjusting device automatically collects and adjusts the schedule of each participant. There is also a method to do this, but it will force the participants to enter their own schedule, so there is a limit to the registration of the schedule at the date level, and it is not possible to adjust the schedule on a schedule. However, it was difficult to adjust the time schedule.

The present invention has been made in consideration of the above circumstances, and it is possible to surely and easily adjust the time schedule without forcing the participants of the cooperative work to perform a special work. An object is to provide a cooperative work adjusting device and a cooperative work adjusting method.

[0012]

A collaborative work adjustment apparatus according to the present invention has a plurality of information processing apparatuses having image input means for photographing a surrounding situation including a person involved in the collaborative work and inputting an image. And a communication unit that transmits and receives information to and from the plurality of information processing apparatuses, and the information processing apparatus is included in the image based on the image input by the image input unit. A situation information conversion unit that generates situation information representing a situation of a person and transmits the situation information to another information processing device via the communication unit, and a situation information generated by the situation information conversion unit in any one of the information processing devices. Received via the communication means, determine a person who can participate in the collaborative work among the persons related to the collaborative work according to the status information, generate a message for the person who can participate in the collaborative work, and generate the message. Characterized by comprising the application service providing means for providing to one of the information processing apparatus via the means.

Thus, the start time of the cooperative work is adjusted based on the status information indicating the current status obtained from the image information input from the image input means, and the appropriate information is notified at an appropriate timing. be able to.

A situation in which the applied service providing means receives and accumulates situation information about a person involved in cooperative work from the situation information generated by the situation information converting means and transmitted through the communication means. Information reference means, request input for cooperative work, application input / output means for outputting a response regarding the adjustment result of cooperative work according to the input request, and according to the request input by the application input / output means, A message is sent to a person related to the collaborative work according to the application control means for adjusting the collaborative work using the situation information accumulated in the situation information reference means, and the result of the collaborative work adjustment by the application control means. It is characterized by comprising an action creating means for notifying.

Thus, when a third party (a person outside the group) who is not a participant in the collaborative work requests the group for the collaborative work, the cooperative work can be adjusted by using the application input / output means.

Further, the application control means is stored in the cooperative work information storage means for storing the cooperative work condition included in the request for the cooperative work inputted by the application input / output means, and the cooperative work information storage means. The present invention is characterized by further comprising a day-time management means for adjusting the time of the collaborative work based on the situation information about the persons involved in the collaborative work based on the collaborative work conditions.

Thus, the current status of the person who is the target of the cooperative work is input as status information, and the time level can be adjusted based on this status information and the cooperative work condition of the cooperative work.

Further, the same-day time management means, based on the situation information about the persons involved in the cooperative work, the commissioned evaluation value indicating the possibility of accepting participation in the cooperative work for each person, and the commissioned evaluation. The feature of the present invention is that the holding work evaluation value, which indicates the appropriateness of holding the holding work based on the value, and the holding work evaluation value are obtained, and the working work is adjusted.

Thus, by determining the possibility of accepting the participation of each participant of the cooperative work and the appropriateness of holding the cooperative work in consideration of them, it is possible to accurately adjust the cooperative work using the situation information. It is possible.

In addition, a process executed by a certain application service providing means is recursively divided among a plurality of the application service providing means, and the divided partial processes are activated by other application service providing means, thereby The feature is that the processing results of the processes are commonly used.

As a result, when a plurality of application service providing means are connected, the process is recursively divided into a plurality of applications (application service processes).
It is possible to share processing components (processing results by each process) between the two.

For example, in the current actual office environment,
It was necessary for one of the members to actually walk around the office to "observe" the state of each member, and to call out to "convene" the members. Even if it is not input as, it is possible to judge that “it seems to be busy now” “all members can now gather” by observing the work status of the members individually via the camera.

Therefore, if the input of the camera (image) and the image processing (and various input devices) are used, the state of each member can be observed, the holding time is decided by the scheduler and the judging device, and the holding time is decided through the network. It becomes possible to send messages to members and call them together.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the cooperative work adjusting apparatus according to the first embodiment.
In FIG. 1, the flow of components and signals is represented.

The objects to be adjusted by the cooperative work adjusting apparatus in the first embodiment are schedule and time schedule determination for a plurality of people, such as holding a conference, cleaning the office shared use part, and carrying luggage in a cooperative manner. It is considered that there is a collaborative work that requires. Further, the collaborative work is not necessarily limited to the work by a plurality of people, and the present invention can be applied to any work in which one or more people are requested to work.

As shown in FIG. 1, the collaborative work adjustment apparatus according to the present embodiment includes one or more input units 10, situation information communication unit 12, one or more applied service providing units 14, output information communication unit 16, It is composed of one or more output units 18.

The input unit 10 is composed of, for example, a computer used in an office environment, and has at least one image input unit 10b or the image input unit 10b.
It has at least one b, a plurality of input devices 10a, and a situation information conversion unit 10c.

The status information conversion unit 10c includes the input device 1
0a and the input data from the image input unit 10b are converted into situation information indicating the surrounding environment or situation of the participants in the cooperative work. Details of the situation information conversion unit 10c will be described later.

The input device 10a is a device for inputting to a computer such as a pointing device such as a keyboard and a mouse, a microphone such as a microphone.

The image input unit 10b is a camera installed in the vicinity of the computer (or in the display device or the main body), and displays an image of a person at a position for using the computer (a position relative to the computer) or the surroundings thereof. Take a picture. The image input unit 10b is composed of, for example, a CCD camera, an infrared camera, a high-sensitivity camera, or the like. In this embodiment, a color CCD camera is provided as the image input unit 10b, and a color image having grayscale image information is captured.

The status information communication unit 12 interconnects the one or more input units 10 and the one or more application service providing units 14 with each other. The status information communication unit 12 transmits, for example, information in the form of a packet, and the form (wired or wireless) or protocol level does not matter. A bus structure physically composed of a plurality of electric wires may be adopted. In any case, the configuration is such that the situation information conversion unit 10c of the input unit 10 and the applied service providing unit 14 can mutually transmit information.

FIG. 2 shows an example of the form of a packet transmitted through the status information communication unit 12. Following the transmission ID, the destination ID, and the content ID in the header portion of the packet, the content (information) data is included. Unless a packet is specifically requested by the application service providing unit 14 to be described later and the information is transmitted, the destination ID part is usually "application service providing unit general (any device configuring the application service providing unit 14 Very good) ”. The packet contents (information) include temporal difference information,
That is, the status information converted by the status information conversion unit 10c includes information indicating that the status information is changed from a certain state to a certain state.

The application service providing unit 14 inputs the status information obtained by the status information conversion unit 10c of the input unit 10 via the status information communication unit 12, and grasps the persons who can participate in the cooperative work based on the status information. Then, it provides an application service for arbitrating the date and time of cooperative work execution.

The output information communication unit 16 connects the one or more application service providing units 14 and the one or more output units 18 to each other.

The output unit 18 is used to transmit a message to a person who is grasped by the application service providing unit 14 and who can participate in the cooperative work.

The cooperative work adjusting device in this embodiment is
For example, a plurality of computers (input unit 10) are connected to communication means such as a network (situation information communication unit 12, output information communication unit 1).
It is realized by a computer system connected to each other via 6).

A general computer has at least one pair of an input device such as a keyboard and an output device such as a display device. Therefore, it can be considered that the computer has a pair of the input unit 10 and the output unit 18.

The output unit 18 is not limited to a display device such as a display in a computer, but may be a storage unit such as a memory or a hard disk, a printer, a switch for electrical appliances, or the like.
It may be a communication device such as a pager or a speaker,
Anything that can notify the message to the participants of the collaborative work by some method will do. The message output (stored) in the storage unit is notified to the participants of the cooperative work by being displayed on the display, for example.

Next, details of the situation information conversion unit 10c in the input unit 10 will be described.

The status information conversion unit 10c is the input device 1
The situation information of the person or environment relative to 0a is extracted based on the data obtained from the input device 10a or the image input unit 10b.

The status information is displayed in the image input section 10b (camera).
The attributes of the person in the image taken by, for example, the information that can be extracted from the input image, "who", "sitting, not""standing, not standing""looking in one direction""No,""I'm on the phone, no."
The information includes "gaze direction", "expression", and the like. In addition, an attribute determined based on the data input from the input device 10a, for example, "is the work progressing smoothly" determined from the typing rhythm on the keyboard, and "what work is being done now" from the input data. , Etc. is included. As a method of distinguishing the process being executed from the input data, if the computer realizes a system that executes the process while setting an arbitrary window on the display screen, it is possible to determine the process depending on the input data (command, etc.). It does this by knowing what kind of window it is running (which window it is looking at) by looking at the process number it is running.

FIG. 3 shows a detailed configuration of the situation information conversion unit 10c. As shown in FIG. 3, the situation information conversion unit 10
The c is configured to include a person detection unit 20, a personal authentication unit 21, a head direction estimation unit 22, a gaze direction estimation unit 23, a mouth shape motion estimation unit 24, an arm motion estimation unit 25, and a situation information generation unit 26. It Hereinafter, each component will be described.

The person detecting section 20 is for discriminating and extracting a region corresponding to a human (a region corresponding to a face, a limb, etc.) from an image taken by the image input unit 10b (camera), and is shown in FIG. Is configured.

As shown in FIG. 4, the person detecting section 20 is composed of a difference detecting section 20a, a face detecting section 20b, a limb detecting section 20c, a score integrating section 20d, and a person counting section 20e.

The difference detecting section 20a includes an image input section 10b.
In a shooting environment in which a (camera) is installed, an image in a situation where no one is being shot is acquired in advance, and an area that has changed by generating a difference image with that image A candidate for a region corresponding to a limb is detected. Techniques for generating or detecting difference images include (H.Na
kai: '' Non-Parametrized Bayes Decision Method for M
oving Object Detection '', Proc. Second ACCV. Vol 3.
pp447-451 (1995)).

Two processes are performed on the area detected by the difference detection unit 20a. One is the face detection unit 20b
According to the method described above, the grayscale image information of the detected area is used as it is to detect a face. )) Is used to detect the limbs.

The face detection unit 20b uses the average face image data (dictionary image) in each direction, which is created based on the image data obtained by photographing the face from various directions in advance, and the difference detection unit 20a. A person's face is detected by performing matching with the image data of the detected area.

In the present embodiment, the size of the dictionary image handled by the face detecting section 20b is fixed. Generally, if a large image is prepared as a dictionary image, it takes a very long time for the matching process. Therefore, for example, a dictionary image having a size of about 30 × 30 is prepared. Using a dictionary image with a fixed size, the resolution of the captured image is changed according to the size of the dictionary image so that faces of various sizes included in the captured image can be detected. After letting it
Matching with the dictionary image is performed.

Regarding the degree of matching (similarity),
Similarity according to the matching method used, for example correlation,
Set by simple, compound, mixed similarity, etc. The face detection unit 20b removes a face area candidate from a candidate that is a face area when the similarity value of the area detected by the difference detection unit 20a with respect to the image data is less than or equal to a certain threshold value. Determined as a candidate for the area. The number of candidates may be plural.

The limb detection unit 20c again obtains, for example, hue information, saturation information, and lightness information from the photographed color image for the location of the difference image obtained using the binary image generated by the difference detection unit 20a. Based on this, area division is performed and labeling is performed. The limb detection unit 20c uses the attribute information such as the position information, the shape information, the hue, and the brightness of those regions to extract a combination that is considered to be a region corresponding to a human limb. The limb detection unit 20c defines the reliability of the shape model of the legs, torso, arms, etc. from the shape, area, etc. of the combination of the extracted regions (regions considered to correspond to human limbs), and The region of the combination that maximizes the sum of reliability is extracted as a part of the human limb (hypothesis at this stage).

The face detecting section 20b and the limb detecting section 20c are
Regarding the region in the image where humans (face, limbs, etc.) in the input image are considered to exist, the position of the face, the position of the limbs (foot,
Data indicating the position of the torso, arm, etc. and information describing the reliability are sent to the score integration unit 20d.

The score integration unit 20d integrates the hypotheses of the face position and the limb position obtained from the face detection unit 20b and the limb detection unit 20c, and votes the existence probabilities of the respective hypotheses in a two-dimensional space.

The number-of-people coefficient section 20e is the score integration section 20d.
The existence probabilities of the hypotheses voted by are comprehensively determined, and how many people are photographed in the image are determined.

A certain threshold value is set for the space in which the existence probability is voted by the score integration section 20d, and the number of photographed persons, the positions of the photographed persons are set as the area higher than the threshold value, which corresponds to a human. Position information such as size (area in image), approximate orientation, face position, foot position, torso position, arm position is determined and stored.

Here, the threshold is changed according to the predictable information and the environment. For example, when it is known that the setting direction of the overall threshold value is improved due to a change in sunshine conditions such as morning and afternoon, or that a person is moving within a shooting range by the image input unit 10b from a certain location, Since the size of the limbs and the like are estimated at the previous location, control such as changing the threshold value in the vicinity of the region in advance is performed in the imaging range depending on the predicted position in the image. The parameters used for these are obtained by experiments.

Next, the personal authentication section 21 will be described.
The personal authentication unit 21 authenticates who is the face photographed at the face position detected by the human detection unit 20. The personal authentication unit 21 performs collation on the images in the respective areas for the number of persons previously detected by the person detection unit 20.
As an authentication method, identification is mainly performed from among persons who are registered as recognition targets in advance.

FIG. 5 shows a process of the person detecting section 20. Based on the input image (difference image) shown in FIG. 5A detected by the difference detection unit 20a from the images captured by the image input unit 10b, the face detection unit 20b
As shown in FIG. 5 (b-1), the face position candidates and their respective degrees of reliability are obtained.
As shown in 2), candidates for limb positions and their respective degrees of reliability are obtained. The score integration unit 20d calculates the reliability and each candidate voted by the face detection unit 20b and the limb detection unit 20c.
Integration is performed as shown in FIG. People counting unit 20
The e is subjected to the threshold processing by setting the threshold value to "1.0" from the points integrated by the score integration unit 20d as shown in FIG. 5 (c), as shown in FIG. 5 (d). Get results. In this example, it is estimated that two people are photographed.

The personal authentication section 21 is configured as shown in FIG. 6, and authenticates who is the face photographed at the face position obtained by the person detection section 20. As a method of authentication, identification among registered persons is the main. As shown in FIG. 6, the personal authentication unit 21 includes a normalization unit 21a, a collation unit 21b, a dictionary storage unit 21c, and an evaluation unit 21d.

The normalizing section 21a detects the eye position from the image of the face position detected by the person detecting section 20 (face detecting section 20b), and based on the eye position, the collating section 2 in the subsequent stage.
For the processing in 1b, an image (normalized image) having the same size as the dictionary image stored in the dictionary storage unit 21c is generated (normalized).

The collation unit 21b, for the image of the face normalized by the normalization unit 21a, stores in the dictionary storage unit 21.
Each of the dictionary images stored in c is collated. In the present embodiment, the matching unit 21b calculates the similarity between the input image and the dictionary image using the composite or mixed similarity.

The dictionary storage unit 21c stores in advance dictionary images for each person to be recognized.

The evaluation unit 21d has a dictionary image having the maximum similarity value among the plurality of similarity values obtained for each of the dictionary images (the number of persons to be recognized) registered in the dictionary storage unit 21c. Is determined as the recognition result, that is, the face of the person corresponding to the dictionary image is included in the input image. However, if the similarity value is lower than a threshold value designated in advance, the person who has been photographed is not determined and is not recognized.

The personal authenticating section 21 collates a plurality of persons by the number of persons previously detected by the person detecting section 20.

Next, the head direction estimating section 22 will be described. The head direction estimation unit 22 estimates the head direction from the face direction and the appearance of the limbs (head position, torso position, arm position, etc.) detected by the human detection unit 20.

The head direction estimation unit 22 estimates the direction in which the face is facing by collating with the patterns of a plurality of face models using the grayscale information near the face position detected by the human detection unit 20. . As a degree of similarity in matching with a pattern, a known example using correlation (Tsukamoto, Lee, Tsuji: Head motion estimation using multiple models, IEICE Transactions Vo
177-D-II, No. 8, PP. 1582-159
0 (1994)), the pattern similarity is examined using the composite similarity to determine the face orientation.

Further, the head direction estimating unit 22 determines the direction of the face from the characteristic points such as eyes and nose (disciplinary willow, Kiyoharu Aizawa, Mitsutoshi Hatori: estimation of human face direction:
The orientation of the face may be determined by using, for example, the Technical Bulletin PRU-49-18 (1994)).

Similarly, the head direction estimating section 22 compares the orientation of the limbs with the image of the area group (including the feet, torso, arms, etc.) detected by the human detecting section 20 and the pattern to determine the orientation. Ask for.

Next, the line-of-sight direction estimation unit 23 will be described. The line-of-sight direction estimation unit 23 estimates the line-of-sight direction by using the image input unit 10b (color CCD camera) and the feature amount obtained from the eyes at the position of the face in the image captured from a distant position or the vicinity of the eyes. .

As a method of detecting the line-of-sight direction, Japanese Patent Laid-Open No.
A method for determining the relative amount of movement of the line of sight in 255515 by extracting the contour data of the pupil from changes in the center coordinates of the pupil, and controlling the cursor position of a word processor or the like by the line of sight in JP-A-3-17696. As a method, a method of dividing the eye region by a vertical line and a horizontal line passing through the center of the pupil and using the area ratio of the divided regions as a feature amount to calculate the viewpoint position can be used.

In this embodiment, the relative positional relationship between the inner and outer corners of the eye and the center of the pupil is used as a feature vector, and the feature vector is linearly identified, and by pattern recognition, which direction is seen with respect to the face direction? To estimate.

Next, the mouth shape motion estimation unit 24 will be described. The mouth shape motion estimation unit 24 detects opening / closing of the mouth from the mouth portion in the image of the face position detected by the person detecting unit 20. As a method of detecting the opening and closing of the mouth, (Mase, A.Pent
land: A trial of lip-reading using optical flow, IEICE Transactions D-II, Vo173-D-II, No.6, pp.796-803, (1
990)) and (ALYuille, PWHallman, DSCohen: Featu
re Extraction from Faces Using Deformable Template
s, International Journal on Computer Vision, Vo18, n
o.2, pp, 99-111 (1992)) can be used.

In the present embodiment, for example, five types of opening / closing patterns are prepared according to the opening / closing degree (opening / closing degree) of the mouth in multiple stages, and as a result of matching each pattern with the mouth portion in the input image, the result is the closest. The degree of opening and closing according to the pattern is currently
The image input unit 10b opens and closes the mouth of the person who is the subject of photography. The degree of opening and closing of the mouth at each point of time is discriminated with the passage of time, and the frequency information of the degree of opening and closing of the mouth is obtained to detect talking, yawning, and the like.

Next, the arm motion estimating section 25 will be described. The arm motion estimation unit 25 uses, for example, a method based on hand detection using hue information. Based on the result of the area division performed by the person detecting section 20, the arm motion estimating section 25 determines not the face area but the area having the skin color hue as the hand area candidate. The hand region candidate is selected based on some relative constraints such as the relative positional relationship with the face position and the size of the region. The arm motion estimation unit 25 extracts the barycentric position of the hand region as a feature amount,
Estimate the movement of the arm from the change of the center of gravity position of the hand region in the image input with the passage of time, and further walk,
Detects motions such as waving hands.

Next, the status information generation unit 26 will be described. The situation information generation unit 26 includes the processing units 21 and 2 described above.
The image analysis information in 5 is integrated to generate a description (state information) of a certain human state. The situation information generator 26
As shown in FIG. 7, the integration unit 26a, the time generation unit 26b,
The location information storage unit 26c, the history storage unit 26d, the integrated action recognition unit 26e, and the integrated information storage unit 26f are included.

The integrating unit 26a inputs and integrates the detection results (recognition results, estimation candidates) in the processing units 21 to 25. The integration unit 26a displays the integration result as the integrated action recognition unit 26e.
8 and is prepared in the history storage unit 26d in correspondence with the number of persons detected by the person detection unit 20, FIG.
The detection results of the processing units 21 to 25 are registered in the slots of the situation registration table as shown in FIG.

The time generation unit 26b considers the interval of the image acquisition and recognition processing time and generates the date and time so that the interval can be expressed in units that can be expressed. As for the time, the time when the image acquisition was performed is recorded.

The location information storage section 26c is used for the image input section 10.
b (Color CCD camera) describes the information based on the place where the camera is installed. For example, when considering the shooting range, the distance between the person and the camera depends on the size of the person in the captured image. Information such as the degree,
The setting method of the coordinate system for the image used at the time of detecting each information and the position information of the global installation place are stored in advance. Note that a method may be used in which these pieces of information are integratedly managed in a certain storage location and only the identification number is generated in each status information conversion unit 10c.

The history storage unit 26d stores and accumulates the information generated by the integration unit 26a in time series, and is referred to when the integrated action recognition unit 26e needs past information.

The integrated action recognition section 26e creates a new level of situation information from the temporal change of the information created by the integrated section 26a. For example, when the mouth is continuously opened and closed, situation information such as "talking" is generated,
Since the position of the person is continuously moving and the arm is descending, situation information such as “walking” is input as a time-series pattern, and the dictionary for action recognition stored in the integrated information storage unit 26f is used. Recognize by referring. Other examples include
"I'm not talking,""I'm laughing, I'm not,""I look at the screen lightly, I'm not,""I'm staring at the screen.
Status information such as "No", "Reading / Not reading document" is generated.

The situation information recognized from the temporal change as described above is referred to as “relatively high-level situation information” in the following description.

In order to recognize these relatively high level situation information from the time change of the information held by the integration section 26a, for example, the following method is used. First, the relationship between the time series (temporal transition) of the level status information generated by the integration unit 26a and the status information of a relatively high level is HM.
Learning is performed using M (Hidden Markov Model) and neural networks. The integrated information storage unit 26f stores this learning data as a dictionary. By comparing this learning data with the time-series data input in the actual scene, a relatively high level situation information is obtained.

The status information obtained at the sensor level is shown in FIG.
The situation registration table (No. 1) as shown in FIG. The relatively high-level status information obtained by the integrated behavior recognition unit 26e is also registered in the history storage unit 26d as a status registration table (No. 2) as shown in FIG.

The situation information conversion unit 10c includes the image input unit 10
The situation information is not limited to being generated only based on the input image from b. For example, information obtained from the input device 10a, for example, measurement of a keyboard typing rhythm,
The amount of movement of the chair and the position of the center of gravity of the person sitting on the chair can also be used as situation information.

In the case of measuring the typing rhythm of the keyboard, when the fundamental frequency is measured, two frequencies with different scales are extracted. High frequency rhythms are fast rhythms like when you type a word spelling. The low-frequency rhythm is a slow rhythm that occurs when you think about each sentence or phrase. By measuring the fundamental frequencies of these two rhythms and measuring the disturbance from the fundamental frequencies, the progress of work can be recognized as a rhythm.

Further, when detecting the movement amount of the chair or the variation of the position of the center of gravity to the chair, it indicates that the operator is not concentrated, and the deviation from the basic position or the frequency of change from the stable position. Can be used as a situation indicating the degree of concentration of the operator.

As described above, the status information integration section 26 generates status information of various levels. The status information integration unit 26 transmits the temporal difference information of the changed status information as a packet via the status information communication unit 12.

Regarding the method of sending information (packet form) to the status information communication unit 12, there will be described in detail that there are various methods in the description of the second embodiment described later.

Next, details of the application service providing unit 14 will be described. FIG. 10 is a block diagram showing the basic configuration of the application service providing unit 14 and the signal flow. As shown in FIG. 10, the application service providing unit 14 includes a situation information reference unit 30, an application control unit 32, an application input / output unit 34, and an action creation unit 36. The detailed configuration of each of the units 30, 32, 34, 36 takes various configurations depending on the content of the application.

The application service providing unit 14 is connected to the situation information communication unit 12 by the situation information reference unit 30. The application control unit 32 includes the application service providing unit 14
The central part is connected to the situation information reference unit 30, the application input / output unit 34, and the action creation unit 36. The action creation unit 36 is the output information communication unit 16
Is connected to

The status information reference unit 30 is the status information communication unit 1.
2 transmits and receives the information (transmitted in the form of the packet shown in FIG. 2) flowing through 2. The situation information reference unit 30 determines the destination I
D is compared with its own ID, and if they match, the contents are fetched, stored, and updated. Alternatively, the situation information reference unit 30
When inquiring the input unit 10 about the status information, the destination ID part is set to "input part general", and all input parts 1
A packet is transmitted for 0.

The application control unit 32 executes service processing or data management for performing a service for adjusting the date and time of the cooperative work based on the situation information obtained via the situation information reference unit 30.

The application input / output unit 34 communicates with the situation information communication unit 12 information that is necessary for controlling the application and is not situation information, such as e-mail for requesting or responding to adjustment of the schedule (date and time) of the cooperative work. Input and output via another communication path (not shown). However, the communication path here may be logically different from the status information communication unit 12 or the output information communication unit 16 or may be physically on the same network path.

The action creating section 36 selects the output section 18, creates output information representing a message for requesting the cooperative work to the output section 18, and sends the output information to the output section 18.

The application service providing unit 14 has a basic structure as shown in FIG. 10, but the detailed structure of the constituent elements differs depending on the contents of the applied application. The detailed structure of the application service providing unit 14 in this embodiment will be described later.

Next, details of the output unit 18 will be described.

The output unit 18 transmits a signal (output information) from the application service providing unit 14 via the output information communication unit 16.
In response to this, for example, the window display method on the screen of the display is updated or the message character string is displayed in the window. Alternatively, the application service providing unit 14
However, it is also possible to adopt a configuration in which a control signal for turning on a switch of a certain room is output, and in this case, the switch of the room corresponds to the output unit 18.

Packets flowing through the output information communication unit 16 are also
As shown in FIG. 12, a structure having a transmission ID and a destination ID is adopted so that the destination and the transmission source can be known. Output unit 18
Receives a packet whose destination ID is its own ID,
According to the output information, it is converted into media such as characters, sounds, and pictures (or converted into device control signals) and output.

Next, a method of configuring the application service providing unit 14 for realizing the cooperative work adjusting apparatus of this embodiment will be described.

FIG. 11 shows the application service providing unit 1 of this embodiment.
It is a detailed block diagram of FIG.

The status information reference unit 30 is composed of a communication unit 30a, a member information control unit 30b, and a member information storage unit 30c. The status information communication unit 12 transmits, receives, decodes, and cooperates with packets. Manages, stores, and updates the status information of the members requested to.

The communication section 30a generates request packets, receives status information packets, and interprets them. The member information control unit 30b, based on the request from the application control unit 32, of the member information (information indicating the state regarding the members of the cooperative work (set in the state table)) registered in the member information storage unit 30c. Send, manage, update, etc. The member information storage unit 30c stores member information required by the current day time management unit 32c of the application control unit 32.

The application control unit 32 comprises a cooperative work information storage unit 32a, a schedule management unit 32b, a current day time management unit 32c, and a schedule storage unit 32d. The application control unit 32 stores a request request obtained via the application input / output unit 34, and has a function of adjusting a schedule and adjusting the time of the day.

The application input / output unit 34 includes a request input unit 34a, a request response unit 34b, and a schedule input unit 3
4c. Application input / output unit 34
Is a part for inputting / outputting necessary data other than status information and for inputting work requests. A work request is input in the request input unit 34a, schedule data is input in the schedule input unit 34c, and a response to each input is made from the request response unit 34b.

The action creating section 36 is composed of a message creating section 36a, and sends the message to the output section 18 of the request destination via the output information communication section 16.

Next, the application service providing unit 1 shown in FIG.
The flow of processing for inputting a request and determining a schedule when the configuration of 4 is used will be described based on the flowchart shown in FIG.

First, each member starts by inputting his or her own schedule using the schedule input section 34c (step A1). The schedule input here is a schedule-level schedule, and fills, for example, one's own line (a line corresponding to one's own person index) in the schedule table as shown in FIG. Here, it is not necessary to input a schedule of a fine time level, but it is sufficient to input a time period when the user is absent from the office, such as vacation (vacation schedule R) or going out (going out g).

The schedule of each member is input from the schedule input section 34c of the application input / output section 34 and output to the schedule storage section 32d of the application control section 32. Since the schedule table is recorded as a schedule table based on the date index, any schedule such as weekly schedule, monthly schedule, and yearly schedule may be used. The inputted schedule of each member is organized in the form of the schedule table of FIG. 13 and stored in the schedule storage unit 32d (step A2). No) inputs the information (cooperative work condition) of the cooperative work requested to the request input unit 34a. For example,
If the requested collaborative work is a meeting, the request information that will be entered includes the due date, due date, members (participants), meeting details, the minimum number of people to perform, the priority of the work, and the likelihood from the holding of the meeting to the end. Time (forecast required time) etc. is entered. The request input unit 34a stores the input request information in the cooperative work information storage unit 32a (step A
3).

The schedule management section 32b starts its operation in response to the input of the request request from the client, and carries out the schedule adjustment process. Details of the schedule adjustment process will be described later. (Step A4).

Next, the request responding section 34b has the request input section 3
A request response for notifying the requester whether the request content input from 4a can be accepted is output. For example, it may be a message "I accept" or a response "Meeting cannot be held under this condition" (step A5). If the request cannot be accepted, it returns to a state in which the request content can be input and waits for a new request (or a modified request) (step A6).

Although the request request is described as being input from the request input unit 34a, it may be considered that the request is actually input through a keyboard of a computer forming a network. In order to clarify the flow of information and signals, the request input unit 34a is explained separately.

Further, in the configuration of the present invention, since the application service providing unit 14 can use the status information of each member, the work requester does not specify the member by name but designates the status information by an attribute. You can also do it. For example, "Now
It is possible to specify attributes such as "collect 10 people who are seated" and "collect 5 people who have read e-mail for 10 minutes or more in the next hour".

The schedule management unit 32b can obtain the member who matches the specified condition by extracting the situation information corresponding to the specified attribute.

Based on the result of the schedule adjustment processing by the schedule management unit 32b, the planned holding date of the conference is determined. Until the requested meeting ends, the candidate dates that can be held are stored in the schedule management unit 32b, and the candidate date standby state is set.
The schedule management unit 32b counts the number of days by a calendar counter that is internally provided. When the calendar counter matches the candidate date, the schedule management unit 32b transmits a control signal for activating the current day time adjustment process to the current day time management unit 32c. (Step A
7).

The day time management section 32c is a schedule management section 32.
The start control signal from b is received, and the day time adjustment processing is performed. Details of the current day time adjustment process will be described later (step A8).

At the time suitable for holding the meeting, the current day time management unit 32c sends a message creation process start signal based on the processing result of the current day time adjustment process to the message creation unit 36a.
(Step A9).

Message creation unit 36 that has received the control signal
a creates a work start message to each member based on the information in the cooperative work information storage unit 32a, and sends a message to each member via the output information communication unit 16 to the effect that a meeting will be held, for example. . FIG. 14 is a view showing an image in which a conference holding time is notified when the cooperative work adjusting apparatus of the present invention is used. In FIG. 14, the output unit 18
The display is used as, and the message is notified by characters. Other output unit 18 such as a speaker may be used.

The above is the overall processing flow.

Next, the schedule adjustment process will be described in detail. The flow of the schedule adjustment process will be described based on the flowchart of FIG.

Each member who uses the computer connected to the network environment inputs his or her schedule using the schedule input unit 34c. As a result, the schedule storage unit 32d stores a schedule table as shown in FIG.

Schedule management unit 32b according to the requester's input
Extracts the row of the requesting member from the schedule table of the schedule storage unit 32d (step B1). The schedule management unit 32b performs a process of taking the logical sum of the information of the extracted table for each time period, and extracts the time period in which no one is scheduled (step B2).

The schedule management unit 32b extracts only the time period longer than the required forecast time from the extracted time periods (step B3). The section extracted here is a time zone (candidate holding time zone) in which a conference can be held.

The schedule management unit 32b prioritizes this time zone (generally, the earlier the day, the higher the priority in general unless the client inputs special specifications), and the current day time as the candidate time for the event. It is transmitted to the management unit 32c (step B4).

If there is a candidate time that satisfies the input contents of the requester, the request is accepted (steps B5 and B6).
If it does not exist, the request is not accepted (step B).
7). In this case, the request response unit 34b transmits a message notifying that the request is not accepted to the requester, and returns to the request input waiting state again.

Although the candidate date for holding is determined at this point, it is also possible to send a message for notifying the requesting member of the cooperative work about the candidate date for holding, the contents of the meeting, the purpose of the meeting, etc. at this time. In this case, the schedule managing unit 32b sends a control signal to the message creating unit 36a so that the message creating unit 36a creates a message to the member based on the information stored in the cooperative work information storage unit 32a. Then, the message is transmitted to each output unit 18 corresponding to each member through the output information communication unit 16.

The above is the description of the schedule adjustment process.

Next, the current day time adjustment process will be described in detail. The flow of the current day time adjustment process will be described based on the flowchart of FIG.

The day time management unit 32c is a schedule management unit 32c.
When the control signal for notifying that the candidate date has come from b is received, the member information control unit 3 of the situation information reference unit 30.
0b is requested to refer to the state table as shown in FIG. The status table is stored in the member information storage unit 30c, and the information indicating the current status is updated for each member (person index) according to the status information flowing through the status information communication unit 12.

Since only the change information flows in the status information communication unit 12 when some kind of status information changes, there is no initial status of the status information in the initial stage of the process. That is, since the content of the member information storage unit 30c remains the initial setting value, the member information control unit 30b requests the communication unit 30a to acquire the initial state of the member, and the communication unit 30c creates an inquiry signal. The situation information communication unit 10
An initial state is acquired via 4 (step C2).

On the other hand, since the status change information is flowing in the status information communication unit 12, after the acquisition of the initial state, the member information control unit 30b, based on the information received by the communication unit 30a, the member information storage unit 30c. It is sufficient to continue the process of updating (step C1).

The status information signal flowing through the status information communication unit 104 is from the image input unit 10b, for example, "seated / not seated", "on the phone / not seated", "writing sentence / not seated", etc. The various situations are determined based on the input image information or the input information input from the various input devices 10a.

Depending on the service contents provided by the application service providing unit 14 or the request contents of the client, it may be necessary to interpret a more complicated state. For example, the interpretation is "you are doing a busy job / not so" or "you need to be motivated because you are doing a monotonous job / not so".

The application control unit 32 receives the status information flowing in the status information communication unit 12 via the status information reference unit 30, and the apparatus is configured so that the status can be interpreted from the status information.

For example, with situation information as a feature vector,
The device may be configured by applying a learning type neural network or pattern recognition theory so that the interpretation of the state is used as the output pattern. The construction method for obtaining the pattern output from the input of the feature vector is detailed in, for example, "Noboru Funakubo" Pattern Recognition "Kyoritsu Shuppan (1991)""TaisouIijima""Pattern Recognition Theory""Morikita Shuppan (1989)". Also, if it is difficult to handle the interpretation of the situation deterministically,
A method of processing data attributes with warmth is also described in, for example, “Baba” Fuzzy Database Current Status and Trends “Adva
nced Database System Symposium '90, pp.207-214 (198
9) ”.

Here, a method of interpreting a situation based on a weighted linear sum of feature vectors will be described as an example.

The day time management unit 32c calculates the request acceptance evaluation value P as an evaluation value indicating the possibility of accepting the meeting of each individual from these situation information. Requested consignment evaluation value P
Is calculated by the weighted linear sum of Expression (1) using the discriminant value (situation information) of the state described above.

[0136]

[Equation 1]

In the equation (1), x (i) is a function that takes a value of "1" or "0" depending on the interpretation result of the i-th state. For example, "sitting / not sitting" When interpreted, the value of x is "1" and "when seated",
"0" represents "when not seated".

Similarly, the values "1/0 not being used" and "While writing / not writing" are also taken.

In the equation (1), a (i) is a weight given to the interpretation result of the i-th state, and is a value giving the relative weight of N states from i = 1 to N.

N weights a (i) (i = 1, ..., N)
Is likely to accept the collaborative work of each person,
Adjustments are made so that the commissioned evaluation value P increases. The adjustment may be set experimentally and empirically.

For example, when the user is seated, the output message is more likely to be transmitted and the possibility of participating in the work is higher than when the user is not seated. Therefore, the weight has a positive value. On the contrary, when the user is on the phone, it is less likely to be able to participate in the work than when the user is not on the phone. Therefore, the weight has a negative value. In addition, for example, when there is little movement of the viewpoint and the user is gazing at the characters on the display, it is highly possible that he or she concentrates on the work, and therefore the possibility of participation is low. As described above, the image information includes a large amount of data capable of estimating the work status of each member, and these data are transmitted as status information, so that the current status of the member can be grasped.

If there is no possibility of participating in the work (for example, it is possible that the user is out of the seat and the whereabouts are unknown), the commissioned evaluation value P is adjusted to be "0". To do.

The request acceptance evaluation value P calculated by the current day time management unit 32c is stored in the member information storage unit 30c via the member information control unit 30b as state table data. Each time there is a change in the member status information, the status information reference unit 30 requests the current day time management unit 32c to update the request trust evaluation value P of the member information. The current time management 32c calculates the request acceptance evaluation value P of the changed member (step C3). After the step C2, the request acceptance evaluation values P of all members are sequentially calculated until the work start message is transferred to the creation processing.

In accordance with the calculation result of the request acceptance evaluation value P by the day time management unit 32c, the member information storage unit 3
The request acceptance evaluation value (state table) of 0c is updated (step C4).

Next, the day time management unit 32c determines the holding evaluation value V for judging whether or not the work (conference) should be held.
Is calculated (step C5). Conference evaluation value V
Is calculated as follows.

The request acceptance evaluation value P of each individual is obtained based on the equation (1). Assuming that there are M members in the conference, the commissioned evaluation value of each member is P (j) (j = 1,
..., M). A weighted linear sum of the commissioned evaluation values is taken, and this is set as an evaluation value V indicating suitability for determining whether or not to hold a meeting, and is called a holding evaluation value V. The event evaluation value V is calculated by the equation (2).

[0147]

[Equation 2]

W (j) (j = 1, ..., M) is a weight set according to the necessity of participation of the members making up the conference.
If all of them have the same degree of necessity, a fixed value (for example, 1) may be set for all j. Alternatively, for example, if the leader of the group absolutely needs to participate, the weight w applied to the leader will be a large value. Or,
If the collaborative work is not a meeting but, for example, carrying luggage, an application such as increasing the weight of young members can be considered.

This weight may be set by the collaborative work requester, may be adjusted in advance so as to follow the custom, or may be automatically set from the contents of the collaborative work request. If there is (for example, there is a method of reusing the weight previously used by the same member), it may be determined as such. If there is no requester's input and there is no automatic setting value, all of them should be set to a fixed value.

Next, the day time management unit 32c prepares an evaluation judgment value for comparison with the holding evaluation value V (step C6). This evaluation judgment value is a value that changes with time. Using this evaluation judgment value as a threshold, it is judged whether or not the holding is appropriate. In the following description, this threshold value is represented as T.

FIG. 18 shows an example for explaining the evaluation judgment value (threshold value T).

In FIG. 18 (e), the vertical axis is the holding evaluation value V axis and the horizontal axis is the time axis. In the time axis direction, the holding candidate time zones transmitted from the schedule management unit 32b are arranged in time order from the time axis origin.

For example, as shown in FIG. 18A, it is assumed that the holding candidate time period is from 8:00 to 14:00. Here, as shown in FIG. 18 (b), since the time required for the conference to be in sight is 2 hours, unless the conference is started between 8:00 and 12:00, the conference will not be completed within the time in which everyone can participate. Therefore, as shown in FIG. 18C, the time from 8:00 to 12:00 is extracted as a start candidate zone and arranged from O (origin) on the time axis shown in FIG. 18E.

In FIG. 18 (e), the deadline time of the end of the conference is the end E in the horizontal axis direction, and the line-of-sight required time of the conference is the position D traced back in time. A straight line is drawn between D and S on the evaluation value axis, and this is set as a threshold value T. The height of S is the value of the holding evaluation value V when the commissioned evaluation value P of each member is set to the maximum.

The day time management unit 32c calculates the holding evaluation value V within the operating time, and judges that the meeting can be held when V is equal to or greater than the threshold value T (step C7). .

The threshold value T shown in FIG. 18 is higher as the time is earlier and lower as the time is later. Therefore, while there is enough time, it is not determined that the event can be held until the holding evaluation value V becomes sufficiently high, but when the time becomes insufficient, the determination that the event can be held is facilitated. Also, from the deadline, make sure that the event can be held before the expected time for the conference.

In the example shown in FIG. 18 (e), the threshold value T is linearly changed from S to D. However, the half-life is set to 8 hours or the like and the threshold value T is changed by a decreasing exponential function. Is also good. This half-life may be set according to the experience of the constituent members. Alternatively, if it is desirable that the conference is preferentially set in the morning, the threshold value T in the morning may be set in a polygonal line so as to be low. The method of changing the threshold value T with time may be determined based on the customs and experience of members who use the environment. However, it is necessary to use a decreasing function and set it so that the meeting is held by the time when the forecast time goes back from the deadline.

If the minimum number of people to entertain is set, the number of members whose request trust evaluation value P is not 0 is
A process for confirming that the number of persons who have performed is exceeded is added (step C8).

With the above steps, the day time management unit 32c
Can determine the appropriate time to start the collaborative work (meeting time is determined in the case of the convening system).

When the start time of the cooperative work is determined, the current day time management unit 32c sends a control signal to the message creation unit 36a.

The action creating section 36 that has received the control signal
Creates a work start message to each member based on the information in the cooperative work information storage unit 32a, and sends the work start message to the output unit 18 including the members through the output information communication unit 16 (step C9).

In the case of this embodiment, assuming that the output unit 18 is a computer display or a speaker connected to the computer, the input unit 10 and the output unit 18 are paired. Therefore, the message creating unit 36a sends a message to the output unit 18 of the computer having the corresponding member based on the personal authentication result obtained from the situation information from the situation information converting unit 10c (step C9). In this case, even if the member to whom the message is to be sent is not in the place (computer) where it should be, the message can be transmitted as long as the output unit 18 in the network environment exists.

If it takes a long time to create a message, the message creating unit 36a creates a part of the transmission message that can be created in advance at the stage when the candidate schedule is determined. It is also possible to adopt a method in which the signal is used only for controlling the transmission timing.

As described in detail above, by using the present invention, the start time can be adjusted according to the current situation of the participating members of the cooperative work without inputting a fine schedule in terms of time. A collaborative work adjustment device can be configured.

Next, as a second embodiment, a modified example of the information flow will be described.

In the first embodiment, the situation information communication unit 1
The situation information flowing through 2 has been described in the form of being communicated only when the situation information of the input unit 10 has changed. At this time, the format of the packet flowing on the communication path (situation information communication unit 12) has the structure shown in FIG.

Besides this, the form of the information to be communicated can be variously modified.

(1) Broadcast type This is a form in which the structure of the status information is transmitted at a certain time interval regardless of whether the status information has changed.
In this case, a structured packet as shown in FIG. 19, for example, flows on the communication path every 1/100 second. The structured packet contents of FIG. 19 correspond to the individual contents of the state table shown in FIG. 17, for example.

In this case, the status information reference section 30
It is not necessary to have a storage unit (member information storage unit 30c in FIG. 11), and the application control unit 32 may use the information in the situation information disassembled by the communication unit 30a.

(2) Request and Response Type In this mode, the application service providing unit 14 inquires the input unit 10 via the situation information communication unit 12 about the situation information necessary for interpretation. In this case, inside the situation information reference unit 30 of the application service providing unit 14 (for example, the communication unit 30a), a packet for inquiring necessary information is created according to a request from the application control unit 32, and the situation information communication unit 12
Send to

(3) Information Difference Type This form is the form of the packet described in the first embodiment.

In any form, the application service providing unit 1
If 4 is a communication format that can use status information, the present invention can be easily applied.

Next, as a third embodiment, the case where the input object of the situation information is one person will be described.

In the first embodiment, the convening of a meeting in which a plurality of members participate as a collaborative work has been described. However, even when the target for detecting the situation information is one,
The configuration of the present invention can be applied. In this case, it is only necessary to interpret it as the case where one person performs the cooperative work. For example, it is possible to configure an electronic mail system that makes it more likely that an electronic mail will be read upon arrival than in conventional methods.

Now, a method of realizing an electronic mail system in which the possibility that the other party will read the electronic mail upon arrival will be higher than that of the conventional method will be described below.

In the first embodiment, the request acceptance evaluation value P is defined as an internal procedure of the application control unit 32, and it is obtained by the weighted linear sum based on the equation (1). In the third embodiment, instead of the request acceptance evaluation value P, an evaluation value that increases in numerical value in a situation where there is a high possibility of reading a sentence is introduced.

The possibility of reading the received e-mail on the spot is higher, for example, in the situation where the eyes are directed to the display, and when the typing is performed with a good rhythm, it may be concentrated on writing the text. It is unlikely that it will be read. On the other hand, when typing is stopped and the posture of the body does not change much, and when the user is gazing at the display, he / she is thinking about something and is likely to be read.

As described above, the image information input from the image input unit 10b includes a large amount of information for estimating the possibility of reading an electronic mail to some extent. It is possible to infer this information, including individuality, using pattern recognition methods or fuzzy measure methods.

From the above, the flow of processing of the third embodiment will be described with reference to the flowchart shown in FIG.

First, a request for sending an electronic mail (address, text) is input from the application input / output unit 34 of the application service providing unit 14 (step D1). The application control unit 32 extracts the status information, which is the initial state of the member information, from the status information reference unit 30 (step D2).
Step D4).

The request acceptance evaluation value is calculated based on the coefficient of the equation (1) adjusted in advance (step D5). As described above, the request acceptance evaluation value has a high value in a situation in which the text is likely to be read.

The threshold value T which is the preset value of the commissioned evaluation value
If it is larger, the application control unit 32 sends a control signal for displaying the message of mail arrival to the action creation unit 36 (steps D6 and D7).
For example, a small icon-shaped mark is displayed on the screen of the display.

Next, as a fourth embodiment, a plurality of applications will be described.

In the embodiment described here, a plurality of application executing means will be described in detail.

In FIG. 1 showing the basic configuration of the present invention, a plurality of input units 10 (N units) and a plurality of application service providing units 14 are provided.
(M) and output section 18 (L). When there are a plurality of application service providing units 14 according to the configuration of the present invention, a situation occurs in which the plurality of application service providing units 14 use the interpretation results.

For example, the person authentication process is performed by the situation information conversion unit 10c of each input unit 10 in the first embodiment. The person authentication process is a process with a large load on the input unit 10 because there is much reference data. Now, as a new mode, in order to reduce the processing amount of the input unit 10, one of the application service providing units 14 intensively performs this person authentication process. That is, the processing is unified.

There are some advantages to this configuration. The first advantage is that the amount of data required for person authentication processing is enormous, but since this data can be centrally managed, updating and correction are easy. The second advantage is that since the knowledge that "the same person does not exist in a plurality of places at the same time" can be used, the person authentication process can be executed efficiently.

However, there are many application service providing units 14 that require the result of the person authentication process. For example, the person authentication process is indispensable because the information "who is where" is required in both the cooperative work adjusting apparatus of the first embodiment and the electronic mail system of the third embodiment.

However, it is not efficient that the same processing is performed by a plurality of application service providing units 14.

Therefore, it is necessary that the application service providing unit 14 can share the interpretation result of the person authentication processing (in this case, the person authentication processing result).

As in the above example, in the structure of the present invention,
When a plurality of application service providing units 14 are connected, common data and intermediate processing results are generated. In order to improve the overall efficiency, it is necessary to have a mechanism for transferring data between a plurality of application service providing units 14.

In order to create a mechanism for transferring data between a plurality of application service providing sections 14, for example, sharing of the result of person authentication processing will be described as an example, and it can be realized as follows.

As described in the first embodiment, the packet flowing through the status information communication section 12 is an information difference type packet which is transmitted only when the data changes.
It is assumed that the format has the configuration shown in FIG.

The application service providing unit that has performed the person authentication process sets the destination ID to “undefined (all application service providing units 14
Address) ”and transmits the packet to the status information communication unit 12. Since the destination ID of the transferred packet is “indefinite”, the situation information reference unit 30 which is a component of the application service providing unit 14 checks the content ID and if the ID indicates the result of the person authentication process, Take that information and use it as input to your interpreter.

As in the case of person authentication processing, the data that is clearly referred to by a plurality of application service providing units 14 is one application service providing unit 14 from the initial stage of creating the entire system (FIG. 1) of the present invention. It is possible to connect as, or it is possible to take a startup form different from this.

For example, when a new application service providing unit 14 is connected to the entire system of the present invention (FIG. 1), the application service providing unit 14 recursively divides its own processing process and executes a part of the processing. The process can be configured to be independent as another application service providing unit 14. The newly divided application service providing unit 14 is referred to as a “child application service providing unit”, and the parent application service providing unit 14 is referred to as a “parent application service providing unit”. In the case of recursive division, a plurality of "child application service providing units" are activated.

Even if the process is divided, as in the description of the "application service providing section for centrally performing person authentication processing", by exchanging information via the situation information communication section 12, application of parent and child can be achieved. It is possible to perform the same processing as when the services are integrated.

When a plurality of application service providing units 14 that recursively divide are connected to the entire system (FIG. 1), the probability that child application service providing units can be shared increases, and the total processing amount is increased. Can be suppressed.

The parent application service providing section investigates the content ID of the packet flowing through the status information communication section 12 for a predetermined time before separating and dividing its own processing process, and provides the application service of the child to be separated. The application service providing unit 14, which is the same processing process as the unit, may take a procedure for separating the processes after confirming that the application service providing unit 14 does not exist in the entire system.

[0200] Further, for example, since the distribution amount of packets is small, the application service providing unit 1 having the same processing process
Even if a plurality of 4 are started, there is no problem unless the processing results are inconsistent. If the redundant processing process is omitted and the overall efficiency is desired to be improved, each application service providing unit 14 checks the packet transmitted by another application service providing unit 14 and determines the content ID of the packet transmitted by itself. It is sufficient to adopt a configuration in which the application service providing unit 14 on the side that is activated later in time terminates itself when a packet that matches and the transmission ID indicating the transmission source is different from itself is detected. When it is necessary to compare the activation times of the application service providing unit 14, the transmission IDs of the packets transmitted by the application service providing unit 14 may be given serial numbers in the order of activation. By comparing the calling IDs, it is easy to compare the activated times.

[0201]

As described above, according to the present invention, it is possible to surely and easily adjust a time schedule without forcing a participant of a cooperative work to perform a special work.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a cooperative work adjusting device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of a form of a packet transmitted through a situation information communication unit 12 in FIG.

FIG. 3 is a diagram showing a detailed configuration of a situation information conversion unit 10c in FIG.

4 is a diagram showing a detailed configuration of a person detection unit 20 in FIG.

FIG. 5 is a diagram for explaining a processing process by the person detection unit 20.

6 is a diagram showing a detailed configuration of a personal authentication unit 21 in FIG.

7 is a diagram showing a detailed configuration of a status information generation unit 26 in FIG.

8 is a diagram showing an example of a situation registration table (No. 1) in which situation information obtained at the sensor level by the integration unit 26a in FIG. 7 is registered.

9 is a diagram showing an example of a situation registration table (No. 2) in which the situation information of a relatively high level generated by the integrated action recognition unit 26e in FIG. 7 is registered.

FIG. 10 is a block diagram showing a basic configuration of an application service providing unit 14 and a signal flow.

FIG. 11 is a block diagram showing a detailed configuration of an application service providing unit 14.

FIG. 12 is a flowchart for explaining a flow of processing for inputting a request and determining a schedule in the application service providing unit 14.

FIG. 13 is a diagram showing an example of a schedule table for registering a schedule level schedule.

FIG. 14 is a diagram illustrating an image in which a conference holding time is notified when the cooperative work adjusting apparatus according to the present embodiment is used.

FIG. 15 is a flowchart illustrating the flow of a schedule adjustment process according to the first embodiment.

FIG. 16 is a flowchart for explaining the flow of current day time adjustment processing according to the first embodiment.

FIG. 17 is a diagram showing an example of a state table stored in a member information storage unit 30c.

FIG. 18 is a diagram for explaining an evaluation determination value (threshold value T) in the first embodiment.

FIG. 19 is a diagram showing an example of a structured packet used in the second embodiment.

FIG. 20 is a flowchart for explaining the operation of the third embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Input part 10a ... Input device 10b ... Image input part 10c ... Situation information conversion part 12 ... Situation information communication part 14 ... Application service provision part 16 ... Output information communication part 18 ... Output part 20 ... Person detection part 20a ... Difference detection Part 20b ... Face detection part 20c ... Limb detection part 20d ... Score integration part 20e ... People counting part 21 ... Personal authentication part 21a ... Normalization part 21b ... Collation part 21c ... Dictionary storage part 21d ... Evaluation part 22 ... Head direction estimation Part 23 ... Gaze direction estimation part 24 ... Mouth shape motion estimation part 25 ... Arm motion estimation part 26 ... Situation information generation part 26a ... Integration part 26b ... Time generation part 26c ... Location information storage part 26d ... History storage part 26e ... Integrated behavior Recognition unit 26f ... Integrated information storage unit 30 ... Situation information reference unit 30a ... Communication unit 30b ... Member information control unit 30c ... Member information storage unit 32 ... Application Application control unit 32a ... Cooperative work information storage unit 32b ... Schedule management unit 32c ... Current time management unit 32d ... Schedule storage unit 34 ... Application input / output unit 34a ... Request input unit 34b ... Request response unit 34c ... Schedule input unit 36 ... Action Creation unit 36a ... Message creation unit

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Kaoru Suzuki Kaoru Suzuki 1-30-1 Oyonaka, Kita-ku, Osaka-shi, Osaka In-house Toshiba Kansai Branch (72) Eiji Tanaka 1 Oyonaka, Kita-ku, Osaka-shi, Osaka 1-30 No. 1 in Toshiba Kansai Branch Co., Ltd. (72) Inventor Yasuhiro Taniguchi 1-30 No. 1 Odanaka, Kita-ku, Osaka-shi, Osaka In-house Toshiba Kansai Branch

Claims (6)

[Claims] 1. A plurality of information processing devices having image input means for photographing a surrounding situation including a person involved in a collaborative work and inputting an image, and mutual information between the plurality of information processing devices. And a communication unit that transmits and receives, the information processing device, based on the image input by the image input unit, generates situation information representing a situation of a person included in the image,
Status information conversion means for transmitting to the other information processing device via the communication means, and status information generated by the status information conversion means in any of the information processing devices is received via the communication means, A person who can participate in the collaborative work is determined among the persons related to the collaborative work according to the information, a message for the person who can participate in the collaborative work is generated, and the message is provided to any information processing device via the communication means. A collaborative work adjusting device comprising: an applied service providing means. 2. The situation in which the application service providing means receives and accumulates situation information about a person involved in cooperative work from the situation information generated by the situation information conversion means and transmitted via the communication means. Information reference means, request input for cooperative work, application input / output means for outputting a response regarding the adjustment result of cooperative work according to the input request, and according to the request input by the application input / output means, Application control means for adjusting the cooperative work using the status information accumulated in the status information reference means, and a message to a person related to the cooperative work according to the adjustment result of the cooperative work by the application control means. The collaborative work adjustment system according to claim 1, further comprising: an action creating means for notifying. . 3. The application control means stores cooperative work information storage means for storing cooperative work conditions included in a request for cooperative work input by the application input / output means, and stores in the cooperative work information storage means. 3. The collaborative work adjustment according to claim 2, further comprising: same-day time management means for adjusting the time of the collaborative work based on the situation information about the persons involved in the collaborative work based on the established collaborative work conditions. apparatus. 4. The on-the-day time management means, based on the situation information about the parties involved in the collaborative work, a commissioned evaluation value indicating the possibility of accepting participation in the collaborative work for each person, and the commissioned acceptance. 4. The collaborative work adjustment apparatus according to claim 3, wherein the collaborative work adjustment is performed by obtaining a holding evaluation value indicating suitability for holding the collaborative work based on the evaluation value. 5. A process executed by a certain application service providing means is recursively divided among a plurality of the application service providing means, and the divided partial processes are activated in another application service providing means, 2. The cooperative work adjusting device according to claim 1, wherein the processing results of the partial processes are commonly used. 6. An information processing system in which a plurality of information processing devices having image input means are connected to each other through communication means, in the information processing device, a surrounding area including a person involved in cooperative work from the image input means. The situation is photographed, an image is input, situation information representing the situation of a person included in the image is generated based on the input image, and any information processing device is provided via the communication means. The person who can participate in the collaborative work among the persons related to the collaborative work is determined according to the transmitted situation information, and a message is generated for the determined person who can participate in the collaborative work to generate the communication means. A collaborative work adjustment method provided to any information processing device via the.