JP2019101566A - Information processing system, information processing method, information processing program, and information processing apparatus - Google Patents

Abstract

To provide a system capable of confirming user's presence without moving to the seat position.SOLUTION: A presence information management system includes a user terminal and a server. The user terminal is provided with a camera. The user terminal detects a user face based on an input image from the camera, determines whether the user is present, and transmits presence information indicating the determination result to the server. The server stores the presence information transmitted from each user terminal. Further, when the server receives a request for confirmation of the presence information of the user from other user terminal, the server transmits the stored presence information to the other user terminal.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing system, an information processing method, an information processing program, and an information processing apparatus that manage the presence status of a user.

  Conventionally, there is a system for presenting information of absentee. For example, in an information processing system described in Patent Document 1, a person who visits a seat captures an AR marker using a smart device, reads a seat position linked to the AR marker from a database, and a person corresponding to the seat position Determine whether you are present or absent. Then, the person determined to be absent is identified, and the presence information of the identified person is presented.

JP, 2016-192746, A

  However, in the above-mentioned prior art, the user needs to move to the seat position and confirm whether the person corresponding to the seat position is present or not.

  Therefore, an object of the present invention is to provide a system capable of confirming the user's presence without moving to the seat position.

  The present invention adopts the following configuration in order to solve the above-mentioned problems. In addition, parentheses indicate, for example, the correspondence with the examples, and do not limit the scope of the present invention.

  One example of the present invention is an image acquisition unit for acquiring an image from a camera provided around a seat of the user, and a face detection unit for detecting the face of the user based on the image acquired by the image acquisition unit A presence determination means for determining whether or not the user is present based on the detection result of the face detection means; and presence information of the user according to the determination result of the presence determination means. Storage control means to be stored in a server on a network, and presentation means for acquiring the presence information of the user stored in the server via the network and presenting the acquired presence information of the user to other users And.

  According to the above, it is determined by the camera whether or not the user is present, and the user's presence information is stored in the server. Another user can confirm the user's presence information by acquiring the user's presence information from the server via the network.

  In another configuration, the information processing system includes a user terminal provided around the seat of the user, and the image acquisition unit, the face detection unit, and the presence determination unit are disposed in the user terminal. It may be done. The user terminal transmits presence information according to the determination result of the presence determination means to the server via a network. The server stores the presence information transmitted from the user terminal in storage means. The server receives a confirmation request receiving unit that receives a confirmation request for presence information stored in the storage unit from the terminal of the other user, and the presence stored in the storage unit when the confirmation request is received Transmitting means for transmitting information to the other user's terminal via the network. The terminal of the other user presents the presence information transmitted by the transmission means to the other user.

  According to the above configuration, in the user terminal, it is possible to acquire an image from the camera and determine whether the user is present. The user terminal transmits the presence information of the user according to the determination result to the server. Since the image from the camera is not transmitted to the server, the load on the network and the server can be reduced. In addition, the presence information is transmitted from the user terminal and the image from the camera is not transmitted to the server, so that the problem in personal information protection can be avoided.

  In another configuration, the user terminal determines whether or not the user's presence has changed based on the determination result of the presence determination means, and determines that the user's presence has changed. The presence information may be sent to the server when it occurs.

  According to the above configuration, the presence information is transmitted to the server when the user's presence changes. Therefore, the presence information can be prevented from being frequently transmitted to the server, and the load on the network and the server can be reduced.

  In addition, in another configuration, the storage control unit may further store time information according to the determination result of the presence determination unit in the server.

  According to the above configuration, time information is stored in the server in addition to the presence information as to whether the user is present. Therefore, it is possible to grasp the user's presence time and absence time. Note that the time information stored in the server may be time information when it is determined that the user is present, or when it is determined that the user is away, or while the user is present It may be information indicating the time of or the time while away from the seat.

  In another configuration, the presenting unit presents, to the other user, the past presence information of the user based on the user's presence information and the time information stored in the server. It is also good.

  According to the above configuration, the other user who is the administrator of the user can confirm the past presence information of the user, for example, the presence time and the absence time of the user in a predetermined period may be confirmed. it can.

  In another configuration, when the presence determination unit determines that the user is present, the information processing system sleeps the user on the basis of the image acquired by the image acquisition unit. The vehicle may further include a sleep determination unit that determines whether the user is on the road or not. The presentation means may present the other user with drowsiness information indicating the determination result of the drowsiness determination means.

  According to the above configuration, another user can check whether the user is sleeping or not.

  In another configuration, when the presence determination unit determines that the user is present, the information processing system determines a predetermined object based on the image acquired by the image acquisition unit. The information processing apparatus may further include work determining means for detecting and determining whether the user is performing a predetermined work using the predetermined object. The presentation means may present work information indicating the determination result of the work determination means to the other user.

  According to the above configuration, another user can check whether the user is performing a predetermined task using a predetermined object.

  In another configuration, the information processing system may further include a schedule management unit that manages schedule information of the user.

  According to the above configuration, schedule information can be managed in addition to user presence information.

  In another configuration, the information processing system may further include integrated information generation means for generating integrated information in which the schedule information managed by the schedule management means and the presence information are integrated. The presentation means may present the integrated information generated by the integrated information generation means to the other user.

  According to the above configuration, the schedule information and the presence information can be integrated, and the user's schedule and the actual presence information can be presented to other users.

  In another mode of the present invention, an information processing method performed in the above information processing system may be used. Further, the information processing program may be an information processing program to function as each means of the information processing system, or may be an information processing apparatus provided with each of the means.

  According to the present invention, whether or not a user is present can be determined by the camera, and the user's presence information can be stored in the server. Another user can confirm the user's presence information by acquiring the user's presence information from the server via the network.

The figure which shows an example of a structure of the presence information management system 1 of this embodiment The figure which shows an example of the hardware constitutions of the user terminal 20 A diagram showing an example of the logical configuration of the user terminal 20 according to the first embodiment A diagram showing an example of the logical configuration of the server 10 A diagram showing an example of the logical configuration of the user terminal 20 according to the second embodiment A diagram showing an example of information stored in the database 13 A diagram schematically representing the information shown in FIG. 6 stored in the database 13 The figure which shows an example of the image displayed on the display part of a terminal when the user who is an administrator accesses the server 10. The figure which shows an example of the image displayed on the display part of the user terminal 20, when the user (general user) who is not an administrator accesses the server 10. A diagram showing an example of data stored in the user terminal 20 Flow chart showing an example of processing performed by the user terminal 20 Flow chart showing an example of processing performed by the server 10 A diagram showing an example of a logical configuration of the server 10 in another embodiment The figure which shows an example of cooperation with the schedule management part 15 and the presence status management part 11.

(Overview of Presence Information Management System)
Hereinafter, the presence information management system 1 according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of the presence information management system 1 of the present embodiment. The presence information management system 1 monitors the presence status, working attitude, etc. of users (hereinafter referred to as “users”) such as employees belonging to companies, countries, local governments, other organizations, national government officials, local government officials etc. It is a system to manage.

  As shown in FIG. 1, the presence information management system 1 includes a server 10 and a plurality of user terminals 20. The server 10 and the plurality of user terminals 20 are connected to a network (for example, an in-house LAN, a WAN, the Internet, etc.) in a wired or wireless manner.

  The user terminal 20 is a terminal used by each user, and is installed, for example, at the desk of each user. The user terminal 20 may be, for example, a stationary personal computer, a notebook computer, a tablet terminal, or the like. In the user terminal 20, application programs such as document creation / editing software, spreadsheet software, web browser software, electronic mail software and the like used by the user for normal operations are installed. In addition to these programs, a program for determining whether or not the user is present is installed in the user terminal 20.

  Hereinafter, for example, the user terminal 20 used by the user A will be referred to as “user terminal 20 a”, and the user terminal 20 used by the user B as “user terminal 20 b”. In the case of representing the user terminal 20 corresponding to a certain user, the "user terminal 20" is written with a lower case letter of the alphabet. When a plurality of user terminals are represented collectively, it is written as "user terminal 20".

  The user terminal 20 determines whether a user who uses the terminal is present (seated in a seat), and transmits presence information according to the determination result to the server 10. Specifically, a camera 22 is installed around the seat of the user. The camera 22 may be installed on the screen of the display unit of the user terminal 20, for example. The camera 22 may be built in the user terminal 20 or may be externally attached to the user terminal 20. The user terminal 20 acquires a real-time image captured by the camera 22, detects the face of the user based on the image, and determines whether the user is present based on the detection result. Then, the user terminal 20 transmits presence information indicating whether the user is present to the server 10 via the network.

  The server 10 receives the presence information transmitted from the user terminal 20, and stores the presence information in the database together with the time information. In the server 10, the presence information of each user (information as to whether each user is at a seat) transmitted from each user terminal 20 is stored together with time information.

  Each user can access the server 10 using, for example, the web browser of the user terminal 20 used by the user, and confirm the presence information of other users. For example, when the seat of the user A and the seat of the user B are physically separated, the user A can not visually check whether the user B is currently present. In such a case, the user A can check whether the user B is present or not by accessing the server 10.

  It should be noted that all users may be able to confirm the presence information of all other users in real time by accessing the server 10 using their own user terminal 20, or presence for each user The authority to refer to the information may be different. For example, only the administrator has the authority to confirm the presence information of each user, and each administrator accesses the server 10 using the user terminal 20 (or other dedicated management terminal) of the administrator. It may be possible to confirm the presence information of the user in real time. Also, for example, while the superior of a certain department can confirm the presence information of the user (subordinate) who belongs to that department, it may be configured not to confirm the presence information of the user who belongs to another department. In addition, personnel personnel may be able to confirm presence information of all users. In addition, the authority to access the presence information may be suppressed for each user.

  For example, a user who is not an administrator (general user) may have the right to confirm the other user's current presence information, but may not have the right to confirm the other user's past presence information . On the other hand, the manager (for example, the head of a certain organization or a person in charge of personnel management) has the authority to confirm the present presence information of the user to be managed (for example, subordinates) and confirms the past presence information of the user May have the right to

(Configuration of user terminal and server)
Next, the configuration of the user terminal 20 will be described. FIG. 2 is a diagram showing an example of the hardware configuration of the user terminal 20. As shown in FIG.

  As shown in FIG. 2, the user terminal 20 includes an information processing unit 21, a camera 22, an input unit 23, a storage unit 24, a communication unit 25, and a display unit 26.

  The information processing unit 21 has a CPU and a memory. The information processing unit 21 acquires an image from the camera 22 and executes an information processing program (an image processing program described later) to perform image processing. The information processing unit 21 functions as a face detection unit 27 and a presence determination unit 28 (see FIG. 3) described later by executing an image recognition program.

  Further, the information processing unit 21 can execute the application program other than the image processing program. In addition, the information processing unit 21 performs information processing based on a signal from the input unit 23 and causes the display unit 26 to display an image or a character according to the result. The information processing unit 21 also reads a program or data stored in the storage unit 24 or writes data in the storage unit 24. Further, the information processing unit 21 controls the communication unit 25.

  The camera 22 captures an image at a predetermined time interval (for example, 1/60 second interval). The captured image is input to the information processing unit 21. The camera 22 may be incorporated in the user terminal 20 or may be externally attached to the user terminal 20. The camera 22 is provided at a position where it is easy to capture the face of the user when the user is seated. For example, the camera 22 is provided at a position (e.g., on the screen) at which the user's face can be imaged from the front when the user is looking at the screen of the display unit 26. For example, when the main body of the user terminal 20 (the terminal main body including the information processing unit 21) and the display unit 26 are configured to be separable, the camera 22 is externally installed on the screen of the display unit 26. May be For example, the camera 22 may be a web camera (a camera accessible from another terminal via the web). When the main body of the user terminal 20 and the display unit 26 are integrally configured, the camera 22 may be integrally provided on the same surface as the screen of the display unit 26. Note that the installation position of the camera 22 is merely an example, and the camera 22 may be provided at any position as long as the user who has seated can be imaged.

  The input unit 23 is a keyboard, a mouse or the like, and receives an operation of the user.

  The storage unit 24 is a storage device that stores the image processing program and data of the present embodiment, and may be, for example, a hard disk or a non-volatile memory. Further, the storage unit 24 stores data for performing presence determination as to whether or not the user is present. The storage unit 24 also stores, in addition to the image processing program, an application program and the like used by the user in normal business operations.

  The communication unit 25 is an interface for connecting the user terminal 20 to a network (for example, an in-house LAN). The communication unit 25 transmits data to the server 10 and the Internet, and receives data from the server 10 and the Internet.

(Logical configuration of user terminal)
Next, the logical configuration of the user terminal 20 according to the first embodiment will be described. FIG. 3 is a diagram showing an example of the logical configuration of the user terminal 20 according to the first embodiment. As shown in FIG. 3, the user terminal 20 includes a face detection unit 27 and a presence determination unit 28. The image recognition program stored in the storage unit 24 causes the information processing unit 21 to function as the face detection unit 27 and the presence determination unit 28.

  The face detection unit 27 acquires an image (input image) transmitted from the camera 22 at predetermined time intervals, and detects a human face image from the input image. Specifically, the face detection unit 27 detects a face area of a person without specifying the user (that is, without authenticating the user). As a program for detecting a face, a known commercial program or open source program may be used.

  When the face detection unit 27 detects a human face, the presence determination unit 28 determines whether the user is present. For example, the face detection unit 27 may detect a face of a person who has passed behind a seat from among input images transmitted from a camera 22 provided around the seat. The presence determination unit 28 does not determine that the user is present when only a person passes as described above. For example, the presence determination unit 28 may determine that the user is present when the face detection unit 27 detects a human face for a predetermined period or more (for example, 10 seconds or more). The presence determination unit 28 may also determine that the user is present when the size of the face image detected by the face detection unit 27 is equal to or larger than a predetermined threshold. Further, the presence determination unit 28 may determine that the user is present when a face image is detected in a predetermined area (for example, the central portion of the image) in the input image. Further, the presence determination unit 28 detects eyes, a nose, a mouth, an ear, and the like of the user to improve the determination accuracy, and determines whether the user is present based on the detection results. You may For example, the presence determination unit 28 determines the direction of the face from detection results of eyes, nose, mouth, ears, etc., and determines whether the user is present based on the direction or movement of the face. Good. Further, after the presence determination unit 28 determines that the user is present, even when the user's face is not detected, the user is present based on the movement of the face immediately before that, etc. It may be determined that For example, when the camera 22 shoots the user's face from the front when the user faces the front, the face detection unit 27 detects the face of the user when the user faces sideways, faces backward, or faces down. Stop detecting faces. Even in such a case, the presence determination unit 28 may detect part of the head or body of the user and determine that the user continues to be present. In addition to the face of the user, the presence determination unit 28 may determine that the user is away when the user's head is not detected. Thus, the presence determination unit 28 detects the entire face of the user, the face, such as eyes, nose, and mouth, ears, head, etc., and comprehensively determines the user's action based on the detection results. Alternatively, it may be determined whether the user is present.

  The presence determination unit 28 determines whether a person is present or not (specifically, whether or not a person is seated at a seat except when a person passes) without specifying the person. May be That is, the presence determination unit 28 may not specify the person who should be in the seat by face authentication. For example, the presence determination unit 28 may determine that the user A is present even if the user B is seated at the seat of the user A.

  Basically, when each user performs work while sitting on a predetermined fixed seat, there is a low possibility that user A will perform work while sitting for another user B's seat for a predetermined time or more. In such a case, if it is confirmed that a person is seated in the seat of the user A, the user A can be checked to some extent without checking whether the person sitting in the seat of the user A is really the user A. It is possible to understand the presence situation of In addition, even when the presence determination unit 28 does not specify the user by face authentication, the user authentication may be performed by another authentication unit. For example, if the presence determination unit 28 detects that a person is seated in the seat of the user A and the user logged in to the user terminal 20 a is the user A, the presence determination unit 28 The user A may be determined to be present.

  As described above, when the presence determination unit 28 does not specify the user, it is not necessary to register face data indicating the features of the face of each user in advance, and thus no problem in personal information protection occurs.

  In another embodiment, the presence determination unit 28 performs face authentication based on the face image included in the input image from the camera 22 and the face data indicating the feature of the face of each user stored in advance. The person may be identified. Specifically, the presence determination unit 28 detects eyes, a nose, a mouth, an ear, and the like from an input image, and uses features based on their shapes, sizes, positional relationships, and the like, and face data stored in advance. Identify the user based on it. In this case, the presence determination unit 28 determines that the user A is present, for example, only when the user A is seated at the seat of the user A.

  For example, if each user does not have a fixed seat (in the case of a so-called free address), it is necessary to specify who is sitting in the seat. In such a case, the presence determination unit 28 identifies the face of each user and determines who is sitting in the seat. Then, the seating determination unit 28 transmits, to the server 10, seating information of the user who is sitting at the imaging target seat. Thereby, even when each user does not have a fixed seat, each user can confirm which user is sitting at which seat by accessing the server 10. Even when each user does not have a fixed seat, the presence determination unit 28 does not have to specify a person by face authentication. In this case, the presence determination unit 28 may specify the present user based on, for example, the user name of the logged-in user.

  The presence determination unit 28 transmits presence information (information indicating whether or not there is a presence, a presence flag described later) to the server 10 via the communication unit 25 based on the result of the presence determination. . Specifically, the presence determination unit 28 changes the presence state of the user, that is, changes from the presence state to the absence state, or the absence state. In the case of a change to the present state, the presence information is transmitted to the server 10 together with the time information.

  The detection of the user's face by the face detection unit 27 and the presence determination of the user by the presence determination unit 28 are realized by the developer presetting and programming rules for detection and determination. It may be realized by causing a computer to read a large amount of data and performing machine learning (including deep learning).

  The communication unit 25 transmits the presence information from the presence determination unit 28 to the server 10 together with time information.

(Logical configuration of server)
FIG. 4 is a diagram showing an example of the logical configuration of the server 10. As shown in FIG. The server 10 includes at least one CPU that executes a predetermined program, a memory, a storage device, and a communication unit for connecting to a network. The predetermined program causes the CPU of the server 10 to function as each unit shown in FIG.

  As shown in FIG. 4, the server 10 includes a presence information acquisition unit 11, a presence status management unit 12, a database 13, and an output unit 14.

  The seating information acquisition unit 11 acquires the seating information and the time information transmitted from each user terminal 20, and outputs the acquired information to the seating condition management unit 12.

  The presence status management unit 12 stores the presence information and the time information acquired by the presence information acquisition unit 11 in the database 13. When the presence information acquisition unit 11 acquires the presence information from the user terminal 20, the presence status management unit 12 uses the clock means (RTC) on the server 10 when the presence information is acquired. Time information to be clocked may be stored in the database 13 together with the acquired presence information.

  Further, the presence status management unit 12 reads the presence information stored in the database 13 and outputs information corresponding to the read presence information to the output unit 14. For example, when there is a request for obtaining presence information of a specific user from the user terminal 20, the presence status management unit 12 accesses the database 13 to obtain presence information and time information of the specific user, Output to the output unit 14.

  The output unit 14 outputs the information from the presence status management unit 12 to the outside (for example, the user terminal 20 or another external device). For example, the output unit 14 may transmit the presence information of a specific user to the user terminal 20 via the network, or may transmit the presence information of a plurality of users to the user terminal 20.

  In addition, the output unit 14 is, for example, statistical data of presence information of one or more users in a predetermined period (for example, one day, one month, half a year, one year, etc.) in response to a request from the user terminal 20 May be output as a report. In this case, the output unit 14 may output statistical data in a predetermined period as an image or may output as text. For example, the output unit may output, as a graph such as a bar graph or a pie graph, the sum of the user's sitting times in a predetermined period and the sum of the leaving time. For example, for a certain user, one month's presence and absence times may be output. In addition, the output unit 14 may output the ratio of the occupancy time and the absence time of one or more users in a predetermined period.

  As described above, in the present embodiment, the user terminal 20 detects the face of the user using the camera 22 installed around the seat of the user, and performs presence determination as to whether the user is present or not. . Presence information indicating the result of presence determination by the user terminal 20 is transmitted to the server 10 and stored in the server 10. Each user can access the server 10 to check the presence information of other users in real time.

  In another embodiment, the user terminal 20 may determine another state of the user in addition to the presence determination as to whether or not the user is present. For example, the user terminal 20 may make a doze determination as to whether or not the user is asleep.

  FIG. 5 is a diagram showing an example of the logical configuration of the user terminal 20 according to the second embodiment. As shown in FIG. 5, the user terminal 20 includes a sleep detection unit 29 in addition to the face detection unit 27 and the presence determination unit 28. The user terminal 20 also includes an object detection unit 30. The image recognition program stored in the storage unit 24 causes the information processing unit 21 to function as the face detection unit 27, the presence determination unit 28, the drowsiness detection unit 29, and the object detection unit 30.

  When the presence determination unit 28 determines that the user is present, the doze detection unit 29 determines whether the user is falling asleep based on the face image included in the input image from the camera 22. Make a nap decision. For example, the drowsiness detection unit 29 detects the user's eyes from the face image, and determines the user's drowsiness based on whether or not the eyes are open, the time the eyes are closed, the blink time interval, these patterns, etc. I do. In addition to the open / close state of the eyes and the movement of the eyes, the doze detection unit 29 performs a doze by comprehensively determining the user's actions such as head movement and tilt, hand position and movement, user posture, and the like. The determination may be made. For example, by capturing an image of the user sleeping with the camera 22 and causing the user terminal 20 to learn, it is possible for the sleep detection unit 29 to determine whether or not the user is sleeping. In addition, the drowsy detection unit 29 may determine that the user is falling asleep even after the face detection unit 27 no longer detects the user's face. For example, when the user falls asleep with his face turned down, the face detection unit 27 may not detect the user's face. Even in such a case, the presence determination unit 28 determines that the user continues to be present, and the drowsy detection unit 29 determines whether the user is falling asleep. For example, the drowsiness detection unit 29 may detect the head of the user and may determine whether the user is snoozing based on the inclination of the head.

  The doze detection unit 29 transmits doze information (a doze flag to be described later) indicating whether or not the user is doze on the basis of the result of the doze determination to the server 10 via the communication unit 25. Specifically, the drowsiness detection unit 29 changes the drowsy state of the user (when the drowsy state is changed to the drowsy state, or from the drowsy state to the drowsy state). (If it changes), the drowsiness information is sent to the server 10 together with the time information.

  Further, when the presence determination unit 28 determines that the user is present, the object detection unit 30 detects a predetermined object based on the input image from the camera 22. For example, the object detection unit 30 may detect a telephone from the input image from the camera 22 and determine whether the user is calling. In addition to the telephone, the object detection unit 30 detects the user's eyes, nose, mouth, ears, hands (one who holds the phone and / or one who does not hold the phone). The user comprehensively determines the user's behavior such as movement, positional relationship between the telephone and face (for example, mouth and ear), positional relationship between the telephone and hand, eye and mouth movement, hand movement, etc. It may be determined whether or not it is present. For example, the object detection unit 30 can determine whether the user is calling by imaging the situation where the user is calling by using the camera 22 and making the user terminal 20 learn. The object detection unit 30 transmits the determination result as to whether the user is calling or not to the server 10 through the communication unit 25. At this time, time information (the time when the call started and the time when the call ended) is also transmitted to the server 10. The server 10 stores in the database 13 information on whether the user is calling and the time information thereof. By accessing the server 10, a user (for example, an administrator) can grasp the time during which another user is calling.

  Further, the object detection unit 30 may detect not only the telephone but also other objects, and determine whether the user is performing a predetermined work using the object. The object detection unit 30 detects a predetermined object, the user's eyes, nose, mouth, head, hands, etc., and comprehensively determines the user's action based on the movement of these, the positional relationship between them, etc. It may be determined whether or not a predetermined task is being performed. The object detection unit 30 transmits, to the server 10 via the communication unit 25, the determination result as to whether or not the user is performing a predetermined task, and the time information thereof. The server 10 stores these pieces of information in the database 13.

  Note that the doze determination by the doze detection unit 29 may be realized by the developer predetermining and programming a rule for the determination, or realized by causing the computer to read a large amount of data to perform machine learning. May be Further, the detection of a predetermined object (for example, a telephone) by the object detection unit 30 and the determination of whether or not the user is performing a predetermined operation using the predetermined object, the developer defines a rule for the determination in advance. It may be realized by programming, or may be realized by causing a computer to read a large amount of data and machine learning. For example, the user may perform machine learning by capturing an image of the user performing a predetermined task using a predetermined object and causing the user terminal 20 to read the image.

  FIG. 6 is a diagram showing an example of information stored in the database 13. FIG. 7 is a diagram schematically representing the information shown in FIG. 6 stored in the database 13.

  As shown in FIG. 6, the database 13 stores a user ID, an attendance time, an attendance time, a nap time, and a telephone time as a predetermined work time. For example, when it is determined that the user A is imaged by the camera 22 installed around the seat and the presence determination unit 28 determines that the user A is present, the information indicating that the user A is present is the user terminal of the user A 20a to the server 10.

  For example, when the presence determination unit 28 of the user terminal 20a determines that the user A is present at time 8:45, the presence information (information indicating that the user A is present) is The time information is sent to the server 10 together with the time information and stored in the database 13. Next, when the user A is away at 12:00, the presence determination unit 28 of the user terminal 20a determines that the user A is away, and the presence information (the user A is away Information indicating the presence or absence and time information are transmitted to the server 10 and stored in the database 13. Thereafter, when the user A is seated at 13:00, the presence determination unit 28 of the user terminal 20a determines that the user A is present, and the presence information and the time information are transmitted to the server 10 , Stored in the database 13. Thereafter, when the user A is away at 17:30, the presence determination unit 28 of the user terminal 20a determines that the user A is away, and the presence information and the time information are sent to the server 10 It is sent and stored in the database 13.

  Similarly, with respect to other users, when the user is seated or is away from the user, presence information indicating that the user is present or absent from the user terminal 20 corresponding to the user is the time information Together with the server 10. Then, the information is stored in the database 13 of the server 10.

  For example, as shown in FIG. 6 and FIG. 7 (2), the user B is present from time 8:58 to time 9:59 and from time 9:59 to 10:04 for a break. When leaving the seat and sitting again at the time 10:04, the database 13 stores the time 8:58 and the time 10:04 as the presence time, and stores the time 9:59 as the leaving time. As described above, when the user B shifts from the state of being away to the state of being present, the time at that time (the time measured by the clocking means of the user terminal 20b of the user B, or the server The clocked time by the clocking means 10 is stored in the database 13 as the sitting time. When the user B shifts from being in the presence to being away from the seat, the time at that time is stored in the database 13 as the leaving time.

  Thus, by storing the attendance time and the absence time in the database 13, as shown in FIG. 7, each user is present from what time to what time, and from which time to what time It is possible to easily grasp the attitude of each user.

  For example, as shown in FIG. 7 (1), it can be seen that the user A was performing work without leaving from work attendance time (8:45) until 12:00. In addition, it can be seen that the user A performs work without taking a seat from 13:00 when the break is finished to 17:30 as the lunch break with 12:00 to 13:00 as a lunch break.

  On the other hand, as shown in FIG. 7 (2), it can be seen that the user B is away from the seat for about 5 minutes to about 10 minutes every hour. It can be understood that the user B repeats a short break approximately every hour, and the superior of the user B or the personnel in charge of personnel can refer to these pieces of information when evaluating the working attitude of the user B .

  Also, as shown in FIG. 6 and FIG. 7 (3), the user C was present from 9:55 to 9:55 and was away from 9:55 to 13:00 I understand that. Thereafter, the user C is present from 13:00 to 13:30, is absent from 13:30 to 14:30, and is present from 14:30 to 15:00, and 15 It can be seen that he was away from 00:00 to 16:30. From these pieces of information, it can be inferred that the user B has often been away for a relatively long time on this day, and the user B has held a meeting in a meeting room or the like, or has gone out.

  In addition, when it is determined by the drowsy detection unit 29 of the user terminal 20 that the user is drowsy, drowsy information and time information indicating that the user is drowsy are transmitted from the user terminal 20 to the server 10 and are stored in the database 13 It is memorized. In addition, when the object detection unit 30 determines that the user is performing a predetermined operation using a predetermined object (for example, when it is determined that a user is calling), the predetermined operation is being performed. And time information are transmitted from the user terminal 20 to the server 10.

  For example, at time 11:00, when it is determined by the doze detection unit 29 of the user terminal 20d corresponding to the user D that the user D is dozing, the user terminal 20d is the doze information (the user D is dozing And time information are sent to the server 10. Next, at time 11:30, when it is determined that the user D does not sleep by the drowsiness detection unit 29 of the user terminal 20 d, the user terminal 20 d is drowsy information (information indicating that the user D does not sleep asleep And time information to the server 10. In this case, as shown in FIG. 6, the database 13 of the server 10 stores the time when the user D started snoozing and the time when the snoozing ended.

  In addition, when it is determined that the user D is calling by the object detection unit 30 of the user terminal 20 d at time 13:30, information indicating that the user is calling and time information are transmitted to the server 10 Ru. Further, when it is determined by the object detection unit 30 that the user D is not calling at time 14:00, information indicating that the user D is not calling and time information are transmitted to the server 10. In this case, as shown in FIG. 6, in the database 13 of the server 10, the time when the user D starts the call and the time when the call is ended are stored.

  The user terminal 20 may store the image captured by the camera 22 when the user is asleep. Specifically, when the drowsiness detection unit 29 detects no drowsiness of the user, the user terminal 20 does not store the input image from the camera 22 in the user terminal 20, and based on the input image, Seat determination and doze determination are made. The user terminal 20 may store the input image from the camera 22 in the user terminal 20 when the user does not feel sleepy by the doze detection unit 29. In addition, when the user does not feel sleepy by the doze detection unit 29, the user terminal 20 transmits the input image from the camera 22 to another device on the network (for example, a storage device for image storage or the server 10). The other device may store the input image.

  Similarly, the user terminal 20 may store an input image from the camera 22 while the user is performing a predetermined task using a predetermined object. Specifically, when a predetermined object is detected by the object detection unit 30 and it is determined that the user is performing a predetermined operation using the predetermined object, the user terminal 20 receives an input from the camera 22. When it is determined that the image is stored in the storage unit 24 and it is determined that the user has not performed a predetermined operation, the input image may not be stored in the storage unit 24. In addition, the user terminal 20 may store an input image from the camera 22 in another device on the network while the user is performing a predetermined work.

  As described above, the drowsiness detection unit 29 detects drowsiness of the user and stores information indicating that drowsy in the database 13, whereby the administrator (supervisor of each user or person in charge of personnel) works for each user These information can be used as a reference when evaluating attitudes.

  For example, when the administrator accesses the server 10 using the user terminal 20 (or a dedicated management terminal) and refers to the user's presence information, in addition to the user's presence information, drowsy information or a telephone call is made. , May be displayed on the display unit of the user terminal 20.

  FIG. 8 is a view showing an example of an image displayed on the display unit of the terminal when a user who is an administrator accesses the server 10.

  As shown in FIG. 8, on the left side of the screen of the administrator, a list 41 for selecting a user is displayed. For example, when the user A is selected, the user A is highlighted using a cursor or the like. On the right side of the administrator's screen, the presence information of the selected user is displayed. For example, a present time 42, a present state 43 indicating whether present or absent, and a camera button 44 are displayed. Further, on the right side of the screen, a graph 45 may be displayed that indicates the present state of the user. As shown in FIG. 8, in the graph 45, the time when the user A was present, the time when he was away, the time when he was asleep, and the time when he was on the phone are distinguishably displayed. Be done. In addition, when the user clicks the camera button 44 using a mouse or the like, the real time image 46 from the camera 22 installed around the user terminal 20 of the selected user is displayed at the lower right of the screen .

  In addition, when a user who is not an administrator (general user) accesses the server 10 using the user terminal 20 and refers to the user's presence information, the user's presence information is displayed, and the drowsy information or telephone call is made. Information indicating whether or not to be displayed may not be displayed.

  FIG. 9 is a view showing an example of an image displayed on the display unit of the user terminal 20 when a user (general user) who is not the administrator accesses the server 10.

  As shown in FIG. 9, a list 41, a current time 42, a presence state 43, and a camera button 44 are displayed on the screen of the user who is not the administrator (general user). When the user clicks the camera button 44 using a mouse or the like, the real time image 46 from the camera 22 installed around the user terminal 20 of the selected user is displayed at the lower right of the screen. On the other hand, the graph 45 shown in FIG. 8 is not displayed on the screen of the general user. As shown in FIG. 8, the graph 45 includes information as to whether the user is asleep or not, and information as to whether or not the user is calling. Thus, information related to personnel evaluation, such as information on whether or not to fall asleep, may not be presented to a non-administrative user. In addition, the real-time video 46 from the camera 22 may not be displayed on the screen of the general user.

  8 and 9 are merely examples, and the information presented to the user when the user accesses the server 10 is not limited to this. For example, a user who is a manager, in addition to the presence state 43 of the user who is his subordinate, the graph 45, the real time image 46, etc., information accumulated for a predetermined period (for example, the past one month, half a year, one year) You may be able to see statistical data (total attendance time, total absence time, total asleep time, and their ratio etc.) based on these information in a predetermined period of time. .

(Details of user terminal and server processing)
Next, processing performed in the user terminal 20 and the server 10 of the presence information management system 1 will be described. In the following, an example in which the user terminal 20 includes the presence determination unit 28 and the sleep detection unit 29 will be described. First, data stored in the memory of the user terminal 20 will be described.

  FIG. 10 is a diagram showing an example of data stored in the user terminal 20. As shown in FIG. As shown in FIG. 10, an image recognition program 100 is stored in the memory of the user terminal 20. The image recognition program 100 may include a program for machine learning. In addition, a part of the image recognition program 100 may be an open source program. The image recognition program 100 includes a program for performing the above-described presence determination and dozing determination. The image recognition program 100 causes the information processing unit 21 to function as the face detection unit 27, the presence determination unit 28, and the sleep detection unit 29.

  Further, in the user terminal 20, the determination data 101, the image data 102, and the state information are stored. The state information includes an attendance flag 103 and a nap flag 104.

  The determination data 101 includes data for detecting a human face, data for performing presence determination of whether or not the user is present, and determination of whether or not the user is falling asleep Contains the data of These data may be prepared in advance by a developer or may be generated by machine learning (including deep learning).

  The image data 102 is data indicating an image transmitted from the camera 22 at predetermined time intervals.

  The presence flag 103 is a flag indicating whether or not the user is present (whether the user is away). When the user is present, the presence flag 103 is set to ON, and when the user is away, the presence flag 103 is set to OFF.

  The nap flag 104 is a flag indicating whether the user is sleeping or not. When the user sleeps, the sleep flag 104 is set to ON, and when the user does not sleep, the sleep flag 104 is set to OFF.

(Details of user terminal processing)
FIG. 11 is a flowchart illustrating an example of processing performed by the user terminal 20. The process illustrated in FIG. 11 is performed by the information processing unit 21 of the user terminal 20 executing the image recognition program 100. The process of step S100 to step S112 shown in FIG. 11 is repeatedly performed at predetermined time intervals. Note that the processing of step S100 to step S112 may be performed at the same interval as the time interval at which the camera 22 captures an image, or from the time interval (eg, 1/60 second interval) at which the camera 22 captures an image Also, it may be performed at long intervals (for example, one second intervals).

  The process shown in FIG. 11 is started when the power of the user terminal 20 is turned on, and is always executed in the background even when the user terminal 20 is executing an application program used for normal work. . Note that the user using the user terminal 20 may not notice that the process shown in FIG. 11 is being performed. For example, even while the process shown in FIG. 11 is being performed, a window or an icon indicating that the process is being performed may not be displayed on the screen of the user terminal 20. Conversely, an icon or the like indicating that the process shown in FIG. 11 is being executed may be displayed on the screen of the user terminal 20 so that the user notices that the process shown in FIG. 11 is being executed. Further, the process illustrated in FIG. 11 may be configured such that each user can not finish the process without permission, or may be allowed to finish by an instruction of each user.

  As illustrated in FIG. 11, the user terminal 20 acquires an input image captured by the camera 22 (step S100). Next, the user terminal 20 performs face detection processing based on the acquired input image (step S101). Specifically, the user terminal 20 detects a human face image from among the acquired input images, and determines a face area. Here, the user terminal 20 determines whether or not there is a human face without specifying the user corresponding to the user terminal 20.

  Subsequently, the user terminal 20 determines whether or not a human face is detected in the face detection process of step S101 (step S102), and when a face is detected (step S102: YES), next in step S103. Do the processing. On the other hand, when the user terminal 20 does not detect a human face in the face detection process of step S101 (step S102: NO), the process of step S109 is performed next. Even when the user terminal 20 does not detect the user's face in the face detection process, if the presence flag is set to ON, the user terminal 20 determines YES in step S102, and then the process of step S103. I do. That is, when the user terminal 20 determines that the user is present in the presence determination process of step S103 in the previous processing loop, the user terminal 20 determines YES in step S102, and then executes the process of step S103.

  In step S103, the user terminal 20 performs presence determination processing. Specifically, in step S103, the user terminal 20 determines the size of the face area detected in the face detection process, the movement of the face area, the number of face areas, whether the face is detected for a predetermined time or more, etc. Presence determination of whether the user is present or not. For example, when the number of detected face areas is one and the size of the face area is equal to or larger than a predetermined size, the user terminal 20 may determine that the user is present. Further, when the face of the user is detected for a predetermined time or more, the user terminal 20 may determine that the user is present. In addition, the user terminal 20 may detect eyes, nose, mouth, ears, head, and the like, and determine whether the user is present based on the detection results. The user terminal 20 may accumulate data for the presence determination by performing machine learning. Also, the user terminal 20 may determine whether the detected face is the user corresponding to the user terminal 20 or not. For example, face data indicating the feature of the user's own face is stored in the user terminal 20, and it is determined based on the face data and the face image included in the input image from the camera 22 whether or not the user is the same. It is also good.

  When it is determined in the presence determination process of step S103 that the user is present (step S104: YES), the user terminal 20 next executes the process of step S105. On the other hand, when it is determined in the presence determination process of step S103 that the user is not present (step S104: NO), the user terminal 20 next executes the process of step S109.

  In step S105, the user terminal 20 sets the presence flag to ON. Next, the user terminal 20 performs doze determination processing (step S106). For example, in step S106, the user terminal 20 detects the user's eyes, and based on whether or not the eyes are closed for a predetermined time or more, the number and frequency of blinks, eye movements, patterns of these, etc. Make a decision. In addition, the user terminal 20 may detect eyes, nose, mouth, ears, head and the like, and based on the detection results, may determine whether or not the user is asleep. Note that the user terminal 20 may accumulate data for determination of sleepiness by performing machine learning.

  When it is determined that the user is dozing in the doze determination process of step S106 (step S107: YES), the user terminal 20 sets a doze flag to ON (step S108). On the other hand, when it is determined that the user is not dozing in the dozing determination process of step S106 (step S107: NO), the user terminal 20 next executes the process of step S110.

  On the other hand, when it is determined NO in step S102 or step S104, the user terminal 20 sets the presence flag and the drowsy flag to OFF (step S109).

  When step S108 is performed, step S109 is performed, or when it is determined NO in step S107, the user terminal 20 determines whether the state information of the user has changed (step S110). Here, the state information is a presence flag or a doze flag. Specifically, the user terminal 20 stores the presence flag and the doze flag before the presence flag and the doze flag are changed in step S105, step S108 or step S109 in the present processing loop. . If at least one of the presence flag and the doze flag changes in the processing loop this time, the user terminal 20 determines that the state information of the user has changed in step S110.

  When it is determined that the state information has changed (step S110: YES), the user terminal 20 acquires time information indicating the current time from the clocking unit (for example, RTC) of the own device (step S111). Then, the user terminal 20 transmits the state information (the presence flag or the drowsy flag) and the time information to the server 10 (step S112).

  When it determines with NO by step S110, or when performing step S112, the user terminal 20 performs the process of step S100 again. This is the end of the description of the process performed in the user terminal 20.

(Details of server processing)
FIG. 12 is a flowchart showing an example of processing performed by the server 10. The process shown in FIG. 12 is performed by the CPU of the server 10 executing a predetermined program. The server 10 repeatedly executes the process shown in FIG. 12 at predetermined time intervals.

  As illustrated in FIG. 12, the server 10 determines whether or not the state information (the presence flag or the doze flag) is acquired from the user terminal 20 (step S201). When the state information is acquired from the user terminal 20 (step S201: YES), the server 10 stores the acquired state information in the database 13 (step S202).

  Specifically, the server 10 specifies the user terminal 20 that has transmitted the state information, stores the state information and the time information in the database 13 in association with the user ID of the user corresponding to the user terminal 20. For example, when the server 10 acquires state information and time information from the user terminal 20a, the server 10 updates the information of the user A in the database 13. For example, when status information in which the presence flag is set to ON is obtained from the user terminal 20a, the server 10 stores the time information obtained from the user terminal 20a at the user A's presence time (see FIG. 6). . Further, for example, when the state information in which the presence flag is set to OFF is acquired from the user terminal 20a, the server 10 acquires the time information acquired from the user terminal 20a at the user A's absence time (see FIG. 6). Remember. Further, when the state information in which the nap flag is set to ON is acquired from the user terminal 20a, the server 10 stores the time information acquired from the user terminal 20a during the nap time of the user A (see FIG. 6).

  When step S202 is executed, or when it is determined NO in step S201, the server 10 determines whether or not the presence confirmation request for confirming the presence of the user is received from the user terminal 20. (Step S203). The presence confirmation request from the user terminal 20 includes the user ID of the target user whose presence is to be confirmed.

  When the presence confirmation request is received (step S203: YES), the server 10 accesses the database 13 and acquires the presence information of the user included in the presence confirmation request (step S204). Then, the server 10 transmits the information acquired from the database 13 to the user terminal 20 that has transmitted the presence confirmation request (step S205).

  For example, when the administrator requests confirmation of the presence information of the user A, the server 10 accesses the database 13 to acquire information on the presence time, absence time, sleep time, and telephone time of the user A. Do. Then, the server 10 transmits the information of the user A acquired from the database 13 to the user terminal 20 that has transmitted the presence confirmation request. Thus, a screen as shown in FIG. 8 is displayed on the user terminal 20. In addition, when a general user who is not the manager requests a confirmation of the presence information of the user A, the server 10 accesses the database 13 to acquire information on the presence time and the absence time of the user A. Then, the server 10 transmits the information of the user A acquired from the database 13 to the user terminal 20 that has transmitted the presence confirmation request. Thus, a screen as shown in FIG. 9 is displayed on the user terminal 20.

  This is the end of the description of the process shown in FIG.

  In addition, although the process by the object detection part 30 was abbreviate | omitted in FIG. 11, in addition to the process mentioned above, the process by the object detection part 30 may be performed. In this case, the process by the object detection unit 30 may be performed in any of the steps S102 to S108. For example, between step S105 and step S106, as processing by the object detection unit 30, processing of detecting another object, and whether or not the user is performing predetermined work using the other object Processing may be performed. Moreover, the process by the object detection part 30 may be performed between said step S108 and step S110.

  For example, when the object detection unit 30 detects a telephone as another object, a process of detecting a telephone and a process of determining whether a user is calling are performed as processing by the object detection unit 30. In this case, in addition to the presence flag and the drowsy flag, the memory of the user terminal 20 stores a telephone use flag indicating whether the user is calling. When it is determined that the user is calling, the telephone use flag is set to ON, and when it is determined that the user is not calling, the telephone use flag is set to OFF. When the value of the telephone use flag has changed, it is determined in step S110 that the state information has changed. That is, in step S110, it is determined that the state information has changed when the user's presence changes, when the user's drowsy changes, or when the user starts or ends a call. Thus, when the state of the user changes, information indicating the state of the user is transmitted to the server 10.

  As described above, in the present embodiment, the user terminal 20 performs the presence determination of the user based on the input image output in real time from the camera 22, and indicates whether the user is present or not. The information and time information are sent to the server 10. The server 10 stores the presence information transmitted from each user terminal 20 in the database 13. Thus, user presence information can be transmitted and stored in real time to the server 10. Each user can access the server 10 to check if another user is currently present.

  Further, in the present embodiment, presence determination is performed in the user terminal 20, and presence information indicating a result of the presence determination is transmitted to the server 10. Since the image taken by the camera 22 is not usually stored in the user terminal 20 and the server 10, while protecting the personal information of each user, other users can confirm the real-time presence information of each user .

  Further, in the present embodiment, the presence information of each user is accumulated in the server 10 together with the time information. For this reason, a user (for example, an administrator) can refer to the past presence information of other users. The user can check how long the other users are actually present during working hours, whether they are away or not, and can guess how long each user is resting etc. . A user who is not the administrator may also be able to refer to the past presence information of each user.

  Further, in the present embodiment, the user terminal 20 determines whether the user is sleeping by using the input image from the camera 22, and transmits the sleep information indicating the determination result to the server 10. The server 10 stores the drowsy information transmitted from the user terminal 20 in the database 13. Thereby, the user (administrator or a user who is not the administrator) can confirm whether or not another user is sleeping at present. In addition, since the server 10 stores the sleepiness information and the time information in the database 13, the user can confirm whether or not another user has slept in the past, and the time.

  Further, in the present embodiment, the user terminal 20 detects a predetermined object based on an input image from the camera 22, and determines whether the user is performing a predetermined operation using the predetermined object. Information indicating the determination result is transmitted to the server 10. The server 10 stores the information transmitted from the user terminal 20 in the database 13. Thereby, the user (administrator or a user who is not the administrator) can check whether another user is currently performing a predetermined operation (for example, whether he is calling or not). In addition, the server 10 accumulates in the database 13 information indicating whether the user is performing a predetermined task and time information. For this reason, the user can check the time that another user has been performing a predetermined task, and can check, for example, whether he / she is calling for a relatively long time.

  Further, in the present embodiment, when the state of the user (the present state, the drowsy state, the state as to whether or not the predetermined work is performed) changes, information indicating the state is transmitted to the server 10 and the user's If the state has not changed, the information is not sent to the server 10. For this reason, the load on the server 10 and the network can be reduced as compared with the case of always transmitting these pieces of information. These pieces of information may be periodically transmitted to the server 10.

  As described above, in the presence information management system 1 according to the present embodiment, the user accesses the server 10 to determine whether another user is present, whether he is sleeping or not, and performs predetermined work. Whether or not the user is on the phone (for example, on the phone) can be confirmed without going near the seat of the other user. For this reason, the user's presence situation of the remote user can be grasped in real time, for example, even for a home-working user, the user at home working is actually at home by imaging the seat periphery of the home with the camera 22 You can remotely check if you are doing business.

  In the presence information management system 1 of the present embodiment, since the actual working conditions such as the presence status of the user and whether the user is performing the original work are visualized, unnecessary rest time and idle time are reduced. Work efficiency can be improved.

  The presence information management system 1 may have a function of managing a user's schedule, in addition to the function of managing the above-described presence information and the like. FIG. 13 is a diagram showing an example of the logical configuration of the server 10 in another embodiment.

  As shown in FIG. 13, the server 10 includes a schedule management unit 15 in addition to the presence status management unit 11, the presence information acquisition unit 12, the database 13, and the output unit 14.

  The schedule management unit 15 provides a function of managing the schedule of each user. Each user registers his own schedule in the schedule management unit 15. For example, each user registers the time of a meeting, the time of going out, the time of other work, and the like. For example, the schedule management unit 15 has a web interface. Each user accesses the schedule management unit 15 of the server 10 using the browser application of the user terminal 20. Then, each user registers, changes, or confirms their own (or another user's) schedule. The schedule information of each user can also be referenced from other users.

  As shown in FIG. 13, the schedule management unit 15 is disposed in the server 10 having the presence status management unit 11, and the schedule management unit 15 and the presence status management unit 11 operate in cooperation with each other. The schedule management unit 15 may be arranged on a server different from the server 10 having the presence status management unit 11. When the presence status management unit 11 is disposed in the server 10 and the schedule management unit 15 is disposed in another server, the server 10 and the other server are linked via the network.

  FIG. 14 is a diagram showing an example of cooperation between the schedule management unit 15 and the presence status management unit 11.

  As shown in FIG. 14A, the schedule management unit 15 manages the schedule of each user. For example, according to the schedule information of a certain day of the user C managed by the schedule management unit 15, the user C is scheduled for a meeting from 10:00 to 12:00, from 13:30 to 14:30. It is scheduled for a meeting, and is scheduled to go out from 15:00 to 16:30. Other than that time, there are no plans in particular, and we plan to carry out regular work.

  On the other hand, as shown in FIG. 14B, according to the user C's presence information determined by the presence determination unit 28 of the user terminal 20, the user is present from 9:00 to 10:00. From 10:00 to 13:00, from 13:00 to 13:30, from 13:30 to 14:30, from 14:30 to 16:00, which is the current time I am present. For example, the user C was scheduled to go out from 15:00 to 16:30, but is still present as of 16:00 because the schedule has been changed. In this case, integrated information in which the schedule information of the user C shown in FIG. 14A and the presence information of the user C shown in FIG. 14B are integrated is generated and presented to other users.

  Specifically, as shown in FIG. 14C, the user C is present from 9:00 to 10:00, and is meeting from 10:00 to 12:00, They are away from 12:00 to 13:00, are present from 13:00 to 13:30, have a meeting from 13:30 to 14:30, and are from 14:30 to the present I am present. If another user accesses the server 10, integrated information shown in FIG. 14 (c) may be presented.

  The integrated information may be presented to each user as part of the function of the schedule management unit 15 that manages the user's schedule. For example, when a user accesses the schedule management unit 15 to check the schedule of another user, even if the integrated information in which the schedule information and the presence information of the other user are integrated is presented to the user Good. Further, the schedule information shown in FIG. 14A and the presence information shown in FIG. 14B may be separately presented to the user on one screen.

  Also, when a certain user accesses the presence status management unit 11 to check the presence information of another user, the integrated information in which the schedule information of the other user and the presence information are integrated corresponds to the particular user. It may be presented. Also, these pieces of information may be presented to the user separately on one screen.

  As described above, when the schedule management unit 15 and the presence status management unit 11 operate in cooperation with each other, it is possible to present the schedule of the user and the actual presence status to other users. Thus, even when the user's schedule is changed, other users can check the actual presence status.

  Note that a function for confirming whether the user is present (presence confirmation function), a function for confirming whether the user is sleeping or not (sleeping detection function), and the user using a predetermined object The function (work detection function) for confirming whether or not a predetermined work is performed may be provided integrally with the schedule management function by the schedule management unit 15. The presence confirmation function, the drowsiness detection function, and the work detection function may be provided as options of the schedule management function by the schedule management unit 15.

  In the above embodiment, the user refers to the presence information and the like of other users using a web browser, but refers to the presence information and the like of other users using dedicated application software. May be

  Further, in the above-described embodiment, the presence determination unit 28 transmits, to the server 10, time information on the determined time together with the presence information, and the time information is stored in the database 13 of the server 10. In another embodiment, based on the determination result by the presence determination unit 28, time information regarding the time during which the user is present or the time when the user is away (for example, the user may The information on the presence or absence and the information on how many hours are left may be stored in the database 13. That is, the time when the user is present or not may be stored in the database 13 instead of the time when the user is present or not.

  The server 10 of the presence information management system 1 may be configured by one server device or may be configured by a plurality of server devices.

  Further, the arrangement of each unit of the above-mentioned presence information management system 1 is an example, and each of these units may be arranged in any of the user terminal 20 and the server 10, and arranged in an apparatus different from the user terminal 20 and the server 10. It may be done. For example, at least one of the face detection unit 27, the presence determination unit 28, the sleep detection unit 29, and the object detection unit 30 is a device different from the user terminal 20 (for example, a dedicated device for image recognition or a server 10). For example, an apparatus different from the user terminal 20 may acquire an input image from the camera 22, and face detection, presence determination, and the like may be performed in the apparatus. Also, any one of the plurality of user terminals 20 may function as the server 10. Further, the server 10 acquires an input image from the camera 22, and the server 10 performs the presence determination, the drowsy determination, and the determination as to whether or not the user is performing a predetermined task, according to the results of these determinations. Information may be stored in database 13.

  Further, although the database 13 is disposed in the server 10 in the above embodiment, the database 13 is disposed in an apparatus different from the server 10 so that the server 10 can access the database 13 via the network. It may be done. For example, the database 13 may be located on the Internet. Also, the database 13 may be configured by a plurality of devices. Each piece of information stored in the database 13 may be distributed and stored on different databases. That is, the database 13 may be physically located at any location as long as it can be accessed from the server 10. In this case, when the user accesses the server 10 and confirms the presence information of the other user, the server 10 accesses the database 13 on the network to obtain the presence information of the other user, Send the presence information to the other user. From the user's point of view, the server 10 and the database 13 appear as one, and the above-mentioned presence information, drowsy information, and information as to whether or not the user is performing predetermined work are substantially stored in the server 10 Is the same as

  In addition, part or all of the above-described processing may be performed by a dedicated circuit or may be performed by a general purpose processor.

DESCRIPTION OF SYMBOLS 1 presence information management system 10 server 11 presence condition management part 12 presence information acquisition part 13 database 14 output part 15 schedule management part 20 user terminal 21 information processing part 22 camera 27 face detection part 28 presence determination part 29 dozing detection Unit 30 Object detection unit 30

Claims (12)

An image acquisition unit that acquires an image from a camera provided around the seat of the user;
Face detection means for detecting the face of the user based on the image acquired by the image acquisition means;
Presence determination means for determining whether the user is present based on the detection result of the face detection means;
Storage control means for causing the server on the network to store the presence information of the user according to the determination result of the presence determination means;
An information processing system, comprising: presentation means for acquiring the presence information of the user stored in the server via a network, and presenting the acquired presence information of the user to another user. The information processing system includes a user terminal provided around the seat of the user,
The image acquisition unit, the face detection unit, and the presence determination unit are disposed in the user terminal,
The user terminal transmits presence information according to the determination result of the presence determination means to the server via a network,
The server is
Storing presence information transmitted from the user terminal in a storage unit;
Confirmation request receiving means for receiving a confirmation request for presence information stored in the storage means from the terminal of the other user;
And transmission means for transmitting the presence information stored in the storage means to the terminal of the other user via the network when the confirmation request is received,
The information processing system according to claim 1, wherein the terminal of the other user presents the presence information transmitted by the transmission means to the other user.   The user terminal determines whether the presence state of the user has changed based on the determination result of the presence determination means, and determines that the presence state of the user has changed. The information processing system according to claim 2, wherein information is transmitted to the server.   The information processing system according to any one of claims 1 to 3, wherein the storage control means causes the server to further store time information according to the determination result of the presence determination means.   The information according to claim 4, wherein the presenting means presents the other user's past presence information to the other user based on the user's presence information and the time information stored in the server. Processing system. When the presence determination means determines that the user is present, the sleep determination means further determines whether or not the user is asleep based on the image acquired by the image acquisition means. Equipped
The information processing system according to any one of claims 1 to 5, wherein the presenting means presents, to the other user, doze information indicating the determination result of the doze determining means. When the presence determination means determines that the user is present, a predetermined object is detected based on the image acquired by the image acquisition means, and the user uses the predetermined object. The system further comprises work determination means for determining whether or not a predetermined work is being performed,
The information processing system according to any one of claims 1 to 6, wherein the presenting unit presents, to the other user, task information indicating the determination result of the task determining unit.   The information processing system according to any one of claims 1 to 7, further comprising schedule management means for managing schedule information of the user. It further comprises: integrated information generation means for generating integrated information in which the schedule information managed by the schedule management means and the presence information are integrated,
The information processing system according to claim 8, wherein the presenting unit presents the integrated information generated by the integrated information generating unit to the other user. An information processing method executed in an information processing system, comprising:
An image acquisition step of acquiring an image from a camera provided around the seat of the user;
A face detection step of detecting a face of the user based on the image acquired in the image acquisition step;
A presence determination step of determining whether the user is present based on the detection result of the user's face in the face detection step;
A storage control step of causing the server on the network to store the presence information of the user according to the determination result of the presence determination step;
A presentation step of acquiring presence information of the user stored in the server via a network, and presenting the acquired presence information of the user to another user. A program executed by a computer of an information processing apparatus, said computer comprising:
An image acquisition unit that acquires an image from a camera provided around the seat of the user;
Face detection means for detecting the face of the user based on the image acquired by the image acquisition means;
Presence determination means for determining whether the user is present based on the detection result of the face detection means;
An information processing program that causes a user's presence information according to the determination result of the presence determination means to function as a transmission means for transmitting to a server via a network. An image acquisition unit that acquires an image from a camera provided around the seat of the user;
Face detection means for detecting the face of the user based on the image acquired by the image acquisition means;
Presence determination means for determining whether the user is present based on the detection result of the face detection means;
An information processing apparatus, comprising: transmission means for transmitting presence information of the user according to the determination result of the presence determination means to a server via a network.

Images