JP5319260B2 - Work monitoring device - Google Patents

Description

  The present invention relates to a technique for monitoring work conditions in offices, classrooms, and various work environments.

  In recent years, there has been a growing need for monitoring work conditions from the viewpoint of improving operational efficiency in offices and various work environments and compliance. In addition, a business management system that monitors work on a work terminal such as a personal computer (PC) has appeared as a means for that purpose. For example, Patent Document 1 describes a method in which a remote person knows the presence status, busyness of work, and the like by acquiring the keystroke interval of a keyboard or pointing device as statistical information.

  A method for monitoring using not only a keyboard and a pointing device but also other information has been proposed. For example, in Patent Document 2, not only key input and pointing device input data but also display output signals are collected to facilitate analysis in the event of a trouble. Further, Patent Document 3 describes a method of reproducing a work state using video information about a document change history and an operator's operation status during a meeting or the like.

JP 2005-332344 A JP 10-171844 A Japanese Patent Laying-Open No. 2005-234641

  In the monitoring method based on the operation log recording of the terminal such as the computer described above, it is difficult to know the action pattern when the keyboard or pointing device is not operated. Even when the keyboard and pointing device are not operated, there are many cases where the business is efficiently carried out, for example, the paper medium is read and written, or the discussion is performed. Therefore, even if there is no operation of the keyboard or pointing device, it cannot be determined that the business efficiency is low in all cases. Conversely, even if a keyboard or pointing device is operated, it cannot be said that the work efficiency is necessarily high. For example, when there is a deadlock in the work and the pointing device is moved meaninglessly, it is more appropriate to regard the work as stagnant. For this reason, in order to know the work efficiency, using only the pause of the keyboard or pointing device operation as a clue is not a correct judgment.

  On the other hand, in the conventional technology that performs work monitoring based on video information, it is necessary to check the content of work by viewing the video as much as possible, and much time is required for the monitoring work.

  In addition, since the method using video is effective only when a person is in the imaging field of view and is performing a clear motion, it is difficult to judge the content of the operation because it is far from the imaging field of view. In some cases, monitoring was difficult.

  An object of the present invention is to provide a work monitoring method, an apparatus thereof, and a recording / analysis server capable of solving the problems in the case of detecting only pause of operation of an input device.

  In order to solve the above-mentioned object, the present invention is characterized by detecting and estimating a work stagnation state by analyzing various operational characteristics as well as a pause state of an input device such as a keyboard or a pointing device. To do.

  Further, in the present invention, in order to detect and estimate the work stagnation situation, not only the operation history of the input device, but also the screen of the display device and the video pattern taken by the worker are analyzed, the input operation is paused, and the input operation Is characterized by extracting features when there is no substantial work progress.

  Furthermore, the present invention is characterized in that an administrator or a user can set the setting contents when detecting and estimating the degree of work stagnation from various monitoring characteristics.

  More preferably, the display screen and the user's video recording procedure are controlled in accordance with the degree of work stagnation.

  Still further, the present invention is characterized in that it is added to the monitoring information by analyzing and recognizing the video recording at the time of detecting the work stagnation state.

  Furthermore, by analyzing the video pattern taken by the worker, it identifies the seating area, and if there is a work stagnation when seated, the user is inquired. It is characterized by waiting and inquiring about behavior during leaving.

  According to the present invention, by monitoring and estimating the degree of stagnation by analyzing operation monitoring characteristics, business monitoring based on the stagnation status of work is performed more accurately than simple determination based on whether or not the keyboard or pointing device is operated. be able to.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1A is a diagram showing a system configuration in the first embodiment. Worker 101 performs work at work terminal 105 using both input devices of key input device 102 and pointing device 103, and display 104. Here, the key operation of the worker 101 and the operation of the pointing device can be acquired and recorded one by one by a software program installed in the work terminal 105. As a mounting method therefor, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2007-304758 is used. The recording format for recording will be described later with reference to FIG.

  In addition, a camera 106 is connected to the work terminal 105 so that an image of the worker 101 can be acquired. In addition, the work terminal 105 is provided with a recording device 107 that is a temporary recording buffer, and records operation logs of keys and pointing devices, video on the display, and video of the user by the camera in the procedure described later. .

  The work terminal 105 is connected to the recording / analysis server 109 via the network 108. The recording / analysis server 109 collects the operation logs collected by the work terminal 105 and recorded in the recording device 107, the video on the display 104 and the user's video through the camera 106, and analyzes the situation. The necessary data is stored in the recording device 110. If necessary, information and alarms are transmitted to the manager 111.

  FIG. 1B is a diagram illustrating an example of a hardware configuration of the work terminal 105 in the present embodiment. This is a standard computer architecture in which each module exchanges data via a data bus 1001. The module includes an interface (IF) 1002 of the external camera 106, a video display unit 1003 for displaying on the display 104, a key input device 102, and a pointing device 103, and respective interfaces (IF) 1004 and 1005, a central processing unit (Central An arithmetic unit 1006 that is a processing unit such as a processing unit (CPU), a memory 1007 that is a storage unit, a monitoring program 1008 that is stored in the memory 1007 during operation, and an interface (IF) to the recording unit 107 that is a storage unit 1009 includes a communication unit 1010 for connecting to the network 108. Note that a setting table 1011 to be described later and recording data 1012 such as an operation log are stored in the storage device 107. Here, the setting table 1011 is created by, for example, a recording / analysis server, which will be described later, and a copy of the setting table 1011 is sent via the network, or stored by a method such as a user creating it on the work terminal itself. Is done. On the other hand, the record data 1012 records operation logs and the like by a method described later by the work terminal itself.

  FIG. 1C shows the hardware configuration of the recording / analysis server 109 in the present embodiment, including the peripheral devices. The module is a processing unit such as a video display unit 2101 for displaying on the display 2100, a key input device 2102, and a pointing device 2103, respective interfaces (IF) 2104 and 2105, and a central processing unit (CPU). An arithmetic device 2106, a memory 2107 as a storage unit, and a storage / analysis program 2114 stored in the memory 2107 during operation, an interface (IF) 2109 to a recording device 2110 as a storage unit, and a network 108 connection The communication unit 2120 is included. The recording data (accumulation) 2113 stored in the storage device 2110 includes data obtained by receiving recording data 1012 transmitted from one or more work terminals via a network.

  FIG. 2 is a diagram illustrating a processing procedure in the work terminal 105 in the present embodiment. The procedure in FIG. 2 is executed by the monitoring program 1008 in FIG. 1B. In this procedure, first, the key operation, pointing device operation, display screen, and operator's video are always buffered and updated in the temporary recording buffer 107 by a certain amount of time back (201). Using them, the arithmetic unit 1006 as the processing unit monitors the stagnation status of the worker according to the procedure described later (202). As a result, if it is determined that there is at least a slight work stagnation (203), the display screen image and additional information are recorded and transferred to the server 109 (204). Here, the additional information includes information on the opened file name (generally a plurality), information on the name of the file operated last, and the like, as described in the above-mentioned Japanese Patent Application Laid-Open No. 2007-304758. It is automatically acquired from the basic software information of the work terminal 105 by the method described. If it is determined that there is a non-mild stagnation (205), the buffered worker image is recorded and transferred to the server (206). The above processing is repeated (207) until an end instruction is given.

  FIG. 3 is a flowchart showing an example of a stagnation status monitoring routine (202) during the processing procedure at the work terminal 105 of FIG. 2 in the present embodiment. The procedure in FIG. 3 is a part of the procedure in FIG. 2 and is executed by the monitoring program 1008 in FIG. 1B. In this embodiment, the determination is made based on the key operation of the key input device 102 and the input of the pointing device 103. First, it is determined whether or not a key operation has been stopped for a short time (301). If it has not occurred, it is determined that there is no stagnation (302). On the other hand, if the stop has occurred for a short time, it is determined whether or not the operation characteristic of the pointing device 103 is unique to the stagnation (303). Here, as an operation characteristic peculiar to stagnation, when there is no substantial work progress, that is, (1) No pointing instruction (button pressing etc.) is performed, and only the reciprocating movement at the indicated position is repeated for the specified time or longer. (2) Detect that the screen has been scrolled up and down or left and right more than the specified time. In such a case, there is a high possibility that the worker is facing some problem that the work cannot be performed. When such an operation peculiar to stagnation is obtained, it is determined as “mild stagnation” in this embodiment (304). On the other hand, if there is no unique stagnation feature, it is determined that the key operation (305) and the pointing device (306) both stop for a long time as “stagnation” (307). On the other hand, when viewed for a long time, if there is at least one of the operations, it is determined again as “mild stagnation”.

  Here, the “short time” and “long time” settings are parameters that the system determines and has in advance in consideration of the business content, environment, and the like. For example, the short time is set to 2 minutes and the long time is set to 10 minutes.

  FIG. 4 is a flowchart showing a processing procedure in the recording / analysis server 109 in this embodiment. The procedure of FIG. 4 is a procedure when the arithmetic unit 2106 executes the recording / analysis program 2114 in FIG. 1C. Data including the display screen and buffered video from the work terminal 105 is monitored (401), and when received (402), the data is analyzed (403). Data analysis refers to the work of converting the work content into a specified format and storing it in the database, and the work of determining whether an alarm to the administrator is necessary when the stagnation is repeated. Point to. If it is determined that an alarm is necessary (404), an alarm is issued to the administrator (405). Here, depending on operation settings, an alarm may be set to the work terminal 105 via a network. When a certain time has passed (406), the display of the monitor displaying the work is updated (407). An example of display on the monitor will be described later. The above processing is repeated until an end instruction is given (408).

  FIG. 5 is a diagram showing an example of the configuration of recording data in the present embodiment. The data includes an operation event 502 that records what software was operating, etc., determined from time 501, key operations, etc., a stagnation status 503, a display that indicates the stagnation status of the above-described work and the video that recorded it State 504, a video state 505 that can be recognized from the worker's video, and a worker memo 506 such as a comment input by the worker. For example, at 10:20:03 in FIG. 5, there is a long work stagnation (non-mild work stagnation), the reference video is No. 33, the display is stagnant, and the worker is present from the video state. Has been recorded. Also, at 10:45:50, a long work stagnation (non-mild work stagnation) was recognized, the display was activated, the worker was seated as a video event, and the operator entered "Meeting" .

  In this embodiment, it is assumed that the recorded data is finally stored on the server side. However, the recorded data is referred to on the terminal side, or some operation is performed using the data. If it is necessary to do so, the record / analysis server 109 may refer to the record even if the record is left in the work terminal 105 that collected the data. In the following description, it is assumed that the recorded data can be referred to from either the terminal side or the server side. Alternatively, depending on the policy, it may be set so that the administrator can browse only on the recording / analysis server 109.

  FIG. 6 shows an example of the screen configuration when the administrator monitors on the server side in this embodiment. A belt-like graph 601 is a graph showing the degree of operation in each time zone, and shows that each of the three time zones 602, 603, and 604 indicated by hatching in this belt graph 601 is stagnant. The lower three areas 605, 606, and 607 are areas for displaying images, comments, and the like at the respective stagnation times. By viewing the video, it is possible to know what the worker was doing during each stagnation time period. On the contrary, since the video of the time zone when the work is not stagnated is not displayed, the administrator can know the outline of the work in a short time.

  As described above, in this embodiment, the display is viewed by the administrator side. However, in this embodiment, since the browsing on the work terminal 105 side is also possible as described above, when the worker finishes the work of the day. For example, it is possible to use it for checking the work contents by looking at its own record.

  In the present embodiment, an embodiment in which the stagnation state can be set in detail will be described. In the first embodiment, the short stop of the key operation is a necessary condition for the stop determination. However, in this embodiment, the stagnation state can be determined by various conditions without being limited to such a condition.

  The entire processing procedure of the work terminal 105 is shown in FIG. This procedure is a processing procedure when the arithmetic device 1006 executes the monitoring program 1008 in FIG. 1B. In this embodiment, the inside of the stagnation status monitoring routine (202) is different, and FIG. 7A is a flowchart showing this. The procedure in FIG. 7A is a part of the procedure in FIG. 2 and is executed by the monitoring program 1008 in FIG. 1B. Here, as described in the first embodiment, the stagnation is determined using data obtained by tracing the input key operation, the pointing device operation, the display screen, and the operator's video by a predetermined time. For example, 15 minutes is set as the fixed time. If necessary, key operations and images or videos may be stored for different times.

  First, the stagnation state of key operation of the key input device 102 during a certain time is calculated (701). Here, the key operation stagnation basically uses the time during which the key operation is paused, but the accumulated input amount itself can be obtained by pressing a specific key abnormally long or repeating erasing and inputting. Even if there is no key, since there is no substantial work progress, it is included in the stagnation of key operations. Details of the procedure will be described later with reference to FIG. 7B.

  Next, the stagnation state of the pointing device 103 is calculated (702). In the case of the pointing device 103, as in the case of the key operation, basically, the time during which the operation is paused is used, but in addition, when the reciprocating motion that is difficult to judge from the net input pattern is repeated, Included in work stagnation as no substantial progress has been made. Such detection of the reciprocating motion of the pointing device 103 assumes a state in which the operator cannot simply touch the pointing device 103 without touching the actual work. Details of the procedure will be described later with reference to FIG. 7C.

  Next, the stagnation state of the screen of the display 104 is calculated (703). Even on the display screen, if there is basically no screen update, no substantial progress of the work is seen, so the work is stagnant. Similarly to the case of the key input device 102 and the pointing device 103 described above, the case where there is no accumulation of input and there is no essential change is included in the stagnation. Details of the procedure will be described later with reference to FIG. 7D.

  Next, the stagnation state by the worker image is calculated (704). The stagnation state due to the worker image is determined by recognizing the image captured by the camera described above. For example, when it is determined that the worker is not working while facing down, when it is determined that the worker is sleeping, even when the worker is absent in front of the camera, there is no substantial work progress. It is determined that the work is stagnant. The recognition processing is performed using pattern recognition processing from video such as face detection, face direction determination, line-of-sight detection, and change detection. Details of the procedure will be described later with reference to FIG. 7E.

  The stagnation is determined using the above-described work stagnation situation. First, it is determined whether or not a mild stagnation condition is satisfied (705), and then whether or not a non-mild stagnation condition is satisfied (706). Then, the stagnation (709) is determined. Here, as described later, the mild stagnation condition and the stagnation condition are determined based on a group of conditions using the plurality of stagnation situations. The contents of condition setting and the description method will be described later.

  FIG. 7B shows a procedure for calculating the stagnation state of the key operation described above (701). The procedure in FIG. 7B is a part of the procedure in FIG. 7A. Further, as described above, the procedure in FIG. 7A is a part of the procedure in FIG. It is executed at 1008.

  In the procedure, first, the key operation pause time of the latest key input device 102 is measured (7001). This time is defined by the time interval from the most recent key operation to the present.

  Next, the duration of the latest stagnation pattern is measured (7002). Here, the stagnation pattern is an abnormal continuous keystroke of a specific key or a keystroke that is regarded as having no substantial input accumulation. As an example of the former, the determination is made based on whether the same character input key has been pressed, for example, several tens of times. This corresponds to a case where a key input not related to the progress of work is performed because an obstacle or a hand weight is applied to the keyboard separately from the user's intention. As an example of the latter, there is input of the same number of backward keys after several character input key operations. For this kind of pattern detection, the keystrokes are divided into (1) single character input or (2) backward, erase, or (3) operations not related to input such as control keys, and character conversion. The cumulative number of input characters is calculated at, and if this value does not increase over time, a stagnation pattern is set. Note that the calculation is performed by skipping the pause period during the calculation.

  Both of these indices are output as stagnation of key operations. For example, the output is “5 minute pause, 7 minute stagnation”.

  FIG. 7C shows a procedure for calculating the stagnation state of the pointing device 103 (702). The procedure in FIG. 7C is a part of the procedure in FIG. 7A. Further, as described above, the procedure in FIG. 7A is a part of the procedure in FIG. It is executed at 1008.

  In the procedure, first, the latest pointing device pause time is measured (7101). This time is defined as the time interval from the most recent pointing device operation to the present.

  Next, as in the case of the key operation, the duration of the latest stagnation pattern is measured (7102). In the case of the pointing device 103, as described above, a reciprocating motion that is difficult to determine from the net input pattern is detected. Specifically, in the same way as described in the first embodiment, when (1) pointing instruction (button pressing etc.) is not performed and only the reciprocating motion at the indicated position is repeated for a specified time, (2) screen Is a pattern in which scrolling up and down or left and right is repeated for a specified time or more. As for the detection method (1), when the locus on the display of the pointing device is constructed, it is regarded as a stagnation pattern when only reciprocation is repeated without performing an operation such as clicking. Here, the reciprocating motion means that the trajectory on the display returns many times to the vicinity of the position where it has previously passed without a click operation. At that time, a parameter for determining how far to consider as a neighborhood and how many times to return as a stagnation is determined experimentally. For example, when 1/10 of the horizontal width of the screen is regarded as a neighborhood and the user returns to the neighborhood five times or more, the setting is made. Similarly, when scrolling the screen of (2), it is regarded as a stagnation pattern in the case where the up and down scroll motion is repeated in a reciprocating manner without performing an operation such as clicking.

  Both of these indices are output as the stagnation status of the pointing device operation. For example, the output is “3-minute pause, 3-minute stagnation”, or the like.

  FIG. 7D shows a procedure for calculating the stagnation status of the screen of the display 104. The procedure of FIG. 7D is a part of the procedure of FIG. 7A. Further, as described above, the procedure of FIG. 7A is a part of the procedure of FIG. It is executed at 1008. In subsequent processing, when a change in the display screen is detected, the display screen is recorded at regular time intervals, and processing for obtaining a difference image between the target screen and a past screen pattern is performed.

  The procedure first checks the latest display no-change time (7201). This time is defined as the time interval from the most recent display screen change to the present.

  Next, as in the case of a key operation or a pointing device, the duration of the latest stagnation pattern is measured (7202). Specifically, as in the case of the stagnation situation, when the change location by the time difference image on the display screen occurs only in a specific location, it is determined as a stagnation pattern. At that time, a parameter about how far to be regarded as a specific place is experimentally determined.

  The display screen difference process used above is a process that is widely performed in image processing, and is a process of recording a display screen as an image and obtaining a value difference for each pixel. By looking at the sum of the image values obtained as a result, it is possible to measure the magnitude of the change and to determine whether or not the place where the change is large is only a specific place. More detailed procedures are described in, for example, the chapter on moving image processing in “Digital Image Processing” (CG-ARTS Association, 2006) edited by CG-ARTS Association.

  FIG. 7E shows a procedure for calculating the stagnation status of the worker video. The procedure of FIG. 7E is a part of the procedure of FIG. 7A. Further, as described above, the procedure of FIG. 7A is a part of the procedure of FIG. It is executed at 1008. In the case of calculating the stagnation status in the worker video, various calculations are performed by performing person recognition by combining image recognition. That is, moving person detection using face detection and detection of a moving body is performed.

  In the procedure, first, whether the user is present or not is detected by face image detection and motion body detection (7301). Here, the moving body detection is processing for detecting an operation when a person is present by image processing. The reason for using both face image recognition and moving body detection is a countermeasure when the face cannot be supplemented, such as when the face is facing downward.

  As these processing methods, processing generally performed in image processing can be used. Face image detection can be performed by, for example, Stan Z. Li and Anil K. et al. Jain eds. “Handbook of Face Recognition,” (Springer, 2005), moving body detection, for example, the chapter of “Digital Image Processing” (CG-ARTS Association, 2006) of the above-mentioned CG-ARTS Association Please refer to each.

  As a result of the above processing, when the user is not present (7302), the time after leaving is measured retroactively (7303). That is, assuming that the person is currently away from the seat, the most recent event of standing is detected, and the time from there is measured. Here, the event of standing up is detected by calculating movement information peculiar to the case where a region (person) occupying a large field of view on the screen stands up by image processing. Specifically, it is detected whether a change in the luminance of the video in the vertical direction (optical flow) has occurred near the center of the image.

  On the other hand, when the user is present (7302), the operator's non-concentration state is detected (7304). Here, the non-concentrated situation is a situation in which the face is excessively faced down for a long time, the eyes are kept closed, or the line of sight is not facing the screen. Here, whether or not the face is turned down is recognized by whether or not the round pattern of the head is facing the camera. Whether or not the eyes are kept closed is determined by extracting the pattern of the pupil after the face detection described above. The line of sight detects the direction of the line of sight by the line-of-sight detection process for detecting the direction of the black eye, and determines the ratio of not facing the front area.

  As an image processing method used in these, processing generally performed in image processing can be used. For example, a technique such as Hough transform can be used to extract a round pattern such as the head. For the detection of the pupil, for example, a similar method can be used for detecting a round pattern. Further, the detection of the gaze direction uses a method such as secondarily estimating from the pupil detection result and the face detection result. For details, see the chapter on pattern and figure extraction in Non-Patent Document 1, for example.

  FIG. 8A shows a stagnation condition setting table in this embodiment. The items constituting the conditions are the above four items of “key operation” 801, “pointing device” 802, “display screen” 803, and “worker image” 804. It is a condition that each row is judged to be lightly stagnant or stagnant. Four item conditions for each row are determined by AND. However, the state of the item “-” (not filled in) does not matter. Between each row, there is an OR condition for light stagnation or stagnation. With the above configuration, any combination condition composed of four items can be described. For example, in the first line, when there is no key operation for 2 minutes, the pointing device performs a stagnation operation, and the worker image is not concentrated, it is regarded as a slight stagnation.

  When setting the stagnation condition, either the above table may be directly edited, or the system may be input by some kind of graphic user interface (GUI). FIG. 8B is a diagram illustrating an example in which settings are input using a GUI. The determination column 8001 has a pull-down 8002 for selecting a determination result, and selects which determination result is to be defined. In the figure, for example, “stagnation” is selected. Next, the state is selected with a radio button or pull-down in the order of key operation 8003, pointing device 8004, display screen 8005, and worker image 8006. Note that the worker image portion of FIG. 8B shows the appearance during the selection work of “no operation” from the pull-down 8007. Finally, one condition is registered with an “add” button 8008. In the lower column, already registered conditions are displayed (8009).

  The stagnation condition can be set by the administrator on the recording / analysis server 109 side, and the setting file can be transferred to the work terminal 105 side so that the program on the terminal side can read and operate it. Is possible. Alternatively, depending on the form of operation, it is possible for the worker side to set his / her situation in advance at the work terminal 105.

  In this embodiment, an embodiment having a function of making an inquiry when work is stagnant will be described.

  FIG. 9 is a flowchart showing the operation of the work terminal in this embodiment. The procedure of FIG. 9 is executed by the monitoring program 1008 in FIG. 1B as in the first embodiment. The procedure is the same as in the first and second embodiments, up to the start of video buffering (201), monitoring of stagnation of work (202), and transfer of video (206). Next, an attempt is made to recognize a person such as an operator using the installed camera image (901). Here, as a method for recognizing a person, a method for detecting a face pattern and a moving body is used in the same manner as the method used for monitoring work stagnation in the second embodiment.

  If the worker 101 is detected as a result of the person recognition, it is determined that the worker 101 is not away (902), and the situation is inquired (903). The inquiry about the situation uses means such as displaying on the display 104 of the work terminal 105 and prompting input as will be described later.

  On the other hand, when it is considered that the worker is away without being detected (902), a change in the camera image is detected in order to detect that the worker is next seated (904). A change detection method is also performed by processing a video pattern. The basic procedure is to detect whether a change in the luminance of the video in the vertical direction (optical flow) has occurred near the center of a specific image, in the same manner as the method used for monitoring work stagnation in the second embodiment.

  This change detection is repeated until seating is detected (905), and when seating is detected, a return inquiry is made (906). The inquiry at the time of return is also displayed on the display of the work terminal to prompt input as will be described later.

  FIG. 10 is an example of a screen for making an inquiry when an operator is present and work stagnation is observed. In the upper row 1001, the recent work history is displayed to assist the worker in grasping the situation. At the second stage 1002, grasp and record the problems causing the stagnation. First, the button 1003 is selected to classify the problem, and the entry box 1004 is made to write down the contents. Note that these memo contents are stored as contents in the worker memo column stored in the server described in the first embodiment.

  FIG. 11 is an example of a screen for making an inquiry when it is detected that the worker who has left the seat is seated again. The top 1101 prompts the user to select a destination, and the button 1102 selects the destination. Next, in the middle row 1103, an inquiry is made about the requirements for leaving the seat, the classification is made with the button 1104, and the contents are written down in the entry box 1105. The lower row 1106 displays the work history immediately before leaving the seat and assists the worker in grasping the situation.

  In each embodiment, the stagnation of the work is determined from various indexes, so that the content of the inquiry can be changed according to the degree and content. For example, even within the same work stagnation, it is possible to divide within 15 minutes and more, and when it is 15 minutes or more, it is possible to make a change such as requesting a more detailed report including an inquiry that cares about physical condition. By such a device, it is possible to discriminate a serious problem such as poor physical condition and normal stagnation, and to leave a more detailed work record.

  As described above, according to the present invention, the state of work stagnation can be more accurately detected than the simple determination based on the presence or absence of operation of the keyboard or pointing device by detecting and estimating the degree of work stagnation by analyzing operation monitoring characteristics. Business monitoring based on

  Further, by adding and analyzing not only the input device but also the screen of the display device and the video pattern obtained by photographing the worker, it is possible to detect and estimate the work stagnation state with higher accuracy.

  Furthermore, since the administrator or the user can set the setting contents when estimating the degree of work stagnation from various monitoring features, the judgment of business stagnation can be finely changed according to the work site.

  Furthermore, by controlling the screen and the video recording procedure of the user according to the degree of stagnation, video of important parts such as time zones that are not recorded by keyboard or pointing device operations or time zones when changes occur can be recorded. While remaining, the recording capacity can be reduced as compared with the case where video is continuously recorded.

  Furthermore, it is possible to supplement the monitoring by analyzing and recognizing the video recording when the work stagnation state is detected, and adding it to the monitoring information. In particular, by detecting and recording a seating / seating operation, it is possible to distinguish between a time zone in which an operator is captured by video and a time zone in which the worker is not captured.

  In addition, if you take a seat, and if you are seated and make an inquiry to the user when a long period of work stagnation occurs, supplementary information for recording can be obtained, so that the behavior record is more seamless Is possible.

  On the other hand, when the user is away from the desk, the user can wait for the return and inquire about the contents at the time of leaving the seat, so that even when the time is not reflected in the camera, the action can be recorded without interruption.

  The present invention described in detail above can be widely applied not only to monitoring work in an office, but also to systems and services related to techniques for monitoring work conditions in various work environments. For example, if a system for recording the operation of the plant control device instead of recording the key operation of the work terminal is constructed, the work status in the plant control room can be monitored.

1 is a diagram illustrating a system configuration in Embodiment 1. FIG. FIG. 3 is a diagram illustrating a hardware configuration of a work terminal according to the first embodiment. 2 is a diagram illustrating a hardware configuration of a recording / analysis server in Embodiment 1. FIG. It is the figure which showed the process sequence in the work terminal in Example 1. FIG. It is a flowchart which shows the monitoring routine of a stagnation state during the process sequence in the work terminal of FIG. 2 in Example 1. FIG. 3 is a flowchart illustrating a processing procedure at a server in the first embodiment. 6 is a diagram illustrating an example of a configuration of recording data in Embodiment 1. FIG. 6 is a diagram illustrating an example of a GUI when an administrator monitors on the server side in Embodiment 1. FIG. FIG. 10 is a flowchart showing the inside of a stagnation status monitoring routine in Embodiment 2. FIG. 10 is a flowchart illustrating a procedure for calculating a stagnation state of key operations in the second embodiment. FIG. 10 is a flowchart illustrating a procedure for calculating a stagnation state of a pointing device in the second embodiment. It is a flowchart which shows the procedure which calculates the stagnation state of a display screen in Example 2. FIG. It is a flowchart which shows the procedure which calculates the stagnation state of an operator image | video in Example 2. FIG. 10 is a diagram illustrating a stagnation condition setting table according to the second embodiment. It is the figure which showed the example of GUI for setting a stagnation condition. FIG. 10 is a flowchart showing the operation of the work terminal in the third embodiment. FIG. 10 is a diagram illustrating an example of a screen for making an inquiry when a worker is present and work stagnation is observed in the third embodiment. FIG. 12 is a diagram illustrating an example of a screen for making an inquiry when it is detected that an operator who has left a seat is seated again in the third embodiment.

Explanation of symbols

101 ... Worker
102 ... Key input device
103… Pointing device
104 ... Display
105 ... Work terminal
106 ... Camera
107: Temporary recording buffer
108 ... Network
109 ... Recording / Analysis Server
110 ... Recording device
111… Administrator
601 ... strip graph
602, 603, 604 ... time zone
605, 606, 607 ... display area.

Claims (3)

A work monitoring device for monitoring a worker's work,
An input device comprising a keyboard or pointing device, a recording device, a display device, a photographing device, and a processing unit;
The processing unit analyzes the status of the input operation of the input device, the display status of the display device, and the content of the video of the worker by the photographing device, or any combination thereof, thereby Detects work stagnation,
In order to detect the work stagnation status, the processing unit uses an integrated amount of the input content as a result of the key operation of the keyboard, or a moving operation that does not include the pointing operation of the pointing device for a long time, or up / down / left / right of the screen Using the scroll,
A work monitoring device characterized by that . The work monitoring device according to claim 1,
The processing unit detects presence or absence of a change in the content of the display screen of the display device in order to detect the work stagnation situation.
A work monitoring device characterized by that. The work monitoring device according to claim 1 ,
The processing unit analyzes the content of the worker's video by the photographing device in order to detect the work stagnation situation ,
A work monitoring device characterized by that.

Images