JP6241505B2 - Management server, information processing terminal, control method, and program - Google Patents

Description

  The present invention relates to an information security monitoring technique using a gaze estimation technique.

  In recent years, interest in telework has increased in each company. Telework has issues such as preventing third-party access to confidential information and optimizing labor management, and these issues must be resolved before it can be introduced.

  Currently, in order to solve these problems, a telework management support system using face authentication technology has been developed. By using this system, identity authentication is performed from a webcam image, and spoofing or peeping by a third party is performed. Status can be detected and notified to the administrator.

  In teleworking, the administrator cannot directly monitor the teleworking terminal screen, so it is important to understand the confidential information that may be leaked from the displayed contents of the terminal screen when an information security problem occurs. Become.

  Patent Document 1 is a technique for managing the working status of a telecommuter, and when the working status of the telecommuter is “on duty”, the display screen is periodically captured and stored in a server as a work trail. Send.

JP 2012-256329 A

  In the technique of Patent Document 1, the contents displayed on the terminal screen can be confirmed later by capturing the display screen periodically and leaving it as a trail.

  According to this method, the contents displayed on the terminal screen can be left as a trail on a regular basis. However, if a problem with information security, such as a third person peeping, occurs, it is displayed at any position on the screen. You can't leave a trail until you are interested in the information.

  Therefore, for example, there is a case where secret information is peeped from a terminal screen by a third party and it is impossible to determine which information should be preferentially dealt with when taking measures against leakage.

  In this way, when there is a leakage of confidential information, the priority of the countermeasures against leakage is accurately determined by leaving a trail of the information displayed on the terminal screen and which part of the screen is of interest. It becomes possible.

  Accordingly, an object of the present invention is to provide a mechanism capable of confirming the possibility of information leakage by indicating whether the user is paying attention to a display area with a high security level for the content displayed on the screen. To do.

The management server of the present invention is connected in a signal passing the User chromatography The terminal, a management server that previously displayed on the display device the status of Kiyu over The terminal, before acquiring the screen image of Kiyu over The terminal a screen image obtaining unit that, the security level information obtaining means for obtaining confidential level information security level in the screen image has identified a high region, corresponding to the screen image, the line of sight user with the screen image is gazing Table gaze information acquisition means, based on the obtained security level information and the viewing information, the information for the user to visualize how the gaze of the user terminal secret level is high region of the screen image to obtain information for each user display control means for controlling to cause shown, have a, wherein the display control unit, a user who watch the security level is high region of the screen image And controlling so as to separate can be displayed.

The method of the present invention is connected in a signal passing the User chromatography The terminal, a control method of a management server to be displayed before the display device the status of Kiyu over The terminal screen image acquisition means, before Symbol and the screen image acquisition step of acquiring a screen image Yu chromatography the terminal, confidential level information acquisition means, and the security level information acquisition step of acquiring security level information security level in the screen image has identified a high region, The line-of-sight information acquisition means corresponds to the screen image and acquires line-of-sight information for each user of line-of-sight information that the user gazes at the screen image, and the display control means is based on the acquired confidentiality level information and line-of-sight information. Te, in order to display the information to visualize whether the security level is high area of the screen image is a user of the user terminal and gaze control A display control step, a possess that, said display control step, wherein the secret level of the screen image is controlled so as to distinguishably display the user who watch the high region.

Program of the present invention, there is provided a User chromatography The terminal and is connected in a signal passing, a program executable in the management server that previously displayed on the display device the status of Kiyu over The terminal, the management server, before Symbol and the screen image acquiring means for acquiring a screen image Yu chromatography the terminal, the security level information obtaining means for obtaining confidential level information security level in the screen image has identified a high region, corresponding to the screen image, Based on the line-of-sight information acquisition means for acquiring the line-of-sight information that the user gazes on the screen image for each user, and based on the acquired security level information and line-of-sight information, the user of the user terminal determines a region with a high security level of the screen image. information to visualize or not gaze to function as a display control means for controlling to cause displayed, wherein the display control unit, wherein Wherein the secret level of the clean image is controlled so as to distinguishably display the user who watch the high region.

  According to the present invention, it is possible to confirm the possibility of information leakage by indicating whether or not the user is paying attention to a display area with a high security level for the content displayed on the screen.

It is a system configuration figure showing an example of the composition of the telework management support system in the present invention. FIG. 2 is a block diagram illustrating a hardware configuration of an information processing apparatus applicable to the management server 101, the teleworker PC 111, and the administrator PC 121 illustrated in FIG. It is a block diagram which shows an example of a function structure of the telework management assistance system in this invention. It is a flowchart which shows an example of the security monitoring process of the telework management assistance system in this invention. It is a flowchart which shows an example of the state determination detailed process of the telework management assistance system in this invention. It is a flowchart which shows an example of the confidential level determination detailed process of the telework management assistance system in this invention. It is a flowchart which shows an example of a gaze point estimation detailed process of the telework management assistance system in this invention. It is a flowchart which shows an example of the trail display detailed process of the telework management assistance system in this invention. It is a flowchart which shows an example of the trail summary display detailed process of the telework management assistance system in this invention. It is an example of parameter setting of the telework management support system in the present invention. It is a data block diagram which shows an example of the gaze destination information of the telework management assistance system in this invention. It is a data block diagram which shows an example of the screen image information of the telework management assistance system in this invention, display area information, and confidential level information. It is a screen image which shows an example of the trail summary screen of the telework management assistance system in this invention. It is a screen image which shows an example of the trail detail screen of the telework management assistance system in this invention. It is a screen image which shows an example of the trail detail screen of the telework management assistance system in this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram showing an example of a configuration of a telework management support system 100 to which the personal authentication process of the present invention is applied.

  The telework management support system 100 is configured such that one or more management servers 101, one or more databases 102, one or more teleworker PCs 111, one or more administrator PCs 121 are connected via the Internet 130. ing.

  The management server 101 is a server that centrally manages the teleworker's absence status, peeping by others, and impersonation trail data as telework information, and is constructed on the service environment network 104. The teleworker PC 111 and the administrator PC 121 are connected to the management server 101 by an authentication process using an account ID and a password. When the management server 101 receives telework information from the teleworker PC 111, it stores it in the database 102. Further, when there is an acquisition request from the administrator PC 121, necessary telework information is extracted from the database 102.

  The database 102 is a server that stores telework information based on the operation of the management server 101, and is constructed on the service environment network 104. The database 102 stores an account ID and password for connection to the management server 101 in addition to the telework information.

  The database 102 permits connection only from the management server 101 and cannot connect from the outside.

  The teleworker PC 111 is a terminal that creates trail data that is the basis of telework information, and exists on the home network 110. The trail data is created by a dedicated application and transmitted to the management server 101 via the router 112, the Internet 130, and the router 103. The personal authentication process of this embodiment operates as a function in this dedicated application.

  The administrator PC 121 is a terminal for confirming telework information, and exists on the in-house network 120. For confirmation of telework information, a web management console (operating on a web browser) is used to connect to the management server 101 via the router 122, the Internet 130, and the router 103.

  Hereinafter, the hardware configuration of the information processing apparatus applicable to the management server 101, the database 102, the teleworker PC 111, and the administrator PC 121 illustrated in FIG. 1 will be described with reference to FIG.

  FIG. 2 is a block diagram illustrating a hardware configuration of an information processing apparatus applicable to the management server 101, the database 102, the teleworker PC 111, and the administrator PC 121 illustrated in FIG.

  In FIG. 2, reference numeral 201 denotes a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, the ROM 202 or the external memory 212 is necessary to realize a BIOS (Basic Input / Output System), an operating system program (hereinafter referred to as OS), which is a control program of the CPU 201, and a function executed by each server or each PC. Various programs to be described later are stored.

  A RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for execution of processing from the ROM 202 or the external memory 212 to the RAM 203 and executing the loaded program.

  An input controller 205 controls input from a keyboard (KB) 209, a web camera 210, a pointing device such as a mouse (not shown), and the like. A video controller 206 controls display on the display 211. The display 211 is a display device such as a CRT or a liquid crystal display.

  A memory controller 207 is provided in an external storage device (hard disk (HD)), flexible disk (FD), or PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, and the like. Controls access to an external memory 212 such as a compact flash (registered trademark) memory connected via an adapter.

  A communication I / F controller 208 is connected to and communicates with an external device via a network (for example, the network 104 shown in FIG. 1), and executes communication control processing on the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the display 211 by executing outline font rasterization processing on a display information area in the RAM 203, for example. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the display 211.

  Various programs to be described later for realizing the present invention are recorded in the external memory 212, and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. Further, a setting file used when executing the program is also stored in the external memory 212, and detailed description thereof will be described later.

  FIG. 3 is a functional block diagram for explaining the functions of the management server 101, database 102, teleworker PC 111, and administrator PC 121 shown in FIG.

  In the telework management support system 100, the teleworker PC 111 mainly performs labor monitoring by personal authentication processing, the management server 101 and the database 102 manage monitoring information, and the administrator PC 121 displays monitoring information.

  The teleworker PC 111 includes an authentication control unit 311, an image acquisition unit 312, an event detection unit 313, a screen display content acquisition unit 314, a screen display content analysis unit 315, a line-of-sight analysis unit 316, and a trail data transmission control unit 317.

  The teleworker PC 111 uses a dedicated application, and the authentication control unit 311 performs personal authentication processing from an image taken by the web camera 210. The authentication control unit 311 establishes a communication connection with the management server 101 using the account ID and password of the authenticated user. After establishing a connection with the management server, the labor and security of the teleworker are monitored.

  The image acquisition unit 312 acquires an image of a space facing the screen of the display 211 of the teleworker PC using the web camera 210 or a monitoring camera (not shown).

  The event detection unit 313 identifies a person shown in the image based on the image acquired by the image acquisition unit 312, and detects states such as leaving the desk, looking into the screen by another person, and impersonating the person.

  The screen display content acquisition unit 314 acquires the display content of the screen of the display 211 regularly or irregularly such as when an event is detected.

  The screen display content analysis unit 315 analyzes the importance of the content displayed on the screen based on the display content acquired by the screen display content acquisition unit 314.

  The line-of-sight analysis unit 316 detects the line of sight of the person identified by the event detection unit 313 using a known line-of-sight detection technique based on the line-of-sight detection image acquired by the web camera 210 or an infrared camera (not shown). Then, the part of the display content acquired by the screen display content acquisition unit 314 is specified.

  The trail data transmission control unit 317 combines the event content by the event detection unit 313, the display content by the screen display content acquisition unit 314, the analysis result by the screen display content analysis unit 315, and the line-of-sight position information by the line-of-sight analysis unit 316. 101 is transmitted as trail data.

  The management server 101 includes a trail data reception control unit 301, a trail data registration unit 302, and a trail display control unit 303.

  In the management server 101, when the trail data is transmitted from the teleworker PC 111, the trail data reception control unit 301 receives the data, and the trail data registration unit 302 stores it in the trail data storage unit 304 of the database 102. When receiving the trail data, the management server 101 may notify the user of the administrator PC 121 by e-mail.

  When there is a trail display request from the administrator PC 121, the trail display control unit 303 creates display data based on the trail data stored in the trail data storage unit 304, and transmits the display data to the administrator PC.

  The administrator PC 121 has a trail display unit 321.

  The trail display unit 321 is realized by a web management console that accesses a dedicated URL from a web browser, and transmits a trail display request to the management server 101 to receive display data and display trail information. In addition to the trail information, the web management console can check the teleworker photo used for the personal authentication process, the communication status to the management server of the teleworker PC 111, and the like.

  In this configuration example, the event detection unit 313, the screen display content analysis unit 315, and the line-of-sight analysis unit 316 have the configuration of the teleworker PC 111. However, the image acquired on the teleworker PC 111, the screen display content, and the line-of-sight detection The management server 101 may be configured to perform processing based on the business image.

  Further, the teleworker PC 111 may have the function of the trail display control unit 303, and display data created based on the trail data may be transmitted to the management server 101 and registered.

  In addition, the administrator PC 121 has a function of the trail display control unit 303, creates display data based on the trail data stored in the trail data storage unit of the management server 101, and displays the trail information. Also good.

  Hereinafter, the security monitoring process in the telework management support system 100 of the present embodiment will be described with reference to FIG.

  This process estimates the line of sight on the display when a person other than the person (third party) looks into the screen or impersonates the person from the image of the webcam installed on the PC 111 for the teleworker. It is acquired as a trail which display area the user is interested in. When a person other than the teleworker looks into the display, the screen image, the line-of-sight position information, and the confidential level information are transmitted as trail information to the management server.

  As a point of this processing, when a person other than the teleworker himself peeks at the display or impersonates himself, the estimated position of the line of sight on the person's display, the content displayed on the display, and its confidential level are used as trail information. To obtain.

  The estimated line-of-sight position on the display indicates an estimated position on which area on the display the line of sight is directed to other than the teleworker himself. The confidentiality level indicates the degree of confidentiality calculated based on the attribute of the file displayed on the display and the displayed character information.

  Below, with reference to FIG. 4, each step of the gaze point estimation process of this embodiment is demonstrated.

  In step S401, the teleworker PC 111 reads various parameters from the setting file. Each read parameter is stored in the RAM 203 and referred to when necessary. A description example of each parameter setting file will be described with reference to FIG.

  In step S402, the authentication control unit 311 establishes a communication connection with the management server so that the trail information can be transmitted in S417. To establish the connection, the account ID and login password for the teleworker system login registered in advance in the management server are used.

  In step S <b> 403, the image acquisition unit 312 acquires an image from the web camera 210. The acquired image is stored in the RAM 203 and used in steps S501, S502, S601, S602, and S603.

  In step S404, the event detection unit 313 determines the state from the image acquired in step S403. Details of the state determination process will be described with reference to FIG.

  In step S405, processing is determined according to the determination state. If the determination state is “peeps” or “spoofing”, which has a security problem, the process proceeds to step S406. On the other hand, in the case of “present seat” or “leave away” with no security problem, the process waits until the next process d execution timing in step S412.

  In this example, a trail is acquired only for “peek” or “spoofing”, but a trail is also acquired for “attended” for security monitoring and confirmation of work status. May be used.

  In step S406, the screen display content acquisition unit 314 determines whether the determination state and the screen display content have changed. If the determination state is different between the previous process and the current time, or if the display content of the screen is different, the process proceeds to step S407 to newly acquire the screen display content. If the determination state and the display content are the same, the process proceeds to step S410.

  Here, in a situation where the screen is scrolled when determining the change in the display content of the screen, if the processing cycle is short, the screen display content may be acquired as usual when the display content of the screen has changed. However, when the processing cycle is long, it is difficult to estimate the line-of-sight destination during screen scrolling, so the screen display content may be acquired when screen scrolling is completed.

  Also, if the display content is the same as a result, for example, when another window is once displayed in front and the window is hidden, the display content of the previous screen continues. The processing may be performed assuming that As a result, it is possible to more accurately obtain the result of gazing at the same information.

  In step S407, the screen display content acquisition unit 314 acquires the content displayed on the screen of the display 211 of the teleworker PC 111 from the OS or the like. The information to be acquired includes a screen image of the screen, an application activated in the displayed window, an open file, and the like. Screen image information and display area information acquired in this step will be described with reference to FIGS. 12 (a) and 12 (b), respectively.

  In step S408, the screen display content analysis unit 315 determines the security level based on the display content of the screen acquired in step S407. Details of the security level determination processing will be described with reference to FIG.

  In step S409, the trail data transmission control unit 317 transmits the screen display content acquired in step S407 and the confidential level information determined in step S408 to the management server 101, and the management server 101 registers them in the database 102. In this example, the screen display content is transmitted every time it is acquired. However, the data may be accumulated and transmitted at a time such as regular or irregular, event occurrence, or the like. Detailed examples of information to be transmitted will be described with reference to FIGS.

  In step S410, the line-of-sight analysis unit 316 performs line-of-sight destination estimation processing on the face shown in the webcam image, calculates which area on the PC screen is being viewed, and creates line-of-sight position information. Details of the line-of-sight destination estimation processing will be described with reference to FIG.

  In step S411, the trail data transmission control unit 317 transmits the line-of-sight position information created in step S410 to the management server 101, and registers it in the database 102 in the management server 101. In this example, transmission is performed every time the line-of-sight position information is acquired. However, data may be accumulated and transmitted at a time such as regular or irregular, event occurrence, or the like. A detailed example of information to be transmitted will be described with reference to FIG.

  In step S <b> 412, the image acquisition unit 312 performs sleep processing for time adjustment until the next image acquisition timing. As the sleep time, the processing cycle parameter read in S401 is used.

  In step S413, it is determined whether to end the security monitoring process. To end the security monitoring process, the teleworker himself / herself explicitly commands from the menu or the like. If the security monitoring process is not terminated, the process returns to step S403 again.

  Hereinafter, the state determination process (S404) of the present embodiment will be described with reference to FIG.

  The event determination unit 313 executes the state determination process.

  In step S501, a face is detected from the image acquired in step S403. An external library is used for face detection, and the number of faces and the position on the image are specified. The detection result is stored in the RAM 203 and used in subsequent processing.

  In step S502, the state is determined based on the face detection result in step S501. When the number of detected faces is 0 (not detected), it is determined as “seated” in step S503 described later. If the number of detection is 1, face authentication processing for confirming whether or not the person is the teleworker is performed in step S505 described later. If the number of detections is two or more, it is determined as “peep” in S504 described later.

  In step S503, the determination state is set to “seated”. The determination state is stored in the RAM 203 and used in subsequent processing.

  In step S504, the determination state is set to “peep”. The determination state is stored in the RAM 203 and used in subsequent processing.

  In step S505, the degree of similarity is calculated as to whether the face shown in the webcam image and the teleworker are the same person by face authentication processing. An external library is used for the face authentication process. The calculated similarity is stored in the RAM 203 and used in subsequent processing.

  In step S506, the similarity result of the face authentication process is compared with the similarity threshold read in step S401, and it is determined whether the face shown in the image is the teleworker himself. If the degree of similarity is greater than or equal to the threshold value, it is determined that the user is the person himself / herself, and the presence is determined in step S507. If the degree of similarity does not satisfy the threshold value, it is determined as someone else, and it is determined as “spoofing” in step S508.

  Hereinafter, the confidential level determination process (S408) of the present embodiment will be described with reference to FIG.

  In the security level determination process, the security level (in other words, the importance level) is calculated based on the display content of the screen acquired in step S407. The secret level is expressed by an integer of 1 to 4, and if it is 1, it indicates that the most confidential content is displayed. If it is 4, it indicates that content with low confidentiality was displayed.

The security level determination process is executed by the screen display content analysis unit 315.
Below, the process of each step is demonstrated.

  In step S601, a display area list is acquired from the acquired screen display content. The display area basically indicates each window, but may be a desktop other than the window. The acquired display area list information is stored in the RAM 203 and used in step S602.

  In step S602, it is determined whether the confidential level calculation process has been performed on all display areas. If there is an unprocessed display area, display position information of the display area is acquired in step S603. If all the display areas have been processed, the secret level value is registered in the secret level information in step S606.

  In step S603, information on which position on the screen the display area to be processed is displayed is acquired from the display area list information. The acquired display position information is stored in the RAM 203 and used in step S604.

  In step S604, character information displayed in the display area is extracted using the display position information of the display area. An external library is used to extract character information. The extracted character information is stored in the RAM 203 and used in step S605.

  In step S605, the extracted character information is analyzed, and the security level of the display content in the display area is calculated. The confidential level is calculated as an integer of 1 to 4 from the analysis result of the character information displayed in the display area. The value of the confidential level indicates that 1 means the most confidential content, and 4 indicates the lowest content. The calculation unit includes the entire display area, a paragraph unit, a sentence unit, a word unit, and the like, and may be a combination thereof.

  For the analysis of the confidential level, a dictionary database in which confidential terms are registered in advance is used. For example, a usage method such as counting the number of appearances of terms registered in the dictionary database or combining a Bayes classifier with a dictionary database is conceivable.

  When the display content is a chart, the security level is calculated from sentences and paragraphs explaining the chart. For example, a calculation method may be considered in which the confidential level of a paragraph in which a chart number is written is calculated so as to be used as the confidential level of the chart.

  Further, the security level of the display area may be determined from the attribute of the file opened in the display area.

  The calculated security level is stored in the RAM 203 and used in step S606.

  In step S606, the confidential level acquired in step S605 is registered as part of the trail information. When registering the confidential level, the calculation unit is converted into coordinate information on the screen so that the display position can be specified. A detailed example of the security level information will be described with reference to FIG.

  Hereinafter, the line-of-sight destination estimation process (S410) of the present embodiment will be described with reference to FIG.

In the line-of-sight destination estimation process, it is estimated to which area of the screen the line of sight is directed with respect to all faces including the teleworker who is reflected in the webcam image, and is calculated as line-of-sight position information.
The line-of-sight analysis unit 316 executes the line-of-sight estimation process.
Below, the process of each step is demonstrated.

  In step S <b> 701, it is determined whether the line-of-sight estimation process has been performed on all the faces reflected in the webcam image. If there is an unprocessed face, the degree of similarity with the teleworker is calculated by face authentication processing in step S702. If all the faces have been processed, the line-of-sight position information is registered as part of the trail information in step S705.

  In step S <b> 702, face authentication is performed, a similarity degree is calculated as to whether the face to be processed and the teleworker are the same person, and it is determined whether the person is the teleworker himself. By determining whether the person is the teleworker himself / herself, the gaze position information can be leaked and the information can be narrowed down and also used for managing the work situation of the teleworker.

  An external library is used for the face authentication process. The calculated similarity is stored in the RAM 203 and used in step S705.

  In step S703, line-of-sight detection is performed on the face to be processed. An external library is used for gaze detection. The detection result is stored in the RAM 203 and used in step S705.

  In step S704, based on the result of eye gaze detection, the position on the screen image where the face eye gaze is located is calculated as eye gaze position information. The line-of-sight position information is expressed by pixel coordinates in the X direction and the Y direction. The calculation result is stored in the RAM 203 and used in step S705.

In step S705, the calculated line-of-sight position information is registered as part of the trail information.
A detailed example of the line-of-sight position information will be described with reference to FIG.

  Hereinafter, the trail display process of the present embodiment will be described with reference to FIG.

  In the trail display process, after selecting an event to be confirmed from the list of acquired events, the third party's line of sight when a spoofing and spoofing event occurs from the screen image, line-of-sight position information, and confidential level information that are trail information. The tip position is displayed on the heat map. The heat map is displayed in a darker color as the gaze time is longer, that is, as the detected gaze position density is higher, and in a lighter color as the gaze time is shorter.

  The display screen includes a trail summary screen (FIG. 13) and a trail detail screen (FIG. 14), and the display can be switched by performing a screen operation. Detailed examples of each screen will be described with reference to FIGS.

  The trail summary screen is a screen for confirming the trail information acquired when an event occurs. On this screen, images (line-of-sight heat map images) on which a screen image and a heat map are superimposed are displayed in a list in thumbnail (reduced image) format and can be continuously confirmed. Thereby, it is visualized which area of which screen the third party was gazing at.

  Furthermore, the color of the heat map changes according to the confidential level of the display content. A color is assigned so that the security level can be judged at a glance, such as a conspicuous red color for the area 1 with the highest security level and a light yellow color for the area 4 with the lowest security level.

  In contrast to the above method, the length of the gaze time may be represented by a color difference and the confidential level may be represented by shading. Further, the confidential level may be represented by a symbol or pattern, and the length of the gaze time may be represented by the size of the symbol or pattern, as long as each content can be identified.

  In this example, the heat map is displayed only for the third party, but the teleworker himself can change the color, symbol, etc. to display the heat map so that it can be distinguished from the third party. You may do it. In addition, when there are a plurality of third parties, the heat map may be displayed so that the person can be distinguished by changing the color or the symbol.

  When a thumbnail displayed in the list is selected with a mouse or the like on the trail summary screen, the screen is changed to the trail detail screen by another window display.

  The trail detail screen is a screen for confirming details of display contents leaked to a third party. On this screen, the screen display content, that is, the screen image and the heat map are superimposed and displayed.

The trail display process is executed by the trail display control unit 303 and displayed on the trail display unit 321, that is, the web management console.
Below, the process of each step is demonstrated.

  In step S801, a list of events sent to the management server is displayed. The user can confirm the contents by selecting an event to be confirmed from the event list.

  In step S802, a user operation in the event list is determined. If one of the events in the event list is selected, the process proceeds to step S803. When the “Home” button is selected, the display of the event list is terminated.

  In step S803, trail information of the event selected in step S802 is acquired. The trail information includes all screen images acquired during the event occurrence period, line-of-sight position information, and confidential level information. The acquired trail information is stored in the RAM 203 and used in steps S804 and S806.

  In step S804, the trail information of the event selected in step S802 is displayed as a summary. Details of the trail information summary display processing will be described with reference to FIG.

  In step S805, a user operation in the trail summary display is determined. If one is selected from the list of thumbnails displayed in the trail summary, the details of the trail information are displayed in step S806. When the “Return” button is selected, the display returns to the event list display in S801.

  In step S806, the trail information associated with the thumbnail selected in step S805 and the line-of-sight heat map image are displayed in a separate window as trail details.

  In step S807, the user operation in the trail detail display is determined. When the “Return” button is selected, the display returns to the trail summary display in S804.

  Hereinafter, the trail summary display process of this embodiment will be described with reference to FIG.

  In the trail summary display process, a list of thumbnails of the line-of-sight heat map image for the event selected in step S802 is mainly displayed.

The trail summary display process is executed by the trail display control unit 303 and displayed on the trail display unit 321, that is, the web management console.
Below, the process of each step is demonstrated.

  In step S901, a screen image list of the event is acquired based on the trail information of the event selected in step S802. The acquired all-screen image is stored in the RAM 203 and used in step S906.

  In step S902, line-of-sight information during the event occurrence period is acquired from the trail information of the event selected in step S802. The line-of-sight information includes a pair of screen image IDs, a line-of-sight detection time, and pixel coordinates serving as a line-of-sight destination on the screen image. The acquired line-of-sight information is stored in the RAM 203 and used in steps S903 and S905.

  In step S903, the gaze time for each line of sight is calculated based on the line-of-sight information extracted in step S902. For the gaze time, the time until the next gaze detection time is calculated in milliseconds. At the end of the event, the next line-of-sight information does not exist, so only the event end time is calculated as the gaze time. The calculated gaze time is stored in the RAM 203 and used in step S905.

  In step S904, confidential level information is acquired from the trail information of the event selected in step S802. The security level information is composed of the ID of the screen image to be paired, the application area displayed in the screen, and its security level. Obtained based on the ID of each screen image obtained in step S901. The acquired security level information is stored in the RAM 203 and used in step S905.

  In step S905, a heat map corresponding to each screen image is generated based on the line-of-sight information, gaze time, and confidential level information acquired and calculated in steps S902, S903, and S904.

  An external library is used to generate the heat map. For an external library, a heat map is generated for all the lines of sight to be processed by giving shade parameters calculated from pixel coordinates that are line-of-sight position information, gaze time, and confidential level information.

  The density parameter is calculated based on the pixel coordinates of the line-of-sight position, application display area information, and the confidential level. The density is set according to the length of the gaze time, and the parameters are set to change the color according to the confidential level. If the pixel coordinates are outside the display area, the density parameter is set as the confidential level 4.

  The generated heat map is stored in the RAM 203 and used in step S906.

  In step S906, thumbnails of images (line-of-sight destination heat map images) in which the heat map generated in step S905 and the screen image are displayed in a superimposed manner are displayed as a list. The image size of the list display can be changed by a user operation. About a heat map, the presence or absence of a display can be switched by user operation.

  By selecting one thumbnail from the displayed thumbnails, the related trail detailed information can be displayed.

  Hereinafter, a description example of a setting file in the teleworker PC according to the present embodiment will be described with reference to FIG.

  FIG. 10 is a setting example of the size of the web camera image, the threshold value of the similarity at the time of personal authentication, and the cycle of personal authentication processing. In this example, each setting item is set by surrounding it with a tag.

  The size of the webcam image is specified by an image_size tag 1001. Specify in pixels in the format of “horizontal x vertical”.

  The similarity threshold at the time of personal authentication is specified by a similarity_threshold tag 1002. If the similarity calculated during the authentication process falls below the threshold set here, it is determined that the person is another person. From the viewpoint of false negative, the specifiable range is 0.5 to 1.0.

  The cycle of the personal authentication process is designated by a processing_cycle tag 1003. The unit of the specified value is milliseconds.

  Hereinafter, a data configuration example of the line-of-sight position information of the present embodiment will be described with reference to FIG.

  The line-of-sight position information includes the line-of-sight position information ID, the ID of the screen image targeted for line-of-sight detection, the gaze position, the detection time, and the presence / absence of the teleworker's own line of sight.

  The ID of the line-of-sight position information is designated by “line-of-sight ID” 1101. This ID is unique throughout the system.

  The ID of the screen image that is the target of eye-gaze detection is designated by “image ID” 1102. It is assumed that the screen image specified by this ID always exists on the system. The relationship between the line-of-sight position information and the screen image is assumed to be many-to-one.

  The gaze position of the line of sight is specified by XY coordinates. The X coordinate is designated by “X” 1103 and the Y coordinate is designated by “Y” 1104. The value that can be specified is a pixel value.

  The eye gaze detection time is designated by “time” 1105. In this example, the time is specified by UNIX (registered trademark) time.

  The presence / absence of whether or not the target of eye gaze detection is the teleworker is specified by “teleworker” 1106. The value that can be specified is 1 or 0. The case of 1 represents the line-of-sight information of the teleworker, and the case of 0 represents the line-of-sight information of a person other than the teleworker.

  Hereinafter, description examples of screen image information, display area information, and confidential level information according to the present embodiment will be described with reference to FIG.

  FIG. 12A shows a data configuration example of screen image information.

  The screen image information is composed of the ID of the screen image, the ID of the terminal that acquired the image, the user ID of the teleworker, the screen image acquisition time, and the stored file information.

  The screen image ID is designated by “image ID” 1201 and is unique throughout the system.

  The ID of the terminal that acquired the image is designated by “terminal ID” 1202 and the computer name and MAC address of the terminal that acquired the screen image are set.

  The user ID of the teleworker is designated by “user ID” 1203.

  The screen image acquisition time is designated by “time” 1204 and expressed in UNIX (registered trademark) time.

  The stored file information is designated by “image file” 1205, and an image file name including the folder path storing the screen image is set.

  FIG. 12B is a data configuration example of display area information.

  The display area information includes a screen image ID, a display area ID in the screen image, a running program, an open file, a display reference position, a display size, and active information.

  The screen image ID is designated by “image ID” 1211.

  The ID of the display area in the screen image is specified by “area ID” 1212 and is unique to the image ID 1211. Basically, each window is set, but a desktop other than the window or the entire screen image may be set as one display area.

  The activated program is designated by “program” 1213, and the name and program ID of the activated program are set in the display area.

  An open file is designated by “file” 1214, and an open file name is set in the display area.

  The display reference position is designated by XY coordinates with the upper left corner of the display range as a reference. The X coordinate is designated by “X” 1215 and the Y coordinate is designated by “Y” 1216. The value that can be specified is a pixel value.

  The display size is specified by length and width. The vertical length is specified by “vertical width” 1217, and the horizontal length is specified by “horizontal width” 1218. The value that can be specified for the length is a pixel value.

  The active information is designated by “active” 1219 and indicates whether the display area is being operated by the user. The value that can be specified is 1 or 0. A case of 1 indicates that the user is operating the display area.

  FIG. 12C is a data configuration example of confidential level information. The confidential level information includes an ID of confidential level information, an ID of a screen image for which the confidential level is calculated, an ID of a display area, a reference position for calculating the confidential level, a display range, and a calculated confidential level.

  The ID of the confidential level information is designated by “confidential ID” 1221. This ID is unique for the entire system, and is set for each security level calculation unit (the entire display area, paragraph, sentence, word, etc.).

  The ID of the screen image that is the target of the security level calculation is designated by “image ID” 1222. It is assumed that the screen image specified by this ID always exists on the system.

  The ID of the display area is designated by “area ID” 1223 and corresponds to “area ID” 1212 of the display area information. This means that the confidential level is calculated for the entire display area or a part thereof.

  The reference position for the security level calculation is designated by XY coordinates with the upper left corner of the display range of the security level calculation unit as a reference. The X coordinate is designated by “X” 1224 and the Y coordinate is designated by “Y” 1225. The value that can be specified is a pixel value.

  The display range is specified by the length and width of the security level calculation unit. The vertical length is specified by “vertical width” 1226 and the horizontal length is specified by “horizontal width” 1227. The value that can be specified for the length is a pixel value.

  The calculated security level is designated by “security level” 1228. The secret level is specified in the range of an integer from 1 to 4. The confidential level is the value calculated in step S605.

  Hereinafter, screen image examples of the trail summary screen and the trail detail screen of this embodiment will be described with reference to FIGS. 13 and 14.

  FIG. 13 is a screen image example of the trail summary screen.

  The trail summary screen 1300 includes a teleworker list 1301, an event list 1302, and a trail summary 1303. Further, a calendar or the like may be displayed as shown in the figure.

  The teleworker list 1301 displays a list of teleworkers managed in the telework management support system 100, and a teleworker for which an event is to be displayed can be selected from the list.

  The event list 1302 displays a list of events corresponding to the selected teleworker. Here, the event corresponds to the determination state determined in step S404. You can select an event for which you want to display a trail from the list.

  The trail summary 1303 displays a list of trails corresponding to the selected event. The trail to be displayed is a thumbnail image 1304 of the line-of-sight heat map image. By selecting a trail to be displayed in detail from the list, a trail detail screen can be called.

  FIG. 14 is a screen image example of the trail detail screen.

  The trail detail screen 1400 includes a trail explanation list 1401 and a trail detail 1402.

  The trail explanation list 1401 displays explanation information for the target trail, and displays the classification of the event, the date and time of occurrence of the event, the activated application (program), the security level, and the like.

In the trail details 1402, the original image corresponding to the thumbnail image selected in the trail summary 1303 is enlarged and displayed. Since the screen image and the heat map are displayed in a superimposed manner, it is possible to confirm the specific content that the third party is watching.
FIG. 15 shows another display form of the trail detail screen.

  FIG. 15A shows an example of a display form in which a display area with a high security level is highlighted when the user has watched the display area for a certain period of time or longer. The display area 1501 has a confidentiality level of “1”, and the display area is shaded and displayed as 1502 by being watched by a user (for example, a third party) for a predetermined time (for example, “5 seconds”) or longer. The The highlighting method may be a format that can be recognized by the administrator, such as inversion and color painting.

  FIG. 15B is an example of a display form in which a message is displayed on the display area when a high-confidential display area is watched by a user for a certain period of time or longer. The display area 1511 has a confidentiality level “1”, and a message 1512 is displayed on the display area 1511 when the user (for example, a third party) has watched for more than a certain time (for example, “5 seconds”). The The message 1512 may be output with information that allows the administrator to recognize the fact that the user is watching, such as the confidential level, the name of the user who was watching, and the watching time.

  As described above, it is possible to confirm whether or not the user is gazing at the display content with a high confidential level.

  It should be noted that the configuration and contents of the various data described above are not limited to this, and it goes without saying that the various data and configurations are configured according to the application and purpose.

  In addition, the program according to the present invention is a program that causes a computer to execute the processes in FIGS. 4, 5, 6, 7, 8, and 9. The program in the present invention may be a program for each process in FIGS. 4, 5, 6, 7, 8, and 9.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The present invention may be applied to a system constituted by a plurality of devices or an apparatus constituted by a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

100 Telework Management Support System 101 Management Server 102 Database Server 103 Router 104 Service Environment Network 110 Home Network 111 Teleworker PC
112 router 120 in-house network 121 PC for administrator
122 router 130 internet 200 computer 201 CPU
202 ROM
203 RAM
204 System bus 205 Input controller 206 Video controller 207 Memory controller 208 Communication I / F controller 209 KB
210 Web Camera 211 Display 212 External Memory 301 Trail Data Receiving Unit 302 Trail Data Registration Unit 303 Trail Display Control Unit 304 Trail Data Storage Unit 311 Authentication Control Unit 312 Image Acquisition Unit 313 Event Detection Unit 314 Screen Display Content Acquisition Unit 315 Screen Display Content Analysis unit 316 Gaze analysis unit 317 Trail data transmission control unit 321 Trail display unit

Claims (6)

User chromatography The terminal and connected communication capable, a management server to be displayed on the display device the situation before Kiyu over The terminal,
And the screen image acquiring means for acquiring a screen image before Kiyu over The terminal,
Security level information acquisition means for acquiring security level information specifying a region with a high security level in the screen image;
Line-of-sight information obtaining means for obtaining line-of-sight information for each user corresponding to the screen image,
Based on the obtained security level information and the viewing information, and a display control means for the security level is high area of the screen image is a user of the user terminal to control in order to display the information to visualize whether gazing,
I have a,
The management server according to claim 1, wherein the display control unit performs control so that a user who has watched an area with a high security level of the screen image is identifiable . User chromatography The terminal and connected communication capable, a management server to be displayed on the display device the situation before Kiyu over The terminal,
And the screen image acquiring means for acquiring a screen image before Kiyu over The terminal,
Security level information acquisition means for acquiring security level information specifying a region with a high security level in the screen image;
Line-of-sight information acquisition means for acquiring line-of-sight information that the user gazes at the screen image corresponding to the screen image;
Based on the obtained security level information and the viewing information, and a display control means for the security level is high area of the screen image is a user of the user terminal to control in order to display the information to visualize whether gazing,
I have a,
Wherein the display control unit, based on the obtained security level information and line-of-sight information, and displays the map information indicating the distribution of the line of sight by the user gaze confidential level higher region of the screen image before Kiyu over The terminal Management server characterized by controlling as necessary. Said display control means, the distribution of the line of sight, the concentration depending on the security level, color, symbol, to claim 2, wherein the controller controls to display the map information with different display patterns such as pattern The management server described. Screen image acquisition means for acquiring a screen image of the terminal;
Security level information acquisition means for acquiring security level information specifying a region with a high security level in the screen image;
Line-of-sight information obtaining means for obtaining line-of-sight information for each user corresponding to the screen image,
Display control means for controlling to display information for visualizing whether or not the user gazes at a high-security area of the screen image based on the acquired security level information and line-of-sight information;
Have
The information processing terminal characterized in that the display control means performs control so that a user who has watched an area with a high security level of the screen image is identifiable. User chromatography The terminal and connected communication capable, a control method of a management server to be displayed before the display device the status of Kiyu over The terminal,
Screen image obtaining means, a screen image obtaining step of pre-obtaining a screen image of Kiyu over The terminal,
A security level information acquisition means for acquiring security level information specifying a region with a high security level in the screen image;
The line-of-sight information acquisition means corresponds to the screen image, and acquires line-of-sight information for each user that is line-of-sight information that the user gazes at the screen image
Display control means, based on the obtained security level information and the viewing information, the display control step confidentiality level is a user of the high region the user terminal to control in order to display the information to visualize whether gazing of the screen image When,
I have a,
The method for controlling a management server, wherein the display control step performs control so that a user who has focused on an area with a high security level of the screen image is identifiable . User chromatography The terminal and is connected in a signal passing, a program executable in the management server that previously displayed on the display device the status of Kiyu over The terminal,
The management server
And the screen image acquiring means for acquiring a screen image before Kiyu over The terminal,
Security level information acquisition means for acquiring security level information specifying a region with a high security level in the screen image;
Line-of-sight information obtaining means for obtaining line-of-sight information for each user corresponding to the screen image,
Based on the obtained security level information and the viewing information, and a display control means for the security level is high area of the screen image is a user of the user terminal to control in order to display the information to visualize whether gazing,
To function as,
The program according to claim 1, wherein the display control means performs control so that a user who has watched an area with a high security level of the screen image is identifiable .

Images