JP2005084991A - Terminal user monitoring system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a terminal user monitoring system that keeps the accuracy of user matching from deteriorating with changes in the user's hairstyle or face over time which is used for matching. <P>SOLUTION: A face image data updating means D is provided which updates and stores the current extracted face image data in a face image data storage means J only if determination by a face image data comparison means B results in a rough match, so that the newest face image data of the user is updated and stored in the face image data storage means J at all times. When a determination is made as to whether or not the user being logged in can be authenticated, the face image data extracted at that time is compared with the newest face image data stored in the face image data storage means J so as to increase the accuracy of authenticating the user, while reducing identification errors due to changes in the user's hairstyle or face across the ages. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an improvement in a terminal user monitoring system that monitors the usage status of a computer terminal and applies usage restrictions to the terminal according to the usage status.

  As a means for preventing unauthorized use of a computer terminal, the use of a terminal is generally restricted by using a user ID, a password, or the like. However, there is a problem that once the user ID, password, etc. are leaked, the use of the computer terminal is allowed without limitation.

  Therefore, recently, it has been strongly desired to develop a technology that constantly monitors the usage status of a computer terminal and applies usage restrictions to the terminal as necessary.

  As a terminal user monitoring system that monitors the usage status of a computer terminal and restricts the use of the terminal, the user's face image data registered in advance and images are taken with a CCD camera or the like at predetermined intervals when the terminal is used For example, Japanese Patent Application Laid-Open No. H10-228667 discloses a method in which the operation of the terminal is automatically locked when the user's face image data is compared and the difference exceeds a predetermined threshold value.

  However, with such a technology, after a long time has passed since face image data was registered, the matching accuracy dropped due to changes in hairstyle and aging of the face, limiting the use of terminals by appropriate users, Alternatively, there is a possibility that a third party who should not allow the use of the terminal is mistaken as an appropriate user and the terminal is illegally used.

  On the other hand, as a technique for preventing the deterioration of matching accuracy due to the aging of the biometric features provided by the user and achieving the user's suitability determination, for example, as disclosed in Patent Document 2, for terminal operation A fingerprint reader is installed on the mouse, and the user's fingerprint is detected at predetermined intervals, and the authenticity of the user is determined by comparing this fingerprint with the user's fingerprint registered in advance. Things have been proposed.

  Certainly, if fingerprints that do not easily change shape due to aging etc. are used as collation means, the problem of degradation of collation accuracy may be improved, but in order to detect fingerprints constantly while using the terminal In this case, the user's finger must always be placed at a specific position on the mouse, and the movement and posture of the user are restricted, so that the workability of the terminal operation is significantly deteriorated.

  In addition, in a terminal that is exclusively used for calculation processing, there is actually little opportunity to operate the mouse, and it is difficult to detect the fingerprint of the user itself. In order to ensure the detection of the fingerprint, It is almost impossible in terms of technology and cost to install fingerprint readers on all the key tops of the keyboard and always read fingerprints.

  As described above, in the conventional security system, the collation accuracy deteriorates due to the change of the hairstyle used as the object of collation or the aging of the face, or the contact type reading with the fingerprint having a small secular change as the object of collation. Since it is necessary to perform authentication using a device, there is a problem that the user's operation and posture are restricted and the workability of the terminal operation is significantly deteriorated.

JP 2003-141088 (FIGS. 1 and 3) JP 2002-7351 (FIGS. 2 and 15)

  Accordingly, an object of the present invention is to solve the inconveniences of the above-described prior art, the collation accuracy is not deteriorated due to the change of the hairstyle used as the object of collation or the aging of the face, and the movement and posture of the user It is an object of the present invention to provide a terminal user monitoring system that can accurately determine the suitability of a user without restricting the user.

The present invention is a terminal user monitoring system that monitors the usage status of a computer terminal and applies usage restrictions to the terminal according to the usage status.
Imaging means installed at a position where the face of the terminal user can be photographed;
Face image data extraction means for analyzing the image captured by the imaging means and extracting face image data sufficient to identify a human face;
Face image data storage means for storing face image data of a user permitted to log in to the terminal;
Face image data comparing means for comparing the face image data repeatedly extracted by the face image data extracting means during login and the face image data stored in the face image data storage means to determine the degree of coincidence between the two,
Terminal lock means for restricting the processing operation using the terminal only when the result of determination by the face image data comparison means is inconsistent.

When such a configuration is applied, face image data of a user who is permitted to log in to the terminal is stored in advance in the face image data storage unit.
The imaging means for monitoring the usage status of the computer terminal is installed at a position where the face of the terminal user can be photographed, and while the terminal is in the login state, the face image data extraction means repeatedly operates and the imaging means The captured image is analyzed to extract face image data.
The face image data comparison means compares the face image data extracted by the face image data extraction means during login with the face image data stored in the face image data storage means, and determines the degree of coincidence between them.
If the result of determination by the face image data comparison means is inconsistent, the terminal lock means is activated to limit processing operations using the terminal.
In this way, the face image data that is a user's biometric feature is detected while maintaining a non-contact state with respect to the user, and this face image data is used as a target for collation. It is possible to accurately determine the suitability of the user without restricting the movement and posture of the user, and the burden on the user required for maintaining security is reduced.
In addition, since the usage status of the computer terminal is constantly monitored and the determination result by the face image data comparison means is inconsistent, the terminal lock means is activated immediately, so the terminal using the user ID, password, etc. Unlike the case of restricting the use of the device, it is difficult for an improper user who has stolen a user ID or password to become a legitimate user and to use the terminal intentionally. Invite other people to use the terminal by instructing the user ID, password, etc., and also prevent unauthorized acts such as unauthorized use of the terminal by another person while the appropriate user is out of the seat. Is done.

  In addition, in order to prevent the accuracy of matching from being deteriorated due to a change in hairstyle used as a target for matching or a secular change in face, the face image data extracted by the face image data extracting means at the time of login to the terminal is stored. It is desirable to configure the face image data storage means.

  If such a configuration is applied, the suitability of the user is determined using at least the face image data captured on the day as a comparison reference. It is possible to effectively prevent the problem that the collation accuracy is greatly deteriorated due to the change.

  Furthermore, in order to surely eliminate the deterioration of the collation accuracy, the face image data storage means for storing the face image data extracted by the face image data extraction means at the time of executing login to the terminal, and the determination by the face image data comparison means Only when the results are substantially the same, face image data update means for updating the face image data extracted this time in the face image data storage means and storing it is provided.

When such a configuration is applied, the latest face image data of the user is always updated and stored in the face image data storage means, and the face image data comparison means is updated based on the latest face image data. Since the authentication process regarding suitability is performed, the problem that the matching accuracy is deteriorated due to a change in hairstyle used as a target of matching or a secular change of the face can be prevented completely.
Note that, depending on the imaging cycle of the imaging means and the processing capability (required extraction time) of the face image data extraction means, the meaning of the change in hairstyle and the aging of the face in this case is the change in the facial expression of the user and the imaging. A variation element such as a change in the separation distance from the means to the user (specifically, a change in the size of the user's face on the imaging surface of the imaging means) can be included as one aspect of the secular change.
That is, if the imaging period of the imaging unit is sufficiently short and the processing capability of the face image data extraction unit is sufficient, the facial expression of the user changes drastically, or the user changes the posture, etc. Even if the size, position, and posture of the user's face on the imaging plane change, it is possible to determine the same person as the same person and another person as another person.

  In addition, when the face image data extracting means has a function of extracting two or more face image data from the image at the same time, when two or more face image data are simultaneously extracted by the face image data extracting means, they are automatically mismatched. It is desirable to configure the face image data comparison means so as to output the determination result.

  By applying such a configuration, it becomes possible to automatically limit the use of the terminal by detecting a third party looking into the display.

  Furthermore, it is also effective in terms of security to include a face image data transmission means for automatically transmitting the face image data extracted this time to another terminal when the judgment result by the face image data comparison means does not match. It is.

  If such a configuration is applied, a third party who tries to become a proper user or a third party who tries to look into the display illegally can be grasped by other terminals at remote locations. The unauthorized user of the terminal can be easily identified.

  It is also possible to provide a second face image data storage means for storing the face image data extracted this time together with log information when the result of determination by the face image data comparison means does not match.

When the second face image data storage means is provided, the face image data of a third party who has attempted to impersonate an appropriate user or who has illegally looked into the display is stored in the terminal. It is stored in the second face image data storage means.
Therefore, by confirming the contents of the second face image data storage means, it is possible to easily identify an unauthorized user as described above.
The log information can include various information that can be recognized internally by the terminal, including the time, the name of the file opened at the time, the name of the application program started at the time, and the like.
Therefore, in addition to specifying an unauthorized user, there is an advantage that a file or the like in which confidential information may be leaked can be easily specified.
Note that it is also possible to transfer the face image data and log information of a third party who is an unauthorized user to another terminal at a remote location by using the above-described face image data transmission means.

  Further, an alarm output means for outputting an alarm signal when the terminal lock means is operated may be provided.

If the alarm is activated by this alarm output, the surrounding people can be warned of unauthorized use of the terminal.
As the alarm means, a speaker incorporated in the terminal or an external device such as a warning light can be arbitrarily used.

  Further, it is desirable to provide a terminal unlocking means for detecting that the determination result by the face image data comparing means is substantially coincident after the operation of the terminal locking means and releasing the operation of the terminal locking means.

As described above, in the present invention, it is possible to realize appropriate authentication processing by absorbing variable factors such as changes in the facial expression of the user and changes in the separation distance from the imaging means to the user. If there is a special circumstance such as the person turning away from the imaging means, the use of the terminal may be unfairly restricted even though the appropriate user is using the terminal Is inherent.
When the result of the determination by the face image data comparison means becomes substantially coincident after the operation of the terminal lock means, the terminal lock release means that automatically releases the operation of the terminal lock means causes an inconvenience in the determination process. Even if it is a case, an appropriate user can continue to use the terminal normally without performing a troublesome operation such as inputting a user ID or password or re-inputting a fingerprint or a retina pattern. . As a result of reducing the burden on the appropriate user in this way, the efficiency of work using the terminal is also improved.

  Further, face image data display means for automatically displaying the face image data stored in the second face image data storage means on the display of the terminal when the terminal unlocking means is operated may be provided.

When a proper user is recognized and the use restriction of the terminal is released, the face image data stored in the second face image data storage means, that is, a third party who tries to impersonate the proper user Or since the information of the third party who tried to look into the display illegally is automatically displayed, it is possible to reliably identify the unauthorized user.
By applying such a configuration, it is possible to prevent inconvenience such as oversight of unauthorized use of the terminal caused by forgetting to look at log information.

  Further, the terminal unlocking means is provided with a scale determination function for determining the scale of the face image data extracted by the face image data extracting means, and when the scale of the extracted face image data is less than the set value, The terminal lock releasing means is maintained in an inoperative state regardless of the determination result by the face image data comparing means.

As described above, in the present invention, it is possible to realize appropriate authentication processing by absorbing variable factors such as changes in the facial expression of the user and changes in the separation distance from the imaging means to the user. Therefore, even if the user is away from the terminal, as long as the proper user's face is imaged by the imaging means, the terminal lock means does not start, or the proper user's face away from the terminal If recognized, there is a possibility that the terminal once locked is automatically released from the use restriction by the terminal unlocking means.
When the scale of the extracted face image data is less than the set value, the terminal unlocking means is maintained in an inactive state, so that the third party can look into the display or the terminal Unauthorized use can be surely prevented.

  Furthermore, it is possible to provide a monitoring cycle adjustment unit that randomly changes the setting values of the operation cycles of the imaging unit, the face image data extraction unit, and the face image data comparison unit during login.

  If such a configuration is applied, it becomes impossible to predict the operation timing of the imaging means, the face image data extraction means, and the face image data comparison means, that is, the means required for user monitoring. It is possible to significantly reduce the possibility that the terminal is illegally used in a period when the device is inactive.

  In addition, depending on the operating state of the terminal locking unit, when the terminal locking unit is inactive, a relatively long cycle is set as the operating cycle setting value of the imaging unit, the face image data extracting unit, and the face image data comparing unit. On the other hand, in the state where the terminal lock means is in operation, a second monitoring period adjusting means for setting a relatively short period setting value as the setting value of the operation period may be provided together with the value being set. .

When such a configuration is applied, the number of executions of image processing and determination processing per unit time under the situation where the terminal is used by an appropriate user is reduced, the load on the CPU is reduced, and application programs are used. Processing operations can be performed smoothly.
Also, once the use of the terminal is restricted and the internal processing load is reduced, the number of times image processing and judgment processing per unit time is automatically increased, so the appropriate user is seated in front of the terminal. It is possible to quickly detect this and operate the terminal unlocking means, and the unnecessary waiting time on the user side is eliminated, thereby improving workability.

  Furthermore, it is possible to provide a third monitoring period adjusting means for setting the operation period of the imaging means, the face image data extracting means, and the face image data comparing means corresponding to the type of application program being used.

  By applying such a configuration, for example, while using an application program that handles important data, the number of executions of image processing and determination processing per unit time is increased to ensure security rather than use efficiency of the application program. While giving priority, while using an application program that handles data with low confidentiality, reduce the number of times image processing or judgment processing is executed per unit time to give priority to the use efficiency of the application program, or an application with low load While using the program, increase the number of executions of image processing and judgment processing per unit time to ensure security, while using application programs that require high-speed processing, the number of executions of image processing and judgment processing per unit time Reduce It is possible, such as speed up the operation of the publication program.

  In addition, the imaging unit and the face image data extraction unit are instantly received in response to a collation request from the application program in use regardless of the setting values of the operation cycles of the imaging unit, the face image data extraction unit, and the face image data comparison unit. Further, a comparison determination forcing unit that operates the face image data comparison unit may be provided.

  If such a configuration is applied, the imaging unit, the face image data extraction unit, and the face image data comparison unit are forcibly driven by a collation request from the application program in use. By configuring so that a verification request is sent from the application program side at the time of display, it becomes possible to selectively execute data display permission or terminal use restriction according to the suitability of the user, and it is related to the terminal. Security related to data safety as well as access rights is improved.

The terminal user monitoring system according to the present invention detects face image data that is a biometric feature of a user while maintaining a non-contact state with respect to the user, and uses the face image data for determining the suitability of the user. Therefore, it is possible to accurately determine the suitability of the user without restricting the movement and posture of the user, and the burden on the user required for maintaining security is reduced.
In addition, since the usage status of the computer terminal is constantly monitored and the determination result by the face image data comparison means is inconsistent, the terminal lock means is activated immediately, so a user ID, password, etc. are used. Unlike the case of restricting the use of a terminal, it is extremely difficult for an inappropriate user to become a legitimate user and to use the terminal. It is also possible to reliably prevent an illegal act such as using the terminal or another person using the terminal while an appropriate user is out of the seat.

  In particular, by applying a configuration in which the face image data extracted by the face image data extraction unit at the time of executing login to the terminal is stored in the face image data storage unit and used as a comparison reference, at least the face image captured on that day It becomes possible to determine the suitability of the user using the data as a comparison criterion, and to effectively prevent the problem that the matching accuracy deteriorates due to a change in the hairstyle used as the target of matching or the aging of the face. it can.

  Furthermore, face image data updating means for updating and storing the face image data extracted this time in the face image data storage means only when the result of determination by the face image data comparison means is substantially coincident is provided. By applying a configuration in which the latest facial image data of the user is updated and stored in the storage means, the matching accuracy is improved due to changes in hairstyle and face changes that affect the facial image data used as the target of matching. Problems such as deterioration can be almost completely prevented, and furthermore, appropriate judgment results can be obtained by dealing with variable factors such as changes in the facial expression of the user and changes in the separation distance from the imaging means to the user. Is possible.

  In addition, when the face image data extracting means has a function of extracting two or more face image data from the image at the same time, when two or more face image data are simultaneously extracted by the face image data extracting means, they are automatically mismatched. Since the determination result is output, it is possible to automatically limit the use of the terminal by detecting a third party looking into the display.

  In addition, since the face image data transmitting means for automatically transmitting the face image data extracted this time to other terminals when the judgment result by the face image data comparing means is inconsistent is provided, it is imitated as an appropriate user. Such a third party or a third party who has illegally looked into the display can be grasped by another terminal at a remote location, and an unauthorized user of the terminal can be easily identified.

Further, since the second face image data storage means for storing the face image data extracted this time together with the log information when the result of the determination by the face image data comparison means is inconsistent is provided. By confirming, the unauthorized user can be easily identified as described above.
In this second face image data storage means, the time when the judgment result by the face image data comparison means does not match, the name of the file opened at the time, the name of the application program started at the time, etc. Since various kinds of information that can be recognized internally by the terminal can be included, in addition to specifying an unauthorized user, there is an advantage that a file or the like in which confidential information may be leaked can be easily specified.

  Furthermore, since alarm output means is provided to output an alarm signal when the terminal lock means is activated, the alarm is activated, so that unauthorized use of the terminal can be warned to the surroundings and the attention of surrounding people can be alerted. .

  In addition, since the terminal lock release means that detects that the result of the determination by the face image data comparison means substantially matches after the operation of the terminal lock means and releases the operation of the terminal lock means is also provided, proper use Even if the user is using the terminal improperly, even if the use of the terminal is unfairly restricted, the user ID and password can be entered without re-inputting the fingerprint or retina pattern The user can continue to use the terminal and the burden on the user required for maintaining security is reduced, so the efficiency of work using the terminal is improved.

  Further, since the face image data display means for automatically displaying the face image data stored in the second face image data storage means on the display when the terminal unlocking means is actuated, an appropriate user is recognized. Look at the face image data stored in the second face image data storage means, that is, a third party who has attempted to impersonate a proper user or illegally display when the use restriction of the terminal is lifted The third party can be identified with certainty, and inconveniences such as overlooking of unauthorized use of the terminal due to forgetting to see log information, etc. can be solved.

  Further, the terminal unlocking means is provided with a scale determining function for determining the scale of the face image data extracted by the face image data extracting means. When the scale of the extracted face image data is less than the set value, the face is unlocked. Regardless of the result of the determination by the image data comparison means, the terminal unlocking means is maintained in an inoperative state, so that the user's face away from the terminal is imaged by the imaging means and the terminal locking means cannot be activated. , The user's face away from the terminal is re-recognized, and the terminal lock release means is activated to prevent the terminal use restriction from being inadvertently canceled, and the inadvertent use restriction is released. It is also possible to reliably prevent the peeping of the display and unauthorized use of the terminal caused by the above.

  Furthermore, since the monitoring period adjusting means for randomly changing the setting values of the operation periods of the imaging means, the face image data extracting means, and the face image data comparing means during login is additionally provided, the timing when the user is monitored It is impossible to predict the problem, and the inconvenience that the terminal is illegally used while the respective means are inactive is also eliminated.

  Further, since the second monitoring period adjusting means for changing the setting values of the operation periods of the imaging means, the face image data extracting means, and the face image data comparing means according to the operating state of the terminal lock means is provided, the CPU load is reduced. The processing operation using the application program etc. can be performed smoothly, and when an appropriate user is seated after restricting the use of the terminal, this is quickly detected and the terminal unlocking means is activated. be able to.

  Further, since the third monitoring period adjusting means for setting the operation period of the imaging means, the face image data extracting means and the face image data comparing means corresponding to the type of the application program being used is provided, important data is handled. While using application programs, priority is given to ensuring security over application program usage efficiency, while application programs that handle data with low confidentiality are given priority over application program usage efficiency, or load. It is possible to ensure security while using a few application programs, while speeding up the operation of an application program while using an application program that requires high-speed processing.

  In addition, since a comparison determination forcing means for forcibly operating the imaging means, the face image data extraction means, and the face image data comparison means in response to a collation request from the application program in use is provided, a simple modification to the application program or the like It is also possible to selectively execute permission or prohibition of data display according to the suitability of the user simply by applying the above.

  FIG. 1 is a functional block diagram showing an outline of means included in a terminal user monitoring system to which the present invention is applied. As an example, a case where a terminal user monitoring system is constructed using a computer terminal itself is shown. Yes.

  Among these means, face image data extraction means A, face image data comparison means B, terminal lock means C, face image data update means D, face image data transmission means E, alarm output means F, terminal lock release means G , Face image data display means H, monitoring period adjustment means L, second monitoring period adjustment means M, third monitoring period adjustment means N, and comparison determination forcing means P are each a microprocessor (hereinafter referred to as CPU) of the terminal. It is configured as a function realization means.

  The imaging means I is composed of a CCD camera or the like installed at a position where the face of the terminal user can be photographed. The face image data storage means J and the second face image data storage means K are connected to the CPU. The storage area of the non-volatile memory.

  The face image data extraction means A analyzes the image picked up by the image pickup means I and sequentially extracts face image data.

The face image data storage means J stores the face image data extracted by the face image data extraction means A when the user logs in to the terminal as a comparison reference for authentication.
It is also possible to store face image data of a user permitted to log in to the terminal in the face image data storage means J in advance.

  The face image data comparison means B compares the face image data extracted by the face image data extraction means A and the face image data stored in the face image data storage means J while the user is logged into the terminal. Then, the degree of coincidence between the two is determined. At this time, if the difference between the two is within a preset threshold range, the two match substantially, and if the difference exceeds the threshold, the two match.

  While the result of determination by the face image data comparison means B is substantially the same, the image pickup process by the image pickup means I, the face image data extraction process by the face image data extraction means A, and the face image data comparison means B Only the determination process by is repeatedly executed, and no use restriction is imposed on the terminal.

During this time, the face image data update means D sequentially updates the face image data newly extracted by the face image data extraction means A, that is, the latest face image data of the user to the face image data storage means J. Remember.
Therefore, the face image data comparison means B can always perform an authentication process relating to the suitability of the user based on the latest face image data, and the matching accuracy can be determined by changes in the hairstyle used as a target for matching and changes in the face over time. The problem of deterioration is virtually prevented.

On the other hand, when the result of determination by the face image data comparison unit B is inconsistent, the terminal lock unit C is activated to limit the processing operation using the terminal.
The restriction of processing operation is, for example, processing that rejects the reception of signals from a manual data input device such as a keyboard or mouse on the CPU side, and prohibits the use of application programs, data read, write, modification, etc. is there.

  There are the following three possibilities for the case where the result of determination by the face image data comparison means B is inconsistent.

  First of all, the replacement of users. In this case, since the face image data of the logged-in user stored in the face image data storage means J is clearly different from the face image data of the replaced user, the determination by the face image data comparison means B is performed. The result will inevitably be inconsistent.

Further, when the face image data extracting means A has a function of extracting two or more face image data at the same time, two or more face image data are simultaneously extracted by the face image data extracting means A, one of which is a face image. The face image data of the logged-in user stored in the data storage means J may substantially match, but when two or more face image data are extracted at the same time, the face image data comparison means B It is determined that a third party other than the already-used user is looking into the display of the terminal, and an unmatched determination result is output unconditionally.
Therefore, it is possible to automatically limit the use of the terminal by detecting a third party looking into the display.
If two or more face image data are extracted, it means that at least one third person other than the logged-in user is recognized, so each of the extracted face image data and the face It is not necessary to compare the face image data of the logged-in user stored in the image data storage means J, and the mismatch determination result may be output.

  Further, when the scale of the face image data extracted by the face image data extracting means A is small, for example, when the number of pixels is small, the face image data extracted by the face image data extracting means A is temporarily Even when the face image data of the logged-in user stored in the face image data storage means J substantially matches, it is determined that the logged-in user is away from the terminal, and the face image The data comparison means B finally outputs a mismatch determination result.

  As described above, when the result of determination by the face image data comparison means B is inconsistent, in addition to the terminal lock means C, the face image data transmission means E, the second face image data storage means K, and the alarm output means F are activated. To do.

When the result of determination by the face image data comparison means B is inconsistent and the terminal lock means C is activated, the face image data transmission means E sends the face image data extracted by the face image data extraction means A at this time, That is, the face image data that causes the determination result by the face image data comparison means B to be inconsistent is connected to the terminal via the in-house LAN or the Internet, for example, a monitoring server Automatically send to etc.
If a monitoring server for storing and displaying the face image data transmitted from the face image data transmitting means E of each terminal is installed on the net, a third party who has attempted to impersonate an appropriate user, or A third party who tries to look into the display illegally can be grasped by a monitoring server in a remote place, and an unauthorized user of the terminal can be easily identified.

Further, the second face image data storage means K, when the determination result by the face image data comparison means B is inconsistent, the face image data extracted by the face image data extraction means A, that is, face image data comparison. The face image data that causes the determination result by means B to be inconsistent is stored together with various log information.
The log information can include various information that can be recognized internally by the terminal, including the time, the name of the file opened at the time, the name of the application program started at the time, and the like.
Therefore, if the contents of the second face image data storage means K are confirmed later, it is possible to identify an unauthorized user or a file that may have been leaked.
Furthermore, the face image data and log information stored in the second face image data storage means K can be transferred to the monitoring server using the face image data transmission means E described above.
In order to confirm the information stored in the second face image data storage means K possessed by the terminal, it is necessary to read data from the second face image data storage means K as post-processing, and this data is confirmed. The contents of the second face image data storage means K may be falsified by an unauthorized user until the end of the period, but the same data is used for monitoring that the face image data transmitting means E cannot reach. Since it is transferred to the server, higher security can be guaranteed.

Also, the alarm output means F outputs an alarm signal and activates an alarm when the result of determination by the face image data comparison means B is inconsistent and the terminal lock means C is activated, to alert the surrounding human beings Let
Since a computer terminal such as a workstation or a personal computer generally includes a speaker, this speaker can be used as an alarm means. Of course, an external device such as a warning light may be activated.

The terminal unlocking means G is a means for canceling the terminal usage restriction without imposing a burden on an appropriate user, that is, a logged-in user. The terminal lock release means G detects that the determination result by the face image data comparison means B is substantially the same after the operation of the terminal lock means C, and releases the operation of the terminal lock means C. Allow continued use of the terminal.
Since the proper user can continue to use the terminal normally without performing cumbersome work such as entering the user ID and password or re-entering the fingerprint, retina pattern, etc., improving work efficiency using the terminal It is beneficial to.

Further, the terminal unlocking means G is provided with a scale determining function for determining the scale of the face image data extracted by the face image data extracting means B, and the scale of the extracted face image data is less than the set value. In this case, the non-operating state of the terminal unlocking means G is maintained as it is.
Therefore, when the user's face away from the terminal is recognized, there is no fear that the use restriction of the terminal is inadvertently released and problems such as a third party looking into the display or illegal use of the terminal occur.

If the result of determination by the face image data comparison means B is substantially coincident after the operation of the terminal lock means C and the terminal lock release means G is activated, in addition to the terminal lock release means G, the face image data display means H is activated, and the face image data stored in the second face image data storage means K is read and automatically displayed on the display of the terminal.
The face image data stored in the second face image data storage means K when the appropriate user is re-recognized and the use restriction of the terminal is lifted, that is, the third that tries to impersonate the appropriate user Since the information of the person who is trying to look into the display or the unauthorized person is automatically displayed on the display, the unauthorized user can be reliably identified, and the second face image data storage means K can Since an appropriate user can quickly refer to the stored log information, it becomes difficult to perform data tampering or the like by an unauthorized user on the second face image data storage means K.

  Each of the monitoring period adjustment means L, the second monitoring period adjustment means M, and the third monitoring period adjustment means N includes the imaging means I, the face image data extraction means A, and the face image data comparison means B. This is for adjusting the set value of the operation cycle of each means required for performing image processing and determination processing required for authenticity determination.

  Among these, the monitoring cycle adjustment means L randomly changes the set values of the operation periods of the imaging means I, the face image data extraction means A, and the face image data comparison means B using a random number generation process by the CPU. The terminal is prevented from being illegally used while monitoring each unit being inactive.

Further, the second monitoring cycle adjusting means M is in accordance with the operating state of the terminal locking means C, and when the terminal locking means C is inactive, the imaging means I, the face image data extracting means A, and the face image data comparing means B A setting value of a relatively long cycle is set as the setting value of the operation cycle of the image, the image capturing process executed per unit time, the face image data extraction process, and the extracted face image data and face image data storage means J The processing operation using the application program or the like can be performed smoothly by reducing the number of comparison processing with the face image data and reducing the load on the CPU.
On the other hand, in a state in which the terminal lock unit C is operating, that is, in an environment in which an application program or the like is not used, the set values of the operation cycles of the imaging unit I, the face image data extraction unit A, and the face image data comparison unit B By setting a relatively short cycle setting value and putting most of the CPU's processing capacity into processing related to user authentication, it is possible to quickly detect that an appropriate user is seated in front of the terminal. Thus, the terminal unlocking means G is activated to eliminate unnecessary waiting time on the user side and improve user workability.

Then, the third monitoring cycle adjustment unit N changes the set values of the operation cycles of the imaging unit I, the face image data extraction unit A, and the face image data comparison unit B in accordance with the type of application program being used.
Here, how the types of application programs correspond to the operation cycles depends on the purpose of use.
For example, if it is desired to ensure the security related to the use of the terminal during the use of an application program that handles important data, a short cycle setting value is set corresponding to the application program that places importance on security. Since the user's authentication is frequently performed, the operation of the application program may be delayed to some extent, but it cannot be helped.
When it is desired not to interfere with the operation of an application program that requires high-speed processing, a long cycle setting value is set in correspondence with the application program that requires high-speed processing. In this case, since it becomes impossible to frequently determine the authenticity of the user, the security capability is reduced to some extent, but it cannot be helped.

  The monitoring cycle adjustment unit L, the second monitoring cycle adjustment unit M, and the third monitoring cycle adjustment unit N can be used in combination, but only one or two of these units can be used in combination. Is also possible.

The comparison determination forcing means P is for actuating the imaging means I, the face image data extraction means A, and the face image data comparison means B immediately upon receiving a collation request from the application program in use.
When the comparison determination forcing means P is activated, the set values of the operation periods set by the monitoring period adjusting means L, the second monitoring period adjusting means M, and the third monitoring period adjusting means N are ignored, and immediately the imaging means I and the face image Data extraction means A and face image data comparison means B operate.
However, in order to operate the comparison determination forcing means P, it is necessary to send a verification request from the application program being used. It is necessary to have a function of sending a verification request according to the processing operation.
However, for example, the processing of reading data from a hard disk drive or the like necessary for displaying information is performed by management on the operating system side, so the command is transferred from the application program to the operating system side. At this stage, if the operating system recognizes this read command as a collation request from the application program side, the application program itself does not necessarily need to be modified.
Further, if the application program side is configured to send a verification request when displaying confidential data, data display permission or terminal use restriction is selectively executed according to the suitability of the user. Security related to data safety is further improved.

  FIG. 2 is a functional block diagram showing an outline of the configuration of a computer terminal 1 (hereinafter referred to as a terminal) according to an embodiment to which the terminal user monitoring system of the present invention is applied.

  The terminal 1 is a normal computer system including a workstation or a personal computer, a microprocessor 2 (hereinafter referred to as a CPU) as a calculation means, a ROM 3 storing a part of a startup program, etc., a temporary storage of calculation data, etc. A nonvolatile memory 5 comprising a RAM 4 and a hard disk drive, etc. used in the system, and an input / output circuit 6 having a display comprising a manual data input device 7 such as a keyboard and mouse functioning as a man-machine interface, a CRT, etc. 8 and a speaker 9 are connected.

The imaging means I in this embodiment is constituted by a CCD camera 10. The CCD camera 10 performs an imaging process in response to an image capture command input from the CPU 2 via the input / output circuit 6 and inputs the captured image to the CPU 2 via the input / output circuit 6.
The interface 11 is used to input / output data to / from an external device, and is connected to another terminal, for example, a monitoring server 13 installed on the net, via a line 12 such as an in-house LAN or the Internet. Has been.

  The CPU 2 functions as conventional means for executing various application programs stored in the non-volatile memory 5, and in this embodiment, the face image data extracting means A and face image data comparing means shown in FIG. B, terminal lock means C, face image data update means D, face image data transmission means E, alarm output means F, terminal lock release means G, face image data display means H, monitoring period adjustment means L, second monitoring period adjustment It also functions as a substantial function realizing means of the means M, the third monitoring cycle adjusting means N, and the comparison determination forcing means P.

Further, a part of the storage area of the nonvolatile memory 5 functions as the face image data storage means J and the second face image data storage means K, and is further imaged in a state where the user is not seated in front of the terminal 1. It also functions as background image storage means for storing the background image. The background image is used as a reference for extracting face image data. The CPU 2 recognizes the presence of a person based on the difference between the image captured by the CCD camera 10 and the background image, and is a known image. The face image data is extracted by processing.
The non-volatile memory 5 also includes a second monitoring cycle adjusting unit M and a third monitoring cycle adjusting unit N that adjust the operating cycle of the CCD camera 10, the face image data extraction unit A, and the face image data comparison unit B. A table storing set values to be used is stored (see FIG. 9).

Next, face image data extraction means A, face image data comparison means B, terminal lock means C, face image data update means D, face image data transmission means E, alarm output means F, terminal lock release means G, face image data The processing operations of the CPU 2 functioning as the display means H, the monitoring period adjusting means L, the second monitoring period adjusting means M, the third monitoring period adjusting means N, and the comparison determination forcing means P are the flowcharts of FIGS. Will be described in detail with reference to FIG.
Among these, FIG. 3 is a flowchart showing an outline of processing executed when logging into the terminal 1, FIGS. 4 to 7 are flowcharts showing an outline of terminal user monitoring processing executed during login, FIG. 8 is a simplified flowchart showing a part of a subroutine program added to an application program that can be executed during login. At least the processes in FIGS. 4 to 7 are repeated as processes at predetermined intervals after execution of login. It is supposed to be executed.

  When the terminal 1 is powered on and the initialization process is completed, the CPU 2 repeatedly executes the determination process of step a1 and the determination process of step a20 at predetermined intervals, and the user inputs a login request to the terminal 1. (Step a1, step a20).

When the login request from the user is detected in the determination process in step a1, with CPU2 resets the value of the counter C ₁ to count the number of failed facial image data extraction processing in the time of login (step a2), An image capture command is sent to the CCD camera 10 via the input / output circuit 6, and the image captured by the CCD camera 10 is temporarily stored in the RAM 4 (step a3).

  Next, the CPU 2 functioning as the face image data extraction means A compares the image temporarily stored in the RAM 4 with the background image stored in advance in the nonvolatile memory 5 and performs image analysis on the different portions, thereby performing the current analysis. The face image data of the user who is going to log in is extracted (step a4).

  Then, the CPU 2 determines whether or not the user's face image data is extracted by appropriately performing the process of step a4 (step a5), and whether or not there are two or more extracted face image data (step a6). Furthermore, it is determined whether or not the scale of the user's face image data captured by the current imaging is normal based on the number of pixels included in the face image data, for example (step a7).

  Here, the user's face image data is appropriately extracted, and there is only one face image data, and the face image data scale is normal and the user's face is imaged at a distance within a certain range. In other words, if the determination result in step a5 is true, the determination result in step a6 is false, and the determination result in step a7 is true, the CPU 2 is extracted this time. The face image data is stored in the face image data storage means J of the nonvolatile memory 5 as face image data serving as a comparison reference for authenticity determination (step a8).

On the other hand, when it is confirmed in step a5 that the face image data of the user has not been extracted because the process of step a4 has not been performed properly, or there are two or more pieces of extracted face image data. Is confirmed in the determination process of step a6, or if the determination process of step a7 confirms that the scale of the user's face image data captured by the current imaging is not normal, the CPU 2 , is incremented by one substantial extraction process to have failed regarded to the value of the counter C ₁ of the face image data (step a15), the failure count the count value of the counter C ₁ is preset allowable value C It is determined whether or not _1S has been reached (step a16).

Here, if the count value of the counter C ₁ has not reached the allowable value C _1S, CPU 2, the same manner as above repeatedly executes step a3 subsequent steps as retry process, in order to extract the proper face image data Try.

Then, if the decision result in the step a16 count value of the counter C ₁ While such processes are repeatedly executed reaches the allowable value C _1S is true, CPU 2 is in substantial extraction process of the face image data Assuming that some kind of problem has occurred, a dialog is displayed on the display 8 to inform the user that either logout execution or continuation of retry processing related to login is selected (step a17).

  Here, when the user selects execution of logout, the CPU 2 detects the selection operation by the user in the determination process in step a18, executes the logout process (step a19), and then performs the determination process in steps a1 and a20. Is returned to the initial standby state in which is repeatedly executed at predetermined intervals.

  On the other hand, when the user selects to continue the processing operation required for login, the CPU 2 repeatedly executes the processing after step a3 as the retry processing in the same manner as described above, and tries to extract appropriate face image data. .

Finally, when the determination result in step a7 is true and the face image data of the user who is going to log in this time is appropriately stored in the face image data storage means J (step a8), the CPU 2 The value of the counter C ₂ that counts the number of failures of the face image data extraction process during login after accepting the login request by the user and executing the login process and allowing the user to operate the terminal 1 (step a9). And the value of the counter C ₃ that counts the number of times of continuous output of the mismatch determination result of the face image data comparison means B during login, the value of the flag F ₁ that indicates that the retry process of the face image data extraction process is being performed, and the terminal resets the value of the flag _{F 2} indicating that the operation of the locking means C (step a10~ step a13), control the imaging period of the CCD camera 10 Gosuru resets the timer t ₁ is restarted (step a14), and ends the processing of this cycle.

  Since the login has already been executed in the processing after the next cycle, the determination result of step a1 is false and the determination result of step a20 is true, and the processing during login by the CPU 2 is executed.

  When the process during login is started, first, the CPU 2 functioning as a part of the comparison determination forcing means P determines whether or not there is a collation request from the application program (step a21).

If no collation request is detected from the application program, the CPU 2 functioning as a part of the third monitoring period adjusting means N determines whether or not the application program is being executed and its type, as shown in FIG. A record corresponding to the type of the application program currently being executed is identified from the current table (step a22), and the CPU ₂ functioning as a part of the second monitoring period adjusting means M has already flag F ₂ at this time. Is set, that is, whether or not the terminal lock means C has already been activated (step a26).

Here, when the determination result of step a26 is false, that is, when it is determined that the terminal lock means C is not operating, the CPU 2 functioning as a part of the second monitoring cycle adjustment means M to select the setting value for relatively long periods out of the record is stored in the set value storage register T ₁ that value as the working cycle of the CCD camera 10 and the face image data extracting means a and the face image data comparison means B (Step a27).
Thus, for example, in a situation where the terminal locking means C is not operating during the execution of the application program A shown in FIG. 9, a relatively longer set value T _a1 is set to the set value storage register T ₁ It will be.

On the other hand, when the determination result of step a26 is true, that is, when it is determined that the terminal lock means C is operating, the CPU 2 functioning as a part of the second monitoring cycle adjustment means M to select the setting value for a relatively short period from the record, is stored in the set value storage register T ₁ that value as the working cycle of the CCD camera 10 and the face image data extracting means a and the face image data comparison means B ( Step a28).
Thus, for example, if the operating terminal locking means C is executing the application program A shown in FIG. 9, that the set value storage register T ₁ set value T _a2 relatively shorter to be set Become.

On the other hand, when the CPU 2 functioning as a part of the third monitoring cycle adjusting means N determines that there is no application program being executed in the determination process in step a22, a table as shown in FIG. Thus, a record corresponding to no application program being executed is identified.
In this case, the CPU ₂ functioning as a part of the second monitoring cycle adjusting unit M determines whether or not the flag F2 is set, that is, whether or not the terminal lock unit C is operating (step a23). If the terminal lock means C is not activated, a set value T _X1 having a relatively long period is selected from the record and stored in the set value storage register T ₁ (step a24). There if activated, by selecting the set value T _X2 relatively short period is stored in the set value storage register T ₁ from of the record (step a25).
Therefore, the set value T _X1 in situations where the application program is not running in the set value storage register T ₁ unless the terminal lock means C is operated is set, also the terminal in a situation where an application program is not running if the locking means C are long operated, the set value T _X2 is to be set in the set value storage register T _1.

To change the setting of the set value storage register T ₁ according to the operation or non-operation of the terminal locking means C, under circumstances where use of the terminal 1 is allowed, that is, the terminal locking means C is under the inoperative status This is because if the CCD camera 10, the face image data extraction means A, and the face image data comparison means B are frequently operated, an excessive load may be applied to the CPU 2.

Then, CPU 2 is actuated to function as a monitoring cycle adjusting unit L, based on the set value of the operation period set in the setting value storage register T ₁ at this time generates a random number, the value set value storage register T _It is re-stored to ₁ (step a29).

This processing is achieved by, for example, using a function RND (X) that randomly generates real values from 0 to ₁ , and executing an arithmetic expression of T ₁ ← 2 · T ₁ · RND (X). Can be done.
Here, if, step a24, step a25, if the 10 seconds as the initial value of _{T 1} in the processing of step a27 or step a28 is set, the value of the random number generated by the processing in step a29 0 seconds to The range is 20 seconds, and the average value is equal to the initial value of 10 seconds.

As already mentioned, the value of the set value T ₁ of the case where the terminal lock means C is not actuated to relatively longer compared to the set value T ₁ of the case where the terminal lock means C is operating a principle, if the initial value of the set value T ₁ of the case where the terminal locking means C are not operated for 10 seconds, and 5 seconds the initial value setting T ₁ of the case where the terminal lock means C is operating Then, in the example of T ₁ ← 2 · T ₁ · RND (X), the set value T ₁ when the terminal lock means C is not in operation can take a value in the range of 0 to 20 seconds, and the terminal lock The possible value of the set value T ₁ when the means C is operating is in the range of 0 to 10 seconds. In some cases, the value of the set value T ₁ when the terminal lock means C is not operating is the terminal value. relatively shorter compared to the set value T ₁ of the case where the locking means C is operating There is also potential.

If necessary the terminal locking means C is set to be always longer than the value of the set value T ₁ of the case where the set value T ₁ of the case is not operating is operating the terminal locking means C is, for example, , configured to generate a random number in the range of 80% to 120% of the initial value, the value of the set value T ₁ of the case where the terminal locking means C are not operated, the terminal locking means C is operated and it may be set such that the size of more than 150% compared to the set value T ₁ of the case are. The random number generation process is a design problem.

In this embodiment, the third monitoring period adjusting means N that changes the set value T ₁ corresponding to the type of the application program in use, and the _first that changes the set value T ₁ according to the operating state of the terminal lock means C. 2 and the monitoring cycle adjusting means M, the set value T ₁ by using a random number generation processing, and the like to randomly change monitoring cycle adjusting unit L and the CCD camera 10 by the combined in series face image data extracting unit a and although so as to change the value of the set value T ₁ of the operating cycle of the face image data comparison means B, 1 only these means or, configured to change the set value T ₁ in combination two It is also possible to do.

Then, CPU 2, the flag F ₁ indicating that the execution of the retry process relating to the face image data extracting process is determined whether it is set (step a30), if the retry processing has been performed, further, CCD camera 10 determines whether the time count of the timer t ₁ for controlling the capturing cycle has reached a set value T ₁ by (step a31).

Then, if no counting of the timer t ₁ has reached the set value T _1, CPU 2 determines whether a logout request from the user is inputted (step a35), if no logout request is input further, the terminal locking means C of determining whether the flag F ₂ indicating that the operation has been set (step a38), since the immediately following login current terminal locking means C are not operated yet, the CPU2 The user is allowed to use the terminal 1, and the normal processing operation using various application programs or the like is executed for one cycle to finish the processing of the cycle (step a39), and the processing of the next cycle is performed again. Start.

Or then either enter the verification request from the application program in use in the processing of step a21 is detected, the count of the timer t ₁ has reached the set value T ₁ is is confirmed in the determination process in step a31, Alternatively, until the logout request from the user is detected in the determination processing in step a35, the CPU 2 performs the processing in the next cycle and subsequent steps in the same manner as described above, step a1, step a20 to step a31, step a35, and step a38. The process of Step a39 is repeatedly executed, and during this time, the execution of the process of Step a39 is allowed, and the normal processing operation by the user using various application programs or the like is performed.

Of course, it is conceivable that the application program used during this time is changed or the application program being used is terminated, but the third monitoring period adjusting means N operates in real time (steps a22 to step a). a28 reference), the set value T ₁ according to the change of the presence and the type of use of the application program is also automatically updated.
Furthermore, since the value of the set value T ₁ is randomly changed by the randomizing process, it is practically impossible to specify the value from the outside.

While such processing is repeatedly executed, when the counting of the timer t ₁ has reached the set value T ₁ is is confirmed in the determination process in step a31, CPU 2 controls the image pickup period by the CCD camera 10 after the timer _{t 1} is restarted by resetting (step a32), an image capture command is sent to the CCD camera 10 in the same manner as described above, captures the image captured by the CCD camera 10 (step a33), a face image The CPU 2 functioning as the data extraction means A extracts the user's face image data from the image captured by the CCD camera 10 (step a34).

  Then, the CPU 2 functioning as the face image data comparison unit B determines whether or not the user's face image data is appropriately extracted (step a40), and whether or not there are two or more extracted face image data (step a41). Further, whether or not the user's face image data captured by the current imaging and the face image data stored in the face image data storage means J at the time of login substantially match (step a42). It is determined whether or not the scale of the user's face image data captured by imaging is normal (step a43).

  Then, the user's face image data is appropriately extracted (the determination result of step a40 is true), the extracted face image data is only one (the determination result of step a41 is false), and this face image data And the face image data stored in the face image data storage means J of the non-volatile memory 5 substantially match (the determination result of step a42 is true) and the scale is equal to or greater than the set value, and the user The CPU 2 functioning as the face image data update means D only uses the face image data stored in the non-volatile memory 5 only when it is confirmed that the face image data has not been moved away from the front of 1 (the determination result in step a43 is true). The data storage means J is updated and stored (step a44).

  By updating the contents of the face image data storage means J of the nonvolatile memory 5 only when the scale of the face image data is equal to or larger than the set value, it is possible to prevent the deterioration of the face image data serving as a comparison reference. is there.

Next, the CPU ₂ determines whether or not the flag F2 indicating that the terminal lock means C is in operation is set, that is, whether or not the terminal lock means C is already in operation at this time (step a45). because at the moment immediately after the terminal locking means C flag F ₂ not operating is held in the reset state, CPU 2 after the determination process executed in step a35 and step a38, allowed to use the terminal 1 by the user Then, a normal processing operation using various application programs or the like is executed for one cycle to end the processing of the cycle (step a39), and the processing of the next cycle is started again.

Thereafter, the input of the verification request from the application program being used is detected in the determination process of step a21 that is repeatedly executed at predetermined intervals, or the logout request from the user is detected in the determination process of step a35. until that, each time the course of setting period T ₁ is is confirmed in the determination process in step a31, on condition that the determination result in step a40~ step a43 is appropriate, serves as the face image data updating unit D The CPU 2 updates and stores new face image data in the face image data storage means J of the nonvolatile memory 5 in the process of step a44. During this time, use of the terminal 1 by the user continues to be permitted.

As described above, every time the timer t ₁ measures the set value T ₁ , the contents of the face image data storage means J are updated to the latest one. As a result, the CPU 2 functioning as the face image data comparison means B always has the latest face. Authentication processing regarding the suitability of the user can be performed based on the image data, and the problem that collation accuracy deteriorates due to a change in hairstyle used as a collation target or a secular change in face is solved.

Further, since the set value T ₁ of the timer t ₁ for controlling the image pickup period by the CCD camera 10 can be set short if even sufficient CPU2 processing capacity, other aging hairstyle changes or face, temporarily Changes in the distance between the CCD camera 10 and the user due to a change in the user's posture, a change in the user's posture, that is, the size of the face on the imaging surface of the CCD camera 10 By coping with variable factors such as changes, it is possible to make a proper determination that the same person is the same person and another person is another person.

  On the other hand, when such processing is repeatedly executed, if the user's face image data is not properly extracted in step a34 and the determination result in step a40 is false, The face image data cannot be extracted because the user is not facing the correct direction, and the user's authenticity determination itself may be impossible.

On the other hand, when the scale of the face image data is less than the set value and the determination result in step a43 is false, only one user who has logged in first is detected, but the position of the user May be away from the front of the terminal 1, and since a proper user is away from the seating position, it is easy for a third party without usage authority to look into the display 8. It can also be considered.
Further, even if this face image data belongs to an appropriate user, if this image data is overwritten on the face image data storage means J of the nonvolatile memory 5, the amount of information (scale) is insufficient. There is also a risk that the face image data serving as a comparison reference may be deteriorated.

  These can be interpreted as phenomena that mean that substantial extraction processing of face image data has failed.

Therefore, this case, CPU 2 is (step a51) with sets a flag F ₁ stores the start of the retry process indicating that the execution of the retry process of the face image data extraction processing, the face image data extracted in the log and increments the value of counter C ₂ to count the number of failed processing (step a52), and determines whether the count value of the counter C ₂ has reached the allowable value C _2S number of failures that have been set in advance (step a53).

Here, if the count value of the counter C ₂ has not reached the allowable value C _2S, CPU 2, the same manner as above repeatedly performs the processes of steps a35 and step a38~ step a39, using various application programs, The normal processing operation of the user is allowed for one cycle, the processing of the cycle is terminated, and the processing of the next cycle is started again.
Even if the substantial extraction process of the face image data fails, use of the terminal 1 is not immediately limited.

Thus result flag F ₁ is set, the processing of the next cycle determination result in step a30 is the determination process of step a31 becomes true is not executed, the timer t ₁ is set value to control the imaging cycle T ₁ regardless of whether measures the immediately so that the retry processing related to authentication judgment of the user reaching step a33~ step a34 and step a40~ step a43 is executed.

Retry processing related to face image data extraction is continuously allowed up to C _2S times, and during this time, use of the terminal 1 is continuously allowed.

However, if the retry processing of the facial image data extraction processing has failed C _2S consecutive detected in the determination process in step a53, CPU 2 may face the failure, from the intentionally CCD camera 10 by the user Failure of image extraction processing caused by turning away from the user, or failure of image extraction processing caused by the right user being away from the terminal 1, that is, face image data does not match in substantial authentication and it was regarded as abnormal with weights comparable occurs in the case, to abandon the continuation of retry processing, and resets the counter C ₂ and flag F ₁ (step a54, step a55).

Next, the CPU ₂ determines whether or not the flag F2 has already been set at this stage, that is, whether or not the terminal lock means C has already been operated at this time (step a56).

If the flag F ₂ is not set and the terminal lock means C is not activated, the CPU ₂ functioning as the terminal lock means C sets the flag F ₂ again and stores the start of use restriction of the terminal 1 ( Step a62).
At the same time, the CPU 2 functioning as the alarm output means F outputs an alarm signal to the speaker 9 of the terminal 1 via the input / output circuit 6, operates the speaker 9 and outputs an alarm sound, and is illegally used by people in the vicinity. The generation is warned (step a63).

  At this time, the drive of the input / output circuit 6 is restricted and display output to the display 8 is temporarily stopped, or a screen saver is started, etc. You may make it prohibit referencing.

Then, CPU 2, the step a35 in the same manner as described above, but will execute the determination process in step a38, the results being set flag F ₂ in the manner described above, the determination result in step a38 becomes false, step The process of a39 is not executed, and the use of the terminal 1 by the user is completely prohibited. This is the state when the terminal lock means C is activated.

  On the other hand, when the determination result of step a41 is true and two or more face image data are detected, or the determination result of step a43 is false, the face image data captured this time and the nonvolatile memory 5, when it is confirmed that the face image data stored in the face image data storage means J does not match, at least the face image data has been successfully extracted, and the display 8 is looked into. The face image data of one or more third parties or the third person who is trying to impersonate the first logged-in user has been extracted, that is, the authenticity of the user's face image data It means that the result of judgment is clearly inconsistent.

In this case, CPU 2 adds 1 to the value of the counter C ₃ to count the consecutive number of output inconsistency determination result of the face image data comparison means B (step a57), the failure count the count value of the counter C ₃ is set in advance It determines whether or not reached the allowable value _{C 3S} (step a58).

Here, if the count value of the counter C ₃ does not reach the allowable value C _3S , the CPU 2 repeatedly executes the processing of step a35 and step a38 to step a39 in the same manner as described above, and uses various application programs and the like. The normal processing operation of the user is allowed for one cycle, the processing of the cycle is terminated, and the processing of the next cycle is started again.
That is, as in the case of the failure of the face image data extraction process described above, just because the determination result by the face image data comparison means B does not match, the use of the terminal 1 is not immediately limited. However, if C _3S = 1 is set, the use of the terminal 1 can be restricted immediately when the determination result by the face image data comparison unit B first does not match.

Abnormalities related to mismatch of face image data are allowed up to C _3S times, and during this time, the use of the terminal 1 continues to be allowed.

However, if it is detected in the determination process of step a58 that the abnormality in the determination result by the face image data comparison unit B has occurred C _3S times, the CPU 2 has a third party who looks into the display 8 illegally. or, or, first to third parties that try Narikawaro the logged-in user is regarded as seated in front of the terminal 1, whether or not it is already set flag F ₂ at this stage, That is, it is determined whether or not the terminal lock means C is already operating at this time (step a59).

Then, if the flag F ₂ has not terminal locking means C a not set is activated, CPU 2 is extracted in this extraction processing facial image data, that is, a third party or proper looking into the display 8 Face image data holding information of a third party trying to impersonate a new user is stored in the second face image data storage means K of the non-volatile memory 5 together with log information (step a60), and the face image data is transmitted. The CPU 2 functioning as the means E adds an ID unique to the terminal 1 to the information stored in the second face image data storage means K, and this information is sent to the monitoring server 13 via the interface 11 and the line 12. (Step a61).

  The log information includes not only the current time but also various information that can be recognized internally by the CPU 2 including the name of the file opened at the time and the name of the application program started at the time. To.

  In this manner, the face image data of one or more third parties who looked into the display 8 or the third party who tried to impersonate the first logged-in user is immediately transmitted by the face image data transmitting means E to the unauthorized user. Since it is transferred to the unreachable monitoring server 13, even if it is technically possible for an unauthorized user to tamper or erase the data in the second face image data storage means K, the monitoring server The information related to the history of unauthorized use can be surely left on the side 13 and high security is guaranteed.

Next, the CPU ₂ functioning as the terminal lock means C sets the flag F ₂ , stores the start of use restriction of the terminal 1 (step a 62), and the CPU 2 functioning as the alarm output means F passes through the input / output circuit 6. Then, an alarm signal is output to the speaker 9 of the terminal 1 and the speaker 9 is operated to output an alarm sound to warn people in the vicinity of unauthorized use (step a63).

  Similarly to the above, a third party other than the user who has logged in the open file by limiting the drive of the input / output circuit 6 and temporarily stopping display output to the display 8 or starting a screen saver. It is possible to prohibit the reference by.

Then, CPU 2, the step a35 in the same manner as described above, but will execute the determination process in step a38, the results being set flag F ₂ in the manner described above, the determination result in step a38 becomes false, step The process of a39 is not executed, and the use of the terminal 1 by the user is completely prohibited. This is the state when the terminal lock means C is activated.

After the terminal lock means C is operated in this way, if the determination result by the face image data comparison means B continues to be inconsistent in the processing after the next cycle, the value of the counter C ₃ is C ₃ > C _3S , but since the flag F ₂ has already been set at that time, the determination result of step a59 is true, and the processing of step a62, step a63, etc. related to the activation of the terminal lock means C is not performed. A process of additionally storing the problematic face image data in the second face image data storage means K of the nonvolatile memory 5 together with the log information (step a64), and adding the additional information to the interface 11 and the line 12 only the processing to be transferred to the monitoring server 13 through a is executed (step a65), the value of the counter C ₃ so that is reset (step a 6).

Even if the value of the counter C ₃ is reset in this way, the setting state of the flag F ₂ is maintained as it is, so that the determination result by the face image data comparison unit B does not match in the processing after the next cycle. if There continuing, each time the face image data comparison means B mismatch determination counting the number of output results counter C ₃ values step counts the set value C _3S a58, the determination result in step a59 becomes true, the The face image data in question at the time is additionally stored together with log information in the second face image data storage means K of the nonvolatile memory 5 (step a64), and these additional information is transmitted via the interface 11 and the line 12. Are sequentially transferred to the monitoring server 13 (step a65).

Then, while the face image data comparison unit determination result by B continues mismatch state since the setting state of the flag F ₂ is held, is carried out also maintains the restriction of the use of the terminal 1.

The setting value T ₁ of the operating cycle of the CCD camera 10 and the face image data extracting means A and the face image data comparison means B is relatively short by the second monitoring cycle adjusting means M described above with the terminal locking means C once activated Since it is switched to the value, image processing and determination processing relating to the authenticity determination of the user shown in step a32 to step a34, step a40 to step a43, etc. are frequently performed, and an appropriate user is When seated before, this is in a state where it can be detected immediately.

  While such processing is repeatedly executed, the user's face image data is appropriately extracted (the determination result at step a40 is true), and the extracted face image data is only one (step the determination result of a41 is false), the face image data and the face image data stored in the face image data storage means J of the nonvolatile memory 5 substantially match (the determination result of step a42 is true), and When it is confirmed that the scale is equal to or larger than the set value and the user is properly seated in front of the terminal 1 (the determination result at step a43 is true), the CPU 2 functioning as the face image data update unit D is activated again. Then, the face image data is updated and stored in the face image data storage means J of the nonvolatile memory 5 (step a44).

Next, the CPU ₂ determines whether or not the flag F2 is set, that is, whether or not the terminal lock means C is operating at this time (step a45). In this case, the terminal lock means C is operated. Therefore, the determination result at step a45 is true.

In this way, when the proper user is seated in front of the terminal 1 is verified, CPU 2 is the counter C ₃ together to stop the output of the alarm signal to terminate the output of the alarm sound by the speaker 9 The value is reset (step a46, step a47), and the CPU ₂ functioning as the terminal lock release means G resets the flag F2 to release the use restriction of the terminal 1 (step a48).

  If the output of the input / output circuit 6 is limited and the display output is temporarily stopped, the display output is resumed at that time. If the screen saver is activated, the screen saver is activated at this time. To stop.

  There is no need to perform a cumbersome operation such as inputting a user ID or password or re-inputting a fingerprint or a retina pattern in order to cancel the use restriction. If a proper user sits in front of the terminal 1, the use restriction is automatically made. Since it can cancel | release, the working efficiency using the terminal 1 improves.

  Further, the terminal unlocking means G is provided with a scale determination function for maintaining the use restriction of the terminal 1 as it is when the scale of the face image data is less than the set value and the determination result of step a43 is false. Therefore, when a user's face away from the terminal 1 is recognized, the use restriction of the terminal 1 is inadvertently released and the display 8 is looked into or the terminal 1 is illegally used. Absent.

Next, whether the CPU 2 functioning as the face image data display means J accesses the second face image data storage means K of the nonvolatile memory 5, and whether the face image data and log information are stored in the second face image data storage means K It is determined whether or not (step a49).
If face image data and log information are stored, the face image data and log information, that is, the face of a third party looking into the display 8 or a third party who is trying to impersonate an appropriate user. The image data, the names of files that may be used by these unauthorized users, the names of application programs, the time of use thereof, and the like are displayed on the display 8 to notify an appropriate user (step a50).

As described above, the data contents of the second face image data storage means K are automatically displayed on the display 8 when the use restriction of the terminal 1 is released. There is no worry of forgetting to confirm.
In addition, since it is possible to promptly notify the appropriate user of the face image data and log information stored in the second face image data storage means, an unauthorized user can instruct the second face image data storage means K of the data. Opportunities for tampering and erasing can also be reduced.

Then, CPU 2, the a executes the determination process in step a35 and step a38 in the same manner, since already the flag F ₂ is reset at this time, the determination result in step a38 is true.

  Therefore, the CPU 2 permits the use of the terminal 1, allows the user to execute normal processing operations using various application programs, etc. for one cycle, ends the processing of the cycle (step a39), and again The processing of the next cycle is started.

Thus the flag F ₂ is reset, the operation of the terminal locking means C is canceled, the processing CCD camera 10 and the face image data extracting means A and the face image data compared by the second monitoring cycle adjusting means M described above since the set value T ₁ of the operating cycle of the device B is switched to the relatively long value, step a32~ step a34, per unit time of the image processing and determination processing relating to authentication judgment shown in step a40~ step a43 etc. Since the number of executions is relatively reduced and the substantial load on the CPU 2 is reduced, the user can smoothly perform processing using various application programs.

  Hereinafter, until the input of the collation request from the application program being used in the determination process in step a21 is detected or until the logout request from the user is detected in the determination process in step a35, the CPU 2 In the processing after the next cycle, the processing of step a1, step a20 to step a35 and step a38 to step a39 is repeatedly executed in the same manner as described above. A normal processing operation by a user using a program or the like is performed.

  FIG. 8 is a flowchart showing a simplified example of a subroutine program for adding a function for sending a collation request for activating the comparison determination forcing means P to the application program executed by the user in the normal processing of step a39. .

When an important data display request is input by the user during execution of the application program (step b1), the CPU 2 that detects this sets a verification request flag in the RAM 4 (step b2), that is, after temporarily stopping the write and display reading data requested (step b3), executes the reset and start processing of the timer t ₂ for counting a waiting time (step b4), the timer t ₂ is set enters a wait state to wait for the counting value _{T 2} (step b5, step b6).

  This application program is repeatedly executed after the next cycle, but the processing on the application program side is in a paused state, the determination result in step b1 is false, the determination result in step b5 is true, and the determination in step b6 Since the result is false, at least processing related to reading and display of important data is not executed.

However, the set value T ₂ of the timer t ₂ executes the retry process involving extraction of the facial image data as needed CCD camera 10 and face image data extracting unit A and the face image data comparison means B described above use A sufficient time shall be determined to determine the suitability of the person.

  The collation request flag set in the RAM 4 in this way is detected as a collation request from the application program in the determination process of step a21 by the CPU 2 executing the process during login.

When verification request from the application program is detected, CPU 2 that functions as a comparison determination forced unit P is all skipped confirmation processing relating to various setting processes and the measuring time timer t ₁ reaches step a22~ step a32 ignores the setting _{T 1} of the operating cycle of the CCD camera 10 and the face image data extracting means a and the face image data comparison means B, immediately step a33~ step a34, the authenticity judgment of the user reaching step a40~ step a42 processing is executed, similarly to the above, according to the determination result, whether to set the flag F _2, or, as it holds the reset state of the flag F _2.

On the other hand, the CPU 2 that executes the application program sets the flag F ₂ when the timer t ₂ that counts the waiting time detects that the set value T ₂ has elapsed, that is, when the CPU ₂ has reliably finished authenticating the user. Is in a reset state or a set state (step b7).

When the flag F ₂ is held in a reset state, that is, only if the user is a proper user resets the collating request flag RAM 4 (step b8), demanding a All data Reading and display are permitted, and this data is displayed on the display 8 (step b9).

Also, if the flag F ₂ is in the set state, i.e., if the user is determined not to be the proper user, the execution itself of the application program by the determination process in step a38 described above (Step a39) Because it is prohibited, there is no worry that important data will be stolen by inappropriate users.

  Here, as an example, a case has been described in which a simple modification is made to an application program and a verification request is sent from the application program side, but the application program does not necessarily need to be modified. A command such as download or display can be interpreted as a collation request and the same processing as in FIG. 8 can be performed.

  Finally, when an appropriate user inputs a logout request and terminates the use of the terminal 1, the CPU 2 detects the logout request in the determination process of step a35, and the face image data storage means of the nonvolatile memory 5 The face image data stored in J, that is, data used as a reference for collation in the user's authenticity determination is cleared and the logout process is executed (step a36, step a37), and the determination in steps a1 and a20 The process returns to the initial standby state in which the process is repeatedly executed at predetermined intervals.

  That is, in this embodiment, the user who has stored the face image data at the time of login is recognized as an appropriate user until the logout.

  Note that whether or not the first logged-in user is appropriate can be easily determined by processing operations similar to those in the past, for example, input of a user ID or password.

  Next, with reference to the flowcharts of FIGS. 10 to 14, an embodiment in which face image data of a user who is permitted to log in is stored in advance in the face image data storage unit J will be briefly described. To do.

  Since the hardware configuration related to the terminal 1 is the same as that in FIG. 2, the description thereof is omitted, and in the following description, the same reference numerals as those used in the description of FIG. 2 are used.

  When the terminal 1 is powered on and the initialization process is completed, the CPU 2 repeatedly executes the determination process in step c1 and the determination process in step c21 at predetermined intervals, and the user inputs a login request to the terminal 1. (Step c1, step c21).

When the login request from the user is detected by the determination processing in step c1, with CPU2 resets the value of the counter C ₁ to count the number of failed facial image data extraction processing in the time of login (step c2), An image capture command is sent to the CCD camera 10 via the input / output circuit 6, and the image captured by the CCD camera 10 is temporarily stored in the RAM 4 (step c3).

  Next, the CPU 2 functioning as the face image data extraction means A compares the image temporarily stored in the RAM 4 with the background image stored in advance in the nonvolatile memory 5 and performs image analysis on the different portions, thereby performing the current analysis. The face image data of the user who is going to log in is extracted (step c4).

  Then, the CPU 2 determines whether or not the process of step c4 is appropriately performed and the user's face image data is extracted (step c5), and whether or not there are two or more extracted face image data (step c6). Further, it is determined whether or not the scale of the user's face image data captured by the current imaging is normal (step c7).

  Here, the user's face image data is appropriately extracted, and there is only one face image data, and the face image data scale is normal and the user's face is imaged at a distance within a certain range. In other words, if the determination result of step c5 is true, the determination result of step c6 is false, and the determination result of step c7 is true, the face image data comparison means B is used. The functioning CPU 2 compares the user's face image data captured at the time of the login request with the user's face image data stored in advance in the face image data storage means J (step c8).

In this case, the face image data storage means J for storing the user's face image data in advance is not necessarily provided in the nonvolatile memory 5.
For example, the face image data storage means J is provided in a remote monitoring server 13 or a data management server, and the face image data of the user is captured from the face image data storage means J to the terminal 1 via the line 12. It is possible.

Further, the face image data storage means J can store in advance face image data of a plurality of users permitted to use the terminal 1, and in the determination process of step c8, when a login request is made. Is compared with the face image data of all users stored in advance in the face image data storage means J.
If the user's face image data captured at the time of the login request substantially matches any one of the user's face image data stored in advance in the face image data storage means J, the determination result is Become true.

  Here, if the determination result in step c8 is false, the face image data is the correct user's information even though the face image data extraction process is properly performed at the time of user login. Since it does not substantially match the face image data, that is, it means that the user who is logging in at the present time is not an appropriate user, the CPU 2 forcibly executes logout processing (step c20). Log the person out. Therefore, the use of the terminal 1 by this user is not allowed.

  On the other hand, if the user's face image data is not properly extracted and the determination result in step c5 is false, or the third party's face is reflected in the CCD camera 10 and the determination result in step c6 is true. If the determination result in step c7 is false due to a problem in the seating position of the user or the like, the appropriate user can be obtained by repeatedly executing the same process as the retry process. Login is allowed.

However, since the value of the counter C ₁ is incremented along with the execution of the retry process (step c16), the allowable number of retries processing is limited to C _1S times (step c17).
If it is necessary to repeat the retry process, the CPU 2 refers to the dialog displayed on the display 8 (step c18), and selects to continue the processing operation required for login (step c19). ).

Finally, when the determination result in step c8 is true and it is confirmed that a proper user is seated in front of the terminal 1, the CPU 2 functioning as the face image data updating means D is fetched this time. The face image data is updated and stored in the corresponding storage area of the face image data storage means J (step c9).
The corresponding storage area referred to here is a storage area that stores the face image data determined to be substantially coincident with the face image data at the time of the login request in the determination process of step c8, that is, a plurality of storage areas. This is a storage area for one person's face image data in the face image data storage means J that stores the user's face image data.

  Next, the CPU 2 allows a login request by the user and executes a login process (step c10), and permits the user to use the terminal 1.

  Since the processing of step c11 to step c36 and step c38 to step c66 is substantially the same as the processing of step a10 to step a35 and step a38 to step a66 described in the first embodiment, description thereof will be omitted.

  In the second embodiment, even when a proper user logout is detected in the determination process in step c36, the face image data corresponding to the user is not cleared, and the latest face image data corresponding to the user is displayed as the face. Since it is held in the image data storage means J as it is, when the user logs in to the terminal 1 at the next opportunity, the authenticity determination of step c8 is executed based on this latest face image data. can do.

That is, in the second embodiment, the user who is attempting to log in by comparing the face image data of the user captured at the time of login with all the face image data stored in the face image data storage means J in advance. It is determined whether or not the user is an appropriate user, and the user permitted to log in by this determination is recognized as an appropriate user until the logout.
Until the appropriate user logs out from the terminal 1, the latest face image data of the user is sequentially updated and stored in the face image data storage means J, and then the user is stored in the terminal 1. When logging in, the user's face image data captured at the time of login is compared with the latest face image data of the user stored in the face image data storage means J in advance (at the previous logout). An authenticity judgment will be made.

Here, if face image data of a plurality of users is stored in the face image data storage means J and the use of the terminal 1 by a plurality of users is permitted, each time one of the users uses the terminal 1, The user's face image data can be automatically updated to the latest one.
However, even in this case, once a specific user logs in, the user who can use the terminal 1 is limited to the user who logs in first, unless the user logs out by himself / herself.

  On the other hand, when only the face image data of one user is stored in the face image data storage means J, other users are completely prohibited from logging in to the terminal 1, so that the security of the terminal 1 is further enhanced. Can be secured.

  In the case where there are a plurality of terminals 1 on the net, a definition file defining the relationship between the user's face image data and the log-in terminals 1 in a one-to-many or many-to-one manner together with the face image data storage means J You may make it deploy.

  In the first embodiment and the second embodiment described above, only face image data that is a user's biological feature is detected while maintaining a non-contact state with respect to the user, and this face image data is used as a target for collation. Since it is used, it is possible to accurately determine the suitability of the user with respect to the terminal 1 without restricting the user's movement and posture, and input of a user ID, password, fingerprint, retinal pattern, etc. Compared to the case of using the conventional authentication means, the burden on the user required for maintaining security is reduced.

  In addition, since the usage status of the terminal 1 is constantly monitored and the determination result by the face image data comparison means B is inconsistent, the terminal lock means C is activated immediately. Unlike the case of restricting the use of the terminal 1 using a password or the like, it is difficult for an inappropriate user to become a legitimate user and to use the terminal 1, and the user intentionally Unauthorized acts such as causing other persons to use the terminal 1 and other persons using the terminal 1 while an appropriate user is out of their seats are also reliably prevented.

  Further, when a plurality of human face image data are extracted at the same time, the use of the terminal 1 is completely restricted even if a proper user's face image data is extracted. It is virtually impossible for a third party to threaten an appropriate user and operate the terminal 1.

Moreover, as the set value T ₁ that identifies the operating cycle of the CCD camera 10 and the face image data extracting means A and the face image data comparison means B is randomly changed by the action of the monitoring cycle adjusting means L Therefore, it is very difficult to illegally use the terminal 1 while monitoring.

In addition, even if it is possible to take advantage of monitoring, when reading or displaying important information using an application program, a comparison judgment enforcement means that receives a verification request from the application program P forcibly drives the CCD camera 10, the face image data extraction means A, and the face image data comparison means B to immediately determine whether or not the user is appropriate. If the user is inappropriate, the terminal 1 By restricting the use of the device itself, important information is not displayed, so that there is almost no worry that important data is stolen by an inappropriate user.
In particular, when the user tries to read information by referring to the display 8, the user needs to face his face in an appropriate direction, so that a positional relationship suitable for extracting face image data is realized. Therefore, it is possible to accurately determine the authenticity of the user, and it is effective for ensuring the security of the data itself.

Further, the second monitoring cycle adjusting unit M is relatively set as the set value T ₁ of the operating cycle of the CCD camera 10, the face image data extracting unit A, and the face image data comparing unit B when the terminal lock unit C is not operated. while setting a long period of set values, since in the state in which the terminal locking means C is operating is adapted to set the set value of the relatively short period as the set value T ₁ of the operation period, the terminal In a situation where 1 is being used by an appropriate user (terminal lock means C is inactive), the load on the CPU 2 can be reduced and processing operations using application programs etc. can be performed smoothly. Once the use of the terminal 1 is restricted and the load on the CPU 2 is reduced (when the terminal lock means C is activated), the processing relating to the authenticity determination of the user is frequently executed. It is possible to operate the terminal unlocking means G to quickly detect a positive a user.
In this way, as a result of eliminating the useless waiting time of an appropriate user waiting for the restriction release of the terminal 1, workability using the terminal 1 is improved.

  FIG. 15 is a functional block diagram simply showing an example in which the biometric authentication unit 14 is provided in the terminal 1 of the first embodiment or the second embodiment.

  As the biometric authentication means 14, a fingerprint collation device, a voiceprint collation device, a retinal pattern collation device attached to glasses or the like can be used.

  By using these biometric authentication means 14 together and determining whether login is possible and whether the terminal unlocking means G is operable, security can be further enhanced.

  In this case, an appropriate user's fingerprint, voiceprint, or retinal pattern is registered in advance in the nonvolatile memory 5 of the terminal 1, the monitoring server 13 or a data management server, and the like. The determination result of the biometric authentication means 14 is used together when determining whether or not the operation is possible.

  Although it is possible to use a determination result using the biometric authentication unit 14 in the authenticity determination process for each predetermined period, there is a possibility of increasing the burden on the user. Therefore, in the third embodiment, the biometric authentication unit 14 is used. Is used only when it is determined whether or not the login is possible and whether or not the terminal unlocking means G is activated, and is not used in the determination process during login.

  FIG. 16 is a diagram in which log information storage means 15 for storing records such as monitoring statuses and operation histories is added to the terminal 1 of the first embodiment, the second embodiment, and the third embodiment.

  Records such as monitoring status and operation history of the terminal 1 are automatically stored in the log information storage means 15 and can be arbitrarily referred to by a system administrator and an appropriate user (a user who can log in to the terminal 1). It is like that. Each administrator can take appropriate measures as necessary.

  In addition, when the face image data extraction unit A cannot detect the face image data or the processing result such as the face does not match in the determination process by the face image data comparison unit B is stored in the log information storage unit 15, It is possible to store logs such as when an appropriate user leaves or when another person tries to look into the display 8. An unauthorized user can be specified by adding a face image to the log.

  Furthermore, after the terminal lock means C is activated to try to look into the display 8 by another person, another person tries to look into the log information storage means 15 at the timing when the person is seated and the use restriction of the terminal 1 is released. By reading the log and displaying the face image of the person who is looking into the terminal 1, it is possible to immediately confirm who has tried to look into.

  This invention can be diverted to the use of a bank deposit / savings card or the use of a credit card at a store in addition to the use of a computer terminal.

  When diverting the present invention to such a field, since the number of users is large and the storage capacity of the face image data becomes enormous and there is a possibility that the face image data comparison and determination process may be hindered. The user's face image data corresponding to the password is read from the face image data storage means based on the input operation such as, and the face image data is compared with the face image data captured by the image pickup means at the time of use of the card. Thus, the suitability of the card use by the user is determined.

  When the use of the card is permitted, the face image data captured this time is updated and stored in the face image data storage means, so that the suitability of the user is always determined using the latest face image data as a comparison criterion. Will be able to.

It is the functional block diagram shown about the outline of the means which the terminal user monitoring system to which this invention is applied. (Best Mode for Carrying Out the Invention) It is the functional block diagram which showed the outline of the structure of the computer terminal of one Example. (Example 1 and Example 2) Face image data extraction means, face image data comparison means, terminal lock means, face image data update means, face image data transmission means, alarm output means, terminal lock release means, face image data display means, monitoring cycle adjustment means, second It is the flowchart which showed the outline of the processing operation of CPU which functions as a monitoring cycle adjustment means, a 3rd monitoring cycle adjustment means, and a comparison determination forced means. (Example 1) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 1) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 1) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 1) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 1) It is the flowchart which simplified and showed a part of subroutine program added to the application program executable during login. (Example 1) It is the conceptual diagram which showed an example of the table which memorize | stored the setting value utilized for adjustment of an operation period. (Example 1) Face image data extraction means, face image data comparison means, terminal lock means, face image data update means, face image data transmission means, alarm output means, terminal lock release means, face image data display means, monitoring cycle adjustment means, second It is the flowchart which showed the outline of the processing operation of CPU which functions as a monitoring cycle adjustment means, a 3rd monitoring cycle adjustment means, and a comparison determination forced means. (Example 2) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 2) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 2) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 2) It is a continuation of the flowchart shown about the outline of processing operation of CPU. (Example 2) It is the functional block diagram which showed the outline of the structure of another one Example. Example 3 It is the functional block diagram which showed the outline of the structure of another one Example. (Example 4)

Explanation of symbols

1 terminal 2 CPU (face image data extraction means, face image data comparison means, terminal lock means, face image data update means, face image data transmission means, alarm output means, terminal lock release means, face image data display means, monitoring cycle Adjusting means, second monitoring period adjusting means, third monitoring period adjusting means, comparison determination forcing means)
3 ROM
4 RAM
5 Nonvolatile memory (face image data storage means, second face image data storage means)
6 Input / output circuit 7 Manual data importer 8 Display 9 Speaker 10 CCD camera (imaging means)
11 Interface 12 Line 13 Monitoring server 14 Biometric authentication means 15 Log information storage means A Face image data extraction means B Face image data comparison means C Terminal lock means D Face image data update means E Face image data transmission means F Alarm output means G Terminal lock release means H Face image data display means I Imaging means J Face image data storage means K Second face image data storage means L Monitoring cycle adjustment means M Second monitoring cycle adjustment means N Third monitoring cycle adjustment means P Compulsory judgment means

Claims (14)

In a terminal user monitoring system that monitors the usage status of a computer terminal and applies usage restrictions to the terminal according to the usage status,
Imaging means installed at a position where the face of the terminal user can be photographed;
Face image data extracting means for analyzing the image picked up by the image pickup means and extracting face image data sufficient to specify a human face;
Face image data storage means for storing face image data of a user permitted to log in to the terminal;
Face image data comparing means for comparing the face image data repeatedly extracted by the face image data extracting means during login and the face image data stored in the face image data storing means to determine the degree of coincidence between them. ,
A terminal user monitoring system comprising terminal lock means for restricting a processing operation using the terminal only when the result of determination by the face image data comparison means is inconsistent. In a terminal user monitoring system that monitors the usage status of a computer terminal and applies usage restrictions to the terminal according to the usage status,
Imaging means installed at a position where the face of the terminal user can be photographed;
Face image data extracting means for analyzing the image picked up by the image pickup means and extracting face image data sufficient to specify a human face;
Face image data storage means for storing the face image data extracted by the face image data extraction means when executing login to the terminal;
Face image data comparing means for comparing the face image data repeatedly extracted by the face image data extracting means during login and the face image data stored in the face image data storing means to determine the degree of coincidence between them. ,
A terminal user monitoring system comprising terminal lock means for restricting a processing operation using the terminal only when the result of determination by the face image data comparison means is inconsistent. In a terminal user monitoring system that monitors the usage status of a computer terminal and applies usage restrictions to the terminal according to the usage status,
Imaging means installed at a position where the face of the terminal user can be photographed;
Face image data extracting means for analyzing the image picked up by the image pickup means and extracting face image data sufficient to specify a human face;
Face image data storage means for storing the face image data extracted by the face image data extraction means when executing login to the terminal;
Face image data comparing means for comparing the face image data repeatedly extracted by the face image data extracting means during login and the face image data stored in the face image data storing means to determine the degree of coincidence between them. ,
Face image data updating means for updating and storing the face image data extracted this time in the face image data storage means only when the result of determination by the face image data comparison means is substantially coincident;
A terminal user monitoring system comprising terminal lock means for restricting a processing operation using the terminal only when the result of determination by the face image data comparison means is inconsistent. The face image data extraction means has a function of extracting two or more face image data simultaneously from the image,
2. The face image data comparing means is configured to automatically determine a mismatch when two or more face image data are simultaneously extracted by the face image data extracting means. The terminal user monitoring system according to any one of claims 2 and 3.   The face image data transmitting means for automatically transmitting the face image data extracted this time to another terminal only when the result of determination by the face image data comparing means is inconsistent is provided. The terminal user monitoring system according to any one of claims 2, 3 and 4.   2. The second face image data storage means for storing the face image data extracted this time together with log information only when the determination result by the face image data comparison means is inconsistent. The terminal user monitoring system according to claim 2, claim 3, claim 4, or claim 5.   An alarm output means for outputting an alarm signal when the terminal lock means is actuated is provided together with any one of claims 1, 2, 3, 4, 5 and 6. The terminal user monitoring system according to the item.   The terminal lock releasing means for releasing the operation of the terminal locking means only when the result of determination by the face image data comparing means becomes substantially coincident after the operation of the terminal locking means is provided. The terminal user monitoring system according to claim 1, claim 2, claim 3, claim 4, claim 5, claim 6, or claim 7.   Only when the result of determination by the face image data comparison means is inconsistent, second face image data storage means for storing the face image data extracted this time together with log information, and after the operation of the terminal lock means, the face image Only when the result of determination by the data comparison means is substantially coincident, the terminal lock release means for releasing the operation of the terminal lock means, and stored in the second face image data storage means when the terminal lock release means is operated. 6. The apparatus according to claim 1, further comprising face image data display means for displaying the face image data being displayed on the display of the terminal. The terminal user monitoring system according to one item.   The terminal lock release means has a scale determination function for determining the scale of the face image data extracted by the face image data extraction means, and when the scale of the extracted face image data is less than a set value 10. The terminal user according to claim 8, wherein the terminal user is configured to maintain an inoperative state regardless of a result of determination by the face image data comparison unit. Monitoring system.   The monitoring period adjusting means for randomly changing the set values of the operation periods of the imaging means, the face image data extracting means, and the face image data comparing means during login is provided. The terminal user monitoring system according to any one of claims 2, 3, 4, 5, 6, 7, 8, 9, or 10.   Depending on the operating state of the terminal locking means, when the terminal locking means is inactive, the setting values of the operating periods of the imaging means, the face image data extracting means, and the face image data comparing means are relatively long While setting the set value of the cycle, in the state where the terminal lock unit is in operation, the second monitoring cycle adjusting unit for setting a relatively short cycle set value as the set value of the operating cycle is also provided Any one of claims 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or 11. The terminal user monitoring system according to one item.   The third monitoring period adjusting means for setting the operation period of the imaging means, the face image data extracting means, and the face image data comparing means corresponding to the type of the application program in use is provided. 1, claim 2, claim 3, claim 4, claim 5, claim 6, claim 8, claim 9, claim 10, claim 11 or claim 12 The terminal user monitoring system described in 1.   Regardless of the set values of the operation cycles of the imaging means, the face image data extraction means, and the face image data comparison means, the imaging means and the face image data extraction are received in response to a collation request from an application program in use. And a comparison determination forcing means for forcibly operating the face image data comparing means and the face image data comparing means. The terminal user monitoring system according to claim 7, claim 8, claim 9, claim 10, claim 11, claim 12, or claim 13.

Images