JP2021136016A - Method and system for identifying unauthorized image capture of secure monitor in work environment - Google Patents

Abstract

To provide a method for detecting an unauthorized image capture of a content displayed on a secure monitor, a security system, a computer program and a computer-readable storage medium.SOLUTION: In a work environment 01, a computing device 04, in a case where a content 09 which is associated with at least one authorized user and displayed on a secure monitor 08 can be browsed by at least one potential bad person 13 in a viewing cone of the secure monitor, captures a monitoring feed for each secure monitor, analyzes the monitoring feed, identifies at least one trigger event, and executes at least one correction action when the trigger event is identified.SELECTED DRAWING: Figure 1

Description

[0001] The present disclosure relates to a security system for the protection of confidential content. More specifically, the present disclosure relates to methods of the system that monitor and identify unauthorized image captures (eg, by photography) of critical content displayed on a secure monitor.

background

[0002] Analog holes are recognized as a fundamental and unavoidable vulnerability in digital data copy protection schemes. DRM and data encryption protect only digital copies. Important content displayed on the secure monitor can be viewed inside the viewing cone, as defined by the shape of the secure monitor screen, and the viewing angle of the pixels on the surface of the screen. can. The viewing angle is the angle at which the display of the secure monitor can be viewed with acceptable visual performance. The angular range of viewing angles is referred to as the viewing cone, which includes a number of viewing directions.

[0003] Inside this viewing cone, a potentially malicious third party (hereinafter referred to as the bad guy) can take unlicensed pictures of the content. A privacy screen (or privacy filter), mechanical blinds beside the security monitor, and office and cubicle walls can limit the viewing cone of the secure monitor. However, Secure Monitor content is visible when viewed by a potential bad guy sitting directly in front of the Secure Monitor or standing. A potential bad guy could be, for example, a person looking over the shoulder of an authorized user of a computing device, or an authorized user acting in violation of security rules applicable to the content displayed. ..

[0004] A well-known method of preventing unauthorized photography of critical secure monitoring content is the complete ban or involvement of mobile phones, as is common in the "red zone" of military and government security agencies. One is required to own only non-camera mobile phones. While this solution is effective, it is often impractical in civilian or moderately secure work environments where mobile phone use is widely permitted. In such a work environment, forcing a mobile phone camera ban is not practical, as most mobile phone users consider the camera to be an essential feature of their mobile phone.

[0005] Employee monitoring software is also inadequate to prevent unauthorized photography. Employee monitoring software typically includes file access restrictions, keystroke monitoring, and regular capture of screenshots of Secure Monitor content. Computer File Access Restrictions addresses the risk of authorized users (or other potential bad guys) taking unauthorized pictures of the content when the content is displayed as part of the authorized user's normal employment obligations. do not. Therefore, the employee monitoring software does not prevent unauthorized photography during the authorized viewing of computer files by authorized users.

[0006] Monitoring of suspicious activity using video recognition technology may be known. For example, U.S. Patent Application Publication No. 2012/0998918 analyzes audio-visual feeds to identify any triggering event and takes certain actions against it. One application of this recognition technique is the removal of an object from an area or the search for an additional object to an area. The addition of objects may indicate the need for an alarm to convey an evacuation warning for the area in the worst case scenario. Removal of an item can convey the theft of an item from a retail store by a customer. Optional notices may be given to security forces. This solution does not link to viewing employee content on secure monitors and does not utilize the cameras and microphones available on employee laptop computers.

[0007] Prevention of unauthorized photography has been addressed for the presentation of entertainment content in large venues. U.S. Patent Application Publication No. 2006/0998165 discloses the use of infrared light emission to interfere with the quality of images acquired by unlicensed digital recorders located within the venue. Infrared light is invisible to the human eye, but is recorded on digital recorders. This technique is useful for deterring reproduction of entertainment content when image quality is important to potential unlicensed viewers. However, this solution does not provide sufficient protection for potential bad guys who are more interested in confidential information or have advanced photography equipment.

[0008] The detection of camera use was by Dhulekar et al., entitled "Arduino based Anti-Photography System for Photography Prohibited Areas", Journal of Science, May 17 Specific measures are taken on the page. In this publication, the design involves the installation of an infrared transmitter around the camera mounted on the laptop computer. In this design, there is continuous transmission of infrared rays in the field of view of the camera. When the infrared rays hit an unlicensed camera lens, a white circular speckle appears in the image captured by the camera of the laptop computer. This white circular speckle is then detected from the video feed of the laptop computer's camera and creates a trigger event. This design requires customization of all laptop computers in the work environment and is an expensive and impractical approach given that most employee computers are available from standard commercial vendors.

[0009] Another solution to reduce unauthorized photography utilizes software installed in mobile phones.

[0010] For example, in U.S. Pat. No. 8,346,072, software installed on a mobile phone allows a user to take a picture of a subject if the device is not firmly focused on the subject for a given period of time. To prevent. This process prominently takes pictures for the camera user because it prolongs the time required for a normal camera. The rationale is that if such an action is easily recognizable, it is unlikely that the camera user will take an unauthorized photo. Camera restrictions allow the device to communicate over a wireless communication network. In addition, limits and boundary values can be communicated to the device as part of the IT security policy. In US Patent Application Publication No. 2013/0040632, the software installed on a mobile phone uploads all the pictures taken on that mobile phone to the server. Initially, the software does not allow any local duplication of image files on mobile phones. The server analyzes the photo to determine if it is authorized. If approved, the photo will be returned to the mobile phone, otherwise the mobile phone user will be denied copying the photo.

[0011] These two software solutions are limited in practical use because they require the installation of intrusive software on all mobile phones present in the work environment. Potential bad guys alternately remove such software from their cell phones and use another cell phone unknown to their employer to take unlicensed pictures, or instead of unlicensed photography May use a stand-alone camera for.

[0012] As discussed above, prior art limits the types of mobile phones allowed on the premises (eg, providing a red zone in the work environment), modifying laptop computer hardware (eg, providing a red zone). For example, adding infrared light radiation), or without installing security software on each cell phone, cannot fully address the need to prevent potential bad guys from taking unauthorized pictures. .. Furthermore, the prior art does not disclose or propose any method of documenting the content viewed on the secure monitor at the time of the attempted unauthorized photography.

Overview

[0013] A first embodiment of the invention is a method of detecting an unauthorized image capture of content displayed on at least one secure monitor, configured by a computing device connected to the secure monitor. Each computing device is associated with at least one authorized user, and the content displayed on each secure monitor can be viewed within the viewing cone of the secure monitor by at least one potential bad guy. The method is (a) a step of capturing a surveillance feed for each secure monitor, wherein the surveillance feed contains a stream of images for at least a portion of the viewing cone of the secure monitor, and (b) each secure monitor. The step of analyzing the monitoring feed associated with is to identify at least one trigger event, (c) when each trigger event is identified, the step of storing the trigger record for the trigger event, and (d) each trigger event. Identifying includes a step of performing at least one corrective action.

[0014] In a first embodiment, the stream of images for the surveillance feed is (i) a first computer connected to a computing device connected to a secure monitor that captures the first substream of images. Camera, second computer camera connected to a computing device connected to a secure monitor, capturing a second substream of (iii) images, capturing a third substream of (iii) images, secure Captured by at least one of at least one CCTV camera attached to the work environment associated with the monitor. Each trigger event is (i) captured in the viewing cone of the secure monitor in at least one of the streams of the image, and (ii) the content displayed on the secure monitor as the property of a potential bad guy. The displayed content, including at least the identification information of the camera unit positioned to be captured, is taken from at least one computer file. Each computer file is stored in at least one of (1) a computing device connected to the secure monitor and (2) at least one content repository accessible by the computing device connected to the secure monitor. .. The trigger recordings for the trigger event include (i) a time stamp for the trigger event, (ii) (1) a screenshot of the secure monitor that encountered the trigger event at the time of the trigger event, and (2) the trigger event. Includes at least one of the content summaries displayed on the secure monitor at the time, as well as at least one image from the surveillance feed associated with the trigger event that captures the (iii) camera unit. Each corrective action includes (i) a step of locking down a secure monitor that encounters a trigger event with one or more lockdown actions, and (ii) a step of sending a security alert to at least one designated person. Includes at least one of.

[0015] The trigger recording for each trigger event is the meta for each computer file displayed on the secure monitor that encountered the trigger event in (a) the first time window extending before and after the trigger event. Data, (b) Login identification information for each authorized user who logged in to the computing device connected to the secure monitor in the first time window, (c) Secure monitor that encountered a trigger event in the first time window. Associated with the first captured block of the monitoring feed associated with, and (d) other secure monitors co-located in the work environment associated with the secure monitor that encountered the trigger event in the first time window. It may further include at least one of the second captured blocks of the additional surveillance feed provided.

[0016] The step of analyzing the surveillance feed can be performed by AI / ML.

The monitoring feed for each secure monitor is (a) a stream of audio captured by a computer microphone connected to the computing device connected to the secure monitor, and (b) to the computing device connected to the secure monitor. At least one of a stream of typing rhythms captured by a connected computer keyboard and (c) a stream of pointer control characteristics captured by a computer pointer connected to a computing device connected to a secure monitor. Additional biometric information can be included. The steps of analyzing the surveillance feed to identify each trigger event are: (a) a stream of voice and a pre-recorded voice of at least one of the authorized users of the computing device connected to the secure monitor. A comparison of 1, (b) a stream of typing rhythms and a second comparison of pre-recorded typing rhythms of at least one authorized user of a computing device connected to a secure monitor, and (c) pointer control characteristics. It can include at least one analysis of a third comparison between the stream and the pre-recorded pointer control characteristics of at least one authorized user of the computing device connected to the secure monitor. The step of analyzing the surveillance feed to identify each trigger event is latent by at least one of (a) a stream of audio, (b) a stream of typing rhythms, and (c) a stream of pointer control characteristics. A second behavioral analysis of the bad guys can be additionally included. Trigger recording for each trigger event is for a second time window that extends before and after the trigger event, (a) a third captured block of the audio stream in the second time window, and (b) a second time. It further comprises at least one of a fourth captured block of the typing rhythm stream in the window and (c) a fifth captured block of the pointer control characteristic stream in the second time window. be able to.

[0018] In a second embodiment of the invention, the method of the first embodiment is implemented in a security system, the system including a server with a database. The server is connected to the computing device connected to the secure monitor via the network. The system comprises the method of the first embodiment of the present invention by at least one of (a) a first processor in a computing device connected to a secure monitor and (b) a second processor in a server. Is configured to run.

[0019] In a third embodiment of the invention, the invention is contained in a computer program containing instructions, the instructions being (a) a first processor in a computing device connected to a secure monitor, and (b) secure. A computing device connected to a monitor, when executed by at least one of a second processor in a server connected over a network, causes the method of the first embodiment to be executed.

[0020] In a fourth embodiment of the invention, the invention is contained in a computer-readable storage medium containing instructions, the instructions being (a) a first processor in a computing device connected to at least one secure monitor. And (b) a first embodiment of the invention when executed by at least one of a second processor in a server connected over a network to a computing device connected to a secure monitor. To execute.

[0021] Embodiments of the present disclosure will be described below with reference to the drawings.
FIG. 1 is a diagram of items placed in the work environment during execution of an embodiment of the present invention. FIG. 2 is a block diagram of items in a security system connected via a network according to an embodiment of the present invention. FIG. 3 is a flow chart of steps implemented in an embodiment of the invention for identifying an unauthorized image capture of content displayed on a secure monitor. FIG. 4 is a detailed chart of a stream of information generated by the equipment in the system. FIG. 5 is a list of potential lockdown steps for corrective action after a trigger event.

Detailed explanation

[0022] The following detailed description makes reference to the accompanying drawings that form part of this specification. The detailed description, drawings, and exemplary embodiments described in the claims are not meant to be limiting. Other embodiments may be utilized and other modifications may be made without departing from the spirit or scope of the subject matter presented herein. Unless otherwise specified, the terms "comprising," "complying," "inclusion," and "include," as used herein, and their terminology. Grammatic variants are intended to express "open" or "inclusive" terminology to allow the inclusion of additional, unlisted elements as well as those that contain them.

[0023] As used herein, the computing device 04 can be a desktop computer or laptop computer, a single stand-alone computer, a thin client, a tablet computer, or a mobile phone. The computing device 04 can run a local operating system and store computer files on a local storage drive. The computing device 04 can access files and applications through a gateway 22 to one or more content repositories 26, which hosts the files and / or runs virtual applications and computes. A virtual desktop for the wing device 04 can be generated.

[0024] As used herein, the secure monitor 08 is a display device for viewing content 09. The secure monitor 08 may be a dedicated screen (such as a laptop computer, a thin client, a tablet computer, or a screen display for a mobile phone) built into the computing device 04. The secure monitor 08 may be a stand-alone display (such as a computer display, projection screen, or conference room monitor connected to the computing device 04).

[0025] As used herein, the server 24 is a single stand-alone computer, a single dedicated server, a plurality of dedicated servers, and / or a virtual server running on a large network 23 of servers 24. And / or cloud-based services. As used herein, database 25 sends data to a single stand-alone computer, a data server, multiple dedicated data servers, cloud-based services, and / or virtual servers running on a large network of servers. You can store and access data from them.

[0026] As used herein, the network 23 may be any means of achieving communication between the computing device 04 and content 09 stored somewhere else. As used herein, the network 23 can be a personal area network, a local area network, a storage area network, a system area network, a wide area network, a virtual private network, and an enterprise private network. When implemented in the present invention, the network 23 may or may not include one or more gateways 22. Network 23 communication can be performed by a publicly available standard protocol or a proprietary protocol. The communication of data through the network 23 includes (i) encrypted or unencrypted, (ii) wired network, wireless network, or transmission via a combination of wired and wireless. Can be mentioned. Wireless communication is feasible in any practical manner, including Wi-Fi 802.11 networks, Bluetooth® networks, or mobile phone networks (3G, 4G, LTE, and 5G, etc.). As used herein, the terms "connected," "connected," and "connecting" refer to a communication link between two devices and are used in this paragraph. It is feasible as discussed in.

[0027] As used herein, the network 23 can be connected, at least in part, along with proprietary network 23 protocols, such as for CCTV and motion detector 03 devices. CCTV and motion detector 03 devices are common Internet of Things (IoT) such as narrowband IoT (“NB-IoT”), 3G SIM card-based mobile communication networks, or Message Queuing Telemetry Transport. ) Communication is also possible by protocol. In the case of IoT devices, the IoT device may be in communication with the gateway 22 and / or network 23 via at least one of a wireless transmitter / receiver and a wired link. Communication between the IoT sensor, gateway 22, server 24, and user interface can also be encrypted and bidirectional. For example, at least one of the IoT sensors can be configured to receive at least one command from the server 24 through the gateway 22.

[0028] As used herein, the user interface device connected to the computing device 04 can be connected to the computing device 04 via a dedicated wire or wirelessly. The touchpad 07 can be integrated into a laptop computer or a separate tracking device. The computer microphone 12 can be integrated into a computing device 04 (typically designed within laptop computers, tablet computers, and mobile phones), and a separate device that can be connected to the computing device 04.

[0029] As used herein, computer image capture devices include video cameras and cameras. The camera can be any type of camera connected to the computing device 04. For example, a dedicated camera can be incorporated at the top of the display screen of laptop computers, tablet computers, and mobile phones. Modern mobile phones have two or more cameras, a "selfie" camera specifically aimed at the intended device user, and a second camera oriented in the opposite direction of the display screen. Another general option for computer cameras is a webcam that can be connected to the computing device 04.

[0030] As used herein, the viewing angle is the angle at which the display of Secure Monitor 08 can be viewed with acceptable visual performance. The angular range of viewing angles is referred to as the viewing cone, which includes a number of viewing directions. Inside this viewing cone, a potential bad guy 13 can take an unlicensed image, such as photo 15, of content 09. In general, the content 09 of the secure monitor 08 is visible when viewed by a potential bad guy 13 sitting or standing directly in front of the secure monitor 08.

[0031] As used herein, the camera unit 14 can be any device capable of taking a picture of content 09 on a secure monitor 08. The most common camera unit 14 typically used by employees will be mobile phones and stand-alone digital cameras (such as compact, bridge, mirrorless, and DSLR). The camera unit 14, as used herein, also includes film camera and cover camera designs of different dimensions.

[0032] As used herein, the designated person receiving a security warning may be an onsite or offsite security provider. The designated person may be a supervisor or another member of the Work Environment 01 staff designated to receive security alerts. Two or more persons can be listed in the system 21 as designated persons.

[0033] As used herein, the terms "associate", "associate", "associate" and "associate" are two items and / Or indicate a defined relationship (or cross-reference) between persons. For example, the computing device 04 can be associated with at least one authorized user, indicating that such user is authorized to log in to the computing device 04. The computing device 04 can have a plurality of licensed users such as employees and IT administrators. For example, the computing device 04 can be associated with at least one trigger event and indicates a trigger event for a potential unauthorized image capture (eg, by photography) of the connected secure monitor 08 of the computing device 04. For example, the work environment 01, the surveillance feed, the CCTV camera 02, or the motion detector 03 can be associated with the secure monitor 08 or its connected computing device 04. For example, the work environment 01 includes many work stations, and each work station may be accompanied by a secure monitor 08 connected to the computing device 04. The database 25 in the server 24 allows various associations to be entered, cross-referenced, and updated.

[0034] At least one potential bad guy 13 can be someone who seeks to obtain an unauthorized copy or image (such as Photo 15) of Content 09. Each potential bad guy 13 could be one of the authorized users of the device or one of its colleagues. The potential bad guy 13 may be a staff member who has authorized access to work environment 01 but does not have authorized access to content 09. The potential bad guy 13 may be an intruder into the work environment 01. There are multiple bad guys, and each potential bad guy 13 can have varying levels of criminality, from negligent to reckless, intentional and / or malicious. For example, the negligence or deception of one potential bad guy 13 can be abused by a second potential bad guy 13. For this reason, the present invention considers a wide window of surveillance feed capture both before and after a trigger event, capturing potential activity by multiple potential bad guys 13 and even new security or other professional staff. Provide potential training footage for use.

Content 09 that can be viewed on the secure monitor 08 may be generated by a computer file that resides in the computing device 04 and / or in the content repository 26. The content repository 26 can be one or more servers, computers, or data storage devices accessible by the computing device 04 through the network 23.

[0036] As used herein, work environment 01 can be a desk, cubicle, office, or office setting with other suitable placement configurations in the work setting. Since the present invention can be implemented by a device (laptop computer, tablet computer, etc.) that can be taken out of the office setting, the work environment 01 is a professional home office, library, conference room, cafe, train. , Airplanes, buses, or other places where portable work can be done. Outside the office setting, CCTV camera 02 and motion detector 03 are typically not available, and computer audio is too noisy for reliable analysis. However, if network 23 is available, a first substream of images from a first camera (as in laptop computers, tablet computers, and mobile phones), a stream of typing rhythms from computer keyboard 05, And a stream of pointer control of computer pointers is available for analysis.

[0037] As used herein, a computer pointer can be any device used to position a cursor on a secure monitor 08. Examples of pointer devices for the computing device 04 include a computer mouse 06, a touch pad 07, a touch screen, and a pen stylus.

[0038] FIG. 1 is a diagram of items placed in the work environment 01 during execution of an embodiment of the present invention. As shown, the computing device 04 is depicted as a laptop computer. The computing device 04 includes a secure monitor 08 for viewing content 09, a keyboard 05, a touchpad 07, a first computer camera 10 aimed at potential users of the computing device 04, and a computer microphone 12. Other devices connected to the computing device 04 include a computer mouse 06 and a second computer camera 11 (depicted as a webcam in FIG. 1). The secure monitor 08 is arranged in the work environment 01. A CCTV camera 02 and a motion detector 03 are installed in the work environment 01. In this work environment 01, a potential bad guy 13 has a camera unit 14 (depicted here as a mobile phone) in the viewing cone of the secure monitor 08. The camera 14 includes a camera (not shown) in the viewing cone of the secure monitor 08, and the screen of the camera unit 14 displays the unlicensed photograph 15 of the content 09 displayed on the secure monitor 08. As depicted, first and second computer cameras 11 are placed as a stream of images to capture the actions of a potential bad guy 13 taking an unauthorized photo 15 of content 09 on secure monitor 08. Has been done.

[0039] FIG. 2 is a block diagram of items in the security system 21 connected via the network 23 of the embodiment of the present invention. The work environment 01 includes a computing device 04, a CCTV camera 02, and a motion detector 03 connected to the network 23 via a gateway 22. The network 23 is connected to a plurality of content repositories 26 and servers 24. The server 24 includes a database 25. The computing device 04 includes a keyboard 05, a computer mouse 06, a touchpad 07, a first computer camera 10, a second computer camera 11, and a computer microphone 12, all of which are network interface cards (“NIC”). ) 27 to connect to the gateway 22.

[0040] FIG. 3 is a flowchart 3-0 of a general step implemented in an embodiment of the present invention. Steps 3--013-07 of FIG. 3 include:
3-01 Computing device for work environment 01 to capture a stream of images from each computing device 04 for at least a portion of the viewing cone of Secure Monitor 08 for analysis as part of the surveillance feed. Install surveillance software on 04 3-02 Connect CCTV camera 02 and motion detector 03 to gateway 22 and route these substreams to server 24 for analysis as an additional part of the surveillance feed 3-03 Time stamp a 3-04 trigger recording that analyzes the stream of images to identify the camera unit 14 that is in the viewing cone of the secure monitor 08 and is located to capture the content 09 displayed on the secure monitor 08. 3-05 Remember with screenshots of Secure Monitor 08 and images of potential bad guys 3-06 Security alert to security vendors locking down compute device 04 for associated Secure Monitor 08 encountering a trigger event 3-07 Attempts to identify and locate potential bad guys 13 based on information from the monitoring feed

[0041] FIG. 4 is a chart 4-00 detailing a stream of information generated by equipment in the system 21 that may be included in the surveillance feed. As detailed in this chart, the first computer camera 10 produces a first substream of images of the potential bad guy 13 and the second computer camera 11 is of the image of the potential bad guy 13. A second substream is generated, the CCTV camera 02 generates a third substream of the image of the potential bad guy 13, the computer microphone 12 generates a stream of audio of the potential bad guy 13, and the computer keyboard 05. Generates a stream of typing rhythms for the user of computing device 04 (which can also be the potential bad guy 13), and the computer pointer is a pointer control characteristic of the user of computing device 04 (which can also be the potential bad guy 13). Generate a stream. Note that all this information is potentially assembled and analyzed to determine any trigger event and potentially document the trigger record for the trigger event.

[0042] FIG. 5 is a list 5-00 of potential lockdown steps for corrective action after a trigger event. The order of the lockdown steps is exemplary and all or part of these steps may be performed in a different order or in parallel. Potential lockdown steps are listed as 5-01-5-07 in FIG. 5, including:
5-01 Deactivate Secure Monitor 08 5-02 Close one or more windows displaying Content 09 on Secure Monitor 08 5-03 Lock each computer file displayed on Secure Monitor 08 5-04 Delete each computer file displayed on Secure Monitor 08 from Computing Device 04 5-05 The compute of an authorized user who logged in to Compute Device 04 connected to Secure Monitor 08 during the trigger event. Restricting Access to Ing Device 04 5-06 Restricting Access to Each Content Repository 26 Accessable by the Computing Device 04 Associated with the Trigger Event 5-07 Computer Device 04 Associated with the Trigger Event Wipe at least part of your local storage drive

[0043] A first embodiment of the present invention is a method of detecting an unauthorized image capture (eg, by photography) of content 09 displayed on at least one secure monitor 08. The content 09 displayed on each secure monitor 08 is composed of a computing device 04 connected to the secure monitor 08, and each computing device 04 is associated with at least one authorized user and is associated with each secure monitor 08. The displayed content 09 can be viewed in the viewing cone of the secure monitor 08 by at least one potential bad guy 13. The method is (a) a step of capturing a surveillance feed for each secure monitor 08, wherein the surveillance feed contains a stream of images for at least a portion of the viewing cone of the secure monitor 08, and (b) each. A step of analyzing the monitoring feed associated with Secure Monitor 08 to identify at least one trigger event, and (c) a step of storing a trigger record for each trigger event when each trigger event is identified, and (d). Identifying each trigger event includes a step of performing at least one corrective action. The stream of images for the surveillance feed is (i) a first computer camera 10 connected to a computing device 04 connected to a secure monitor 08, which captures a first substream of images, (ii) images. A second computer camera 11 connected to a computing device 04 connected to a secure monitor 08 that captures a second substream, (iii) associated with a secure monitor 08 that captures a third substream of an image. Captured by at least one of at least one CCTV camera 02 attached to the working environment 01. Each trigger event is (i) captured in the viewing cone of the secure monitor 08 in at least one of the streams of the image and displayed on the secure monitor 08 as the property of (ii) the potential bad guy 13. The displayed content 09 is taken from at least one computer file, including at least the identification of the camera unit 14, which is positioned to capture the content 09. Each computer file is at least one of (1) the computing device 04 connected to the secure monitor 08 and (2) at least one content repository 26 accessible by the computing device 04 connected to the secure monitor 08. It is remembered at one time. The trigger recordings for the trigger event include (i) a time stamp for the trigger event, (ii) (1) a screenshot of the secure monitor 08 that encountered the trigger event at the time of the trigger event, and (2) the trigger event. Includes at least one of the summary of content 09 displayed on the secure monitor 08 at the time of, and at least one image from the surveillance feed associated with the trigger event that captures (iii) camera unit 14. .. Each corrective action involves (i) locking down the secure monitor 08 that encountered the trigger event with one or more lockdown actions, and (ii) sending a security alert to at least one designated person. Includes at least one of.

[0044] To identify a camera, the computing device 04 or server 24 stores a library of exemplary images, or a model of a sample camera unit 14, and then secures it in at least one of the streams of images. A pattern recognition step can be performed to identify any camera unit 14 captured within the viewing cone of monitor 08. A step of evaluating whether the camera unit 14 is positioned to capture the content 09 displayed on the secure monitor 08 is performed to reduce the number of false alarms. For example, to distinguish between normal cell phone use for text entry, web surfing, and phone calling from the act of taking a picture. For pattern recognition, the advent of AI / ML software can be utilized. Considering the high computational effort of AI / ML, the pattern recognition step is more likely to be performed on the server 24 than on the computing device 04.

[0045] The trigger recording for each trigger event is for each computer file displayed on the secure monitor 08 that encountered the trigger event in (a) the first time window for the first time window extending before and after the trigger event. Metadata, (b) Login identification information for each authorized user who logged in to the computing device 04 connected to Secure Monitor 08 in the first time window, (c) Trigger event in the first time window. Placed together in the first captured block of the monitoring feed associated with the Encountered Secure Monitor 08, and (d) the Work Environment 01 associated with the Encountered Secure Monitor 08 in the first time window. It may further include at least one of a second captured block of additional monitoring feeds associated with the other secure monitor 08.

[0046] The step of analyzing the surveillance feed to identify each trigger event can include a first behavioral analysis of the potential bad guy 13 with a stream of images.

The step of analyzing the surveillance feed to identify each trigger event can include comparing the stream of images with pre-recorded images of at least one authorized user of the computing device 04.

[0048] The step of analyzing the surveillance feed can be performed by AI / ML.

The monitoring feeds are (a) motion detection information from at least one motion detector 03 in the work environment 01 associated with the secure monitor 08, and (b) a computing device connected to the secure monitor 08. It may additionally include at least one of the location information accessed by at least one mobile device associated with one or more of the 04 authorized users. The mobile device can be a mobile phone, tablet computer, or other personal device that tracks the location of its users.

[0050] The monitoring feed for each secure monitor 08 is connected to (a) a stream of audio captured by the computer microphone 12 connected to the computing device 04 connected to the secure monitor 08, and (b) to the secure monitor 08. A stream of typing rhythms captured by the computer keyboard 05 connected to the computing device 04, and (c) pointer control characteristics captured by the computer pointer connected to the computing device 04 connected to the secure monitor 08. Additional biometric information, including at least one of the streams, can be included. The steps to analyze the surveillance feed and identify each trigger event are: (a) a stream of voice and pre-recorded voice of at least one of the authorized users of computing device 04 connected to Secure Monitor 08. First comparison of (b) a stream of typing rhythms and a second comparison of pre-recorded typing rhythms of at least one authorized user of the computing device 04 connected to Secure Monitor 08, and (c). It may include at least one analysis of a stream of pointer control characteristics and a third comparison of the pre-recorded pointer control characteristics of at least one authorized user of the computing device 04 connected to Secure Monitor 08. can. The step of analyzing the surveillance feed to identify each trigger event is latent by at least one of (a) a stream of audio, (b) a stream of typing rhythms, and (c) a stream of pointer control characteristics. A second behavioral analysis of the bad guy 13 can be additionally included. Trigger recording for each trigger event is for a second time window that extends before and after the trigger event, (a) a third captured block of the audio stream in the second time window, and (b) a second time. It further comprises at least one of a fourth captured block of the typing rhythm stream in the window and (c) a fifth captured block of the pointer control characteristic stream in the second time window. be able to.

[0051] The pre-recorded voice of at least one of the authorized users of the compute device 04 connected to the secure monitor 08 is even if the entry to the compute device 04 is due to a proper login and password. It can act like a fingerprint when identifying authorized users. The pre-recording typing rhythm or pre-recording pointer control characteristic of at least one authorized user of the computing device 04 connected to the secure monitor 08 also acts like a fingerprint in identifying the authorized user. For example, the difference between an authorized user and a fake can be identified by comparing voice patterns, typing rhythms, and / or pointer control characteristics with pre-recorded samples.

[0052] During the trigger event, one or more lockdown actions for the computing device 04 connected to the secure monitor 08 may (a) deactivate the secure monitor 08, (b). Closing one or more windows displaying content 09 on Secure Monitor 08, (c) locking each computer file displayed on Secure Monitor 08, (d) displayed on Secure Monitor 08 Deleting each computer file from the computing device 04, (e) accessing the computing device 04 by an authorized user logged in to the computing device 04 connected to the secure monitor 08 during the trigger event. Restricting, (f) restricting access to each content repository 26 accessible by the computing device 04 associated with the trigger event, and (g) restricting access to each content repository 26 associated with the trigger event, and (g) local computing device 04 associated with the trigger event. It can include at least one of wiping at least a portion of the storage drive.

Security alerts for each trigger event are associated with (a) email, (b) text messages, (c) phone calls, (d) voice alarms, and (e) Secure Monitor 08 encountering trigger events. It can include at least one of at least a portion of the surveillance feed sent.

Each trigger recording includes (a) a computing device 04 connected to a secure monitor 08 receiving a trigger event, and (b) a computing device 04 connected to a secure monitor 08 receiving a trigger event. It can be stored in at least one of the databases 25 of the server 24 connected via the network 23.

The steps of analyzing the monitoring feed were (a) a computing device 04 connected to the secure monitor 08, and (b) a computing device 04 connected to the secure monitor 08 connected via network 23. This can be done on at least one of the servers 24.

[0056] In a second embodiment of the invention, the method of the first embodiment is implemented in a security system 21, which includes a server 24 with a database 25. The server 24 is connected to the computing device 04 connected to the secure monitor 08 via the network 23. The system 21 is according to the first embodiment by at least one of (a) a first processor in the computing device 04 connected to the secure monitor 08 and (b) a second processor in the server 24. Configured to perform the method. For example, the computing device 04 can be loaded with software that captures the surveillance feed and pushes it to the server 24, which performs computer-intensive analysis of the surveillance feed.

[0057] In a third embodiment of the invention, the invention is included in a computer program containing instructions, the instructions being (a) a first processor in a computing device 04 connected to a secure monitor 08, (b). When executed by at least one of the second processors in the server 24 connected via the network 23 to the computing device 04 connected to the secure monitor 08, the method of the first embodiment. Let it run. For example, the computing device 04 can be loaded with software that captures the surveillance feed and also analyzes the surveillance feed. Alternatively, the computing device 04 can be loaded with software that captures the surveillance feed and pushes it to the server 24, which performs a computer-intensive analysis of the surveillance feed.

[0058] In a fourth embodiment of the invention, the invention is contained in a computer-readable storage medium containing instructions, the instructions being (a) a first in a computing device 04 connected to at least one secure monitor 08. The processor, (b) the computing device 04 connected to the secure monitor 08, is executed by at least one of the second processors in the server 24 connected via the network 23, according to the present invention. The first embodiment is executed. For example, the computing device 04 can be loaded with software that captures the surveillance feed and also analyzes the surveillance feed. Alternatively, the computing device 04 can be loaded with software that captures the surveillance feed and pushes it to the server 24, which performs a computer-intensive analysis of the surveillance feed.

Although various aspects and embodiments have been disclosed herein, those skilled in the art will appreciate the various aspects of the invention without departing from the spirit and scope of the invention. It will be clear that other modifications and conformances will be apparent and that all such modifications and conformances are intended to fall within the scope of the appended claims. The various aspects and embodiments disclosed herein are for illustration purposes only and are not intended to be limiting, and the true scope and intent of the invention is in the appended claims. Shown.

01 Work environment 02 CCTV camera 03 Motion detector 04 Computing device 05 Keyboard 06 Computer mouse 07 Touch pad 08 Secure monitor 09 Content 10 First computer camera 11 Second computer camera 12 Computer microphone 13 Potential bad guy 14 Camera unit 15 Photo 21 System 22 Gateway 23 Network 24 Server 25 Database 26 Content Repository 27 Network Interface Card [Summary]
A first embodiment of the present invention captures an unauthorized image capture (eg, by photography) of content 09 displayed on at least one secure monitor 08, configured by a computing device 04 connected to the secure monitor 08. It is a method of detection. Each computing device 04 is associated with at least one authorized user, and the content 09 displayed on each secure monitor 08 is viewed in the viewing cone of the secure monitor 08 by at least one potential bad guy 13. It is possible. The method is (a) a step of capturing a surveillance feed for each secure monitor 08, wherein the surveillance feed contains a stream of images for at least a portion of the viewing cone of the secure monitor 08, and (b) each. A step of analyzing the monitoring feed associated with Secure Monitor 08 to identify at least one trigger event, and (c) a step of storing a trigger record for each trigger event when each trigger event is identified, and (d). Identifying each trigger event includes a step of performing at least one corrective action. The present invention can also be executed by a computer program instruction or by a server 24 in the security system 21.

Claims (17)

A method of detecting an unauthorized image capture of content displayed on at least one secure monitor.
The content displayed on each secure monitor is composed of a computing device connected to the secure monitor, and the computing device is associated with at least one authorized user and displayed on each secure monitor. Content is viewable within the viewing cone of the secure monitor by at least one potential bad guy.
The above method
(A) A step of capturing a surveillance feed for each secure monitor, wherein the surveillance feed includes a stream of images for at least a portion of the viewing cone of the secure monitor and a stream of the images for the surveillance feed. but,
(I) A first computer image capture device connected to the computing device connected to the secure monitor that captures the first substream of the image.
(Ii) Capture a second substream of the image, a second computer image capture device connected to the computing device connected to the secure monitor, and (iii) capture a third substream of the image. And the steps captured by at least one of the at least one CCTV image capture device attached to the work environment associated with the secure monitor.
(B) A step of analyzing the monitoring feed associated with each secure monitor to identify at least one trigger event, each of the at least one trigger event.
(I) At least one of the streams of the image is captured in the viewing cone of the secure monitor.
(Ii) The displayed content includes at least the identification information of the camera unit, which is positioned to acquire the image of the content displayed on the secure monitor as the property of the potential bad guy, and the displayed content is at least 1. Extracted from two computer files, each computer file is accessible by (1) the computing device connected to the secure monitor and (2) the computing device connected to the secure monitor. Steps and steps stored in at least one of the at least one content repository ...
(C) When each trigger event is identified, it is a step of storing the trigger record for the trigger event, and as the trigger record for the trigger event.
(I) Timestamp for the trigger event,
(Ii) (1) A screenshot of the secure monitor that encountered the trigger event at the time of the trigger event, and (2) an overview of the content displayed on the secure monitor at the time of the trigger event. , And (iii) a step comprising at least one image from the surveillance feed associated with the trigger event that captures the camera unit.
(D) When each trigger event is identified, it is a step of executing at least one corrective action, and each corrective action is
(I) A step of locking down the secure monitor that has encountered the trigger event with one or more lockdown actions.
(Ii) A method comprising, including, including at least one of a step of sending a security alert to at least one designated person. For a first time window in which the trigger recording for each trigger event extends before and after the trigger event.
(A) Metadata for each computer file displayed on the secure monitor that encountered the trigger event in the first time window.
(B) Login identification information for each authorized user who logged in to the computing device connected to the secure monitor in the first time window (c) The trigger event was encountered in the first time window. Placed together in the first captured block of the monitoring feed associated with the secure monitor and (d) in the work environment associated with the secure monitor that encountered the trigger event within the first time window. The method of claim 1, further comprising at least one of a second captured block of additional monitoring feed associated with another secure monitor. The method of claim 1 or 2, wherein the step of analyzing the surveillance feed to identify each trigger event comprises a first behavioral analysis of the potential bad guy with a stream of images. The steps of analyzing the surveillance feed to identify each trigger event include comparing the stream of images with pre-recorded images of at least one authorized user of the computing device. The method according to any one of 3. The method of any one of claims 1-4, wherein the step of analyzing the monitoring feed is performed by AI / ML. The monitoring feed
Of the motion detection information from at least one motion detector in the work environment associated with the secure monitor, and (b) the authorized user of the computing device connected to the secure monitor. Location information, accessed by at least one mobile device associated with one or more of the
The method according to any one of claims 1 to 5, further comprising at least one of the above. The monitoring feed for each secure monitor
(A) A stream of audio captured by a computer microphone connected to the computing device connected to the secure monitor.
(B) A stream of typing rhythms captured by a computer keyboard connected to the computing device connected to the secure monitor, and (c) a computer pointer connected to the computing device connected to the secure monitor. The method of any one of claims 1-6, further comprising additional biometric information including at least one of a stream of pointer control characteristics captured by. The step of analyzing the monitoring feed to identify each trigger event is
(A) A first comparison between the audio stream and the pre-recorded audio of at least one of the authorized users of the computing device connected to the secure monitor.
A second comparison of the typing rhythm stream with the pre-recorded typing rhythm of at least one authorized user of the computing device connected to the secure monitor, and (c) of the pointer control characteristics. 7. The 7. Method. The step of analyzing the monitoring feed to identify each trigger event is
(A) The audio stream,
The seventh aspect of claim 7, further comprising a second behavioral analysis of the potential bad guy by (b) a stream of the typing rhythm and (c) a stream of the pointer control characteristics. Method. For a second time window in which the trigger recording for each trigger event extends before and after the trigger event.
(A) A third captured block of the audio stream in the second time window,
(B) A fourth captured block of the typing rhythm stream in the second time window, and (c) a fifth captured block of the pointer control characteristic stream in the second time window. The method of claim 7, further comprising at least one of the blocks. The one or more lockdown actions for the computing device connected to the secure monitor during the trigger event.
(A) Deactivating the secure monitor,
(B) Closing one or more windows displaying the content on the secure monitor.
(C) Locking each computer file displayed on the secure monitor,
(D) Deleting each computer file displayed on the secure monitor from the computing device.
(E) Restricting access to the computing device by the authorized user who logged in to the computing device connected to the secure monitor during the trigger event.
(F) Restricting access to each content repository accessible by the computing device associated with the trigger event, and (g) of the local storage drive of the computing device associated with the trigger event. The method of any one of claims 1-10, comprising at least one of wiping at least a portion. The security warning for each trigger event
(A) Email,
(B) Text message,
(C) Calling up the phone
Any one of claims 1-11, comprising (d) a voice alarm and (e) at least one of at least one of the transmissions of the monitoring feed associated with the secure monitor that has encountered the trigger event. The method described in the section. Each trigger record is
(A) The computing device connected to the secure monitor receiving the trigger event, and (b) the computing device connected to the secure monitor receiving the trigger event via a network. The method according to any one of claims 1 to 12, which is stored in at least one of the connected server databases. The steps of analyzing the monitoring feed are (a) the computing device connected to the secure monitor, and (b) the server connected to the computing device connected to the secure monitor via a network. The method according to any one of claims 1 to 12, which is carried out by at least one of them. A security system with a server with a database
The server is connected to a computing device connected to a secure monitor via a network.
The system
Any one of claims 1-12 by at least one of (a) a first processor in the computing device connected to the secure monitor and (b) a second processor in the server. A security system configured to perform the method described in. A computer program containing instructions, the instructions to (a) a first processor in a computing device connected to a secure monitor, and (b) a computing device connected to a secure monitor over a network. A computer program that, when executed by at least one of the second processors in a connected server, causes the method according to any one of claims 1-12 to be performed. A computer-readable storage medium containing an instruction, wherein the instruction is attached to (a) a first processor in a computing device connected to at least one secure monitor, and (b) to a computing device connected to the secure monitor. A computer-readable storage medium that, when executed by at least one of the second processors in a server connected via a network, causes the method of any one of claims 1-12 to be performed.

Images