CN114285761B - Diving board machine illegal operation detection method based on video recording and OCR technology - Google Patents
Diving board machine illegal operation detection method based on video recording and OCR technology Download PDFInfo
- Publication number
- CN114285761B CN114285761B CN202111615997.8A CN202111615997A CN114285761B CN 114285761 B CN114285761 B CN 114285761B CN 202111615997 A CN202111615997 A CN 202111615997A CN 114285761 B CN114285761 B CN 114285761B
- Authority
- CN
- China
- Prior art keywords
- user
- character string
- illegal
- input
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method for detecting illegal operations of a springboard machine based on video screen recording and OCR technology, which comprises the steps of firstly recording a screen of a user operation interface, then detecting and judging the position of a command line input by a user through cursor detection or fixed character string detection aiming at the video of the recorded screen, converting the command input by the user into a text through the OCR technology, and then detecting illegal operation words of the text. The method for detecting the illegal operation of the gangway machine based on the video recording and OCR technology provided by the invention realizes the monitoring and operation and maintenance personnel operation of the gangway machine, audits and controls the operation and maintenance personnel operation of the gangway machine, audits and determines responsibility for misoperation and illegal operation of a user, reduces the cost for audit and operation and maintenance by using the fort machine, and additionally does not need hardware facilities, software installation and the like, so that the monitoring, audit and operation and maintenance of the operation of the user at a terminal are lighter and simpler.
Description
Technical Field
The invention relates to the technical field of network communication, in particular to a diving board machine illegal operation detection method based on video recording and OCR technology.
Background
In order to remotely log in for centralized management of operation and maintenance personnel, a Jump board machine (a Server of a Unix/Windows system) is often deployed, and a schematic diagram of the Jump board machine is shown in fig. 1, and all operation and maintenance personnel need to remotely log in the Jump board machine and log in to other servers from the Jump board machine to perform operation and maintenance operations. The jump board machine can be used as a remote device for batch operation of the jump board, and is a common operation platform for system administrators or operation and maintenance personnel. However, because the springboard machine does not realize audit and control on operation and maintenance personnel, accidents such as misoperation and illegal operation can be generated in the process of using the springboard machine.
The fort machine functionally integrates two main functions of core system operation and security audit management, and in technical aspect, the terminal computer is cut off from directly accessing network and server resources, and the protocol agent mode is adopted to take over the access of the terminal computer to the network and the server. Therefore, the operation and maintenance security audit can intercept illegal access and malicious attack, block illegal commands, filter out all illegal access behaviors to target equipment, and audit and monitor misoperation and illegal operation of internal personnel so as to facilitate postmortem responsibility tracking.
The bastion machine is used as an operation audit means, and the most core function is to realize authority control and operation behavior audit of operation operators. It must be possible to intercept the operations of the operation and maintenance personnel and analyze the contents of the operations. A schematic diagram of the fort machine is shown in figure 2. The deployment mode of the fort machine ensures that the fort machine can intercept all operation behaviors of operation and maintenance personnel, analyzes the operation contents in the operation behaviors to realize the purposes of authority control and behavior audit, and adopts the technology of an application agent.
The operation and maintenance audit type fort machine corresponds to a Proxy server (Proxy server) for operation and maintenance operators, and the working flow of the operation and maintenance audit type fort machine is shown in figure 3. The operation and maintenance personnel are firstly connected to the fort machine in the operation process, and then submit an operation request to the fort machine; after the request passes the permission check of the fort machine, the application proxy module of the fort machine replaces the user to be connected to the target equipment to complete the operation, then the target equipment returns the operation result to the fort machine, and finally the fort machine returns the operation result to the operation and maintenance personnel. Through the connection mode, the fort machine logically isolates operation and maintenance personnel from target equipment, establishes a management mode of operation and maintenance personnel, fort machine user account, authorization, target equipment account, target equipment, and solves operation authority control and behavior audit.
The operating principle of the fort machine is shown in figure 4. In the actual use scene, the bastion machine users can be divided into three types of users, namely an administrator, an operation and maintenance user and an auditor. The manager needs to configure the security policy of the fort machine according to the corresponding security policy and the operation authority of the operation and maintenance personnel. After the bastion machine manager logs in the bastion machine, a policy management component in the bastion machine is responsible for interaction of the manager, and the security policy input by the manager is stored in a policy configuration library in the bastion machine. The core part of the bastion machine is an application agent component which is responsible for transferring operation of a dimension operation user and interacting with other components in the bastion machine. The application agent component calls the policy management component to check the operation behavior after receiving the operation request of the operation and maintenance personnel, the checking basis is that the administrator has configured the policy configuration library, the operation does not accord with the security policy, and the application agent component refuses the execution of the operation behavior. After the operation behaviors of the operation and maintenance personnel pass through the verification of the policy management component, the application proxy component replaces the operation and maintenance personnel to connect with the target equipment to finish corresponding operation, and the operation result is returned to the corresponding operation and maintenance operator; at the same time, the operation process is submitted to an 'audit module' in the bastion machine, and then the operation process is recorded in an audit log database. Finally, when the historical operation records of the operation and maintenance personnel are required to be investigated, the auditor logs in the fort machine to inquire, and then the audit module reads the corresponding log records from the audit log database and displays the corresponding log records on the auditor interaction interface.
The fort machine is the latest safety protection technology platform which is common in the industry with higher informatization degree and information safety requirements at present. However, the traditional fort machine has higher input cost and high maintenance cost in the later period of the fort machine, and small and medium enterprises cannot bear tens of millions of expenses of the hardware fort mobile. Although the open source fort reduces the input cost to a certain extent, the post maintenance cost is higher. The open source fort requires special personnel for maintenance and secondary development, and can also select commercial support services, but high technical support cost is paid. The open source fort also means security threat and potential vulnerability brought by opening, and the enterprise uses the open source fort, so that the service, software and product of the enterprise may not be able to apply for sales permission, level protection and even cannot obtain trust of the user. Even if any problem occurs with the use of an open source fort machine, it is not possible to catch up.
Therefore, the invention designs a novel detection method for illegal operation of the jump board machine, which realizes the monitoring and control of operation and maintenance personnel on the jump board machine, audits and determines responsibility for misoperation and illegal operation of a user, reduces the cost for audit and operation and maintenance by using the fort machine, and further does not need hardware facilities, software installation and the like, so that the monitoring, audit and operation and maintenance on the operation of a terminal of the user are lighter and simpler.
Disclosure of Invention
The invention aims to provide a diving board machine illegal operation detection method based on video recording and OCR technology.
In order to achieve the above object, the present invention provides the following technical solutions:
the invention provides a method for detecting illegal operations of a springboard machine based on video screen recording and OCR technology, which comprises the steps of firstly recording a screen of a user operation interface, then detecting and judging the position of a command line input by a user through cursor detection or fixed character string detection aiming at the video of the recorded screen, converting the command input by the user into a text through the OCR technology, and then detecting illegal operation words of the text.
Further, the screen recording of the user operation interface is realized through Asciinema.
Further, screen recording is carried out on the user operation interface from the time when the user accesses the trigger to the time when all operations are finished.
Further, the splicing method of the video content of the recording screen comprises the following steps: when the video content rolls downwards, whether the content in the screen rolls or not is judged by comparing the whole pixels to upwards shift, and when the content rolls, new content screenshots are spliced behind old content to form long screenshots of all input and output of a user.
Further, the fixed string is composed of user rights and hostnames.
Further, the method for positioning the input command line through the fixed character string comprises the following steps: in the initial frames of the video, the user does not perform any operation, the initial fixed character string can be extracted by intercepting the end command line, the character string is recorded and compared with the line by line in the long screenshot, the position of the fixed character string is determined, the position of the input command can be determined, and the input command is obtained after the fixed character string.
Further, when a command which can be changed by user input is detected, the fixed character string is assigned again, and then the new fixed character string is used for matching row by row, so that the position of the input command line is determined.
Further, the method for positioning and inputting the command line through the cursor comprises the following steps: by positioning the cursor position, the horizontal coordinate value of the cursor is returned, and whether the cursor is suddenly changed or not is determined, so that the part where the cursor is suddenly changed is determined as output, and the rest is input.
Further, cursor differentiation becomes more than 8 units of cursor movement per second.
Further, after judging the position of the input command line through cursor detection or fixed character string detection, performing OCR extraction on the input part; comparing the text content extracted by OCR with the content in the illegal word library to confirm whether the user inputs the illegal command, and recording the user name, the operation time and the illegal operation if the command contains the illegal operation word.
Compared with the prior art, the invention has the beneficial effects that:
the method for detecting the illegal operation of the gangway machine based on the video recording and OCR technology provided by the invention realizes the monitoring and operation and maintenance personnel operation of the gangway machine, audits and controls the operation and maintenance personnel operation of the gangway machine, audits and determines responsibility for misoperation and illegal operation of a user, reduces the cost for audit and operation and maintenance by using the fort machine, and additionally does not need hardware facilities, software installation and the like, so that the monitoring, audit and operation and maintenance of the operation of the user at a terminal are lighter and simpler.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
Fig. 1 is a schematic diagram of a diving board machine.
FIG. 2 is a schematic diagram of a fort machine.
FIG. 3 is a schematic diagram of the fort machine workflow.
Fig. 4 is a schematic diagram of the working principle of the fort machine.
Fig. 5 is a schematic block diagram of a method for detecting illegal operations of a springboard machine based on video recording and OCR technology according to an embodiment of the present invention.
Fig. 6 is a schematic diagram of a fixed string and a cursor according to an embodiment of the present invention.
Fig. 7 is a video recording screen of the user operation interface of the springboard machine provided by the embodiment of the invention.
Fig. 8 is a schematic diagram of a fixed string positioning input command line according to an embodiment of the present invention.
Detailed Description
The design idea of the invention is as follows: firstly, a screen recording user operates on a plate jumping machine, then judges user input, performs OCR (Optical Character Recognize, optical character recognition) extraction on input content, and finally performs detection on illegal operation words on the input content, so that monitoring on operation behaviors of operation and maintenance personnel is realized, and misoperation and illegal operation of the user can be audited and responsibility can be determined later.
For a better understanding of the present technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings.
The invention provides a diving board machine illegal operation detection method based on video screen recording and OCR technology, as shown in figure 5, firstly screen recording is carried out on a user operation interface, then the position of a user input command line is judged by 'cursor detection' or 'fixed character string detection' (the fixed character string refers to a character string consisting of user authority and host name) aiming at the video of the screen recording, the fixed character string (in a box) and a cursor schematic diagram are shown in figure 6, then the command input by the user is converted into a text through OCR (Optical Character Recognize) technology, and then illegal operation words are detected on the text. If the command contains the illegal operation words, the user name, the operation time, the illegal operation and the like are recorded, and later period of misoperation, audit and responsibility determination of the illegal operation and the like of the user are facilitated, so that the whole process of illegal operation detection of the gangway machine is realized. The specific implementation process is as follows.
1. Implementation of operation interface screen recording
The video recording of the springboard machine user interface may be implemented by Asciinema, which is a terminal recording tool that saves the input, output and time of the command line in a file and may be played back through a terminal or web browser, as shown in fig. 7. By adding the script on the host, the automatic recording can be realized when the user accesses each time.
2. Video content stitching
The video content splicing process is that the screen recording of the user operation interface is started from the time when the user accesses the springboard machine until all operations are finished. When the video content rolls downwards, whether the content in the screen rolls or not can be judged by comparing the whole pixels to upwards shift, and when the content rolls, new content screenshots are spliced behind old content to form long screenshots of all inputs and outputs of a user.
3. Fixed string positioning input command line
Positioning the input command line is intended to take two ways: one is to locate the input command line by the position of a fixed string (composed of user rights and hostname) before the input command line; one is to position the input command line with a cursor.
1) Fixed string positioning input command line
This section achieves locating the input command line by determining the position of the fixed string. As shown in fig. 8, in the first few frames of the video, the user can extract the first fixed character string by intercepting the last command line without any operation. The character string is recorded and compared with the long screenshot line by line, and the position of the fixed character string is determined, so that the position of the input command can be determined: the fixed string is followed by the entered command.
However, this fixed string can be changed by a command, so that when a command is detected that the user input can change it, the fixed string is reassigned. Then the new 'fixed character string' is used for matching line by line, and the position of the input command line is determined.
2) Cursor positioning input command line
In the process of inputting the command, the abscissa and the ordinate of the cursor move in a slow step mode, and the position of the cursor is suddenly changed in the process of outputting. Therefore, by positioning the cursor position, the horizontal coordinate value of the cursor is returned to determine whether the cursor is suddenly changed, so that the part where the cursor is suddenly changed is determined as output, and the rest is input. The abrupt change of the cursor position needs to be determined according to the normal input speed, and the cursor movement per second is more than 8 units, namely the cursor movement.
4. OCR (optical character) extraction input command
After locating the position of the input command line, the commands entered by the user are converted to text and recorded by OCR (Optical Character Recognize, optical character recognition) technology.
5. Illegal operation detection
Comparing the text content extracted by OCR with the content in the illegal word stock to confirm whether the user inputs the illegal command, if yes, recording the user name, the operation time, the illegal operation and the like, and facilitating later audit and responsibility setting of misoperation and illegal operation of the user, thereby realizing the whole flow of illegal operation detection of the springboard machine.
The invention realizes the detection of the illegal operation of the springboard machine by the video screen recording and combining with the OCR technology, audits and determines the responsibility of the misoperation and the illegal operation of the user, and positions the input command line by a cursor or by a fixed character string (consisting of the user authority and the host name), thereby carrying out OCR extraction on the whole input video or long screenshot, and automatically analyzing whether the illegal command is input by the user or not only needing the screen recording video of the terminal without manual participation. Compared with the fort machine which is additionally provided with special equipment and special operation and maintenance personnel, the method does not need hardware facilities, software installation and the like, so that the monitoring, auditing and operation and maintenance of the user in terminal operation are lighter and simpler, the cost of manpower and material resources is saved, and the risk of larger loss caused by broken open source fort machine is avoided.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus embodiments, the electronic device embodiments, the computer-readable storage medium embodiments, and the computer program product embodiments, the description is relatively simple, as relevant to the description of the method embodiments in part, since they are substantially similar to the method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (5)
1. A diving board machine illegal operation detection method based on video screen recording and OCR technology is characterized in that firstly, a screen recording is carried out on a user operation interface, and the splicing method of the video content of the screen recording is as follows: when the video content rolls downwards, judging whether the content in the screen rolls or not by comparing the whole pixels to upwards shift, and splicing the new content screenshot to the rear of the old content every time when the content rolls, so as to form long screenshots of all input and output of a user; and then, judging the position of a command line input by a user through fixed character string detection aiming at the video of the screen, converting the command input by the user into a text through OCR technology, and detecting illegal operation words of the text.
2. The method for detecting the illegal operation of the springboard machine based on the video recording and OCR technology as claimed in claim 1, wherein the fixed character string is composed of user rights and hostnames.
3. The method for detecting the illegal operation of the springboard machine based on the video recording and OCR technology as claimed in claim 1, wherein the method for positioning the input command line through the fixed character string is as follows: in the initial frames of the video, the user does not perform any operation, the initial fixed character string can be extracted by intercepting the end command line, the character string is recorded and compared with the line by line in the long screenshot, the position of the fixed character string is determined, the position of the input command can be determined, and the input command is obtained after the fixed character string.
4. The method for detecting the illegal operation of the springboard machine based on the video recording and OCR technology as claimed in claim 3, wherein when the command which can be changed by the user input is detected, the fixed character string is assigned again, and then the new fixed character string is used for matching line by line, so as to determine the position of the input command line.
5. The method for detecting the illegal operation of the springboard machine based on the video recording and OCR technology according to claim 1, wherein the input part is subjected to OCR extraction after the position of the input command line is judged through cursor detection or fixed character string detection; comparing the text content extracted by OCR with the content in the illegal word library to confirm whether the user inputs the illegal command, and recording the user name, the operation time and the illegal operation if the command contains the illegal operation word.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111615997.8A CN114285761B (en) | 2021-12-27 | 2021-12-27 | Diving board machine illegal operation detection method based on video recording and OCR technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111615997.8A CN114285761B (en) | 2021-12-27 | 2021-12-27 | Diving board machine illegal operation detection method based on video recording and OCR technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114285761A CN114285761A (en) | 2022-04-05 |
| CN114285761B true CN114285761B (en) | 2023-04-25 |
Family
ID=80876464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111615997.8A Active CN114285761B (en) | 2021-12-27 | 2021-12-27 | Diving board machine illegal operation detection method based on video recording and OCR technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114285761B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114979108A (en) * | 2022-05-05 | 2022-08-30 | 北京精一强远科技有限公司 | System, method, equipment and medium for remote assistance and safety audit |
| CN116048363B (en) * | 2023-04-03 | 2023-08-25 | 数孪模型科技(北京)有限责任公司 | Display method, system, equipment and medium of software interface based on artificial intelligence |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113485615A (en) * | 2021-06-30 | 2021-10-08 | 福州大学 | Method and system for making typical application intelligent image-text tutorial based on computer vision |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10779056B2 (en) * | 2016-04-14 | 2020-09-15 | Contec, Llc | Automated network-based test system for set top box devices |
| CN108920690B (en) * | 2018-07-13 | 2020-06-19 | 北京朋创天地科技有限公司 | Visual network security audit method and system |
| CN110780865B (en) * | 2019-09-20 | 2022-06-07 | 苏州浪潮智能科技有限公司 | Method, equipment and medium for right-side alignment display of command line |
| CN113836972A (en) * | 2020-06-23 | 2021-12-24 | 中国移动通信集团辽宁有限公司 | Security audit method, device, equipment and storage medium based on OCR |
-
2021
- 2021-12-27 CN CN202111615997.8A patent/CN114285761B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113485615A (en) * | 2021-06-30 | 2021-10-08 | 福州大学 | Method and system for making typical application intelligent image-text tutorial based on computer vision |
Non-Patent Citations (1)
| Title |
|---|
| 陈亮 ; 汪景福 ; 王娜 ; 李霞 ; .基于DNN算法的移动视频推荐策略.计算机学报.2016,(第08期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114285761A (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114285761B (en) | Diving board machine illegal operation detection method based on video recording and OCR technology | |
| KR101053680B1 (en) | Software management apparatus and method, user terminal controlled by it and management method thereof | |
| CN114978584A (en) | Network security protection safety method and system based on unit cell | |
| CN102306258B (en) | UNIX host safety configuration auditing method based on configurable knowledge base | |
| CN105139139A (en) | Data processing method, device and system for operation and maintenance audit | |
| CN108063755B (en) | Vulnerability scanning method, system, storage medium and electronic equipment | |
| CN107566350B (en) | Security configuration vulnerability monitoring method and device and computer readable storage medium | |
| CN110290138B (en) | Restricted login method and system suitable for test database | |
| CN111092910B (en) | Database security access method, device, equipment, system and readable storage medium | |
| US10296182B2 (en) | Managed access graphical user interface | |
| CN113407949A (en) | Information security monitoring system, method, equipment and storage medium | |
| US20020184406A1 (en) | Method and system for handling window-based graphical events | |
| CN106778264A (en) | The application program analysis method and analysis system of a kind of mobile client | |
| CN112765611B (en) | Unauthorized vulnerability detection method, device, equipment and storage medium | |
| CN110958236A (en) | Dynamic authorization method of operation and maintenance auditing system based on risk factor insight | |
| CN116015824A (en) | Unified authentication method, equipment and medium for platform | |
| CN114157464B (en) | Network test monitoring method and monitoring system | |
| KR102156359B1 (en) | A Method for Checking Vulnerability Diagnosis Command Execution through Sending Pre-Command and Its System | |
| CN114745203A (en) | Method and device for monitoring full life cycle of user account | |
| CN112905983B (en) | Equipment leasing method, equipment and storage medium | |
| CN115801472B (en) | Authority management method and system based on authentication gateway | |
| US11822916B2 (en) | Correlation engine for detecting security vulnerabilities in continuous integration/continuous delivery pipelines | |
| Anisetti et al. | Moon cloud: a cloud platform for ICT security governance | |
| KR102107415B1 (en) | Method for providing cyber secure guide | |
| CN114979108A (en) | System, method, equipment and medium for remote assistance and safety audit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |