CN109815100B - Behavior monitoring method for CABAO software by utilizing image contrast analysis - Google Patents
Behavior monitoring method for CABAO software by utilizing image contrast analysis Download PDFInfo
- Publication number
- CN109815100B CN109815100B CN201910006169.0A CN201910006169A CN109815100B CN 109815100 B CN109815100 B CN 109815100B CN 201910006169 A CN201910006169 A CN 201910006169A CN 109815100 B CN109815100 B CN 109815100B
- Authority
- CN
- China
- Prior art keywords
- interface
- image
- mask
- position information
- template
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a behavior monitoring method for Called Numbers software by utilizing image contrast analysis, which comprises the following steps: firstly, pre-storing an interface template; secondly, acquiring a Mask area; thirdly, image comparison; fourthly, OCR recognition; the invention can automatically monitor the software interface of the number caller and extract effective information from the software interface, thereby reducing manual operation and improving working efficiency.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a behavior monitoring method for Calqueuer software by utilizing image contrast analysis.
Background
With the development of computer technology, networking, informatization, automation and intellectualization, the computer technology is widely applied to various aspects of society, the behavior of user operation software is analyzed by analyzing a user operation interface, and the behavior of user operation is judged, so that the demand is more and more, for example, parents monitor the behavior of children playing computers by detecting the time and the sequence of each page; and for example, a new student is known to learn the weakness of certain software by monitoring the change of the operation interface, so that targeted guidance is performed.
There are many types of medical operation software, and a queueing treasure is a software system widely used in hospitals for charging and registering information of patients. It is very cumbersome to manually monitor the interface and obtain the relevant information of the patient from the interface.
The traditional method only monitors the operation interface of the user, and judges which interface the user operates according to the change of the operation interface, but the operation behavior of the user on the interface can not be judged. The method is a static monitoring method, and dynamic behaviors cannot be analyzed.
The prior advanced traditional method compares the operation behavior of the user with the preset behavior of the user in the user model library, intelligently analyzes the subsequent operation of the user, and automatically presents the interface required by the user.
Disclosure of Invention
The invention aims to provide a behavior monitoring method for Called software by using image contrast analysis, aiming at the defects and shortcomings of the prior art.
In order to achieve the purpose, the invention adopts the technical scheme that: the method comprises the following steps:
firstly, interface template prestoring: the number calling software mainly comprises two main interfaces, namely a charging interface and a new agriculture and government interface of the software, and the two interfaces are stored in advance through a storage interface module; after the two interfaces are opened, images of the two interfaces are stored as interface templates, and position information of the two interfaces in a doctor computer screen is recorded;
secondly, acquiring a Mask region: in the template image, the areas of the text boxes of names, charging types, drug columns and total fees are Mask areas, and the position information of the Mask areas is stored;
thirdly, image comparison: acquiring a screenshot of a current doctor operation interface, acquiring a software interface part from the screenshot according to the position of a previously stored interface template area, only storing a non-Mask area according to the position information of the previously stored Mask area, comparing the non-Mask area with the interface template to acquire an interface corresponding to the optimal similarity, and judging whether the current interface is a charging interface or a new agriculture and union interface;
fourthly, OCR recognition: and after determining whether the interface type of the current screenshot is a charging interface or a new agriculture interface, extracting image content of a corresponding area according to the position information of text boxes of the name, the charging type and the drug column, performing OCR (optical character recognition), and acquiring detailed text information of the patient seeing a doctor fee.
After the scheme is adopted, the invention has the beneficial effects that: the behavior monitoring method for the Called-Taobao software by utilizing image contrast analysis can automatically monitor the software interface of the Called-Taobao and extract effective information from the Called-Taobao, so that manual operation is reduced, and the working efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is an overall flow diagram of the present invention;
fig. 2 is an image comparison flow chart.
Detailed Description
The invention will be further described with reference to the accompanying drawings.
The technical scheme adopted by the specific implementation mode is as follows: referring to fig. 1, it comprises the following steps:
firstly, interface template prestoring: the number calling software mainly comprises a charging interface and a new agriculture and government interface, the two interfaces are used as key monitoring objects, and corresponding effective information is extracted from the interfaces;
The two interfaces are stored in advance through a storage interface module; when the two interfaces are opened, the positions of the rectangular frames of the charging interface and the new agriculture and forestry interface in the computer screen are recorded and sequentially represented as L1 and L2; the two interface images are sequentially stored and marked as P1 and P2 to be used as comparison templates for subsequent operations;
secondly, acquiring a Mask region: in the new agriculture and government interface and the charging interface, some information of different patients is different, such as the name, the charging type, the drug column and the like of the patient; if the areas participate in the comparison with the template, the judgment of the type of the current interface is interfered, and therefore the areas need to be removed, and the removed areas are called Mask areas; the specific operation steps are as follows:
A. for a charging interface, recording position information of a patient name text box and a charging type rectangular text box, and taking the text box areas as Mask areas;
B. for a new agricultural and social interface, recording position information of two text boxes, namely total cost and central terminal total cost, and taking the text box areas as Mask areas;
thirdly, image comparison: acquiring a screenshot of a current doctor operation interface, acquiring a software interface part from the screenshot according to a position of a template region stored previously, only storing a non-Mask region according to position information of a Mask region stored previously, comparing the non-Mask region with the template to acquire an interface corresponding to the optimal similarity, and judging whether the current interface is a charging interface or a new rural-rural interface;
Referring to fig. 2, the specific operation steps of image comparison are as follows:
(1) image screen capture: calling a background screen capture program, and capturing an image of the current computer interface every 1s, wherein i represents the current moment;
(2) interface cutting: because the type of the current interface is unknown, two stored templates need to be compared in sequence; according to the position information of the two templates in the computer screen, cutting the corresponding rectangular area in the current image Ti according to the position information, and storing as Tl；
(3) Cutting Mask areas: since the Mask region has a large variation amplitude, interference is caused to template comparison, and therefore the Mask region needs to be removed to reduce the interference;
if the implementation is carried out according to a charging interface, acquiring the Mask area position of the stored image Ti, removing the corresponding Mask area, marking the rest image as Ai, and if the implementation is carried out according to a new agriculture and union interface, marking the rest image as Ai after removing the corresponding Mask area; in addition, for the two template images, the corresponding Mask regions are also removed respectively, and after the corresponding Mask regions are removed respectively, the two template images can be marked as P1 and P2;
(4) Image comparison: comparing the image Ai with the interface template, and judging the type of the image Ai; the specific implementation steps are as follows:
A. image graying
Whether it is a template image or an image Ai, it is a three-channel image outputting colors, and is relatively complex to implement for image matching; the gray level image belongs to a single-channel image, the matching is simple, and most information of the original image is reserved, so that the method can be well suitable for matching the image; images Ai, P1, and P2, after undergoing a graying operation, are labeled Ai, P1, and P2, respectively;
B. determining image type
i) Cutting according to the position information of the charging interface and Mask position information corresponding to the position information to obtain an image Ai, matching pixel-by-pixel of a gray image Ai and a gray image P1, if the number of pixels with different values of Ai and P1 pixels in the same position is less than 200, indicating that the matching is successful, and if the interface corresponding to the moment belongs to the charging interface, continuously monitoring, otherwise, continuing to perform the step ii;
ii) cutting according to the position information of the new farming-joining interface and Mask position information corresponding to the new farming-joining interface to obtain an image Ai, matching pixel points by pixel points of the gray image Ai and the gray image P2, and if the number of pixel points with different values of the pixel points of Ai and P2 in the same position is less than 200, indicating that the matching is successful, wherein the interface corresponding to the moment belongs to the new farming-joining interface, otherwise, the interface at the moment does not belong to a charging interface or the new farming-joining interface, and continuing to continuously monitor;
(5) And (3) continuously monitoring: the step (1) to the step (4) are executed every 1s because the type of the interface with the moment i is acquired, so that the continuous monitoring of the computer interface of the doctor is realized;
in addition, in the image comparison of the step (4), since the corresponding interface type (except for the first detection) at the last moment is known, namely a charging interface, a new agriculture and union interface or other interfaces; in the current moment, preferentially adopting a template corresponding to the interface type known at the previous moment for matching, except a non-charging interface and a new agricultural interface, thereby improving the detection efficiency;
fourthly, OCR recognition: after determining whether the interface type of the current screenshot is a charging interface or a new agriculture and union interface, extracting image content of a corresponding area according to the position information of text boxes such as names, charging types and drug columns, performing OCR (optical character recognition), and acquiring detailed text information of the patient medical expense;
for the moment i, judging the interface type of the interface by acquiring the type of the corresponding interface, if the interface type belongs to a charging interface or a new agriculture and union interface, executing an OCR recognition part, and if not, continuing to execute interface monitoring of the next moment;
if the current interface belongs to a charging interface, extracting corresponding image areas according to the position information of the Mask area acquired by the charging interface, wherein the areas comprise information such as a patient name text box, a charging type and the like, and then calling an already-disclosed OCR (optical character recognition) program to extract information such as the information of the patient, the charging type and the like from the areas in a character form;
If the current interface belongs to a new agriculture and government interface, extracting corresponding image areas according to the position information of the Mask area acquired by the new agriculture and government interface, wherein the areas comprise information such as total cost, total cost of a center terminal and the like, and then calling an OCR recognition program which is disclosed to extract information such as total cost of a patient, total cost of the center terminal and the like from the areas in a text mode.
The specific implementation mode has the following advantages:
1. the template matching method is adopted to accurately know the type of the current interface in the CALL software;
2. after the interface type at the current moment is obtained, the relevant information of the patient can be extracted in a text form from the relevant position in the image (in different types of interfaces, the positions of different effective information in the image are different);
3. because the software interface for monitoring the number calling treasures in real time, the time for opening a certain page or the time for jumping from a certain page to another page can be known, and whether the page jumps or not is judged by detecting whether the type of the software interface changes or not.
4. By the template matching method, the time for judging the interface type is greatly shortened, and the efficiency is improved.
The above description is only for illustrating the technical solution of the present invention, and it is not intended to limit other modifications or equivalent substitutions of the technical solution of the present invention by those of ordinary skill in the art, and the technical solution of the present invention is covered by the scope of the claims of the present invention as long as it does not depart from the spirit and scope of the technical solution of the present invention.
Claims (1)
1. A behavior monitoring method for Called Nuobao software by utilizing image contrast analysis is characterized by comprising the following steps:
firstly, interface template pre-storing: the number calling software mainly comprises two interfaces, namely a charging interface and a new agriculture and government interface of the software, and the two interfaces are stored in advance through a storage interface module; after the two interfaces are opened, images of the two interfaces are stored as interface templates, and position information of the two interfaces in a doctor computer screen is recorded;
secondly, acquiring a Mask area: in the template image, the areas of the text boxes of names, charging types, drug columns and total fees are Mask areas, and the position information of the Mask areas is stored;
thirdly, comparing the images: acquiring a screenshot of a current doctor operation interface, acquiring a software interface part from the screenshot according to the position of a previously stored interface template area, only storing a non-Mask area according to the position information of the previously stored Mask area, comparing the non-Mask area with the interface template to acquire an interface corresponding to the optimal similarity, and judging whether the current interface is a charging interface or a new rural cooperative interface;
The specific operation steps of image comparison are as follows:
(1) image screen capture: calling a background screen capture program, and capturing an image of the current computer interface every 1s, wherein i represents the current time;
(2) interface cutting: because the type of the current interface is unknown, two stored templates need to be compared in sequence; according to the position information of the two templates in the computer screen, cutting the corresponding rectangular area in the current image Ti according to the position information, and storing as Tl；
(3) Cutting Mask areas: since the Mask region has a large variation amplitude, interference is caused to template comparison, so that the Mask region needs to be removed, and the interference is reduced;
if the implementation is carried out according to a charging interface, acquiring the Mask area position of the stored image Ti, removing the corresponding Mask area, marking the rest image as Ai, and if the implementation is carried out according to a new agriculture and union interface, marking the rest image as Ai after removing the corresponding Mask area; in addition, for the two template images, the corresponding Mask regions are also removed respectively, and after the corresponding Mask regions are removed respectively, the two template images can be marked as P1 and P2;
(4) image comparison: comparing the image Ai with the interface template, and judging the type of the image Ai; the specific implementation steps are as follows:
A. Image graying
Whether it is a template image or an image Ai, which is a three-channel image outputting colors, it is relatively complex to implement for image matching; the gray level image belongs to a single-channel image, the matching is simple, and most information of the original image is reserved, so that the method can be well suitable for matching the image; images Ai, P1, and P2, after undergoing a graying operation, are labeled Ai, P1, and P2, respectively;
B. determining image type
i) Cutting according to the position information of the charging interface and Mask position information corresponding to the position information to obtain an image Ai, matching pixel-by-pixel of a gray image Ai and a gray image P1, if the number of pixels with different values of Ai and P1 pixels in the same position is less than 200, indicating that the matching is successful, and if the interface corresponding to the moment belongs to the charging interface, continuously monitoring, otherwise, continuing to perform the step ii;
ii) cutting according to the position information of the new farming-joining interface and Mask position information corresponding to the new farming-joining interface to obtain an image Ai, matching pixel points by pixel points of the gray image Ai and the gray image P2, and if the number of pixel points with different values of the pixel points of Ai and P2 in the same position is less than 200, indicating that the matching is successful, wherein the interface corresponding to the moment belongs to the new farming-joining interface, otherwise, the interface at the moment does not belong to a charging interface or the new farming-joining interface, and continuing to continuously monitor;
(5) And (3) continuously monitoring: the step (1) to the step (4) are executed every 1s because the type of the interface with the moment i is acquired, so that the continuous monitoring of the computer interface of the doctor is realized;
in addition, in the image comparison of the step (4), since the corresponding interface type at the last moment is known, namely the charging interface, the new agriculture and forestry interface or other interfaces; in the current moment, preferentially adopting the template corresponding to the interface type known at the previous moment for matching except the non-charging interface and the new agriculture and government interface, thereby improving the detection efficiency;
fourthly, OCR recognition: and after determining whether the interface type of the current screenshot is a charging interface or a new agriculture interface, extracting image content of a corresponding area according to the position information of text boxes of the name, the charging type and the drug column, performing OCR (optical character recognition), and acquiring detailed text information of the patient seeing a doctor fee.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910006169.0A CN109815100B (en) | 2019-01-04 | 2019-01-04 | Behavior monitoring method for CABAO software by utilizing image contrast analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910006169.0A CN109815100B (en) | 2019-01-04 | 2019-01-04 | Behavior monitoring method for CABAO software by utilizing image contrast analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109815100A CN109815100A (en) | 2019-05-28 |
| CN109815100B true CN109815100B (en) | 2022-06-28 |
Family
ID=66603913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910006169.0A Active CN109815100B (en) | 2019-01-04 | 2019-01-04 | Behavior monitoring method for CABAO software by utilizing image contrast analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109815100B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110888812B (en) * | 2019-11-26 | 2023-11-07 | 国核自仪系统工程有限公司 | Test system and test method for response time of terminal page |
| CN111913873A (en) * | 2020-06-17 | 2020-11-10 | 浙江数链科技有限公司 | Picture verification method, device and system and computer readable storage medium |
| CN112835807B (en) * | 2021-03-02 | 2022-05-31 | 网易（杭州）网络有限公司 | Interface identification method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016197710A1 (en) * | 2015-11-27 | 2016-12-15 | 中兴通讯股份有限公司 | Method and device for identifying fake software interface for mobile terminal |
| US10116861B1 (en) * | 2017-05-01 | 2018-10-30 | Ricoh Company, Ltd. | Guided image capture user interface |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207777A (en) * | 2012-01-11 | 2013-07-17 | 天津中青意达软件技术有限公司 | Development technology of automatic game goods receiving and delivery system based on image workflows |
| CN108121648B (en) * | 2016-11-29 | 2020-09-01 | 北京京东尚科信息技术有限公司 | Interface error monitoring method |
| CN107256485B (en) * | 2017-05-27 | 2020-12-04 | 北京小米移动软件有限公司 | Transaction record information acquisition method and device and computer readable storage medium |
| US10360473B2 (en) * | 2017-05-30 | 2019-07-23 | Adobe Inc. | User interface creation from screenshots |
| CN108390818B (en) * | 2018-02-23 | 2020-01-14 | 平安科技（深圳）有限公司 | Method, device, equipment and storage medium for interaction of instant messaging and mails |
| CN108710880A (en) * | 2018-05-16 | 2018-10-26 | 深圳市众信电子商务交易保障促进中心 | A kind of data grab method and terminal |
| CN109036523B (en) * | 2018-09-04 | 2022-08-09 | 珠海横琴圣澳云智科技有限公司 | Method, device, equipment and storage medium for inquiring case ID diagnosis information |
-
2019
- 2019-01-04 CN CN201910006169.0A patent/CN109815100B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016197710A1 (en) * | 2015-11-27 | 2016-12-15 | 中兴通讯股份有限公司 | Method and device for identifying fake software interface for mobile terminal |
| US10116861B1 (en) * | 2017-05-01 | 2018-10-30 | Ricoh Company, Ltd. | Guided image capture user interface |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109815100A (en) | 2019-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109815100B (en) | Behavior monitoring method for CABAO software by utilizing image contrast analysis | |
| CN101667251B (en) | OCR recognition method and device with auxiliary positioning function | |
| CN111124888B (en) | Method and device for generating recording script and electronic device | |
| CN109784304B (en) | Method and apparatus for labeling dental images | |
| CN111291661B (en) | Method and equipment for identifying text content of icon in screen | |
| CN108235115B (en) | Method and terminal for positioning human voice area in song video | |
| CN112784835B (en) | Method and device for identifying authenticity of circular seal, electronic equipment and storage medium | |
| CN107786867A (en) | Image identification method and system based on deep learning architecture | |
| CN112101386B (en) | Text detection method, device, computer equipment and storage medium | |
| CN112559341A (en) | Picture testing method, device, equipment and storage medium | |
| WO2021073117A1 (en) | Oct image segmentation method, apparatus and device, and storage medium | |
| CN110929562A (en) | Answer sheet identification method based on improved Hough transformation | |
| CN114549557A (en) | Portrait segmentation network training method, device, equipment and medium | |
| CN110888812B (en) | Test system and test method for response time of terminal page | |
| CN110599520B (en) | Open field experiment data analysis method, system and terminal equipment | |
| CN110248235B (en) | Software teaching method, device, terminal equipment and medium | |
| JP2006309405A (en) | Meter recognition system, meter recognition method, and meter recognition program | |
| CN113822907A (en) | Image processing method and device | |
| Vaishak et al. | Currency and Fake Currency Detection using Machine Learning and Image Processing–An Application for Blind People using Android Studio | |
| CN108877030A (en) | Image processing method, device, terminal and computer readable storage medium | |
| CN113554022A (en) | Automatic acquisition method and device for detection test data of power instrument | |
| CN110737687A (en) | Data query method, device, equipment and storage medium | |
| CN111783737B (en) | Mathematical formula identification method and device | |
| CN113141433B (en) | Method and device for testing screen sensitivity and processor | |
| CN116469156A (en) | Method, apparatus, computer device and computer readable storage medium for identifying body state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |