CN113204455A - Method, equipment and storage medium for automatically detecting user interface display abnormity - Google Patents
Method, equipment and storage medium for automatically detecting user interface display abnormity Download PDFInfo
- Publication number
- CN113204455A CN113204455A CN202110491107.0A CN202110491107A CN113204455A CN 113204455 A CN113204455 A CN 113204455A CN 202110491107 A CN202110491107 A CN 202110491107A CN 113204455 A CN113204455 A CN 113204455A
- Authority
- CN
- China
- Prior art keywords
- image
- user interface
- current display
- pixel points
- machine equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2205—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested
- G06F11/2221—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested to test input/output devices or peripheral units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Abstract
The invention discloses a method, equipment and a storage medium for automatically detecting display abnormity of a user interface, which are applied to all-in-one machine equipment and comprise the following steps: step S1, judging whether a starting instruction is received; if the starting-up instruction is received, the test image is displayed on a user interface; step S2, intercepting the current display image of the user interface, and judging whether the current display image is a normal image or not through an image comparison algorithm; if the current display image is a normal image, normally starting up the computer; if the currently displayed image is an abnormal image, executing step S3; and step S3, collecting the log files of the all-in-one machine equipment and sending notification information in a preset mode for checking the all-in-one machine equipment. The invention automatically detects the display condition of the user interface of the built-in computer and automatically collects the log files when abnormal conditions occur, thereby facilitating the overhaul of workers.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method, equipment and a storage medium for automatically detecting abnormal display of a user interface.
Background
With the development of science and technology, the application of all-in-one machine equipment is more and more extensive. The all-in-one machine equipment is generally configured with a built-in computer, and the built-in computer is started by default after the all-in-one machine equipment is started. In the starting process, the all-in-one machine equipment module is initialized, the output signal of the built-in computer is easy to be unstable, and the problems of probability abnormity exist, such as the conditions that the picture of the built-in computer is screen-blooming, the picture has color difference, the picture is screen-blacking and the like.
The display abnormality is random, but since the multiple integrated machines are controlled by the built-in computer, if the display abnormality occurs, the user cannot normally operate the integrated machines. Meanwhile, due to the fact that abnormal problems do not have occurrence rules and are short in time, the abnormal problems often need to be repeatedly detected through a large amount of pressure, and the abnormal problems are used for judging the reasons of the abnormal problems. The current solution is to adopt manual testing to restart repeatedly in the maintenance process, reproduce the problem and analyze the problem, but the mode is unable to be used normally in the maintenance process, the efficiency is low, and the input cost is higher.
Disclosure of Invention
In order to overcome the defects of the prior art, one of the objectives of the present invention is to provide a method for automatically detecting the abnormal display of a user interface, which automatically detects the display condition of the user interface of a built-in computer, and automatically collects log files when abnormal conditions occur, so as to facilitate the overhaul of workers.
The second objective of the present invention is to provide a device for performing the above method for automatically detecting the abnormal display of the user interface, which automatically detects the display condition of the user interface of the built-in computer, and automatically collects the log files when the abnormal condition occurs, so as to facilitate the maintenance of the staff.
The invention also aims to provide a storage medium device, which executes the method for automatically detecting the abnormal display of the user interface, automatically detects the display condition of the user interface of the built-in computer, automatically collects log files when the abnormal condition occurs, and is convenient for workers to overhaul.
One of the purposes of the invention is realized by adopting the following technical scheme:
a method for automatically detecting abnormal display of a user interface is applied to all-in-one machine equipment and comprises the following steps:
step S1, judging whether a starting instruction is received; if the starting-up instruction is received, the test image is displayed on a user interface;
step S2, intercepting the current display image of the user interface, and judging whether the current display image is a normal image or not through an image comparison algorithm; if the current display image is a normal image, normally starting up the computer; if the currently displayed image is an abnormal image, executing step S3;
and step S3, collecting the log files of the all-in-one machine equipment and sending notification information in a preset mode for checking the all-in-one machine equipment.
Further, the step S2 of determining whether the currently displayed image is a normal image through an image comparison algorithm includes the following steps:
step S21, preprocessing the current display image, and respectively calculating the average gray value of all pixels and the gray value of each pixel in the preprocessed current display image and the test image;
step S22, comparing the gray value of each pixel with the average value of the gray values in sequence, and recording the pixel points not less than the average value of the gray values as 1 and the pixel points less than the average value of the gray values as 0;
step S23, combining all pixel points in the current display image and the test image according to a preset sequence respectively, and comparing different pixel point numbers in two groups of pixel points; if the number of different pixel points is larger than the threshold value, the current display image is an abnormal image; and if the number of the different pixel points is less than a threshold value, the current display image is a normal image.
Further, the preprocessing in step S21 is to reduce the currently displayed image to a preset size and convert the currently displayed image into a grayscale image.
Furthermore, the test image is an image which is obtained by intercepting a user interface and displaying normally through manual test in advance.
Further, the test image is a static white picture, the preset size is 8 × 8, and the preset sequence is that the current display image and the test image adopt the same sequence.
Further, the log file is a log file, and the log file comprises logcat printing information and kernel printing information with abnormality, so as to be used for analyzing the cause of the abnormality.
Further, in the step S3, the notification information is sent in a preset manner to send the log file to the maintenance end.
The second purpose of the invention is realized by adopting the following technical scheme:
an apparatus comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the processor when executing the computer program implementing a method of automatically detecting a user interface display anomaly as described above.
The third purpose of the invention is realized by adopting the following technical scheme:
a storage medium having stored thereon a computer program which, when executed, implements a method of automatically detecting user interface display anomalies as described above.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a method, equipment and a storage medium for automatically detecting user interface display abnormity. The staff can directly analyze the reason of the display abnormity of the all-in-one machine equipment through the log file and overhaul the all-in-one machine equipment without manual testing and repeated restarting and recurring problems, so that the identification accuracy and efficiency of the display abnormity of the user interface are improved, and the cost of the display abnormity detection is reduced.
Drawings
FIG. 1 is a flowchart illustrating a method for automatically detecting an abnormal user interface display according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a method step S2 for automatically detecting an abnormal user interface display according to an embodiment of the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The invention provides a method for automatically detecting abnormal display of a user interface, which is applied to all-in-one machine equipment. The all-in-one machine equipment is generally provided with a built-in computer, information of the all-in-one machine equipment is displayed on a user interface of the built-in computer, and a user controls the all-in-one machine equipment to execute specified operation through the built-in computer. And the built-in computer is connected with the serial port of the all-in-one machine equipment.
As shown in fig. 1, the method for automatically detecting an abnormal display of a user interface includes the following steps:
step S1, judging whether a starting instruction is received; if the starting-up instruction is received, the test image is displayed on a user interface; the built-in computer is connected with the serial port of the all-in-one machine device, and the built-in computer is automatically turned on after the all-in-one machine device receives a starting instruction. Therefore, after the starting-up instruction is received, the abnormal display test is automatically started, and the test image is displayed in the user interface. After the serial port program of the built-in computer inquires that the image program opens the test image, a specified serial port instruction is sent to the all-in-one machine device, and image diagnosis is started.
The test image is typically a static image. In this embodiment, the test image is an image obtained by manually testing in advance and capturing a user interface to display a normal image. The screenshot image of the user interface is selected as the test image, and the image normally displayed by the user interface can be restored to the maximum extent. Meanwhile, in order to avoid that different colors in the test image affect the accuracy of judgment, in this embodiment, the test image is a pure white image.
Step S2, intercepting the current display image of the user interface, and judging whether the current display image is a normal image or not through an image comparison algorithm; if the current display image is a normal image, normally starting up the computer; if the currently displayed image is an abnormal image, step S3 is executed. The image comparison algorithm can adopt a perceptual hash algorithm, and whether the currently displayed image is a normal image or not is judged through the image comparison algorithm. As shown in fig. 2, the method comprises the following steps:
and step S21, preprocessing the current display image and the test image, and respectively calculating the average gray value of all pixels and the gray value of each pixel in the preprocessed current display image and the preprocessed test image.
The preprocessing is to reduce the current display image and the test image to a preset size and convert the images into gray images. In this embodiment, the current display image and the test image are reduced to 8 × 8 sizes, and there are 64 pixels respectively, so as to remove the image details, only retain the basic information of structure/brightness, etc. in the image, and discard the image difference caused by different sizes/scales in different user interfaces. And then the reduced image is converted into 64-level gray scale, namely, only 64 colors exist in all pixel points, and the inaccuracy of comparison caused by a plurality of colors is avoided. After conversion into a grayscale image, the grayscale average of all 64 pixels is calculated.
And step S22, comparing the gray value of each pixel with the average gray value in sequence, and recording the pixel points not less than the average gray value as 1 and the pixel points less than the average gray value as 0.
Step S23, combining all pixel points in the current display image and the test image according to a preset sequence respectively, and comparing different pixel point numbers in two groups of pixel points; if the number of different pixel points is larger than the threshold value, the current display image is an abnormal image; and if the number of the different pixel points is less than a threshold value, the current display image is a normal image.
Specifically, all the pixels of the current display image and the test image are combined according to the same sequence to obtain two groups of 64-bit integers. Comparing the number of different pixel points in the two groups of pixel points, if the number of different pixel points is more than 10, the current display image is an abnormal image, the comparison is not passed, and the step S3 needs to be executed; if the number of the different pixel points is less than 10, the current display image is a normal image, the comparison is passed, the user interface display is normal, and the computer can be started normally.
And step S3, collecting the log files of the all-in-one machine equipment and sending notification information in a preset mode for checking the all-in-one machine equipment. After the current display image is judged to be an abnormal image, the all-in-one machine equipment automatically collects a log file, the log file comprises the abnormal logcat (a command line tool in Android) printing information and kernel (real-time operating system) printing information, and a worker can analyze the reason of the abnormality through the log file. In this embodiment, after the user interface displays an abnormality, the log file is automatically collected, and an email including the log file is sent to the maintenance end, so as to inform the staff of the abnormality and check the log file to analyze the cause of the abnormality.
The invention provides a method for automatically detecting user interface display abnormity, which automatically performs user interface display test after receiving a starting instruction, collects log files of all-in-one machine equipment when the display abnormity is detected, and notifies workers. The staff can directly analyze the reason of the display abnormity of the all-in-one machine equipment through the log file and overhaul the all-in-one machine equipment without manual testing and repeated restarting and recurring problems, so that the identification accuracy and efficiency of the display abnormity of the user interface are improved, and the cost of the display abnormity detection is reduced.
In addition, the present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program realizes the steps of the method for automatically detecting the display abnormality of the user interface when being executed by the processor.
The invention is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
Based on the same inventive concept, there is also provided an apparatus comprising a memory, a processor, and a program stored in the memory, the program being configured to be executed by the processor, the program when executed by the processor implementing the above-mentioned method of automatically detecting a user interface display anomaly.
The apparatus in this embodiment and the method in the foregoing embodiment are based on two aspects of the same inventive concept, and the method implementation process has been described in detail in the foregoing, so that those skilled in the art can clearly understand the structure and implementation process of the system in this embodiment according to the foregoing description, and for the sake of brevity of the description, details are not repeated here.
The above embodiments are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby, and any insubstantial changes and substitutions made by those skilled in the art based on the present invention are within the protection scope of the present invention.
Claims (9)
1. A method for automatically detecting abnormal display of a user interface is applied to all-in-one machine equipment and comprises the following steps:
step S1, judging whether a starting instruction is received; if the starting-up instruction is received, the test image is displayed on a user interface;
step S2, intercepting the current display image of the user interface, and judging whether the current display image is a normal image or not through an image comparison algorithm; if the current display image is a normal image, normally starting up the computer; if the currently displayed image is an abnormal image, executing step S3;
and step S3, collecting the log files of the all-in-one machine equipment and sending notification information in a preset mode for checking the all-in-one machine equipment.
2. The method as claimed in claim 1, wherein the step S2 of determining whether the currently displayed image is a normal image through an image comparison algorithm comprises the following steps:
step S21, preprocessing the current display image, and respectively calculating the average gray value of all pixels and the gray value of each pixel in the preprocessed current display image and the test image;
step S22, comparing the gray value of each pixel with the average value of the gray values in sequence, and recording the pixel points not less than the average value of the gray values as 1 and the pixel points less than the average value of the gray values as 0;
step S23, combining all pixel points in the current display image and the test image according to a preset sequence respectively, and comparing different pixel point numbers in two groups of pixel points; if the number of different pixel points is larger than the threshold value, the current display image is an abnormal image; and if the number of the different pixel points is less than a threshold value, the current display image is a normal image.
3. The method as claimed in claim 2, wherein the step S21 is pre-processing to reduce the currently displayed image to a preset size and convert it into a gray image.
4. The method of claim 3, wherein the test image is an image obtained by manually testing the user interface in advance and intercepting the user interface display.
5. The method as claimed in claim 4, wherein the test image is a static white picture, the predetermined size is 8 x 8, and the predetermined sequence is the same sequence for the currently displayed image and the test image.
6. The method of claim 1, wherein the log file is a log file, and the log file comprises logcat print information and kernel print information of the occurrence of the abnormality, so as to analyze the cause of the abnormality.
7. The method of claim 1, wherein the step S3 is performed by sending a notification message in a predetermined manner to a service maintenance end.
8. An apparatus comprising a processor, a memory, and a computer program stored on the memory and operable on the processor, wherein the processor implements a method for automatically detecting a user interface display anomaly of any one of claims 1-7 when executing the computer program.
9. A storage medium having stored thereon a computer program which, when executed, implements a method of automatically detecting user interface display anomalies according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110491107.0A CN113204455A (en) | 2021-05-06 | 2021-05-06 | Method, equipment and storage medium for automatically detecting user interface display abnormity |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110491107.0A CN113204455A (en) | 2021-05-06 | 2021-05-06 | Method, equipment and storage medium for automatically detecting user interface display abnormity |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113204455A true CN113204455A (en) | 2021-08-03 |
Family
ID=77029928
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110491107.0A Pending CN113204455A (en) | 2021-05-06 | 2021-05-06 | Method, equipment and storage medium for automatically detecting user interface display abnormity |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113204455A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113467743A (en) * | 2021-09-03 | 2021-10-01 | 广州朗国电子科技股份有限公司 | Method and device for automatically detecting and configuring single channel and double channels of display screen |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120099782A1 (en) * | 2010-10-20 | 2012-04-26 | Samsung Electronics Co., Ltd. | Image processing apparatus and method |
| EP2605211A1 (en) * | 2011-12-16 | 2013-06-19 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and program |
| CN106686372A (en) * | 2015-11-11 | 2017-05-17 | 天津三星电子有限公司 | Display terminal image display detection method and display terminal |
| CN106844179A (en) * | 2017-02-07 | 2017-06-13 | 上海与德信息技术有限公司 | log storing method and device |
| CN106951899A (en) * | 2017-02-24 | 2017-07-14 | 李刚毅 | Method for detecting abnormality based on image recognition |
| CN108121648A (en) * | 2016-11-29 | 2018-06-05 | 北京京东尚科信息技术有限公司 | A kind of interface error monitoring method |
| CN108579094A (en) * | 2018-05-11 | 2018-09-28 | 深圳市腾讯网络信息技术有限公司 | A kind of user interface detection method and relevant apparatus, system and storage medium |
-
2021
- 2021-05-06 CN CN202110491107.0A patent/CN113204455A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120099782A1 (en) * | 2010-10-20 | 2012-04-26 | Samsung Electronics Co., Ltd. | Image processing apparatus and method |
| EP2605211A1 (en) * | 2011-12-16 | 2013-06-19 | Canon Kabushiki Kaisha | Image processing apparatus, image processing method, and program |
| CN106686372A (en) * | 2015-11-11 | 2017-05-17 | 天津三星电子有限公司 | Display terminal image display detection method and display terminal |
| CN108121648A (en) * | 2016-11-29 | 2018-06-05 | 北京京东尚科信息技术有限公司 | A kind of interface error monitoring method |
| CN106844179A (en) * | 2017-02-07 | 2017-06-13 | 上海与德信息技术有限公司 | log storing method and device |
| CN106951899A (en) * | 2017-02-24 | 2017-07-14 | 李刚毅 | Method for detecting abnormality based on image recognition |
| CN108579094A (en) * | 2018-05-11 | 2018-09-28 | 深圳市腾讯网络信息技术有限公司 | A kind of user interface detection method and relevant apparatus, system and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113467743A (en) * | 2021-09-03 | 2021-10-01 | 广州朗国电子科技股份有限公司 | Method and device for automatically detecting and configuring single channel and double channels of display screen |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10866852B2 (en) | Image based fault state determination | |
| US11853150B2 (en) | Method and device for detecting memory downgrade error | |
| CN110609778A (en) | Method and system for storing server downtime log | |
| WO2015081765A1 (en) | Automatic fault diagnosis method and device for sorting machine | |
| CN114996103A (en) | Page abnormity detection method and device, electronic equipment and storage medium | |
| CN113204455A (en) | Method, equipment and storage medium for automatically detecting user interface display abnormity | |
| CN111124774B (en) | Method and related device for testing stability of server in starting process | |
| JP7132021B2 (en) | Notification device, notification method and program | |
| CN113342627B (en) | Application service monitoring method and monitoring system of terminal | |
| CN110874311A (en) | Database detection method and device, computer equipment and storage medium | |
| CN115576737A (en) | Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium | |
| CN115525499A (en) | Data analysis method and device for hard disk, electronic equipment and storage medium | |
| CN111124841B (en) | Alarm method and device for abnormal page and computer system | |
| CN114257534A (en) | Test result processing method, device and system and storage medium | |
| JP2000293411A (en) | Device and method for fault management and recording medium | |
| CN111855155B (en) | Detection method and detection device for display fault and readable storage medium | |
| CN113992778B (en) | Equipment detection method, device, chip and module equipment | |
| CN110457177B (en) | Startup abnormity detection method and device, electronic equipment and storage medium | |
| CN113127317B (en) | Log acquisition processing method, system, device and storage medium | |
| JP2009039953A (en) | Printing apparatus | |
| CN113849274A (en) | Process inspection method and device, storage medium and electronic equipment | |
| CN116501637A (en) | Printing test method and device, electronic equipment and storage medium | |
| CN112651655A (en) | Attendance management method and device, computing equipment and storage medium | |
| JP2021026754A (en) | Information processing apparatus, method for controlling the same, and program | |
| CN117130810A (en) | Error information collection method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 238, room 406, 1 Yichuang street, Huangpu District, Guangzhou, Guangdong 510700 Applicant after: Guangzhou langguo Electronic Technology Co.,Ltd. Address before: Room 238, room 406, 1 Yichuang street, Huangpu District, Guangzhou, Guangdong 510700 Applicant before: GUANGZHOU LANGO ELECTRONIC SCIENCE & TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210803 |
|
| RJ01 | Rejection of invention patent application after publication |