CN115525891A - Non-invasive detection method and device for pop-up window button - Google Patents
Non-invasive detection method and device for pop-up window button Download PDFInfo
- Publication number
- CN115525891A CN115525891A CN202110707217.6A CN202110707217A CN115525891A CN 115525891 A CN115525891 A CN 115525891A CN 202110707217 A CN202110707217 A CN 202110707217A CN 115525891 A CN115525891 A CN 115525891A
- Authority
- CN
- China
- Prior art keywords
- pop
- window
- image
- button
- screen image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/554—Detecting local intrusion or implementing counter-measures involving event detection and direct action
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a non-invasive detection method of pop-up window buttons, which is used for identifying a pop-up window comprising at least one pop-up window button in a display device. The method comprises the following steps: capturing a screen image of the display device; comparing the screen image with the preset screen image, and generating a difference image area according to the screen image and the preset screen image; when the difference image area is larger than the image area threshold value, judging the difference image area as a pop-up window of the screen image; detecting and screening a plurality of contour lengths in the pop-up window which are more than a contour length threshold value by using the canny edge; and interpreting a plurality of profile lengths by using Douglas-Peucker algorithm (Douglas-Peucker algorithm) and the number of the end points to generate a profile edge corresponding to at least one pop-up window button.
Description
Technical Field
The present invention relates to a non-invasive detecting method and device, and more particularly, to a non-invasive detecting method and device for automatically detecting a pop-up window button of a pop-up window.
Background
With the rapid development of industry, the equipment of many factories is gradually going to be automated. Generally, the equipment is matched with a display screen, and the equipment displays the current production progress or various parameters on the screen so as to enable an operator to check the operation condition of the equipment. In recent years, with the development of industrial intelligence, many factories are provided with remotely controllable non-intrusive device connection equipment, and the equipment is used to monitor and collect data on the display screen of the equipment, thereby replacing manual operation.
However, when the non-invasive device is disposed in the process of monitoring the display screen of the apparatus, unexpected pop-up windows may appear on the peripheral device or the display screen disposed in the apparatus, such as: an alarm window and a selective window. Therefore, when the pop-up window appears in the equipment, the pop-up window may block the production parameters or the data to be detected in the screen, so that the non-invasive device cannot recognize the data and generates an abnormality, which may interrupt the operation of the equipment, thereby reducing the efficiency.
Therefore, it is necessary to develop a new pop-up window and a pop-up button detection mechanism to solve the problems of the prior art.
Disclosure of Invention
Accordingly, the present invention is directed to a non-intrusive detecting method and apparatus for pop-up windows, so as to solve the problems of the prior art, and the method and apparatus can automatically detect the pop-up windows of the device and identify the pop-up window buttons in the pop-up windows, thereby effectively improving the practicability, and also improving the processing efficiency, reducing the labor and production costs, and improving the processing efficiency. .
To achieve the above object, the present invention discloses a non-intrusive detection method for pop-up window buttons, which is used to identify at least one pop-up window button of a pop-up window of a display device, and is characterized in that the non-intrusive detection method for pop-up window buttons comprises the following steps:
s1: capturing a screen image of the display device;
s2: comparing the screen image with a preset screen image, and generating a difference image area according to the screen image and the preset screen image;
s3: judging whether the difference image area is larger than an image area threshold value;
s4: when the difference image area is larger than an image area threshold, judging the difference image area as the pop-up window of the screen image;
s5: in the pop-up window, a contour length threshold value is preset, and then a plurality of contour lengths which are more than the contour length threshold value are screened out by canny edge detection; and
s6: the method includes presetting an end number of the at least one pop-up window button, and then interpreting the contour lengths by using a Douglas-Pock algorithm and the end number to generate a contour edge corresponding to the at least one pop-up window button.
Wherein: further comprising the steps of:
s11: judging whether the preset screen image exists or not, and if so, executing a step S2; and
s12: if the determination result in the step S11 is negative, the screen image is replaced with the predetermined screen image.
Wherein: in step S2, the method further includes the steps of:
s21: subtracting the screen image from the preset screen image to generate a difference image; and
s22: the difference image is calculated through binarization and image erosion to generate the difference image area.
Wherein: in step S6, the method further includes the steps of:
s61: interpreting the contour lengths by using the Douglas-Puck algorithm and the number of the end points to generate a button contour approximating the at least one pop-up window button; and
s62: generating the outline edge corresponding to the at least one pop-up window button according to the button outline and an outline correction precision, wherein the outline correction precision is between 1% and 10% of the outline length.
Wherein: further comprising the steps of:
s7: analyzing the at least one pop-up window button of the pop-up window by optical character recognition and generating a button text in the at least one pop-up window button.
Wherein: the minimum value of the number of endpoints is 4.
Also disclosed is a non-invasive detecting device for pop-up window buttons, which is used to identify at least one pop-up window button of a pop-up window of a display device, and is characterized in that the non-invasive detecting device for pop-up window buttons comprises:
an image capturing module for capturing a screen image of the display device;
the image analysis module is connected with the image acquisition module and stores a preset screen image and an image area threshold, the image analysis module is used for comparing the screen image with the preset screen image and generating a difference image area according to the screen image and the preset screen image, and when the difference image area is larger than the image area threshold, the image analysis module judges the difference image area as the pop-up window of the screen image; and
the calculation module is connected with the image analysis module and prestores a contour length threshold value and an end point number, screens out a plurality of contour lengths which are in accordance with the contour length threshold value and are greater than the contour length threshold value in the pop-up window through canny edge detection, and judges the contour lengths by utilizing a Douglas-Pock algorithm and the end point number to generate a contour edge corresponding to at least one pop-up window button.
Wherein: when the preset screen image does not exist, the screen image is replaced by the preset screen image through the image analysis module.
Wherein: the image analysis module subtracts the screen image from the preset screen image to generate a difference image, and calculates the difference image in a binarization and image erosion manner to generate the difference image area.
Wherein: the calculation module pre-stores a contour correction precision, and the calculation module interprets the plurality of contour lengths by using a Douglas-Pack algorithm and the number of the end points to generate a button contour approximating the at least one pop-up window button and generates a contour edge corresponding to the at least one pop-up window button according to the button contour and the contour correction precision, wherein the contour correction precision is between 1% and 10% of the contour length.
Wherein: further comprises an optical character recognition module connected with the computing module, wherein the optical character recognition module is used for analyzing the at least one pop-up window button of the pop-up window and generating a button character in the at least one pop-up window button.
Wherein: the minimum value of the number of endpoints is 4.
In summary, the non-invasive detection method and device for pop-up window buttons of the present invention can automatically detect the pop-up window of the device and identify the pop-up window button in the pop-up window, thereby improving the practicability. In addition, the non-invasive detection method and device of the pop-up window button can provide and identify complete pop-up window data, so that an operator can provide a solution according to all data in the pop-up window, the processing efficiency is improved, and the labor and production cost is reduced. In addition, the non-invasive detection method and device of the pop-up window button of the invention can also effectively identify the pop-up window of the display device through various image processing methods and image identification algorithms, thereby improving the processing efficiency.
Drawings
Fig. 1 is a flowchart illustrating the steps of a method for non-intrusive detection of pop-up window buttons according to an embodiment of the present invention.
FIG. 2 is a functional block diagram of a non-invasive detection apparatus for pop-up window buttons according to an embodiment of the present invention.
FIG. 3A is a schematic diagram of the display device of FIG. 2.
FIG. 3B is a diagram illustrating a pop-up window included in the display screen of the display device shown in FIG. 2.
FIG. 4 is a flowchart illustrating further steps in a method for non-intrusive detection of pop-up window buttons in accordance with the embodiment of FIG. 1.
FIG. 5 is a flowchart illustrating further steps in a method for non-intrusive detection of a pop-up window button according to the embodiment of FIG. 1.
Fig. 6A and 6B are schematic diagrams illustrating the identification of the pop-up window button in the non-invasive detection method of the pop-up window button according to the embodiment of fig. 1.
Detailed Description
In order that the advantages, spirit and features of the invention will be readily understood and appreciated, embodiments thereof will be described in detail hereinafter with reference to the accompanying drawings. It is noted that these embodiments are merely representative of the present invention, and the particular methods, devices, conditions, materials, etc., recited herein are not intended to limit the present invention or the corresponding embodiments. Also, the devices shown in the drawings are merely for relative positional representation and are not drawn to scale as they are actually drawn.
Reference throughout this specification to "one embodiment," "another embodiment," or "some embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments.
Please refer to fig. 1 and fig. 2. FIG. 1 is a flowchart illustrating a method for non-intrusive detection of a pop-up window button according to one embodiment of the present invention. Fig. 2 is a functional block diagram of the non-invasive detection apparatus 1 for pop-up window buttons according to an embodiment of the present invention. The non-invasive detection method of fig. 1 can be performed by the non-invasive detection apparatus 1 of the pop-up window button of fig. 2. As shown in fig. 2, the non-invasive detecting apparatus 1 of pop-up window button includes an image capturing module 11, an image analyzing module 12, a calculating module 13 and an optical character recognizing module 14, and the non-invasive detecting apparatus 1 of pop-up window button can be connected to the display device 5 of the device to capture and detect the screen image and data of the display device 5. The image analysis module 12 is connected to the image capturing module 11 and stores the preset screen image and the image area threshold. The calculation module 13 is connected to the image analysis module 12 and pre-stores the contour length threshold and the number of end points. The optical character recognition module 14 is connected to the calculation module 13 for analyzing the pop-up window button of the pop-up window and generating the button text in the pop-up window button. In practice, the image capturing module 11 may be an image capturing chip, the image analyzing module 12 may be an image analyzing chip, the computing module 13 may be a Central Processing Unit (CPU) and the optical character recognizing module 14 may be an optical character recognizing chip, but not limited thereto, the image capturing module 11 and the image analyzing module 12 may be integrated in the same image processing chip, and the optical character recognizing module 14 may be other chips capable of recognizing characters.
As shown in fig. 1 and fig. 2, in the present embodiment, the non-invasive pop-up window button detection method is used to identify the pop-up window 51 of the display device 5 and at least one pop-up window button 52 in the pop-up window 51. The non-invasive detection method of the pop-up window button comprises the following steps:
step S1: the image capturing module 11 captures a screen image of the display device 5;
step S2: the image analysis module 12 compares the screen image with the preset screen image, and generates a difference image area according to the screen image and the preset screen image;
and step S3: the image analysis module 12 determines whether the difference image area is greater than the image area threshold;
and step S4: when the difference image area is larger than the image area threshold, the image analysis module 12 determines the difference image area as a pop-up window 51 of the screen image;
step S5: the calculation module 13 filters a plurality of contour lengths in the pop-up window 51 that satisfy a contour length threshold by Canny edge detector;
step S6: the calculation module 13 determines a plurality of profile lengths by Douglas-Peucker algorithm (Douglas-Peucker algorithm) and the number of end points to generate a profile edge corresponding to the pop-up window button 52;
and step S7: the optical character recognition module 14 analyzes the pop-up window button 52 of the pop-up window 51 by optical character recognition and generates a button character in the pop-up window button 52.
As shown in fig. 1, in addition to the above steps S1-S7, the non-invasive detection method for pop-up window buttons of the present embodiment further includes the following steps: after the image capturing module 11 captures the screen image of the display device 5 (as shown in step S1), in step S11, the image analyzing module 12 first determines whether the predetermined screen image exists. If the predetermined screen image does not exist, in step S12, the image analysis module 12 replaces the screen image captured by the image capture module 11 with the predetermined screen image; if the default screen image already exists, the image analysis module 12 performs the following step S2. In practice, the image capturing module 11 can capture the screen images of the display device 5 at a plurality of different times, and the time interval for capturing the screen images by the image capturing module 11 can be preset or manually set. The preset screen image may be a screen image of the apparatus in normal operation. When the non-invasive detection apparatus 1 of the pop-up window button is first started or restarted, the image analysis module 12 does not exist or may empty the predetermined screen image. At this time, the image analysis module 12 takes the screen image captured by the image capture module 11 for the first time as the default screen image.
Further, after the image analysis module 12 of step S12 uses the screen image captured by the image capture module 11 for the first time as the preset screen image, in step S13, the operator may control whether the non-invasive detecting device 1 of the pop-up window button is to end the detection. If the detection is to be finished, the non-invasive detection device 1 of the pop-up window button will stop the detection of the pop-up window of the display device. If the detection is not finished, the process returns to step S1, and the image capturing module 11 captures the screen image of the display device 5 at the next time point.
Please refer to fig. 1, fig. 2, fig. 3A and fig. 3B together. Fig. 3A is a schematic diagram of the display device 5 of fig. 2. FIG. 3B is a schematic diagram illustrating that the display frame of the display device 5 shown in FIG. 2 includes a pop-up window 51. As shown in step S2 of fig. 1, fig. 3A and fig. 3B, after the image analysis module 12 stores the preset screen image and the image capture module 11 captures the screen image of the display device 5, the image analysis module 12 compares the screen image with the preset screen image and generates a difference image area according to the screen image and the preset screen image. Taking fig. 3A and fig. 3B as an example, fig. 3A is a screen image of the display device 5 when the apparatus is in normal operation, and fig. 3B is a screen image of the pop-up window 51 appearing on the display device 5 of the apparatus. In practice, fig. 3A may be the default screen image stored in the image analysis module 12, and fig. 3B may be the screen images captured by the image capture module 11 at other time points. The image analysis module 12 can compare fig. 3A and 3B and generate a difference image area between fig. 3A and 3B.
Please refer to fig. 1, fig. 3B and fig. 4. FIG. 4 is a flowchart illustrating further steps in a method for non-intrusive detection of a pop-up window button according to the embodiment of FIG. 1. As shown in fig. 1 and 4, in the present embodiment, the non-invasive detection method for the pop-up window button further includes the following steps, which can be executed by the image analysis module 12 to generate the difference image area between the screen image and the predetermined screen image. The method comprises the following steps: step S21: the image analysis module 12 subtracts the screen image and the preset screen image to generate a difference image; and step S22: the image analysis module 12 calculates the difference image by means of binarization and image erosion to generate a difference image region. In practice, the detailed process of step S21 may be that the image analysis module 12 converts the screen image and the preset screen image into the grayscale image, and deletes the screen image with the grayscale value at the same position by subtracting the grayscale image from the screen image. When the display device 5 jumps out of the pop-up window 51, the pop-up window 51 blocks a partial area of the display device 5, and the gray level value of the area blocked by the pop-up window 51 is changed. Therefore, when the image analysis module 12 subtracts the screen image from the predetermined screen image, the grayscale image hidden by the pop-up window 51 cannot be deleted, and the image analysis module 12 determines the grayscale image that cannot be deleted as the difference image.
As described above, in step S22, the image analysis module 12 performs binarization processing on the difference image. In practice, when the image analysis module 12 generates the difference image, each pixel of the difference image includes a gray scale difference. Further, the image analysis module 12 may include a gray level threshold, and the image analysis module 12 may highlight the difference image according to the gray level difference of the difference image and the gray level threshold. In practice, the gray level threshold value may be determined according to the frame characteristic of the display device 5 or the gray level value of the difference image. For example, the grayscale threshold is 45. When the gray scale difference value of one pixel of the difference image is smaller than 45, the image analysis module 12 adjusts the gray scale value of the pixel to 0; when the gray-scale difference of one of the pixels in the difference image is greater than or equal to 45, the image analysis module 12 adjusts the gray-scale value of the pixel to 225. Therefore, the image analysis module 12 can optimize and highlight the difference image through image binarization.
In addition, in step S22, the image analysis module 12 also processes the difference image by means of image erosion (exposure). Image erosion (erosion) is to eliminate the micro-variation region and noise region in the difference image through the structural element. The structural elements may be predetermined regions or may be determined manually. In practice, the production data or numbers will be displayed on the display device 5 when the apparatus is operating, and will change as the apparatus is operating. Therefore, the number in the screen image captured by the image capturing module 11 may be different from the number in the default screen image. That is, when the image analysis module 12 generates the difference image, the gray-scale difference region generated by the variation of the production data or number becomes a minute variation region and a noise region of the difference image. Therefore, the image analysis module 12 calculates a difference image through image erosion and generates a difference image region.
Please refer to fig. 1, fig. 2, and fig. 3B again. In this embodiment, the image analysis module 12 further stores the image region threshold. As shown in step S3 of fig. 1, after the difference image area is generated, the image analysis module 12 determines whether the difference image area is greater than the threshold of the storage image area. In practice, the image region threshold may be the size or area of the pop-up window. The threshold of the image area may be the minimum size of all pop-up windows that the display device will jump out of, or may be set manually. Therefore, when the image analysis module 12 analyzes and determines that the difference image area is greater than the threshold of the stored image area, it means that the difference image area is not an image difference that may occur during normal operation of the apparatus, and therefore, the image analysis module 12 determines that the difference image area is the pop-up window 51 of the screen image of the display device 5, as shown in step S4.
On the other hand, if the image analysis module 12 analyzes and determines in step S3 that the difference image area is smaller than the threshold of the stored image area, it indicates that there is a possibility that only the character string may change in the display device rather than flying out of the pop-up window. At this time, returning to step S13, the operator can control whether the non-invasive detecting device 1 of the pop-up window button is to end the detection. If the detection is to be finished, the non-invasive detection device 1 of the pop-up window button will stop the detection of the pop-up window of the display device. If the detection is not finished, the process returns to step S1, and the image capturing module 11 captures the screen image of the display device 5 at the next time point.
Please refer to fig. 1, fig. 2, fig. 3B, fig. 5 and fig. 6. FIG. 5 is a flowchart illustrating further steps in a method for non-intrusive detection of a pop-up window button according to the embodiment of FIG. 1. Fig. 6A and 6B are schematic diagrams illustrating the identification of the pop-up window button 52 in the non-invasive detection method of the pop-up window button according to the embodiment of fig. 1. When the image analysis module 12 recognizes the pop-up window 51, the calculation module 13 then determines and recognizes the pop-up window button 52 in the pop-up window 51. In step S5 of fig. 1, the calculation module 13 first uses Canny edge detector (Canny edge detector) to screen out a plurality of contour lengths in the pop-up window 51 that meet the contour length threshold. In practice, the pop-up window 51 may contain text, button outline, pattern, and graphics. When the calculation module 13 calculates the pop-up window 51 by Canny edge detector (Canny edge detector), the calculation module 13 traces and generates a contour of each data (such as characters, lines, etc.) in the pop-up window 51, and the contour of each data includes a contour length. And the contour length may be the length of a pixel. The contour length threshold may be the length of the pixels of the outline of the button in the pop-up window, and may be determined manually, but is not limited thereto. For example, as shown in fig. 6A, each line segment a in fig. 6A is the contour length meeting or greater than the contour length threshold. Therefore, if the threshold value of the contour length is 200 pixels, after the calculation module 13 calculates the contour length of all the data in the pop-up window 51, the calculation module 13 will screen out the contour length of the pop-up window 51 that is equal to or greater than 200 pixels.
After the calculation module 13 generates the profile length of all the data in the pop-up window 51, the calculation module 13 further uses Douglas-Peucker algorithm (Douglas-Peucker algorithm) and the number of the end points to interpret the screened profile length to generate the profile edge corresponding to the pop-up window button 52.
In this embodiment, the method for non-invasive detection of pop-up window buttons further comprises: step S61: the calculation module 13 uses Douglas-Peucker algorithm (Douglas-Peucker algorithm) and the number of endpoints to interpret the profile length to generate a button profile approximating the pop-up window button 52; and step S62: the calculation module 13 generates a contour edge corresponding to the pop-up window button 52 according to the button contour and the contour correction accuracy. In practice, the number of endpoints may be the number of endpoints of the pop-up window 51. When the shape of the pop-up window is rectangular, the number of the end points is 4, but is not limited thereto, and the number of the end points may be determined according to the shape of the pop-up window. As shown in fig. 6A, when the calculating module 13 utilizes Douglas-Peucker algorithm (Douglas-Peucker algorithm) and the number of the end points is 4 to determine the contour length, the calculating module 13 will find out 4 end points B in all the contour lengths. At this time, the outline length in the rectangle surrounded by the 4 end points B forms a button outline approximating the pop-up window button. Then, the calculation module 13 generates the outline edge corresponding to the pop-up window button 52 according to the button outline and the outline correction precision. The accuracy of the profile correction can be a distance dimension (epsilon) in a Douglas-Peucker algorithm (Douglas-Peucker algorithm), and the accuracy of the profile correction can be between 1% and 10% of the length of the profile, but is not limited thereto, and the accuracy of the profile correction can also be preset or manually determined. As shown in fig. 6B, when the calculating module 13 calculates the button outline according to the outline correction accuracy, the calculating module 13 will find the line segment with the maximum distance based on the endpoint B and delete the excess outline length (as shown by the dotted line in the figure), so as to interpret and generate the outline edge C corresponding to the pop-up window button. Please note that, in the present embodiment, the pop-up window 51 in FIG. 3B includes 2 pop-up window buttons 52, 53, but the embodiment is not limited thereto, and the number of pop-up window buttons may be 1 or more than 3.
As shown in fig. 1 to fig. 5, the non-invasive detecting method for pop-up window buttons of the present invention first captures screen images of the display device 5 at different time points through the image capturing module 11, then compares the preset screen image (shown in fig. 3A) and the screen image (shown in fig. 3B) through the image analyzing module 12, and then simplifies the processes of image subtraction, binarization and image erosion to generate a difference image area, so as to determine the pop-up window 51 in the display device 5. Furthermore, the calculation module 13 calculates and identifies the pop-up window button 52 of the pop-up window 51 by Canny edge detector (Canny edge detector) and Douglas-Peacker algorithm (Douglas-Peucker algorithm), respectively. Finally, the OCD module 14 recognizes all the characters in the pop-up window 51. Therefore, the non-invasive detection method of the pop-up window button can automatically detect the pop-up window in the screen image of the display device, and utilizes the optical character recognition module to recognize all data of the pop-up window button.
In summary, the non-intrusive detection method and device for pop-up window buttons of the present invention can automatically detect the pop-up window of the device and identify the pop-up window button in the pop-up window, so as to improve the practicability. In addition, the non-invasive detection method and device of the pop-up window button can provide and identify complete pop-up window data, so that an operator can provide a solution according to all data in the pop-up window, the processing efficiency is improved, and the labor and production cost is reduced. In addition, the non-invasive detection method and device of the pop-up window button of the invention can also effectively identify the pop-up window of the display device through various image processing methods and image identification algorithms, thereby improving the processing efficiency.
The above detailed description of the preferred embodiments is intended to more clearly illustrate the features and spirit of the present invention, and is not intended to limit the scope of the present invention by the preferred embodiments disclosed above. On the contrary, it is intended to cover various modifications and equivalent arrangements included within the scope of the claims. The scope of the claims is thus to be accorded the broadest interpretation so as to encompass all such modifications and equivalent arrangements as is within the scope of the appended claims.
Claims (12)
1. A non-invasive detecting method of pop-up window buttons is used for identifying at least one pop-up window button of a pop-up window of a display device, and is characterized in that the non-invasive detecting method of the pop-up window buttons comprises the following steps:
s1: capturing a screen image of the display device;
s2: comparing the screen image with a preset screen image, and generating a difference image area according to the screen image and the preset screen image;
s3: judging whether the difference image area is larger than an image area threshold value;
s4: when the difference image area is larger than an image area threshold, judging the difference image area as the pop-up window of the screen image;
s5: in the pop-up window, a contour length threshold value is preset, and then a plurality of contour lengths which are more than the contour length threshold value are screened out by canny edge detection; and
s6: the method includes presetting an end number of the at least one pop-up window button, and then interpreting the contour lengths by using a Douglas-Pock algorithm and the end number to generate a contour edge corresponding to the at least one pop-up window button.
2. The method for non-invasive detection of pop-up window buttons according to claim 1, further comprising the steps of:
s11: judging whether the preset screen image exists or not, and if so, executing a step S2; and
s12: if the determination result in the step S11 is negative, the screen image is replaced with the predetermined screen image.
3. The non-invasive detecting method for pop-up window button as claimed in claim 1, wherein the step S2 further comprises the steps of:
s21: subtracting the screen image from the preset screen image to generate a difference image; and
s22: the difference image is calculated through binarization and image erosion to generate the difference image area.
4. The non-invasive detecting method for pop-up window button as claimed in claim 1, wherein the step S6 further comprises the steps of:
s61: interpreting the contour lengths by using the Douglas-Puck algorithm and the number of the end points to generate a button contour approximating the at least one pop-up window button; and
s62: generating the outline edge corresponding to the at least one pop-up window button according to the button outline and an outline correction precision, wherein the outline correction precision is between 1% and 10% of the outline length.
5. The method for non-invasive detection of pop-up window buttons according to claim 1, further comprising the steps of:
s7: analyzing the at least one pop-up window button of the pop-up window by optical character recognition and generating a button text in the at least one pop-up window button.
6. The method as claimed in claim 1, wherein the minimum number of the endpoints is 4.
7. A non-invasive detecting device for pop-up window buttons is used to identify at least one pop-up window button of a pop-up window of a display device, and the non-invasive detecting device for pop-up window buttons comprises:
an image capturing module for capturing a screen image of the display device;
the image analysis module is connected with the image acquisition module and stores a preset screen image and an image area threshold, the image analysis module is used for comparing the screen image with the preset screen image and generating a difference image area according to the screen image and the preset screen image, and when the difference image area is larger than the image area threshold, the image analysis module judges the difference image area as the pop-up window of the screen image; and
and the calculation module is connected with the image analysis module and prestores a contour length threshold value and an end point number, screens out a plurality of contour lengths which are more than the contour length threshold value in the pop-up window by canny edge detection, and judges the contour lengths by utilizing a Douglas-Pock algorithm and the end point number to generate a contour edge corresponding to at least one pop-up window button.
8. The apparatus of claim 7, wherein the image analysis module replaces the screen image with the default screen image when the default screen image is absent.
9. The apparatus of claim 7, wherein the image analysis module subtracts the screen image and the predetermined screen image to generate a difference image, and calculates the difference image by binarization and image erosion to generate the difference image region.
10. The apparatus of claim 7, wherein the computing module pre-stores a profile calibration accuracy, the computing module determines the profile lengths using the Douglas-Puck algorithm and the endpoint number to generate a button profile approximating the at least one pop-up window button, and generates a profile edge corresponding to the at least one pop-up window button according to the button profile and the profile calibration accuracy, wherein the profile calibration accuracy is between 1% and 10% of the profile length.
11. The apparatus for non-invasive detecting of pop-up window buttons according to claim 7, further comprising an optical character recognition module connected to the computing module, the optical character recognition module being configured to analyze the at least one pop-up window button of the pop-up window and generate a button text of the at least one pop-up window button.
12. The apparatus for non-invasive detecting the pop-up window button as claimed in claim 7, wherein the minimum value of the number of end points is 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110707217.6A CN115525891A (en) | 2021-06-25 | 2021-06-25 | Non-invasive detection method and device for pop-up window button |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110707217.6A CN115525891A (en) | 2021-06-25 | 2021-06-25 | Non-invasive detection method and device for pop-up window button |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115525891A true CN115525891A (en) | 2022-12-27 |
Family
ID=84693540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110707217.6A Pending CN115525891A (en) | 2021-06-25 | 2021-06-25 | Non-invasive detection method and device for pop-up window button |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115525891A (en) |
-
2021
- 2021-06-25 CN CN202110707217.6A patent/CN115525891A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1970839B1 (en) | Apparatus, method, and program for face feature point detection | |
| JP6794737B2 (en) | Information processing equipment, information processing methods, programs and inspection systems | |
| US10043090B2 (en) | Information processing device, information processing method, computer-readable recording medium, and inspection system | |
| CN112577969B (en) | Defect detection method and defect detection system based on machine vision | |
| US20160314368A1 (en) | System and a method for the detection of multiple number-plates of moving cars in a series of 2-d images | |
| JP2009064175A (en) | Object detection device and object detection method | |
| CN110927184A (en) | Visual detection method for chip packaging defects | |
| CN111912846A (en) | Machine vision-based surface defect and edge burr detection method | |
| JP3756507B1 (en) | Image processing algorithm evaluation method and apparatus, image processing algorithm generation method and apparatus, program, and program recording medium | |
| JP6903966B2 (en) | Information processing equipment, information processing systems and programs | |
| EP3904865A1 (en) | Image processing device, image processing method, and image processing program | |
| CN116559177A (en) | Defect detection method, device, equipment and storage medium | |
| US20120237118A1 (en) | Image processing device, image processing method, and image processing program | |
| CN115525891A (en) | Non-invasive detection method and device for pop-up window button | |
| US20230154144A1 (en) | Method and system or device for recognizing an object in an electronic image | |
| JP2010140444A (en) | Inspection system and method of controlling same, and computer program | |
| JP5046148B2 (en) | Image analysis apparatus and object recognition method | |
| JP2003044832A (en) | Method, device and program for setting inspection parameter | |
| TWI787885B (en) | Non-intrusive detection method and device for pop-up window button | |
| CN114757888A (en) | Windshield dirt detection method based on textural features | |
| CN112329561A (en) | Finger abnormity detection method and system for optical fingerprint acquisition under screen | |
| US6208756B1 (en) | Hand-written character recognition device with noise removal | |
| US20220365673A1 (en) | Non-intrusive shared processing method and system for pop-up window | |
| Saad et al. | Real-time LCD digit recognition system | |
| KR101689705B1 (en) | Method for detecting pattern information area using pixel direction information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |