TWI784720B - Electromagnetic susceptibility tesing method based on computer-vision - Google Patents

Abstract

The present disclosure proposes an electromagnetic susceptibility (EMS) tesing method based on computer-vision, wherein the method is applicable to an electronic device with a monitor. The method includes a training stage and a testing stage. During the training stage, the electronic deivce receives a testing data and the monitor displays a first picture. A camera captures the first picture to generate a template video. The processor generates a plurality of template image according to the template video at least. During the testing stage, after the processor generates the plurality of template images, an antenna emits an interference signal to the electronic deivce. The electronic device receives the testing data an the monitor displays a second picture. The camera captures the second picture to generate a testing video. The processor generates a testing image according to the testing video and calculates a difference ratio between the testing image and each template image. The processor sends an alert signal when the difference ratio is greater than a threshold.

Description

Electromagnetic Susceptibility Testing Method Based on Computer Vision

The invention relates to electromagnetic susceptibility testing and image comparison, in particular to an electromagnetic susceptibility testing method based on computer vision.

Electromagnetic susceptibility (Electromagnetic Susceptibility, EMS) refers to the tolerance or immunity to electromagnetic interference (Electromagnetic Interference, EMI) or radio frequency interference (Radio Frequency Interference, RFI). If an electronic device does not generate EMI or RFI to other devices, and has good EMS, then the electronic device has good electromagnetic compatibility (EMC). In other words, the goal of EMC is the correct operation of different devices in a common electromagnetic environment.

Radiated field susceptibility testing for EMS typically uses a high-power radio frequency or electromagnetic device and a radiating antenna to direct energy to the electronic device under test, such as a laptop computer. The electronic device to be tested is placed in an RF anechoic chamber, and one of various test data is displayed on the screen in turn. While the antenna emits electromagnetic waves, the camera shoots the screen of the electronic device under test, and the testers outside the RF anechoic room watch the captured test images for abnormalities and check Record the electromagnetic wave frequency and setting parameters used when the abnormality occurs. The so-called anomalies include complete black screen, flickering screen, distorted lines in the test pattern, etc.

However, labor-intensive testing is easy on the eyes, and testers may overlook some anomalies.

In view of this, the present invention proposes a computer vision-based electromagnetic susceptibility test method, using a computer vision-based EMS test system to continuously receive test images from cameras in a radio frequency anechoic chamber, and imitate testers to watch the images and conduct Judgment mechanism, so as to judge whether there is any abnormality in the test image, and then automatically record various setting parameters when abnormality occurs.

According to an embodiment of the present invention, a method for testing electromagnetic susceptibility based on computer vision is applicable to an electronic device with a screen. The method includes: a training phase, including: the electronic device receives a test data, and uses the A firefly display corresponds to a first frame of the test data; a camera shoots the first frame to generate a template video; and a processor at least generates a plurality of template images according to the template video; and a test phase includes: After the template images, an antenna is used to send an interference signal to the electronic device; the electronic device receiving the interference signal receives the test data, and displays a second picture corresponding to the test data on the screen; the video camera shooting the second frame to generate a test video; the processor generates a test image according to the test video; the processor calculates a difference ratio between the test image and each of the template images; and when the difference ratio is greater than a threshold , the processor sends out a warning signal.

To sum up, the present invention proposes a method for testing electromagnetic susceptibility based on computer vision. In the training phase, the template image belonging to the normal picture is dynamically updated from the video captured by the camera. The test image is captured in the test image and compared with the template image, so as to automatically detect any abnormalities in the screen displayed on the screen. In addition, the present invention can dynamically adjust the threshold for abnormality judgment according to past abnormality detection records.

The above description of the disclosure and the following description of the implementation are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the patent application scope of the present invention.

S: training phase

T: testing stage

S1~S3, S31~S33, S331~S334, T1~T7, T51~T53, T5’, T51’~T52’: steps

Fig. 1 is the flowchart of the electromagnetic susceptibility testing method based on computer vision of an embodiment of the present invention; Fig. 2 is the detailed flowchart of step S3 among Fig. 1; Fig. 3 is the detailed flowchart of step S33 among Fig. 2; Fig. 4 is a detailed flowchart of an embodiment of step T5 in FIG. 1 ; and FIG. 5 is a detailed flowchart of another embodiment of step T5 in FIG. 1 .

The detailed features and characteristics of the present invention are described in detail below in the implementation mode, and its content is enough to enable any person familiar with the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings , anyone who is familiar with the related art can easily understand the ideas and features related to the present invention. The following examples are to further describe the concept of the present invention in detail, but not to limit the scope of the present invention in any way.

The present invention proposes an electromagnetic susceptibility (EMS) testing method based on computer vision, which is suitable for testing electronic devices under test (hereinafter referred to as electronic devices) with screens.

FIG. 1 is a flowchart of a method for testing electromagnetic susceptibility based on computer vision according to an embodiment of the present invention, the method includes a training phase S and a testing phase T. The training stage S includes steps S1~S3, and the testing stage T includes steps T1~T6. The implementation details of each step are introduced in order below.

Step S1 is "the electronic device receives the test data, and the screen displays the first picture". The first frame corresponds to test data, and the types of test data include static test charts, semi-static test charts and dynamic test charts. Figure 2 is a schematic diagram corresponding to the above three test data. The static test chart is a picture, and the color of each pixel in the picture is always the same; the dynamic test chart is a video, and the color of each pixel in the picture changes with time. ;The semi-static test pattern is equivalent to the integration of the static test pattern and the dynamic test pattern. In other words, all the pixels in the screen are divided into two parts, the color of some pixels is always the same, and the color of the other part of pixels changes with time.

Step S2 is "the camera captures the first frame to generate a template video". A template video is a video obtained by taking a screen shot.

Step S3 is "the processor generates a plurality of template images at least according to the template video", please refer to FIG. 2 , which is a detailed flow chart of step S3. Step S31 is "setting ROI in the template video", step S32 is "performing correction operation according to ROI", and step S33 is "executing algorithm to generate multiple template images".

In step S31, the processor sets a region of interest (ROI) in the template video. In practice, the image captured by the camera may include the surrounding environment of the screen, such as the desktop on which the electronic device is placed and the wall behind the screen, but the subsequent process only requires that the template image belongs to the first Therefore, it is necessary to perform step S31 to filter out the parts irrelevant to the first frame. Step S31 can be divided into automatic and manual implementations.

The method of automatically setting the ROI is to segment the part belonging to the first frame from each frame of the template video according to the color (or brightness) of the environment where the electronic device is located and an adaptive threshold setting (adaptive thresholding) algorithm. The adaptive threshold setting algorithm is, for example, the Otsu algorithm (OTSU), K-means (K-means) or the adaptiveThreshold function in OpenCV. In addition, if there are no other light-emitting elements except the screen in the RF anechoic chamber, the color of the environment where the electronic device is located is black, so it is easy to separate the light-emitting area from the template video as a template image.

The way to manually set the ROI is to use another screen to display the template video, and the user decides the range of the first frame on the screen displayed on the other screen.

In practice, the electronic device is placed on a rotating table, and the electronic device can be evenly affected by the interference signal during the test stage T through rotation, and the camera is usually set at a fixed position, so in the template video captured by the camera, the shape of the screen And the position will change with the rotation angle. Therefore, if the camera can rotate synchronously with the electronic device so that the lens is always facing the screen, in other embodiments, step S31 can also be omitted. For example, the processor first judges whether the template video contains parts other than the first frame, if the judgment result is "No", step S31 can be skipped, and if the judgment result is "Yes", step S31 is executed.

In step S32, the correcting operation is, for example, perspective transformation. Assuming another screen is used to display the template video, the picture that the user sees on the other screen may have a visual difference compared with the first picture, which is caused by the internal parameters of the camera itself and the optical parameters of the lens. For example: if the camera uses a fisheye lens, the edge of the picture displayed on the other screen may be distorted or curved compared with the edge of the first picture, so it is necessary to borrow The deformed ROI in the template video is restored to the shape of the first frame (usually a rectangle, but the invention is not limited thereto) by the correction operation.

In step S33, the processor can extract a plurality of frames from the template video according to the specified sampling frequency, and then execute an algorithm based on these frames to determine the foreground attribute and the background attribute of each pixel in each frame, Pixels with background attributes are the standard for subsequent comparisons, and pixels with foreground attributes will be regarded as abnormal pixels in most cases.

In the first embodiment of step S33, the algorithm is Gaussian mixture model or Visual Background Extractor (ViBE, Visual Background Extractor).

In the second embodiment of step S33, the details of the algorithm are as follows: first, the processor executes a corresponding program according to the type of test data to generate a template image.

If the test data is a dynamic test pattern, the processor captures at least one frame of the template image as the template image, and the present invention does not limit the number of frames to be captured, nor the rules for frame capture. For example, if the template video includes 30 frames F1~F30, the processor can set all the frames F1~F30 as the template image; or only select F1~F10 as the template image. The more template images are selected, the higher the alignment accuracy in the test stage T will be.

If the test data is a static test pattern, the processor will execute the background initialization procedure according to the first frame of the template image, and execute the background update procedure according to the second frame, wherein the second frame is one of multiple frames Or, and the timing of these frames is after the timing of the first frame. Please refer to FIG. 3 , which is a flowchart of step S33 when the test data is a static test pattern.

8。N代表參考像素的取樣深度，除非位置位於訊框的邊緣，每一個第一中心像素基本上都可從周邊的多個像素中選擇N個參考像素做為像素比對時的標準。鄰接於第一中心像素的距離依據取樣深度N進行調整，例如當取樣深度N為20時，參考像素的選擇範圍須擴展至一個5×5的區域，此區域為以第一中心像素為中心，向四周各延伸兩個像素的距離所形成的範圍。 The background initialization procedure is shown in step S331, "obtain a plurality of reference pixels adjacent to the first central pixel". The first central pixel is each pixel of the first frame. Assuming that the frame size is 3×3, the frame has 9 pixels arranged in a nine-square grid. At this time, the first central pixel is located in the central position of the nine-square grid, and the reference pixel can be selected from the remaining eight pixels N, N

8. N represents the sampling depth of the reference pixel. Unless the position is located at the edge of the frame, each first central pixel can basically select N reference pixels from the surrounding pixels as a standard for pixel comparison. The distance adjacent to the first central pixel is adjusted according to the sampling depth N. For example, when the sampling depth N is 20, the selection range of reference pixels must be extended to a 5×5 area, which is centered on the first central pixel. The range formed by extending a distance of two pixels in each direction.

The background update procedure is shown in steps S332, S333, and S334. Step S332 is "judging the similarity between the second central pixel and each reference pixel", step S333 is "determining the attribute of the second central pixel according to the similarity", and step S334 is "replacing the first central pixel or reference pixel with the second central pixel according to the specified probability, attribute and number of consecutive frames".

In steps S332 and S333, the second center pixel is each pixel of the second frame. When the number of pixels similar to the second central pixel among the N reference pixels of the first frame is less than the threshold value m, the processor sets the foreground attribute of the second central pixel. On the contrary, when the number of pixels similar to the second central pixel among the N reference pixels of the first frame is not less than the threshold value, the processor sets the background attribute of the second central pixel. In one embodiment, the condition for judging that two pixels are similar is that the difference between the Euclidean distances of the two pixels is smaller than a preset value. In another embodiment, the condition for judging that two pixels are similar is that the difference between the pixel intensities of the two pixels is smaller than a preset value.

In step S334, when the second central pixel has the background attribute, the first central pixel or the reference pixel is replaced by the second central pixel according to a specified probability r.

In step S334, if in the consecutive C frames of the template video, the second center pixel at the same position has the foreground attribute, then the processor replaces the second center pixel in the first frame The first center pixel of . Take a scenario as an example: during the training phase S, the user accidentally moves the mouse cursor from the upper left corner of the screen to the lower right corner at a certain point in time, and then stays in the lower right corner. In this scenario, the multiple pixels that make up the cursor will be set to the foreground attribute at the beginning of the training phase S, because the mouse cursor is not part of the test data, but since these pixels persist in multiple consecutive frames, Therefore these pixels will be considered as part of the template image.

In one embodiment, based on considerations of computing speed and accuracy, the following parameter settings are adopted: reference pixel sampling depth N=5, threshold value m=2, specified probability r=1/16, and number of consecutive frames C=15 .

If the test data is a semi-static test chart, the test data can be divided into a dynamic part and a static part. For these two parts, refer to the methods of the dynamic test chart and the static test chart for adaptive adjustment, and then generate a template image.

During the training phase S, the processor continuously updates the background model according to the new second frame in the template video and the process shown in FIG. 3 . The background model is composed of a basic template image and a reference pixel dictionary. The basic template image is composed of a plurality of first central pixels (equivalent to the updated first frame), and the reference pixel dictionary records the position of each pixel of the basic template image. Corresponding multiple reference pixels. From another point of view, the background model can be regarded as composed of multiple template images, where each template image is equivalent to each pixel position of the basic template image selected from the first center pixel and a plurality of reference pixels. combination.

After the training phase S is completed, the processor has generated one or more template images and recorded them into the storage device, so that these template images can be read as norms during the testing phase T.

In the embodiments described above, the processor directly uses the first frame of the template video as the initial template image, and then updates the template image according to the second frame of the template video. In another embodiment In this method, the processor may first use the test data as an initial template image, and then update the template image according to the template image. According to the above method, the number of frames used in the training stage S can be reduced under the premise of maintaining the comparison accuracy, thereby reducing the calculation cost during training.

The test phase T includes steps T1~T6, wherein step T1 is "the antenna sends out interference signals to the electronic device".

Step T2 is "the electronic device receives the test data, and the screen displays a second frame", where the second frame corresponds to the test data. In step T2 and step S1, the types of test data used are the same.

Step T3 is "the camera shoots a second frame to generate a test video". Steps T2~T3 are similar to steps S1~S2, but the difference is that in the training phase T, the electronic device is affected by the interference signal, so the second frame may be Abnormal phenomena occur, such as black screen, flickering screen, distorted lines in the test pattern, etc. In the training phase S, the electronic device is not affected by the interference signal, so a normal screen model can be derived from it.

Step T4 is "the processor generates a test image according to the test video", the step T4 is similar to the step S3, the difference is: in the step T4, the processor generates a test image according to each frame of the test video. In one embodiment, if the process of setting ROI and performing correction shown in FIG. 2 is included in step S3, the process shown in FIG. 2 is adaptively modified to be applicable to step T4. In addition, if it is desired to speed up the execution speed of the test stage T, in step T4, the processor selects some frames from all the frames of the test video to generate a plurality of test images.

Step T5 is "the processor calculates the difference ratio between the test image and each template image". According to the type of the test data, the processor executes a corresponding program to calculate the difference ratio. When the test data is a dynamic test chart, the method of calculating the difference ratio is shown in step T5 of Figure 4; when the test data is a static test chart, the method of calculating the difference ratio is shown in step T5' of Figure 5.

When the test data is a dynamic test image, multiple template images will be generated after the training phase S, and the processor compares the features of each template image with the input image in the neural network based on the Perceptual Similarity Metric similarity.

Please refer to Figure 4, step T51 is "input the test image and each template image to the neural network model", the neural network has multiple layers of convolution operations, and the neural network model is for example AlexNet, which is used to calculate two images perceived difference. In step T51 , an operation of resizing the test image to the size of the template image may also be included.

Step T52 is "calculate multiple difference values based on the feature map of the test image and the feature map of each template image". In each layer of convolution operation, the feature map of the test image and the feature map of each template image will be obtained. In one embodiment, in the convolution operation of the first three layers, the processor upsamples the feature map of the template image and the feature map of the test image, and after enlarging the two feature maps to the same size, calculates the two The difference value between feature maps. The calculation method is as follows: for two pixels located at the same position L in two feature maps, the processor calculates the Euclidean distance D between these two pixels as the difference value of the difference map at position L. The smaller the Euclidean distance D, the more similar the two pixels are; the larger the Euclidean distance D, the less similar the two pixels are. In one embodiment, the Euclidean distance D between two pixels is calculated according to the respective RGB values, so the difference value is a three-dimensional data.

Step T53 is "generating a difference map based on multiple difference values and calculating the difference ratio". In the difference map, if the Euclidean distance D of the pixel at position L is greater than the preset value, the processor marks position L of the difference map as a difference pixels. The difference ratio is related to the number of all difference pixels in the difference map and the total number of pixels in the difference map, wherein the difference value of each difference pixel is greater than a preset value. In one embodiment, the processor calculates the area of the block formed by all the difference pixels, and the ratio of the area to the total area of the difference map is the difference ratio.

After the process of steps T51-T5, each template image and test image will generate a difference map and a difference ratio. In one embodiment, the processor can select the largest one from these difference ratios as the criterion for judging whether the test image passes the test. standard, the weighted average of these difference ratios can also be calculated as the standard for judging whether the test image passes the test, and the present invention is not limited to this. When the test data is a static test chart, the method of calculating the difference ratio is as shown in Figure 5. T5', which includes step T51' and step T52'. During the testing phase T, the background model has stopped updating.

Step T51' is "calculate the number of differences for each pixel based on the background model and the test image", in detail, for the pixel PT at the position L of the test image and the pixel _{PM corresponding} to the position L in the background model, the processing The detector calculates how many pixels in _PM are similar to pixel PT _. The processor can calculate the number of difference pixels (difference number) according to the number of similar pixels (similar number) and the number of pixels _PM , wherein _PM includes the first central pixel and its corresponding plurality of reference pixels.

Step T52' is "determining difference pixels according to difference quantity, and generating difference map and difference ratio".

In one embodiment, for each pixel of the test image, if the difference quantity of the pixel exceeds a tolerance value, the pixel is marked as the difference pixel in the difference map, otherwise, the pixel is not marked as the difference pixel; and the difference ratio is the proportion of the difference pixels in the difference map to all the pixels in the difference map.

Please refer to FIG. 1, step T6 is "judging whether the difference ratio is greater than the threshold".

If the judgment result is "Yes", go to step T7, if the judgment result is "No", go back to step S1, and the processor selects another type of test data and then returns to step S1 to execute the next round of EMS testing.

In step S7, the processor sends out a warning signal. In addition, the processor records the abnormality type, the frequency of the interference signal, the type of test data, the configuration information of the electronic device, and the currently used threshold into the database of the storage device.

In one embodiment, a step of dynamically adjusting the threshold is further included before step T6. Specifically, the processor reads configuration information of the electronic device, and searches a database for historical test records similar to the configuration information. If the historical test record and the historical threshold used therein are found, the processor adjusts the threshold used in the next step T6 according to the historical threshold.

To sum up, the present invention proposes a method for testing electromagnetic susceptibility based on computer vision. In the training phase, the template image belonging to the normal picture is dynamically updated from the video captured by the camera. The test image is captured in the test image and compared with the template image, so as to automatically detect any abnormalities in the screen displayed on the screen. In addition, the present invention can dynamically adjust the threshold for abnormality judgment according to past abnormality detection records.

Although the present invention is disclosed by the aforementioned embodiments, they are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, all changes and modifications are within the scope of patent protection of the present invention. For the scope of protection defined by the present invention, please refer to the appended scope of patent application.

S: training phase

T: testing stage

S1~S3,T1~T7: steps

Claims (10)

An electromagnetic susceptibility testing method based on computer vision, suitable for an electronic device with a screen, the method includes: a training phase, including: the electronic device receives a test data, and displays the corresponding test data on the screen a first frame of a camera; a camera captures the first frame to generate a template video; and a processor at least generates a plurality of template images according to the template video; and a test stage includes: after generating the template images, using An antenna sends an interference signal to the electronic device; the electronic device receiving the interference signal receives the test data, and displays a second frame corresponding to the test data on the screen; the camera captures the second frame to generate a test video; the processor generates a test image according to the test video; the processor calculates a difference ratio between the test image and each of the template images; and when the difference ratio is greater than a threshold, the processor sends a warning sign. According to the computer vision-based electromagnetic susceptibility testing method described in Claim 1, the processor at least generates the template images according to the template video including: setting an interest region in the template video; performing a calibration according to the interest region operation; and executing an algorithm to generate the template images. The electromagnetic susceptibility testing method based on computer vision as described in Claim 2, wherein the algorithm is a Gaussian mixture model or a visual background extractor (ViBE, Visual Background Extractor). The electromagnetic susceptibility testing method based on computer vision as described in claim 2, wherein the test data is a dynamic test pattern, and the algorithm includes: the processor captures at least one frame of the template image as the template images . The electromagnetic susceptibility test method based on computer vision as described in claim 2, wherein the test data is a static test pattern, and the algorithm includes: a background initialization program, including: obtaining multiple pixels adjacent to a first central pixel a reference pixel, the first center pixel is each pixel of a first frame in the template image; and the background update procedure includes: judging the similarity between a second center pixel and each of the reference pixels, wherein The second central pixel is each pixel of a second frame in the template image, and the timing of the second frame is after the timing of the first frame; when the reference pixels are similar to the second When the number of pixels in the center pixel is less than a threshold value, set a foreground attribute of the second center pixel; otherwise, set a background attribute of the second center pixel; when the second center pixel has the background attribute, according to a specified probability , replace the first center pixel or the reference pixel with the second center pixel; and in a plurality of consecutive frames of the template video, when the second center pixel has the foreground attribute, replace with the second center pixel The first center pixel. The electromagnetic susceptibility testing method based on computer vision as described in Claim 1, wherein the test image is a dynamic test chart, and the calculation of the difference ratio between the test image and each of the template images by the processor includes: inputting the The test image and each of the template images are fed to a neural network model, and the neural network has multiple layers of convolution operations; in each of the layers of convolution operations, a feature map of the test image and each of the a feature map of the template image; upsampling the feature map of the test image and the feature map of each of the template images for each of the layers, and based on the pixel scale, for the feature map of the test image and each For the feature map of the template images, a Euclidean distance is calculated as a difference value in a difference map; wherein the difference ratio is related to the number of difference pixels in the difference map and a total number of pixels in the difference map, The difference value of each of the difference pixels is greater than a preset value. The electromagnetic susceptibility testing method based on computer vision as described in claim 1, wherein the test data is a static test chart, and the calculation of the difference ratio between the test image and each of the template images by the processor includes: according to a The background model and the test image calculate a difference quantity for each pixel, wherein the background model is associated with the template images; and determine a difference pixel according to the difference quantity, and generate a difference map and the difference ratio. The electromagnetic susceptibility testing method based on computer vision as described in Claim 1, wherein the processor at least generating the template images according to the template video includes: generating the template images according to the test data and the template video by the processor. The electromagnetic susceptibility testing method based on computer vision as described in Claim 1, wherein the testing phase further includes: the processor obtains a configuration information of the electronic device; the processor searches a database to obtain a historical test record a historical threshold, wherein a historical configuration information of the historical test record is similar to the configuration information; and the processor adjusts the threshold according to the historical threshold. The electromagnetic susceptibility testing method based on computer vision as described in claim 1, before the processor calculates the difference ratio between the test image and each of the template images, further includes: according to the size of each of the template images Resize the test image.

Images