KR102660883B1 - A method for testing media processing on embedded devices and computing devices using the process - Google Patents

Abstract

The present invention provides a testing method for an embedded device with a hierarchical host structure, comprising: receiving, by a main host, setting information of an execution environment of a target device; Building a virtualized execution environment on a subhost according to the configuration information; performing one or more test cases in the constructed execution environment; and monitoring the execution result, and performing the test case includes testing a physical connection operation between the target device and the subhost.

Description

A method for testing media processing on embedded devices and computing devices using the process}

The present invention relates to a testing method for embedded devices. Preferably, it relates to a method for testing media processing of an embedded device designed to perform a specific function.

Conventionally, in order to test an embedded device, all hardware environments related to the embedded device for testing are established in each host device, and then software is downloaded to the embedded device and testing is performed.

At this time, in order to test multiple embedded devices at the same time in the same environment, you can connect multiple embedded devices to one host and test multiple devices at the same time.

The user must be equipped with the relevant test equipment and must proceed with the test while changing the hardware configuration and software according to the user's desired environment.

Therefore, if multiple users share a single hardware device and conduct testing, it is necessary to reconfigure the desired environment each time and conduct testing, which is an inefficient repetitive task.

In addition, in order to distribute the test equipment to all users, high production costs are required in the early stages of development, and frequent hardware changes have the disadvantage of being difficult in terms of cost.

In particular, in the case of testing media processing equipment such as multimedia, a person plays back media such as a sound source or video and determines whether the media is actually played normally, whether there are interruptions or delays during playback, and whether there is noise. I am making judgments and inputting them while watching the execution process. In this case, there is a problem that remote, large-scale and automated testing cannot be conducted because humans are involved in all processes and devices without being able to test large quantities of devices through automation.

In addition, the media processing test device must always be placed near the tester, and the tester must also remain near the test device for a person to continuously perform and judge it. Therefore, it is difficult to conduct the same test on multiple devices at the same time, and each device must be tested separately. As the same task is repeated, there is a problem in that very inefficient testing is performed in terms of time and cost.

The purpose of the present invention is to propose a method for efficiently testing a limited number of embedded devices that have high development costs in the early stages of development or are constantly being improved.

In addition, the purpose of the present invention is to propose a method of configuring hardware and configuring an environment in which software can be installed according to the environment and combination for testing in various environments during the software development process.

Specifically, the purpose of the present invention is to propose an environment in which software can be tested in all preset environments or conditions by selecting software in an environment that virtualizes expected environments or conditions.

In addition, the present invention records the media processing performance of the embedded device and verifies it by comparing it with the original media to determine the results of the test remotely or automatically, and improves media processing test efficiency at the development stage. The purpose is to improve.

In addition, the present invention aims to enable remote and real-time monitoring of the media processing test performance status for each test device and improve error detection performance of the test based on this.

A method according to an embodiment of the present invention for achieving the above-described object is a method for testing an embedded device, comprising: building a virtualized execution environment in a subhost according to setting information about the execution environment of the target device; performing one or more test cases in the constructed execution environment; and monitoring the execution result, wherein performing the test case includes testing a multimedia playback operation of the target device, and the multimedia includes an audio signal.

In addition, a device according to an embodiment of the present invention for achieving the above-described object includes a processor and a memory that communicates with the processor, wherein the memory stores instructions that cause the processor to perform operations, The operations include: building a virtualized execution environment in a subhost according to setting information about the execution environment of the target device; An operation of performing one or more test cases in the constructed execution environment; and an operation of monitoring the performance result, wherein the operation of performing the test case includes an operation of testing a multimedia playback operation of the target device, and the multimedia includes an audio signal.

Meanwhile, a computer program stored in a computer-readable recording medium according to an embodiment of the present invention to achieve the above-described object may include program code for executing the above-described calculation method.

According to the present invention, resources can be utilized efficiently because there is no need to newly configure and prepare physical hardware resources every time during the development and testing of embedded devices.

Additionally, the present invention can greatly reduce initial and maintenance costs by eliminating the need to purchase actual hardware each time.

Additionally, since the present invention uses various preset virtualization environments, it can easily respond to various test cases and conditions.

Additionally, the present invention can shorten the time to physically prepare hardware and can quickly replicate or scale the required environment.

Additionally, the present invention can easily replicate the same test environment, thereby increasing the accuracy and reliability of testing.

In addition, the present invention can be easily automated and standardized, reducing human errors and maintaining consistency of test results.

Additionally, the present invention can centrally manage all virtualized environments from a centralized main host, greatly reducing complexity and making management easier.

Additionally, the present invention allows the main host to manage desired devices, environments, TCs, etc. for each user, allowing multiple developers or teams to work efficiently at the same time.

In addition, the present invention records the media processing performance of the embedded device and verifies it by comparing it with the original media to determine the results of the test remotely or automatically, and improves media processing test efficiency at the development stage. can be improved.

In addition, the present invention can monitor the performance status of media processing tests for each test device remotely and in real time, and based on this, the error detection performance of the test can be improved.

1 is an exemplary diagram conceptually showing the configuration of a testing system for a computing device according to an embodiment of the present invention.
Figure 2 is a flowchart showing the process of a testing method for an embedded device according to an embodiment of the present invention.
Figure 3 is a diagram illustrating the main host configuration of a computing device for testing according to an embodiment of the present invention.
4 to 5 are flowcharts showing detailed processes of a testing method for an embedded device according to an embodiment of the present invention.
6 to 8 are diagrams illustrating a subhost configuration of a computing device for testing according to an embodiment of the present invention.
Figure 9 is an exemplary diagram conceptually showing the configuration of a testing system for a computing device according to another embodiment of the present invention.
Figure 10 is a flowchart showing the process of a media processing test method for an embedded device according to an embodiment of the present invention.
11 to 13 are exemplary diagrams for explaining audio signal processing and verification methods according to an embodiment of the present invention.
Figure 14 is a flowchart showing an image verification process of a media processing test method for an embedded device according to an embodiment of the present invention.
Figure 15 is an example diagram for explaining an image signal processing and verification method according to an embodiment of the present invention.
Figure 16 is an exemplary diagram showing the hardware implementation of a computing device for testing according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Therefore, a person skilled in the art can invent various devices that embody the principles of the invention and are included in the concept and scope of the invention, although not clearly described or shown herein. In addition, all conditional terms and embodiments listed in this specification are, in principle, clearly intended only for the purpose of ensuring that the inventive concept is understood, and should be understood as not limiting to the specifically listed embodiments and states. .

The above-mentioned purpose, features and advantages will become clearer through the following detailed description in relation to the attached drawings, and accordingly, those skilled in the art in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

Additionally, when describing the invention, if it is determined that a detailed description of the known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is an exemplary diagram showing a test system with a hierarchical host structure according to the present invention.

Referring to FIG. 1, the system according to this embodiment includes a user 10 who inputs setting information for testing, a test device 100 that builds an environment according to the setting information of the user 10 and performs a test, and a test subject. It may be configured as a target device 200.

The user 10 may input test setting information using the interface provided by the test device 100.

At this time, the user 10 can remotely access the test device 100 and input setting information. In addition, the user 10 may be composed of a plurality of user groups, and each user 10 can remotely access the test device 100 and enter setting information for the target device 200 to be tested. The device 100 can manage the test schedules of users 10.

The test device 100 may be configured in a hierarchical structure of a main host 110 and a sub host 120.

The main host 110 manages all connected devices, environments, users 10, and TC (Test Case). The subhost 120 performs actual tests where each embedded target device 200 is connected.

When the user 10 selects a target device 200 and an environment to be tested on the main host 110, the corresponding environment is virtualized and set on the sub-host 120 device, and software can be installed.

The installed virtual environment 1250 is connected to the target device 200 for testing, so the device and environment desired by the user 10 can be variably constructed, and software or TC selected by the user 10 is performed.

The subhost 120 sets up various environments in terms of hardware or software in a virtualized environment and then installs them as needed to variably perform tests in various environments.

In this embodiment, the main host 110 and the sub-host 120 can be physically separated, and therefore, when the target device 200 is added later, the test device 100 is connected through logical combination with various sub-hosts 120. can be configured.

Hereinafter, a specific test method will be described with reference to FIG. 2.

Figure 2 is a flowchart showing a testing method according to an embodiment of the present invention.

The main host 110 receives setting information for the execution environment of the target device 200.

Setting information can be input by the user 10 through an interface provided by the test device 100.

The main host 110 provides various test environments to the user 10. Here, the user 10 can select the device to be tested, the software to be executed, and the environment in which they will operate.

The user 10 selects desired settings through a GUI (Graphical User Interface) or a text-based user interface CLI (Command Line Interface).

Figure 3 shows the configuration of the main host 110 according to an embodiment of the present invention.

The main host 110 for testing the target device 200 according to this embodiment may include various configurations.

For example, the web server 112 can provide a web-based interface that allows the user 10 to easily set up and manage the test environment, and the user 10 can remotely access and request testing. .

Additionally, the web server 112 may request external information or provide processing results to the device of the user 10 by providing an API endpoint so that external systems or services can interact with the host system.

In addition, the web server 112 supports transmitting data using the HTTP/HTTPS protocol between the user's client and the main host 110, and the user 10 transmits data to the main host 110 and the main host 110 ) allows you to perform tests by referring to the data.

Job Manager (114) manages when and on what device tests will be run, and can dynamically allocate computing resources and devices required for testing.

It also determines the priority of which of several test tasks should be executed first.

Specifically, referring to FIG. 4, the main host 110 schedules the test request of the user 10 (S10).

Next, the main host 110 allocates the sub host 120 when the target device 200 and execution environment for the user 10 become available according to the scheduled schedule (S20).

The device manager 116 manages the target devices 200 to be tested and monitors the current status of the devices in real time. Additionally, it is possible to select an appropriate device to run the test and allocate resources according to the job manager 114.

Log Manager 118 collects logs generated from device testing and requests or feedback from users 10 and can store log data for problem diagnosis or performance monitoring.

Referring to FIG. 5, in this embodiment, the main host 110 stores the test case and the revision history of the test case (S30). It is possible to reuse test cases through the saved history or check the performance history of the target device 200 through the history (S40).

Next, the main host 110 allows the previously built execution environment to be reused according to the history of the configuration information.

Furthermore, ML Perf (Machine Learning Performance) 119 measures the performance of the machine learning model when the target device 200 includes machine learning, analyzes performance data to find optimization points, or later hyperparameters using AutoML. so that it can be used for their control. Additionally, a report can be generated and provided by summarizing AutoML-based performance measurement results.

TC Manager (Test Case Manager) 115 manages the list and properties of test cases to be used for testing, and tracks and analyzes the execution results of each test case by test unit. Additionally, the change history of test cases is tracked and managed as versions.

TC Manager (Test Case Manager) (115) analyzes/learns accumulated normal/abnormal logs and results based on AI and can be used to identify abnormal conditions and causes during operation based on logs generated during future execution. It is also possible.

In other words, the above configurations can be linked to each other within the main host 110 to help efficiently test the target device 200 even in a complex test environment.

Next, the test device 100 builds a virtualized execution environment on the sub host 120 according to the setting information (S200).

In this embodiment, the system automatically sets the environment selected by the user 10 as a Docker container or a virtual machine (VM) on the subhost 120, and necessary software and libraries in the virtual environment can be installed.

In this embodiment, the system configures a virtual environment on the OS 105 for testing the target device 200 based on a VM (Virtual Machine) 109 using a hypervisor 103 and a guest OS 107 and Docker. Container 104-based configuration can be determined according to specific purposes and situations.

First, referring to FIG. 6, the system according to this embodiment can configure a VM (Virtual Machine)-based virtual environment 125.

Hypervisor 103 emulates hardware and allows running multiple independent virtual machines.

The guest OS (Operating System) 107 is an operating system that runs on a virtual machine and includes the entire operating system, allowing a high level of isolation and complex applications to be executed.

For example, when the contents of a test case require a high degree of isolation and testing in various OS environments is required, a VM-based configuration can be performed.

Additionally, if you need to test complex network topologies and interactions, you can implement the environment on a subhost using a virtual machine.

On the other hand, since virtual machines have high resource usage, slow start/end times, and high overhead, the main host 110 according to this embodiment can determine whether to configure a VM by referring to the schedule of the job manager 114.

Additionally, referring to FIG. 7, the system is also capable of performing test cases through a Docker-based virtual environment 125.

Docker Engine 102 runs and manages containerized applications.

The container 104 configures the application, its necessary libraries, settings, etc. into a single package to execute, and ensures rapid deployment and scalability.

Additionally, the system according to this embodiment can maintain the environment from development to distribution through Docker in the case of tests that require environmental consistency to be maintained.

However, isolation at the OS level may be weaker than that of virtual machines, and because the Linux kernel is shared, testing in various OS environments may be limited.

In other words, the system according to this embodiment can be configured based on a virtual machine when performing tests that require a high level of isolation and various OS environments, and conversely, when fast and efficient deployment and execution is desired, a Docker-based test environment can be configured. You can.

Therefore, the first threshold defining the isolation level set in the TC manager 115 of the main host and the second threshold defining the distribution level are managed for each test case, and the job manager 114 manages the assigned isolation level for each subhost. Subhosts can be determined and scheduling performed through the isolation level and deployment level of the test case.

Next, the test device 100 performs specific software or test cases in the built execution environment (S300).

Next, the test device 100 monitors the test case execution results of the subhost 120 (S400).

As described above, the log manager 118 collects logs generated from device testing, requests or feedback from the user 10, and stores log data for problem diagnosis or performance monitoring.

Furthermore, in this embodiment, the system may additionally configure an NPU or GPU driver 106 on Docker for machine learning-based testing.

Specifically, the system can install GPU drivers for GPUs attached to or attached to the subhost so that the container can use the main host's GPU resources, and assign the GPU to the container. Additionally, driver libraries and utilities are mounted on the container so that they can be used within the container.

At this time, ML Perf (119) analyzes performance data to find optimization points or adjust hyperparameters. Additionally, a report can be generated and provided by summarizing AutoML-based automated test results.

Furthermore, the TC manager 115 manages the list and properties of test cases to be used for testing, and tracks and analyzes the execution results of each test case by test unit. Additionally, the change history of test cases is tracked and managed as versions.

As described above, the main host 110 allows reuse of a previously built execution environment according to the history of configuration information.

Specifically, snapshots can be used to reuse virtual machines.

The specific state of the virtual machine can be saved as a snapshot and restored when necessary. Snapshots allow you to easily compare the status before and after testing, and enable restoration in the event of an error.

Alternatively, a virtual machine with specific settings and software installed is saved as a template, and if multiple virtual machines with the same settings are needed, they can be quickly created through templates.

Furthermore, it is also possible to create an identical environment by cloning a running virtual machine. In other words, a new environment can be created while maintaining the current state, so it can be used under dynamic test conditions.

On the other hand, as a way to reuse Docker, the state of the Docker container is saved as an image and a new container can be executed through the image.

Using images, quick environment configuration and distribution is possible, and usability can be improved by uploading images to the registry so that multiple people can easily access them.

Alternatively, you can write a script to create a Docker image in a Docker file and recreate the same environment through it. Unlike images, the environment is configured through code, so it has the advantage of clear environment settings and easy version management.

In addition, by storing data outside the container, the same data can be reused in multiple containers, data persistence is maintained, and data can be easily shared across multiple containers.

At this time, the isolation level and distribution level of the reused virtual machine or Docker container can be updated through the log manager 118, and the virtual environment can be determined when reused.

Additionally, the main host 110 authenticates the user 10, and in the step of receiving setting information of the execution environment, it is also possible to receive setting information according to the access rights of the authenticated user 10.

Figure 9 is an exemplary diagram conceptually showing the configuration of a testing system for a computing device according to another embodiment of the present invention.

Referring to FIG. 9, the test system according to an embodiment of the present invention includes each subhost 120 and a target device 200 to perform a media processing test case of a media device in the system described in FIG. 1. A media processing test unit 140 may be further provided to process a media playback operation test and monitor the processing results.

Here, the media processing test unit 140 obtains the original media signal for testing the media playback operation provided to the target device 200 from the main host 110 or the sub host 120, and obtains the playback result of the target device 200. The signal is acquired, and normal operation can be verified by comparing the original media signal and the playback result signal. To this end, the media processing test unit 140 may include a separate audio signal capture module and a video capture module, and may be further equipped with one or more cameras and microphones if necessary to obtain precise video and audio playback signals.

This test device 100 may be externally connected to the media processing test unit 140 or may include the media processing test unit 140 internally, and may monitor verification results, construct a test report, and provide it to the user. You can. To this end, the media processing test unit 140 may be connected wirelessly or wired to the main host 110, sub-host 120, and target device 200, and a signal transmission channel for this may be established in advance within the system. there is.

In particular, the media processing test unit 140 verifies the media playback operation test of the target device 200 according to the test case, and since the media may include an audio signal, the test verification involves noise according to the execution environment. A learning model in which signals are pre-trained based on an artificial neural network can be used.

More specifically, when performing a test case, the media processing test unit 140 may obtain a playback result of the original audio signal in the target device 200, and send the playback result to the target device 200. A noise signal can be derived by applying a pre-trained learning model based on an artificial neural network, along with a correspondingly set execution environment, and the playback result can be filtered using the noise signal.

Here, the noise signal may include at least one of white noise, pink noise, and brown noise included in the audio signal, and the media processing test unit 140 determines the playback result to derive the noise signal. Spectral conversion may be performed, and the spectrum converted playback result may be applied to the learning model along with execution environment information to obtain the white noise, pink noise, or brown noise signal predicted from the learning model.

For example, when performing a test case, the subhost 120 inputs a standard audio test signal (e.g., 1 kHz sine wave, voice file, etc.) to the target device 200, and the media processing test unit 140 targets the target device ( The playback result can be obtained by re-recording the speaker output of 200) with a microphone. This playback result may include noise resulting from hardware limitations of the device or algorithm-based errors, so the learning model receives the playback result for learning and the original audio signal for learning and creates a learning model that derives noise components resulting from the difference. It can be built.

For example, specific noise types include hardware noise, quantification noise, and algorithm-based distortion, and the learning model can predict the white noise, pink noise, or brown noise signal from these noises.

Here, white noise refers to noise evenly distributed throughout the frequency band, and because all frequency components have the same power, it may sound similar to noise to the ear.

Additionally, pink noise is a noise whose power decreases in proportion to 1/f as the frequency increases, and the power in the low-frequency region is more concentrated, resulting in a softer sound than white noise.

Meanwhile, brown noise is noise whose power decreases more rapidly in proportion to 1/f^2 as the frequency increases, and the power concentration phenomenon in the low-frequency region is more pronounced, giving a feeling similar to a crackling sound.

In addition, the media processing test unit 140 can obtain improved audio quality by performing filtering by adding the signal from which this noise component has been removed to the playback result, and by comparing the added filtering result with the original audio signal, normal operation is achieved. can be verified.

More specifically, for example, the media processing test unit 140 may obtain the filtered playback result by subtracting the obtained noise signal from the waveform of the playback result, and obtain the filtered playback result from the filtered playback result. Delay verification can be performed by extracting a verification area based on the start sound and end sound and comparing the length of the original audio signal with the length between the verification areas to verify whether a delay has occurred.

In addition, more specifically, for example, the media processing test unit 140 normalizes the filtered playback result and each waveform of the original audio signal within the same range, and based on the similarity between each normalized waveform, audio You can verify whether there is a difference in context.

Here, the media processing test unit 140 may extract pattern information for each section from each waveform or the spectrum conversion information of each waveform to verify whether there is a difference in the audio context, and each waveform of the pattern information for each section. A vector value according to the inter-cosine similarity calculation can be obtained, and a section in which the vector value is derived more than a threshold can be detected as an abnormal section.

Meanwhile, the media may further include a video signal, and the media processing test unit 140 uses the normal operation result corresponding to the audio signal to produce a video corresponding to the video section between the normally verified start sound and end sound. Further verification can be performed. This will be described in more detail later.

Figure 10 is a flowchart showing the process of a media processing test method for an embedded device according to an embodiment of the present invention.

Referring to FIG. 10, the system including the test device 100 first drives the media processing test unit 140 to command playback of test media including an audio signal between the target device 200 and the sub host 120. Deliver (S500).

Then, the media processing test unit 140 applies the media playback results of the target device to a pre-trained learning model to derive a noise signal (S550).

Thereafter, the media processing test unit 140 obtains a filtered playback result by subtracting the noise signal from the playback result (S600).

Then, the media processing test unit 140 extracts a verification area from the filtered playback result and normalizes it to within the same range as the original media signal of the test media (S650).

Thereafter, the media processing test unit 140 verifies the difference in media context based on the similarity between the waveform of the normalized verification area and the original media signal (S700).

In addition, the media processing test unit 140 monitors the performance results for each test case, and the monitoring information is used to store and report in the test device 100 (S750).

By performing this method, as described above, the system according to the embodiment of the present invention plays the sound source or image of the media on the device under test 200, compares the playback result with the original media, and You can verify and monitor whether playback operates normally on each device.

Here, the criteria for verification success or failure for each test case may be as follows.

- If the sound of the sound source or video is low or inaudible,

- If noise is included in the sound source or video during playback,

- If actual playback does not work properly (no screen or sound)

- If there is a delay or the speed is not normal and the playback is fast or slow.

In the case of these test cases, the success/failure of the operation must be judged by manually performing it and listening or watching it directly, but this has the problem of not being able to test multiple devices remotely, and the actual playback depending on the network status. It may be difficult to accurately judge the quality of video or audio, errors may occur due to lack of concentration or subjective judgment standards, and ambient noise or other sounds depending on each playback environment may be recorded together, so the same test is performed at the same time. There are also problems that prevent the process from proceeding.

Accordingly, the test device 100 according to an embodiment of the present invention can process scheduling so that media playback can be performed sequentially rather than simultaneously, depending on the characteristics of the media being tested. . For example, when performing a TC that must analyze the playback results of all target devices 200 for a TC that plays media requested by the device under test 200, the subhost to which the target device 200 is assigned At 120, processing to sequentially adjust the playback command execution order so that the playback order does not overlap may be performed.

In addition, the media processing test unit 140 plays a specific audio or video on the device under test 200 and monitors it to automatically determine whether the specific audio or video is played normally, at the playback start and end points of the specific audio or video. Sound sources of corresponding specific waveforms can be inserted into the original media. Accordingly, the media processing test unit 140 can accurately identify the area for comparing the recorded file and the original sound source using the time position of the specific waveform. For example, this specific waveform sound source is a high-frequency sound source that has a specific value for a certain period of time, and can be used in the form of a start sound or an end sound.

In addition, the media processing test unit 140 can accurately record or obtain the playback results of the target device 200 by using a specific signal or event synchronization corresponding to the start or end of a video or audio signal. In this case, there is no need for additional classification information within the media, but it is necessary to process the playback results taking into account that some errors may occur during synchronization.

Meanwhile, Figures 11 to 13 are exemplary diagrams for explaining audio signal processing and verification methods according to an embodiment of the present invention.

First, in order to accurately compare sound sources by the media processing test unit 140, there must be almost no difference between the sound source played and recorded in the device and the original sound source to enable accurate comparison. To this end, referring to FIGS. 11 to 13, the media processing test unit 140 uses an artificial neural network-based learning model to remove white/pink/brown noise mixed during recording from the recorded sound source to provide better sound quality. Comparisons can be performed.

As shown in FIG. 11, the White/Pink/Brown Noise generated by each target device 200 may be formed differently depending on its installation environment.

Accordingly, the media processing test unit 140 according to an embodiment of the present invention collects the white/pink/brown noise of the continuously occurring audio signal using a single background daemon processor and changes it into a spectrogram. Thus, a CNN-based learning model can be built.

The media processing test unit 140 according to an embodiment of the present invention uses known deep learning techniques such as CNN, RNN, DNN, and LSTM to generate noise that can be universally applied to various continuously occurring environments or conditions. By performing base learning, the noise value in the execution environment where specific equipment is installed can be continuously predicted, and performance can be improved by comparing it with the actual original media for learning and playback results for learning.

Afterwards, in order to remove the actual noise, the media processing test unit 140 may convert the spectrogram format of the noise obtained in response to the acquired playback result back into a wave pulse format.

Accordingly, as shown in the waveform shown in FIG. 12, the media processing test unit 140 removes noise that may be mixed in when recording the playback result before the test is later performed and compared with the original media sound source to verify it. can be removed, and the playback result sound source after removal can be compared with the original media sound source.

Meanwhile, referring to FIG. 13, the media processing test unit 140 compares each frame of the original media sound source and the noise-removed playback result sound source, and defines a section in which the cosine similarity is outside the preset normal range as an abnormal section. It can be determined by:

For example, the media processing test unit 140 may extract a verification section area based on the above-described start sound and end sound for the noise-removed playback result sound source and the original media sound source, respectively.

Additionally, the media processing test unit 140 can first determine whether a delay has occurred during playback by comparing the lengths of these two sound sources (delay test).

In addition, the media processing test unit 140 can process the waveforms of the two sound sources to be normalized within the same range through normalization, and compare the waveforms of the two audios to see how much they overlap or whether there are any nonexistent parts. Accordingly, a comparison of the contexts of the two audios can be performed (context test). For such context comparison, cosine similarity between waveforms can be used, and if the similarity is within a certain range, it can be judged as normal, and if it is outside a certain range, it can be judged as abnormal. If the rate or time of these abnormal sections is greater than a threshold or error range, the test may be judged to have failed.

Meanwhile, Figure 14 is a flowchart showing the video verification process of the media processing test method of an embedded device according to an embodiment of the present invention, and Figure 15 is an example for explaining the video signal processing and verification method according to an embodiment of the present invention. It is also a degree.

First, referring to FIG. 14, the media processing test unit 140 obtains media playback results for a multimedia video signal test case from the target device (S800).

Then, the media processing test unit 140 identifies a verification section from the media playback results of the target device and normalizes it to the same range as the original video signal (S850).

Thereafter, the media processing test unit 140 performs normal operation verification processing by performing at least one of waveform comparison and pixel change amount comparison between the normalized reproduced video signal and the original video signal (S900).

More specifically, the step of verifying normal operation of the video playback may include verifying by performing at least one of waveform comparison and pixel change amount comparison between the normalized playback video signal and the original video signal, Extracting a first semantic segment object from a normalized reproduced video signal, extracting a second semantic segment object from the normalized original video signal, and overlapping between the first semantic segment object and the second semantic segment object A step of extracting an Intersection over Union (IoU) representing the degree of each frame for each frame, and a step of verifying normal operation of the video playback according to the degree of maintenance of the IoU for each frame section can be performed.

More specifically, the media processing test unit 140 performs audio verification if audio verification is required in the case of media containing video, but the verification section is based on the start sound and end sound from the audio verification. can be extracted accurately.

However, in cases where only video and not audio are verified, the test video media may include a start point screen and an end point screen. For example, an entry point screen may be configured to display a red background and a special wildcard format pattern, and an end point screen may be configured to display a blue background and a special wildcard format pattern. The media processing test unit 140 can identify each screen, identify the start and end points of the image, and perform verification by accurately extracting only the verification section area.

In addition, the media processing test unit 140 can determine whether video playback is normal by comparing the video signal of the original media and the playback result extracted from the reproduced video signal.

First, the media processing test unit 140 can determine whether the file is played accurately and whether there is a delay by comparing the playback time of the file (delay test).

In addition, the media processing test unit 140 can convert the two images into waveform signals and match the color range through normalization, and then perform a first comparison by comparing the waveforms of the two files as with audio (waveform comparison test).

Thereafter, the media processing test unit 140 may compare the amount of change in each pixel for each frame. For example, the media processing test unit 140 may determine whether the same image is maintained by comparing changes in pixel values of the same image (pixel comparison test).

Lastly, referring to FIG. 15, the media processing test unit 140 performs semantic segmentation on the playback result video and the original media video in order to determine whether semantic content is maintained in the entire video. can be performed. In addition, the media processing test unit 140 calculates an Intersection over Union (IoU) indicating the degree of overlap for each extracted semantic segment object, and compares the IoU to determine to what extent the IoU for each frame is maintained as each frame progresses. Wow, by measuring the change in IoU, you can find the difference in meaning between the original media video and the resulting playback video. Here, the media processing test unit 140 configures a frame window of about 1 to 5 seconds based on the initial starting point, slides the frame window over time, and sets the frame with the highest IoU as the reference frame.

According to this configuration, the user can not only directly remotely monitor media playback of the target device 200 in real time, but also remotely control the media modules (camera, speaker, microphone, etc.) of the target device 200, During the test, you can directly inspect while playing audio or video, and if voice recognition or a specific sound source environment is required as an input during performance, you can conduct the test by speaking the voice directly or playing the necessary sound source in the distance. .

Accordingly, through the present invention, various media playback tests can be performed simultaneously on various devices, and it is also possible to prevent a situation where errors are accidentally missed when the verifier's concentration is low or the amount of change in the media is small, and a person's subjective judgment standard can be prevented. As a result, unclear verification criteria can be expressed numerically, making verification results objective.

Meanwhile, referring to FIG. 16, in some embodiments of the present invention, the service server 300 may be implemented in the form of a computing device. One or more of each module constituting the service server 300 is implemented on a general-purpose computing processor, and therefore includes a processor 388, input/output I/O 382, memory 384, and interface. It may include (386) and bus (, bus) (385). The processor 388, input/output device 382, memory 384, and/or interface 386 may be coupled to each other through a bus 385. The bus 385 corresponds to a path through which data moves.

Specifically, the processor 388 includes a Central Processing Unit (CPU), Micro Processor Unit (MPU), Micro Controller Unit (MCU), Graphic Processing Unit (GPU), microprocessor, digital signal processor, microcontroller, and application processor (AP). , application processor) and logic elements capable of performing similar functions.

The input/output I/O 382 may include at least one of a keypad, keyboard, touch screen, and display device. The memory device 384 may store data and/or programs.

The interface 386 may perform a function of transmitting data to or receiving data from a communication network. Interface 386 may be wired or wireless. For example, the interface 386 may include an antenna or a wired or wireless transceiver. The memory 384 is a volatile operating memory that improves the operation of the processor 388 and protects personal information, and may further include high-speed DRAM and/or SRAM.

Additionally, memory 384 stores programming and data configurations that provide the functionality of some or all of the modules described herein. For example, it may include logic to perform selected aspects of the learning method described above.

A program or application is loaded with a set of instructions including each operation for performing the above-described learning method stored in the memory 384 and allows the processor to perform each operation.

Various embodiments described herein may be implemented, for example, in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). In some cases, it may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. The described embodiments may be implemented as a control module itself.

According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented as a software application written in an appropriate programming language. The software code may be stored in a memory module and executed by a control module.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications, changes, and substitutions can be made by those skilled in the art without departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments and the attached drawings. . The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

Claims (15)

In a test method for an embedded device,
Building a virtualized execution environment on a subhost according to setting information about the execution environment of the target device;
performing one or more test cases in the constructed execution environment; and
Including monitoring the performance results,
The steps for performing the test case are:
testing media playback operation of the target device, wherein the media includes an audio signal,
The testing step is,
Obtaining a playback result on the target device for the audio signal;
applying the playback result to a pre-trained learning model based on an artificial neural network corresponding to the execution environment, thereby deriving a noise signal;
filtering the playback result using the noise signal; and
Verifying normal operation by comparing the original audio signal corresponding to the audio signal and the filtered playback result,
The verification step is,
extracting a verification area based on the start sound and end sound from the filtered playback result; and
Comprising the step of verifying whether a delay occurs by comparing the length of the original audio signal and the length between the verification areas.
Test methods for embedded devices. delete According to paragraph 1,
The noise signal includes at least one of white noise, pink noise, and brown noise included in the audio signal.
Test methods for embedded devices. According to paragraph 3,
The step of deriving the noise signal is,
performing spectral conversion of the reproduction result; and
Comprising the step of applying the spectrum-converted reproduction result to the learning model to obtain the white noise, the pink noise, or the brown noise signal predicted from the learning model;
Test methods for embedded devices. According to paragraph 4,
The filtering step is,
Comprising the step of subtracting the obtained noise signal from the waveform of the reproduction result to obtain the filtered reproduction result.
Test methods for embedded devices. delete According to paragraph 1,
The verification step is,
normalizing the filtered reproduction result and each waveform of the original audio signal within the same range; and
Comprising the step of verifying whether there is a difference in audio context based on the similarity between each normalized waveform.
Test methods for embedded devices. In clause 7,
The step of verifying whether the audio context is different is,
extracting pattern information for each section from each waveform or spectral conversion information of each waveform;
Obtaining a vector value according to cosine similarity calculation between each waveform of the pattern information for each section; and
Including the step of detecting a section in which the vector value is derived above a threshold as an abnormal section.
Test methods for embedded devices. According to paragraph 1,
The media further includes a video signal,
The testing step is,
Using the normal operation result corresponding to the audio signal, further performing video verification corresponding to the video section between the normally verified start sound and end sound.
Test methods for embedded devices. According to paragraph 1,
The media further includes a video signal,
The testing step is,
Obtaining a playback result on the target device for the video signal;
identifying a verification section from the playback results;
Normalizing the reproduced video signal of the verification section and the original video signal to the same range; and
Comprising the step of verifying normal operation of video playback by comparing the normalized playback video signal with the original video signal.
Test methods for embedded devices. According to clause 10,
The steps to verify the normal operation of video playback are:
Comprising the step of verifying by performing at least one of waveform comparison and pixel change amount comparison between the normalized reproduced video signal and the original video signal.
Test methods for embedded devices. According to clause 10,
The steps to verify the normal operation of video playback are:
extracting a first semantic segment object from the normalized reproduction video signal;
extracting a second semantic segment object from the normalized original video signal; and
Extracting for each frame an Intersection over Union (IoU) indicating the degree of overlap between the first semantic segment object and the second semantic segment object; and
Comprising the step of verifying normal operation of the video playback according to the degree of maintenance of the IoU for each frame section.
Test methods for embedded devices. processor, and
comprising a memory in communication with the processor,
The memory stores instructions that cause the processor to perform operations,
The above operations are:
In a test method for an embedded device,
An operation to build a virtualized execution environment on a subhost according to setting information about the execution environment of the target device;
An operation of performing one or more test cases in the constructed execution environment, and
Including the operation of monitoring the performance results,
The operation of performing the test case is,
An operation of testing media playback operation of the target device, wherein the media includes an audio signal,
The test operation is,
An operation of obtaining a playback result on the target device for the audio signal,
An operation of deriving a noise signal by applying the playback result to a learning model pre-trained based on an artificial neural network in response to the execution environment,
Filtering the playback result using the noise signal, and
Comprising an operation of verifying normal operation by comparing an original audio signal corresponding to the audio signal and the filtered playback result,
The verification operation is,
An operation of extracting a verification area based on the start sound and end sound from the filtered playback result, and
Comprising the operation of verifying whether a delay occurs by comparing the length of the original audio signal and the length between the verification areas.
Computing device. delete According to clause 13,
The media further includes a video signal,
The test operation is,
Obtaining a playback result on the target device for the video signal;
Identifying a verification section from a reproduction result of the video signal;
Normalizing the reproduced video signal of the verification section and the original video signal to the same range; and
Comprising the operation of verifying normal operation of video playback by comparing the normalized playback video signal with the original video signal.
Computing device.