TWI755020B - Application program monitor and analysis system - Google Patents

Abstract

The invention discloses an application program monitor and analysis system which contains a record module, an image processing module, a model establishing module and a user behavior management module. The record module may record a smart phone’s monitor, so as to obtain a plurality of monitor images. The image processing module and the model establishing module are installed on a monitor server device. The image processing modulemay generate a training dataset based on the plurality of monitor images. The model establishing module may include a PCA model and a GAN model. The former may receive the training dataset to output a vector, and the latter may find out user program classes of the plurality of monitor images by inputting the vector. The user behavior management module may include the model establishing module and it may install on the smart phone. And, the user behavior management modulemay generate an application management data according to the user program class and a user operating behavior.

Description

Application Monitoring Analysis System

The following description is about an application monitoring and analysis system, especially a monitoring and analysis system that uses artificial intelligence algorithms to collect the behavior of users on mobile devices, and then determine the type of applications used by users.

In the 21st century when electronic technology products are becoming more and more popular, the applications in smartphones and tablets are constantly being updated, which can include document software, browser software, e-book applications, shopping mall software or e-sports games. Also due to the popularization of apps and mobile phones, we can see "head bowers" anytime and anywhere in public places such as MRTs and buses, which means users who are immersed in the world of smartphones.

It is worth mentioning that the diseases of civilization brought about by such technological products are also being paid more and more attention. Further, the users of these smart phones may unconsciously suffer from diseases related to mobile phones, such as "smart phone compulsion". Sexual examination” or “smartphone ghost hallucination syndrome”, the former symptom of which is the user’s near-compulsive use of the smartphone’s explicit behavior, and the latter’s symptom is that the user will feel the phone vibrate or hear the phone Ringing, picked up the smartphone to check but found no hallucinatory experience of any incoming calls.

However, in today's social environment, the Internet and mobile phones have become almost indispensable tools, so how to treat the disease of mobile phone addiction is becoming more and more important. Generally speaking, due to the Indispensably, patients with mobile phone addiction cannot be treated in the way of "quitting mobile phone use", but instead by "adjusting" and "controlling" the mode of mobile phone use. Since the user's problem is in the usage habits of mobile phone applications, recording which programs users use, the time spent using these programs, and the degree of addiction will be an important key to the treatment of mobile phone addiction. trend. If daily records and analysis and comparison are carried out in the traditional way, not only is it difficult to carry out in practice, but it will also cause great inconvenience to users. Therefore, the use of mobile phone applications for automatic recording will be imperative.

Furthermore, the above-mentioned disease of curing "mobile phone addiction" will face the biggest problem, that is, how to identify the type of programs currently used by users (that is, the applications that cause users to become addicted), and carry out related work. Record, because the general application can only record the total usage time of the user's mobile phone, but it cannot record according to the program type. For example, the current application can only count the user's mobile phone usage information within 24 hours, or the mobile phone usage information of a single application, but it cannot identify multiple video games installed on the mobile phone (that is, the same category) related usage information.

In view of the foregoing, the inventors of the present invention have considered and designed an application monitoring and analysis system, in order to improve the deficiencies of the prior art, thereby enhancing the implementation and utilization in the industry.

Based on the above purpose, the present invention provides an application monitoring and analysis system, which is suitable for an intelligent electronic device and a monitoring servo device, and includes a side recorder module, an image processing module, and a model building module and a user behavior management module. The skimming program module is installed in the intelligent electronic device, and the skimming program module can skimming the screen of the intelligent electronic device to obtain a plurality of screen skimming images. The image processing module and the model building module are installed on the monitoring servo device, and the image processing module calculates the plurality of screen skimming images to generate a training sample data. mold The model building module includes a Generative Adversarial Network (GAN, Generative Adversarial Network) model and a Principal Component Analysis (PCA, Principal Component Analysis) model, which can receive the training sample data to output a PCA feature A vector to which the Generative Adversarial Network model can input the PCA feature vector to identify a user program category into which the plurality of screen skimming images are classified. The user behavior management module includes the model building module and can be installed on the intelligent electronic device. Create an application to manage data.

Preferably, the generative adversarial network model includes a GAN generator and a GAN discriminator, the GAN generator is based on the PCA feature vector to generate a simulated image that approximates one of the screen skimming images, and the GAN discriminator is According to the simulated image and the plurality of screen skimming images for comparison and identification, the GAN generator continuously deceives the GAN discriminator, and the GAN discriminator continuously identifies the authenticity of the simulated image through The balance of the two against each other and the constant adjustment of the respective network parameters can produce the learning results of the custom model, which in turn generates the class of user programs.

Preferably, the image processing module calculates the plurality of screen skimming images according to a CIE-Lab color space to generate the training sample data.

Preferably, the user behavior management module downloads and installs the model building module from the monitoring server device.

1: Application monitoring analysis system

10:Side recorder module

11: Screen recording video

20: Monitor the servos

21: Image processing module

211: Training sample data

22: Model building module

221: Generative Adversarial Network Models

2211: GAN Generator

11': Simulated image

2212: GAN Discriminator

2221: PCA eigenvectors

222: Principal Component Analysis Models

223: User program class

30: User behavior management module

31: User behavior

32: Application Management Data

A: Smart Electronic Device

FIG. 1 is a block diagram of the application monitoring and analysis system of the present invention.

FIG. 2 is a first schematic diagram of the image processing of the application monitoring and analysis system of the present invention.

Figures 3A to 3C are the second schematic diagrams of the image processing of the application monitoring and analysis system of the present invention.

FIG. 4 is a flow chart of the system operation of the application monitoring and analysis system of the present invention.

In order to facilitate the examiners to understand the features, contents and advantages of the present invention, as well as the effects that can be achieved, the present invention is hereby described in detail as follows in the form of an embodiment in conjunction with the accompanying drawings. The subject matter is only for illustration and auxiliary description, and may not necessarily be the real scale and precise configuration after the implementation of the present invention. Therefore, the proportion and configuration relationship of the attached drawings should not be interpreted or limited to the scope of rights of the present invention in actual implementation.

The advantages, features, and technical means of achieving the present invention will be more easily understood by being described in more detail with reference to the exemplary embodiments and the accompanying drawings, and the present invention may be implemented in different forms, so it should not be construed as being limited to what is described herein. Rather, the embodiments are provided so that this disclosure will be thorough, complete and complete to convey the scope of the invention to those of ordinary skill in the art, and the invention will only be appended Defined by the scope of the patent application.

Please refer to FIG. 1 , which is a block diagram of the application monitoring and analysis system of the present invention. As shown in the figure, the application monitoring and analysis system of the present invention is suitable for an intelligent electronic device A and a monitoring server 20, which may include a side recorder module 10, an image processing module 21, a model The building module 22 and a user behavior management module 30 are created. of which this The smart electronic device A can be a smart phone or a tablet computer, and the monitoring server device 20 can be a server, a workstation, a desktop computer or a notebook computer.

In this embodiment, the skimming program module 10 can be a mobile phone application or a tablet computer application, which can be installed on the smart electronic device A, and is used for skimming the screen on the smart electronic device A for Obtain a plurality of screen skimming images 11 . More specifically, the screen skimming image 11 includes a screenshot image when the smart electronic device A executes at least one application program.

The monitoring server 20 includes an image processing module 21 and a model building module 22, wherein the image processing module 21 and the model building module 22 are both software or application programs executable on a computer. Wherein, the image processing module 21 can calculate the plurality of screen skimming images 11 to generate a training sample data 211 , the schematic diagram of which is shown in FIG. 2 .

In a preferred embodiment, the image processing module 21 can calculate a plurality of screen skimming images 11 according to a CIE-Lab color space to generate the training sample data 211, wherein the images generated through the CIELAB color space are The visual difference of the training sample data 211 can be more linear and approach the visual perception result, and it can also avoid the problem of light and dark interference that affects the image during skimming when different smart electronic devices A are used for skimming. content. For example, when using different smartphones and skimming the screen of the same application, the recorded images may be different due to the brightness of the mobile phone screen or the current environment. The schematic diagrams are shown in Figures 3A and 3B. As shown, when the CIE-Lab color space is used to process the image on the screen skimming image 11, the interference problem of the image can be greatly reduced, and the schematic diagram is shown in FIG. 3C.

The model building module 22 may include a Generative Adversarial Network (GAN, Generative Adversarial Network) model 221 and a Principal Component Analysis (PCA, Principal Component Analysis) model 222 . The principal component analysis model 222 receives the training sample data 211 after image processing from the image processing module 21 , and then outputs a PCA feature vector 2221 . Next, the generative adversarial network model 221 can determine the user program category 223 classified by the plurality of screen skimming images 11 according to the PCA feature vector 2221 , and the user program category 223 can be established according to the model. 22. The self-defined model trained to determine the category of the application currently being executed by the user belongs to which category, the application can include e-sports programs, document software, browser programs, map software, e-book applications, etc. one. It should be understood that, the scope of the above-mentioned application program is only illustrative, but not limited thereto.

The user behavior management module 30 can be an application program including the model building module 22, which can be downloaded from the monitoring server device 20 and installed on the intelligent electronic device A. The user behavior management module 30 can be An application program management data 32 is generated according to the user program type 223 and a user usage behavior 31, wherein the user usage behavior 31 may include the user's operation time length, operation frequency, and the light intensity of the operating environment Or the user's posture when holding the intelligent electronic device A.

More specifically, after the user installs the user behavior management module 30, the model building module 22 included in it will interpret the type of the application program on the smart electronic device A executed by the user, and then use Act 31 to generate application management data 32 to give the user the most appropriate advice. For example, when the model building module 22 determines that it is an e-sports application, the user management module 30 can operate the Give users an appropriate suggestion or reminder based on the length of time and the frequency of operation of e-sports applications.

In the embodiment of the present invention, the generative adversarial network model 221 may include a GAN generator 2211 and a GAN discriminator 2212, wherein the GAN generator 2211 can generate a simulated image 11' according to the PCA feature vector 2221, The GAN discriminator 2212 can compare and discriminate according to the simulated image 11 ′ and the plurality of screen recording images 11 . In other words, the GAN generator 2211 is responsible for simulating the production of data similar to the screen skimming image 11, and is sent to the GAN discriminator 2212 for comparison with the real data (ie, the screen skimming image 11), and the GAN generator 2211 tries its best to deceive The GAN discriminator 2212 and the GAN discriminator 2212 try their best to identify true and false data. The balance between the two confronting each other and continuously adjusting their respective network parameters can produce the learning results of the self-defined model, and then generate a user program. Category 223.

It is worth mentioning that the generative neural network in the traditional generative adversarial network model (ie, the GAN generator corresponding to the present invention) mainly outputs results by randomly selecting potential variables, and the output results must be as close as possible to the training set data. The advantage is that it does not need to collect enough training data to train the AI model, but can use the generative neural network to generate a large amount of simulated data/fake data. Different from the prior art, the GAN generator 2211 of the present invention does not use random selection of latent variables, but uses the PCA feature vector 2221 generated by the principal component analysis model 222. The reason is that the simulation data of the traditional generative neural network is generated based on data However, such simulation data may affect the discrimination problem of accuracy, and cannot meet the basic requirements of data richness when training AI models. Therefore, the present invention improves the simulation data part of the generative adversarial network model, and adopts the principal component analysis model 222 to make the simulation data more representative.

FIG. 4 is a flow chart of the system operation of the application monitoring and analysis system of the present invention. As shown in the figure, the present invention can use a side recorder module to record the data of the user's smart mobile device (touch device) screen operation and use screen, and then upload it to the remote remote computer through 3G/4G/WiFi. A set of multiple AI workstations is received by a monitoring server. In this embodiment, a Web Server is used as an example. At this time, the Web Server will generate a user behavior management module and send it to the user's touch device for installation. At this time, the model building module in the user behavior management module establishes a custom model through continuous training to determine the user program type. After the user program type is identified, the user behavior management A module to make self-management suggestions or alerts when the user runs the application.

The above-mentioned embodiments are only to illustrate the technical ideas and characteristics of the present invention, and the purpose is to enable those who are familiar with the art to understand the content of the present invention and implement it accordingly. It should not be used to limit the patent scope of the present invention. That is, all equivalent changes or modifications made according to the spirit disclosed in the present invention should still be covered within the patent scope of the present invention.

1: Application monitoring analysis system

10:Side recorder module

11: Screen recording video

20: Monitor the servos

21: Image processing module

211: Training sample data

22: Model building module

221: Generative Adversarial Network Models

2211: GAN Generator

11': Simulated image

2212: GAN Discriminator

2221: PCA eigenvectors

222: Principal Component Analysis Models

223: User program class

30: User behavior management module

31: User behavior

32: Application Management Data

A: Smart Electronic Device

Claims (4)

An application monitoring and analysis system, which is suitable for an intelligent electronic device and a monitoring server device, comprises: a side recorder module, which is installed on the intelligent electronic device, and the side recorder module records the a screen of an intelligent electronic device to obtain a plurality of screen skimming images; an image processing module is installed in the monitoring server device, the image processing module calculates the plurality of screen skimming images to generate a training sample data; a A model building module is installed on the monitoring servo device. The model building module includes a generative adversarial network (GAN, Generative Adversarial Network) model and a principal component analysis (PCA, Principal Component Analysis) model. The analysis model receives the training sample data to output a PCA feature vector, and the generative adversarial network model inputs the PCA feature vector to determine a user program category to which the plurality of screen skimming images are classified; and a user The behavior management module includes the model building module and is installed on the smart electronic device. The user behavior management module generates an application program management data according to the user program type and a user usage behavior , wherein the user's use behavior includes the operation time length, the operation frequency, the light intensity of the operation environment, or the user's posture when holding the intelligent electronic device. The application monitoring and analysis system as described in claim 1, wherein the generative adversarial network model comprises a GAN generator and a GAN discriminator, and the GAN generator generates approximations of the screens according to the PCA feature vector A simulated image of the profiled image, the GAN discriminator is based on the simulated image and the complex The GAN generator continuously deceives the GAN discriminator, and the GAN discriminator continuously identifies the real and fake images of the simulated images by confronting each other and constantly adjusting. Balancing the respective network parameters produces the learning outcomes of the custom model, which in turn generates the class of user programs. The application monitoring and analysis system as described in claim 1, wherein the image processing module calculates the plurality of screen skimming images according to a CIE-Lab color space to generate the training sample data. The application monitoring and analysis system as described in claim 1, wherein the user behavior management module is downloaded from the monitoring server device and the model building module is installed.

Images