KR102169580B1 - System and method for detecting abnormal display screen - Google Patents

Abstract

The present invention is a method for detecting an abnormal display screen performed in a terminal, the method comprising: detecting an edge region in a terminal in which a game service is being used; Detecting a specific area within the detected edge area; And transmitting the terminal-related information to a reporting server when it is determined that the number of the detected specific regions is greater than or equal to a preset number.

Description

Abnormal display screen detection method and system {SYSTEM AND METHOD FOR DETECTING ABNORMAL DISPLAY SCREEN}

The present invention relates to a method and system for detecting an abnormal display screen, and more particularly, to a method and system for detecting and reporting an abnormal output of a screen displayed in a terminal playing a game.

With the rapid spread of portable terminals and expansion of network networks, it is possible to use high-spec game services using the Internet anytime, anywhere. In particular, as high-performance portable terminals capable of supporting 3D rendering technology have recently been developed, game contents that were previously only playable in some high-performance PC environments can be played in a mobile environment.

Game developers are also developing content that can realize more colorful and impactful directing effects through such hardware support. As a game service user, you can experience a sense of immersion in the game through realistic graphics.

However, as portable terminals are manufactured by many manufacturers, their types have become diverse, and it is difficult for game developers to satisfy all of the various portable terminal specifications, and as the degree of implementation of contents varies depending on the network environment, There are cases in which the screen of the game content is displayed abnormally.

In the case of the conventional abnormal display detection method, it is only the level of analyzing a recorded crash or reporting log information about a test failure to a server along with the cause. Therefore, in the case of such a phenomenon, it is output as a normal case on the game play log, but there is a problem that it is difficult for game developers to report it because the screen cracking phenomenon occurs in the actual game service user's portable terminal. Although it is determined that there is no presence, it can act as a factor that hinders the user's immersion in the game.

Therefore, research on a method of detecting whether a game play screen is displayed abnormally in a portable terminal is required.

The present invention is to solve the problems of the prior art, and it is possible to detect an abnormally output screen by analyzing a screen displayed on a portable terminal.

In addition, as a game developer, by detecting an abnormally displayed screen and receiving a report, an error that cannot be detected by the game log can be promptly corrected.

An embodiment of the present invention is a method for detecting an abnormal display screen performed in a terminal, the method comprising: detecting an edge region in a terminal in which a game service is being used; Detecting a specific area within the detected edge area; And transmitting the terminal-related information to a reporting server when it is determined that the number of the detected specific regions is greater than or equal to a preset number.

In addition, the specific area within the detected edge area is a rectangular area displayed in black in the terminal.

In addition, the terminal-related information is device information of the terminal, and includes at least one of manufacturer information, model name information, OS information, communication method information, battery usage information, and remaining memory information of the terminal.

In addition, the terminal-related information is log information of a game being serviced by the terminal, and includes at least one of game level information, experience value information, play time information, and purchase record information.

Another embodiment of the present invention is a method of detecting an abnormal display screen performed in a terminal, the method comprising: resizing an image displayed on a terminal in which a game service is being used; Detecting an edge region in the resized image; Detecting a specific area within the detected edge area; And transmitting the terminal-related information to a reporting server when it is determined that the number of the detected specific regions is greater than or equal to a preset number.

In addition, the resizing of the image may include resizing to an image lower than a screen resolution at which the image is displayed.

Another embodiment of the present invention is an abnormal display screen detection system, comprising: an edge area detection unit for detecting an edge area in a terminal in which a game service is being used; A specific area detection unit for detecting a specific area within the edge area detected by the edge area detection unit; An abnormal display determination unit configured to determine that the terminal is abnormally displayed when the number of the detected specific regions is determined to be greater than or equal to a preset number; And a communication unit transmitting the terminal-related information to a reporting server when it is determined that the abnormal display is being performed.

In addition, the specific area detection unit may detect a rectangular area displayed in black in the terminal.

The present invention relates to an abnormal display screen detection method and system, and has an advantageous effect of detecting an abnormally output screen by analyzing a screen displayed on a portable terminal.

In addition, as a game developer, by detecting an abnormally displayed screen and receiving a report, there is an advantageous effect of promptly correcting an error that cannot be detected by the game log.

That is, there is no crash record in the game server, but there is an advantageous effect of being able to detect and report a situation caused by an error of a graphic component.

1 is a diagram conceptually showing a game service system according to an example of the present invention.
2 is a flowchart illustrating a process of detecting an abnormal display according to an example of the present invention according to the sequence of time.
3 is a flowchart illustrating a process of detecting an abnormal display according to another example of the present invention according to the sequence of time.
4 is a block diagram conceptually showing the configuration of a game terminal according to an example of the present invention.
5 is a diagram illustrating a normal display screen and an abnormal display screen according to an embodiment of the present invention.
6 is a diagram illustrating a method of detecting an edge region on a display screen of a terminal according to an exemplary embodiment of the present invention.
7 is a diagram for describing a method of detecting a specific area in an edge area according to an example of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware. Meanwhile,'~ unit' is not meant to be limited to software or hardware, and'~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. The components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units', or may be further divided into additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

The "user terminal" mentioned below may be implemented as a computer or portable terminal that can access a server or other terminal through a network. Here, the computer includes, for example, a notebook equipped with a web browser, a desktop, a laptop, and the like, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), LTE (Long Term Evolution) communication-based terminal, smart It may include all kinds of handheld-based wireless communication devices such as phones and tablet PCs. In addition, the “network” refers to a wired network such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), or a mobile radio communication network or satellite. It can be implemented in any type of wireless network such as a communication network.

1 is a diagram conceptually showing a game service system according to an example of the present invention.

As shown in FIG. 1, in the game service system, a game server 100 and game terminals 210, 220, and 230 are connected via a network.

The game server 100 provides game content to the game terminals 210, 220, and 230 so that the game terminals 210, 220 and 230 can use the game service, and the game terminals 210, 220 and 230 Game content is received from the game server 100 and a game service is used.

The game service system may further include a reporting server 300 in addition to the game server 100 and game terminals 210, 220, and 230. The reporting server 300 may check whether the game is normally being serviced by receiving information on game content used in the game terminal. The reporting server 300 may exist separately from the game server 100, or may be understood as being distinguished from the game server 100 in terms of its functionality.

The game server 100 may receive and analyze game log information whether a game is normally played in the game terminals 210, 220, and 230. However, in the case of analysis using game log information, although it is determined to be normal on the log information, there is a problem in that the screen displayed abnormally in the actual game terminals 210, 220 and 230 cannot be analyzed.

Accordingly, in order to detect abnormal display screens in the game terminals 210, 220 and 230, the screens of the game terminals 210, 220 and 230 may be analyzed by the game terminals 210, 220 and 230. It is also possible to transmit the screen information of the game terminals 210, 220, and 230 itself to the game server 100 or the reporting server 300 and analyze it in the game server 100 or the reporting server 300, but the transmitted and received data And since it may be inefficient in consideration of time and the like, a description will be made of a method of detecting whether the game terminal 210, 220, and 230 is displayed abnormally by analyzing the game screen in the game terminals 210, 220 and 230.

2 is a flowchart illustrating a process of detecting an abnormal display according to an example of the present invention according to the sequence of time.

As shown in FIG. 2, when a game terminal is using a game service, an edge region may be detected in a game image (S210). The edge area in the present invention means an area of a certain size displayed in a certain color in the game screen. That is, it means a single color without gradation and an area of a certain size or more due to the nature of the game graphic component error.

The method of detecting the edge area on the game screen may be various. In general, Laplacian filtering may be used as an edge detection image processing technique.

In addition, a Canny edge detector method may be used as a technique for detecting the edge region. In the case of the canny edge detection method, an edge is detected using an edge detection algorithm after converting an image to a gray image.

In the present invention, the above two methods have been described as an example for convenience of description, but it is preferable to broadly understand that various methods of detecting an edge region in an image can be applied.

The terminal may detect a specific area within the detected edge area. In the present invention, the specific area is an area in the form of a sliding window having a predetermined square size (Width * Height) or more, and may be a single color area such as black.

The terminal may selectively determine and detect the size of a specific area according to the image size. For example, after detecting the detected edge area as a specific area of a large size, it is detected as a specific area of a smaller size, and it is possible to determine whether the number of specific areas is greater than or equal to a preset threshold. Yes (S230).

When it is determined that the number of specific regions is greater than or equal to a preset number, the terminal may determine that the current screen is displayed abnormally. The terminal may transmit terminal-related information to the reporting server along with information that the screen is displayed abnormally (S240).

In another case, when the number of specific areas is less than the preset number, the terminal may determine that it is normally displayed.

When it is determined that the screen is displayed abnormally, the terminal may transmit information related to the terminal to the reporting server.

In the present invention, terminal-related information can be largely divided into two types.

The terminal-related information may be game terminal information, that is, device information. In this case, this is to analyze the cause information of the abnormal display through hardware/software analysis of a specific terminal.

The device information may include manufacturer information, model name information, operating system (OS) information, communication method information, battery usage information, remaining memory information, and the like.

In the case of manufacturer information, there may be differences in HW/SW according to the manufacturing policy or method of the manufacturer that manufactured the game terminal, and it is traced back during the game verification process to determine whether a problem occurs only in the game terminal of a specific manufacturer. You can use it as a basis.

In the case of manufacturer information, there may be differences in HW/SW according to the manufacturing policy or method of the manufacturer that manufactured the game terminal, and it is traced back during the game verification process to determine whether a problem occurs only in the game terminal of a specific manufacturer. You can use it as a basis.

In the case of model name information, even for the same manufacturer, since manufacturing standards and manufacturing parts information are different for each model, it can be used as a basis for determining the cause of abnormal display.

In the case of the operating system (OS), since the supported standards are different for each version, and game contents may be different for each OS, OS information can also be used as a basis for determining the cause of abnormal display.

In the case of communication method, communication speed and standard are different depending on whether game service is performed through a cellular (LTE, 3~5G) network or a game service is performed through a short-range wireless network (Wi-Fi, tethering, etc.). It can also be used as a basis for determining the cause of abnormal display.

Even in the case of battery usage, if the amount of remaining battery in the terminal is below a certain level, the terminal itself can implement a different performance mode, and thus battery usage information can also be used as a basis for determining the cause of abnormal display.

The terminal-related information in the present invention may be game log information. Game log information can also be understood as terminal-related information because it is operated in the corresponding game terminal.

The game log information includes information that can be recorded while using the game service, such as the level of the user, the level of the character, the level of the team, the experience of the character, the play time, the accumulated play time, and purchase record information.

The game log information is also information that can be analyzed whether it is a hardware problem of the game terminal or a content problem provided by the game server.When a problem occurs in the game content, log information about the operation of the game user is converted into big data. It can be collected and used as a basis for determining abnormal play or abnormal display of the game.

3 is a flowchart illustrating a process of detecting an abnormal display according to another example of the present invention according to the sequence of time.

As shown in FIG. 3, as a step before the terminal detects the edge region, the image displayed on the terminal may be resized (S310).

The step of resizing an image displayed on the terminal may be selectively implemented, and the image may be downsized to reduce the amount of computation in the terminal. However, resizing does not necessarily mean image downsizing, and it is also possible to upsize the image to detect abnormal displays with higher accuracy.

Since the processes after resizing (S320 to S360) are duplicated with those described in FIG. 2, they will be omitted.

4 is a block diagram conceptually showing the configuration of a game terminal according to an example of the present invention.

As shown in FIG. 4, the game terminal 200 may include an edge region detection unit 201, a specific region detection unit 202, an abnormal display determination unit 203, and a communication unit 204.

The edge area detection unit 201 in the present invention detects an edge area on a screen in which a game service is being used. The edge region detector may detect the edge region using an algorithm such as a Laplacian filtering method and a canny edge detection method.

In the present invention, the specific region detection unit 202 detects a specific region within the edge region detected by the edge region detection unit 201. The specific area detection unit is an area in the form of a sliding window having a predetermined square (Width * Height) or more, and may be a single color area such as black.

The specific area detection unit 202 may selectively determine and detect the size of a specific area according to the image size. For example, after detecting the detected edge area as a specific area of a large size, it is detected as a specific area of a smaller size, and it is possible to determine whether the number of specific areas is greater than or equal to a preset threshold. have.

In the present invention, when the number of detected specific regions is determined to be greater than or equal to a preset number, the abnormal display determination unit 203 determines that the terminal is displaying abnormally.

That is, the specific area is an abnormal display area, and when the area being abnormally displayed is larger than a certain size, it may be determined that the terminal is abnormally displayed.

The communication unit 204 in the present invention transmits terminal-related information to the reporting server. When it is determined by the abnormal display determination unit 203 that the terminal is displaying abnormally, the terminal-related information together with the abnormal screen information may be transmitted to the reporting server. If it is being displayed normally, it can be transmitted to the reporting server that it is being displayed normally.

The communication unit 204 receives content provided by the game server, and transmits play information from the game terminal, that is, game log information, to the game server. In the present invention, the game server and the reporting server may exist separately, and may be servers classified according to functions.

The reporting server and the game server that have received the abnormal display information and the terminal-related information may analyze various information to track the cause of the abnormal display.

The error detection unit on the reporting server may compare and analyze whether it is an error of a specific manufacturer's terminal, a specific model's terminal, an OS problem, a communication method problem, a battery usage problem, or a residual memory problem.

The error detection unit on the reporting server can accumulate and learn the detected error information as a database (DB) as big data, and the cause of the error can be determined by comparing and analyzing the newly detected error with the learned data.

In addition, the error detection unit on the reporting server may learn a method of detecting the abnormal display on the log by matching game log information at the time of abnormal display and analyzing the user's play log in case of abnormal display.

5 is a diagram illustrating a normal display screen and an abnormal display screen according to an embodiment of the present invention.

The image shown in FIG. 5(a) is a diagram showing a case where game content is normally displayed on a terminal, and the image shown in FIG. 5(b) is a diagram showing a case where the game content is displayed abnormally.

As shown in Fig. 5(b), even when a graphic error occurs on the screen displayed on the terminal, it is difficult to detect an error in such a situation on the game server because information recorded on the game log is normally recorded.

In addition, since the presentation effect is different for each object in the game and the distribution ratio for the graphic resource is also different, it may be displayed abnormally in one area of the terminal screen. This abnormal display screen does not significantly affect the operation of the game, but It can act as a factor that considerably hinders the degree of immersion.

6 is a diagram illustrating a method of detecting an edge region on a display screen of a terminal according to an exemplary embodiment of the present invention.

6(a) and 6(b) are diagrams showing a case where a game screen is displayed abnormally in a terminal, and FIGS. 6(c) and 6(d) are diagrams showing a case where a game screen is normally displayed in a terminal. These are the drawings.

An image may be converted to detect an edge region on each terminal screen, and in this case, a Laplacian filtering or a canny edge detection method may be used.

FIG. 6(e) is an image converted to detect an edge area in FIG. 6(a), and FIG. 6(f) is an image converted to detect an edge area in FIG. 6(b). In the case of FIGS. 6(e) and 6(f), a black area may be displayed, and such a black area may be detected as an edge area.

In the case of FIGS. 6(g) and 6(h), it can be seen that a black area, which is considered to be an abnormal screen, is not detected even if the edge detection algorithm is passed through the normal display in the terminal.

7 is a diagram for describing a method of detecting a specific area in an edge area according to an example of the present invention.

7(a) to 7(b) are images of the detected edge area, and FIG. 7(e) is a diagram for explaining the detection of a specific area within the edge area of FIG. 7(a). As shown in FIGS. 7(e) and 7(f), a specific area may be detected by being divided into various sizes within the edge area in a rectangular shape.

In the case of FIGS. 7(g) and 7(h), since the edge area was not detected, it can be confirmed that the specific area was also not detected.

When the number of the detected specific regions is more than a certain number, it may be determined that the terminal is displaying abnormally, and terminal-related information may be transmitted to the reporting server for error reporting.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery medium.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: game server
210, 220, 230: game terminal
300: reporting server

Claims (10)

In the abnormal display screen detection method performed in a terminal,
Converting a screen output from a terminal in which the game service is being used into a gray image and then detecting an edge region;
Detecting a specific area within the detected edge area; And
When it is determined that the number of the detected specific regions is greater than or equal to a preset number, transmitting terminal-related information to a reporting server,
The step of detecting the specific region,
This is a step of detecting a specific area of a smaller size after detecting the detected edge area as a specific area of a larger size,
The screen output from the terminal is characterized in that the screen determined to be normal on the game log,
How to detect abnormal display screen. The method of claim 1,
A specific area within the detected edge area,
An abnormal display screen detection method, characterized in that the rectangular area displayed in black in the terminal. The method of claim 1,
The terminal-related information is device information of the terminal,
An abnormal display screen detection method comprising at least one or more of manufacturer information, model name information, OS information, communication method information, battery usage information, and remaining memory information of the terminal. The method of claim 1,
The terminal-related information is log information of a game being serviced by the terminal,
An abnormal display screen detection method comprising at least one of game level information, experience value information, play time information, and purchase record information. In the abnormal display screen detection method performed in a terminal,
Resizing an image displayed on a terminal in which the game service is being used;
Converting the resized image to a gray image and detecting an edge region;
Detecting a specific area within the detected edge area; And
When it is determined that the number of the detected specific regions is greater than or equal to a preset number, transmitting terminal-related information to a reporting server,
The step of detecting the specific region,
This is a step of detecting a specific area of a smaller size after detecting the detected edge area as a specific area of a larger size,
The screen output from the terminal is characterized in that the screen determined to be normal on the game log,
How to detect abnormal display screen. The method of claim 5,
The step of resizing the image,
An abnormal display screen detection method, characterized in that the image is resized to an image lower than the displayed screen resolution. In the abnormal display screen detection system,
An edge area detection unit configured to detect an edge area after converting a screen output from a terminal in which a game service is being used into a gray image;
A specific area detection unit for detecting a specific area within the edge area detected by the edge area detection unit;
An abnormal display determination unit configured to determine that the terminal is abnormally displayed when the number of the detected specific regions is determined to be greater than or equal to a preset number; And
Communication unit that transmits terminal-related information to the reporting server when it is determined that the display is abnormal
Including,
The specific area detection unit,
After detecting the detected edge area as a specific area of a larger size, it is detected as a specific area of a smaller size,
An abnormal display screen detection system, characterized in that the screen output from the terminal is a screen determined to be normal on the game log. The method of claim 7,
The specific area detection unit,
An abnormal display screen detection system, characterized in that the terminal detects a rectangular area displayed in black. The method of claim 7,
The terminal-related information is device information of the terminal,
An abnormal display screen detection system comprising at least one or more of manufacturer information, model name information, OS information, communication method information, battery usage information, and remaining memory information of the terminal. The method of claim 7,
The terminal-related information is log information of a game being serviced by the terminal,
An abnormal display screen detection system comprising at least one of game level information, experience value information, play time information, and purchase record information.

Images