WO2022062772A1 - 图像检测方法、装置、计算机设备及计算机可读存储介质 - Google Patents

图像检测方法、装置、计算机设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2022062772A1
WO2022062772A1 PCT/CN2021/113175 CN2021113175W WO2022062772A1 WO 2022062772 A1 WO2022062772 A1 WO 2022062772A1 CN 2021113175 W CN2021113175 W CN 2021113175W WO 2022062772 A1 WO2022062772 A1 WO 2022062772A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
value
matching
preset
detection
Prior art date
Application number
PCT/CN2021/113175
Other languages
English (en)
French (fr)
Inventor
任明星
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP21871145.5A priority Critical patent/EP4129430A4/en
Priority to JP2022567668A priority patent/JP7490082B2/ja
Publication of WO2022062772A1 publication Critical patent/WO2022062772A1/zh
Priority to US17/901,707 priority patent/US20220415038A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/352—Details of game servers involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/355—Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/54—Controlling the output signals based on the game progress involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/70—Game security or game management aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/32—Normalisation of the pattern dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/955—Hardware or software architectures specially adapted for image or video understanding using specific electronic processors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/50—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers
    • A63F2300/53—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers details of basic data processing
    • A63F2300/535—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers details of basic data processing for monitoring, e.g. of user parameters, terminal parameters, application parameters, network parameters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2404—Monitoring of server processing errors or hardware failure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781—Games

Definitions

  • the present application relates to the field of communication technologies, and in particular, to an image detection method, an apparatus, a computer device, and a computer-readable storage medium.
  • the game host of the cloud game is on the server, and players connect to the server through the local network.
  • the server plays the game, the server transmits the game screen in real time through the network, and performs vivid interactive entertainment.
  • the computer device can obtain the usage frequency of the central processing unit (CPU) in real time, and use the When the frequency is abnormal, it is determined that the game is stuck and the corresponding game optimization process is performed.
  • CPU central processing unit
  • Various embodiments of the present application provide an image detection method, apparatus, computer device, and computer-readable storage medium. These include:
  • An image detection method executed by a server, the method comprising:
  • An image detection device comprising:
  • an intercepting unit used for intercepting the first image and the second image at intervals of a preset time period from the video stream
  • a matching unit configured to perform pixel matching on the first image and the second image to obtain a total matching value of pixels between the first image and the second image
  • a detection unit configured to perform screen content detection on the second image when it is determined based on the total matching value of the pixels that the first image and the second image satisfy a preset matching condition
  • a determining unit configured to determine that the video stream is abnormal when it is detected that there is no picture content in the second image.
  • a computer device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein, when the processor executes the program, any image detection method provided by the embodiments of the present application is implemented steps in .
  • a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image detection methods provided in the various optional embodiments described above.
  • FIG. 1 is a schematic diagram of a scene of an image detection system provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of another scene of the image detection system provided by the embodiment of the present application.
  • FIG. 3 is a schematic flowchart of an image detection method provided by an embodiment of the present application.
  • FIG. 5a is a schematic diagram of a scene of an image detection method provided by an embodiment of the present application.
  • FIG. 5b is a schematic diagram of another scene of the image detection method provided by the embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of an image detection device provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • Embodiments of the present application provide an image detection method, an apparatus, a computer device, and a computer-readable storage medium.
  • FIG. 1 is a schematic diagram of an image detection system provided by an embodiment of the present application, including: a basic server A and a virtualized cloud host B (the basic server A and the virtualized cloud host B may also include: More, the specific number is not limited here), the basic server A is a physical machine, also called a physical server, which is the name of a physical computer relative to a virtual machine (Virtual Machine), and the physical machine is provided to the virtual machine.
  • hardware environment also known as "host” or "host”.
  • the basic server A can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, and cloud communications.
  • each basic server A can virtualize multiple cloud hosts B.
  • the cloud host B is a virtual machine, which can also be called a virtual private server (Virtual Private Server, VPS). It is a technology that partitions a server into multiple virtual independent dedicated servers. Each virtual independent server using VPS technology has its own independent Internet Protocol (Internet Protocol Address, IP) address, operating system, hard disk space, memory space, processor (Central Processing Unit, CPU) resources, etc. Installing programs, restarting the server, etc. is exactly the same as running a standalone server.
  • IP Internet Protocol Address
  • cloud host B is a VPS, but cloud host B further virtualizes all basic resources, such as memory bandwidth, etc., on all basic servers A or virtual machines.
  • the advantage of cloud host B is that it can store data in a distributed manner and dynamically expand basic resources. Strong security and scalability.
  • Each cloud host B has an independent operating system and hardware structure, which is exactly the same as running an independent host, except that the physical address in each cloud host B is the physical address of the virtual machine.
  • Multiple processors are installed. For example, multiple graphics processors (Graphics Processing Unit, GPU) are installed in one cloud host B.
  • One cloud host B can be similar to a VMware virtual machine, and one physical machine can virtualize multiple Android operations.
  • a system instance, a game board or container can be installed in the cloud host B, simulating the user's terminal environment, but without a physical display screen.
  • the running environment of the game can be installed on a board or container C of a cloud host B, and the board or container C is similar to a user's terminal. , but there is no physical display screen for screen display, the board or container C will have a streaming process, and the game video and sound will be pushed to the terminal through the streaming server D.
  • the terminal can be a smartphone, tablet computer, laptop computer , desktop computers, smart speakers, smart watches, etc., but not limited to this.
  • the terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited in this application.
  • the terminal can install and open an application or webpage E, and receive game video and sound for loading through the application or webpage E.
  • the user can input control events on the terminal to control the actions of the virtual game characters in the video, and the terminal will The control event is sent back to the board or container C of the cloud host B to implement game control.
  • the image detection apparatus may be specifically integrated in a server, and the server may be implemented by an independent server or a server cluster composed of multiple servers.
  • the server may be a cloud host or a physical server with a storage unit and a microprocessor installed with computing capability.
  • FIG. 3 is a schematic flowchart of an image detection method provided by an embodiment of the present application.
  • the image detection method includes:
  • Step 101 intercepting a first image and a second image spaced by a preset time period from a video stream.
  • the video stream may be composed of multiple frames of game images, that is, the video stream may be a game screen. It can be understood that, in the related art, due to the heavy load of game screen rendering or the incompatibility of some components of the cloud host The operation of the game will cause the game to freeze when it starts or during the game. When the game freezes, it is difficult to determine whether the game is really stuck by visual observation of the image.
  • the implementation of the present application can periodically intercept the first image and the second image at intervals of a preset time period from the game video stream through the server, and the preset time period can be freely set according to requirements, such as 10 seconds, 20 30 seconds, which is not specifically limited here.
  • the preset time period can be freely set according to requirements, such as 10 seconds, 20 30 seconds, which is not specifically limited here.
  • the first image of the current frame is captured
  • the second image of the 10-second frame is captured.
  • the first image and The number of pixels in the second image is the same.
  • the pixel is composed of small squares in the image. These small squares have a clear position and assigned color value. The color and position of the small squares determine the image.
  • the appearance of the image can be regarded as an indivisible unit or element in the entire image. Each image contains a certain amount of pixels, which determine the size of the image displayed on the screen.
  • Step 102 Perform pixel matching on the first image and the second image to obtain a total matching value of pixels between the first image and the second image.
  • the server can calculate the similarity between each pixel in the first image and the pixel at the corresponding position in the second image, for example, calculate the similarity between the first image and the second image.
  • the similarity score value is the total matching value of the pixels between the first image and the second image
  • the similarity score value reflects the degree of similarity between the first image and the second image, and it can be determined whether the first image and the second image meet the preset matching conditions according to the similarity score value, that is, whether the game screen is still.
  • the server may calculate a total matching value of pixels between the first image and the second image through a squared difference matching algorithm.
  • the corresponding squared difference matching algorithm may be determined based on the template matching (TemplateMatching) principle.
  • the template matching principle can find the most similar area to the template image in a pair of images to be matched.
  • the method is simple in principle and fast in calculation, and can be applied to target recognition, target tracking and other fields.
  • the server may use each pixel in the first image as a template image, and use the pixel at the corresponding position in the second image as the image to be matched, and for each pixel in the plurality of pixels in the second image, the matching algorithm is based on the squared difference. Calculate the matching value between the current pixel in the second image and the pixel corresponding to the first image, count the matching value of each pixel between the first image and the second image, and obtain the pixel between the first image and the second image.
  • the total matching value reflects the matching degree of the first image and the second image. When the total matching value of the pixel is 0, the first image and the second image are exactly the same; The first image and the second image are less similar.
  • the step of performing pixel matching on the first image and the second image to obtain a total matching value of pixels between the first image and the second image may include:
  • the total matching value of pixels between the first image and the second image can be determined by a squared difference matching algorithm, and the squared difference matching algorithm is:
  • the T(x', y') is a template image matrix.
  • the template image matrix is the first image as a template image, and a matrix formed by each pixel in the first image
  • I(x, y ) is the source image matrix
  • the source image matrix is a matrix formed by each pixel in the second image
  • I(x+x', y+y') is the pixel of the first image that covers the second image.
  • the R(x,y) is the total matching value of the pixels.
  • the server can count the sum of the squares of the difference between each pixel of each image and the corresponding pixel of the second target image, and obtain the total matching value R(x, y) of the pixel.
  • Step 103 when it is determined that the first image and the second image satisfy a preset matching condition based on the total matching value of the pixels, perform screen content detection on the second image.
  • a preset matching condition can be set, and the preset matching condition can be a preset score threshold, and the preset score threshold is a critical value that defines whether the first image and the second image match, that is, the total matching value of pixels is higher than
  • the server determines that the first image and the second image meet the preset matching conditions, and determines that the first image and the second image are successfully matched, that is, the similarity between the first image and the second image satisfies the conditions, and determines that it is a game screen
  • the server determines that the pixels between the first image and the second image do not meet the preset matching conditions, and determines that the matching between the first image and the second image fails, That is, the similarity between the first image and the second image does not satisfy the condition, and it is determined that the game screen is still changing and not stuck.
  • the game screen of the video stream separated by the preset time period is static and unchanged, that is, the game video stream. It may be stuck, because there may be a short-term static image in some game screens, and the stuck game screen is usually a solid color or an image with little change in brightness. Perform screen content detection.
  • the overall blurriness is bound to be less than a certain range, which can be less than 1.5, and the screen content detection can be correct.
  • the second image is subjected to image blur detection.
  • the image blur of the second image can be comprehensively detected by the Laplacian algorithm to realize the corresponding picture content detection.
  • the Laplacian algorithm is used for edge detection of the image and can be used for the image. Detection of light and dark changes to determine the blurriness of an image.
  • the step of intercepting the first image and the second image at intervals of a preset time period from the video stream can be returned to continue to perform detection.
  • the above-mentioned image detection method further includes the step of determining that the first image and the second image satisfy a preset matching condition based on the total matching value of the pixels, wherein, based on the total matching value of the pixels, determining the first image
  • the step of an image and the second image satisfying a preset matching condition may include:
  • the embodiment of the present application adopts a normalization process to The total matching value is scaled between 0 and 1. The closer the normalized matching value is to 0, the closer the first image and the second image are, and the closer the normalized matching value is to 1, the first image The closer it is to the second image.
  • the normalized matching value can be converted into a score value, the closer to 0, the higher the score, the closer to 1, the lower the score , and then the matching degree can be defined by a preset score threshold.
  • the server detects that the score value is greater than the preset score threshold, it is determined that the first image and the second image meet the preset matching conditions, and the first image and the second image meet the preset matching conditions. The two images are matched successfully.
  • the server detects that the score value is less than or equal to the preset score threshold it is determined that the first image and the second image do not meet the preset matching condition, and the first image and the second image fail to match.
  • the step of performing normalization processing on the matching value to obtain a normalized matching value may include:
  • the total matching value of the pixels can be normalized by the following formula to obtain the normalized matching value, and the formula can be the normalized squared difference matching method:
  • the T(x', y') is a template image matrix.
  • the template image matrix is the first image as a template image, and a matrix formed by each pixel in the first image
  • I(x, y ) is the source image matrix
  • the source image matrix is a matrix formed by each pixel in the second image
  • I(x+x', y+y') is the pixel of the first image that covers the second image.
  • the R(x, y) is the matching value after normalization processing.
  • the numerator of the above formula is the total matching value of the pixel, calculate the ratio of the total matching value of the pixel in the numerator to the target value of the denominator, and obtain the matching value after normalization processing, so as to reduce the matching value Between 0 and 1, the closer the normalized matching value is to 0, the closer the first image and the second image are, and the closer the normalized matching value is to 1, the first image and the second image are closer. The closer the image is.
  • the step of converting the normalized matching value into a score value may include:
  • the part close to 0 is not conducive to setting the threshold value for determination.
  • the preset amplification threshold may be 1000, so that 1000 is the maximum score, and the preset score threshold may be set to 950, and when it is detected that the score is greater than the preset score threshold When , it is determined that the first image and the second image match.
  • the step of performing screen content detection on the second image may include:
  • the frame area of the second image may be filtered out first to obtain a filtered image to be detected.
  • the server can perform Gaussian blurring through data smoothing technology. Understandably, the calculated weight can be weighted to the corresponding pixels to achieve image smoothing, where the weight can be the average value of surrounding pixels. . In this way, the server can perform Gaussian blur processing on the image to be detected, and smooth the image to be detected to obtain a Gaussian blurred image of the target to be detected, so that the subsequent blurriness calculation is more accurate.
  • the server may calculate the target blurriness of the target image to be detected.
  • the server may calculate the target blurriness of the target image to be detected by using the Laplace algorithm. The lower the target blurriness, the description The lower the pixel change rate, the greater the probability that there is no picture content in the second image, the higher the target blurriness, the greater the pixel change rate, the lower the probability that there is no picture content in the second image.
  • the critical value that defines whether there is picture content in the second image can be set as a preset blurriness threshold, and when detecting that the target blurriness is lower than the preset blurriness threshold, the server determines that there is no screen in the second image. content.
  • the step of calculating the target blurriness of the target image to be detected may include:
  • the server can obtain the waveform data set of the light and dark values of each pixel in the target image to be detected through the second-order differential in the Laplace algorithm, the waveform data with brighter pixels is negative, and the waveform data with darker pixels is positive. .
  • the server calculates the average value of the waveform data set, the standard deviation is the arithmetic square root of the variance, and the standard deviation can reflect the degree of dispersion of the waveform data set.
  • the difference between the average values is larger, and the smaller the standard deviation, the smaller the difference between most of the data in the waveform data set and the average value is. Therefore, the server can judge whether the brightness of the pixels in the second image is larger based on the standard deviation. , that is, the standard deviation corresponding to the waveform data set is obtained according to the average value, and the standard deviation is determined as the target ambiguity.
  • Step 104 when it is detected that there is no picture content in the second image, it is determined that the video stream is abnormal.
  • the server when the server detects that there is no screen content in the second image, it indicates that the overall brightness and darkness of the pixels in the second image change within a certain range, that is, it is determined that the second image basically does not have any pixel changes, and the screen has no content. , is a solid color picture, it is determined that the video stream is abnormal, and the game is stuck.
  • the first image and the second image at intervals of a preset time period are intercepted from the video stream; the pixels between the first image and the second image are obtained by performing pixel matching on the first image and the second image.
  • the second image is subjected to picture content detection; when it is detected that there is no picture content in the second image, it is determined The video stream is abnormal.
  • an image recognition method can be used to detect the image frames of the video stream at a preset time period, and when it is detected that the image frames at the preset time period remain unchanged, the second image is subjected to screen content detection.
  • the embodiment of the present application can use the total matching value of pixels between images to determine whether the video picture is still on the basis of code design or log extraction that does not invade the game, and then perform content detection on the still image. , to determine whether the video stream is stuck, compatible with various colors and brightness, and will not cause inaccurate detection due to changes in screen brightness and color, which greatly improves the accuracy of image detection.
  • the image detection device is specifically integrated in a server, and the server is a cloud host as an example for description, and the following description is specifically referred to.
  • FIG. 4 is another schematic flowchart of the image detection method provided by the embodiment of the present application.
  • the method flow may include:
  • Step 201 the server intercepts the first image and the second image at a preset time interval from the video stream.
  • the video stream of the present application may be a cloud game video stream, because the game screen corresponding to the cloud game video stream is prone to heavy game screen rendering load or temporary incompatibility, causing the game to freeze when starting or during the game.
  • the server may periodically intercept the first game image and the second game image from the video stream at an interval of 10 seconds.
  • the number of pixels is the same.
  • Step 202 the server overlays the pixels of the first image on the pixels of the second image to obtain a second target image after covering, and counts the sum of the squares of the difference between each pixel of the first image and the pixel corresponding to the second target image , to get the total matching value of pixels between the first image and the second image.
  • the total matching value of pixels between the first image and the second image can be determined by the following formula, and the formula can specifically be a square difference matching algorithm:
  • the T(x', y') is a template image matrix.
  • the template image matrix is the first image as a template image, and a matrix formed by each pixel in the first image
  • I(x, y ) is the source image matrix
  • the source image matrix is a matrix formed by each pixel in the second image
  • I(x+x', y+y') is the pixel of the first image that covers the second image.
  • the obtained matrix formed by each pixel of the second target image after coverage is obtained, and in this step, the R(x, y) is the total matching value of the pixel.
  • the server covers the pixels of the first image on the pixels of the second image to obtain the covered second target image, and based on the above formula, counts the pixels of the corresponding relationship between each pixel of each image and the second target image The sum of the squares of the differences is obtained to obtain the total matching value R(x, y) of the pixel.
  • Step 203 the server counts the sum of the products of each pixel of the first image and the pixel corresponding to the second target image, performs quadratic radical calculation on the sum of the statistical products, obtains the target value, and calculates the ratio of the matching value to the target value, Get the normalized matching value.
  • the total matching value can be normalized by the following formula, and the formula can be the normalized squared difference matching method:
  • the T(x', y') is a template image matrix.
  • the template image matrix is the first image as a template image, and a matrix formed by each pixel in the first image
  • I(x, y ) is the source image matrix
  • the source image matrix is a matrix formed by each pixel in the second image
  • I(x+x', y+y') is the pixel of the first image that covers the second image.
  • the R(x, y) is the matching value after normalization processing.
  • the matching value after normalization processing is obtained, so as to reduce the matching value to between 0 and 1.
  • the matching value after normalization processing is closer to 0, the closer the first image and the second image are, the closer the normalized matching value is to 1, and the less close the first image and the second image are.
  • Step 204 the server calculates the difference between the preset basic value and the normalized matching value, and multiplies the difference by the preset amplification threshold to obtain a score value, and when it is detected that the score value is greater than the preset score threshold, it determines the first An image and a second image satisfy a preset matching condition.
  • the preset basic value may be 1, and the server calculates the difference between the preset basic value 1 minus the normalized matching value, so that the judgment rule after normalization is adjusted in reverse, and the difference is closer to 0 , the more dissimilar the first image and the second image are, the closer the difference is to 1, the more similar the first image and the second image are.
  • the preset magnification threshold can be set to 1000, and the difference value can be multiplied by the preset magnification threshold of 1000 to obtain a score between 0 and 1000.
  • the score is equal to When the value is 1000, it means that the first image and the second image are exactly the same.
  • the preset score threshold value can be set to 950.
  • the detected score value is greater than the preset score threshold value, it is determined that the first image is detected.
  • Matching with the second image it is determined that the first image and the second image satisfy the preset matching condition.
  • the game video stream When it is detected that the score value is not greater than the preset score threshold, it means that the first image and the second image have changed, the game video stream is running normally, and the execution server can return to the video stream to intercept the first interval of the preset time period. Image and second image steps, continue to detect.
  • Step 205 The server performs pixel matching between the first image and the second image at a preset time period, and records the number of times that pixel matching between the first image and the second image is detected.
  • the server since some game video streams may be loaded for a short time, such as 12 seconds of loading time, in order to prevent the short-time loading situation from being misjudged as the game video stream being stuck, the server performs pixel matching at preset time intervals.
  • the number of times that the first image and the second image are successfully matched is recorded, and the condition for a successful match may be: the first image and the second image at an interval of 10 seconds satisfy the preset matching condition.
  • the first image and the second image satisfy the preset matching condition, it may be determined that the first image and the second image are successfully matched.
  • step 206 the server detects whether the number of times exceeds a preset number of times.
  • the preset number of times is a defined value that defines the real stillness of the game screen, for example, 3 times.
  • the server detects that the number of times exceeds the preset number of times, it means that neither the first image nor the second image has changed within 30 seconds, and they are still on the screen.
  • step 207 is executed.
  • the server detects that the number of times does not exceed the preset number of times, it means that the first image and the second image have changed within 30 seconds, and the game video stream is not stuck.
  • Step 207 The server performs filtering processing on the frame area of the second image to obtain an image to be detected after filtering processing, and performs blurring processing on the image to be detected to obtain a target image to be detected after blurring processing.
  • the server when the server detects that the number of times exceeds the preset number of times, it means that the first image and the second image have not changed within 30 seconds, and are in a state of still picture, that is, the game video stream may be stuck, due to the stuck game screen. Usually it is a solid color or an image with little change in brightness and darkness.
  • a second image needs to be further detected, and the border area of the second image can be filtered out, please refer to Figure 5a.
  • the title bar on the border of the second image 1 also has "King X Yao". If this part is processed, it will affect the processing result. Therefore, the server can filter the border area of the second image to obtain the filtered image.
  • the processed image to be detected 11 when the server detects that the number of times exceeds the preset number of times, it means that the first image and the second image have not changed within 30 seconds, and are in a state of still picture, that is, the game video stream may be stuck, due to the stuck game screen. Usually it is a solid
  • Gaussian blurring can be performed on the image to be detected 11 to obtain the target image to be detected after Gaussian blurring.
  • the RGB value of the pixel at the center point of the partial area 2 is 2.
  • the RGB value of the pixel at the center point of the partial area 3 of the target image to be detected after the Gaussian blurring is obtained by referring to the average value of the surrounding pixels, When it becomes 1, the pixel loses some details and realizes image blur processing.
  • the Gaussian blur processing has a key parameter ksize, which represents the blur radius. The larger the radius, the blurrier the effect of Gaussian blur processing.
  • the blur radius refers to the surrounding pixels. Quantity value, in this embodiment of the present application, the blur radius may be set to 1.
  • Step 208 the server calculates the image to be detected by the Laplacian algorithm, obtains the waveform data set, calculates the average value of the waveform data set, and obtains the standard deviation corresponding to the waveform data set according to the average value, and determines the standard deviation as the target ambiguity.
  • the server can also perform grayscale processing on the target image to be detected.
  • Grayscale means no color.
  • Set the RGB color components of the target image to be detected equal to obtain a grayscale image and specify the operator size of the Laplacian algorithm.
  • the grayscale image is calculated by the Laplace algorithm to obtain a waveform data set composed of waveform data of each pixel in the grayscale image, and the waveform data can reflect the brightness of the pixel.
  • the average value of the waveform data set can be calculated, and the standard deviation corresponding to the waveform data set can be obtained according to the average value, and the standard deviation reflects the difference between the waveform data in the waveform data set and the average value. This indicates that most of the data in the waveform data set has a large difference with the average value, and the smaller the standard deviation, the smaller the difference between most of the data in the waveform data set and the average value.
  • Step 209 when the server detects that the target blurriness is lower than the preset blurriness threshold, it is determined that there is no picture content in the second image, and that the video stream is abnormal.
  • the preset blurriness threshold is a critical value that defines whether there is picture content in the second image.
  • the preset blurriness threshold may be set to 1.5.
  • the embodiments of the present application may upload the second image that is determined to be stuck to a Convolutional Neural Networks (CNN) model for learning, so that through continuous learning, the convolutional neural network can Learn the ability to identify the screen of the game video stream stuck, and realize fast identification.
  • CNN Convolutional Neural Networks
  • intercepting the first image and the second image at a preset time interval from the video stream includes intercepting the first image and the second image at the preset time interval from the video stream at the current detection moment before the second image is detected by the screen content
  • the method further includes: when based on the total matching value of the pixels, it is determined that the total matching value of the pixels between the first image and the second image corresponding to the current detection moment satisfies the preset matching condition
  • the current matching result is determined as successful matching
  • the stored historical matching result is obtained
  • the historical matching result is the matching result obtained by performing pixel matching processing on the first image and the second image of the intercepted interval preset time period at the historical detection moment ;
  • Based on the historical matching result and the current matching result determine the number of successful matches within the preset detection period; and when based on the total matching value of the pixel, determine that the total matching value of the pixel between the first image and the second image satisfies the preset matching
  • the server since some game video streams may be loaded for a short time, such as 12 seconds of loading time, in order to prevent the short-time loading situation from being misjudged as the game video stream being stuck, the server performs pixel matching at preset time intervals.
  • the number of times that the first image and the second image are successfully matched is recorded, and the condition for a successful match may be: the first image and the second image at an interval of 10 seconds satisfy the preset matching condition.
  • the first image and the second image satisfy the preset matching condition, it may be determined that the first image and the second image are successfully matched.
  • the server intercepts the first image and the second image at a preset time interval from the video stream at the current detection moment.
  • the current detection moment refers to the current time point, and the first image and the second image are separated by a preset time period, for example, the first image and the second image are separated by 10 seconds.
  • the server determines whether the total matching value of the pixels between the first image and the second image intercepted from the video stream at the current moment satisfies the preset matching condition through the above method, and when it is determined that the preset matching condition is met, that is, the current matching When the result is that the matching is successful, the stored historical matching results are obtained.
  • the historical matching result is a matching result obtained by performing pixel matching processing on the intercepted first image and the second image at a preset time interval at the historical detection moment.
  • a historical moment refers to a moment prior to the current moment.
  • the server intercepts the first image and the second image from the video stream at regular intervals, performs pixel matching processing on the first image and the second image, obtains a matching result, and stores the matching result in the memory, thereby
  • the server may acquire the historical matching results within the preset detection time period from the memory.
  • the preset detection time period refers to a preset period of time ending at the current detection time.
  • the preset detection time period may include the first 30 seconds of the current detection time.
  • the server determines the number of successful matches within the preset detection period according to the historical matching results and the current matching results.
  • the preset number of times is a defined value that defines the real stillness of the game screen, such as 3 times.
  • the server detects that the number of times is greater than or equal to the preset number of times, it means that neither the first image nor the second image has changed within 30 seconds.
  • the server performs picture content detection on the current second image.
  • the server may perform image detection on the current second image by using the method for performing image content detection on the second image in the foregoing embodiment.
  • the number of detections by the server is less than the preset number of times, it means that the first image and the second image have changed within 30 seconds, the game video stream is not stuck, enter the next detection time, and use the next detection time as the current time , returning to the step of intercepting the first image and the second image at a preset time interval from the video stream at the current detection moment, and continuing the detection until the video stream finishes playing. .
  • the server when the total matching value of pixels between the first image and the second image captured from the video stream at the current moment does not meet the preset matching condition, the server returns the data from the video stream at the current detection moment.
  • the step of capturing the first image and the second image separated by a preset time period continues to detect.
  • the first image and the second image at intervals of a preset time period are intercepted from the video stream; the pixels between the first image and the second image are obtained by performing pixel matching on the first image and the second image.
  • the second image is subjected to picture content detection; when it is detected that there is no picture content in the second image, it is determined The video stream is abnormal.
  • an image recognition method can be used to detect the image frames of the video stream at a preset time period, and when it is detected that the image frames at the preset time period remain unchanged, the second image is subjected to screen content detection.
  • the embodiment of the present application can use the total matching value of pixels between images to determine whether the video picture is still on the basis of code design or log extraction that does not invade the game, and then perform content detection on the still image. , to determine whether the video stream is stuck, compatible with various colors and brightness, and will not cause inaccurate detection due to changes in screen brightness and color, which greatly improves the accuracy of image detection.
  • an embodiment of the present application further provides an apparatus for an image detection method.
  • the device can be integrated in the server.
  • the meanings of the nouns are the same as those in the above-mentioned image detection method, and the specific implementation details can refer to the description in the method embodiment.
  • FIG. 6 is a schematic structural diagram of an image detection apparatus provided by an embodiment of the present application, wherein the image detection apparatus may include an intercepting unit 301 , a matching unit 302 , a detection unit 303 , and a determination unit 304 .
  • the intercepting unit 301 is configured to intercept the first image and the second image spaced apart by a preset time period from the video stream.
  • the matching unit 302 is configured to perform pixel matching on the first image and the second image to obtain a total matching value of pixels between the first image and the second image.
  • the matching unit 302 includes:
  • a covering subunit used for covering the pixels of the first image on the pixels of the second image to obtain the covered second target image
  • a statistical subunit configured to count the sum of the squares of the differences between each pixel of the first image and the pixels corresponding to the second target image, to obtain the total matching value of the pixels between the first image and the second image.
  • the detection unit 303 is configured to perform screen content detection on the second image when it is determined based on the total matching value of the pixels that the first image and the second image satisfy a preset matching condition.
  • the detection unit 303 includes:
  • the normalization subunit is used to normalize the total matching value to obtain the normalized matching value
  • the conversion subunit is used to convert the normalized matching value into a score value
  • the determining subunit is configured to determine that the first image and the second image satisfy a preset matching condition when it is detected that the score value is greater than a preset score threshold, and perform screen content detection on the second image.
  • the determining subunit is further configured to determine that the first image and the second image do not meet the preset matching condition when it is detected that the score value is less than or equal to the preset score threshold.
  • the normalization subunit is further configured to count the sum of the products of each pixel of the first image and the pixels corresponding to the second target image; perform quadratic radical calculation on the sum of the statistical products, Obtain the target value; calculate the ratio of the total matching value of the pixel to the target value, and obtain the normalized matching value.
  • the conversion subunit is further configured to calculate the difference between the preset basic value and the normalized matching value; multiply the difference by the preset amplification threshold to obtain a score value.
  • the detection unit 303 includes:
  • the filtering subunit is used for filtering out the frame area of the second image when the total matching value of the pixels between the first image and the second image satisfies the preset matching condition, to obtain a filtered image. image to be detected;
  • a processing subunit used for blurring the to-be-detected image to obtain a blurred-processed target to-be-detected image
  • a determination subunit configured to determine that there is no picture content in the second image when it is detected that the target blurriness is lower than a preset blurriness threshold.
  • the calculation subunit is further configured to calculate the image to be detected by the Laplacian algorithm to obtain a waveform data set; calculate the average value of the waveform data set, and obtain the waveform data set according to the average value.
  • the standard deviation corresponding to the waveform data set; this standard deviation is determined as the target ambiguity.
  • the determining unit 304 is configured to determine that the video stream is abnormal when it is detected that there is no picture content in the second image.
  • the image detection apparatus may further include a recording unit for: performing pixel matching between the first image and the second image at a preset time period; recording the detection of the first image and the second image The number of times of pixel matching for the image; when it is detected that the number of times exceeds the preset number of times, the step of performing screen content detection on the second image is performed; when it is detected that the number of times does not exceed the preset number of times, it returns to perform interception from the video stream The step of spacing the first image and the second image for a preset time period.
  • the image detection device is further configured to intercept the first image and the second image at a preset time interval from the video stream at the current detection moment; when the current detection moment is determined based on the total matching value of the pixels When the total matching value of the pixels between the corresponding first image and the second image satisfies the preset matching conditions, it is determined that the current matching result is a successful match; the stored historical matching results are obtained; the historical matching results are the intercepted results at the historical detection moment.
  • the image detection device is further configured to enter the next detection moment when the number of successful matches within the preset detection period is less than the preset number of times, and use the next detection moment as the current moment, and return to the current moment.
  • the step of intercepting the first image and the second image at intervals of a preset time period from the video stream continues to detect, and stops when the video stream finishes playing.
  • the intercepting unit 301 intercepts the first image and the second image at a preset time interval from the video stream; the matching unit 302 performs pixel matching on the first image and the second image to obtain the first image and the second image.
  • an image recognition method can be used to detect the image frames of the video stream at a preset time period, and when it is detected that the image frames at the preset time period remain unchanged, the second image is subjected to screen content detection.
  • the frequency of use of the central processing unit is abnormally determined, and the threshold for abnormal determination is difficult to accurately set, and the detection accuracy is poor.
  • the embodiment of the present application can use the total matching value of pixels between images to determine whether the video picture is still on the basis of code design or log extraction that does not invade the game, and then perform content detection on the still image. , to determine whether the video stream is stuck, compatible with various colors and brightness, and will not cause inaccurate detection due to changes in screen brightness and color, which greatly improves the accuracy of image detection.
  • the embodiment of the present application further provides a server, as shown in FIG. 7 , which shows a schematic structural diagram of the server involved in the embodiment of the present application, specifically:
  • the server may be a cloud host, and may include a processor 401 of one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403 and an input unit 404 and other components.
  • a processor 401 of one or more processing cores may include a processor 401 of one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403 and an input unit 404 and other components.
  • FIG. 7 does not constitute a limitation on the server, and may include more or less components than the one shown, or combine some components, or arrange different components. in:
  • the processor 401 is the control center of the server, using various interfaces and lines to connect various parts of the entire server, by running or executing the software programs and/or modules stored in the memory 402, and calling the data stored in the memory 402, Execute various functions of the server and process data to monitor the server as a whole.
  • the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc. , the modem processor mainly deals with wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 401.
  • the memory 402 can be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402 .
  • the memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required for at least one function, and the like; Data created by the use of the server, etc.
  • memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, memory 402 may also include a memory controller to provide processor 401 access to memory 402 .
  • the server also includes a power supply 403 for supplying power to various components.
  • the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system.
  • Power source 403 may also include one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and any other components.
  • the server may also include an input unit 404 that may be used to receive input numerical or character information and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control.
  • an input unit 404 may be used to receive input numerical or character information and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control.
  • the server may also include a display processor and the like, which will not be described herein again.
  • the processor 401 in the server loads the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the execution and stores them in the memory 402 in order to achieve various functions, as follows:
  • the second image is subjected to picture content detection; when it is detected that there is no picture content in the second image, the video is determined An exception occurred in the stream.
  • the server in this embodiment of the present application can obtain the first image and the second image by performing pixel matching on the first image and the second image by intercepting the first image and the second image at a preset time interval from the video stream;
  • the screen content detection is performed on the second image; when it is detected that there is no screen content in the second image , it is determined that the video stream is abnormal.
  • an image recognition method can be used to detect the image frames of the video stream at a preset time period, and when it is detected that the image frames at the preset time period remain unchanged, the second image is subjected to screen content detection.
  • the embodiment of the present application can use the total matching value of pixels between images to determine whether the video picture is still on the basis of code design or log extraction that does not invade the game, and then perform content detection on the still image. , to determine whether the video stream is stuck, compatible with various colors and brightness, and will not cause inaccurate detection due to changes in screen brightness and color, which greatly improves the accuracy of image detection.
  • a computer apparatus comprising a memory and one or more processors, the memory storing computer readable instructions that, when executed by the processor, cause the one or more processors Execute the steps of executing the above-mentioned method for processing the audio mixing of the call audio.
  • the steps of the call audio mixing processing method here may be the steps in the image detection methods of the above embodiments.
  • one or more non-volatile readable storage media are provided that store computer-readable instructions that, when executed by one or more processors, cause the one or more processors to execute Perform the steps of the image detection method described above.
  • the steps of the image detection method here may be the steps in the image detection methods of the above embodiments.
  • a computer program product or computer program comprising computer instructions stored in a computer readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementation manners provided by the foregoing embodiments.
  • the computer-readable storage medium may include: a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Acoustics & Sound (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Business, Economics & Management (AREA)
  • Computer Security & Cryptography (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Hardware Design (AREA)
  • Image Analysis (AREA)

Abstract

一种图像检测方法，由服务器执行。该方法包括：从视频流中截取间隔预设时间段的第一图像和第二图像；将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值；当基于像素的总匹配值，确定第一图像和第二图像满足预设匹配条件时，对第二图像进行画面内容检测；在检测到第二图像中不存在画面内容时，确定视频流出现异常。以此，可以采用图像识别的方式，对视频流间隔预设时间段的图像画面进行检测，当检测到间隔预设时间段的图像画面不变时，对第二图像进行画面内容检测，在第二图像同时不存在画面内容时，判定为视频流出现异常。

Description

图像检测方法、装置、计算机设备及计算机可读存储介质
本申请要求于2020年09月25日提交中国专利局，申请号为202011024483.0、发明名称为“图像检测方法、装置及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信技术领域，具体涉及一种图像检测方法、装置、计算机设备及计算机可读存储介质。
背景技术
随着互联网技术的飞速发展，计算机设备的处理能力也越来越强，从而衍生出很多基于人机交互的应用程序，例如云游戏，该云游戏的游戏主机在服务器，玩家通过本地网络连接服务器，在服务器玩游戏时，服务器将游戏画面通过网络进行实时传输，进行生动的互动娱乐。
现有技术中，在云游戏的启动以及使用中，很可能出现游戏卡死的情况，计算机设备可以通过实时获取中央处理器(central processing unit，CPU)的使用频率，在该中央处理器的使用频率出现异常时，判定为游戏卡死的情况并进行相应的游戏优化处理。
在对现有技术的研究和实践过程中，本申请的发明人发现，现有技术中，由于游戏中不同场景对于中央处理器的消耗不同，使得异常判定的阈值很难进行准确设置，导致检测准确率较差。
发明内容
本申请的各种实施例提供一种图像检测方法、装置、计算机设备及计算机可读存储介质。其中包括：
一种图像检测方法，由服务器执行，所述方法包括：
从视频流中截取间隔预设时间段的第一图像和第二图像；
将所述第一图像和所述第二图像进行像素匹配获得所述第一图像和所述第二图像之间像素的总匹配值；
当基于所述像素的总匹配值，确定所述第一图像和所述第二图像满足预设匹配条件时，对所述第二图像进行画面内容检测；及
在检测到所述第二图像中不存在画面内容时，确定所述视频流出现异常。
一种图像检测装置，包括：
截取单元，用于从视频流中截取间隔预设时间段的第一图像和第二图像；
匹配单元，用于将所述第一图像和所述第二图像进行像素匹配获得第一图像和所述第二图像之间像素的总匹配值；
检测单元，用于当基于所述像素的总匹配值，确定所述第一图像和所述第二图像满足预设匹配条件时，对所述第二图像进行画面内容检测；及
确定单元，用于在检测到所述第二图像中不存在画面内容时，确定所述视频流出现异常。
一种计算机设备，包括存储器，处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现本申请实施例提供的任一种图像检测方法中的步骤。
一种计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现本申请实施例提供的任一种图像检测方法中的步骤。
一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述各种可选实施例中提供的图像检测方法。
附图说明
图1是本申请实施例提供的图像检测系统的场景示意图；
图2是本申请实施例提供的图像检测系统的另一场景示意图
图3是本申请实施例提供的图像检测方法的流程示意图；
图4是本申请实施例提供的图像检测方法的另一流程示意图；
图5a为本申请实施例提供的图像检测方法的场景示意图；
图5b为本申请实施例提供的图像检测方法的另一场景示意图；
图6是本申请实施例提供的图像检测装置的结构示意图；
图7是本申请实施例提供的服务器的结构示意图。
具体实施方式
本申请实施例提供一种图像检测方法、装置、计算机设备及计算机可读存储介质。
请参阅图1，图1为本申请实施例所提供的图像检测系统的场景示意图，包括：基础服务器A、和虚拟化的云主机B(该基础服务器A和虚拟化的云主机B还可以包括更多，具体个数在此不作限定)，该基础服务器A即为物理机，也称为实体服务器，是相对于虚拟机(Virtual Machine)而言的实体计算机的称呼，物理机提供给虚拟机的硬件环境，也称为“宿主”或者“寄主”。基础服务器A可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人 工智能平台等基础云计算服务的云服务器。通过对该基础服务器A进行虚拟化，每台基础服务器A都可以虚拟化出多个云主机B，该云主机B即为虚拟机，也可以称为虚拟专用服务器(Virtual Private Server，VPS)，是将一个服务器分区成多个虚拟独立专享服务器的技术。每个使用VPS技术的虚拟独立服务器拥有各自独立的公网互联网协议(Internet Protocol Address，IP)地址、操作系统、硬盘空间、内存空间、处理器(Central Processing Unit，CPU)资源等，还可以进行安装程序、重启服务器等操作，与运行一台独立服务器完全相同。也就是说通过软件层面，对一台服务器进行虚拟划分，虚拟出来多台服务器，这样就能让只需要一点点计算能力用户享用到大型服务器的计算资源。从广义上讲，云主机B就是VPS，只不过云主机B是在所有基础服务器A或者虚拟机上进一步虚拟化所有基础资源，例如内存带宽等等。云主机B的优势在于它可以分布式存储数据，动态扩展基础资源。安全性和扩展性较强。
该每台云主机B拥有独立的操作系统和硬件结构，与运行一台独立主机完全相同，只不过每台云主机B中的物理地址都为虚拟机的物理地址，每台云主机B中可以安装有多个处理器，如一台云主机B中安装有多个图形处理器(Graphics Processing Unit，GPU)，一台云主机B可以类似于VMware虚拟机，一个物理机可以虚拟化多个安卓操作系统实例，该一台云主机B中可以安装游戏的板卡或者容器，模拟用户的终端环境，但是无物理显示屏。
为了更好的描述本申请实施例，请一并参阅图2所示，游戏的运行环境可以安装在一台云主机B的板卡或者容器C中，该板卡或者容器C类似于用户的终端，但是没有物理显示屏进行画面显示，该板卡或者容器C会有一个推流进程，通过推流服务器D将游戏视频和声音推送到终端上，该终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等，但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接，本申请在此不做限制。该终端可以安装并开启应用或者网页E，通过该应用或者网页E接收游戏视频和声音进行加载，在一实施方式中，用户可以在终端输入控制事件，控制视频中的虚拟游戏人物行动，终端将该控制事件回传至云主机B的板卡或者容器C中，实现游戏控制。
以此，当云主机B中的板卡或容器C中的游戏视频出现卡死时，会影响终端测的显示，进而导致用户无法操作，导致游戏无法进行，以此，需要实时有效的监测游戏的视频是否异常，以实现及时处理，防止游戏宕机。
基于上述应用场景的描述，以下分别进行详细说明。需说明的是，以下实施例的序号不作为对实施例优选顺序的限定。
本实施例将从图像检测装置的角度进行描述，该图像检测装置具体可以集成在服务器中，该服务器可以用独立的服务器或者是多个服务器组成的服务器集群来实现。服务器具体可以是具备储存单元并安装有微处理器而具有运算能力的云主机或物理服务器。
请参阅图3，图3是本申请实施例提供的图像检测方法的流程示意图。该图像检测方法包括：
步骤101，从视频流中截取间隔预设时间段的第一图像和第二图像。
其中，该视频流可以由多帧游戏图像组成，即该视频流可以为游戏画面，可以理解的是，在相关技术中，由于游戏画面渲染的负荷较大，或者云主机的某些组件不兼容游戏的运行，会导致游戏启动时或者游戏过程中卡死，而游戏卡死时，光凭视觉观察图像很难让人确定游戏是否真的卡死不动。
以此，本申请实施可通过服务器从游戏视频流中定期的截取间隔预设时间段的第一图像和第二图像，该预设时间段可以根据需求自由设置，例如为10秒、20秒或者30秒，此处不作具体限定，例如游戏视频流在运行的过程中，截取当前帧的第一图像，在运行过去的10秒时，截取第10秒帧的第二图像，该第一图像和第二图像的像素数量是相同的，该像素是指由图像的小方格组成的，这些小方块都有一个明确的位置和被分配的色彩数值，小方格颜色和位置就决定该图像所呈现出来的样子，可以将像素视为整个图像中不可分割的单位或者是元素，每一图像包含了一定量的像素，这些像素决定图像在屏幕上所呈现的大小。
步骤102，将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值。
其中，为了判断游戏视频流对应的游戏画面是否静止不动，需要将间隔预设时间段的第一图像和第二图像进行像素匹配。由于该第一图像和第二图像的像素数量相同，所以可通过服务器计算第一图像中每一像素与第二图像中相应位置处的像素之间的相似度，例如，计算第一图像和第二图像中具有相同位置的两个像素之间的相似度，将全部的相似度进行整合，得到相似分数值，该相似分数值即为第一图像和第二图像之间像素的总匹配值，该相似分数值反应了第一图像和第二图像相似的程度，可以根据该相似分数值判定该第一图像和第二图像是否满足预设匹配条件，即判定该游戏画面是否静止不动。
在其中一个实施例中，服务器可以通过平方差匹配算法计算该第一图像和该第二图像之间像素的总匹配值。其中，可基于模板匹配(TemplateMatching)原理确定相应的平方差匹配算法。该模板匹配原理可以在一副待匹配图像中寻找和模板图像最相似的区域，该方法原理简单计算速度快，能够应用于目标识别，目标跟踪等多个领域。
服务器可以将第一图像中的每一像素当作模板图像，将第二图像中对应位置的像素作为待匹配图像，对于第二图像中多个像素中的每个像素，均基于平方差匹配算法计算第二图像中当前像素与第一图像对应位置像素之间的匹配值，统计出第一图像和第二图像之间各像素各自对应的匹配值，得到第一图像和第二图像之间像素的总匹配值，该总匹配值反应第一图像和第二图像的匹配度，该像素的总匹配值为0时，第一图像和第二图像一模一样；该像素的总匹配值越大，第一图像和第二图像越不相似。
在其中一个实施例中，该将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值的步骤，可以包括：
(1)将第一图像的像素覆盖在第二图像的像素上，得到覆盖后的第二目标图像；
(2)统计该第一图像的每一像素和第二目标图像对应的像素的差的平方之和，得到该第一图像和该第二图像之间像素的总匹配值。
其中，可通过平方差匹配算法确定第一图像和第二图像之间像素的总匹配值，平方差匹配算法为：
Figure PCTCN2021113175-appb-000001
该T(x′，y′)为模板图像矩阵，在本申请实施例中，该模板图像矩阵为第一图像作为模板图像，第一图像中的每一像素构成的矩阵，I(x，y)为源图像矩阵，该源图像矩阵为第二图像中的每一像素构成的矩阵，I(x+x′,y+y′)为将该第一图像的像素覆盖在该第二图像的像素上，得到的覆盖后的第二目标图像的每一像素构成的矩阵。在本步骤中该R(x,y)为像素的总匹配值。
以此，基于上述公式，服务器可统计每一图像的每一像素和第二目标图像对应关系的像素的差的平方之和，得到像素的总匹配值R(x,y)，该像素的总匹配值越接近于0，第一图像和第二图像越接近；该像素的总匹配值越大，第一图像和第二图像越不接近。
步骤103，当基于像素的总匹配值，确定第一图像和第二图像满足预设匹配条件时，对第二图像进行画面内容检测。
其中，可以设定预设匹配条件，该预设匹配条件可以为预设分数阈值，该预设分数阈值为界定第一图像和第二图像是否匹配的临界值，即像素的总匹配值高于预设分数阈值时，服务器确定第一图像和第二图像满足预设匹配条件，判定为第一图像和第二图像匹配成功，即第一图像和第二图像相似度满足条件，确定为游戏画面静止不动，当像素的总匹配值不高于预设分数阈值时，服务器确定第一图像和第二图像之间像素不满足预设匹配条件，判定为第一图像和第二图像匹配失败，即第一图像和第二图像相似度不满足条件，确定游戏画面仍在变化，未卡死。
进一步的，在确定第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，说明在间隔了预设时间段的视频流的游戏画面为静态不变的，即游戏视频流可能卡死，由于在一些游戏画面可能存在短期静态不变的画面，而卡死的游戏画面通常为纯色或者明暗度变化不大的图像，为了防止卡死误判，服务器需要进一步对第二图像进行画面内容检测。
如果图像中都是纯色，或者明暗度变化不大的图像，即第二图像中不存在画面内容，那么整体模糊度势必小于一定的范围，该范围可以为小于1.5，该画面内容检测可以为对第二图像进行图像模糊度检测。
在其中一个实施例中，可以通过拉普拉斯算法对第二图像的图像模糊度进行综合检测，实现相应的画面内容检测，该拉普拉斯算法用于图像的边缘检测，可以用于图像的明暗变化检测，以确定图像的模糊度。
在其中一个实施例中，当第一图像和第二图像之间像素的总匹配值不满足预设匹配 条件时，说明在间隔了预设时间段的视频流的游戏画面不为静态不变的，可以返回继续执行从视频流中截取间隔预设时间段的第一图像和第二图像的步骤继续进行检测。
在其中一个实施方例中，上述图像检测方法还包括基于像素的总匹配值，确定第一图像和所述第二图像满足预设匹配条件的步骤，其中，基于像素的总匹配值，确定第一图像和所述第二图像满足预设匹配条件的步骤的步骤可以包括：
(1)对该像素的总匹配值进行归一化处理，得到归一化处理后的匹配值；
(2)将归一化处理后的匹配值转换为分数值；
(3)当检测到该分数值大于预设分数阈值时，确定该第一图像和该第二图像满足预设匹配条件。
其中，为了防止该像素的总匹配值的区间较大，不利于用一个标准的分数去判定第一图像和第二图像之间是否匹配，因此，本申请实施例采用了归一化处理将该总匹配值缩放至0至1之间，归一化处理后的匹配值越接近于0，第一图像和第二图像越接近，归一化处理后的匹配值越接近于1，第一图像和第二图像越不接近。
在实际的使用过程中，为了更好的判定两者的相似度，可以将归一化处理后的匹配值转换为分数值，越接近于0，分数越高，越接近于1，分数越低，进而可以通过预设分数阈值进行匹配度界定，当服务器检测到该分数值大于预设分数阈值时，确定该第一图像和该第二图像满足预设匹配条件，该第一图像和该第二图像匹配成功。当服务器检测到该分数值小于或等于预设分数阈值时，确定该第一图像和该第二图像不满足预设匹配条件，该第一图像和该第二图像匹配失败。
在其中一个实施例中，该对该匹配值进行归一化处理，得到归一化处理后的匹配值的步骤，可以包括：
(2.1)统计该第一图像的每一像素和第二目标图像对应的像素的乘积之和；
(2.2)对统计的乘积之和进行二次根式计算，得到目标值；
(2.3)计算该像素的总匹配值和该目标值的比值，得到归一化处理后的匹配值。
其中，可通过以下公式对像素的总匹配值进行归一化处理，得到归一化处理后的匹配值，该公式具体可为归一化平方差匹配法：
Figure PCTCN2021113175-appb-000002
该T(x′，y′)为模板图像矩阵，在本申请实施例中，该模板图像矩阵为第一图像作为模板图像，第一图像中的每一像素构成的矩阵，I(x，y)为源图像矩阵，该源图像矩阵为第二图像中的每一像素构成的矩阵，I(x+x′,y+y′)为将该第一图像的像素覆盖在该第 二图像的像素上，得到的覆盖后的第二目标图像的每一像素构成的矩阵，该在本步骤中该R(x,y)为归一化处理后的匹配值。
以此，基于上述公式的分母部分，统计该第一图像的每一像素和第二目标图像对应的像素的乘积之和，对统计的乘积之和进行二次根式计算(即开2次根计算)，得到目标值，上述公式的分子为像素的总匹配值，计算分子的像素的总匹配值和该分母的目标值的比值，得到归一化处理后的匹配值，以此将匹配值缩小至0至1之间，该归一化处理后的匹配值越接近于0，第一图像和第二图像越接近，归一化处理后的匹配值越接近于1，第一图像和第二图像越不接近。
在其中一个实施例中，该将归一化处理后的匹配值转换为分数值的步骤，可以包括：
(3.1)计算预设基础值与该归一化处理后的匹配值的差值；
(3.2)将该差值乘以预设放大阈值，得到分数值。
其中，由于归一化处理后的匹配值处于0至1之间，由于后期需要设定阈值，接近0的部分不利于设定阈值进行判定，可以设置该预设基础值为1，计算该预设基础值与该归一化处理后的匹配值的差值，使得判定规则逆向调整，实现接近1为匹配，接近0为不匹配，更利于人工设定阈值进行判定，该差值越接近于0，第一图像和第二图像越不相似，差值越接近于1，第一图像和第二图像越接相似。
进一步的，将该差值乘以预设放大阈值，例如该预设放大阈值可以为1000，使得1000为最大分数，可以设定预设分数阈值为950，在检测到分数值大于预设分数阈值时，判定为检测到该第一图像和该第二图像匹配。
在其中一个实施例中，该对第二图像进行画面内容检测的步骤，可以包括：
(4.1)将该第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像；
(4.2)对该待检测图像进行模糊处理，得到模糊处理后的目标待检测图像；
(4.3)计算该目标待检测图像的目标模糊度；
(4.4)当检测到该目标模糊度低于预设模糊度阈值时，判定为检测到该第二图像中不存在画面内容。
其中，由于第二图像的边框区域中往往会存在标题等内容，为了排除干扰，可以先将该第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像。
服务器可通过数据平滑技术(data smoothing)进行高斯模糊处理，可以理解地，可将计算出的权重对相应像素进行加权处理，以实现图像的平滑处理，其中，该权重可为周边像素的平均值。以此，服务器可以对该待检测图像进行高斯模糊处理，对待检测图像进行平滑处理，得到高斯模糊处理后的目标待检测图像，使得后续的模糊度计算更为准确。
进一步的，服务器可以计算该目标待检测图像的目标模糊度，在一实施方式中，服务器可以通过拉普拉斯算法，计算出目标待检测图像的目标模糊度，该目标模糊度越低，说明像素变化率越低，表明第二图像中不存在画面内容的概率越大，该目标模糊度越高，说明像素变化率越大，第二图像中不存在画面内容的概率越低。
可以设定界定第二图像是否存在画面内容的临界值为预设模糊度阈值，当检测到该目标模糊度低于预设模糊度阈值时，服务器判定为检测到该第二图像中不存在画面内容。
在其中一个实施例中，该计算该目标待检测图像的目标模糊度的步骤，可以包括：
(5.1)通过拉普拉斯算法对该目标待检测图像进行计算，得到波形数据集；
(5.2)计算该波形数据集的平均值，并根据该平均值得到该波形数据集对应的标准差；
(5.3)将该标准差确定为目标模糊度。
其中，服务器可以通过拉普拉斯算法中的二阶微分得到目标待检测图像中每一像素的明暗值的波形数据集，像素较亮的波形数据为负数，像素较暗的波形数据为正数。
进一步的，服务器计算出该波形数据集的平均值，该标准差为方差的算数平方根，该标准差可以反映出该波形数据集的离散程度，标准差越大，说明波形数据集中大部分数据与平均值的差异较大，标准差越小，说明波形数据集中大部分数据与平均值的差异越小，以此，服务器可以通过该标准差判断第二图像内的像素的明暗度是否有较大的变化，即根据平均值得到波形数据集对应的标准差，将该标准差确定为目标模糊度。
步骤104，在检测到第二图像中不存在画面内容时，确定视频流出现异常。
其中，在服务器检测到第二图像中不存在画面内容时，说明该第二图像中像素的整体明暗度的变化处于一定的范围内，即判定为第二图像基本没有任何像素变化，画面没有内容，为纯色画面，确定为视频流出现异常，为游戏卡死。
由上述可知，本申请实施例通过从视频流中截取间隔预设时间段的第一图像和第二图像；将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值；当第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，对第二图像进行画面内容检测；在检测到第二图像中不存在画面内容时，确定视频流出现异常。以此，可以采用图像识别的方式，对视频流间隔预设时间段的图像画面进行检测，当检测到间隔预设时间段的图像画面不变时，对第二图像进行画面内容检测，在第二图像同时不存在画面内容时，判定为视频流出现异常，相对于现有技术中对于中央处理器的使用频率进行异常判定，且异常判定的阈值很难进行准确设置，检测准确率较差的方案而言，本申请实施例可以在不侵入游戏的代码设计或日志提取的基础上，采用图像之间像素的总匹配值确定视频画面是否静止不动，然后将静止不动的图像进行内容检测，确定视频流是否卡死不动，可以兼容各种颜色和亮度，不会因为画面亮度和颜色变化而导致检测不准，极大的提升了图像检测的准确率。
以下将举例作进一步详细说明。
在本实施例中，将以该图像检测装置具体集成在服务器中，该服务器为云主机为例进行说明，具体参照以下说明。
请参阅图4，图4为本申请实施例提供的图像检测方法的另一流程示意图。该方法流程可以包括：
步骤201，服务器从视频流中截取间隔预设时间段的第一图像和第二图像。
其中，本申请的视频流可以为云游戏视频流，由于该云游戏视频流对应的游戏画面容易出现游戏画面渲染负荷较大或者临时不兼容的情况，使得游戏启动时或者游戏过程中卡死。
为了防止游戏卡死未进行修复使得用户等待时间过长，本申请实施例可以通过服务器定期从视频流中截取间隔10秒的第一游戏图像和第二游戏图像，该第一图像和第二图像的像素数量是相同的。
步骤202，服务器将第一图像的像素覆盖在第二图像的像素上，得到覆盖后的第二目标图像，统计第一图像的每一像素和第二目标图像对应的像素的差的平方之和，得到第一图像和第二图像之间像素的总匹配值。
其中，可通过如下公式确定第一图像和第二图像之间像素的总匹配值，该公式具体可为平方差匹配算法：
Figure PCTCN2021113175-appb-000003
该T(x′，y′)为模板图像矩阵，在本申请实施例中，该模板图像矩阵为第一图像作为模板图像，第一图像中的每一像素构成的矩阵，I(x，y)为源图像矩阵，该源图像矩阵为第二图像中的每一像素构成的矩阵，I(x+x′,y+y′)为将该第一图像的像素覆盖在该第二图像的像素上，得到的覆盖后的第二目标图像的每一像素构成的矩阵，在本步骤中该R(x,y)为像素的总匹配值。
以此，服务器将第一图像的像素覆盖在第二图像的像素上，得到覆盖后的第二目标图像，并基于上述公式，统计每一图像的每一像素和第二目标图像对应关系的像素的差的平方之和，得到像素的总匹配值R(x,y)，该像素的总匹配值越接近于0，表明第一图像和第二图像越接近，该像素的总匹配值越大，表明第一图像和第二图像越不接近。
步骤203，服务器统计第一图像的每一像素和第二目标图像对应的像素的乘积之和，对统计的乘积之和进行二次根式计算，得到目标值，计算匹配值和目标值的比值，得到归一化处理后的匹配值。
其中，可通过如下公式对总匹配值进行归一化处理，该公式具体可为归一化平方差匹配法：
Figure PCTCN2021113175-appb-000004
该T(x′，y′)为模板图像矩阵，在本申请实施例中，该模板图像矩阵为第一图像作 为模板图像，第一图像中的每一像素构成的矩阵，I(x，y)为源图像矩阵，该源图像矩阵为第二图像中的每一像素构成的矩阵，I(x+x′,y+y′)为将该第一图像的像素覆盖在该第二图像的像素上，得到的覆盖后的第二目标图像的每一像素构成的矩阵，该在本步骤中该R(x,y)为归一化处理后的匹配值。
以此，基于上述公式的分母部分，统计该第一图像的每一像素和第二目标图像对应的像素的乘积之和，对统计的乘积之和进行二次根式计算，得到目标值，计算分子的像素的总匹配值和该分母的目标值的比值，得到归一化处理后的匹配值，以此将匹配值缩小至0至1之间，该归一化处理后的匹配值越接近于0，第一图像和第二图像越接近，归一化处理后的匹配值越接近于1，第一图像和第二图像越不接近。
步骤204，服务器计算预设基础值与归一化处理后的匹配值的差值，将差值乘以预设放大阈值，得到分数值，当检测到分数值大于预设分数阈值时，确定第一图像和第二图像满足预设匹配条件。
其中，该预设基础值可以为1，服务器计算该预设基础值1减去归一化处理后的匹配值的差值，使得归一化处理后判定规则逆向调整，差值越接近于0，第一图像和第二图像越不相似，差值越接近于1，第一图像和第二图像越相似。
以此，为了利于人工设定阈值进行判定，可以设定该预设放大阈值为1000，将差值乘以预设放大阈值1000，得到分数值的区间为0至1000之间，在分数值等于1000时，说明第一图像和第二图像完全相同，在实际的使用场景中，可以设定预设分数阈值为950，当检测到分数值大于预设分数阈值时，判定为检测到第一图像和第二图像匹配，确定第一图像和第二图像满足预设匹配条件。而当检测到分数值不大于预设分数阈值时，说明第一图像和第二图像有发生过变化，游戏视频流正常运行，可以返回执行服务器从视频流中截取间隔预设时间段的第一图像和第二图像的步骤，继续进行检测。
步骤205，服务器对第一图像和第二图像间隔预设时间段进行像素匹配，记录检测到第一图像和第二图像进行像素匹配的次数。
其中，由于部分游戏视频流可能会存在短时间加载的情况，例如12秒的加载时间，为了防止将短时间加载的情况误判定为游戏视频流卡死，服务器间隔预设时间段进行像素匹配，记录检测到第一图像和第二图像匹配成功的次数，发生一次匹配成功的条件可以为：间隔10秒的第一图像和第二图像满足预设匹配条件。其中，当第一图像和第二图像满足预设匹配条件时，可确定第一图像和第二图像匹配成功。
在步骤206中，服务器检测次数是否超过预设次数。
其中，该预设次数为界定游戏画面真正静止的界定值，例如3次，当服务器检测到次数超过预设次数时，说明第一图像和第二图像在30秒内均未发生变化，处于画面静止的状态，执行步骤207。当服务器检测到次数未超过预设次数时，说明第一图像和第二图像在30秒内发生变化，游戏视频流未卡死，返回执行步骤201，继续进行检测。步骤207，服务器将第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像，对待检测图像进 行模糊处理，得到模糊处理后的目标待检测图像。
其中，当服务器检测到次数超过预设次数时，说明第一图像和第二图像在30秒内均未发生变化，处于画面静止的状态，即游戏视频流可能卡死，由于卡死的游戏画面通常为纯色或者明暗度变化不大的图像，为了防止卡死误判，需要进一步第二图像进行检测，可以将第二图像的边框区域进行滤除处理，请一并参阅图5a。第二图像1的边框上的标题栏还存在“王者X耀”，如果将此部分进行处理，会影响处理结果，以此，服务器可以将第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像11。
进一步的，可以对该待检测图像11进行高斯模糊处理，得到高斯模糊处理后的目标待检测图像，为了更好的解释高斯模糊处理的原理，请一并参阅图5b，假设待检测图像11的部分区域2的中心点的像素的RGB值为2，经过高斯模糊处理之后，得到高斯模糊处理后的目标待检测图像的部分区域3的中心点的像素的RGB值经过参考周边像素的平均值，变成1，使得像素点失去部分细节，实现图像模糊处理，该高斯模糊处理具有一个关键参数ksize，代表模糊半径，该半径越大，高斯模糊处理的效果越模糊，该模糊半径为参考周边像素数量值，本申请实施例可以将该模糊半径设定为1。
步骤208，服务器通过拉普拉斯算法对目标待检测图像进行计算，得到波形数据集，计算波形数据集的平均值，并根据平均值得到波形数据集对应的标准差，将标准差确定为目标模糊度。
其中，服务器还可以将目标待检测图像进行灰度处理，灰度就是没有色彩，将该目标待检测图像的RGB色彩分量设置为相等，得到灰度图，指定拉普拉斯算法的算子大小与模糊半径相同，通过拉普拉斯算法对灰度图进行计算，得到由灰度图中每一像素的波形数据组成的波形数据集，该波形数据可以反映出像素的明暗度。
进一步的，可以计算波形数据集的平均值，并根据该平均值得到波形数据集对应的标准差，该标准差反映出波形数据集中的波形数据与平均值的差异情况，该标准差越大，说明波形数据集中大部分数据与平均值的差异较大，标准差越小，说明波形数据集中大部分数据与平均值的差异越小。
步骤209，当服务器检测到目标模糊度低于预设模糊度阈值时，确定第二图像中不存在画面内容，确定视频流出现异常。
其中，该预设模糊度阈值为界定第二图像中是否存在画面内容的临界值，本申请实施例中，可以设定该预设模糊度阈值为1.5，当服务器检测到目标模糊度低于预设模糊度阈值时，说明第二图像中像素的明暗度变化率越低，第二图像中不存在画面内容，在纯色画面，确定为游戏视频流出现异常，为游戏卡死，可以采取对应的修复手段。
在一些实施方式中，本申请实施例可以将判定为卡死的第二图像上传给卷积神经网络(Convolutional Neural Networks，CNN)模型进行学习，以此，通过不断地学习，卷积神经网络可以学习识别出游戏视频流卡死的画面的能力，实现快速识别。
在其中一个实施例中，从视频流中截取间隔预设时间段的第一图像和第二图像，包括 在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像；对第二图像进行画面内容检测之前，上述方法还包括：当基于像素的总匹配值，确定当前检测时刻所对应的第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，确定当前匹配结果为匹配成功；获取存储的历史匹配结果；历史匹配结果为在历史检测时刻对截取的间隔预设时间段的第一图像和第二图像进行像素匹配处理，得到的匹配结果；基于历史匹配结果和当前匹配结果，确定在预设检测时段内匹配成功的次数；及当基于像素的总匹配值，确定第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，对第二图像进行画面内容检测，包括：当在预设检测时段内匹配成功的次数大于或等于预设次数时，对当前第二图像进行画面内容检测。
其中，由于部分游戏视频流可能会存在短时间加载的情况，例如12秒的加载时间，为了防止将短时间加载的情况误判定为游戏视频流卡死，服务器间隔预设时间段进行像素匹配，记录检测到第一图像和第二图像匹配成功的次数，发生一次匹配成功的条件可以为：间隔10秒的第一图像和第二图像满足预设匹配条件。其中，当第一图像和第二图像满足预设匹配条件时，可确定第一图像和第二图像匹配成功。
服务器在当前检测时刻从视频流中截取间隔预设时间段的第一图像和第二图像。其中，当前检测时刻指的是当前时间点，第一图像和第二图像间隔预设时间段，比如，第一图像和第二图像间隔10秒。服务器通过上述方法确定在当前时刻从视频流中截取的第一图像和第二图像之间像素的总匹配值是否满足预设匹配条件，并在确定满足预设匹配条件时，也即确定当前匹配结果为匹配成功时，获取存储的历史匹配结果。其中，历史匹配结果为在历史检测时刻对截取的间隔预设时间段的第一图像和第二图像进行像素匹配处理，得到的匹配结果。历史时刻指的是在当前时刻之前的时刻。
容易理解地，服务器每隔一段时间从视频流中截取出第一图像和第二图像，并对第一图像和第二图像进行像素匹配处理，得到匹配结果，将匹配结果存储于存储器中，从而在确定在当前时刻截取的第一图像和第二图像匹配成功时，服务器可从存储器中获取预设检测时间段内的历史匹配结果。其中，预设检测时间段指的是以当前检测时刻为终点的一段预设时间，例如，预设检测时间段可以为包括当前检测时刻的前30秒。
服务器根据历史匹配结果和当前匹配结果，判断在预设检测时段内匹配成功的次数。其中，该预设次数为界定游戏画面真正静止的界定值，例如3次，当服务器检测到次数大于或等于预设次数时，说明第一图像和第二图像在30秒内均未发生变化，处于画面静止的状态，此时服务器对当前第二图像进行画面内容检测。其中，服务器可通过上述实施例中对第二图像进行画面内容检测的方法，对当前第二图像进行画面检测。
当服务器检测到次数小于预设次数时，说明第一图像和第二图像在30秒内发生变化，游戏视频流未卡死，进入下一个检测时刻，并将所述下一个检测时刻作为当前时刻，返回在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像的步骤继续进行检测，直至所述视频流播放完毕时停止。。
在其中一个实施例中，当在当前时刻从视频流中截取的第一图像和第二图像之间像素的总匹配值不满足预设匹配条件时，服务器返回在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像的步骤继续进行检测。
由上述可知，本申请实施例通过从视频流中截取间隔预设时间段的第一图像和第二图像；将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值；当第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，对第二图像进行画面内容检测；在检测到第二图像中不存在画面内容时，确定视频流出现异常。以此，可以采用图像识别的方式，对视频流间隔预设时间段的图像画面进行检测，当检测到间隔预设时间段的图像画面不变时，对第二图像进行画面内容检测，在第二图像同时不存在画面内容时，判定为视频流出现异常，相对于现有技术中对于中央处理器的使用频率进行异常判定，且异常判定的阈值很难进行准确设置，检测准确率较差的方案而言，本申请实施例可以在不侵入游戏的代码设计或日志提取的基础上，采用图像之间像素的总匹配值确定视频画面是否静止不动，然后将静止不动的图像进行内容检测，确定视频流是否卡死不动，可以兼容各种颜色和亮度，不会因为画面亮度和颜色变化而导致检测不准，极大的提升了图像检测的准确率。
为了更好地实施以上方法，，本申请实施例还提供一种图像检测方法的装置。该装置可以集成在服务器中。其中名词的含义与上述图像检测方法中相同，具体实现细节可以参考方法实施例中的说明。
请参阅图6，图6为本申请实施例提供的图像检测装置的结构示意图，其中该图像检测装置可以包括截取单元301、匹配单元302、检测单元303、及确定单元304等。
截取单元301，用于从视频流中截取间隔预设时间段的第一图像和第二图像。
匹配单元302，用于将该第一图像和该第二图像进行像素匹配获得第一图像和该第二图像之间像素的总匹配值。
在其中一个实施例中，该匹配单元302，包括：
覆盖子单元，用于将该第一图像的像素覆盖在该第二图像的像素上，得到覆盖后的第二目标图像；
统计子单元，用于统计该第一图像的每一像素和第二目标图像对应的像素的差的平方之和，得到该第一图像和该第二图像之间像素的总匹配值。
检测单元303，用于当基于像素的总匹配值，确定该第一图像和该第二图像满足预设匹配条件时，对该第二图像进行画面内容检测。
在其中一个实施例中，该检测单元303，包括：
归一化子单元，用于对该总匹配值进行归一化处理，得到归一化处理后的匹配值；
转换子单元，用于将归一化处理后的匹配值转换为分数值；
确定子单元，用于当检测到该分数值大于预设分数阈值时，确定该第一图像和该第二图像满足预设匹配条件，对该第二图像进行画面内容检测。
在其中一个实施例中，确定子单元还用于当检测到分数值小于或等于预设分数阈值时，确定第一图像和第二图像不满足预设匹配条件。
在其中一个实施例中，该归一化子单元还用于统计该第一图像的每一像素和第二目标图像对应的像素的乘积之和；对统计的乘积之和进行二次根式计算，得到目标值；计算该像素的总匹配值和该目标值的比值，得到归一化处理后的匹配值。
在其中一个实施例中，该转换子单元还用于计算预设基础值与该归一化处理后的匹配值的差值；将该差值乘以预设放大阈值，得到分数值。
在其中一个实施例中，该检测单元303，包括：
滤除子单元，用于当该第一图像和该第二图像之间像素的总匹配值满足预设匹配条件时，将该第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像；
处理子单元，用于对该待检测图像进行模糊处理，得到模糊处理后的目标待检测图像；
计算子单元，用于计算该目标待检测图像的目标模糊度；
确定子单元，用于当检测到该目标模糊度低于预设模糊度阈值时，确定该第二图像中不存在画面内容。
在其中一个实施例中，该计算子单元还用于通过拉普拉斯算法对该目标待检测图像进行计算，得到波形数据集；计算该波形数据集的平均值，并根据该平均值得到该波形数据集对应的标准差；将该标准差确定为目标模糊度。
确定单元304，用于在检测到该第二图像中不存在画面内容时，确定该视频流出现异常。
在其中一个实施例中，该图像检测装置还可以包括记录单元，用于：对该第一图像和该第二图像间隔预设时间段进行像素匹配；记录检测到该第一图像和该第二图像进行像素匹配的次数；当检测到该次数超过预设次数时，执行对该第二图像进行画面内容检测的步骤；当检测到该次数不超过预设次数时，返回执行从视频流中截取间隔预设时间段的第一图像和第二图像的步骤。
在其中一个实施例中，该图像检测装置还用于在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像；当基于像素的总匹配值，确定当前检测时刻所对应的第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，确定当前匹配结果为匹配成功；获取存储的历史匹配结果；历史匹配结果为在历史检测时刻对截取的间隔预设时间段的第一图像和第二图像进行像素匹配处理，得到的匹配结果；基于历史匹配结果和所述当前匹配结果，确定在预设检测时段内匹配成功的次数；当在预设检测时段内匹配成功的次数大于或等于预设次数时，对当前第二图像进行画面内容检测。
在其中一个实施例中，该图像检测装置还用于当在预设检测时段内匹配成功的次数小于预设次数时，进入下一个检测时刻，并将下一个检测时刻作为当前时刻，返回在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像的步骤继续进行检测， 直至视频流播放完毕时停止。
以上各个单元的具体实施可参见前面的实施例，在此不再赘述。
由上述可知，本申请实施例通过截取单元301从视频流中截取间隔预设时间段的第一图像和第二图像；匹配单元302将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值；检测单元303当第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，对第二图像进行画面内容检测；确定单元304在检测到第二图像中不存在画面内容时，确定视频流出现异常。以此，可以采用图像识别的方式，对视频流间隔预设时间段的图像画面进行检测，当检测到间隔预设时间段的图像画面不变时，对第二图像进行画面内容检测，在第二图像同时不存在画面内容时，判定为视频流出现异常，相对于现有技术中对于中央处理器的使用频率进行异常判定，且异常判定的阈值很难进行准确设置，检测准确率较差的方案而言，本申请实施例可以在不侵入游戏的代码设计或日志提取的基础上，采用图像之间像素的总匹配值确定视频画面是否静止不动，然后将静止不动的图像进行内容检测，确定视频流是否卡死不动，可以兼容各种颜色和亮度，不会因为画面亮度和颜色变化而导致检测不准，极大的提升了图像检测的准确率。
本申请实施例还提供一种服务器，如图7所示，其示出了本申请实施例所涉及的服务器的结构示意图，具体来讲：
该服务器可以为云主机，可以包括一个或者一个以上处理核心的处理器401、一个或一个以上计算机可读存储介质的存储器402、电源403和输入单元404等部件。本领域技术人员可以理解，图7中示出的服务器结构并不构成对服务器的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。其中：
处理器401是该服务器的控制中心，利用各种接口和线路连接整个服务器的各个部分，通过运行或执行存储在存储器402内的软件程序和/或模块，以及调用存储在存储器402内的数据，执行服务器的各种功能和处理数据，从而对服务器进行整体监控。可选的，处理器401可包括一个或多个处理核心；优选的，处理器401可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器401中。
存储器402可用于存储软件程序以及模块，处理器401通过运行存储在存储器402的软件程序以及模块，从而执行各种功能应用以及数据处理。存储器402可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据服务器的使用所创建的数据等。此外，存储器402可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。相应地，存储器402还可以包括存储器控制器，以提供处理器401对存储器402的访问。
服务器还包括给各个部件供电的电源403，优选的，电源403可以通过电源管理系统与处理器401逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功 能。电源403还可以包括一个或一个以上的直流或交流电源、再充电系统、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。
该服务器还可包括输入单元404，该输入单元404可用于接收输入的数字或字符信息，以及产生与用户设置以及功能控制有关的键盘、鼠标、操作杆、光学或者轨迹球信号输入。
尽管未示出，服务器还可以包括显示处理器等，在此不再赘述。具体在本实施例中，服务器中的处理器401会按照如下的指令，将一个或一个以上的应用程序的进程对应的可执行文件加载到存储器402中，并由处理器401来运行存储在存储器402中的应用程序，从而实现各种功能，如下：
从视频流中截取间隔预设时间段的第一图像和第二图像；将该第一图像和该第二图像进行像素匹配获得该第一图像和该第二图像之间像素的总匹配值；当基于像素的总匹配值，确定第一图像和第二图像满足预设匹配条件时，对该第二图像进行画面内容检测；在检测到该第二图像中不存在画面内容时，确定该视频流出现异常。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见上文针对图像检测方法的详细描述，此处不再赘述。
由上述可知，本申请实施例的服务器可以通过从视频流中截取间隔预设时间段的第一图像和第二图像；将第一图像和第二图像进行像素匹配获得第一图像和第二图像之间像素的总匹配值；当第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，对第二图像进行画面内容检测；在检测到第二图像中不存在画面内容时，确定视频流出现异常。以此，可以采用图像识别的方式，对视频流间隔预设时间段的图像画面进行检测，当检测到间隔预设时间段的图像画面不变时，对第二图像进行画面内容检测，在第二图像同时不存在画面内容时，判定为视频流出现异常，相对于现有技术中对于中央处理器的使用频率进行异常判定，且异常判定的阈值很难进行准确设置，检测准确率较差的方案而言，本申请实施例可以在不侵入游戏的代码设计或日志提取的基础上，采用图像之间像素的总匹配值确定视频画面是否静止不动，然后将静止不动的图像进行内容检测，确定视频流是否卡死不动，可以兼容各种颜色和亮度，不会因为画面亮度和颜色变化而导致检测不准，极大的提升了图像检测的准确率。
在一些实施例中，提供了一种计算机设备，包括存储器和一个或多个处理器，存储器存储有计算机可读指令，计算机可读指令被所述处理器执行时，使得一个或多个处理器执行执行上述通话音频混音处理方法的步骤。此处通话音频混音处理方法的步骤可以是上述各个实施例的图像检测方法中的步骤。
在一些实施例中，提供了一个或多个存储有计算机可读指令的非易失性可读存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行执行上述图像检测方法的步骤。此处图像检测方法的步骤可以是上述各个实施例的图像检测方法中的步骤。
根据本申请的一个方面，提供了一种计算机程序产品或计算机程序，该计算机程序产 品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述实施例提供的各种可选实现方式中提供的方法。
以上各个操作的具体实施可参见前面的实施例，在此不再赘述。
其中，该计算机可读存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)、磁盘或光盘等。

Claims (20)

  1. 一种图像检测方法，由服务器执行，所述方法包括：
    从视频流中截取间隔预设时间段的第一图像和第二图像；
    将所述第一图像和所述第二图像进行像素匹配，获得所述第一图像和所述第二图像之间像素的总匹配值；
    当基于所述像素的总匹配值，确定所述第一图像和所述第二图像满足预设匹配条件时，对所述第二图像进行画面内容检测；及
    在检测到所述第二图像中不存在画面内容时，确定所述视频流出现异常。
  2. 根据权利要求1所述的图像检测方法，其特征在于，所述将所述第一图像和所述第二图像进行像素匹配，获得所述第一图像和所述第二图像之间像素的总匹配值，包括：
    将所述第一图像的像素覆盖在所述第二图像的像素上，得到覆盖后的第二目标图像；
    统计所述第一图像的每一像素和第二目标图像对应的像素的差的平方之和，得到所述第一图像和所述第二图像之间像素的总匹配值。
  3. 根据权利要求1所述的图像检测方法，其特征在于，所述方法还包括：
    对所述像素的总匹配值进行归一化处理，得到归一化处理后的匹配值；
    将归一化处理后的匹配值转换为分数值；及
    当检测到所述分数值大于预设分数阈值时，确定所述第一图像和所述第二图像满足预设匹配条件。
  4. 根据权利要求3所述的图像检测方法，其特征在于，所述方法还包括：
    当检测到所述分数值小于或等于预设分数阈值时，确定所述第一图像和所述第二图像不满足预设匹配条件。
  5. 根据权利要求3所述的图像检测方法，其特征在于，所述对所述像素的总匹配值进行归一化处理，得到归一化处理后的匹配值，包括：
    统计所述第一图像的每一像素和第二目标图像对应的像素的乘积之和；
    对统计的乘积之和进行二次根式计算，得到目标值；及
    计算所述像素的总匹配值和所述目标值的比值，得到归一化处理后的匹配值。
  6. 根据权利要求3所述的图像检测方法，其特征在于，所述将归一化处理后的匹配值转换为分数值，包括：
    计算预设基础值与所述归一化处理后的匹配值的差值；及
    将所述差值乘以预设放大阈值，得到分数值。
  7. 根据权利要求1所述的图像检测方法，其特征在于，所述对所述第二图像进行画面内容检测，包括：
    将所述第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像；
    对所述待检测图像进行模糊处理，得到模糊处理后的目标待检测图像；
    计算所述目标待检测图像的目标模糊度；及
    当检测到所述目标模糊度低于预设模糊度阈值时，确定所述第二图像中不存在画面内容。
  8. 根据权利要求7所述的图像检测方法，其特征在于，所述计算所述目标待检测图像的目标模糊度，包括：
    通过拉普拉斯算法对所述目标待检测图像进行计算，得到波形数据集；
    计算所述波形数据集的平均值，并根据所述平均值得到所述波形数据集对应的标准差；及
    将所述标准差确定为目标模糊度。
  9. 根据权利要求1所述的图像检测方法，其特征在于，所述从视频流中截取间隔预设时间段的第一图像和第二图像，包括：
    在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像；
    所述对所述第二图像进行画面内容检测之前，所述方法还包括：
    当基于所述像素的总匹配值，确定当前检测时刻所对应的第一图像和第二图像之间像素的总匹配值满足预设匹配条件时，确定当前匹配结果为匹配成功；
    获取存储的历史匹配结果；所述历史匹配结果为在历史检测时刻对截取的间隔预设时间段的第一图像和第二图像进行像素匹配处理，得到的匹配结果；
    基于所述历史匹配结果和所述当前匹配结果，确定在预设检测时段内匹配成功的次数；及
    所述当基于所述像素的总匹配值，确定所述第一图像和所述第二图像之间像素的总匹配值满足预设匹配条件时，对所述第二图像进行画面内容检测，包括：
    当在预设检测时段内匹配成功的次数大于或等于预设次数时，对所述当前第二图像进行画面内容检测。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    当在预设检测时段内匹配成功的次数小于预设次数时，进入下一个检测时刻，并将所述下一个检测时刻作为当前时刻，返回所述在当前检测时刻，从视频流中截取间隔预设时间段的第一图像和第二图像的步骤继续进行检测，直至所述视频流播放完毕时停止。
  11. 一种图像检测装置，其特征在于，包括：
    截取单元，用于从视频流中截取间隔预设时间段的第一图像和第二图像；
    匹配单元，用于将所述第一图像和所述第二图像进行像素匹配获得第一图像和所述第二图像之间像素的总匹配值；
    检测单元，用于当基于所述像素的总匹配值，确定所述第一图像和所述第二图像满足预设匹配条件时，对所述第二图像进行画面内容检测；及
    确定单元，用于在检测到所述第二图像中不存在画面内容时，确定所述视频流出现异常。
  12. 根据权利要求11所述的图像检测装置，其特征在于，所述匹配单元，包括：
    覆盖子单元，用于将所述第一图像的像素覆盖在所述第二图像的像素上，得到覆盖后的第二目标图像；及
    统计子单元，用于统计所述第一图像的每一像素和第二目标图像对应的像素的差的平方之和，得到所述第一图像和所述第二图像之间像素的总匹配值。
  13. 根据权利要求11所述的图像检测装置，其特征在于，所述检测单元，包括：
    归一化子单元，用于对所述像素的总匹配值进行归一化处理，得到归一化处理后的匹配值；
    转换子单元，用于将归一化处理后的匹配值转换为分数值；及
    确定子单元，用于当检测到所述分数值大于预设分数阈值时，确定所述第一图像和所述第二图像满足预设匹配条件，对所述第二图像进行画面内容检测。
  14. 根据权利要求13所述的图像检测装置，其特征在于，所述确定子单元还用于当检测到所述分数值小于或等于预设分数阈值时，确定所述第一图像和所述第二图像不满足预设匹配条件。
  15. 根据权利要求13所述的图像检测装置，其特征在于，所述归一化子单元还用于：
    统计所述第一图像的每一像素和第二目标图像对应的像素的乘积之和；
    对统计的乘积之和进行二次根式计算，得到目标值；及
    计算所述像素的总匹配值和所述目标值的比值，得到归一化处理后的匹配值。
  16. 根据权利要求13所述的图像检测装置，其特征在于，所述转换子单元还用于：
    计算预设基础值与所述归一化处理后的匹配值的差值；及
    将所述差值乘以预设放大阈值，得到分数值。
  17. 根据权利要求11所述的图像检测装置，其特征在于，所述检测单元，包括：
    滤除子单元，用于当所述第一图像和所述第二图像像素匹配值满足预设匹配条件时，将所述第二图像的边框区域进行滤除处理，得到滤除处理后的待检测图像；
    处理子单元，用于对所述待检测图像进行模糊处理，得到模糊处理后的目标待检测图像；
    计算子单元，用于计算所述目标待检测图像的目标模糊度；及
    确定子单元，用于当检测到所述目标模糊度低于预设模糊度阈值时，确定所述第二图像中不存在画面内容。
  18. 根据权利要求17所述的图像检测装置，其特征在于，所述计算子单元还用于：
    通过拉普拉斯算法对所述目标待检测图像进行计算，得到波形数据集；
    计算所述波形数据集的平均值，并根据所述平均值得到所述波形数据集对应的标准差；及
    将所述标准差确定为目标模糊度。
  19. 一个或多个存储有计算机可读指令的非易失性可读存储介质，所述计算机可读指 令被一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1-10任一项所述方法的步骤。
  20. 一种计算机设备，包括存储器，处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如权利要求1-10任一项所述方法的步骤。
PCT/CN2021/113175 2020-09-25 2021-08-18 图像检测方法、装置、计算机设备及计算机可读存储介质 WO2022062772A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP21871145.5A EP4129430A4 (en) 2020-09-25 2021-08-18 IMAGE DETECTION METHOD AND APPARATUS, COMPUTER DEVICE AND COMPUTER-READABLE STORAGE MEDIUM
JP2022567668A JP7490082B2 (ja) 2020-09-25 2021-08-18 画像検出方法、装置、コンピュータプログラム及びコンピュータ機器
US17/901,707 US20220415038A1 (en) 2020-09-25 2022-09-01 Image detection method and apparatus, computer device, and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011024483.0 2020-09-25
CN202011024483.0A CN112206508A (zh) 2020-09-25 2020-09-25 一种图像检测方法、装置及计算机可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/901,707 Continuation US20220415038A1 (en) 2020-09-25 2022-09-01 Image detection method and apparatus, computer device, and computer-readable storage medium

Publications (1)

Publication Number Publication Date
WO2022062772A1 true WO2022062772A1 (zh) 2022-03-31

Family

ID=74051130

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/113175 WO2022062772A1 (zh) 2020-09-25 2021-08-18 图像检测方法、装置、计算机设备及计算机可读存储介质

Country Status (5)

Country Link
US (1) US20220415038A1 (zh)
EP (1) EP4129430A4 (zh)
JP (1) JP7490082B2 (zh)
CN (1) CN112206508A (zh)
WO (1) WO2022062772A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116156157A (zh) * 2023-04-24 2023-05-23 长沙海信智能系统研究院有限公司 一种摄像头遮挡异常的检测方法及电子设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112206508A (zh) * 2020-09-25 2021-01-12 腾讯科技（深圳）有限公司 一种图像检测方法、装置及计算机可读存储介质
CN113112448A (zh) * 2021-02-25 2021-07-13 惠州华阳通用电子有限公司 一种显示画面检测方法及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010056729A (ja) * 2008-08-27 2010-03-11 Mitsubishi Electric Corp フレーム補間装置及び方法並びに画像表示装置
CN105872760A (zh) * 2015-12-02 2016-08-17 乐视网信息技术（北京）股份有限公司 一种视频播放监测方法及装置
CN106375755A (zh) * 2016-08-31 2017-02-01 福建省天奕网络科技有限公司 一种视频数据卡顿检测方法及系统
CN106599783A (zh) * 2016-11-09 2017-04-26 浙江宇视科技有限公司 一种视频遮挡检测方法及装置
CN107509107A (zh) * 2017-09-20 2017-12-22 广州视源电子科技股份有限公司 视频播放故障的检测方法、装置及设备、可读介质
CN109996063A (zh) * 2019-04-04 2019-07-09 广东省安心加科技有限公司 视频图像花屏检测方法、装置、计算机设备和存储介质
CN112206508A (zh) * 2020-09-25 2021-01-12 腾讯科技（深圳）有限公司 一种图像检测方法、装置及计算机可读存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003108051A (ja) * 2001-09-28 2003-04-11 Namco Ltd ゲーム装置
JP5600954B2 (ja) 2010-02-10 2014-10-08 セイコーエプソン株式会社 検査システム、その方法及びプログラム
JP5557189B2 (ja) 2010-03-31 2014-07-23 トヨタ自動車株式会社 位置推定装置、位置推定方法及びプログラム
US8786625B2 (en) * 2010-09-30 2014-07-22 Apple Inc. System and method for processing image data using an image signal processor having back-end processing logic
CN104166614B (zh) * 2013-05-17 2017-12-15 腾讯科技（深圳）有限公司 一种移动设备的帧率探测方法和相关装置
CN103810106A (zh) * 2014-03-06 2014-05-21 安一恒通（北京）科技有限公司 一种测试界面流畅度的方法和装置
CN107657591A (zh) * 2017-09-05 2018-02-02 维沃移动通信有限公司 一种图像处理方法及移动终端
US11039146B2 (en) * 2018-03-01 2021-06-15 Arris Enterprises Llc Visual artifact detector
CN109062651A (zh) * 2018-08-01 2018-12-21 深圳Tcl新技术有限公司 Ui界面的图片加载方法、装置、存储介质及智能终端
CN109766953B (zh) 2019-01-22 2021-07-13 中国人民公安大学 对象识别方法及装置
CN111654756B (zh) * 2020-06-03 2021-06-11 腾讯科技（深圳）有限公司 卡顿检测方法、装置、设备及可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010056729A (ja) * 2008-08-27 2010-03-11 Mitsubishi Electric Corp フレーム補間装置及び方法並びに画像表示装置
CN105872760A (zh) * 2015-12-02 2016-08-17 乐视网信息技术（北京）股份有限公司 一种视频播放监测方法及装置
CN106375755A (zh) * 2016-08-31 2017-02-01 福建省天奕网络科技有限公司 一种视频数据卡顿检测方法及系统
CN106599783A (zh) * 2016-11-09 2017-04-26 浙江宇视科技有限公司 一种视频遮挡检测方法及装置
CN107509107A (zh) * 2017-09-20 2017-12-22 广州视源电子科技股份有限公司 视频播放故障的检测方法、装置及设备、可读介质
CN109996063A (zh) * 2019-04-04 2019-07-09 广东省安心加科技有限公司 视频图像花屏检测方法、装置、计算机设备和存储介质
CN112206508A (zh) * 2020-09-25 2021-01-12 腾讯科技（深圳）有限公司 一种图像检测方法、装置及计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4129430A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116156157A (zh) * 2023-04-24 2023-05-23 长沙海信智能系统研究院有限公司 一种摄像头遮挡异常的检测方法及电子设备
CN116156157B (zh) * 2023-04-24 2023-08-18 长沙海信智能系统研究院有限公司 一种摄像头遮挡异常的检测方法及电子设备

Also Published As

Publication number Publication date
CN112206508A (zh) 2021-01-12
JP7490082B2 (ja) 2024-05-24
EP4129430A1 (en) 2023-02-08
US20220415038A1 (en) 2022-12-29
JP2023525283A (ja) 2023-06-15
EP4129430A4 (en) 2023-11-01

Similar Documents

Publication Publication Date Title
WO2022062772A1 (zh) 图像检测方法、装置、计算机设备及计算机可读存储介质
US11170210B2 (en) Gesture identification, control, and neural network training methods and apparatuses, and electronic devices
US11057583B2 (en) Bullet comment processing method and apparatus, and storage medium
WO2019105337A1 (zh) 基于视频的人脸识别方法、装置、设备、介质及程序
US8818037B2 (en) Video scene detection
US11586842B2 (en) System and method for machine learning based video quality assessment
WO2022089170A1 (zh) 字幕区域识别方法、装置、设备及存储介质
US10863230B1 (en) Content stream overlay positioning
WO2022111201A1 (zh) 一种图像处理方法、装置及计算机可读存储介质
WO2020151156A1 (zh) 视频流播放方法、系统、计算机装置及可读存储介质
CN111479129A (zh) 直播封面的确定方法、装置、服务器、介质及系统
US11282179B2 (en) System and method for machine learning based video quality assessment
CN113507630B (zh) 比赛视频的拆条方法和装置
CN114845158A (zh) 视频封面的生成方法、视频发布方法及相关设备
CN112257729B (zh) 图像识别方法、装置、设备及存储介质
WO2023236757A1 (zh) 视频图像噪声评估方法及装置
CN108629766A (zh) 图像模糊度检测方法、装置、计算机设备及可读存储介质
CN114764821B (zh) 移动物体检测方法、装置、电子设备和存储介质
CN113066024B (zh) 图像模糊检测模型的训练方法、图像模糊检测方法及装置
US20220122341A1 (en) Target detection method and apparatus, electronic device, and computer storage medium
CN115294162A (zh) 目标识别方法、装置、设备及存储介质
EP3586240A1 (en) Apparatus and method of adjusting a sensitivity buffer of semi-supervised machine learning principals for remediation of issues
CN113627534A (zh) 识别动态图像的类型的方法、装置及电子设备
CN110971895A (zh) 视频抖动检测方法和装置
US20230196724A1 (en) Video frame analysis for targeted video browsing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21871145

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 21871145.5

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021871145

Country of ref document: EP

Effective date: 20221027

ENP Entry into the national phase

Ref document number: 2022567668

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE