CN112351337A - Video quality inspection method and device, computer equipment and storage medium - Google Patents

Video quality inspection method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN112351337A
CN112351337A CN202110001121.8A CN202110001121A CN112351337A CN 112351337 A CN112351337 A CN 112351337A CN 202110001121 A CN202110001121 A CN 202110001121A CN 112351337 A CN112351337 A CN 112351337A
Authority
CN
China
Prior art keywords
quality inspection
subtask
target video
target
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110001121.8A
Other languages
Chinese (zh)
Other versions
CN112351337B (en
Inventor
严石伟
丁凯
蒋楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202110001121.8A priority Critical patent/CN112351337B/en
Publication of CN112351337A publication Critical patent/CN112351337A/en
Application granted granted Critical
Publication of CN112351337B publication Critical patent/CN112351337B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44008Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • H04N21/4394Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams

Abstract

The application relates to parallel scheduling and batch processing of video quality inspection tasks, in particular to a video quality inspection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video; determining at least one quality inspection subtask corresponding to the target video according to the task configuration information; determining target video intervals respectively matched with the quality inspection subtasks from the target video; frame skipping processing is carried out on the corresponding target video interval according to frame skipping information corresponding to each quality inspection subtask respectively, and target quality inspection fragments corresponding to each quality inspection subtask are obtained; and scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are jointly used as execution objects. By adopting the method, the video quality inspection efficiency can be improved.

Description

Video quality inspection method and device, computer equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a video quality inspection method and apparatus, a computer device, and a storage medium.
Background
The quality inspection refers to detecting key links in the video, for example, in the process of insurance sales, a salesperson can collect the video in the process of insurance sales through a sound recording and video recording technology and perform quality inspection on key links such as a signature link and a certificate display link in the video so as to determine whether signature content in the signature link and certificate content displayed in the certificate display link meet requirements or not.
At present, quality inspection is mainly performed on videos in a manual inspection mode, an inspector browses the whole video from beginning to end, and the communication content and the display content in the whole video are inspected. However, since the video to be quality-checked is usually as long as ten and several minutes, it is inefficient to manually quality-check the video.
Disclosure of Invention
In view of the above, it is necessary to provide a video quality inspection method, an apparatus, a computer device, and a storage medium capable of improving quality inspection efficiency.
A video quality inspection method, the method comprising:
acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video;
determining at least one quality inspection subtask corresponding to the target video according to the task configuration information;
determining target video intervals respectively matched with the quality inspection subtasks from the target video;
frame skipping processing is carried out on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask;
and scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are used as execution objects together.
A video quality inspection device, the device comprising:
the subtask determining module is used for acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video; determining at least one quality inspection subtask corresponding to the target video according to the task configuration information;
the quality inspection fragment determining module is used for determining target video intervals respectively matched with all quality inspection subtasks from the target video; frame skipping processing is carried out on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask;
and the execution module is used for scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are jointly used as execution objects.
In one embodiment, the subtask determining module further includes a configuration module, configured to receive a target video and a target task configuration item, which are sent by the terminal and are to be subjected to quality inspection; the target task configuration item is obtained by selecting a plurality of candidate task configuration items in a task configuration page displayed by the terminal; and taking the received target task configuration item as task configuration information corresponding to the target video.
In one embodiment, the quality inspection fragment determining module further includes an interval determining module, configured to perform video cleaning processing on the target video to obtain an audio file of the target video; determining target keywords respectively corresponding to each quality inspection subtask; performing keyword monitoring on an audio file in the target video to determine the occurrence time of the target keyword in the audio file; and carrying out segmentation processing on the target video based on the occurrence time to obtain a target video interval respectively matched with each quality inspection subtask.
In one embodiment, the interval determining module is further configured to perform speech recognition on the audio file to obtain an initial recognition result of the audio file; performing semantic analysis on the initial recognition result to obtain a structural recognition result of the audio file; and monitoring the structural identification result by keywords to obtain the appearance moment of the target keywords in the audio file.
In one embodiment, the quality inspection segment determining module further includes a frame skipping module, configured to determine frame skipping factors respectively corresponding to the quality inspection subtasks; screening out target video frames from target video intervals corresponding to the quality inspection subtasks according to frame skipping factors corresponding to the quality inspection subtasks respectively; and for each quality inspection subtask, combining the corresponding target video frames to obtain a target quality inspection segment corresponding to the corresponding quality inspection subtask.
In one embodiment, the frame skipping module is further configured to determine an original frame rate of the target video interval; adjusting the original frame rate of the target video interval to a target frame rate; and for each quality inspection subtask, screening a target video frame from a target video interval which corresponds to the corresponding quality inspection subtask and is adjusted by the frame rate according to the frame skipping factor respectively corresponding to the corresponding quality inspection subtask.
In one embodiment, the execution module is further configured to determine execution resources required by each of the quality inspection subtasks when executed; and scheduling the quality inspection subtasks in parallel and executing the quality inspection subtasks based on the execution resources.
In one embodiment, the execution module further comprises a batch processing module, configured to generate, in parallel, quality inspection requests for video frames to be quality inspected in corresponding target quality inspection fragments through a plurality of request threads for execution of each quality inspection subtask; writing the generated quality inspection request into a cache queue through the request thread; and when the cache queue meets the execution condition, responding to the quality inspection requests in the cache queue in batch.
In one embodiment, the video quality inspection device is further configured to determine a quality inspection task corresponding to a target video to be subjected to quality inspection each time the target video is acquired; adding the quality inspection task to a task queue; and performing parallel scheduling execution on more than one quality inspection task in the task queue, and executing the steps of acquiring the target video to be subjected to quality inspection and the task configuration information corresponding to the target video when each quality inspection task is executed, and performing parallel scheduling and execution on each quality inspection subtask until a quality inspection result corresponding to the target video is obtained.
In one embodiment, the video quality inspection device is further configured to determine a current quality inspection task in the task queue, and schedule and execute each quality inspection subtask included in the current quality inspection task; monitoring the use condition of the execution resources corresponding to each quality inspection subtask in the current quality inspection task; when unused execution resources exist, executing a quality inspection task in a next sequence of the current quality inspection task in the task queue based on the unused execution resources.
In one embodiment, the video quality inspection device is further configured to acquire a target video; the target video is a double-recording video recorded based on an insurance service scene; the quality inspection subtask corresponding to the target video comprises at least one of a signature identification subtask, a document identification subtask, an identity card identification subtask, a face detection subtask, an action identification subtask, and a human body detection subtask.
In one embodiment, the video quality inspection device further comprises a display module, configured to return a quality inspection result to a terminal when a quality inspection result corresponding to the target video is obtained, and trigger the terminal to display the quality inspection result; the displayed quality inspection result is used for assisting a user to manually check a target video, and the quality inspection result comprises at least one of signature content obtained by executing the signature identification subtask, a document type obtained by executing the document identification subtask, identity card content obtained by executing the identity card identification subtask, a character identifier obtained by executing the face detection subtask, action content obtained by executing the action identification subtask, and a human body detection result obtained by executing the human body detection subtask.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video;
determining at least one quality inspection subtask corresponding to the target video according to the task configuration information;
determining target video intervals respectively matched with the quality inspection subtasks from the target video;
frame skipping processing is carried out on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask;
and scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are used as execution objects together.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video;
determining at least one quality inspection subtask corresponding to the target video according to the task configuration information;
determining target video intervals respectively matched with the quality inspection subtasks from the target video;
frame skipping processing is carried out on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask;
and scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are used as execution objects together.
According to the video quality inspection method, the video quality inspection device, the computer equipment and the storage medium, when the target video and the task configuration information corresponding to the target video are obtained, at least one quality inspection subtask to be executed corresponding to the target video can be determined based on the task configuration information, so that the quality inspection subtask to be executed can be executed only subsequently without executing all the quality inspection subtasks, and therefore the quality inspection efficiency of video quality inspection is greatly improved. When the quality inspection subtasks corresponding to the target video are determined, the target video intervals respectively matched with the quality inspection subtasks can be determined from the target video, frame skipping processing is performed on the corresponding target video intervals according to the frame skipping information, and target quality inspection fragments respectively corresponding to each quality inspection subtask are obtained, so that quality inspection processing can be performed only on video frames in the target quality inspection fragments subsequently, and quality inspection processing is not required to be performed on all video frames in the whole target video, and therefore quality inspection efficiency of video quality inspection is further improved. When the target quality inspection fragment corresponding to each quality inspection subtask is obtained, each quality inspection subtask can be dispatched and executed in parallel, and a plurality of video frames in the target quality inspection fragment corresponding to the quality inspection subtask are used as execution objects together, so that the quality inspection efficiency of video quality inspection is further improved. Because can carry out quality inspection to the target video automatically, compare in traditional mode of taking artifical the audit to carry out quality inspection to the video, this application can promote the quality inspection efficiency of video quality inspection greatly.
Drawings
FIG. 1 is a diagram of an exemplary video quality inspection system;
FIG. 2 is a flow chart illustrating a video quality inspection method according to an embodiment;
FIG. 3 is a diagram illustrating the configuration of quality control subtasks in one embodiment;
FIG. 4 is a flowchart illustrating a video quality inspection method according to another embodiment;
FIG. 5 is a schematic flow chart of video cleaning in one embodiment;
FIG. 6 is a flow diagram illustrating video frame cleaning in one embodiment;
FIG. 7 is a flow diagram that illustrates the scheduling of multiple quality inspection subtasks for execution in parallel, according to one embodiment;
FIG. 8 is a flow diagram that illustrates the scheduling of multiple quality inspection tasks in parallel, according to one embodiment;
FIG. 9 is a system diagram of a video quality inspection system in accordance with one embodiment;
FIG. 10 is a flow chart illustrating a video quality inspection method according to an embodiment;
FIG. 11 is a block diagram of a video inspection apparatus according to an embodiment;
FIG. 12 is a block diagram showing the structure of a video inspection apparatus according to another embodiment;
FIG. 13 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Fig. 1 is an application environment diagram of a video quality inspection method according to an embodiment. Referring to fig. 1, the video quality inspection method is applied to a video quality inspection system 100. The video quality inspection system 100 includes a terminal 102 and a server 104. The terminal 102 and the server 104 are connected via a network. The terminal 102 may be a desktop terminal or a mobile terminal, and the mobile terminal may be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The terminal 102 and the server 104 can be used separately to execute the video quality inspection method provided in the embodiment of the present application. The terminal 102 and the server 104 may also be cooperatively used to perform the video quality inspection method provided in the embodiments of the present application. When the terminal 102 and the server 104 are cooperatively used for executing the video quality inspection method provided in the embodiment of the present application, a video to be quality inspected can be displayed through the graphical interface 102-11 in the terminal 102, the video to be quality inspected is sent to the server 104 through the detection control 102-12, the server 104 detects the video to be quality inspected to obtain a quality inspection result, and the quality inspection result is returned to the terminal 102, so that the terminal 102 correspondingly displays the quality inspection result through the graphical interface 102-21.
The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, web services, cloud communications, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms.
Cloud services can be implemented based on Cloud technology, which refers to a hosting technology for unifying resources of hardware, software, network, etc. in wide area network or local area network to implement calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied in the cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Background services of the technical network system require a large amount of computing and storage resources, and schematically, in the quality inspection process of videos, the videos need to be stored, and artificial intelligence computation needs to be performed on different key video segments in the videos. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.
Cloud computing (cloud computing) refers to a mode of delivery and use of Internet Technology (IT) infrastructure, and refers to obtaining required resources through a network in an on-demand, easily-extensible manner; the generalized cloud computing refers to a delivery and use mode of a service, and refers to obtaining a required service in an on-demand and easily-extensible manner through a network. Such services may be IT and software, internet related, or other services. Cloud computing is a product of development and fusion of traditional computers and Network Technologies, such as grid computing (G ri d C o m p u t i n G), distributed computing (Dis tri distributed computing), parallel computing (Pa roller computing), utility computing (utility type computing), Network Storage (Network Storage Technologies), Virtualization (Virtualization), Load balancing (Load Balance), and the like.
It is also noted that the present application relates to the field of Artificial Intelligence (AI) technology, which is a theory, method, technique and application system that utilizes a digital computer or a machine controlled by a digital computer to simulate, extend and extend human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
It should be noted that, the numbers of "a plurality" or "a plurality" and the like mentioned in the embodiments of the present application each refer to a number of "at least two", for example, "a plurality" means "at least two", and "a plurality" means "at least two".
In an embodiment, as shown in fig. 2, a video quality inspection method is provided, where the method is applied to a computer device, and the computer device may specifically be described as an example of the terminal or the server in fig. 1, and includes the following steps:
step S202, a target video to be subjected to quality inspection and task configuration information corresponding to the target video are obtained.
The quality inspection refers to a detection process for detecting at least one key link in a video to determine whether the key link meets a preset rule or not. For example, when a target video acquired for the insurance sales process is acquired, the quality inspection may be a detection process of identifying a face, a human body, a signature process, an identity card display process and a document display process appearing in the target video to obtain a corresponding identification result, and judging whether the identification result meets a preset quality inspection requirement.
It can be understood that the process of detecting the target video can be called a quality inspection task, one quality inspection task can include at least one quality inspection subtask, and through task configuration, a user can freely select the quality inspection subtask to be executed, so as to achieve the purpose of improving the quality inspection efficiency. The quality inspection subtask refers to a subtask in a quality inspection task, and in order to facilitate quality inspection of a target video, the quality inspection task for the target video can be divided into a plurality of quality inspection subtasks according to a quality inspection function, for example, a complete quality inspection task can be divided into a signature identification subtask, a document identification subtask, an identity card identification subtask, a face detection subtask, a signature action identification subtask, a display action identification subtask, a human body detection subtask, and the like.
The task configuration information comprises at least one task configuration item, the task configuration item refers to a configuration item used for carrying out corresponding configuration on the quality inspection task of the target video, and the task configuration item is in one-to-one correspondence with the quality inspection subtasks, so that a user can freely select the task configuration item according to requirements to determine the quality inspection subtasks to be executed.
Specifically, when quality inspection of the target video is required, a user can upload the target video and perform task configuration on the target video through the user terminal, so that the user terminal can send task configuration information obtained after the task configuration is performed on the target video and the target video uploaded by the user to the computer equipment, the computer equipment can perform quality inspection on the target video according to the task configuration information, and whether key stages in the target video are in compliance or not is determined.
In one embodiment, acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video includes: receiving a target video to be subjected to quality inspection and a target task configuration item sent by a terminal; the target task configuration item is obtained by selecting a plurality of candidate task configuration items in a task configuration page displayed by the terminal; and taking the received target task configuration item as task configuration information corresponding to the target video.
Specifically, a target application for performing quality inspection on the video runs in the user terminal, and when the quality inspection needs to be performed on the target video, the user can start the target application and upload the target video to be subjected to quality inspection through the target application. When the user is determined to trigger the task configuration control in the target application, the user terminal can correspondingly display a task configuration page comprising at least one candidate task configuration item, so that the user can select the target task configuration item from the candidate task configuration items according to the quality inspection requirement. For example, the task configuration page may include a signature recognition configuration item, a document recognition configuration item, an identification card recognition configuration item, a face detection configuration item, a signature action recognition configuration item, a display action recognition configuration item, and a human body detection configuration item, and waits for the selected task configuration item, and when only document detection and face detection are performed on the target video, the user may select the document recognition configuration item and the face detection configuration item as the target task configuration item.
Further, when the target video and the target task configuration item are obtained, the user terminal can send the target video and the target task configuration item to the computer device, and the computer device takes the received target task configuration item as task configuration information corresponding to the target video.
In one embodiment, the user may not select the target task configuration item, so that the subsequent server executes the quality inspection subtask corresponding to each candidate task configuration item.
In one embodiment, the target application may automatically present the task configuration page upon determining that the user uploaded the target video, such that the user selects the target task configuration item through the task configuration page.
And step S204, determining at least one quality inspection subtask corresponding to the target video according to the task configuration information.
The quality inspection task comprises at least one of a signature identification subtask, a document identification subtask, an identity card identification subtask, a face detection subtask, a signature action identification subtask, a display action identification subtask and a human body detection subtask. The signature recognition subtask refers to a quality inspection subtask which performs content recognition on the signature in the target video through optical characters to obtain a signature result; the document identification subtask refers to a quality inspection subtask which identifies the content of the document in the target video through optical characters to obtain the type of the document and the content of the document; the identity card identification subtask refers to a quality inspection subtask which is used for identifying the content of the identity card in the target video through optical characters to obtain an identification result; the human face detection task refers to a human face recognition task for recognizing human faces appearing in a target video to determine the quality inspection subtask of human characters; the signature action identification subtask and the display action identification subtask refer to a quality inspection subtask which identifies the signature action and the display action in the target video so as to determine whether the signature action and the display action are in compliance; the human body detection subtask refers to a quality inspection subtask for detecting a human body in a target video and avoiding signing on behalf of others and wrong signing.
Specifically, the computer device prestores the association relationship between task configuration items and quality inspection subtasks. When the task configuration information corresponding to the target video is obtained, the computer device can extract at least one target task configuration item included in the task configuration information, and determine the quality inspection subtask corresponding to the target task configuration item according to the incidence relation between the task configuration item and the quality inspection subtask, that is, determine the quality inspection subtask corresponding to the target video according to the incidence relation between the task configuration item and the quality inspection subtask.
In an embodiment, as shown in fig. 3, the computer device has a task configuration pool and a task scheduling pool, where task identifiers of all quality inspection subtasks that can be scheduled are stored in the task configuration pool, and when receiving a target video and task configuration information corresponding to the target video, the computer device may perform screening processing on the task identifiers in the task configuration pool based on the task configuration information, screen out a target task identifier of a quality inspection subtask to be executed from the task configuration pool, and store the screened target task identifier in the task scheduling pool, so that subsequent computer devices can schedule and execute the quality inspection subtasks corresponding to the target task identifier until a quality inspection result corresponding to the target video is obtained. FIG. 3 is a diagram that illustrates the configuration of the quality control subtask in one embodiment.
And step S206, determining target video intervals respectively matched with the quality inspection subtasks from the target video.
Specifically, the computer device can divide the target video according to the task attributes of the quality inspection subtasks to obtain target video intervals respectively matched with the quality inspection subtasks. For example, when the quality inspection subtask is a signature action identification subtask, the computer device may determine, based on the signature action identification subtask, that the corresponding task attribute is the signature attribute, so that the computer device may extract a video interval including a signature process in the target video, and use the video interval as a target video interval matched with the signature action identification subtask; for another example, when the quality inspection subtask is an identity card identification subtask, the computer device may extract a video interval containing the display identity card in the target video, and use the video interval as the target video interval matched with the identity card identification subtask.
In one embodiment, in the process of insurance sales, the salesperson can correspondingly record the acquisition time of each key stage, for example, the time for the insured person to display the identity card, the signature time, the document display time and the like, so that the computer equipment can perform segmented processing on the target video according to the acquisition time recorded by the salesperson to obtain target video intervals corresponding to each quality inspection subtask.
In one embodiment, determining the target video interval respectively matched with each quality inspection subtask from the target video comprises: carrying out video cleaning processing on the target video to obtain an audio file of the target video; determining target keywords respectively corresponding to each quality inspection subtask; performing keyword monitoring on an audio file in a target video to determine the occurrence time of a target keyword in the audio file; and carrying out segmentation processing on the target video based on the occurrence time to obtain a target video interval respectively matched with each quality inspection subtask.
Specifically, when the target video is obtained, the computer device may perform video cleaning processing on the target video, extract sound information in the target video, and obtain an audio file. The audio file comprises audio contents such as voice and voice broadcast voice sent by a person in the target video recording process. For convenience of description, the quality inspection subtask corresponding to the target video will be referred to as a target quality inspection subtask hereinafter. Furthermore, the computer device prestores the corresponding relation between the quality inspection subtasks and the target keywords, and when the target quality inspection subtasks corresponding to the target videos are obtained, the computer device can determine the target keywords corresponding to each target quality inspection subtasks according to the corresponding relation between the quality inspection subtasks and the target keywords, and perform keyword monitoring on the audio files to determine the time when the target keywords appear in the audio files.
For example, the target keyword corresponding to the id card identification subtask may be "id card showing", so that when the "id card showing" in the audio file is monitored, the computer device correspondingly records the occurrence time of the "id card showing". For another example, the target keyword corresponding to the signature action recognition subtask may be "please sign", so that when the "please sign" in the audio file is monitored, the computer device correspondingly records the occurrence time of the "please sign".
Further, the computer device can perform segmentation processing on the target video based on the occurrence time of the target keyword in the audio file to obtain target video intervals respectively matched with each quality inspection subtask.
In one embodiment, a plurality of target keywords corresponding to one quality inspection subtask may be provided, for example, the target keywords corresponding to the id identification subtask may be "id card presentation" and "id card presentation completed", so that the computer device may record a time when each target keyword corresponding to the current target quality inspection subtask appears in the audio file, and segment the target video according to the time when each target keyword appears in the audio file, so as to obtain a target video interval corresponding to the current target quality inspection subtask. For example, when the time when the "id card presentation" appears in the audio file is 32 minutes 23 seconds, and the time when the "id card presentation is completed" appears in the audio file is 36 minutes 2 seconds, a video clip from 32 minutes 23 seconds to 36 minutes 2 seconds can be used as the target video interval matched with the id card identification subtask.
In one embodiment, the computer device may use an appearance time of a target keyword corresponding to a current quality inspection subtask as a start time, use an appearance time of a target keyword corresponding to a next sequential quality inspection subtask as a termination time, and partition a target video interval corresponding to the current quality inspection subtask, for example, when the current quality inspection subtask is an id card identification subtask, the target keyword corresponding to the id card identification subtask is "id card presentation", the next sequential quality inspection subtask is a ticket identification subtask, and the target keyword corresponding to the ticket identification subtask is "ticket presentation", the computer device may partition the target video by using an appearance time of "id card presentation" of 32 minutes and 23 seconds as the start time, and using an appearance time of "ticket presentation" of 36 minutes and 2 seconds as the termination time.
In one embodiment, the computer device may divide the target video interval within a preset time length by taking the occurrence time of the target keyword in the audio file as a starting time. For example, when the preset time length is 1 minute and the appearance time of the target keyword "document showing" is 36 minutes 2 seconds, the computer device divides the target video by using 36 minutes 2 seconds as the starting time and 37 minutes 2 seconds as the ending time.
In one embodiment, after the target keyword is identified, the computer device may perform motion identification on the video content with the occurrence time of the target keyword as a starting point, and divide a part of the identified motion that meets the target keyword into the target video interval. For example, the appearance time of the target keyword "identity card exhibition" is 3 minutes and 20 seconds of the audio file, the computer device performs motion recognition on the video content from 3 minutes and 20 seconds, divides the video clip with the exhibition motion obtained through recognition into a target video interval corresponding to the "identity card exhibition", and the termination time of the target video interval is the end time of the exhibition motion.
In one embodiment, the target video may be a double-recording video obtained by recording a sound during the insurance sales process, and when the target video is obtained, the computer device may split the target video to obtain an audio file for recording the insurance sales process and a video file which does not include sound information and is used for recording the insurance sales process. And when the appearance moment of the target keyword in the audio file is determined, the computer equipment can perform segmentation processing on the video file by taking the appearance moment as a reference to obtain a target video interval respectively matched with each quality inspection subtask.
In the above embodiment, through carrying out the keyword monitoring to audio file, can follow the keyword monitoring result, accurately mark off from the target video with each quality control subtask assorted target video interval to follow-up can only need carry out quality control to the target interval, compare in traditional all carrying out quality control to whole target, the quality control efficiency of video quality control has been promoted greatly to this embodiment.
And S208, performing frame skipping processing on the corresponding target video interval according to the frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask.
Specifically, the computer device prestores a corresponding relationship between the quality inspection subtasks and the frame skipping information, and when the target quality inspection subtasks corresponding to the target video are obtained, the computer device can determine the frame skipping information corresponding to each target quality inspection subtask according to the corresponding relationship between the quality inspection subtasks and the frame skipping information. The frame skipping information refers to information for performing frame skipping processing on the target video interval, and the proportion of video frames participating in quality inspection in the corresponding target video interval can be determined based on the frame skipping information. Further, the computer device performs frame skipping processing on the corresponding target video interval according to the frame skipping information corresponding to each target quality inspection subtask to obtain a target quality inspection segment corresponding to each quality inspection subtask.
For better understanding of the present embodiment, a process of performing frame skipping processing on a corresponding target video interval according to current frame skipping information corresponding to a current quality inspection subtask to obtain a target quality inspection segment corresponding to the current quality inspection subtask is described below by way of example: when the current quality inspection subtask, the current frame skipping information corresponding to the current quality inspection subtask, and the current target video interval corresponding to the current quality inspection subtask are obtained, the computer device can determine the proportion of video frames participating in quality inspection in the current target video interval according to the current frame skipping information, and screen the target video frames participating in quality inspection in the current target video interval according to the determined proportion information to obtain a target quality inspection segment corresponding to the current quality inspection subtask. For example, when the current frame skipping information is "the proportion of video frames participating in quality inspection is 0.5" and 420 video frames are in the current target video interval, the computer device may extract 0.8 times of target video frames from all video frames in the current target video interval, that is, extract 210 target video frames from the 420 video frames, and combine the extracted target video frames to obtain a target quality inspection segment corresponding to the current quality inspection subtask.
In one embodiment, the frame skipping information may specifically be a numerical value, such as "0.5", or text information, such as "the proportion of video frames participating in quality inspection is 0.5", or a graphic proportion, such as a graphic proportion for representing "0.5".
The frame skipping information can be freely set according to requirements, for example, a user can synthesize the accuracy of the quality inspection result of the quality inspection subtask, the consumption condition of computer resources and the processing efficiency of the quality inspection task, determine the frame skipping information corresponding to the quality inspection subtask, and use the frame skipping information when the accuracy of the quality inspection result, the consumption condition of the computer resources and the processing efficiency reach dynamic balance as the final frame skipping information of the quality inspection subtask.
Frame skipping processing is carried out on the target video interval through the frame skipping information, the number of video frames participating in quality inspection can be further reduced, and therefore the quality inspection efficiency of the target video is improved.
And step S210, scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are used as execution objects together.
Specifically, the computer device allocates independent execution resources to each quality inspection subtask, so that each quality inspection subtask can be executed in parallel based on the independent execution resources, and a quality inspection result output by each quality inspection subtask is obtained. The execution resources refer to Central Processing Unit (CPU) resources, Graphics Processing Unit (GPU) resources, memory resources, hard disk resources, network resources, and the like required by the quality inspection subtask running. Further, in executing each quality inspection subtask, the computer device may collectively use a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask as execution objects, and perform computer vision Computation (CV) on each of the execution objects. For example, when the quality inspection subtask is an identity card identification subtask, the computer device may identify an identity card image in a corresponding video frame based on GPU resources to obtain an identity card number; for another example, when the quality inspection subtask is a face detection subtask, the computer device may detect a face in a corresponding video frame based on the CPU resource, and compare the detection result with a face in a preset face library, thereby obtaining a character identifier.
In one embodiment, when the task configuration information corresponding to the current target video is obtained, the computer device may determine at least one quality inspection subtask corresponding to the target video according to the task configuration information, and perform parallel scheduling processing on each quality inspection subtask. In the process of executing each quality inspection subtask, the computer equipment determines the video frames contained in the target quality inspection segment corresponding to the quality inspection subtask, and performs batch processing on the video frames based on GPU resources through multithreading until a quality inspection result corresponding to the quality inspection subtask is obtained. It is easy to understand that, in the process of executing each quality inspection subtask, the computer device may also process the video frames included in the target quality inspection segment in sequence until the quality inspection result corresponding to the quality inspection subtask is obtained. The present embodiment is not limited thereto. The quality inspection efficiency of the video quality inspection can be improved by executing all the target quality inspection subtasks in parallel, and the quality inspection efficiency of the video quality inspection can be further improved by processing all the video frames in batches.
In one embodiment, the computer device may estimate, in advance, the size of the execution resources required for execution of each quality inspection subtask, so that when the target quality inspection subtask corresponding to the target video is obtained, the computer device may allocate, according to the estimated resource size in advance, execution resources of a corresponding size to each target quality inspection subtask.
In one embodiment, when the target quality inspection subtasks corresponding to the target video are obtained, the computer device may allocate execution resources with the same size to each target quality inspection subtask, so that when the allocated execution resources are insufficient to support the target quality inspection subtasks to run, the target quality inspection subtask may reapply for additional execution resources to the computer device.
In one embodiment, the computer device may dynamically adjust the number of parallel executions of the target quality inspection subtasks according to the idle resource condition. For example, when the idle resources are sufficient, the computer device allocates execution resources to each target quality inspection subtask, so that each target quality inspection subtask is processed in parallel based on the execution resources; when the space resources are not enough to support the execution of each target quality inspection subtask, the computer device may allocate execution resources according to the priority of each target quality inspection subtask, so that the target quality inspection subtask with higher priority may be preferentially executed based on the preferentially allocated execution resources.
In one embodiment, as shown in FIG. 4, step S402: and carrying out task configuration through the user terminal.
When the video to be quality tested needs to be tested, a user can configure the task through the user terminal to obtain task configuration information and send the task configuration information to the server. Step S404: and (4) carrying out audio and video separation on the video to be quality tested. When receiving a target video to be quality tested and task configuration information for performing task configuration on the target video, which are uploaded by a user terminal, the computer equipment can split the target video to obtain an audio file and a video file. Step S406: and monitoring the keywords of the audio file. After audio and video separation is carried out on the video to be quality tested, the computer equipment can carry out keyword monitoring on the audio file. Step S408: the video file is divided. After the audio file is subjected to keyword monitoring, the computer equipment can divide the audio file based on the keyword monitoring result to obtain a target video interval corresponding to each target quality inspection subtask. Step S410: and determining a target quality inspection task. The computer equipment can determine the corresponding target quality inspection subtask according to the task configuration information. Step S412: and judging whether the target quality inspection subtask is empty. The computer device can judge whether a target quality inspection subtask corresponding to the target video exists, and if the target quality inspection subtask does not exist, namely the user does not select the target configuration item, the computer device ends the quality inspection task corresponding to the target video. Step S414: and adjusting the frame rate of the target video interval. And if the target quality inspection subtask exists, the computer equipment determines a target video interval corresponding to the target quality inspection subtask, and adjusts the frame rate of the target video interval. Step S416: and performing frame skipping processing on the target video interval. And the computer equipment performs frame skipping processing on the corresponding target video interval according to the frame skipping information respectively corresponding to each target quality inspection subtask to obtain a target quality inspection fragment respectively corresponding to each target quality inspection subtask. Step S418: and carrying out batch processing on the video frames in the target quality inspection fragment. And C, performing CV calculation on the video frames in the target quality inspection fragment by the computer equipment until a quality inspection result is obtained. Fig. 4 is a flow chart illustrating a video quality inspection method according to an embodiment.
According to the video quality inspection method, when the target video and the task configuration information corresponding to the target video are obtained, at least one quality inspection subtask to be executed corresponding to the target video can be determined based on the task configuration information, so that the quality inspection subtask to be executed can be executed only subsequently, and all quality inspection subtasks do not need to be executed, and therefore the quality inspection efficiency of video quality inspection is greatly improved. When the quality inspection subtasks corresponding to the target video are determined, the target video intervals respectively matched with the quality inspection subtasks can be determined from the target video, frame skipping processing is performed on the corresponding target video intervals according to the frame skipping information, and target quality inspection fragments respectively corresponding to each quality inspection subtask are obtained, so that quality inspection processing can be performed only on video frames in the target quality inspection fragments subsequently, and quality inspection processing is not required to be performed on all video frames in the whole target video, and therefore quality inspection efficiency of video quality inspection is further improved. When the target quality inspection fragment corresponding to each quality inspection subtask is obtained, each quality inspection subtask can be dispatched and executed in parallel, and a plurality of video frames in the target quality inspection fragment corresponding to the quality inspection subtask are used as execution objects together, so that the quality inspection efficiency of video quality inspection is further improved. Because can carry out quality inspection to the target video automatically, compare in traditional mode of taking artifical the audit to carry out quality inspection to the video, this application can promote the quality inspection efficiency of video quality inspection greatly.
In one embodiment, keyword monitoring of an audio file in a target video to determine the occurrence time of a target keyword in the audio file comprises: carrying out voice recognition on the audio file to obtain an initial recognition result of the audio file; performing semantic analysis on the initial recognition result to obtain a structural recognition result of the audio file; and monitoring the structural recognition result by keywords to obtain the appearance moment of the target keywords in the audio file.
Specifically, in the process of performing keyword monitoring on the audio file, the computer device may perform Speech Recognition on the audio file through an Automatic Speech Recognition algorithm ASR (Automatic Speech Recognition), and convert the audio file into original characters to obtain an initial Recognition result. Further, the computer device can perform semantic understanding on the initial recognition result through a Neural Language Programming (NLP), convert the original file into a complete structured recognition result, perform keyword monitoring on the structured recognition result, and determine the occurrence time of the target keyword in the audio file. The structured recognition result refers to a dialog text containing utterances of both parties of the dialog.
In one embodiment, during the process of monitoring the keywords, the computer device may also monitor a preset abnormal word and respond to the abnormal word appearing in the structured recognition result. For example, the abnormal word may be set as "disagreement", and when it is determined that "disagreement" occurs in the structured recognition result, the computer device feeds back a quality inspection result that the quality inspection fails.
In one embodiment, in the process of insurance sales, the salesperson can input own voice in the target application through the user terminal in advance, so that the user terminal can send the voice of the salesperson to the computer device, and the computer device can perform tone recognition on the voice to obtain tone information corresponding to the salesperson. When the abnormal word appears in the structural recognition result, the computer equipment determines the appearance moment of the abnormal word in the audio file and calls the audio segment corresponding to the abnormal word according to the appearance moment. Further, the computer equipment analyzes the tone color information of the audio clip and compares the tone color information with the tone color information of the salesperson to judge whether the abnormal word is the vocabulary of the salesperson statement or the vocabulary of the insured person statement. When the abnormal word is determined to be the word stated by the insured person, the abnormal word can be judged to express the repudiation intention of the insured person, and the computer device feeds back the quality inspection result that the quality inspection does not pass. When the abnormal word is determined to be the word described by the salesperson, it can be determined that the abnormal word does not express the repudiation intention of the insured person, and the computer device ignores the abnormal word.
In the above embodiment, the audio file is preferentially converted into the initial identification result, and then the initial identification result is converted into the structured identification result, so that the obtained structured identification result is more accurate, and the monitoring result of performing keyword monitoring on the structured identification result is more accurate.
In one embodiment, the frame skipping information includes a frame skipping factor, and frame skipping processing is performed on a corresponding target video interval according to frame skipping information corresponding to each quality inspection subtask, so as to obtain a target quality inspection segment corresponding to each quality inspection subtask, including: determining frame skipping factors corresponding to the quality inspection subtasks respectively; screening target video frames from target video intervals corresponding to the quality inspection subtasks according to frame skipping factors corresponding to the quality inspection subtasks respectively; and for each quality inspection subtask, combining the corresponding target video frames to obtain a target quality inspection segment corresponding to the corresponding quality inspection subtask.
Specifically, the frame skipping information includes a frame skipping factor, wherein the frame skipping factor refers to a proportion of video frames participating in quality inspection in the target video interval. The computer device is pre-stored with the corresponding relation between the quality inspection subtask and the frame skipping factor, and when the target quality inspection subtask of the target video is determined, the computer device can determine the frame skipping factor corresponding to the target quality inspection subtask according to the corresponding relation between the quality inspection subtask and the frame skipping factor. Further, the computer device can screen out target video frames from the target video intervals corresponding to the quality inspection subtasks according to the frame skipping factors corresponding to the quality inspection subtasks respectively, and combine the target video frames screened out from the current target video interval corresponding to the current quality inspection subtasks to obtain target quality inspection fragments corresponding to the current quality inspection subtasks.
In one embodiment, a screening algorithm for screening the target number of target video frames from the target video interval according to the frame skipping factor can be freely set according to requirements. For example, when the frame skipping factor is 0.5, the computer device may randomly screen half of the target video frames from the target video interval, may also screen the target video frames from the target video interval every other frame, and may also use the first half of the video frames in the target video interval as the target video frames. The present embodiment is not limited thereto.
In the embodiment, the frame skipping processing is performed on the target video interval through the frame skipping factor, so that the number of video frames for quality inspection can be greatly reduced, and the quality inspection efficiency of the video quality inspection is improved.
In an embodiment, before performing frame skipping processing on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection fragment respectively corresponding to each quality inspection subtask, the video quality inspection method further includes: determining an original frame rate of a target video interval; adjusting the original frame rate of the target video interval to a target frame rate; according to the frame skipping factor corresponding to each quality inspection subtask, screening out a target video frame from a target video interval corresponding to each quality inspection subtask, wherein the method comprises the following steps: and for each quality inspection subtask, screening a target video frame from a target video interval which corresponds to the corresponding quality inspection subtask and is adjusted by the frame rate according to the frame skipping factor respectively corresponding to the corresponding quality inspection subtask.
Specifically, the computer device may determine an original frame rate of the target video interval and determine whether the original frame rate of the target video interval is greater than a preset target frame rate. When the original frame rate is greater than the target frame rate, the computer device discards part of the video frames of the target video interval, and adjusts the original frame rate of the target video interval to the target frame rate, for example, 60 frames per second to 30 frames per second. When the original frame rate is less than the target frame rate, the computer device copies a portion of the video frames in the target video interval and inserts the copied video frames into the target video interval to adjust the original frame rate of the target video interval to the target frame rate, for example, 15 frames per second to 30 frames per second. The target frame rate can be freely set according to requirements. Further, for each quality inspection subtask, the computer device screens out a target video frame from a target video interval corresponding to the corresponding quality inspection subtask and having the frame rate adjusted according to the frame skipping factor corresponding to the corresponding quality inspection subtask.
In one embodiment, the computer device may modify the original frame rate of the target video interval by Ffmpeg.
In one embodiment, when the target video is obtained, the computer device may perform video cleaning on the target video, and adjust the target video from the original frame rate to the target frame rate. More specifically, as shown in fig. 5, when the target video is obtained, the computer device may split the target video to obtain an audio file and a video file, and perform voice recognition and semantic analysis on the audio file to obtain video partition time ranges corresponding to the quality inspection subtasks. Meanwhile, the computer equipment performs stream fetching and decoding on the video file to obtain the original frame rate of the video file, and adjusts the original frame rate of the video file to the target frame rate to obtain the video file with the adjusted frame rate. Further, the computer device divides the time range according to the video corresponding to each quality inspection subtask, and performs segmentation processing on the video file after the frame rate adjustment to obtain a target video interval corresponding to each quality inspection subtask. FIG. 5 shows a schematic flow diagram of video cleansing in one embodiment.
Easily understood, when the target video is obtained, the frame rate of the target video is preferentially adjusted, and then the target video with the frame rate adjusted is divided to obtain a target video interval; the target videos can also be divided preferentially to obtain each target video interval, and then the frame rate of each target video is adjusted. The present embodiment is not limited thereto.
In one embodiment, after performing the video cleaning process on the target video, the computer device may further perform the video frame cleaning process on the video frames in the target video interval. More specifically, as shown in fig. 6, when the target video including 3.6 ten thousand video frames is obtained, the computer device may divide the target video in the above manner to obtain the target video interval corresponding to each quality inspection subtask, and adjust each target video interval from the original frame rate to the target frame rate to obtain the target video including 1.2 ten thousand video frames, the target video interval related to the id card phase including 420 video frames, and the target video interval related to the document phase including 6720 video frames. Further, the computer device can obtain frame skipping factors corresponding to the target quality inspection subtasks, and perform video frame cleaning processing on the target video interval with the adjusted corresponding frame rate based on the frame skipping factors to obtain target quality inspection fragments. For example, the computer device may perform a full-process detection on a human face and a human body appearing in the target video, and therefore, in the full-process detection stage, the computer device may screen out a target video frame 0.8 times (9600 frames) from the target video after the frame rate adjustment according to the frame skipping factor 0.8 corresponding to the human face recognition subtask or the human body detection subtask. For another example, the computer device may screen out 0.5 times (210 frames) of target video frames from the corresponding target video interval after the frame rate adjustment according to the frame skipping factor 0.5 corresponding to the id card identification subtask, and may take the corresponding target video interval as the target quality inspection segment according to the frame skipping factor 1.0 corresponding to the id card presentation subtask. FIG. 6 is a flow diagram that illustrates video frame cleaning in one embodiment.
In the above embodiment, the original frame rate of the target video interval is adjusted to the target frame rate, so that the original frame scale of the target video can be reduced, the total number of video frames participating in quality inspection is reduced, and the quality inspection efficiency of the video quality inspection is further improved.
In one embodiment, scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained includes: determining execution resources required by each quality inspection subtask in execution; and scheduling and executing the quality inspection subtasks in parallel based on the execution resources.
Specifically, when determining each target quality inspection subtask corresponding to the target video, the computer device determines execution resources required by each target quality inspection subtask in execution, and allocates the determined execution resources to the target quality inspection subtasks, so that each target quality inspection subtask can be executed in parallel based on the execution resources independent of each other. The execution resources comprise at least one of CPU resources, GPU resources, memory resources, hard disk resources and network resources.
In the embodiment, the quality inspection efficiency of the video quality inspection can be improved by scheduling and executing each target quality inspection subtask in parallel compared with the serial execution of each target quality inspection subtask.
In one embodiment, in the process of executing each quality inspection subtask, the collectively using a plurality of video frames in the target quality inspection segment corresponding to the quality inspection subtask as an execution object includes: for the execution of each quality inspection subtask, parallelly generating a quality inspection request of a video frame to be quality inspected in a corresponding target quality inspection fragment through a plurality of request threads; writing the generated quality inspection request into a cache queue through a request thread; and when the cache queue meets the execution condition, responding to the quality inspection requests in the cache queue in batches.
Specifically, in the process of executing each target quality inspection subtask, the computer device may create a batch processing thread and multiple request threads, extract the video frames to be quality inspected in the target quality inspection segment corresponding to the target quality inspection subtask in parallel through the multiple request threads, generate a quality inspection request for performing quality inspection on the extracted video frames to be quality inspected, and write the quality inspection request into a pre-created buffer queue. For example, when the request threads are a1, a2 and A3, and the video frames to be quality checked are a1, a2, A3 and a4, the computer device may create a quality check request B1 for performing quality check on a1 through a1, create a quality check request B2 for performing quality check on a2 through a2, and create a quality check request B3 for performing quality check on A3 through A3, and write B1, B2 and B3 into the cache queue.
Further, the computer equipment judges whether the cache queue meets the execution condition, responds to the quality inspection requests in the cache queue in batch through the batch processing thread when the execution condition is met, and correspondingly deletes the responded quality inspection requests from the cache queue when the quality inspection requests are completely responded.
In one embodiment, the computer device may determine whether the number of the quality inspection requests in the buffer queue reaches a preset number, and when the preset number is reached, determine that the buffer queue meets the execution condition, and at this time, respond to the quality inspection requests in the buffer queue in batch through the batch processing thread.
In one embodiment, the computer device may determine whether a newly written quality inspection request exists within a preset time period, for example, may determine whether the requesting thread writes a quality inspection into the cache queue within 5 seconds, and determine that the cache queue meets the execution condition when the newly written quality inspection request does not exist, at which time, the batch processing thread responds to the quality inspection requests in the cache queue in batch.
In one embodiment, as shown in fig. 7, when multiple target quality inspection subtasks are obtained, the computer device may perform task management on each target quality inspection subtask to perform parallel scheduling execution on each target quality inspection subtask. For the execution of each target quality inspection subtask, the computer device performs batch processing on a plurality of video frames in a target quality inspection segment corresponding to the target quality inspection subtask, performs quality inspection processing on the plurality of video frames subjected to batch processing based on the GPU resource in the CV cluster, namely performs image recognition on each video frame based on the GPU resource. FIG. 7 is a flow diagram that illustrates the scheduling of multiple quality inspection subtasks for execution in parallel, according to one embodiment.
In the embodiment, the quality inspection subtasks are called out in parallel through the buffer queue, so that the quality inspection efficiency of video quality inspection can be improved.
In one embodiment, the video quality inspection method further includes: determining a quality inspection task corresponding to a target video when the target video to be subjected to quality inspection is acquired; adding a quality inspection task to a task queue; and performing parallel scheduling execution on more than one quality inspection task in the task queue, and executing the steps from the step of acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video to the step of scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained when each quality inspection task is executed.
Specifically, a process of detecting a target video may be referred to as a quality inspection task, and one quality inspection task may include at least one quality inspection subtask, and when multiple target videos are obtained, the computer device may determine the quality inspection task corresponding to each target video, and add the quality inspection task to a task queue generated in advance. Further, the computer equipment can perform parallel scheduling execution on more than one quality inspection task in the task queue, and execute the quality inspection subtasks in each quality inspection task according to the steps.
In one embodiment, the computer device may dynamically adjust the queue length of the task queue according to the resource vacancy degree, for example, when the vacancy resource is smaller than the preset resource size, the queue length of the task queue is shortened; when the idle resources are larger than the preset resource size, increasing the queue length of the task queue, so that when the task queue is full, the subsequent quality inspection tasks to be added into the task queue are in a waiting response state; and when the task queue is not full, performing parallel scheduling execution on each quality inspection task in the task queue.
In the embodiment, the quality inspection efficiency of the video quality inspection can be improved by performing parallel scheduling execution on more than one quality inspection tasks in the task queue.
In one embodiment, the quality inspection task comprises at least one quality inspection subtask, and the parallel scheduling execution of more than one quality inspection tasks in the task queue comprises: determining a current quality inspection task in a task queue, and scheduling and executing each quality inspection subtask included in the current quality inspection task; monitoring the use condition of execution resources corresponding to each quality inspection subtask in the current quality inspection task; when there are unused execution resources, a quality inspection task in the next order of the current quality inspection task in the task queue is executed based on the unused execution resources.
Specifically, the computer device determines the current quality inspection tasks in the task queue, and sequentially schedules and executes each quality inspection subtask included in the current quality inspection task according to the request execution time of each quality inspection subtask. The computer equipment monitors the use condition of the execution resources corresponding to each quality inspection subtask in the current quality inspection task in real time, and when unused execution resources exist, the computer equipment executes the quality inspection task in the next sequence of the current quality inspection task in the task queue based on the unused execution resources, so that the parallel scheduling execution of a plurality of quality inspection tasks can be realized.
For example, as shown in fig. 8, when the target video 1 is received at time t2, the execution request of the identification card recognition subtask "OCR 1" for performing OCR recognition on the identification card in the target video 1 is received at time t3, the execution request of the action recognition subtask "action 1" for recognizing the identification card showing action in the target video 1 is received at time t4, and the execution request of the face detection subtask "face 1" for recognizing the face in the target video 1 is received at time t5, the computer device may monitor the usage of the execution resources corresponding to each of "OCR 1", "action 1" and "face 1", and when it is determined that the allocated execution resource is not used by "action 1" at time t4, the execution resource is allocated to the "OCR 2" also received at time t4, so that the identification card in the target video 2 can be identified based on the execution resource. It is easily understood that, when it is determined that the "OCR 2" still does not use the allocated execution resource, the computer device may further allocate the execution resource to "action 3" received at time t4, so that the quality inspection tasks may be scheduled in a pipeline according to the time difference between the reception of the quality inspection tasks and the time difference between the execution requests of the quality inspection subtasks, thereby implementing a multi-task parallel scheduling process and further improving the quality inspection efficiency. FIG. 8 is a flowchart illustrating parallel scheduling of multiple quality inspection tasks according to an embodiment.
In the embodiment, the unused execution resources are allocated to the rest quality inspection subtasks, so that the utilization rate of the execution resources can be improved, and the quality inspection efficiency of the video quality inspection is further improved.
In one embodiment, the target video is a double-recording video recorded based on an insurance service scene; the quality inspection subtask corresponding to the target video includes at least one of a signature identification subtask, a document identification subtask, an identification subtask, a face detection subtask, a motion identification subtask, and a human body detection subtask.
Specifically, in the process of insurance sales, sales personnel can acquire a target video to be subjected to quality inspection through technical means such as sound recording and video recording and the like, so that the sales behavior can be played back, important information can be inquired, and the problem responsibility can be confirmed. The quality inspection tasks corresponding to the target video may include at least one of a signature recognition subtask, a document recognition subtask, an identification card recognition subtask, a face detection subtask, a motion recognition subtask, and a human detection subtask. The action identification subtask may include an identity card presentation subtask, an identity card signing action subtask, a document presentation subtask, and a document signing subtask. The signature recognition subtask, the document recognition subtask, and the identification card recognition subtask can be summarized as an OCR recognition task.
When the quality inspection subtask is a signature identification subtask, the computer equipment can identify the signature process through an OCR (Optical Character Recognition) signature identification algorithm to obtain a signature result, and judge whether the signature result meets the preset signature requirement; when the quality inspection subtask is a document identification subtask, the computer equipment can identify the document display process through an OCR document identification algorithm to obtain the type of the document; when the quality inspection subtask is an identity card identification subtask, the computer equipment can identify the identity card display process through an OCR (optical character recognition) identity card identification algorithm to obtain an identity card number and judge whether the identity card number meets the requirement of a preset certificate or not; when the quality detection subtask is a face detection subtask, the computer equipment can identify the face in the video image frame through a face detection algorithm to obtain a face motion track; when the quality inspection subtask is an action identification subtask, the computer equipment can identify an identity card display action, a document display action, an identity card signing action or a document signing action in the target video and judge whether the action in the target video meets a preset action standard or not; when the quality inspection subtask is a human body detection subtask, the computer device can identify a human body in the target video.
In one embodiment, when the target quality inspection subtasks corresponding to the target video are a face recognition subtask, a body recognition subtask, and a signature action recognition subtask, the computer device may recognize a face in the target video based on a face detection algorithm, may recognize a body in the target video based on a body detection algorithm, and may recognize a signature action in the target video based on an action recognition algorithm. When the human face and the human body are determined to appear in the signing process based on the human face recognition result, the human body recognition result and the action recognition result, the computer equipment can bind the human face and the human body in the signing process and judge that the human face and the human body which are signed in the same video frame belong to the same person, so that the probability of mistaken signing and signature generation in the insurance selling process is reduced, and the normalization of insurance selling is improved.
In the embodiment, the multiple quality inspection subtasks are set, so that the user can select the corresponding target quality inspection subtask based on the self requirement, and the user experience is greatly improved.
In one embodiment, the video quality inspection method further includes: when a quality inspection result corresponding to the target video is obtained, returning the quality inspection result to the terminal, and triggering the terminal to display the quality inspection result; the displayed quality inspection result is used for assisting a user to manually check the target video, and the quality inspection result comprises at least one of signature content obtained by executing the signature identification subtask, a document type obtained by executing the document identification subtask, identity card content obtained by executing the identity card identification subtask, a character role identifier obtained by executing the face detection subtask, action content obtained by executing the action identification subtask, and a human body detection result obtained by executing the human body detection subtask.
Specifically, when the quality inspection results of each target quality inspection subtask are obtained, the computer device may return the quality inspection results to the user terminal, so that the user terminal correspondingly displays the quality inspection results. For example, when the target quality inspection subtask is a face identification subtask, the computer device may return a character identifier obtained by performing the face detection subtask to the user terminal, so that the user terminal may frame a face in a corresponding video frame through a face frame and display the character identifier of the framed character, so that the user may distinguish characters appearing in the target video, for example, may distinguish salespersons and insured persons with a service manager. For another example, when the target quality inspection task is an identity card display subtask, the computer device may return a time period for displaying the identity card and an action recognition result of the identity card display to the user terminal, so that the user may accurately position to a target video interval to be manually checked based on the time period, and manually review a display action in the target video interval based on the action recognition result. Furthermore, when the target quality inspection task is a document identification subtask, the computer device can return the identified document type to the user terminal, so that the user can determine whether the document type in the target video is the preset target type according to the returned quality inspection result.
In this embodiment, the quality inspection result is returned to the user terminal, so that faster quality inspection data can be provided for the user, and the user can quickly and quickly know whether key behaviors such as documents, identity cards, signatures, presentations and the like meet preset rules through the quality inspection result and quickly position the key behaviors to the time period when the key behaviors appear, so as to quickly determine whether the whole target video is legal or not.
The video detection method can be applied to a video quality inspection system, and for facilitating understanding of those skilled in the art, referring to fig. 9, fig. 9 shows a system diagram of the video quality inspection system in one embodiment. The video quality inspection system includes an application layer (also referred to as a front end) 902, a processing layer (also referred to as a web backend) 904, an access layer cluster 906, a microservice 908, a management center 910, and a kubernets cluster 912. The kubernets cluster 912 is used for managing containerized applications on multiple hosts in the cloud platform, that is, for managing the application layer 902, the processing layer 904, the access layer cluster 906, the micro-service 908, and the management center 910. The application layer 902 is used for receiving a target video uploaded by a user and performing task configuration on the target video. The processing layer 904 is configured to determine a target quality inspection subtask corresponding to the target video according to the task configuration information, split the target video, and perform video cleaning and video frame cleaning on the target video. The access layer cluster 906 is configured to access each cleaned target quality inspection fragment and determine a target quality inspection subtask corresponding to each target quality inspection fragment, so that the microservice 908 can execute each target quality inspection subtask by balancing loads. The management center 910 may monitor and manage the entire system, continuously integrate the entire system, and release a new system.
In one embodiment, as shown in fig. 10, a flow chart of a video quality inspection method in an embodiment is provided:
s1002, receiving a target video and a target task configuration item which are sent by a terminal and are to be subjected to quality inspection; the target task configuration item is obtained by selecting a plurality of candidate task configuration items in a task configuration page displayed by the terminal.
And S1004, taking the received target task configuration item as task configuration information corresponding to the target video.
S1006, determining at least one quality inspection subtask corresponding to the target video according to the task configuration information; the quality inspection subtasks corresponding to the target video include at least one of a signature identification subtask, a document identification subtask, an identification subtask, a face detection subtask, a motion identification subtask, and a human body detection subtask.
S1008, performing video cleaning processing on the target video to obtain an audio file of the target video; determining target keywords respectively corresponding to each quality inspection subtask; carrying out voice recognition on the audio file to obtain an initial recognition result of the audio file; performing semantic analysis on the initial recognition result to obtain a structural recognition result of the audio file; and monitoring the structural recognition result by keywords to obtain the time when the target keywords appear in the audio file.
And S1010, performing segmentation processing on the target video based on the occurrence time to obtain target video intervals respectively matched with each quality inspection subtask. Determining an original frame rate of a target video interval; and adjusting the original frame rate of the target video interval to the target frame rate.
S1012, determining frame skipping factors corresponding to the quality inspection subtasks respectively; for each quality inspection subtask, screening a target video frame from a target video interval which corresponds to the corresponding quality inspection subtask and is adjusted by frame rate according to frame skipping factors respectively corresponding to the corresponding quality inspection subtasks; and for each quality inspection subtask, combining the corresponding target video frames to obtain a target quality inspection segment corresponding to the corresponding quality inspection subtask.
S1014, determining the execution resources required by each quality inspection subtask in execution; and scheduling and executing the quality inspection subtasks in parallel based on the execution resources. Monitoring the use condition of execution resources corresponding to each quality inspection subtask in the current quality inspection task; when there are unused execution resources, a quality inspection task in the next order of the current quality inspection task in the task queue is executed based on the unused execution resources.
S1016, for the execution of each quality inspection subtask, parallel generating a quality inspection request of a video frame to be quality inspected in a corresponding target quality inspection segment through a plurality of request threads; writing the generated quality inspection request into a cache queue through a request thread; and when the cache queue meets the execution condition, responding to the quality inspection requests in the cache queue in batches.
S1018, when a target video to be subjected to quality inspection is obtained, determining a quality inspection task corresponding to the target video; adding a quality inspection task to a task queue; and performing parallel scheduling execution on more than one quality inspection task in the task queue.
And S1020, executing each quality inspection task from the step of acquiring the target video to be subjected to quality inspection and the task configuration information corresponding to the target video to the step of scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained.
S1022, when a quality inspection result corresponding to the target video is obtained, returning the quality inspection result to the terminal, and triggering the terminal to display the quality inspection result; and the displayed quality inspection result is used for assisting a user to manually check the target video.
According to the video quality inspection method, when the target video and the task configuration information corresponding to the target video are obtained, at least one quality inspection subtask to be executed corresponding to the target video can be determined based on the task configuration information, so that the quality inspection subtask to be executed can be executed subsequently, and all quality inspection subtasks do not need to be executed, and therefore the quality inspection efficiency of the video quality inspection is greatly improved. When the quality inspection subtasks corresponding to the target video are determined, the target video intervals respectively matched with the quality inspection subtasks can be determined from the target video, frame skipping processing is performed on the corresponding target video intervals according to the frame skipping information, and target quality inspection fragments respectively corresponding to each quality inspection subtask are obtained, so that quality inspection processing can be performed only on video frames in the target quality inspection fragments subsequently, and quality inspection processing is not required to be performed on all video frames in the whole target video, and therefore quality inspection efficiency of video quality inspection is further improved. When the target quality inspection fragment corresponding to each quality inspection subtask is obtained, each quality inspection subtask can be dispatched and executed in parallel, and a plurality of video frames in the target quality inspection fragment corresponding to the quality inspection subtask are used as execution objects together, so that the quality inspection efficiency of video quality inspection is further improved. Because can carry out quality inspection to the target video automatically, compare in traditional mode of taking artifical the audit to carry out quality inspection to the video, this application can promote the quality inspection efficiency of video quality inspection greatly.
It should be understood that although the steps in the flowcharts of fig. 2 and 10 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 and 10 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the other steps or stages.
The application also provides an application scenario, wherein the application scenario applies the video quality inspection method, and specifically, the application of the video quality inspection method in the scenario is as follows:
in the invigilating process, the recording and video equipment erected in a classroom can collect the invigilating process to obtain an invigilating video, so that a teacher can upload the invigilating video serving as a target video to be subjected to quality inspection to computer equipment and correspondingly configure task configuration information of the target video. The computer equipment can determine a quality inspection subtask to be executed, perform segmentation processing and frame skipping processing on the target video, and perform parallel scheduling processing on the quality inspection task and the quality inspection subtask until a quality inspection result is obtained, and then return the quality inspection result to the user terminal, so that a teacher can determine the identity information of examinees in a classroom based on the quality inspection result of the face identification subtask; determining the signature content in the examination process based on the quality inspection result of the signature identification subtask; determining whether an examinee is looking at or leaves the seat at will in the examination process based on the quality inspection result of the action identification subtask; and determining whether the surrogate phenomenon exists in the examination process or not based on the quality inspection result of the human body identification subtask.
The application also provides another application scenario, where the video quality inspection method is applied, and specifically, the application of the video quality inspection method in the scenario is as follows:
in the process of handling banking business, the audio and video recording equipment erected in a bank can collect the business handling process to obtain business handling videos, so that a banking staff can upload the business handling videos serving as target videos to be subjected to quality inspection to computer equipment and perform task configuration on the target videos. The computer can obtain a quality inspection result by adopting the mode and return the quality inspection result to the user terminal, so that a manager can distinguish a banking operator and a client in a business handling video based on the quality inspection result of the face recognition subtask; determining signature content in the process of banking business based on the signature identification subtask quality inspection result; determining a display action in the banking process based on the quality inspection result of the display action identification subtask, and judging whether the display action meets the preset requirement; and determining the displayed identity card number based on the quality inspection result of the identity card identification subtask, and judging whether the identity card number meets the preset requirement or not.
In one embodiment, as shown in fig. 11, there is provided a video quality inspection apparatus 1100, which may be a part of a computer device using a software module or a hardware module, or a combination of the two, the apparatus specifically includes: a subtask determination module 1102, a quality inspection segment determination module 1104, and an execution module 1106, wherein:
a subtask determining module 1102, configured to obtain a target video to be subjected to quality inspection and task configuration information corresponding to the target video; and determining at least one quality inspection subtask corresponding to the target video according to the task configuration information.
A quality inspection segment determining module 1104, configured to determine, from the target video, target video intervals respectively matched with the quality inspection subtasks; and performing frame skipping processing on the corresponding target video interval according to the frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask.
And an execution module 1106, configured to schedule and execute each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, where in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are collectively used as an execution object.
In an embodiment, as shown in fig. 12, the subtask determining module 1102 is further configured to receive a target video to be subjected to quality inspection and a target task configuration item, which are sent by the terminal; the target task configuration item is obtained by selecting a plurality of candidate task configuration items in a task configuration page displayed by the terminal; and taking the received target task configuration item as task configuration information corresponding to the target video.
In one embodiment, the quality inspection fragment determining module 1104 further includes an interval determining module 1141, configured to perform video cleaning processing on the target video to obtain an audio file of the target video; determining target keywords respectively corresponding to each quality inspection subtask; performing keyword monitoring on an audio file in a target video to determine the occurrence time of a target keyword in the audio file; and carrying out segmentation processing on the target video based on the occurrence time to obtain a target video interval respectively matched with each quality inspection subtask.
In an embodiment, the interval determining module 1141 is further configured to perform voice recognition on the audio file to obtain an initial recognition result of the audio file; performing semantic analysis on the initial recognition result to obtain a structural recognition result of the audio file; and monitoring the structural recognition result by keywords to obtain the appearance moment of the target keywords in the audio file.
In one embodiment, the quality inspection segment determining module 1104 further includes a frame skipping module 1142 for determining frame skipping factors respectively corresponding to the quality inspection subtasks; screening target video frames from target video intervals corresponding to the quality inspection subtasks according to frame skipping factors corresponding to the quality inspection subtasks respectively; and for each quality inspection subtask, combining the corresponding target video frames to obtain a target quality inspection segment corresponding to the corresponding quality inspection subtask.
In one embodiment, the frame skipping module 1142 is further configured to determine an original frame rate of the target video interval; adjusting the original frame rate of the target video interval to a target frame rate; and for each quality inspection subtask, screening a target video frame from a target video interval which corresponds to the corresponding quality inspection subtask and is adjusted by the frame rate according to the frame skipping factor respectively corresponding to the corresponding quality inspection subtask.
In one embodiment, the execution module 1106 is further configured to determine the execution resources required by each quality inspection subtask during execution; and scheduling and executing the quality inspection subtasks in parallel based on the execution resources.
In one embodiment, the execution module 1106 further includes a batch processing module 1161, configured to generate, in parallel, quality inspection requests of video frames to be quality inspected in corresponding target quality inspection segments through multiple request threads for execution of each quality inspection subtask; writing the generated quality inspection request into a cache queue through a request thread; and when the cache queue meets the execution condition, responding to the quality inspection requests in the cache queue in batches.
In one embodiment, the video quality inspection apparatus 1100 is further configured to determine a quality inspection task corresponding to a target video to be subjected to quality inspection each time the target video is acquired; adding a quality inspection task to a task queue; and performing parallel scheduling execution on more than one quality inspection task in the task queue, and executing the steps from the step of acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video to the step of scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained when each quality inspection task is executed.
In one embodiment, the video quality inspection apparatus 1100 is further configured to determine a current quality inspection task in the task queue, and schedule and execute each quality inspection subtask included in the current quality inspection task; monitoring the use condition of execution resources corresponding to each quality inspection subtask in the current quality inspection task; when there are unused execution resources, a quality inspection task in the next order of the current quality inspection task in the task queue is executed based on the unused execution resources.
In one embodiment, the video quality inspection apparatus 1100 is further configured to obtain a target video; the target video is a double-recording video recorded based on an insurance service scene; the quality inspection subtask corresponding to the target video includes at least one of a signature identification subtask, a document identification subtask, an identification subtask, a face detection subtask, a motion identification subtask, and a human body detection subtask.
In an embodiment, the video quality inspection apparatus 1100 further includes a display module 1108, configured to, when a quality inspection result corresponding to the target video is obtained, return the quality inspection result to the terminal, and trigger the terminal to display the quality inspection result; the displayed quality inspection result is used for assisting a user to manually check the target video, and the quality inspection result comprises at least one of signature content obtained by executing the signature identification subtask, a document type obtained by executing the document identification subtask, identity card content obtained by executing the identity card identification subtask, a character role identifier obtained by executing the face detection subtask, action content obtained by executing the action identification subtask, and a human body detection result obtained by executing the human body detection subtask.
For specific limitations of the video quality inspection apparatus, reference may be made to the above limitations of the video quality inspection method, which are not described herein again. The modules in the video quality inspection device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 13. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing video quality inspection data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a video quality inspection method.
Those skilled in the art will appreciate that the architecture shown in fig. 13 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (15)

1. A method for video quality inspection, the method comprising:
acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video;
determining at least one quality inspection subtask corresponding to the target video according to the task configuration information;
determining target video intervals respectively matched with the quality inspection subtasks from the target video;
frame skipping processing is carried out on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask;
and scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are used as execution objects together.
2. The method according to claim 1, wherein the acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video comprises:
receiving a target video to be subjected to quality inspection and a target task configuration item sent by a terminal; the target task configuration item is obtained by selecting a plurality of candidate task configuration items in a task configuration page displayed by the terminal;
and taking the received target task configuration item as task configuration information corresponding to the target video.
3. The method of claim 1, wherein the determining target video intervals respectively matching the quality inspection subtasks from the target video comprises:
carrying out video cleaning processing on the target video to obtain an audio file of the target video;
determining target keywords respectively corresponding to each quality inspection subtask;
performing keyword monitoring on an audio file in the target video to determine the occurrence time of the target keyword in the audio file;
and carrying out segmentation processing on the target video based on the occurrence time to obtain a target video interval respectively matched with each quality inspection subtask.
4. The method of claim 3, wherein the performing keyword monitoring on the audio file in the target video to determine the occurrence time of the target keyword in the audio file comprises:
carrying out voice recognition on the audio file to obtain an initial recognition result of the audio file;
performing semantic analysis on the initial recognition result to obtain a structural recognition result of the audio file;
and monitoring the structural identification result by keywords to obtain the appearance moment of the target keywords in the audio file.
5. The method according to claim 1, wherein the frame skipping information includes a frame skipping factor, and the frame skipping processing is performed on the corresponding target video interval according to the frame skipping information corresponding to each of the quality inspection subtasks, so as to obtain a target quality inspection segment corresponding to each of the quality inspection subtasks, including:
determining frame skipping factors respectively corresponding to the quality inspection subtasks;
screening out target video frames from target video intervals corresponding to the quality inspection subtasks according to frame skipping factors corresponding to the quality inspection subtasks respectively;
and for each quality inspection subtask, combining the corresponding target video frames to obtain a target quality inspection segment corresponding to the corresponding quality inspection subtask.
6. The method of claim 5, further comprising:
determining an original frame rate of the target video interval;
adjusting the original frame rate of the target video interval to a target frame rate;
the step of screening out the target video frames from the target video intervals corresponding to the quality inspection subtasks according to the frame skipping factors corresponding to the quality inspection subtasks respectively comprises the following steps:
and for each quality inspection subtask, screening a target video frame from a target video interval which corresponds to the corresponding quality inspection subtask and is adjusted by the frame rate according to the frame skipping factor respectively corresponding to the corresponding quality inspection subtask.
7. The method of claim 1, wherein the scheduling and executing the quality inspection subtasks in parallel until a quality inspection result corresponding to the target video is obtained comprises:
determining execution resources required by each quality inspection subtask in execution;
and scheduling the quality inspection subtasks in parallel and executing the quality inspection subtasks based on the execution resources.
8. The method according to claim 1, wherein the collectively performing a plurality of video frames in the target quality inspection segment corresponding to each quality inspection subtask comprises:
for the execution of each quality inspection subtask, parallelly generating a quality inspection request of a video frame to be quality inspected in a corresponding target quality inspection fragment through a plurality of request threads;
writing the generated quality inspection request into a cache queue through the request thread;
and when the cache queue meets the execution condition, responding to the quality inspection requests in the cache queue in batch.
9. The method of claim 1, further comprising:
determining a quality inspection task corresponding to a target video to be subjected to quality inspection when the target video is acquired;
adding the quality inspection task to a task queue;
and performing parallel scheduling execution on more than one quality inspection task in the task queue, and executing the steps of acquiring the target video to be subjected to quality inspection and the task configuration information corresponding to the target video when each quality inspection task is executed, and performing parallel scheduling and execution on each quality inspection subtask until a quality inspection result corresponding to the target video is obtained.
10. The method of claim 9, wherein the quality inspection task comprises at least one quality inspection subtask, and wherein scheduling execution of more than one quality inspection tasks in the task queue in parallel comprises:
determining a current quality inspection task in the task queue, and scheduling and executing each quality inspection subtask included in the current quality inspection task;
monitoring the use condition of the execution resources corresponding to each quality inspection subtask in the current quality inspection task;
when unused execution resources exist, executing a quality inspection task in a next sequence of the current quality inspection task in the task queue based on the unused execution resources.
11. The method according to any one of claims 1 to 10, wherein the target video is a double-recorded video recorded based on an insurance service scenario; the quality inspection subtask corresponding to the target video comprises at least one of a signature identification subtask, a document identification subtask, an identity card identification subtask, a face detection subtask, an action identification subtask, and a human body detection subtask.
12. The method of claim 11, further comprising:
when a quality inspection result corresponding to the target video is obtained, returning the quality inspection result to a terminal, and triggering the terminal to display the quality inspection result; the displayed quality inspection result is used for assisting a user to manually check a target video, and the quality inspection result comprises at least one of signature content obtained by executing the signature identification subtask, a document type obtained by executing the document identification subtask, identity card content obtained by executing the identity card identification subtask, a character identifier obtained by executing the face detection subtask, action content obtained by executing the action identification subtask, and a human body detection result obtained by executing the human body detection subtask.
13. A video quality inspection apparatus, the apparatus comprising:
the subtask determining module is used for acquiring a target video to be subjected to quality inspection and task configuration information corresponding to the target video; determining at least one quality inspection subtask corresponding to the target video according to the task configuration information;
the quality inspection fragment determining module is used for determining target video intervals respectively matched with all quality inspection subtasks from the target video; frame skipping processing is carried out on a corresponding target video interval according to frame skipping information respectively corresponding to each quality inspection subtask to obtain a target quality inspection segment respectively corresponding to each quality inspection subtask;
and the execution module is used for scheduling and executing each quality inspection subtask in parallel until a quality inspection result corresponding to the target video is obtained, wherein in the process of executing each quality inspection subtask, a plurality of video frames in a target quality inspection segment corresponding to the quality inspection subtask are jointly used as execution objects.
14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 12.
15. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 12.
CN202110001121.8A 2021-01-04 2021-01-04 Video quality inspection method and device, computer equipment and storage medium Active CN112351337B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110001121.8A CN112351337B (en) 2021-01-04 2021-01-04 Video quality inspection method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110001121.8A CN112351337B (en) 2021-01-04 2021-01-04 Video quality inspection method and device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112351337A true CN112351337A (en) 2021-02-09
CN112351337B CN112351337B (en) 2022-02-01

Family

ID=74427726

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110001121.8A Active CN112351337B (en) 2021-01-04 2021-01-04 Video quality inspection method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112351337B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113206998A (en) * 2021-04-30 2021-08-03 中国工商银行股份有限公司 Method and device for quality inspection of video data recorded by service
CN113435414A (en) * 2021-07-30 2021-09-24 北京睿芯高通量科技有限公司 Video content auditing system, method and device
CN114640806A (en) * 2022-03-14 2022-06-17 上海哔哩哔哩科技有限公司 Video file synthesis method and device
WO2023024791A1 (en) * 2021-08-27 2023-03-02 上海商汤智能科技有限公司 Frame rate adjustment method and apparatus, electronic device, storage medium, and program

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101957780A (en) * 2010-08-17 2011-01-26 中国电子科技集团公司第二十八研究所 Resource state information-based grid task scheduling processor and grid task scheduling processing method
CN109831665A (en) * 2019-01-16 2019-05-31 深圳壹账通智能科技有限公司 A kind of video quality detecting method, system and terminal device
CN110147726A (en) * 2019-04-12 2019-08-20 财付通支付科技有限公司 Business quality detecting method and device, storage medium and electronic device
CN111741356A (en) * 2020-08-25 2020-10-02 腾讯科技(深圳)有限公司 Quality inspection method, device and equipment for double-recording video and readable storage medium
CN112016538A (en) * 2020-10-29 2020-12-01 腾讯科技(深圳)有限公司 Video processing method, video processing device, computer equipment and storage medium
CN112101311A (en) * 2020-11-16 2020-12-18 深圳壹账通智能科技有限公司 Double-recording quality inspection method and device based on artificial intelligence, computer equipment and medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101957780A (en) * 2010-08-17 2011-01-26 中国电子科技集团公司第二十八研究所 Resource state information-based grid task scheduling processor and grid task scheduling processing method
CN109831665A (en) * 2019-01-16 2019-05-31 深圳壹账通智能科技有限公司 A kind of video quality detecting method, system and terminal device
CN110147726A (en) * 2019-04-12 2019-08-20 财付通支付科技有限公司 Business quality detecting method and device, storage medium and electronic device
CN111741356A (en) * 2020-08-25 2020-10-02 腾讯科技(深圳)有限公司 Quality inspection method, device and equipment for double-recording video and readable storage medium
CN112016538A (en) * 2020-10-29 2020-12-01 腾讯科技(深圳)有限公司 Video processing method, video processing device, computer equipment and storage medium
CN112101311A (en) * 2020-11-16 2020-12-18 深圳壹账通智能科技有限公司 Double-recording quality inspection method and device based on artificial intelligence, computer equipment and medium

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113206998A (en) * 2021-04-30 2021-08-03 中国工商银行股份有限公司 Method and device for quality inspection of video data recorded by service
CN113206998B (en) * 2021-04-30 2022-12-09 中国工商银行股份有限公司 Method and device for quality inspection of video data recorded by service
CN113435414A (en) * 2021-07-30 2021-09-24 北京睿芯高通量科技有限公司 Video content auditing system, method and device
CN113435414B (en) * 2021-07-30 2024-04-26 北京中科通量科技有限公司 Video content auditing system, method and device
WO2023024791A1 (en) * 2021-08-27 2023-03-02 上海商汤智能科技有限公司 Frame rate adjustment method and apparatus, electronic device, storage medium, and program
CN114640806A (en) * 2022-03-14 2022-06-17 上海哔哩哔哩科技有限公司 Video file synthesis method and device

Also Published As

Publication number Publication date
CN112351337B (en) 2022-02-01

Similar Documents

Publication Publication Date Title
CN112351337B (en) Video quality inspection method and device, computer equipment and storage medium
US8990149B2 (en) Generating a predictive model from multiple data sources
CN111741356B (en) Quality inspection method, device and equipment for double-recording video and readable storage medium
JP6751806B2 (en) Entity recognition that uses multiple data streams to supplement missing information related to an entity
CN109918184B (en) Picture processing system, method and related device and equipment
CN111258744A (en) Task processing method based on heterogeneous computation and software and hardware framework system
CN112449750A (en) Log data collection method, log data collection device, storage medium, and log data collection system
US20220270612A1 (en) Cognitive correlation of group interactions
US10423822B2 (en) Video image overlay of an event performance
US20220374219A1 (en) Deployment of service
CN110347389B (en) Method, device and system for processing algorithm file
US20210406981A1 (en) Method and apparatus of determining display page, electronic device, and medium
US20200250608A1 (en) Providing feedback by evaluating multi-modal data using machine learning techniques
CN116775183A (en) Task generation method, system, equipment and storage medium based on large language model
US10146659B2 (en) Large event log replay method and system
US20200251006A1 (en) Dynamic evaluation of event participants using a smart context-based quiz system
CN112085078A (en) Image classification model generation system, method and device and computer equipment
US10762089B2 (en) Open ended question identification for investigations
CN115170085A (en) Approval process generation method and device, storage medium and electronic equipment
US11947894B2 (en) Contextual real-time content highlighting on shared screens
CN114691953A (en) Immersive interactive preference mining method and system combined with big data
US11750671B2 (en) Cognitive encapsulation of group meetings
US11558438B1 (en) Status prediction for meetings and participants
CN114581130A (en) Bank website number assigning method and device based on customer portrait and storage medium
CN113127195B (en) Artificial intelligence analysis vertical solution integrator

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40039046

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant