WO2022156294A1 - Video processing method and apparatus, computer readable storage medium, and electronic device - Google Patents

Video processing method and apparatus, computer readable storage medium, and electronic device Download PDF

Info

Publication number
WO2022156294A1
WO2022156294A1 PCT/CN2021/126446 CN2021126446W WO2022156294A1 WO 2022156294 A1 WO2022156294 A1 WO 2022156294A1 CN 2021126446 W CN2021126446 W CN 2021126446W WO 2022156294 A1 WO2022156294 A1 WO 2022156294A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
event
clip
clipped
video clip
Prior art date
Application number
PCT/CN2021/126446
Other languages
French (fr)
Chinese (zh)
Inventor
成云峰
杨太任
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2022156294A1 publication Critical patent/WO2022156294A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334Recording operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44016Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • the present disclosure relates to the technical field of video processing, and in particular, to a video processing method, a video processing apparatus, a computer-readable storage medium, and an electronic device.
  • a video processing method including: when a first event occurs in a video, starting a video capture task; and within a predetermined time period after the first event ends, determining whether a second event occurs in the video ; If the second event occurs, then within a predetermined time period after the second event ends, determine whether the third event occurs in the video; if the third event occurs, the third event is used as the second event; if the second event does not occur or In the third event, the video clipping task is ended to determine the clipped video segment; wherein, at least two of the first event, the second event and the third event are correlated events with each other.
  • a video processing method comprising: starting a video capture task when a first event occurs in a video; if no event associated with the first event occurs within a predetermined time period after the end of the first event , then end the video clipping task to determine the clipped video clip; if the second event associated with the first event occurs within a predetermined duration after the first event ends, and the second event does not appear within the predetermined duration after the second event ends
  • the video clipping task is ended to determine the clipped video segment.
  • a video processing device comprising: a task initiation module for initiating a video capture task when a first event occurs in a video; an event determination module for initiating a video capture task after the first event ends Within the predetermined duration of the video, determine whether the second event occurs in the video; if the second event occurs, then within the predetermined duration after the second event ends, determine whether the third event occurs in the video; if the third event occurs, then the third event as the second event; the first video clipping module is used to end the video clipping task if the second event or the third event does not occur, so as to determine the clipped video segment; wherein the first event, the second event and the third event At least two of the events are related events.
  • a video processing apparatus comprising: a task initiating module for initiating a video capture task when a first event occurs in a video; a second video capture module for initiating a video capture task if the first event occurs If the associated event of the first event does not occur within the predetermined time period after the end, the video interception task is ended to determine the video clip to be intercepted; the third video interception module is used if the first event occurs within the predetermined time period after the end. A second event associated with an event, and no event associated with the first event occurs within a predetermined period of time after the end of the second event, the video clipping task is terminated to determine the clipped video segment.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the above-mentioned video processing method.
  • an electronic device including a processor; a memory for storing one or more programs, and when the one or more programs are executed by the processor, the processor enables the processor to implement the above-mentioned video processing method.
  • Figure 1 shows a schematic diagram of a video containing user movement events in some technologies
  • Fig. 2 shows the schematic diagram of the interception mode of the fixed duration interception of the video of Fig. 1;
  • Fig. 3 shows the schematic diagram of another example of adopting fixed duration interception
  • FIG. 4 shows a schematic diagram of a video including user movement events in other technologies
  • FIG. 5 shows a schematic diagram of an exemplary system architecture of a video processing solution according to an embodiment of the present disclosure
  • FIG. 6 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure
  • FIG. 7 schematically shows a flowchart of a video processing method according to an exemplary embodiment of the present disclosure
  • FIG. 8 schematically shows a flowchart of the entire process of the video processing solution according to an embodiment of the present disclosure
  • FIG. 9 schematically shows a flowchart of a solution for participating in video capture by the cloud according to another embodiment of the present disclosure.
  • FIG. 10 schematically shows a flowchart of a video processing method according to another exemplary embodiment of the present disclosure
  • FIG. 11 schematically shows a block diagram of a video processing apparatus according to an exemplary embodiment of the present disclosure
  • FIG. 12 schematically shows a block diagram of a video processing apparatus according to another exemplary embodiment of the present disclosure.
  • FIG. 13 schematically shows a block diagram of a video processing apparatus according to yet another exemplary embodiment of the present disclosure
  • FIG. 14 schematically shows a block diagram of a video processing apparatus according to still another exemplary embodiment of the present disclosure.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
  • the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
  • numerous specific details are provided in order to give a thorough understanding of the embodiments of the present disclosure.
  • those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed.
  • well-known solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
  • Figure 1 shows a schematic diagram of a video involving user movement events in some technologies. Referring to FIG. 1 , during the 1 minute from 13:00:00 to 13:01:00, user movement appears in the video.
  • the video shown in FIG. 1 can be intercepted by using a video interception method of intercepting with a fixed duration, so as to obtain a video picture of the user moving.
  • the fixed duration is 5 minutes.
  • a 5-minute video clip from 13:00:00 to 13:05:00 can be captured.
  • a mismatch between the fixed interception duration and the event occurrence duration can cause another result.
  • the event of the user moving appears in the video
  • the fixed duration is configured to be 5 minutes.
  • the 1-minute video clip from 13:05:00 to 13:06:00 cannot be captured, resulting in incomplete user movement events and missing event information.
  • FIG. 4 shows a schematic diagram of a video that includes user movement events in other technologies. Referring to FIG. 4, from 13:00:00 to 13:00:50, there are two events of user movement 1 and user movement 2, and the times are from 13:00:00 to 13:00:10 and 13:00:30, respectively. until 13:00:50.
  • the video clips corresponding to the user movement 1 and the user movement 2 can be extracted separately, and then combined to obtain the clipped video clips.
  • the present disclosure provides a new video processing solution.
  • FIG. 5 shows a schematic diagram of an exemplary system architecture of a video processing solution according to an embodiment of the present disclosure.
  • the system architecture may include a terminal device 51 and a cloud 53 .
  • the terminal device 51 and the cloud 53 may be connected through a network, and the network may include various connection types, such as wired, wireless communication links, or optical fiber cables, and so on.
  • the terminal device 51 can interact with the cloud 53 through the network to receive or send messages and the like.
  • the terminal device 51 may be a mobile phone, a tablet computer, a smart wearable device, a personal computer, various video surveillance devices (doorbell, camera), and the like.
  • the terminal device may also be referred to as a terminal, a mobile terminal, a mobile terminal, a smart terminal, and the like.
  • the cloud 53 may be a single server or a server cluster composed of multiple servers, and the cloud 53 may also be referred to as a cloud server or a server.
  • the terminal device 51 may initiate a video capture task when a first event occurs in the video. Within a predetermined time period after the end of the first event, it is determined whether the second event occurs in the video. If the second event occurs, within a predetermined period of time after the end of the second event, it is determined whether the third event occurs in the video. If the third event occurs, the third event is regarded as the second event, and it continues to determine whether the third event exists within a new predetermined time period, and the loop process is executed. If the terminal device 51 determines that the second event or the third event does not occur, the video clipping task is ended to determine the clipped video segment.
  • the first event, the second event and the third event are mutually related events, more specifically, the first event, the second event and the third event are mutually related events, or, the first event
  • the event may be associated with the second event and the third event, respectively.
  • the associated event may be the same event or a related event, and the related event may be user-defined, or may be preset by the system, for example, a fall event and a crying event are set as associated events.
  • the terminal device 51 can remove the video clips of the last predetermined duration from the clipped video clips to generate the target video clips, and further, can upload the target video clips to the cloud 53 for storage.
  • the target video clip can also be stored locally (which can be understood as a device that performs video capture tasks, such as a camera, a mobile phone, etc.) or on other devices (which can be understood as other devices connected to the local device), such as wirelessly. It is stored in the memory of other devices such as TVs, mobile phones, etc. by means of transmission or cable transmission.
  • the terminal device 51 can transmit the clipped video clips to the designated device, so that the designated device can remove the video clips of the last predetermined duration from the clipped video clips to generate the target video clips.
  • the specified device may be other devices than the terminal device 51, such as a cloud server, a mobile phone, a TV, and the like.
  • the terminal device 51 may upload the clipped video clips to the cloud 53 .
  • the cloud 53 may, in response to the video acquisition request corresponding to the video clip, extract the video clip of the last predetermined duration from the video clip, generate the target video clip, and send the target video clip to the requesting end that initiates the request.
  • the requesting end may be the terminal device 51 or other devices, which are not limited in the present disclosure.
  • the cloud 53 can immediately remove the video clip of the last predetermined duration from the video clip, generate and store the target video clip, so as to receive the above video acquisition request in the cloud 53 In the case of , send the target video clip to the requester.
  • the terminal device 51 may start a video capture task when the first event occurs in the video. If the associated event of the first event does not occur within a predetermined time period after the end of the first event, the video clipping task is ended to determine the clipped video segment. If the second event associated with the first event occurs within a predetermined time period after the end of the first event, and the event associated with the first event does not occur within a predetermined period of time after the end of the second event, the video capture task is terminated to determine the interception out video clips.
  • the cloud 53 may receive video data from the terminal device 51 . Subsequently, the cloud 53 may analyze the video data, and start the video capture task when the first event occurs in the video. Within a predetermined time period after the end of the first event, it is determined whether the second event occurs in the video. If the second event occurs, within a predetermined period of time after the end of the second event, it is determined whether the third event occurs in the video. If the third event occurs, the third event is regarded as the second event, and the loop process is executed. If the cloud 53 determines that the second event or the third event does not occur, the video clipping task is ended to determine the clipped video segment. Wherein, at least two of the first event, the second event and the third event are mutually associated events, and more specifically, the first event, the second event and the third event are mutually associated events.
  • the cloud 53 may further intercept the clipped video clips, so as to eliminate the video clips of the last predetermined duration, and generate target video clips for storage. This process may be performed immediately after the cut-out video segment is determined, or may be performed after a corresponding video acquisition request is received, which is not limited in the present disclosure.
  • the cloud 53 may start a video capture task when the first event occurs in the video. If the associated event of the first event does not occur within a predetermined time period after the end of the first event, the video clipping task is ended to determine the clipped video segment. If a second event associated with the first event occurs within a predetermined time period after the end of the first event, the video clipping task is terminated after the predetermined time period elapses after the end of the second event to determine the clipped video segment.
  • the video processing solution of the present disclosure can be applied in a video surveillance scenario, that is, the video is a video captured by a camera in real time, and real-time analysis is performed to intercept video clips that meet user needs .
  • the video processing solution of the present disclosure can also be used to analyze existing videos.
  • FIG. 6 shows a schematic diagram of an electronic device suitable for use in implementing exemplary embodiments of the present disclosure.
  • the terminal device described in the present disclosure may be configured in the form of an electronic device as shown in FIG. 6 . It should be noted that the electronic device shown in FIG. 6 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device of the present disclosure includes at least a processor and a memory for storing one or more programs, which, when executed by the processor, enable the processor to implement the video processing method of the exemplary embodiment of the present disclosure.
  • the electronic device 600 may include: a processor 610, an internal memory 621, an external memory interface 622, a Universal Serial Bus (USB) interface 630, a charging management module 640, and a power management module 641, battery 642, antenna 1, antenna 2, mobile communication module 650, wireless communication module 660, audio module 670, speaker 671, receiver 672, microphone 673, headphone jack 674, sensor module 680, display screen 690, camera module 691 , an indicator 692, a motor 693, a key 694, a Subscriber Identification Module (SIM) card interface 695, and the like.
  • SIM Subscriber Identification Module
  • the sensor module 680 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.
  • the structures illustrated in the embodiments of the present disclosure do not constitute a specific limitation on the electronic device 600 .
  • the electronic device 600 may include more or less components than shown, or some components may be combined, or some components may be separated, or different component arrangements.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 610 may include one or more processing units, for example, the processor 610 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), controller, video codec, digital signal processor (Digital Signal Processor, DSP), baseband processor and/or neural network processor (Neural-network Processing Unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • a memory may also be provided in the processor 610 for storing instructions and data.
  • the electronic device 600 can realize the shooting function through the ISP, the camera module 691, the video codec, the GPU, the display screen 690, the application processor, and the like.
  • the electronic device 600 may include 1 or N camera modules 691, where N is a positive integer greater than 1. If the electronic device 600 includes N cameras, one of the N cameras is the main camera.
  • Internal memory 621 may be used to store computer executable program code, which includes instructions.
  • the internal memory 621 may include a storage program area and a storage data area.
  • the external memory interface 622 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 600.
  • the present disclosure also provides a computer-readable storage medium.
  • the computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist alone without being assembled into the electronic device.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • the computer-readable storage medium can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • Program code embodied on a computer-readable storage medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • the computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to implement the methods described in the following embodiments.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • the units involved in the embodiments of the present disclosure may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • the video processing method of the exemplary embodiment of the present disclosure may include steps 1 to 4, specifically:
  • step 1 when the first event occurs in the video, a video capture task is started.
  • step 2 within a predetermined period of time after the end of the first event, it is determined whether the second event occurs in the video. If the second event does not occur, go to step 3; if the second event occurs, go to step 4.
  • step 3 the video clipping task is ended to determine the clipped video segment.
  • step 4 within a predetermined time period after the end of the second event, it is determined whether the third event occurs in the video. If the third event occurs, the third event is regarded as the second event to execute step 4 cyclically; if the third event does not occur, step 3 is executed.
  • At least two of the first event, the second event and the third event are mutually associated events.
  • FIG. 7 schematically shows a flowchart of a video processing method according to an exemplary embodiment of the present disclosure.
  • Each step of the video processing method of the present disclosure will be described below by taking the terminal device performing the steps shown in FIG. 7 as an example.
  • the video processing method may include the following steps:
  • the video targeted by the solution of the present disclosure may be a video captured by a camera in real time, and the present disclosure does not limit the content of the video (ie, the object captured by the camera).
  • the camera may be a fixed camera, such as a monitoring camera in a parking lot or a manufacturing workshop.
  • the camera may also be a mobile camera, for example, a camera on a mobile phone, and a user can perform mobile shooting through the camera to obtain surrounding scene information.
  • the video targeted by the solution of the present disclosure may also be a video that has been shot, and is obtained from the memory when the video needs to be analyzed. Similarly, the present disclosure does not limit the video type of the video that has been shot.
  • the first event may be a preset event
  • the preset event may include a user preset event or a system preset event.
  • the user preset event may be an event demonstrated by the user in advance, and the terminal device may photograph and save the event demonstrated by the user. For example, taking the preset event as an event of the appearance of a human face as an example, the terminal device can capture an image containing a human face and an image that does not contain a human face, and then the user can select an image containing a human face on a preset event configuration interface As an image containing preset events.
  • the preset event may also be an event preset by the system when the terminal device leaves the factory, and the present disclosure does not limit the type of the preset event.
  • the first event may be an event of interest to the user, or an event of a predetermined type preset by the system.
  • the first event may be any one or more of situations such as the presence of a human face in the shooting scene, the movement of an object (such as a person, an animal, etc.), the device sending a prompt signal in the scene, crying, screaming, falling, etc.
  • the type of the first event is not limited.
  • the terminal device may extract video frame images from the video at predetermined time intervals.
  • the predetermined time interval is related to the scene type and can be set based on the scene, and the value of the predetermined time interval is not limited in the present disclosure.
  • each frame of image in the video can be extracted for processing.
  • feature extraction can be performed on the video frame images.
  • a machine learning model based on deep learning can be used to process the video frame images to extract features of the video frame images.
  • the present disclosure does not limit the structure and training process of the machine learning model.
  • a method such as a histogram can also be used to extract the features of the video frame images, which is not limited in the present disclosure.
  • the output of a machine learning model can be the result of whether a preset event occurs.
  • further analysis can be performed according to the features extracted by the machine learning model to obtain a result of whether a preset event occurs.
  • the video frame image can be input into the trained convolutional neural network, and the convolutional neural network can perform feature extraction to classify whether there is a cat in the video frame image. .
  • a solution for judging whether a first event (or referred to as a preset event) occurs based on multiple frames is also provided.
  • the target video frame image in which the preset object appears for the first time can be determined from the video according to the extracted features.
  • the preset object is an object that determines that an event is a preset event. It can be understood that the preset object can be used as an identifier of the preset event.
  • the target video frame image is used as the starting point of the preset event.
  • a face appears in the 5th frame, it is judged whether a face also appears in the 6th frame, or whether a face appears in a predetermined number of subsequent video frames (such as the 6th to the 1st frame). 10 frames). If it is determined that there is a human face in these frames, it can be determined that a human face appears in the video, and the fifth frame is taken as the starting point of the human face appearing.
  • the terminal device may start a video capture task.
  • the video clipping task can be started from the target video frame image.
  • the above example is still described, and the video capture task can be started from the fifth frame image.
  • the operation of initiating a video capture task includes starting a video capture operation.
  • the operation of initiating the video capture task includes starting to record the video.
  • the operation of initiating the video capture task includes recording the time when the first event begins to appear in the video as the video capture start time.
  • the time when the first event begins to appear is the time point in the video when the first event starts from nothing, that is, the instantaneous time point from the absence of the first event to the appearance of the first event.
  • the video clipping start time may be a time in the video, that is, it represents a relative time.
  • the video clipping start time may also represent an absolute time in reality, which is not limited in the present disclosure.
  • the association of the second event with the first event means that the second event and the first event are of the same event type. For example, all faces appear, all users move, all other designated objects (eg, cats, designated devices, etc.) exist.
  • the association between the second event and the first event may also refer to: the second event is the same as the first event.
  • both the second event and the first event appear on the face of user A.
  • the same here means that the images corresponding to the events are the same, and it is not necessary that the positions and sizes of the images appear exactly the same.
  • the association of the second event with the first event means that the second event may be a subsequent event of the first event.
  • assembling an item includes two steps: process a and process b. It is necessary to execute process a and then process b. In this case, the event corresponding to process a is the first event, and the event corresponding to process b is the second event. .
  • a timer may be started to determine whether the second event occurs in the video within a predetermined period of time.
  • the predetermined duration is related to the application scenario of the solution of the present disclosure, and may be, for example, 10 seconds, 30 seconds, etc., which is not limited in the present disclosure.
  • step S70 it is determined whether the next event corresponding to the event occurs within a predetermined period of time. For example, in the case of detecting a human face, when the human face disappears from the video, the timer starts, and within a predetermined period of time, it is detected whether there is another human face.
  • the manner of determining whether the second event exists may be the same as the manner of determining the second event in step S70, that is, whether the event occurs may be determined by analyzing the video frame images.
  • step S74 If it is determined that the second event occurs in the video, the terminal device executes step S74; if it is determined that the second event does not appear in the video, the terminal device executes step S78.
  • one or more frames may be combined to detect whether the first event ends.
  • the 20th frame image it is found in the 20th frame image that the first event ends. In this case, the judgment process of one or more frames can be carried out. If the first event does not appear, the 20th frame is regarded as the first event. end image.
  • the second event when using multiple frames of images to determine whether the second event occurs, it can be set so that when multiple frames of images (for example, 3-frame images, 5-frame images, etc.) with objects corresponding to the second event all appear in the Within the predetermined time period, it can be determined that the second event occurs in the video, or it can be determined that the second event occurs in the video as long as the image with the object corresponding to the second event appears within the predetermined time period.
  • multiple frames of images for example, 3-frame images, 5-frame images, etc.
  • the third event may be associated with the first event or the second event, and the meaning of association mentioned here is the same as the association described in step S72, and details are not repeated here. It should be noted that at least two of the first event, the second event and the third event are mutually related events, and more specifically, the first event, the second event and the third event are mutually related events.
  • the terminal device may determine whether the third event occurs in the video within a predetermined period of time after the end of the second event.
  • step S76 If it is determined that the third event occurs in the video, the terminal device executes step S76; if it is determined that the third event does not appear in the video, the terminal device executes step S78.
  • the third event when using multiple frames of images to determine whether the third event occurs, it can be set so that when multiple frames of images (for example, 3-frame images, 5-frame images, etc.) with objects corresponding to the third event all appear in the Within the predetermined time period, it can be determined that the third event occurs in the video, or it can be determined that the third event occurs in the video as long as the image with the object corresponding to the third event appears within the predetermined time period.
  • multiple frames of images for example, 3-frame images, 5-frame images, etc.
  • step S74 If it is determined in step S74 that the third event occurs in the video, the third event is regarded as the second event, and the process returns to step S74 to perform the operation of determining whether the third event occurs in the video within a predetermined time period after the end of the second event.
  • a loop process of steps S74 and S76 is formed.
  • the predetermined duration is 10 seconds. If event b associated with event a occurs within 10 seconds after event a ends, continue to judge whether an event associated with event a (or event b) occurs within 10 seconds after event b ends, and if associated event c occurs, Then continue to judge whether an event associated with the previous event occurs within 10 seconds after the end of event c, and so on.
  • the operation of the terminal device to end the video capture task includes: ending the video capture operation. Specifically, when the video is a video captured by a camera in real time, ending the video capture task includes stopping the video recording.
  • the operation of starting the video interception task includes recording the start time of video interception
  • the operation of the terminal device to end the video interception task includes: recording after determining that the first event ends After a predetermined period of time, it is used as the end time of video interception.
  • a video clipping time period may be determined based on the video clipping start time and the video clipping end time, and a clipping operation is performed for the clipped video segment to determine the clipped video segment.
  • the operation of the terminal device to end the video capture task includes: recording a predetermined time elapsed after the end of the second event is determined as the video capture end time.
  • a video clipping time period may be determined based on the video clipping start time and the video clipping end time, and a clipping operation is performed for the clipped video segment to determine the clipped video segment.
  • the video clipping start time is 01:30
  • the video clipping end time is 03:00.
  • the terminal device can clip the video clip corresponding to 01:30 to 03:00 from the video, that is, determine Take out the clipped video clip.
  • the terminal device After determining the clipped video clip, since there is no corresponding event within the last predetermined duration of the video clip, in this case, the terminal device can remove the clipped video clip with the last predetermined duration, and generate the target video clip . In addition, the terminal device can upload the target video clip to the cloud for storage.
  • the cloud can respond to a video acquisition request corresponding to the target video clip sent by the terminal device or other device, and send the target video clip to the device that sends the request.
  • the terminal device can directly upload the clipped video clips to the cloud.
  • the cloud may, in response to the video acquisition request corresponding to the clipped video clip, remove the video clip of the last predetermined duration from the clipped video clip to generate the target video clip, and use the clipped video clip to generate the target video clip.
  • the target video clip is sent to the requester that initiates the video acquisition request, so that the user can watch it.
  • the cloud can remove the video clips of the last predetermined duration from the clipped video clips, generate and store the target video clips, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips and store the target video clips. It is sent to the requester that initiates the video acquisition request so that users can watch it.
  • at least two consecutive associated events (including the same event) within a preset time interval can be intercepted from the video, and each associated event is not interrupted (any two consecutive events described above). The video between the associated events is also captured), thereby improving the user’s viewing effect. Since the preset time is set, it can avoid that two consecutive associated events with a long interval are included in the same captured video clip, to a certain extent The storage capacity is reduced, so that it is convenient to seek a balance between storage capacity and viewing effect.
  • the terminal device monitors the video captured by the camera in real time.
  • the camera can be integrated on the terminal device, and in addition, the camera can also establish a connection with the terminal device in a wired or wireless manner, so that the terminal device can obtain the video.
  • step S804 the terminal device determines whether a preset event occurs in the video. If it appears, go to step S806; if not, go back to step S802.
  • step S806 after the preset event ends, the video recording is extended for N seconds, where N seconds corresponds to the above-mentioned predetermined duration, for example, 10 seconds, 30 seconds, and the like.
  • step S808 the terminal device determines whether a preset event occurs again within N seconds. If so, go back to step S806; if not, go to step S810.
  • step S810 the terminal device determines a clipped video clip, where the clipped video clip includes a video clip of N seconds after the end of the last preset event.
  • step S812 the terminal device truncates the video clips of the last N seconds from the video clips determined in step S810, and uploads them to the cloud for saving.
  • FIG. 9 schematically shows a flowchart of a solution for participating in video capture by the cloud according to another embodiment of the present disclosure.
  • step S902 the cloud acquires and stores the video clips cut out by the terminal device.
  • the process of determining the clipped video segment by the terminal device may be as shown in the above steps S802 to S810.
  • step S904 the cloud receives a video acquisition request corresponding to the video clip.
  • step S906 the cloud may truncate the last N seconds of the video clip and send it to the requester of the video acquisition request.
  • the present disclosure also provides another video processing method for a scene that only needs to output a video containing two related events.
  • the video processing method may include the following steps:
  • Step S102 is the same as the above-mentioned step S70, and is not repeated here.
  • the terminal device can determine whether an event associated with the first event occurs within a predetermined time period after the first event ends, and if so, ends the video capture task to determine the video clip to be captured.
  • the process of ending the video clipping task to determine the clipped video segment is similar to the process of step S78, and will not be repeated.
  • the terminal device can end the video clipping task to determine the clipped video segment.
  • the video capture task is terminated after a predetermined period of time, so as to avoid some relevant information that may exist in the video within the predetermined period of time being captured. Missing or discarding issues.
  • the solution of the present disclosure can also eliminate the video of the last predetermined duration, and generate a target video segment for storage.
  • the terminal device may remove the video clip of the last predetermined duration from the clipped video clip to generate the target video clip.
  • the terminal device can upload the target video clip to the cloud for storage.
  • the cloud can respond to a video acquisition request corresponding to the target video clip sent by the terminal device or other device, and send the target video clip to the device that sends the request.
  • the terminal device can directly upload the clipped video clips to the cloud.
  • the cloud may, in response to a video acquisition request corresponding to the clipped video clip, remove the video clip of the last predetermined duration from the clipped video clip to generate the target video clip, and use the clipped video clip to generate the target video clip.
  • the target video clip is sent to the requester that initiates the video acquisition request, so that the user can watch it.
  • the cloud can remove the video clips of the last predetermined duration from the clipped video clips, generate and store the target video clips, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips. It is sent to the requester that initiates the video acquisition request so that users can watch it.
  • the solution of the present disclosure can intercept multiple video clips associated with events from the video; on the other hand, the clipped video clips are continuous video clips, ensuring that the video clips viewed by the user Continuous and complete events; on the other hand, storage based on the clipped video clips can greatly save storage space.
  • this exemplary embodiment also provides a video processing apparatus.
  • FIG. 11 schematically shows a block diagram of a video processing apparatus of an exemplary embodiment of the present disclosure.
  • the video processing apparatus 11 may include a task initiation module 111 , an event detection module 113 and a first video capture module 115 .
  • the task initiation module 111 can be used to start the video capture task when the first event occurs in the video; the event detection module 113 can be used to determine whether the second event occurs in the video within a predetermined time period after the first event ends; If the second event occurs, determine whether the third event occurs in the video within a predetermined period of time after the second event ends; if the third event occurs, the third event is taken as the second event; the first video interception module 115 can use If the second event or the third event does not occur, end the video clipping task to determine the clipped video segment; wherein, at least two of the first event, the second event and the third event are correlated events with each other.
  • the first video clipping module 115 may be further configured to perform: excluding a video clip of the last predetermined duration from the clipped video clips to generate a target video clip.
  • the first video clipping module 115 may be further configured to perform: transmitting the clipped video segment to a designated device, so that the designated device can remove the video of the last predetermined duration from the clipped video clip segment to generate the target video segment.
  • the video processing apparatus 12 may further include a video segment uploading module 121 .
  • the video clip uploading module 121 may be configured to perform: uploading the clipped video clips to the cloud.
  • the cloud in response to the video acquisition request corresponding to the clipped video clip, the cloud removes the video clip of the last predetermined duration from the clipped video clip, generates the target video clip, and sends the target video clip to the initiating video clip.
  • the requesting end of the request; or, the cloud removes the video clips of the last predetermined duration from the clipped video clips, generates the target video clips and stores them, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips and send the target video clips.
  • the cloud removes the video clip of the last predetermined duration from the clipped video clips, generates the target video clips and stores them, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips and send the target video clips.
  • the process in which the task initiating module 111 initiates a video capture task may be configured to perform: when a first event occurs in the video, start a video capture operation.
  • the process of the first video capture module 115 ending the video capture task may be configured to perform: end the video capture operation.
  • the process of initiating the video capture task by the task initiation module 111 may be configured to perform: recording the time when the first event starts to appear in the video as the video capture start time.
  • the process of the first video clipping module 115 ending the video clipping task to determine the clipped video segment may be configured to perform: in the case that the second event does not occur, record the process after it is determined that the first event ends.
  • the predetermined length of time is used as the end time of video interception. Based on the start time of video interception and the end time of video interception, the video is intercepted to determine the video clip to be intercepted; if the third event does not occur, record and determine the first video clip. After the second event ends, a predetermined period of time is used as the video clipping end time. Based on the video clipping start time and the video clipping end time, the video clipping operation is performed to determine the clipped video segment.
  • the first event is a preset event
  • the preset event includes a user preset event or a system preset event.
  • the video processing apparatus 13 may further include an image analysis module 131 .
  • the image analysis module 131 may be configured to perform: extracting features from video frame images in the video; and determining whether a preset event occurs in the video according to the extracted features.
  • the process in which the image analysis module 131 determines whether a preset event occurs in the video according to the extracted features may be configured to perform: according to the extracted features, determine from the video the first occurrence of the preset object in the video
  • the target video frame image the preset object is an object that determines an event as a preset event; if there are preset objects in one or more frames of video frame images after the target video frame image, it is determined that a preset event occurs in the video; Wherein, starting from the target video frame image, the video interception task is started.
  • the above video is a video captured by a camera in real time.
  • FIG. 14 schematically shows a block diagram of a video processing apparatus according to another exemplary embodiment of the present disclosure.
  • the video processing apparatus 14 may include a task initiation module 111 , a second video capture module 141 and a third video capture module 143 .
  • the task initiation module 111 can be used to start the video capture task when the first event occurs in the video; the second video capture module 141 can be used to associate the first event if the first event does not occur within a predetermined period of time after the end of the first event event, then end the video clipping task to determine the clipped video segment; the third video clipping module 143 can be used if the second event associated with the first event occurs within a predetermined time period after the first event ends, and the second event occurs in the second event. After the event ends, if no related event of the first event occurs within a predetermined period of time, the video interception task is ended to determine the intercepted video segment.
  • the third video clipping module 143 may be further configured to perform: excluding a video clip of the last predetermined duration from the clipped video clips to generate a target video clip.
  • the third video clipping module 143 may be further configured to perform: transmitting the clipped video segment to a designated device, so that the designated device can remove the video of the last predetermined duration from the clipped video clip segment to generate the target video segment.
  • the video processing apparatus 14 may further include the above-mentioned video clip uploading module 121 .
  • the exemplary embodiments described herein may be implemented by software, or may be implemented by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to an embodiment of the present disclosure.
  • a computing device which may be a personal computer, a server, a terminal device, or a network device, etc.
  • modules or units of the apparatus for action performance are mentioned in the above detailed description, this division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into multiple modules or units to be embodied.

Abstract

A video processing method, a video processing apparatus, a computer readable storage medium, and an electronic device, which relate to the technical field of video processing. The video processing method comprises: when a first event occurs in a video, starting a video interception task (S70); within a predetermined duration after the first event ends, determining whether a second event occurs in the video (S72); if the second event occurs, then, within a predetermined duration after the second event end, determining whether a third event occurs in the videos (S74); if the third event occurs, using the third event as the second event (S76); and if the second event or the third event does not occur, ending the video interception task so as to determine an intercepted video clip (S78). At least two among the first event, the second event, and the third event are associated events. According to said method, a plurality of video clips of associated events can be intercepted from a video, and the video clips are continuous video clips, thus ensuring that the video clips viewed by a user are continuous and the events are complete. (FIG. 7)

Description

视频处理方法及装置、计算机可读存储介质和电子设备Video processing method and apparatus, computer-readable storage medium and electronic device
相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS
本申请要求于2021年01月21日提交的申请号为202110082809.3、名称为“视频处理方法及装置、计算机可读存储介质和电子设备”的中国专利申请的优先权,该中国专利申请的全部内容通过引用全部并入本文。This application claims the priority of the Chinese patent application with the application number 202110082809.3 and the title of "video processing method and device, computer-readable storage medium and electronic equipment" filed on January 21, 2021, the entire content of the Chinese patent application Incorporated herein by reference in its entirety.
技术领域technical field
本公开涉及视频处理技术领域,具体而言,涉及一种视频处理方法、视频处理装置、计算机可读存储介质和电子设备。The present disclosure relates to the technical field of video processing, and in particular, to a video processing method, a video processing apparatus, a computer-readable storage medium, and an electronic device.
背景技术Background technique
视频作为传递信息的一种重要方式,已广泛应用于监控、教育、娱乐、医疗、智能驾驶等众多领域。As an important way to transmit information, video has been widely used in many fields such as monitoring, education, entertainment, medical care, and intelligent driving.
视频中往往存在一些用户不关注的内容,这些内容在视频中的比例可能较大,用户的观看体验差且存储压力大。目前,出现了一些对视频进行截取的方案。然而,这些截取视频的方案可能出现丢失用户关注的信息等截取效果不佳的问题。There are often some content that users do not pay attention to in the video, and the proportion of these content in the video may be large, the user's viewing experience is poor, and the storage pressure is large. At present, there are some solutions for intercepting video. However, these solutions for intercepting video may have problems such as loss of information that the user cares about and poor interception effect.
发明内容SUMMARY OF THE INVENTION
根据本公开的第一方面,提供了一种视频处理方法,包括:在视频中出现第一事件时,启动视频截取任务;在第一事件结束后的预定时长内,确定视频是否出现第二事件;如果出现第二事件,则在第二事件结束后的预定时长内,确定视频是否出现第三事件;如果出现第三事件,则将第三事件作为第二事件;如果未出现第二事件或第三事件,则结束视频截取任务,以确定截取出的视频片段;其中,第一事件、第二事件和第三事件中至少两个互为关联事件。According to a first aspect of the present disclosure, a video processing method is provided, including: when a first event occurs in a video, starting a video capture task; and within a predetermined time period after the first event ends, determining whether a second event occurs in the video ; If the second event occurs, then within a predetermined time period after the second event ends, determine whether the third event occurs in the video; if the third event occurs, the third event is used as the second event; if the second event does not occur or In the third event, the video clipping task is ended to determine the clipped video segment; wherein, at least two of the first event, the second event and the third event are correlated events with each other.
根据本公开的第二方面,提供了一种视频处理方法,包括:在视频中出现第一事件时,启动视频截取任务;如果第一事件结束后的预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段;如果第一事件结束后的预定时长内出现与第一事件关联的第二事件,且在第二事件结束后经历预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。According to a second aspect of the present disclosure, there is provided a video processing method, comprising: starting a video capture task when a first event occurs in a video; if no event associated with the first event occurs within a predetermined time period after the end of the first event , then end the video clipping task to determine the clipped video clip; if the second event associated with the first event occurs within a predetermined duration after the first event ends, and the second event does not appear within the predetermined duration after the second event ends When an event is associated with an event, the video clipping task is ended to determine the clipped video segment.
根据本公开的第三方面,提供了一种视频处理装置,包括:任务启动模块,用于在视频中出现第一事件时,启动视频截取任务;事件确定模块,用于在第一事件结束后的预定时长内,确定视频是否出现第二事件;如果出现第二事件,则在第二事件结束后的预定时长内,确定视频是否出现第三事件;如果出现第三事件,则将第三事件作为第二事件;第一视频截取模块,用于如果未出现第二事件或第三事件,则结束视频截取任务,以确定截取出的视频片段;其中,第一事件、第二事件和第三事件中至少两个互为关联事件。According to a third aspect of the present disclosure, there is provided a video processing device, comprising: a task initiation module for initiating a video capture task when a first event occurs in a video; an event determination module for initiating a video capture task after the first event ends Within the predetermined duration of the video, determine whether the second event occurs in the video; if the second event occurs, then within the predetermined duration after the second event ends, determine whether the third event occurs in the video; if the third event occurs, then the third event as the second event; the first video clipping module is used to end the video clipping task if the second event or the third event does not occur, so as to determine the clipped video segment; wherein the first event, the second event and the third event At least two of the events are related events.
根据本公开的第四方面,提供了一种视频处理装置,包括:任务启动模块,用于在视频中出现第一事件时,启动视频截取任务;第二视频截取模块,用于如果第一事件结束后的预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段;第三视频截取模块,用于如果第一事件结束后的预定时长内出现与第一事件关联的第二事件,且在第二事件结束后经历预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。According to a fourth aspect of the present disclosure, there is provided a video processing apparatus, comprising: a task initiating module for initiating a video capture task when a first event occurs in a video; a second video capture module for initiating a video capture task if the first event occurs If the associated event of the first event does not occur within the predetermined time period after the end, the video interception task is ended to determine the video clip to be intercepted; the third video interception module is used if the first event occurs within the predetermined time period after the end. A second event associated with an event, and no event associated with the first event occurs within a predetermined period of time after the end of the second event, the video clipping task is terminated to determine the clipped video segment.
根据本公开的第五方面,提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现上述的视频处理方法。According to a fifth aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the above-mentioned video processing method.
根据本公开的第六方面,提供了一种电子设备,包括处理器;存储器,用于存储一个或多个程序,当一个或多个程序被处理器执行时,使得所述处理器实现上述的视频处理方 法。According to a sixth aspect of the present disclosure, there is provided an electronic device including a processor; a memory for storing one or more programs, and when the one or more programs are executed by the processor, the processor enables the processor to implement the above-mentioned video processing method.
附图说明Description of drawings
图1示出了一些技术中包含用户移动事件的视频示意图;Figure 1 shows a schematic diagram of a video containing user movement events in some technologies;
图2示出了对图1视频进行固定时长截取的截取方式的示意图;Fig. 2 shows the schematic diagram of the interception mode of the fixed duration interception of the video of Fig. 1;
图3示出了采用固定时长截取的另一个实例的示意图;Fig. 3 shows the schematic diagram of another example of adopting fixed duration interception;
图4示出了另一些技术中包含用户移动事件的视频示意图;FIG. 4 shows a schematic diagram of a video including user movement events in other technologies;
图5示出了本公开实施例的视频处理方案的示例性系统架构的示意图;5 shows a schematic diagram of an exemplary system architecture of a video processing solution according to an embodiment of the present disclosure;
图6示出了适于用来实现本公开实施例的电子设备的结构示意图;FIG. 6 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure;
图7示意性示出了根据本公开示例性实施方式的视频处理方法的流程图;FIG. 7 schematically shows a flowchart of a video processing method according to an exemplary embodiment of the present disclosure;
图8示意性示出了根据本公开实施例的视频处理方案的整个过程的流程图;FIG. 8 schematically shows a flowchart of the entire process of the video processing solution according to an embodiment of the present disclosure;
图9示意性示出了根据本公开另一实施例的由云端参与视频截取的方案的流程图;FIG. 9 schematically shows a flowchart of a solution for participating in video capture by the cloud according to another embodiment of the present disclosure;
图10示意性示出了根据本公开另一示例性实施方式的视频处理方法的流程图;FIG. 10 schematically shows a flowchart of a video processing method according to another exemplary embodiment of the present disclosure;
图11示意性示出了根据本公开示例性实施方式的视频处理装置的方框图;FIG. 11 schematically shows a block diagram of a video processing apparatus according to an exemplary embodiment of the present disclosure;
图12示意性示出了根据本公开另一示例性实施方式的视频处理装置的方框图;FIG. 12 schematically shows a block diagram of a video processing apparatus according to another exemplary embodiment of the present disclosure;
图13示意性示出了根据本公开又一示例性实施方式的视频处理装置的方框图;FIG. 13 schematically shows a block diagram of a video processing apparatus according to yet another exemplary embodiment of the present disclosure;
图14示意性示出了根据本公开再一示例性实施方式的视频处理装置的方框图。FIG. 14 schematically shows a block diagram of a video processing apparatus according to still another exemplary embodiment of the present disclosure.
具体实施方式Detailed ways
现在将参考附图更全面地描述示例实施方式。然而,示例实施方式能够以多种形式实施,且不应被理解为限于在此阐述的范例;相反,提供这些实施方式使得本公开将更加全面和完整,并将示例实施方式的构思全面地传达给本领域的技术人员。所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施方式中。在下面的描述中,提供许多具体细节从而给出对本公开的实施方式的充分理解。然而,本领域技术人员将意识到,可以实践本公开的技术方案而省略所述特定细节中的一个或更多,或者可以采用其它的方法、组元、装置、步骤等。在其它情况下,不详细示出或描述公知技术方案以避免喧宾夺主而使得本公开的各方面变得模糊。Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments, however, can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
此外,附图仅为本公开的示意性图解,并非一定是按比例绘制。图中相同的附图标记表示相同或类似的部分,因而将省略对它们的重复描述。附图中所示的一些方框图是功能实体,不一定必须与物理或逻辑上独立的实体相对应。可以采用软件形式来实现这些功能实体,或在一个或多个硬件模块或集成电路中实现这些功能实体,或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the figures are functional entities that do not necessarily necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
附图中所示的流程图仅是示例性说明,不是必须包括所有的步骤。例如,有的步骤还可以分解,而有的步骤可以合并或部分合并,因此实际执行的顺序有可能根据实际情况改变。另外,下面所有的术语“第一”、“第二”、“第三”仅是为了区分的目的,不应作为本公开内容的限制;本申请的实施例、实施方式以及其中的具体技术特征在不冲突的情况下,可以相互组合。The flow charts shown in the figures are merely illustrative and do not necessarily include all steps. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may be changed according to the actual situation. In addition, all the following terms "first", "second" and "third" are only for the purpose of distinction and should not be used as a limitation of the present disclosure; the examples, implementations and specific technical features of the present application They can be combined with each other without conflict.
图1示出了一些技术中包含用户移动事件的视频示意图。参考图1,在13:00:00至13:01:00这1分钟内,视频中出现了用户移动。Figure 1 shows a schematic diagram of a video involving user movement events in some technologies. Referring to FIG. 1 , during the 1 minute from 13:00:00 to 13:01:00, user movement appears in the video.
根据本公开一些技术的方案,可以采用固定时长截取的视频截取方式对图1所示的视频进行截取,以得到用户移动的视频画面。例如参考图2,固定时长为5分钟,在这种情况下,可以截取出13:00:00至13:05:00这5分钟的视频片段。According to some technical solutions of the present disclosure, the video shown in FIG. 1 can be intercepted by using a video interception method of intercepting with a fixed duration, so as to obtain a video picture of the user moving. For example, referring to FIG. 2 , the fixed duration is 5 minutes. In this case, a 5-minute video clip from 13:00:00 to 13:05:00 can be captured.
然而,13:01:00至13:05:00这4分钟及之后并不存在用户移动的事件,如果也截取这4分钟,会浪费存储空间,用户在观看时,也浪费了用户的时间,体验较差。However, there is no user movement event during the 4 minutes from 13:01:00 to 13:05:00 and after that. If these 4 minutes are also intercepted, the storage space will be wasted, and the user will also waste the user's time when watching. Bad experience.
另外,固定截取时长与事件发生时长不匹配还会造成另一种结果。例如参考图3,在 13:00:00至13:06:00共6分钟内,视频中均出现了用户移动的事件,而固定时长被配置为5分钟。在这种情况下,13:05:00至13:06:00这1分钟的视频片段无法被截取出,造成用户移动事件不完整,遗漏事件信息的问题。In addition, a mismatch between the fixed interception duration and the event occurrence duration can cause another result. For example, referring to FIG. 3 , from 13:00:00 to 13:06:00 for a total of 6 minutes, the event of the user moving appears in the video, and the fixed duration is configured to be 5 minutes. In this case, the 1-minute video clip from 13:05:00 to 13:06:00 cannot be captured, resulting in incomplete user movement events and missing event information.
图4示出了另一些技术中包含用户移动事件的视频示意图。参考图4,在13:00:00至13:00:50内,存在用户移动1和用户移动2两个事件,时间分别为13:00:00至13:00:10和13:00:30至13:00:50。FIG. 4 shows a schematic diagram of a video that includes user movement events in other technologies. Referring to FIG. 4, from 13:00:00 to 13:00:50, there are two events of user movement 1 and user movement 2, and the times are from 13:00:00 to 13:00:10 and 13:00:30, respectively. until 13:00:50.
利用本公开的一些方案,可以对用户移动1和用户移动2对应的视频片段分别提取,然后进行合并,以得到截取出的视频片段。With some solutions of the present disclosure, the video clips corresponding to the user movement 1 and the user movement 2 can be extracted separately, and then combined to obtain the clipped video clips.
然而,一方面,这种合并会造成最终得到的视频片段不连续,影响用户观看;另一方面,视频片段之间拼接的处理较为复杂,不易实施。However, on the one hand, such merging will cause discontinuity of the finally obtained video clips, which affects the viewing of users; on the other hand, the processing of splicing between video clips is complicated and difficult to implement.
鉴于此,本公开提供了一种新的视频处理方案。In view of this, the present disclosure provides a new video processing solution.
图5示出了本公开实施例的视频处理方案的示例性系统架构的示意图。FIG. 5 shows a schematic diagram of an exemplary system architecture of a video processing solution according to an embodiment of the present disclosure.
如图5所示,系统架构可以包括终端设备51和云端53。终端设备51与云端53可以通过网络进行连接,网络可以包括各种连接类型,例如有线、无线通信链路或者光纤电缆等等。As shown in FIG. 5 , the system architecture may include a terminal device 51 and a cloud 53 . The terminal device 51 and the cloud 53 may be connected through a network, and the network may include various connection types, such as wired, wireless communication links, or optical fiber cables, and so on.
终端设备51可以通过网络与云端53交互,以接收或发送消息等。终端设备51可以是手机、平板电脑、智能可穿戴设备、个人计算机、各类视频监控设备(门铃、摄像头)等。在不同场景下,终端设备还可以被称为终端、移动终端、移动端、智能终端等。另外,云端53可以是单个服务器,也可以是由多个服务器组成的服务器集群,云端53还可以被称为云端服务器或服务器。The terminal device 51 can interact with the cloud 53 through the network to receive or send messages and the like. The terminal device 51 may be a mobile phone, a tablet computer, a smart wearable device, a personal computer, various video surveillance devices (doorbell, camera), and the like. In different scenarios, the terminal device may also be referred to as a terminal, a mobile terminal, a mobile terminal, a smart terminal, and the like. In addition, the cloud 53 may be a single server or a server cluster composed of multiple servers, and the cloud 53 may also be referred to as a cloud server or a server.
在由终端设备51执行本公开视频处理方案的一些实例中,终端设备51可以在视频中出现第一事件时,启动视频截取任务。在第一事件结束后的预定时长内,确定视频是否出现第二事件。如果出现第二事件,则在第二事件结束后的预定时长内,确定视频是否出现第三事件。如果出现第三事件,则将第三事件作为第二事件,继续在新的预定时长内确定是否存在第三事件,执行循环过程。如果终端设备51确定出未出现第二事件或第三事件,则结束视频截取任务,以确定截取出的视频片段。其中,第一事件、第二事件和第三事件中至少两个互为关联事件,更具体的,第一事件、第二事件和第三事件三者之间互为关联事件,或者,第一事件可以分别与第二事件和第三事件为关联事件。需要说明的是,关联事件可以是相同事件也可以是相关事件,相关事件可以用户自定义,也可以是系统预设,例如跌倒事件和哭泣事件设为关联事件。In some instances where the terminal device 51 performs the video processing solution of the present disclosure, the terminal device 51 may initiate a video capture task when a first event occurs in the video. Within a predetermined time period after the end of the first event, it is determined whether the second event occurs in the video. If the second event occurs, within a predetermined period of time after the end of the second event, it is determined whether the third event occurs in the video. If the third event occurs, the third event is regarded as the second event, and it continues to determine whether the third event exists within a new predetermined time period, and the loop process is executed. If the terminal device 51 determines that the second event or the third event does not occur, the video clipping task is ended to determine the clipped video segment. Wherein, at least two of the first event, the second event and the third event are mutually related events, more specifically, the first event, the second event and the third event are mutually related events, or, the first event The event may be associated with the second event and the third event, respectively. It should be noted that the associated event may be the same event or a related event, and the related event may be user-defined, or may be preset by the system, for example, a fall event and a crying event are set as associated events.
在一个实施例中,终端设备51可以从截取出的视频片段中剔除最后预定时长的视频片段,以生成目标视频片段,进一步的,可以将目标视频片段上传至云端53进行存储。可以理解的,目标视频片段也可以存储在本地(可以理解为执行视频截取任务的设备上,例如摄像头、手机等)或者其他设备上(可以理解为与本地设备连接的其他设备),例如通过无线传输或有线传输的方式存储到电视、手机等其他设备的存储器上。In one embodiment, the terminal device 51 can remove the video clips of the last predetermined duration from the clipped video clips to generate the target video clips, and further, can upload the target video clips to the cloud 53 for storage. It can be understood that the target video clip can also be stored locally (which can be understood as a device that performs video capture tasks, such as a camera, a mobile phone, etc.) or on other devices (which can be understood as other devices connected to the local device), such as wirelessly. It is stored in the memory of other devices such as TVs, mobile phones, etc. by means of transmission or cable transmission.
可以理解的,终端设备51可以将截取出的视频片段传输至指定设备,以供指定设备从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。指定设备可以是终端设备51之外的其他设备,例如云端服务器、手机、电视等。It can be understood that the terminal device 51 can transmit the clipped video clips to the designated device, so that the designated device can remove the video clips of the last predetermined duration from the clipped video clips to generate the target video clips. The specified device may be other devices than the terminal device 51, such as a cloud server, a mobile phone, a TV, and the like.
在另一个实施例中,终端设备51可以将截取出的视频片段上传至云端53。云端53可以响应于该视频片段对应的视频获取请求,从该视频片段中提取最后预定时长的视频片段,生成目标视频片段,并将目标视频片段发送给发起请求的请求端。该请求端可以是终端设备51,也可以是其他设备,本公开对此不做限制。In another embodiment, the terminal device 51 may upload the clipped video clips to the cloud 53 . The cloud 53 may, in response to the video acquisition request corresponding to the video clip, extract the video clip of the last predetermined duration from the video clip, generate the target video clip, and send the target video clip to the requesting end that initiates the request. The requesting end may be the terminal device 51 or other devices, which are not limited in the present disclosure.
另外,云端53在接收到终端设备51发送的截取出的视频片段后,可以立即从该视频片段中剔除最后预定时长的视频片段,生成目标视频片段并存储,以便在云端53接收上述视频获取请求的情况下,将目标视频片段发送给请求端。In addition, after receiving the clipped video clip sent by the terminal device 51, the cloud 53 can immediately remove the video clip of the last predetermined duration from the video clip, generate and store the target video clip, so as to receive the above video acquisition request in the cloud 53 In the case of , send the target video clip to the requester.
在由终端设备51执行本公开视频处理方案的另一些实例中,终端设备51可以在视频中出现第一事件时,启动视频截取任务。如果第一事件结束后的预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。如果第一事件结束后的预定时长内出现与第一事件关联的第二事件,且在第二事件结束后经历预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。In other examples in which the terminal device 51 performs the video processing solution of the present disclosure, the terminal device 51 may start a video capture task when the first event occurs in the video. If the associated event of the first event does not occur within a predetermined time period after the end of the first event, the video clipping task is ended to determine the clipped video segment. If the second event associated with the first event occurs within a predetermined time period after the end of the first event, and the event associated with the first event does not occur within a predetermined period of time after the end of the second event, the video capture task is terminated to determine the interception out video clips.
在由云端53执行本公开视频处理方案的一些实例中,云端53可以从终端设备51接收视频数据。随后,云端53可以对视频数据进行分析,在视频中出现第一事件时,启动视频截取任务。在第一事件结束后的预定时长内,确定视频是否出现第二事件。如果出现第二事件,则在第二事件结束后的预定时长内,确定视频是否出现第三事件。如果出现第三事件,则将第三事件作为第二事件,执行循环过程。如果云端53确定出未出现第二事件或第三事件,则结束视频截取任务,以确定截取出的视频片段。其中,第一事件、第二事件和第三事件中至少两个互为关联事件,更具体的,第一事件、第二事件和第三事件三者之间互为关联事件。In some instances where the video processing scheme of the present disclosure is performed by the cloud 53 , the cloud 53 may receive video data from the terminal device 51 . Subsequently, the cloud 53 may analyze the video data, and start the video capture task when the first event occurs in the video. Within a predetermined time period after the end of the first event, it is determined whether the second event occurs in the video. If the second event occurs, within a predetermined period of time after the end of the second event, it is determined whether the third event occurs in the video. If the third event occurs, the third event is regarded as the second event, and the loop process is executed. If the cloud 53 determines that the second event or the third event does not occur, the video clipping task is ended to determine the clipped video segment. Wherein, at least two of the first event, the second event and the third event are mutually associated events, and more specifically, the first event, the second event and the third event are mutually associated events.
云端53可以对截取出的视频片段进一步截取,以剔除最后预定时长的视频片段,生成目标视频片段进行存储。这个过程可以在确定出截取出的视频片段之后立即执行,也可以在接收到对应的视频获取请求后,再执行,本公开对此不做限制。The cloud 53 may further intercept the clipped video clips, so as to eliminate the video clips of the last predetermined duration, and generate target video clips for storage. This process may be performed immediately after the cut-out video segment is determined, or may be performed after a corresponding video acquisition request is received, which is not limited in the present disclosure.
在由云端53执行本公开视频处理方案的另一些实例中,云端53可以在视频中出现第一事件时,启动视频截取任务。如果第一事件结束后的预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。如果第一事件结束后的预定时长内出现与第一事件关联的第二事件,则在第二事件结束后经历预定时长后,结束视频截取任务,以确定截取出的视频片段。In other instances where the cloud 53 performs the video processing solution of the present disclosure, the cloud 53 may start a video capture task when the first event occurs in the video. If the associated event of the first event does not occur within a predetermined time period after the end of the first event, the video clipping task is ended to determine the clipped video segment. If a second event associated with the first event occurs within a predetermined time period after the end of the first event, the video clipping task is terminated after the predetermined time period elapses after the end of the second event to determine the clipped video segment.
此外,需要说明的是,一方面,本公开的视频处理方案可以应用于视频监控场景下,也就是说,视频是通过摄像头实时拍摄的视频,并进行实时分析,以截取满足用户需求的视频片段。另一方面,本公开的视频处理方案还可以用于对已有视频进行分析。In addition, it should be noted that, on the one hand, the video processing solution of the present disclosure can be applied in a video surveillance scenario, that is, the video is a video captured by a camera in real time, and real-time analysis is performed to intercept video clips that meet user needs . On the other hand, the video processing solution of the present disclosure can also be used to analyze existing videos.
图6示出了适于用来实现本公开示例性实施方式的电子设备的示意图。本公开所述终端设备可以被配置为如图6所示电子设备的形式。需要说明的是,图6示出的电子设备仅是一个示例,不应对本公开实施例的功能和使用范围带来任何限制。6 shows a schematic diagram of an electronic device suitable for use in implementing exemplary embodiments of the present disclosure. The terminal device described in the present disclosure may be configured in the form of an electronic device as shown in FIG. 6 . It should be noted that the electronic device shown in FIG. 6 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
本公开的电子设备至少包括处理器和存储器,存储器用于存储一个或多个程序,当一个或多个程序被处理器执行时,使得处理器可以实现本公开示例性实施方式的视频处理方法。The electronic device of the present disclosure includes at least a processor and a memory for storing one or more programs, which, when executed by the processor, enable the processor to implement the video processing method of the exemplary embodiment of the present disclosure.
具体的,如图6所示,电子设备600可以包括:处理器610、内部存储器621、外部存储器接口622、通用串行总线(Universal Serial Bus,USB)接口630、充电管理模块640、电源管理模块641、电池642、天线1、天线2、移动通信模块650、无线通信模块660、音频模块670、扬声器671、受话器672、麦克风673、耳机接口674、传感器模块680、显示屏690、摄像模组691、指示器692、马达693、按键694以及用户标识模块(Subscriber Identification Module,SIM)卡接口695等。其中传感器模块680可以包括深度传感器、压力传感器、陀螺仪传感器、气压传感器、磁传感器、加速度传感器、距离传感器、接近光传感器、指纹传感器、温度传感器、触摸传感器、环境光传感器及骨传导传感器等。Specifically, as shown in FIG. 6 , the electronic device 600 may include: a processor 610, an internal memory 621, an external memory interface 622, a Universal Serial Bus (USB) interface 630, a charging management module 640, and a power management module 641, battery 642, antenna 1, antenna 2, mobile communication module 650, wireless communication module 660, audio module 670, speaker 671, receiver 672, microphone 673, headphone jack 674, sensor module 680, display screen 690, camera module 691 , an indicator 692, a motor 693, a key 694, a Subscriber Identification Module (SIM) card interface 695, and the like. The sensor module 680 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.
可以理解的是,本公开实施例示意的结构并不构成对电子设备600的具体限定。在本公开另一些实施例中,电子设备600可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件、软件或软件和硬件的组合实现。It can be understood that the structures illustrated in the embodiments of the present disclosure do not constitute a specific limitation on the electronic device 600 . In other embodiments of the present disclosure, the electronic device 600 may include more or less components than shown, or some components may be combined, or some components may be separated, or different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
处理器610可以包括一个或多个处理单元,例如:处理器610可以包括应用处理器(Application Processor,AP)、调制解调处理器、图形处理器(Graphics Processing Unit,GPU)、图像信号处理器(Image Signal Processor,ISP)、控制器、视频编解码器、数字信号处理器 (Digital Signal Processor,DSP)、基带处理器和/或神经网络处理器(Neural-etwork Processing Unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。另外,处理器610中还可以设置存储器,用于存储指令和数据。The processor 610 may include one or more processing units, for example, the processor 610 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), controller, video codec, digital signal processor (Digital Signal Processor, DSP), baseband processor and/or neural network processor (Neural-network Processing Unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. In addition, a memory may also be provided in the processor 610 for storing instructions and data.
电子设备600可以通过ISP、摄像模组691、视频编解码器、GPU、显示屏690及应用处理器等实现拍摄功能。在一些实施例中,电子设备600可以包括1个或N个摄像模组691,N为大于1的正整数,若电子设备600包括N个摄像头,N个摄像头中有一个是主摄像头。The electronic device 600 can realize the shooting function through the ISP, the camera module 691, the video codec, the GPU, the display screen 690, the application processor, and the like. In some embodiments, the electronic device 600 may include 1 or N camera modules 691, where N is a positive integer greater than 1. If the electronic device 600 includes N cameras, one of the N cameras is the main camera.
内部存储器621可以用于存储计算机可执行程序代码,所述可执行程序代码包括指令。内部存储器621可以包括存储程序区和存储数据区。外部存储器接口622可以用于连接外部存储卡,例如Micro SD卡,实现扩展电子设备600的存储能力。Internal memory 621 may be used to store computer executable program code, which includes instructions. The internal memory 621 may include a storage program area and a storage data area. The external memory interface 622 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 600.
本公开还提供了一种计算机可读存储介质,该计算机可读存储介质可以是上述实施例中描述的电子设备中所包含的;也可以是单独存在,而未装配入该电子设备中。The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist alone without being assembled into the electronic device.
计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
计算机可读存储介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读存储介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:无线、电线、光缆、RF等等,或者上述的任意合适的组合。The computer-readable storage medium can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Program code embodied on a computer-readable storage medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
计算机可读存储介质承载有一个或者多个程序,当上述一个或者多个程序被一个该电子设备执行时,使得该电子设备实现如下述实施例中所述的方法。The computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to implement the methods described in the following embodiments.
附图中的流程图和框图,图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是,框图或流程图中的每个方框、以及框图或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现,也可以通过硬件的方式来实现,所描述的单元也可以设置在处理器中。其中,这些单元的名称在某种情况下并不构成对该单元本身的限定。The units involved in the embodiments of the present disclosure may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
本公开的示例性实施方式的视频处理方法可以包括步骤1至步骤4,具体的:The video processing method of the exemplary embodiment of the present disclosure may include steps 1 to 4, specifically:
在步骤1中,在视频中出现第一事件时,启动视频截取任务。In step 1, when the first event occurs in the video, a video capture task is started.
在步骤2中,在第一事件结束后的预定时长内,确定视频是否出现第二事件。如果未出现第二事件,则执行步骤3;如果出现第二事件,则执行步骤4。In step 2, within a predetermined period of time after the end of the first event, it is determined whether the second event occurs in the video. If the second event does not occur, go to step 3; if the second event occurs, go to step 4.
在步骤3中,结束视频截取任务,以确定截取出的视频片段。In step 3, the video clipping task is ended to determine the clipped video segment.
在步骤4中,在第二事件结束后的预定时长内,确定视频是否出现第三事件。如果出现第三事件,则将第三事件作为第二事件,以循环执行步骤4;如果未出现第三事件,则执行步骤3。In step 4, within a predetermined time period after the end of the second event, it is determined whether the third event occurs in the video. If the third event occurs, the third event is regarded as the second event to execute step 4 cyclically; if the third event does not occur, step 3 is executed.
其中,第一事件、第二事件和第三事件中至少两个互为关联事件。Wherein, at least two of the first event, the second event and the third event are mutually associated events.
图7示意性示出了本公开的示例性实施方式的视频处理方法的流程图。下面将以终端 设备执行图7所示步骤为例对本公开的视频处理方法的各步骤进行说明。参考图7,视频处理方法可以包括以下步骤:FIG. 7 schematically shows a flowchart of a video processing method according to an exemplary embodiment of the present disclosure. Each step of the video processing method of the present disclosure will be described below by taking the terminal device performing the steps shown in FIG. 7 as an example. Referring to Figure 7, the video processing method may include the following steps:
S70.在视频中出现第一事件时,启动视频截取任务。S70. When the first event occurs in the video, start the video capture task.
本公开方案针对的视频可以是摄像头实时拍摄的视频,本公开对视频的内容(即摄像头拍摄的对象)不做限制。其中,摄像头可以是固定式摄像头,例如停车场、制造车间的监控摄像头。另外,摄像头还可以是移动摄像头,例如手机上的摄像头,用户可以通过该摄像头进行移动拍摄,获取周围场景信息。The video targeted by the solution of the present disclosure may be a video captured by a camera in real time, and the present disclosure does not limit the content of the video (ie, the object captured by the camera). Wherein, the camera may be a fixed camera, such as a monitoring camera in a parking lot or a manufacturing workshop. In addition, the camera may also be a mobile camera, for example, a camera on a mobile phone, and a user can perform mobile shooting through the camera to obtain surrounding scene information.
本公开方案针对的视频还可以是已经拍摄完成的视频,需要对该视频进行分析时,再从存储器中获取的视频。类似地,本公开对已拍摄完成的视频的视频类型不做限制。The video targeted by the solution of the present disclosure may also be a video that has been shot, and is obtained from the memory when the video needs to be analyzed. Similarly, the present disclosure does not limit the video type of the video that has been shot.
在本公开的示例性实施方式中,第一事件可以是预设事件,而预设事件可以包括用户预设事件或系统预设事件。其中,用户预设事件可以是用户预先进行演示的事件,终端设备可以对用户演示的事件进行拍摄并保存。例如,以预设事件为出现人脸的事件为例,终端设备可以拍摄包含人脸的图像和未包含人脸的图像,然后,用户可以在一预设事件配置界面上选择包含人脸的图像作为包含预设事件的图像。另外,预设事件还可以是终端设备出厂时系统预设的事件,本公开对预设事件的类型不做限制。In an exemplary embodiment of the present disclosure, the first event may be a preset event, and the preset event may include a user preset event or a system preset event. The user preset event may be an event demonstrated by the user in advance, and the terminal device may photograph and save the event demonstrated by the user. For example, taking the preset event as an event of the appearance of a human face as an example, the terminal device can capture an image containing a human face and an image that does not contain a human face, and then the user can select an image containing a human face on a preset event configuration interface As an image containing preset events. In addition, the preset event may also be an event preset by the system when the terminal device leaves the factory, and the present disclosure does not limit the type of the preset event.
又例如,第一事件可以是用户感兴趣的事件,或者系统预先设定的预定类型的事件。例如,第一事件可以为拍摄场景中存在人脸、有对象(例如人、动物等)移动、场景中设备发出提示信号、哭泣、尖叫、摔倒等任一种或多种情况,本公开对第一事件的类型不做限制。For another example, the first event may be an event of interest to the user, or an event of a predetermined type preset by the system. For example, the first event may be any one or more of situations such as the presence of a human face in the shooting scene, the movement of an object (such as a person, an animal, etc.), the device sending a prompt signal in the scene, crying, screaming, falling, etc. The type of the first event is not limited.
根据本公开的一些实施例,首先,终端设备可以每隔预定时间间隔,从视频中提取视频帧图像。其中,预定时间间隔与场景类型相关,可以基于场景来设定,本公开对其取值不做限制。According to some embodiments of the present disclosure, first, the terminal device may extract video frame images from the video at predetermined time intervals. The predetermined time interval is related to the scene type and can be set based on the scene, and the value of the predetermined time interval is not limited in the present disclosure.
由于不是每一帧均进行处理,而是每隔预定时间间隔提取视频帧图像,由此,大大减轻了终端设备的处理压力,节省了资源。Since each frame is not processed, but video frame images are extracted at predetermined time intervals, the processing pressure of the terminal device is greatly reduced, and resources are saved.
容易理解的是,在一些图像内容变化剧烈的场景或其他需要缜密分析的场景下,可以提取视频中每一帧图像进行处理。It is easy to understand that in some scenes with drastic changes in image content or other scenes that require careful analysis, each frame of image in the video can be extracted for processing.
接下来,可以对视频帧图像进行特征提取。具体的,可以采用基于深度学习的机器学习模型对视频帧图像进行处理,以提取出视频帧图像的特征。其中,本公开对机器学习模型的结构和训练过程不做限制。另外,还可以采用例如直方图的方法提取出视频帧图像的特征,本公开对此也不做限制。Next, feature extraction can be performed on the video frame images. Specifically, a machine learning model based on deep learning can be used to process the video frame images to extract features of the video frame images. Wherein, the present disclosure does not limit the structure and training process of the machine learning model. In addition, a method such as a histogram can also be used to extract the features of the video frame images, which is not limited in the present disclosure.
然后,可以根据提取到的特征,确定出视频帧图像是否出现上述预设事件。可以理解的是,机器学习模型的输出可以是是否出现预设事件的结果。另外,还可以根据机器学习模型提取到的特征,再进行进一步分析,得到是否出现预设事件的结果。Then, according to the extracted features, it can be determined whether the above-mentioned preset event occurs in the video frame image. Understandably, the output of a machine learning model can be the result of whether a preset event occurs. In addition, further analysis can be performed according to the features extracted by the machine learning model to obtain a result of whether a preset event occurs.
例如,在第一事件为判断场景中存在猫的情况下,可以将视频帧图像输入训练后的卷积神经网络,由该卷积神经网络进行特征提取,以分类出视频帧图像中是否存在猫。For example, in the case where the first event is to judge that there is a cat in the scene, the video frame image can be input into the trained convolutional neural network, and the convolutional neural network can perform feature extraction to classify whether there is a cat in the video frame image. .
此外,鉴于单帧的判断可能出现错误,根据本公开的另一些实施例,还提供了一种基于多帧判断是否出现第一事件(或称为预设事件)的方案。In addition, since the judgment of a single frame may be wrong, according to other embodiments of the present disclosure, a solution for judging whether a first event (or referred to as a preset event) occurs based on multiple frames is also provided.
具体的,首先,可以根据提取到的特征,从视频中确定首次出现预设对象的目标视频帧图像。其中,预设对象是确定一个事件是预设事件的对象,可以理解的是,预设对象可以作为预设事件的标识。接下来,如果目标视频帧图像之后的一帧或多帧视频帧图像中均存在预设对象,则确定视频中出现预设事件,并将目标视频帧图像作为预设事件开始的起点。Specifically, first, the target video frame image in which the preset object appears for the first time can be determined from the video according to the extracted features. The preset object is an object that determines that an event is a preset event. It can be understood that the preset object can be used as an identifier of the preset event. Next, if there are preset objects in one or more frames of video frame images after the target video frame image, it is determined that a preset event occurs in the video, and the target video frame image is used as the starting point of the preset event.
例如,在连续的100帧图像中,如果第5帧图像出现人脸,则判断第6帧是否也出现人脸,或者判断之后的预定数量视频帧图像是否出现人脸(如第6帧至第10帧)。如果判断出这些帧均存在人脸,则可以确定视频中出现人脸,并将第5帧作为人脸出现的起点。For example, in 100 consecutive frames of images, if a face appears in the 5th frame, it is judged whether a face also appears in the 6th frame, or whether a face appears in a predetermined number of subsequent video frames (such as the 6th to the 1st frame). 10 frames). If it is determined that there is a human face in these frames, it can be determined that a human face appears in the video, and the fifth frame is taken as the starting point of the human face appearing.
在视频中出现第一事件时,终端设备可以启动视频截取任务。When the first event occurs in the video, the terminal device may start a video capture task.
如上所述,在这种情况下,可以自目标视频帧图像起,启动视频截取任务。仍以上例进行说明,可以自第5帧图像起,启动视频截取任务。As described above, in this case, the video clipping task can be started from the target video frame image. The above example is still described, and the video capture task can be started from the fifth frame image.
根据本公开的一些实施例,启动视频截取任务的操作包括开始对视频进行截取操作。具体的,在视频为摄像头实时拍摄的视频的情况下,启动视频截取任务的操作包括开始对视频进行录制。According to some embodiments of the present disclosure, the operation of initiating a video capture task includes starting a video capture operation. Specifically, in the case that the video is a video captured by a camera in real time, the operation of initiating the video capture task includes starting to record the video.
根据本公开另一些实施例,启动视频截取任务的操作包括记录视频中开始出现第一事件的时间,作为视频截取开始时间。可以理解的是,开始出现第一事件的时间是视频中第一事件从无到有的时间点,即,由未出现第一事件到出现第一事件的瞬时时间点。另外,视频截取开始时间可以是视频中的时间,也就是说,其表示的是相对时间。然而,视频截取开始时间也可以表示现实中的绝对时间,本公开对此不做限制。According to other embodiments of the present disclosure, the operation of initiating the video capture task includes recording the time when the first event begins to appear in the video as the video capture start time. It can be understood that the time when the first event begins to appear is the time point in the video when the first event starts from nothing, that is, the instantaneous time point from the absence of the first event to the appearance of the first event. In addition, the video clipping start time may be a time in the video, that is, it represents a relative time. However, the video clipping start time may also represent an absolute time in reality, which is not limited in the present disclosure.
S72.在第一事件结束后的预定时长内,确定视频是否出现第二事件。S72. Within a predetermined period of time after the end of the first event, determine whether the second event occurs in the video.
下面以第二事件与第一事件相关联为例进行说明。The following description will be given by taking as an example that the second event is associated with the first event.
在本公开一些实施例中,第二事件与第一事件相关联指的是:第二事件与第一事件的事件类型相同。例如,均出现人脸、均存在用户移动、均存在其他指定对象(如,猫、指定设备等)等。In some embodiments of the present disclosure, the association of the second event with the first event means that the second event and the first event are of the same event type. For example, all faces appear, all users move, all other designated objects (eg, cats, designated devices, etc.) exist.
在本公开另一些实施例中,第二事件与第一事件相关联还可以指:第二事件与第一事件相同。例如,第二事件与第一事件均为出现用户A的人脸。另外,可以理解的是,这里的相同指的是事件对应的图像相同,不一定图像出现的位置与尺寸完全相同。In other embodiments of the present disclosure, the association between the second event and the first event may also refer to: the second event is the same as the first event. For example, both the second event and the first event appear on the face of user A. In addition, it can be understood that the same here means that the images corresponding to the events are the same, and it is not necessary that the positions and sizes of the images appear exactly the same.
在本公开又一些实施例中,第二事件与第一事件相关联指的是:第二事件可以是第一事件的后续事件。例如,组装一物品包括过程a、过程b两个步骤,需要先执行过程a再执行过程b,在这种情况下,过程a对应的事件为第一事件,过程b对应的事件为第二事件。In still other embodiments of the present disclosure, the association of the second event with the first event means that the second event may be a subsequent event of the first event. For example, assembling an item includes two steps: process a and process b. It is necessary to execute process a and then process b. In this case, the event corresponding to process a is the first event, and the event corresponding to process b is the second event. .
在由步骤S70检测出的第一事件结束时,可以开始计时,在预定时长内确定视频是否出现第二事件。其中,预定时长与本公开方案的应用场景相关,例如可以是10秒、30秒等,本公开对此不做限制。When the first event detected by step S70 ends, a timer may be started to determine whether the second event occurs in the video within a predetermined period of time. The predetermined duration is related to the application scenario of the solution of the present disclosure, and may be, for example, 10 seconds, 30 seconds, etc., which is not limited in the present disclosure.
也就是说,在由步骤S70检测出的事件结束后,在预定时长内判断有没有与之对应的下一事件发生。例如,在检测出人脸的情况下,当人脸从视频中消失时,开始计时,在预定时长内,检测是否又有人脸出现。That is, after the event detected in step S70 ends, it is determined whether the next event corresponding to the event occurs within a predetermined period of time. For example, in the case of detecting a human face, when the human face disappears from the video, the timer starts, and within a predetermined period of time, it is detected whether there is another human face.
另外,确定是否存在第二事件的方式可以与步骤S70中确定第二事件的方式相同,即均可以通过对视频帧图像的分析而确定是否出现事件。In addition, the manner of determining whether the second event exists may be the same as the manner of determining the second event in step S70, that is, whether the event occurs may be determined by analyzing the video frame images.
在确定出视频中出现第二事件的情况下,终端设备执行步骤S74;在确定出视频中未出现第二事件的情况下,终端设备执行步骤S78。If it is determined that the second event occurs in the video, the terminal device executes step S74; if it is determined that the second event does not appear in the video, the terminal device executes step S78.
此外,针对检测第一事件结束的过程,类似于上述检测第一事件出现的情况,可以再结合一帧或多帧来检测第一事件是否结束。In addition, for the process of detecting the end of the first event, similar to the above-mentioned case of detecting the occurrence of the first event, one or more frames may be combined to detect whether the first event ends.
例如,在第20帧图像中发现第一事件结束,在这种情况下,可以在进行之后一帧或多帧的判断过程,如果均未出现第一事件,则将第20帧作为第一事件结束的图像。For example, it is found in the 20th frame image that the first event ends. In this case, the judgment process of one or more frames can be carried out. If the first event does not appear, the 20th frame is regarded as the first event. end image.
另外,仍可以理解的,在使用多帧图像确定是否出现第二事件时,可以设置成当具有第二事件对应的对象的多帧图像(例如,3帧图像、5帧图像等)均出现在预定时长内,则可以确定出视频出现第二事件,也可以设置成只要具有第二事件对应的对象的图像出现在预定时长内,即可确定出视频出现第二事件。In addition, it can still be understood that when using multiple frames of images to determine whether the second event occurs, it can be set so that when multiple frames of images (for example, 3-frame images, 5-frame images, etc.) with objects corresponding to the second event all appear in the Within the predetermined time period, it can be determined that the second event occurs in the video, or it can be determined that the second event occurs in the video as long as the image with the object corresponding to the second event appears within the predetermined time period.
S74.在第二事件结束后的预定时长内,确定视频是否出现第三事件。S74. Within a predetermined period of time after the end of the second event, determine whether a third event occurs in the video.
在本公开的一些实施例中,第三事件可以与第一事件或第二事件相关联,此处所说的相关联的含义与步骤S72中描述的相关联相同,不再赘述。应当注意的是,第一事件、第二事件和第三事件中至少两个互为关联事件,更具体的,第一事件、第二事件和第三事件 互为关联事件。In some embodiments of the present disclosure, the third event may be associated with the first event or the second event, and the meaning of association mentioned here is the same as the association described in step S72, and details are not repeated here. It should be noted that at least two of the first event, the second event and the third event are mutually related events, and more specifically, the first event, the second event and the third event are mutually related events.
如果在步骤S72中确定出视频中出现第二事件,则终端设备可以在第二事件结束后的预定时长内,确定该视频是否出现第三事件。If it is determined in step S72 that the second event occurs in the video, the terminal device may determine whether the third event occurs in the video within a predetermined period of time after the end of the second event.
具体的,也可以通过提取视频帧图像的特征并进行分析的方式,确定视频中是否出现第三事件的方式。Specifically, it is also possible to determine whether the third event occurs in the video by extracting and analyzing the features of the video frame images.
在确定出视频中出现第三事件的情况下,终端设备执行步骤S76;在确定出视频中未出现第三事件的情况下,终端设备执行步骤S78。If it is determined that the third event occurs in the video, the terminal device executes step S76; if it is determined that the third event does not appear in the video, the terminal device executes step S78.
另外,仍可以理解的,在使用多帧图像确定是否出现第三事件时,可以设置成当具有第三事件对应的对象的多帧图像(例如,3帧图像、5帧图像等)均出现在预定时长内,则可以确定出视频出现第三事件,也可以设置成只要具有第三事件对应的对象的图像出现在预定时长内,即可确定出视频出现第三事件。In addition, it can still be understood that when using multiple frames of images to determine whether the third event occurs, it can be set so that when multiple frames of images (for example, 3-frame images, 5-frame images, etc.) with objects corresponding to the third event all appear in the Within the predetermined time period, it can be determined that the third event occurs in the video, or it can be determined that the third event occurs in the video as long as the image with the object corresponding to the third event appears within the predetermined time period.
S76.将第三事件作为第二事件。S76. Use the third event as the second event.
如果步骤S74确定出视频出现第三事件,则将第三事件作为第二事件,并返回步骤S74,执行在第二事件结束后的预定时长内确定视频是否出现第三事件的操作。由此,如图7所示,形成了步骤S74与步骤S76的循环过程。If it is determined in step S74 that the third event occurs in the video, the third event is regarded as the second event, and the process returns to step S74 to perform the operation of determining whether the third event occurs in the video within a predetermined time period after the end of the second event. Thus, as shown in FIG. 7 , a loop process of steps S74 and S76 is formed.
可以看出,只要在一事件结束后预定时长内出现了关联的另一事件,则循环过程一直执行,直至事件结束后预定时长内不出现关联的事件为止,过程再从步骤S74跳转至步骤S78。It can be seen that as long as another associated event occurs within a predetermined period of time after the end of an event, the loop process will be executed until no associated event occurs within a predetermined period of time after the end of the event, and the process will jump from step S74 to step S74. S78.
例如,预定时长为10秒。如果事件a结束后,10秒内出现了与事件a关联的事件b,则继续判断事件b结束后10秒内是否出现与事件a(或事件b)关联的事件,如果出现关联的事件c,则继续判断事件c结束后10秒内是否出现与前面事件关联的事件,……等等。For example, the predetermined duration is 10 seconds. If event b associated with event a occurs within 10 seconds after event a ends, continue to judge whether an event associated with event a (or event b) occurs within 10 seconds after event b ends, and if associated event c occurs, Then continue to judge whether an event associated with the previous event occurs within 10 seconds after the end of event c, and so on.
S78.如果未出现第二事件或第三事件,则结束视频截取任务,以确定截取出的视频片段。S78. If the second event or the third event does not occur, end the video clipping task to determine the clipped video segment.
在启动视频截取任务的操作包括开始对视频进行截取操作的实施例中,终端设备结束视频截取任务的操作包括:结束对视频的截取操作。具体的,在视频为摄像头实时拍摄的视频的情况下,结束视频截取任务包括停止对视频进行录制。In the embodiment where the operation of initiating the video capture task includes starting the video capture operation, the operation of the terminal device to end the video capture task includes: ending the video capture operation. Specifically, when the video is a video captured by a camera in real time, ending the video capture task includes stopping the video recording.
在启动视频截取任务的操作包括记录视频截取开始时间的实施例中,在步骤S72确定出未出现第二事件的情况下,终端设备结束视频截取任务的操作包括:记录确定出第一事件结束后经历预定时长的时间,作为视频截取结束时间。在这种情况下,可以基于视频截取开始时间与视频截取结束时间,确定视频截取的时间段,并针对该时间段进行截取操作,以确定截取出的视频片段。In the embodiment where the operation of starting the video interception task includes recording the start time of video interception, in the case that it is determined in step S72 that the second event does not occur, the operation of the terminal device to end the video interception task includes: recording after determining that the first event ends After a predetermined period of time, it is used as the end time of video interception. In this case, a video clipping time period may be determined based on the video clipping start time and the video clipping end time, and a clipping operation is performed for the clipped video segment to determine the clipped video segment.
在步骤S74确定出未出现第三事件的情况下,终端设备结束视频截取任务的操作包括:记录确定出第二事件结束后经历预定时长的时间,作为视频截取结束时间。在这种情况下,可以基于视频截取开始时间与视频截取结束时间,确定视频截取的时间段,并针对该时间段进行截取操作,以确定截取出的视频片段。In the case where it is determined in step S74 that the third event does not occur, the operation of the terminal device to end the video capture task includes: recording a predetermined time elapsed after the end of the second event is determined as the video capture end time. In this case, a video clipping time period may be determined based on the video clipping start time and the video clipping end time, and a clipping operation is performed for the clipped video segment to determine the clipped video segment.
例如,视频中,视频截取开始时间为01:30,视频截取结束时间为03:00,在这种情况下,终端设备可以从视频中截取01:30到03:00对应的视频片段,即确定出截取出的视频片段。For example, in the video, the video clipping start time is 01:30, and the video clipping end time is 03:00. In this case, the terminal device can clip the video clip corresponding to 01:30 to 03:00 from the video, that is, determine Take out the clipped video clip.
在确定出截取的视频片段后,鉴于视频片段最后预定时长内不存在对应的事件,在这种情况下,终端设备可以从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。另外,终端设备可以将目标视频片段上传至云端进行保存。After determining the clipped video clip, since there is no corresponding event within the last predetermined duration of the video clip, in this case, the terminal device can remove the clipped video clip with the last predetermined duration, and generate the target video clip . In addition, the terminal device can upload the target video clip to the cloud for storage.
由此,云端可以响应终端设备或其他设备发送的与该目标视频片段对应的视频获取请求,将目标视频片段发送给发送请求的设备。Thus, the cloud can respond to a video acquisition request corresponding to the target video clip sent by the terminal device or other device, and send the target video clip to the device that sends the request.
另外,考虑到终端设备的处理资源有限,终端设备可以将截取出的视频片段直接上传至云端。In addition, considering the limited processing resources of the terminal device, the terminal device can directly upload the clipped video clips to the cloud.
在这种情况下,在一些实施例中,云端可以响应于截取出的视频片段对应的视频获取请求,从截取出的视频片段中剔除最后预定时长的视频片段,以生成目标视频片段,并将目标视频片段发送给发起该视频获取请求的请求端,以便用户观看。In this case, in some embodiments, the cloud may, in response to the video acquisition request corresponding to the clipped video clip, remove the video clip of the last predetermined duration from the clipped video clip to generate the target video clip, and use the clipped video clip to generate the target video clip. The target video clip is sent to the requester that initiates the video acquisition request, so that the user can watch it.
在另一些实施例中,云端可以从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段并存储,以便云端响应于截取出的视频片段对应的视频获取请求,将目标视频片段发送给发起视频获取请求的请求端,以便用户观看。可以理解的,本申请的一些实施例,能够将间隔预设时间内的至少两个连续关联事件(包括相同事件)从视频中截取,并且各个关联事件之间不中断(上述任意两个连续的关联事件之间的视频也被截取出),从而提升用户观看效果,由于设置了预设时间,因此可以避免间隔过长的两个连续关联事件被纳入同一个截取的视频片段中,一定程度上降低了存储量,从而可以方便在存储量和观看效果上寻求平衡。In other embodiments, the cloud can remove the video clips of the last predetermined duration from the clipped video clips, generate and store the target video clips, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips and store the target video clips. It is sent to the requester that initiates the video acquisition request so that users can watch it. It can be understood that in some embodiments of the present application, at least two consecutive associated events (including the same event) within a preset time interval can be intercepted from the video, and each associated event is not interrupted (any two consecutive events described above). The video between the associated events is also captured), thereby improving the user’s viewing effect. Since the preset time is set, it can avoid that two consecutive associated events with a long interval are included in the same captured video clip, to a certain extent The storage capacity is reduced, so that it is convenient to seek a balance between storage capacity and viewing effect.
下面将参考图8,以出现相同的预设事件为例,对本公开实施例的视频处理方案的整个过程进行说明。Referring to FIG. 8 , the entire process of the video processing solution according to the embodiment of the present disclosure will be described below by taking the occurrence of the same preset event as an example.
在步骤S802中,终端设备对摄像头拍摄的视频进行实时监测。其中,摄像头可以集成在终端设备上,另外,摄像头还可以通过有线或无线的方式与终端设备建立连接,以便终端设备可以获取到视频。In step S802, the terminal device monitors the video captured by the camera in real time. The camera can be integrated on the terminal device, and in addition, the camera can also establish a connection with the terminal device in a wired or wireless manner, so that the terminal device can obtain the video.
在步骤S804中,终端设备判断视频中是否出现预设事件。如果出现,则执行步骤S806;如果未出现,则返回步骤S802。In step S804, the terminal device determines whether a preset event occurs in the video. If it appears, go to step S806; if not, go back to step S802.
在步骤S806中,预设事件结束后,延长录制N秒视频,其中,N秒即对应上述预定时长,例如,10秒、30秒等。In step S806, after the preset event ends, the video recording is extended for N seconds, where N seconds corresponds to the above-mentioned predetermined duration, for example, 10 seconds, 30 seconds, and the like.
在步骤S808中,终端设备判断N秒内是否又出现预设事件。如果出现,则返回步骤S806;如果未出现,则执行步骤S810。In step S808, the terminal device determines whether a preset event occurs again within N seconds. If so, go back to step S806; if not, go to step S810.
在步骤S810中,终端设备确定截取出的视频片段,该截取出的视频片段包含最后一个预设事件结束后的N秒视频片段。In step S810, the terminal device determines a clipped video clip, where the clipped video clip includes a video clip of N seconds after the end of the last preset event.
在步骤S812中,终端设备从步骤S810确定的视频片段中截断最后N秒的视频片段,并上传云端进行保存。In step S812, the terminal device truncates the video clips of the last N seconds from the video clips determined in step S810, and uploads them to the cloud for saving.
图9示意性示出了根据本公开另一实施例的由云端参与视频截取的方案的流程图。FIG. 9 schematically shows a flowchart of a solution for participating in video capture by the cloud according to another embodiment of the present disclosure.
在步骤S902中,云端获取由终端设备截取出的视频片段并存储。终端设备确定截取出的视频片段的过程可以如上述步骤S802至步骤S810所示。In step S902, the cloud acquires and stores the video clips cut out by the terminal device. The process of determining the clipped video segment by the terminal device may be as shown in the above steps S802 to S810.
在步骤S904中,云端接收与该视频片段对应的视频获取请求。In step S904, the cloud receives a video acquisition request corresponding to the video clip.
在步骤S906中,云端可以截断视频片段的最后N秒,并发送给视频获取请求的请求端。In step S906, the cloud may truncate the last N seconds of the video clip and send it to the requester of the video acquisition request.
此外,针对仅需要输出包含两个关联事件的视频的场景,本公开还提供了另一种视频处理方法。参考图10,该视频处理方法可以包括以下步骤:In addition, the present disclosure also provides another video processing method for a scene that only needs to output a video containing two related events. Referring to Figure 10, the video processing method may include the following steps:
S102.在视频中出现第一事件时,启动视频截取任务。S102. When the first event occurs in the video, start the video capture task.
步骤S102与上述步骤S70相同,不再赘述。Step S102 is the same as the above-mentioned step S70, and is not repeated here.
S104.如果第一事件结束后的预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。S104. If the associated event of the first event does not occur within a predetermined time period after the end of the first event, end the video clipping task to determine the clipped video segment.
关于确定两个事件是否关联,与步骤S72中第一事件与第二事件相关联的情况类似。就出现第一事件后,终端设备可以确定第一事件结束后预定时长内是否出现第一事件的关联事件,如果出现,则结束视频截取任务,以确定截取出的视频片段。Regarding the determination of whether the two events are related, it is similar to the case where the first event and the second event are related in step S72. After the first event occurs, the terminal device can determine whether an event associated with the first event occurs within a predetermined time period after the first event ends, and if so, ends the video capture task to determine the video clip to be captured.
其中,结束视频截取任务以确定截取出的视频片段过程与步骤S78的过程类似,不再赘述。The process of ending the video clipping task to determine the clipped video segment is similar to the process of step S78, and will not be repeated.
S106.如果第一事件结束后的预定时长内出现与第一事件关联的第二事件,且在第二事件结束后经历预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截 取出的视频片段。S106. If the second event associated with the first event occurs within a predetermined duration after the first event ends, and the associated event of the first event does not occur within the predetermined duration after the second event ends, then end the video interception task, with Determine the clipped video clip.
如果第一事件结束后的预定时长内出现与第一事件关联的事件,记为第二事件,则在第二事件结束后经历预定时长内未出现第一事件(或第二事件)的关联事件,终端设备可以结束视频截取任务,以确定截取出的视频片段。If an event associated with the first event occurs within a predetermined period of time after the first event ends, it will be recorded as a second event, and no event associated with the first event (or the second event) will appear within a predetermined period of time after the second event ends. , the terminal device can end the video clipping task to determine the clipped video segment.
在本示例性方案中,考虑到一些场景中事件往往具有较强的连续性,第二事件结束后,再经历预定时长后结束视频截取任务,避免了预定时长内视频可能存在的一些相关信息被遗漏或丢弃的问题。In this exemplary solution, considering that events in some scenarios often have strong continuity, after the second event ends, the video capture task is terminated after a predetermined period of time, so as to avoid some relevant information that may exist in the video within the predetermined period of time being captured. Missing or discarding issues.
另外,针对另一些场景,本公开方案还可以剔除最后预定时长的视频,并生成目标视频片段进行存储。In addition, for other scenarios, the solution of the present disclosure can also eliminate the video of the last predetermined duration, and generate a target video segment for storage.
在本公开的一些实施例中,终端设备可以从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。另外,终端设备可以将目标视频片段上传至云端进行保存。In some embodiments of the present disclosure, the terminal device may remove the video clip of the last predetermined duration from the clipped video clip to generate the target video clip. In addition, the terminal device can upload the target video clip to the cloud for storage.
由此,云端可以响应终端设备或其他设备发送的与该目标视频片段对应的视频获取请求,将目标视频片段发送给发送请求的设备。Thus, the cloud can respond to a video acquisition request corresponding to the target video clip sent by the terminal device or other device, and send the target video clip to the device that sends the request.
另外,考虑到终端设备的处理资源有限,终端设备可以将截取出的视频片段直接上传至云端。In addition, considering the limited processing resources of the terminal device, the terminal device can directly upload the clipped video clips to the cloud.
在这种情况下,在一些实施例中,云端可以响应于截取出的视频片段对应的视频获取请求,从截取出的视频片段中剔除最后预定时长的视频片段,以生成目标视频片段,并将目标视频片段发送给发起该视频获取请求的请求端,以便用户观看。In this case, in some embodiments, the cloud may, in response to a video acquisition request corresponding to the clipped video clip, remove the video clip of the last predetermined duration from the clipped video clip to generate the target video clip, and use the clipped video clip to generate the target video clip. The target video clip is sent to the requester that initiates the video acquisition request, so that the user can watch it.
在另一些实施例中,云端可以从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段并存储,以便云端响应于截取出的视频片段对应的视频获取请求,将目标视频片段发送给发起视频获取请求的请求端,以便用户观看。In other embodiments, the cloud can remove the video clips of the last predetermined duration from the clipped video clips, generate and store the target video clips, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips. It is sent to the requester that initiates the video acquisition request so that users can watch it.
基于本公开上述的视频处理方法,一方面,本公开方案可以从视频中截取多个关联事件的视频片段;另一方面,截取出的视频片段为连续的视频片段,确保用户观看到的视频片段连续且事件完整;再一方面,基于截取出的视频片段进行存储,可以大大节约存储空间。Based on the above-mentioned video processing method of the present disclosure, on the one hand, the solution of the present disclosure can intercept multiple video clips associated with events from the video; on the other hand, the clipped video clips are continuous video clips, ensuring that the video clips viewed by the user Continuous and complete events; on the other hand, storage based on the clipped video clips can greatly save storage space.
应当注意,尽管在附图中以特定顺序描述了本公开中方法的各个步骤,但是,这并非要求或者暗示必须按照该特定顺序来执行这些步骤,或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的,可以省略某些步骤,将多个步骤合并为一个步骤执行,以及/或者将一个步骤分解为多个步骤执行等。It should be noted that although the various steps of the methods of the present disclosure are depicted in the figures in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps must be performed to achieve the desired the result of. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, and the like.
进一步的,本示例实施方式中还提供了一种视频处理装置。Further, this exemplary embodiment also provides a video processing apparatus.
图11示意性示出了本公开的示例性实施方式的视频处理装置的方框图。参考图11,根据本公开的示例性实施方式的视频处理装置11可以包括任务启动模块111、事件检测模块113和第一视频截取模块115。FIG. 11 schematically shows a block diagram of a video processing apparatus of an exemplary embodiment of the present disclosure. Referring to FIG. 11 , the video processing apparatus 11 according to an exemplary embodiment of the present disclosure may include a task initiation module 111 , an event detection module 113 and a first video capture module 115 .
具体的,任务启动模块111可以用于在视频中出现第一事件时,启动视频截取任务;事件检测模块113可以用于在第一事件结束后的预定时长内,确定视频是否出现第二事件;如果出现第二事件,则在第二事件结束后的预定时长内,确定视频是否出现第三事件;如果出现第三事件,则将第三事件作为第二事件;第一视频截取模块115可以用于如果未出现第二事件或第三事件,则结束视频截取任务,以确定截取出的视频片段;其中,第一事件、第二事件和第三事件中至少两个互为关联事件。Specifically, the task initiation module 111 can be used to start the video capture task when the first event occurs in the video; the event detection module 113 can be used to determine whether the second event occurs in the video within a predetermined time period after the first event ends; If the second event occurs, determine whether the third event occurs in the video within a predetermined period of time after the second event ends; if the third event occurs, the third event is taken as the second event; the first video interception module 115 can use If the second event or the third event does not occur, end the video clipping task to determine the clipped video segment; wherein, at least two of the first event, the second event and the third event are correlated events with each other.
根据本公开的示例性实施例,第一视频截取模块115还可以被配置为执行:从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。According to an exemplary embodiment of the present disclosure, the first video clipping module 115 may be further configured to perform: excluding a video clip of the last predetermined duration from the clipped video clips to generate a target video clip.
根据本公开的示例性实施例,第一视频截取模块115还可以被配置为执行:将截取出的视频片段传输至指定设备,以供指定设备从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。According to an exemplary embodiment of the present disclosure, the first video clipping module 115 may be further configured to perform: transmitting the clipped video segment to a designated device, so that the designated device can remove the video of the last predetermined duration from the clipped video clip segment to generate the target video segment.
根据本公开的示例性实施例,参考图12,相比于视频处理装置11,视频处理装置12 还可以包括视频片段上传模块121。According to an exemplary embodiment of the present disclosure, referring to FIG. 12 , compared to the video processing apparatus 11 , the video processing apparatus 12 may further include a video segment uploading module 121 .
具体的,视频片段上传模块121可以被配置为执行:将截取出的视频片段上传至云端。在这种情况下,云端响应于截取出的视频片段对应的视频获取请求,从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段,并将目标视频片段发送给发起视频获取请求的请求端;或者,云端从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段并存储,以便云端响应于截取出的视频片段对应的视频获取请求,将目标视频片段发送给发起视频获取请求的请求端。Specifically, the video clip uploading module 121 may be configured to perform: uploading the clipped video clips to the cloud. In this case, in response to the video acquisition request corresponding to the clipped video clip, the cloud removes the video clip of the last predetermined duration from the clipped video clip, generates the target video clip, and sends the target video clip to the initiating video clip. The requesting end of the request; or, the cloud removes the video clips of the last predetermined duration from the clipped video clips, generates the target video clips and stores them, so that the cloud can respond to the video acquisition request corresponding to the clipped video clips and send the target video clips. To the requester that initiates the video acquisition request.
根据本公开的示例性实施例,任务启动模块111启动视频截取任务的过程可以被配置为执行:在视频中出现第一事件时,开始对视频进行截取操作。在这种情况下,第一视频截取模块115结束视频截取任务的过程可以被配置为执行:结束对视频的截取操作。According to an exemplary embodiment of the present disclosure, the process in which the task initiating module 111 initiates a video capture task may be configured to perform: when a first event occurs in the video, start a video capture operation. In this case, the process of the first video capture module 115 ending the video capture task may be configured to perform: end the video capture operation.
根据本公开的示例性实施例,任务启动模块111启动视频截取任务的过程可以被配置为执行:记录视频中开始出现第一事件的时间,作为视频截取开始时间。在这种情况下,第一视频截取模块115结束视频截取任务以确定截取出的视频片段的过程可以被配置为执行:在未出现第二事件的情况下,记录确定出第一事件结束后经历预定时长的时间,作为视频截取结束时间,基于视频截取开始时间与视频截取结束时间,对视频进行截取操作,以确定截取出的视频片段;在未出现第三事件的情况下,记录确定出第二事件结束后经历预定时长的时间,作为视频截取结束时间,基于视频截取开始时间与视频截取结束时间,对视频进行截取操作,以确定截取出的视频片段。According to an exemplary embodiment of the present disclosure, the process of initiating the video capture task by the task initiation module 111 may be configured to perform: recording the time when the first event starts to appear in the video as the video capture start time. In this case, the process of the first video clipping module 115 ending the video clipping task to determine the clipped video segment may be configured to perform: in the case that the second event does not occur, record the process after it is determined that the first event ends. The predetermined length of time is used as the end time of video interception. Based on the start time of video interception and the end time of video interception, the video is intercepted to determine the video clip to be intercepted; if the third event does not occur, record and determine the first video clip. After the second event ends, a predetermined period of time is used as the video clipping end time. Based on the video clipping start time and the video clipping end time, the video clipping operation is performed to determine the clipped video segment.
根据本公开的示例性实施例,第一事件为预设事件,预设事件包括用户预设事件或系统预设事件。在这种情况下,参考图13,相比于视频处理装置11,视频处理装置13还可以包括图像分析模块131。According to an exemplary embodiment of the present disclosure, the first event is a preset event, and the preset event includes a user preset event or a system preset event. In this case, referring to FIG. 13 , compared to the video processing apparatus 11 , the video processing apparatus 13 may further include an image analysis module 131 .
具体的,图像分析模块131可以被配置为执行:对视频中的视频帧图像进行特征提取;根据提取到的特征,确定视频中是否出现预设事件。Specifically, the image analysis module 131 may be configured to perform: extracting features from video frame images in the video; and determining whether a preset event occurs in the video according to the extracted features.
根据本公开的示例性实施例,图像分析模块131根据提取到的特征确定视频中是否出现预设事件的过程可以被配置为执行:根据提取到的特征,从视频中确定首次出现预设对象的目标视频帧图像,预设对象是确定一事件为预设事件的对象;如果目标视频帧图像之后的一帧或多帧视频帧图像中均存在预设对象,则确定视频中出现预设事件;其中,自目标视频帧图像起,启动视频截取任务。According to an exemplary embodiment of the present disclosure, the process in which the image analysis module 131 determines whether a preset event occurs in the video according to the extracted features may be configured to perform: according to the extracted features, determine from the video the first occurrence of the preset object in the video The target video frame image, the preset object is an object that determines an event as a preset event; if there are preset objects in one or more frames of video frame images after the target video frame image, it is determined that a preset event occurs in the video; Wherein, starting from the target video frame image, the video interception task is started.
根据本公开的示例性实施例,上述视频为摄像头实时拍摄的视频。According to an exemplary embodiment of the present disclosure, the above video is a video captured by a camera in real time.
进一步的,本示例实施方式中还提供了另一种视频处理装置。Further, another video processing apparatus is also provided in this exemplary embodiment.
图14示意性示出了本公开另一示例性实施方式的视频处理装置的方框图。参考图14,根据本公开的示例性实施方式的视频处理装置14可以包括任务启动模块111、第二视频截取模块141和第三视频截取模块143。FIG. 14 schematically shows a block diagram of a video processing apparatus according to another exemplary embodiment of the present disclosure. Referring to FIG. 14 , the video processing apparatus 14 according to an exemplary embodiment of the present disclosure may include a task initiation module 111 , a second video capture module 141 and a third video capture module 143 .
具体的,任务启动模块111可以用于在视频中出现第一事件时,启动视频截取任务;第二视频截取模块141可以用于如果第一事件结束后的预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段;第三视频截取模块143可以用于如果第一事件结束后的预定时长内出现与第一事件关联的第二事件,且在第二事件结束后经历预定时长内未出现第一事件的关联事件,则结束视频截取任务,以确定截取出的视频片段。Specifically, the task initiation module 111 can be used to start the video capture task when the first event occurs in the video; the second video capture module 141 can be used to associate the first event if the first event does not occur within a predetermined period of time after the end of the first event event, then end the video clipping task to determine the clipped video segment; the third video clipping module 143 can be used if the second event associated with the first event occurs within a predetermined time period after the first event ends, and the second event occurs in the second event. After the event ends, if no related event of the first event occurs within a predetermined period of time, the video interception task is ended to determine the intercepted video segment.
根据本公开的示例性实施例,第三视频截取模块143还可以被配置为执行:从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。According to an exemplary embodiment of the present disclosure, the third video clipping module 143 may be further configured to perform: excluding a video clip of the last predetermined duration from the clipped video clips to generate a target video clip.
根据本公开的示例性实施例,第三视频截取模块143还可以被配置为执行:将截取出的视频片段传输至指定设备,以供指定设备从截取出的视频片段中剔除最后预定时长的视频片段,生成目标视频片段。According to an exemplary embodiment of the present disclosure, the third video clipping module 143 may be further configured to perform: transmitting the clipped video segment to a designated device, so that the designated device can remove the video of the last predetermined duration from the clipped video clip segment to generate the target video segment.
根据本公开的示例性实施例,视频处理装置14还可以包括上述视频片段上传模块121。According to an exemplary embodiment of the present disclosure, the video processing apparatus 14 may further include the above-mentioned video clip uploading module 121 .
由于本公开实施方式的视频处理装置的各个功能模块与上述方法实施方式中相同,因此在此不再赘述。Since each functional module of the video processing apparatus in the embodiment of the present disclosure is the same as that in the above-mentioned method embodiment, it will not be repeated here.
通过以上的实施方式的描述,本领域的技术人员易于理解,这里描述的示例实施方式可以通过软件实现,也可以通过软件结合必要的硬件的方式来实现。因此,根据本公开实施方式的技术方案可以以软件产品的形式体现出来,该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM,U盘,移动硬盘等)中或网络上,包括若干指令以使得一台计算设备(可以是个人计算机、服务器、终端装置、或者网络设备等)执行根据本公开实施方式的方法。From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein may be implemented by software, or may be implemented by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to an embodiment of the present disclosure.
此外,上述附图仅是根据本公开示例性实施例的方法所包括的处理的示意性说明,而不是限制目的。易于理解,上述附图所示的处理并不表明或限制这些处理的时间顺序。另外,也易于理解,这些处理可以是例如在多个模块中同步或异步执行的。In addition, the above-mentioned figures are merely schematic illustrations of the processes included in the methods according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the chronological order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, in multiple modules.
应当注意,尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元,但是这种划分并非强制性的。实际上,根据本公开的实施方式,上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之,上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。It should be noted that although several modules or units of the apparatus for action performance are mentioned in the above detailed description, this division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into multiple modules or units to be embodied.
本领域技术人员在考虑说明书及实践这里公开的内容后,将容易想到本公开的其他实施例。本申请旨在涵盖本公开的任何变型、用途或者适应性变化,这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的,本公开的真正范围和精神由权利要求指出。Other embodiments of the present disclosure will readily suggest themselves to those skilled in the art upon consideration of the specification and practice of what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the claims.
应当理解的是,本公开并不局限于上面已经描述并在附图中示出的精确结构,并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限。It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (20)

  1. 一种视频处理方法,包括:A video processing method, comprising:
    在视频中出现第一事件时,启动视频截取任务;When the first event occurs in the video, start the video capture task;
    在所述第一事件结束后的预定时长内,确定所述视频是否出现第二事件;within a predetermined period of time after the end of the first event, determining whether a second event occurs in the video;
    如果出现所述第二事件,则在所述第二事件结束后的所述预定时长内,确定所述视频是否出现第三事件;If the second event occurs, within the predetermined time period after the second event ends, determine whether a third event occurs in the video;
    如果出现所述第三事件,则将所述第三事件作为所述第二事件;If the third event occurs, use the third event as the second event;
    如果未出现所述第二事件或所述第三事件,则结束所述视频截取任务,以确定截取出的视频片段;If the second event or the third event does not occur, end the video clipping task to determine the clipped video segment;
    其中,所述第一事件、所述第二事件和所述第三事件中至少两个互为关联事件。Wherein, at least two of the first event, the second event and the third event are mutually associated events.
  2. 根据权利要求1所述的视频处理方法,其中,所述视频处理方法还包括:The video processing method according to claim 1, wherein the video processing method further comprises:
    从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段。Eliminate the last video clip of the predetermined duration from the clipped video clips to generate a target video clip.
  3. 根据权利要求1所述的视频处理方法,其中,所述视频处理方法还包括:The video processing method according to claim 1, wherein the video processing method further comprises:
    将所述截取出的视频片段传输至指定设备,以供所述指定设备从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段。The clipped video clip is transmitted to a designated device, so that the designated device can remove the last video clip of the predetermined duration from the clipped video clip to generate a target video clip.
  4. 根据权利要求1所述的视频处理方法,其中,所述视频处理方法还包括:The video processing method according to claim 1, wherein the video processing method further comprises:
    将所述截取出的视频片段上传至云端;uploading the clipped video clips to the cloud;
    以供所述云端响应于所述截取出的视频片段对应的视频获取请求,从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段,并将所述目标视频片段发送给发起所述视频获取请求的请求端;或者,所述云端从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段并存储,以便所述云端响应于所述截取出的视频片段对应的视频获取请求,将所述目标视频片段发送给发起所述视频获取请求的请求端。In order for the cloud to respond to the video acquisition request corresponding to the clipped video clip, remove the last video clip of the predetermined duration from the clipped video clip, generate a target video clip, and use the target video clip. The clip is sent to the requesting end that initiates the video acquisition request; or, the cloud removes the last video clip of the predetermined duration from the clipped video clip, generates and stores the target video clip, so that the cloud can respond to the For the video acquisition request corresponding to the intercepted video clip, the target video clip is sent to the requester that initiates the video acquisition request.
  5. 根据权利要求1所述的视频处理方法,其中,启动视频截取任务包括:开始对所述视频进行截取操作;The video processing method according to claim 1, wherein starting a video capture task comprises: starting a capture operation on the video;
    结束所述视频截取任务包括:结束对所述视频的截取操作。Ending the video capture task includes: ending the video capture operation.
  6. 根据权利要求1所述的视频处理方法,其中,启动视频截取任务包括:记录所述视频中开始出现所述第一事件的时间,作为视频截取开始时间;The video processing method according to claim 1, wherein starting a video interception task comprises: recording the time when the first event begins to appear in the video as the video interception start time;
    结束所述视频截取任务,以确定截取出的视频片段,包括:在未出现所述第二事件的情况下,记录确定出所述第一事件结束后经历所述预定时长的时间,作为视频截取结束时间,基于所述视频截取开始时间与所述视频截取结束时间,对所述视频进行截取操作,以确定截取出的视频片段;在未出现所述第三事件的情况下,记录确定出所述第二事件结束后经历所述预定时长的时间,作为视频截取结束时间,基于所述视频截取开始时间与所述视频截取结束时间,对所述视频进行截取操作,以确定截取出的视频片段。Ending the video clipping task to determine the clipped video clips includes: in the case where the second event does not occur, recording the time that has elapsed after the first event is determined to have passed the predetermined duration, as a video clipping The end time, based on the video clipping start time and the video clipping end time, perform a clipping operation on the video to determine the clipped video segment; in the case where the third event does not occur, record and determine the After the second event ends, the predetermined duration is used as the video clipping end time. Based on the video clipping start time and the video clipping end time, the video clipping operation is performed to determine the clipped video segment. .
  7. 根据权利要求1所述的视频处理方法,其中,所述第一事件为预设事件,所述预设事件包括用户预设事件或系统预设事件;其中,所述视频处理方法还包括:The video processing method according to claim 1, wherein the first event is a preset event, and the preset event includes a user preset event or a system preset event; wherein, the video processing method further comprises:
    对所述视频中的视频帧图像进行特征提取;Feature extraction is performed on the video frame images in the video;
    根据提取到的特征,确定所述视频中是否出现所述预设事件。According to the extracted features, it is determined whether the preset event occurs in the video.
  8. 根据权利要求7所述的视频处理方法,其中,根据提取到的特征,确定所述视频中是否出现所述预设事件,包括:The video processing method according to claim 7, wherein determining whether the preset event occurs in the video according to the extracted features comprises:
    根据提取到的特征,从所述视频中确定首次出现预设对象的目标视频帧图像,所述预设对象是确定一事件为所述预设事件的对象;According to the extracted features, determine from the video a target video frame image in which a preset object appears for the first time, and the preset object is an object for which an event is determined to be the preset event;
    如果所述目标视频帧图像之后的一帧或多帧视频帧图像中均存在所述预设对象,则确定所述视频中出现所述预设事件;If the preset object exists in one or more frames of video frame images after the target video frame image, determine that the preset event occurs in the video;
    其中,自所述目标视频帧图像起,启动所述视频截取任务。Wherein, starting from the target video frame image, the video clipping task is started.
  9. 根据权利要求1至8中任一项所述的视频处理方法,其中,所述视频为摄像头实时拍摄的视频。The video processing method according to any one of claims 1 to 8, wherein the video is a video captured by a camera in real time.
  10. 一种视频处理方法,包括:A video processing method, comprising:
    在视频中出现第一事件时,启动视频截取任务;When the first event occurs in the video, start the video capture task;
    如果所述第一事件结束后的预定时长内未出现所述第一事件的关联事件,则结束所述视频截取任务,以确定截取出的视频片段;If the associated event of the first event does not occur within a predetermined time period after the end of the first event, end the video capture task to determine the video clip to be captured;
    如果所述第一事件结束后的所述预定时长内出现与所述第一事件关联的第二事件,且在所述第二事件结束后经历所述预定时长内未出现所述第一事件的关联事件,则结束所述视频截取任务,以确定截取出的视频片段。If a second event associated with the first event occurs within the predetermined time period after the first event ends, and the first event does not occur within the predetermined time period after the second event ends If the event is associated, the video clipping task is ended to determine the clipped video segment.
  11. 根据权利要求10所述的视频处理方法,其中,所述视频处理方法还包括:The video processing method according to claim 10, wherein the video processing method further comprises:
    从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段。Eliminate the last video clip of the predetermined duration from the clipped video clips to generate a target video clip.
  12. 根据权利要求10所述的视频处理方法,其中,所述视频处理方法还包括:The video processing method according to claim 10, wherein the video processing method further comprises:
    将所述截取出的视频片段传输至指定设备,以供所述指定设备从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段。The clipped video clip is transmitted to a designated device, so that the designated device can remove the last video clip of the predetermined duration from the clipped video clip to generate a target video clip.
  13. 根据权利要求10所述的视频处理方法,其中,所述视频处理方法还包括:The video processing method according to claim 10, wherein the video processing method further comprises:
    将所述截取出的视频片段上传至云端;uploading the clipped video clips to the cloud;
    以供所述云端响应于所述截取出的视频片段对应的视频获取请求,从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段,并将所述目标视频片段发送给发起所述视频获取请求的请求端;或者,所述云端从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段并存储,以便所述云端响应于所述截取出的视频片段对应的视频获取请求,将所述目标视频片段发送给发起所述视频获取请求的请求端。In order for the cloud to respond to the video acquisition request corresponding to the clipped video clip, remove the last video clip of the predetermined duration from the clipped video clip, generate a target video clip, and use the target video clip. The clip is sent to the requesting end that initiates the video acquisition request; or, the cloud removes the last video clip of the predetermined duration from the clipped video clip, generates and stores the target video clip, so that the cloud can respond to the For the video acquisition request corresponding to the intercepted video clip, the target video clip is sent to the requester that initiates the video acquisition request.
  14. 一种视频处理装置,包括:A video processing device, comprising:
    任务启动模块,被配置为在视频中出现第一事件时,启动视频截取任务;a task initiation module, configured to initiate a video capture task when the first event occurs in the video;
    事件确定模块,被配置为在所述第一事件结束后的预定时长内,确定所述视频是否出现第二事件;如果出现所述第二事件,则在所述第二事件结束后的所述预定时长内,确定所述视频是否出现第三事件;如果出现所述第三事件,则将所述第三事件作为所述第二事件;an event determining module, configured to determine whether a second event occurs in the video within a predetermined period of time after the first event ends; if the second event occurs, the second event occurs in the video after the second event ends Within a predetermined period of time, determine whether a third event occurs in the video; if the third event occurs, use the third event as the second event;
    第一视频截取模块,被配置为如果未出现所述第二事件或所述第三事件,则结束所述视频截取任务,以确定截取出的视频片段;a first video clipping module, configured to end the video clipping task if the second event or the third event does not occur, to determine the clipped video segment;
    其中,所述第一事件、所述第二事件和所述第三事件中至少两个互为关联事件。Wherein, at least two of the first event, the second event and the third event are mutually associated events.
  15. 根据权利要求14所述的视频处理装置,其中,所述第一视频截取模块还被配置为从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段。The video processing apparatus according to claim 14, wherein the first video clipping module is further configured to remove the last video clip of the predetermined duration from the clipped video clips to generate a target video clip.
  16. 根据权利要求14所述的视频处理装置,其中,所述视频处理装置还包括:The video processing apparatus according to claim 14, wherein the video processing apparatus further comprises:
    视频片段上传模块,被配置为将所述截取出的视频片段上传至云端;a video clip uploading module, configured to upload the clipped video clips to the cloud;
    以供所述云端响应于所述截取出的视频片段对应的视频获取请求,从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段,并将所述目标视频片段发送给发起所述视频获取请求的请求端;或者,所述云端从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段并存储,以便所述云端响应于所述截取出的视频片段对应的视频获取请求,将所述目标视频片段发送给发起所述视频获取请求的请求端。In order for the cloud to respond to the video acquisition request corresponding to the clipped video clip, remove the last video clip of the predetermined duration from the clipped video clip, generate a target video clip, and use the target video clip. The clip is sent to the requesting end that initiates the video acquisition request; or, the cloud removes the last video clip of the predetermined duration from the clipped video clip, generates and stores the target video clip, so that the cloud can respond to the For the video acquisition request corresponding to the intercepted video clip, the target video clip is sent to the requester that initiates the video acquisition request.
  17. 一种视频处理装置,包括:A video processing device, comprising:
    任务启动模块,被配置为在视频中出现第一事件时,启动视频截取任务;a task initiation module, configured to initiate a video capture task when the first event occurs in the video;
    第二视频截取模块,被配置为如果所述第一事件结束后的预定时长内未出现所述第一 事件的关联事件,则结束所述视频截取任务,以确定截取出的视频片段;The second video clipping module is configured to end the video clipping task if the associated event of the first event does not occur within a predetermined duration after the first event ends, to determine the clipped video clip;
    第三视频截取模块,被配置为如果所述第一事件结束后的所述预定时长内出现与所述第一事件关联的第二事件,且在所述第二事件结束后经历所述预定时长内未出现所述第一事件的关联事件,则结束所述视频截取任务,以确定截取出的视频片段。A third video interception module is configured to, if a second event associated with the first event occurs within the predetermined time period after the first event ends, and the predetermined time period elapses after the second event ends If the associated event of the first event does not occur within the context, the video clipping task is ended to determine the clipped video segment.
  18. 根据权利要求17所述的视频处理装置,其中,所述视频处理装置还包括:The video processing apparatus according to claim 17, wherein the video processing apparatus further comprises:
    视频片段上传模块,被配置为将所述截取出的视频片段上传至云端;a video clip uploading module, configured to upload the clipped video clips to the cloud;
    以供所述云端响应于所述截取出的视频片段对应的视频获取请求,从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段,并将所述目标视频片段发送给发起所述视频获取请求的请求端;或者,所述云端从所述截取出的视频片段中剔除最后所述预定时长的视频片段,生成目标视频片段并存储,以便所述云端响应于所述截取出的视频片段对应的视频获取请求,将所述目标视频片段发送给发起所述视频获取请求的请求端。In order for the cloud to respond to the video acquisition request corresponding to the clipped video clip, remove the last video clip of the predetermined duration from the clipped video clip, generate a target video clip, and convert the target video clip. The clip is sent to the requesting end that initiated the video acquisition request; or, the cloud removes the last video clip of the predetermined duration from the clipped video clip, generates and stores the target video clip, so that the cloud can respond to the For the video acquisition request corresponding to the intercepted video clip, the target video clip is sent to the requester that initiates the video acquisition request.
  19. 一种计算机可读存储介质,其上存储有计算机程序,所述程序被处理器执行时实现如权利要求1至13中任一项所述的视频处理方法。A computer-readable storage medium on which a computer program is stored, which implements the video processing method according to any one of claims 1 to 13 when the program is executed by a processor.
  20. 一种电子设备,包括:An electronic device comprising:
    处理器;processor;
    存储器,被配置为存储一个或多个程序,当所述一个或多个程序被所述处理器执行时,使得所述处理器实现如权利要求1至13中任一项所述的视频处理方法。A memory configured to store one or more programs that, when executed by the processor, cause the processor to implement the video processing method of any one of claims 1 to 13 .
PCT/CN2021/126446 2021-01-21 2021-10-26 Video processing method and apparatus, computer readable storage medium, and electronic device WO2022156294A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110082809.3 2021-01-21
CN202110082809.3A CN114827713B (en) 2021-01-21 2021-01-21 Video processing method and device, computer readable storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
WO2022156294A1 true WO2022156294A1 (en) 2022-07-28

Family

ID=82524330

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/126446 WO2022156294A1 (en) 2021-01-21 2021-10-26 Video processing method and apparatus, computer readable storage medium, and electronic device

Country Status (2)

Country Link
CN (1) CN114827713B (en)
WO (1) WO2022156294A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100123830A1 (en) * 2008-11-17 2010-05-20 On Demand Real Time Llc Method and system for segmenting and transmitting on-demand live-action video in real-time
CN102547139A (en) * 2010-12-30 2012-07-04 北京新岸线网络技术有限公司 Method for splitting news video program, and method and system for cataloging news videos
CN105791730A (en) * 2014-12-23 2016-07-20 北京同步科技有限公司 Prerecording system and method applied to video monitoring
US20190026882A1 (en) * 2015-08-28 2019-01-24 Nec Corporation Analysis apparatus, analysis method, and storage medium
CN110830847A (en) * 2019-10-24 2020-02-21 杭州威佩网络科技有限公司 Method and device for intercepting game video clip and electronic equipment
CN112069937A (en) * 2020-08-21 2020-12-11 深圳市商汤科技有限公司 Event detection method and device, electronic equipment and storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2407943B1 (en) * 2010-07-16 2016-09-28 Axis AB Method for event initiated video capturing and a video camera for capture event initiated video
CN105681749A (en) * 2016-01-12 2016-06-15 上海小蚁科技有限公司 Method, device and system for previewing videos and computer readable media
CN111355990A (en) * 2020-03-17 2020-06-30 网易(杭州)网络有限公司 Video acquisition method and device, computer readable storage medium and electronic equipment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100123830A1 (en) * 2008-11-17 2010-05-20 On Demand Real Time Llc Method and system for segmenting and transmitting on-demand live-action video in real-time
CN102547139A (en) * 2010-12-30 2012-07-04 北京新岸线网络技术有限公司 Method for splitting news video program, and method and system for cataloging news videos
CN105791730A (en) * 2014-12-23 2016-07-20 北京同步科技有限公司 Prerecording system and method applied to video monitoring
US20190026882A1 (en) * 2015-08-28 2019-01-24 Nec Corporation Analysis apparatus, analysis method, and storage medium
CN110830847A (en) * 2019-10-24 2020-02-21 杭州威佩网络科技有限公司 Method and device for intercepting game video clip and electronic equipment
CN112069937A (en) * 2020-08-21 2020-12-11 深圳市商汤科技有限公司 Event detection method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN114827713A (en) 2022-07-29
CN114827713B (en) 2023-08-08

Similar Documents

Publication Publication Date Title
RU2597232C1 (en) Method for providing a video in real time and device for its implementation, as well as a server and a terminal device
TWI765304B (en) Image reconstruction method and image reconstruction device, electronic device and computer-readable storage medium
US9100667B2 (en) Life streaming
EP4096222A1 (en) Live broadcast assistance method and electronic device
WO2021180004A1 (en) Video analysis method, video analysis management method, and related device
CN111147808B (en) Network device, image processing method and computer readable medium
WO2022042389A1 (en) Search result display method and apparatus, readable medium, and electronic device
US20160100149A1 (en) System and methods for simultaneously capturing audio and image data for digital playback
CN111708663A (en) Cloud computing safety monitoring system based on artificial intelligence
CN113055709B (en) Video publishing method, device, equipment, storage medium and program product
WO2020052062A1 (en) Detection method and device
CN110852306A (en) Safety monitoring system based on artificial intelligence
KR20150083491A (en) Methed and system for synchronizing usage information between device and server
WO2022156294A1 (en) Video processing method and apparatus, computer readable storage medium, and electronic device
JPWO2015178234A1 (en) Image search system, search screen display method
US11163822B2 (en) Emotional experience metadata on recorded images
US9955162B2 (en) Photo cluster detection and compression
WO2017045068A1 (en) Methods and apparatus for information capture and presentation
WO2017049474A1 (en) Filming method and smart wristband
WO2021129444A1 (en) File clustering method and apparatus, and storage medium and electronic device
CN109874036B (en) Video analysis method and device, equipment and storage medium
CN110300290B (en) Teaching monitoring management method, device and system
CN108828965B (en) Positioning method, electronic equipment, intelligent household system and storage medium
CN107357423B (en) Information display method and first electronic equipment
CN112437279B (en) Video analysis method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21920672

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21920672

Country of ref document: EP

Kind code of ref document: A1