CN116597351A - Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium - Google Patents

Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium Download PDF

Info

Publication number
CN116597351A
CN116597351A CN202310550709.8A CN202310550709A CN116597351A CN 116597351 A CN116597351 A CN 116597351A CN 202310550709 A CN202310550709 A CN 202310550709A CN 116597351 A CN116597351 A CN 116597351A
Authority
CN
China
Prior art keywords
behavior analysis
target
target video
video
sampling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310550709.8A
Other languages
Chinese (zh)
Inventor
彭礼晖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Yuncong Enterprise Development Co ltd
Original Assignee
Shanghai Yuncong Enterprise Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Yuncong Enterprise Development Co ltd filed Critical Shanghai Yuncong Enterprise Development Co ltd
Priority to CN202310550709.8A priority Critical patent/CN116597351A/en
Publication of CN116597351A publication Critical patent/CN116597351A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022—Mechanisms to release resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses a behavior analysis method, which comprises the following steps: acquiring a target video; sampling the target video to obtain a plurality of target video frames; converting the plurality of target video frames to obtain a plurality of target pictures; and distributing the plurality of target pictures to a plurality of behavior analysis engines to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine. The method comprises the steps of obtaining a plurality of target video frames by sampling one path of video; the same video is multiplexed to enable a plurality of behavior analysis engines to use the same path of decoded image, and the image output analysis result can be analyzed according to the sampling frequency of the behavior analysis engines per se, so that the problem that redundant CPU (Central processing Unit) is consumed when the behavior analysis engines repeatedly decode the same video is solved, and the concurrency upper limit of an analysis system is improved.

Description

Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium
Technical Field
The present application relates to the field of image processing, and in particular, to a behavior analysis method, apparatus, machine readable medium, and device.
Background
The need for multiple behavioral analysis of the same scene is encountered in projects involving public transportation or public places. For example, for a section of urban road, it is detected whether the road has water accumulation, whether the road surface has breakage, whether the road is congested, whether the road has pedestrians, and the like. The sampling detection frequencies required for each behavior analysis are different from each other. For example, the road water accumulation is detected once for 10 seconds, the road surface is detected once for 60 minutes if the road surface is broken, the road is detected once for 1 second if the road is jammed, and the road is detected once for 1 second if the road has pedestrians. Then for the same video source, the video source needs to be simultaneously sent to a plurality of different behavior analysis engines, so that the video source can conduct behavior analysis according to a set sampling frequency to meet the detection requirement.
In the existing analysis system, each engine independently pulls a video stream, and the engine decodes, samples and then analyzes the video. The cost of such a processing mode is that the CPU is additionally consumed, and the performance of the behavior analysis engine is greatly reduced.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, an object of the present application is to provide a behavior analysis method, apparatus, machine-readable medium and device for solving the problems of the prior art.
To achieve the above and other related objects, the present application provides a behavior analysis method, including:
acquiring a target video;
sampling the target video to obtain a plurality of target video frames;
converting the plurality of target video frames to obtain a plurality of target pictures;
and distributing the plurality of target pictures to a plurality of behavior analysis engines to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine.
In an embodiment of the present application, the step of obtaining the target video includes:
acquiring a behavior analysis task, wherein the behavior analysis task carries a video source address;
determining a target video source based on the video source address;
and pulling the video stream from the target video source to obtain a target video.
In an embodiment of the present application, the step of converting the plurality of target video frames includes:
decoding the plurality of target video frames through a decoder to obtain image frames corresponding to each target video frame;
and encoding the image frames corresponding to each target video frame to obtain target pictures corresponding to each target video frame.
In an embodiment of the present application, the behavior analysis task further carries a behavior analysis engine address, and the step of distributing the plurality of target pictures to a plurality of behavior analysis engines includes:
distributing the plurality of target pictures to a plurality of behavior analysis engines based on the behavior analysis engine address.
In an embodiment of the present application, after the target video is acquired, the method further includes: caching the target video;
after decoding the plurality of target video frames by a decoder, the method further comprises: and caching the image frames.
In an embodiment of the present application, the behavior analysis task further carries sampling conditions, and the step of sampling the target video includes:
and sampling the target video under preset sampling conditions, wherein the preset sampling conditions comprise sampling time and sampling frequency.
In an embodiment of the present application, after the video stream is pulled from the target video source, the pulled video stream is filtered to obtain the target video.
To achieve the above and other related objects, the present application also provides a behavior analysis apparatus comprising:
the video acquisition module is used for acquiring a target video;
the sampling module is used for sampling the target video to obtain a plurality of target video frames;
the conversion module is used for converting the plurality of target video frames to obtain a plurality of target pictures;
and the distribution module is used for distributing the plurality of target pictures to a plurality of behavior analysis engines so as to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine.
To achieve the above and other related objects, the present application also provides a behavior analysis apparatus comprising:
one or more processors; and
one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the device to perform one or more of the behavior analysis methods described previously.
To achieve the above and other related objects, the present application also provides one or more machine-readable media having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform one or more of the behavior analysis methods described above.
As described above, the behavior analysis method, apparatus, machine-readable medium, device, machine-readable medium, and apparatus provided by the present application have the following beneficial effects:
the application provides a behavior analysis method, which comprises the following steps: acquiring a target video; sampling the target video to obtain a plurality of target video frames; converting the plurality of target video frames to obtain a plurality of target pictures; and distributing the plurality of target pictures to a plurality of behavior analysis engines to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine. The method comprises the steps of obtaining a plurality of target video frames by sampling one path of video; the same channel of decoded images are used by a plurality of behavior analysis engines through multiplexing the same video technology, and the image output analysis results can be analyzed according to the sampling frequency of the behavior analysis engines, so that the problem that redundant CPU (Central processing Unit) is consumed by repeated decoding of the video of the same video is solved, the concurrency upper limit of an analysis system is improved, and considerable CPU resources are released for other service systems to use, so that the intelligent analysis capability of the engines is improved under the condition that the hardware cost and hardware equipment are not changed, namely, the concurrency analysis task supported by one analysis server can be effectively improved, and the original analysis accuracy is not reduced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an implementation environment of a behavior analysis method according to an exemplary embodiment of the present application;
FIG. 2 is a flow chart of a behavior analysis method according to an exemplary embodiment of the present application;
FIG. 3 is a flowchart of a method for capturing a target video according to an exemplary embodiment of the present application
FIG. 4 is a flow chart illustrating a method of converting a target video frame according to an exemplary embodiment of the present application;
fig. 5 is a schematic structural view of a behavior analysis apparatus according to an exemplary embodiment of the present application;
fig. 6 is a schematic hardware structure of a terminal device according to an embodiment of the present application;
fig. 7 is a schematic hardware structure of a terminal device according to an embodiment of the application.
Detailed Description
Further advantages and effects of the present application will become readily apparent to those skilled in the art from the disclosure herein, by referring to the accompanying drawings and the preferred embodiments. The application may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present application. It should be understood that the preferred embodiments are presented by way of illustration only and not by way of limitation.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present application by way of illustration, and only the components related to the present application are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.
In the following description, numerous details are set forth in order to provide a more thorough explanation of embodiments of the present application, it will be apparent, however, to one skilled in the art that embodiments of the present application may be practiced without these specific details, in other embodiments, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the embodiments of the present application.
The need for multiple behavioral analysis of the same scene is encountered in projects involving public transportation or public places. For example, for a section of urban road, it is detected whether the road has water accumulation, whether the road surface has breakage, whether the road is congested, whether the road has pedestrians, and the like. The sampling detection frequencies required for each behavior analysis are different from each other. For example, the road water accumulation is detected once for 10 seconds, the road surface is detected once for 60 minutes if the road surface is broken, the road is detected once for 1 second if the road is jammed, and the road is detected once for 1 second if the road has pedestrians. Then for the same video source, the video source needs to be simultaneously sent to a plurality of different behavior analysis engines, so that the video source can conduct behavior analysis according to a set sampling frequency to meet the detection requirement.
In the conventional analysis system structure, each engine independently pulls a video stream, and the engine decodes, samples and then analyzes the video. For example, 5 different functional behavior analysis engines analyze the same video source at the same time, then they pull 5 streams at the same time and then each decodes the video. The cost of the processing mode is that the decoding CPU of 4 paths of videos is additionally consumed, one path of 1080P video decoding generally consumes 10% of the performance of the single-core CPU, so that 40% of the single-core CPU is additionally consumed, and the performance of the behavior analysis engine is greatly reduced.
Aiming at the analysis scene, the problem is to promote the capability of the engine for intelligent analysis under the condition of ensuring that the hardware cost and hardware equipment are not changed, namely the concurrent analysis task supported by one analysis server can be effectively promoted, and the original analysis accuracy is not reduced.
In view of the above problems, embodiments of the present application respectively provide a behavior analysis method, a behavior analysis device, a behavior analysis electronic apparatus, and a computer readable storage medium, by which the problem of high performance consumption of an analysis system CPU in the prior art can be solved.
FIG. 1 is a schematic diagram of an exemplary behavior analysis method implementation environment of the present application. Referring to fig. 1, the implementation environment includes an application system, a video source, a flow control unit, a task scheduler, a data output pipeline, and a behavior analysis engine, wherein the flow control unit, the task scheduler, and the data output pipeline form a video snapshot system.
Video source: providing real-time audio and video data, and pulling real-time audio and video streams from the real-time audio and video data through an audio and video stream media transmission protocol, such as a monitoring camera, an image acquisition card and the like;
an application system: adding and deleting behavior analysis tasks, accessing a video snapshot system through an HTTP-API and modifying the behavior analysis tasks, wherein the behavior analysis tasks are managed by a task scheduler in the video snapshot system;
flow control unit: the video access module, the decoder and the encoder are integrated;
the video access module is used for pulling an audio and video ES stream from a specified video source (target video source) and submitting the ES stream to the flow control unit; the decoder is responsible for decoding the video into YUV image frames, and the encoder is responsible for encoding the YUV image frames into JPEG pictures, so that the encoding standards of all pictures are unified;
audio and video ES stream: the elementary streams are audio and video data streams directly coming out of the encoder, and may be encoded video data streams (h.264, MJPEG, etc.), audio data streams (AAC), or other general names of encoded data streams;
video ES frames: a single coded video frame, and a plurality of coded video frames are combined in series to form an audio-video ES stream;
YUV image frame: the video ES frames are decoded by a video decoder to obtain data describing the gray scale and color of the computer image frame, and the computer operating system can easily render the YUV image frames to the display device. Functions related to computer image calculation are generally designed and used by taking YUV image frames as minimum units;
task scheduler: the system comprises a flow control unit, a data output pipeline, a video source, a sampling frequency of a target video, a collection time, a management and control behavior analysis task, an application system and a behavior analysis task.
Data output pipeline: receiving pictures and forwarding the pictures to an affiliated behavior analysis engine; the received pictures are sent to a target behavior analysis engine, and can be specifically output in various modes such as HTTP(s) -Post, kafka, websocket, tcp.
Audio and video ES stream: the elementary streams are audio and video data streams directly coming from the encoder, and may be encoded video data streams (h.264, MJPEG, etc.), audio data streams (AAC), or other general names of encoded data streams.
Video ES frames: a single encoded video frame and a plurality of encoded video frames are concatenated and combined into an audio-video ES stream.
YUV image frame: the video ES frames are decoded by a video decoder to obtain data describing the gray scale and color of the computer image frame, and the computer operating system can easily render the YUV image frames to the display device. Functions generally associated with computer image computation are designed and used with YUV image frames as the smallest unit.
The application system sends a behavior analysis task to the task scheduler, and then the flow control unit acquires a target video from a target video source and samples the target video to obtain a plurality of target video frames; converting the plurality of target video frames to obtain a plurality of target pictures; after a plurality of target pictures are obtained, distributing the target pictures to a plurality of behavior analysis engines through a data output pipeline based on a task scheduler, so that the behavior analysis is carried out on the basis of the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine.
It should be understood that the number of video sources and behavior analysis engines in fig. 1 is merely illustrative. There may be any number of video sources and behavior analysis engines, as practical.
The video snapshot system may correspond to a service end, which may be a server providing various services, may be an independent physical server, may be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing cloud services, a cloud database, cloud computing, a cloud function, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a CDN (content delivery network), basic cloud computing services such as big data and an artificial intelligent platform, which are not limited herein.
The video source may communicate with the video snapshot system through a wireless network such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), 5G (fifth generation mobile information technology), etc., while the video snapshot system may communicate with the behavior analysis engine through a wireless network such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), 5G (fifth generation mobile information technology), etc., which is not limited in this regard.
Referring to fig. 2, fig. 2 is a flowchart illustrating a behavior analysis method according to an exemplary embodiment of the present application. It should be understood that the behavior analysis method may also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and the embodiment is not limited to the implementation environments to which the behavior analysis method is applied.
Referring to fig. 2, fig. 2 is a flowchart of a behavior analysis method according to an exemplary embodiment of the application, the behavior analysis method at least includes steps S210 to S240, and the following details are described:
step S210, obtaining a target video;
target video, i.e., a video stream obtained from a video source. It will be appreciated that the video source provides real-time audio and video data from which real-time audio and video streams, such as surveillance cameras, image capture cards, etc., may be pulled via an audio and video streaming media transmission protocol.
Referring to fig. 3, fig. 3 is a flowchart illustrating a method for acquiring a target video according to an exemplary embodiment of the application. In fig. 3, the step of acquiring the target video includes steps S310 to S330:
step S310, a behavior analysis task is obtained, wherein the behavior analysis task carries a video source address;
step S320, determining a target video source based on the video source address;
step S330, pull the video stream from the target video source, and obtain the target video.
The behavior analysis task is issued to the task scheduler by the application system. The behavior analysis task is a task of completing behavior analysis of a target object in the target video based on the target video. The target object may be a person, a car, a road, etc. For example, the target object is a road, and the behavior of the target object includes, but is not limited to, whether the road is water-bearing, whether the road surface is broken, whether the road is congested, whether the road is pedestrian, and the like.
Since the video source may include a plurality of video sources, such as video source 1 and video source 2, when the behavior analysis of the corresponding target object needs to be completed, it is necessary to determine which video source is to be subjected to the behavior analysis. Therefore, when the task scheduler receives the behavior analysis task, the task scheduler then analyzes the behavior analysis task to obtain the video source address carried in the behavior analysis task, and then determines the target video source through the video source address.
After the target video source is determined, the real-time audio and video stream, such as a monitoring camera, an image acquisition card and the like, can be pulled from the target video source through an audio and video streaming media transmission protocol, so that the purpose of pulling the video stream from the target video source and obtaining the target video is achieved.
In the application, the video stream of the video source is continuously received through the flow control unit, and after the video stream is received, the video stream can be filtered, namely, after the video stream is pulled from the target video source, the pulled video stream is filtered, so that the target video is obtained. After the target video is obtained, the target video is cached, namely, the target video frame is sent into a caching queue for waiting for consumption. After the video snapshot system is accessed to the behavior analysis task, the corresponding video stream is sent to a decoder in the flow control unit.
Step S220, sampling the target video to obtain a plurality of target video frames;
the application aims to solve the problem of large resource consumption caused by simultaneously pulling multiple paths of video sources by analyzing the same video source through a plurality of different behavior analysis engines in the prior art. Therefore, in the present application, it is proposed to meet the analysis requirements of multiple behavior analysis engines by pulling only one video source.
Specifically, the behavior analysis task further carries sampling conditions, and the step of sampling the target video includes:
and sampling the target video under preset sampling conditions, wherein the preset sampling conditions comprise sampling time and sampling frequency.
The sampling time indicates from what time point sampling is started, and the sampling frequency indicates how long time intervals sampling is performed. It should be further noted that the sampling frequency may be set according to different behavior analysis tasks, for example, whether the road is water-logging detected for 10 seconds, whether the road is broken for 60 minutes, whether the road is jammed for 1 second, and whether the road is pedestrian for 1 second.
By the method, the target video is sampled through different sampling frequencies, so that a plurality of target video frames corresponding to one path of video stream are obtained.
Step S230, converting the plurality of target video frames to obtain a plurality of target pictures;
referring to fig. 4, fig. 4 is a flowchart illustrating a method for converting a target video frame according to an exemplary embodiment of the present application. In fig. 4, the step of converting the plurality of target video frames includes steps S410 to S420:
step S410, decoding the plurality of target video frames through a decoder to obtain image frames corresponding to each target video frame;
in an embodiment, after decoding the plurality of target video frames by a decoder, the method further comprises: and caching the image frames. I.e. adding the image frames to the buffer queue.
Step S420, encoding the image frame corresponding to each target video frame to obtain a target picture corresponding to each target video frame.
It should be noted that, the image frame corresponding to the target video decoded by the decoder may be a YUV image frame, and the target picture encoded by the encoder may be a JPEG picture;
the video frames are decompressed into YUV image frames, and then the YUV image frames are encoded into JPEG pictures, so that the encoding standards of all pictures are unified.
Step S240, distributing the plurality of target pictures to a plurality of behavior analysis engines, so as to perform behavior analysis based on the target pictures through the behavior analysis engines, where each target picture corresponds to one behavior analysis engine.
According to the application, the analysis of a plurality of target pictures is completed through a plurality of analysis engines, and a knowledge needs to be made about which analysis engine correspondingly analyzes which target picture. Therefore, the behavior analysis task also carries a behavior analysis engine address, and the step of distributing the plurality of target pictures to a plurality of behavior analysis engines comprises the following steps: distributing the plurality of target pictures to a plurality of behavior analysis engines based on the behavior analysis engine address.
It should be noted that, the type of the behavior analysis task is an adding task, and the task scheduler will notify the stream control unit to open a stream to the video source indicated by the video source address carried in the task after receiving the adding task. After receiving the stream opening request, the stream control unit initiates stream opening to the video source and maintains stream stability, and if the stream is interrupted or abnormal, the stream control unit continuously tries to open the stream until the stream opening is successful or the stream is closed after the task is terminated.
Of course, the type of behavior analysis task may be a delete task, and the traffic conditioner may notify the flow control unit to close the designated video source and delete the task from the task scheduler. The stream control unit receives the stream-closing request, closes the video stream that the specified video source has been or is being opened, and does not receive the ES stream data of this video source.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.
In summary, the method and the device acquire the target video; sampling the target video to obtain a plurality of target video frames; converting the plurality of target video frames to obtain a plurality of target pictures; and distributing the plurality of target pictures to a plurality of behavior analysis engines to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine. The method comprises the steps of obtaining a plurality of target video frames by sampling one path of video; the same channel of decoded images are used by a plurality of behavior analysis engines through multiplexing the same video technology, and the image output analysis results can be analyzed according to the sampling frequency of the behavior analysis engines, so that the problem that redundant CPU (Central processing Unit) is consumed by repeated decoding of the video of the same video is solved, the concurrency upper limit of an analysis system is improved, and considerable CPU resources are released for other service systems to use, so that the intelligent analysis capability of the engines is improved under the condition that the hardware cost and hardware equipment are not changed, namely, the concurrency analysis task supported by one analysis server can be effectively improved, and the original analysis accuracy is not reduced.
Fig. 5 is a block diagram of a behavior analysis apparatus according to an exemplary embodiment of the present application. The apparatus may be applied to the implementation environment shown in fig. 1, and is specifically configured in a server. The behavior analysis device may be applied to other exemplary implementation environments and may be specifically configured in other apparatuses, and the embodiment does not limit the implementation environment to which the behavior analysis device is applied.
As shown in fig. 5, the present application further provides a behavior analysis apparatus, which includes:
the video acquisition module 510 is configured to acquire a target video;
the sampling module 520 is configured to sample the target video to obtain a plurality of target video frames;
the conversion module 530 is configured to convert the plurality of target video frames to obtain a plurality of target pictures;
the distribution module 540 is configured to distribute the plurality of target pictures to a plurality of behavior analysis engines, so as to perform behavior analysis based on the target pictures through the behavior analysis engines, where each target picture corresponds to one behavior analysis engine.
In one embodiment, the step of obtaining the target video includes:
acquiring a behavior analysis task, wherein the behavior analysis task carries a video source address;
determining a target video source based on the video source address;
and pulling the video stream from the target video source to obtain a target video.
In one embodiment, the step of converting the plurality of target video frames comprises:
decoding the plurality of target video frames through a decoder to obtain image frames corresponding to each target video frame;
and encoding the image frames corresponding to each target video frame to obtain target pictures corresponding to each target video frame.
In an embodiment, the behavior analysis task further carries a behavior analysis engine address, and the step of distributing the plurality of target pictures to a plurality of behavior analysis engines includes:
distributing the plurality of target pictures to a plurality of behavior analysis engines based on the behavior analysis engine address.
In an embodiment, after the capturing the target video, the method further includes: caching the target video;
after decoding the plurality of target video frames by a decoder, the method further comprises: and caching the image frames.
In an embodiment, the behavior analysis task further carries sampling conditions, and the step of sampling the target video includes:
and sampling the target video under preset sampling conditions, wherein the preset sampling conditions comprise sampling time and sampling frequency.
In one embodiment, after the video stream is pulled from the target video source, the pulled video stream is filtered to obtain the target video.
It should be noted that, the behavior analysis device provided in the foregoing embodiment and the behavior analysis method provided in the foregoing embodiment belong to the same concept, and a specific manner in which each module and unit perform an operation has been described in detail in the embodiment of the behavior analysis method, which is not described herein again. In practical application provided by the above embodiment, the above function allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the functions described above, which is not limited herein.
The embodiment of the application also provides equipment, which can comprise: one or more processors; and one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the behavior analysis method described in fig. 2. In practical applications, the device may be used as a terminal device or may be used as a server, and examples of the terminal device may include: smart phones, tablet computers, e-book readers, MP3 (dynamic image expert compression standard voice plane 3,MovingPicture ExpertsGroupAudioLayerIII) players, MP4 (dynamic image expert compression standard voice plane 4, movingpictureexpertsgroupuudiolayeriv) players, laptop portable computers, car-mounted computers, desktop computers, set-top boxes, smart televisions, wearable devices, etc., embodiments of the present application are not limited to specific devices.
The embodiment of the application also provides a non-volatile readable storage medium, in which one or more modules (programs) are stored, where the one or more modules are applied to a device, and the device can execute instructions (instructions) of steps included in the behavior analysis method in fig. 2 in the embodiment of the application.
Fig. 6 is a schematic hardware structure of a terminal device according to an embodiment of the present application. As shown, the terminal device may include: an input device 1100, a first processor 1101, an output device 1102, a first memory 1103 and at least one communication bus 1104. The communication bus 1104 is used to enable communication connections between the elements. The first memory 1103 may include a high-speed RAM memory, and may further include a nonvolatile memory NVM, such as at least one magnetic disk memory, where various programs may be stored in the first memory 1103 for performing various processing functions and implementing the behavior analysis method steps of the present embodiment.
Alternatively, the first processor 1101 may be implemented as, for example, a central processing unit (CentralProcessing Unit, abbreviated as CPU), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and the first processor 1101 is coupled to the input device 1100 and the output device 1102 through a wired or wireless connection.
Alternatively, the input device 1100 may include a variety of input devices, for example, may include at least one of a user-oriented user interface, a device-oriented device interface, a programmable interface of software, a camera, and a sensor. Optionally, the device interface facing to the device may be a wired interface for data transmission between devices, or may be a hardware insertion interface (such as a USB interface, a serial port, etc.) for data transmission between devices; alternatively, the user-oriented user interface may be, for example, a user-oriented control key, a voice input device for receiving voice input, and a touch-sensitive device (e.g., a touch screen, a touch pad, etc. having touch-sensitive functionality) for receiving user touch input by a user; optionally, the programmable interface of the software may be, for example, an entry for a user to edit or modify a program, for example, an input pin interface or an input interface of a chip, etc.; the output device 1102 may include a display, sound, or the like.
In this embodiment, the processor of the terminal device may include a function for executing each module in each device, and specific functions and technical effects may be referred to the above embodiments and are not described herein again.
Fig. 7 is a schematic hardware structure of a terminal device according to an embodiment of the present application. Fig. 7 is a specific embodiment of the implementation of fig. 6. As shown, the terminal device of the present embodiment may include a second processor 1201 and a second memory 1202.
The second processor 1201 executes the computer program code stored in the second memory 1202 to implement the steps of the behavior analysis method described in fig. 2 in the above embodiment.
The second memory 1202 is configured to store various types of data to support operations at the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, such as messages, pictures, video, etc. The second memory 1202 may include a random access memory (randomaccess memory, simply RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
Optionally, a second processor 1201 is provided in the processing assembly 1200. The terminal device may further include: a communication component 1203, a power component 1204, a multimedia component 1205, a voice component 1206, an input/output interface 1207, and/or a sensor component 1208. The components and the like specifically included in the terminal device are set according to actual requirements, which are not limited in this embodiment.
The processing component 1200 generally controls the overall operation of the terminal device. The processing assembly 1200 may include one or more second processors 1201 to execute instructions to perform all or part of the steps in the behavior analysis method described above. Further, the processing component 1200 may include one or more modules that facilitate interactions between the processing component 1200 and other components. For example, the processing component 1200 may include a multimedia module to facilitate interaction between the multimedia component 1205 and the processing component 1200.
The power supply component 1204 provides power to the various components of the terminal device. Power supply components 1204 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for terminal devices.
The multimedia component 1205 includes a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.
The voice component 1206 is configured to output and/or input voice signals. For example, the voice component 1206 includes a Microphone (MIC) configured to receive external voice signals when the terminal device is in an operational mode, such as a voice recognition mode. The received voice signals may be further stored in the second memory 1202 or transmitted via the communication component 1203. In some embodiments, the voice component 1206 further includes a speaker for outputting voice signals.
The input/output interface 1207 provides an interface between the processing assembly 1200 and peripheral interface modules, which may be click wheels, buttons, and the like. These buttons may include, but are not limited to: volume button, start button and lock button.
The sensor assembly 1208 includes one or more sensors for providing status assessment of various aspects for the terminal device. For example, the sensor assembly 1208 may detect an on/off state of the terminal device, a relative positioning of the assembly, and the presence or absence of user contact with the terminal device. The sensor assembly 1208 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor assembly 1208 may also include a camera or the like.
The communication component 1203 is configured to facilitate communication between the terminal device and other devices in a wired or wireless manner. The terminal device may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot therein for inserting a SIM card, so that the terminal device may log into a GPRS network and establish communication with a server via the internet.
From the above, the communication component 1203, the voice component 1206, the input/output interface 1207, and the sensor component 1208 in the embodiment of fig. 7 can be implemented as the input device in the embodiment of fig. 7.
The above embodiments are merely illustrative of the principles of the present application and its effectiveness, and are not intended to limit the application. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that all equivalent modifications and variations of the application be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims (10)

1. A method of behavioral analysis, the method comprising:
acquiring a target video;
sampling the target video to obtain a plurality of target video frames;
converting the plurality of target video frames to obtain a plurality of target pictures;
and distributing the plurality of target pictures to a plurality of behavior analysis engines to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine.
2. The behavior analysis method according to claim 1, wherein the step of acquiring the target video includes:
acquiring a behavior analysis task, wherein the behavior analysis task carries a video source address;
determining a target video source based on the video source address;
and pulling the video stream from the target video source to obtain a target video.
3. The behavior analysis method according to claim 1, wherein the step of converting the plurality of target video frames includes:
decoding the plurality of target video frames through a decoder to obtain image frames corresponding to each target video frame;
and encoding the image frames corresponding to each target video frame to obtain target pictures corresponding to each target video frame.
4. The behavior analysis method according to claim 2, wherein the behavior analysis task further carries a behavior analysis engine address, and the step of distributing the plurality of target pictures to a plurality of behavior analysis engines comprises:
distributing the plurality of target pictures to a plurality of behavior analysis engines based on the behavior analysis engine address.
5. A behavior analysis method according to claim 3, characterized in that after the acquisition of the target video, the method further comprises: caching the target video;
after decoding the plurality of target video frames by a decoder, the method further comprises: and caching the image frames.
6. The behavior analysis method according to claim 2, wherein the behavior analysis task further carries sampling conditions, and the step of sampling the target video includes:
and sampling the target video under preset sampling conditions, wherein the preset sampling conditions comprise sampling time and sampling frequency.
7. The behavioral analysis method of claim 2, wherein after pulling a video stream from the target video source, the pulled video stream is filtered to obtain a target video.
8. A behavior analysis apparatus, the apparatus comprising:
the video acquisition module is used for acquiring a target video;
the sampling module is used for sampling the target video to obtain a plurality of target video frames;
the conversion module is used for converting the plurality of target video frames to obtain a plurality of target pictures;
and the distribution module is used for distributing the plurality of target pictures to a plurality of behavior analysis engines so as to conduct behavior analysis based on the target pictures through the behavior analysis engines, wherein each target picture corresponds to one behavior analysis engine.
9. A behavior analysis apparatus, characterized by comprising:
one or more processors; and
one or more machine readable media having instructions stored thereon, which when executed by the one or more processors, cause the apparatus to perform the behavior analysis method of one or more of claims 1-7.
10. One or more machine readable media having instructions stored thereon that, when executed by one or more processors, cause an apparatus to perform the behavior analysis method of one or more of claims 1-7.
CN202310550709.8A 2023-05-16 2023-05-16 Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium Pending CN116597351A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310550709.8A CN116597351A (en) 2023-05-16 2023-05-16 Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310550709.8A CN116597351A (en) 2023-05-16 2023-05-16 Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium

Publications (1)

Publication Number Publication Date
CN116597351A true CN116597351A (en) 2023-08-15

Family

ID=87604028

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310550709.8A Pending CN116597351A (en) 2023-05-16 2023-05-16 Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium

Country Status (1)

Country Link
CN (1) CN116597351A (en)

Similar Documents

Publication Publication Date Title
CN104137146A (en) Method and system for video coding with noise filtering of foreground object segmentation
CN110177300B (en) Program running state monitoring method and device, electronic equipment and storage medium
CN110365973B (en) Video detection method and device, electronic equipment and computer readable storage medium
CN110471709B (en) Method, device, medium and electronic equipment for accelerating webpage opening speed
CN113784049B (en) Camera calling method of android system virtual machine, electronic equipment and storage medium
US20160026728A1 (en) Interaction Method And Device Between Browsers And Browser
CN114679607B (en) Video frame rate control method and device, electronic equipment and storage medium
CN111935663B (en) Sensor data stream processing method, device, medium and electronic equipment
CN114268796A (en) Method and device for processing video stream
CN114647390B (en) Enhanced screen sharing method and system and electronic equipment
WO2021082600A1 (en) Webpage image monitoring method and apparatus, electronic device, and computer readable storage medium
CN111783632B (en) Face detection method and device for video stream, electronic equipment and storage medium
CN115225881A (en) Data transmission method, device, equipment and storage medium
CN109831673B (en) Live broadcast room data processing method, device, equipment and storage medium
CN116597351A (en) Behavior analysis method, behavior analysis device, machine-readable medium and machine-readable medium
KR102445154B1 (en) data transfer
CN116186545A (en) Training and application methods and devices of pre-training model, electronic equipment and medium
CN113395523B (en) Image decoding method, device, equipment and storage medium based on parallel threads
CN109886234B (en) Target detection method, device, system, electronic equipment and storage medium
WO2021082599A1 (en) Wed page image monitoring method and device, electronic equipment and computer readable storage medium
CN115730104A (en) Live broadcast room processing method, device, equipment and medium
CN112347982A (en) Video-based unsupervised difficult case data mining method, device, medium and equipment
CN109756756B (en) Video playing method and video playing device
CN118138787A (en) Live broadcast multi-opening method, device, terminal equipment and storage medium
CN117208725A (en) Staircase state monitoring method, device, machine-readable medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination