WO2022237591A1 - 运动对象的识别方法、装置、电子设备及可读存储介质 - Google Patents

运动对象的识别方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2022237591A1
WO2022237591A1 PCT/CN2022/090390 CN2022090390W WO2022237591A1 WO 2022237591 A1 WO2022237591 A1 WO 2022237591A1 CN 2022090390 W CN2022090390 W CN 2022090390W WO 2022237591 A1 WO2022237591 A1 WO 2022237591A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
interest
image frames
color mode
Prior art date
Application number
PCT/CN2022/090390
Other languages
English (en)
French (fr)
Inventor
何伟
马欣
吴臻志
祝夭龙
Original Assignee
北京灵汐科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京灵汐科技有限公司 filed Critical 北京灵汐科技有限公司
Publication of WO2022237591A1 publication Critical patent/WO2022237591A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • Embodiments of the present disclosure relate to the technical field of image detection, and in particular, to a method, device, electronic device, and readable storage medium for identifying a moving object.
  • monitoring equipment is full of various public places such as streets, communities, and buildings.
  • time when moving objects appear in the video collected by the monitoring equipment is relatively sparse, when analyzing the video collected by the monitoring equipment, it usually includes a large number of image frames without actual content, which consumes a lot of unnecessary computing resources and reduces the computing efficiency. lower.
  • Embodiments of the present disclosure provide a moving object recognition method, device, electronic device, and readable storage medium, so as to save computing resources for recognizing moving objects and improve computing efficiency.
  • an embodiment of the present disclosure provides a method for identifying a moving object, including: acquiring event stream data collected by an event-based visual sensor; performing target detection of interest according to the collected event stream data to obtain a detection result; When the detection result is that the target of interest is detected, the target image frame set is obtained; wherein, the target image frame set includes image frames in the target time period, and the target time period includes the time period in which the target of interest appears; according to the target image frame set , to identify moving objects.
  • an embodiment of the present disclosure provides an apparatus for identifying a moving object, including: a first acquisition module, configured to acquire event flow data collected by an event-based visual sensor; a detection module, configured to , to detect the target of interest to obtain the detection result; the second acquisition module is used to obtain the target image frame set when the detection result is that the target of interest is detected; wherein, the target image frame set includes images within the target time period The frame, the target time period includes the time period when the target of interest appears; the recognition module is used to identify the moving object according to the set of target image frames.
  • the embodiment of the present disclosure also provides an electronic device, which is characterized in that it includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor.
  • a processor a memory
  • a program or instruction stored in the memory and operable on the processor.
  • the embodiments of the present disclosure further provide a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method in the first aspect are implemented.
  • the embodiments of the present disclosure further provide a computer program product, including computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, wherein, when the computer readable codes are stored in the electronic device When running in the processor of the electronic device, the processor in the electronic device executes the steps for realizing the method of the first aspect.
  • the dynamic visual sensor DVS event flow is obtained; according to the DVS event flow, the object of interest is detected; when the object of interest is detected, the target image frame set is obtained; the target image frame set includes the target time period The target time period includes the time period when the target of interest appears; according to the set of target image frames, the moving object is identified.
  • the target of interest is detected based on the DVS event flow, it is determined that the target of interest exists, and the target image frame set determined based on the target of interest is obtained and analyzed to identify the moving object, thereby saving computing resources and improving computing efficiency.
  • FIG. 1 is a flowchart of a method for identifying a moving object provided by an embodiment of the present disclosure
  • FIG. 2 is a structural diagram of a network system in an embodiment of the present disclosure
  • Fig. 3 is a structural block diagram of an identification device for a moving object provided by an embodiment of the present disclosure
  • Fig. 4 is a structural block diagram of an electronic device provided by an embodiment of the present disclosure.
  • first and second are used for description purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features.
  • plural means two or more.
  • the picture in the picture frame without actual content may be any of the following situations: the picture is completely black or completely blue, the picture is not clear (the sharpness of the picture is less than the preset sharpness threshold), or the picture has relatively low Large-area occlusion (the proportion of the occluded picture in the overall area of the video picture exceeds a predetermined ratio threshold).
  • the moving object can be identified by the frame difference method (also called the inter-frame difference method); specifically, the frame difference method is to perform a difference operation on the pixels corresponding to the front and rear frames of the color mode picture to obtain The difference is obtained, and when the absolute value of the difference exceeds a certain threshold, it can be determined whether there is a moving object (or moving object) in the picture, so as to identify the moving object.
  • the frame difference method also called the inter-frame difference method
  • the frame difference method is to perform a difference operation on the pixels corresponding to the front and rear frames of the color mode picture to obtain The difference is obtained, and when the absolute value of the difference exceeds a certain threshold, it can be determined whether there is a moving object (or moving object) in the picture, so as to identify the moving object.
  • the color mode is a color standard in the industry.
  • the color mode may include, but not limited to: one of a red green blue (RED Green Blue, RGB) color mode, a standard RGB (Standard Red Green Blue, sRGB) color mode, and Adobe RGB.
  • the RGB color mode obtains a variety of colors by changing the three color channels of red (RED, R), green (Green, G), and blue (Blue, B) and superimposing each other;
  • the sRGB color mode can Based on independent color coordinates, the color corresponds to the same color coordinate system in different equipment transmissions; Adobe RGB color mode has a larger color gamut space than sRGB color mode.
  • the image acquisition device based on the color mode may include, for example, at least one of the following devices: a camera photosensitive element based on the color mode, a camera lens based on the color mode, and a high-speed camera based on the color mode.
  • the color mode may include, for example, RGB color mode, sRGB color mode and Adobe RGB color mode; Be specific.
  • the RGB color mode can be referred to as RGB or RGB domain for short; and the image frame and video information collected by the image acquisition device based on the color mode can be referred to as is the RGB image frame and the RGB domain video information.
  • the moving object is identified by the frame difference method, for the dark moving object appearing in the dark place of the monitoring area, it is limited by the dynamic range of the sensor element of the camera based on the RGB color mode, and the continuous image in the moving area of the moving object The pixel changes in the picture are very small. At this time, it is difficult to find moving objects using the frame difference method, which is not conducive to effectively finding moving objects.
  • an embodiment of the present disclosure provides a method for identifying a moving object.
  • the method for identifying a moving object in the embodiments of the present disclosure may be executed by a corresponding apparatus for identifying a moving object.
  • the apparatus may be implemented in the form of software and/or hardware, and may generally be integrated into an electronic device.
  • Fig. 1 is a flowchart of a method for identifying a moving object provided by an embodiment of the present disclosure.
  • the method for identifying a moving object in an embodiment of the present disclosure includes:
  • the detection result is that the target of interest is detected, acquire a set of target image frames; wherein, the set of target image frames includes image frames within a target time period, and the target time period includes a time period when the target of interest appears;
  • the method for identifying a moving object in the embodiment of the present disclosure when it is determined based on the collected event stream data that there is a target of interest, acquires a set of target image frames determined based on the target of interest, and analyzes and identifies the moving object. Through this method, it is possible Perform subsequent analysis and processing on the image frame collection of the time period in which the target of interest is found by the event-based visual sensor. In the detection scene, the calculation amount of target recognition and analysis processing can save computing resources and improve computing efficiency.
  • the method for identifying a moving object may be applied to a corresponding identification device, and the identification device may be a computer, server, or other device with data processing functions or a data platform.
  • a device capable of data processing may be, for example, an embedded device.
  • the event flow data can be collected by the dynamic visual sensing module of the recognition device, or can be collected by a dynamic visual sensor independent of the recognition device and then transmitted to the recognition device; the image frame can be collected by the image acquisition module based on the color mode of the recognition device, or It can be collected by an image acquisition device based on the color mode independent of the recognition device and then transmitted to the recognition device.
  • the event-based vision sensor includes at least one of any of the following sensors: a dynamic vision sensor (Dynamic Vision Sensor, DVS) and an event camera (Event Camera) sensor.
  • a dynamic vision sensor Dynamic Vision Sensor, DVS
  • an event camera Event Camera
  • DVS can also be called a dynamic event sensor.
  • the sensor can include a pixel unit array composed of multiple pixel units, and each pixel unit can respond to and record the area where the light intensity changes rapidly when it senses a change in light intensity;
  • An event camera sensor can generate an event by detecting the brightness change of each pixel, which has the advantages of high dynamic range, low latency and no motion blur; for an event camera sensor, when detecting a moving object, whether it is Whether it's a low-light scene or a high-exposure scene, it can make a difference.
  • the event-based visual sensor may output event stream data by using an event-triggered processing mechanism.
  • the event stream data may include information such as time stamps of light intensity changes, light intensity values, and coordinate positions of triggered pixel units.
  • the DVS due to the event-triggered processing mechanism of the DVS, the DVS can detect high-speed objects moving at a higher rate, and has a larger dynamic range, so that it can be illuminated in the monitoring area. It can accurately sense and output scene changes in dark places (low-light scenes) or scenes with high-brightness light sources in the monitoring area (high-exposure scenes). Therefore, compared with the frame difference method, the event-based visual sensor is used for sensing
  • the detection of the target moving object of interest is beneficial to reduce the difficulty of finding the moving object, and is conducive to improving the efficiency and accuracy of finding the corresponding movement, so as to effectively find the moving object.
  • the event stream data is based on an event mechanism, using an event-based visual sensor, and for each pixel position captured, an event signal at the position is generated when the light intensity changes exceed a first preset threshold.
  • a first preset threshold For each captured pixel position, when the light intensity change exceeds the first preset threshold and the pixel position jumps from low brightness to high brightness, a "+1" event signal can be generated; when the light intensity changes When the first preset threshold is exceeded and the pixel point position jumps from high brightness to low brightness, a "-1" event signal may be generated; when the light intensity change does not exceed the first preset threshold, no event signal is sent.
  • the target of interest is detected according to the event stream data.
  • the image frames within the time period when the target of interest appears can be obtained to obtain a set of target image frames.
  • the continuous inter-frame difference method can be used to perform differential operations on two adjacent frames in the target image frame set to calculate whether the inter-frame pixel change exceeds a second preset threshold to determine whether there is a moving object.
  • the embodiments of the present disclosure may also refer to other methods for identifying moving objects other than the frame difference method, which will not be repeated here.
  • the method for identifying a moving object can detect an object of interest based on the acquired event stream data and obtain a detection result; the detection result is that the object of interest is detected In the case of , acquire a target image frame set associated with the target of interest; and then identify a moving object based on the target image frame set.
  • the target of interest is determined based on the event stream data
  • the target image frame set determined based on the target of interest is obtained and analyzed to identify the moving object, thereby saving computing resources, improving computing efficiency, and reducing the difficulty of finding moving objects, thus effectively Discover sporting goals.
  • step S102 may specifically include:
  • the event stream data or the pulse sequence corresponding to the event stream data may be input into the pre-trained neural network to detect the target of interest.
  • event flow data collected by various event-based visual sensors or the pulse sequence corresponding to the event flow data can be used to input the pre-trained neural network to detect the target of interest.
  • object of interest detection can be performed based on event stream data collected by image sensors such as DVS and event camera sensors.
  • DVS event stream data the event stream data collected by the DVS
  • DVS event stream data is used as an example to illustrate the specific implementation manner of detecting the object of interest.
  • DVS event stream data is used as an example to illustrate the specific implementation manner of detecting the object of interest.
  • the processing method for event stream data collected by other event-based visual sensors other than DVS is consistent with the processing method for DVS event stream data.
  • DVS event stream data can be fed into a pre-trained neural network.
  • the pre-trained neural network can be trained with the DVS event stream data with a sampling period of T as a training sample
  • the input information of the pre-trained neural network can include the DVS event stream data with a sampling period of T
  • the output information can include but not limited to sensory At least one item of the indication information of the target of interest, the confidence level of the target of interest, and the location information of the target of interest.
  • the target of interest indication information is used to indicate whether there is a target of interest;
  • the location information of the target of interest may include four dimensions, for example, the location information may be expressed as (x, y, w, h), where x , y respectively represent the horizontal and vertical coordinates of the center point of the target of interest, w, h represent the width and height of the target of interest respectively.
  • the DVS event stream data can be encoded first to obtain the pulse sequence corresponding to the DVS event data, and then the pulse sequence is input into the pre-trained spiking neural network.
  • the spiking neural network can represent spatiotemporal information, which can improve Accuracy of object of interest detection.
  • the pre-trained neural network is any one of the following: a spiking neural network; a neural network fused with a spiking neural network and an artificial neural network.
  • the image frame can be acquired by an image acquisition device based on color mode
  • the event stream data can be acquired by an event-based visual sensor, for example: DVS event stream data can be acquired by a dynamic visual sensor DVS The resulting event stream data.
  • step S103 in the case that the target of interest is detected from the collected event stream data, it may be determined that the number of time periods in which the target of interest appears is at least one.
  • each time period in which the object of interest appears can be determined by the initial moment when the object of interest is detected each time it appears and the moment when the object of interest is detected to disappear; as an example, the object of interest appears
  • the time period can also be determined by the initial time when the object of interest is detected for the first time and the time when the object of interest is last detected to disappear.
  • the time period from time t1 to time t2 can be regarded as the time period when an object of interest appears, and the time period from time t3 to time t4 can be regarded as another object of interest
  • the time period of appearance; alternatively, the time period from time t1 to time t4 may also be the time period when the target of interest appears.
  • the moment when the object of interest disappears after a predetermined duration threshold can be taken as the moment when the object of interest disappears; wherein, the value of the predetermined duration threshold is greater than or equal to zero, which can be set according to actual conditions.
  • the set of target image frames includes the first set of image frames; in step S103, when the detection result is that the target of interest is detected, the step of obtaining the set of target image frames may specifically include: S11, when the target is detected In the case of a target of interest, the image frames captured by the image capture device based on the color mode within the target time period are acquired to obtain a first set of image frames.
  • the number of image frames acquired in step S11 is greater than or equal to 1 and less than or equal to N, where N is the total number of image frames within the target time period.
  • the number of image frames within the acquisition target time period can be set to be greater than or equal to 1 and less than N, and the specific value of the set number can be determined according to actual needs.
  • the embodiment of the present disclosure does not specifically limit it.
  • the event flow data may be collected by a dynamic visual sensor independent of the recognition device, and the recognition device may obtain the event flow data from the dynamic vision sensor.
  • the image frame is collected by an image acquisition device based on a color mode independent of the identification device, and the identification device can obtain the image frame from the image acquisition device.
  • step S11 the step of acquiring the image frames within the target time period collected by the image acquisition device based on the color mode to obtain the first image frame set may specifically include the following steps.
  • S21 Send a first instruction to the image acquisition device based on the color mode at the first moment, the first moment is the start time of the target time period, and the first instruction is used to instruct the image acquisition device based on the color mode to collect and return a real-time image frame ;
  • S22 receiving the real-time image frame sent by the image acquisition device based on the color mode;
  • S23 sending a second instruction to the image acquisition device based on the color mode at the second moment, the second moment being the termination moment of the target time period, the second instruction It is used to instruct the image acquisition device based on the color mode to stop acquiring real-time image frames;
  • S24 obtain a first set of image frames according to the received real-time image frames from the start time to the end time.
  • the current moment when the target of interest is detected may be determined as the initial moment of the target time period, denoted as the first moment here.
  • the identification device may send a first instruction to the image acquisition device based on the color mode at the first moment to instruct the image acquisition device based on the color mode to start capturing real-time image frames and return the real-time image frames to the identification device. After the target of interest disappears, the end moment of the target time period can be determined, which is represented as the second moment here.
  • the identification device may send a second instruction to the image acquisition device based on the color mode at a second moment to instruct the image acquisition device based on the color mode to stop capturing real-time image frames.
  • the identification device may receive the real-time image frames between the first moment and the second moment.
  • the real-time image frame refers to the first image frame set obtained by collecting each frame of original image by the image acquisition device based on the color mode.
  • the first moment may be the moment when the object of interest is detected for the first time, and the determination of the second moment may be delayed by a predetermined duration threshold based on the moment when the object of interest is detected to disappear. Since the appearance of the object of interest may be discontinuous, the number of target time periods can be multiple.
  • the first moment is the moment when the object of interest appears, and the second moment is the moment when the object of interest disappears.
  • the first image The frame set includes multiple real-time image frames received within the target time period, which may be determined according to actual conditions, and is not limited in this embodiment of the present disclosure.
  • the image acquisition device based on the color mode may only need to acquire real-time image frames within the target time period. Energy consumption of the collection device.
  • the identification device can only store the first set of image frames, which saves memory space and improves memory utilization.
  • the set of target image frames includes a first set of image frames
  • the method for identifying a moving object further includes: receiving a real-time image frame sent by an image acquisition device based on a color mode; wherein, the image frame is an image based on a color mode In the process of collecting event stream data by the collection device, it is obtained through synchronous collection by the image collection device based on the color mode.
  • step S11 the step of acquiring the image frames within the target time period collected by the image acquisition device based on the color mode, and obtaining the first image frame set may specifically include: S31, from the received image frames Real-time image frames within the target time period are acquired to obtain a first set of image frames.
  • an image acquisition device based on color mode (such as a camera based on RGB color mode, referred to as RGB camera) can synchronously collect real-time image frames, and the DVS event stream
  • the data can be time aligned with the real-time image frame, for example, the DVS event stream data within a certain time interval can correspond to a real-time image frame.
  • the identification device can receive DVS event stream data and receive real-time image frames.
  • the identification device When detecting the existence of a target of interest according to the DVS event stream data, the identification device can correspondingly acquire real-time image frames aligned with its time based on the current DVS event stream data, thereby Acquiring at least part of the real-time image frames within the target time period to obtain a set of target image frames.
  • the target image frame set obtained in this implementation form can more accurately correspond to the real-time image frame when the target of interest appears, improving the accuracy of moving object recognition.
  • the target set of image frames includes a second set of image frames.
  • the step of acquiring the target image frame set may specifically include: S41, determine the position information of the object of interest according to the event stream data; S42, From the real-time image frames collected by the image acquisition device based on the color mode, the image frames corresponding to the position information are intercepted to obtain the second image frame set.
  • pixels with relative motion and light intensity changes exceeding the first preset threshold can be captured, and these pixels are usually distributed around the outline or boundary of the object, so the location of the target of interest , there is likely to be a moving object.
  • the recognition device may intercept the image corresponding to the position information of the target of interest from the real-time image frame collected by the image collection device based on the color mode to obtain a set of target image frames. That is to say, the image frames in the image frame set are in the real-time image frame collected by the image acquisition device based on the color mode, corresponding to the partial image of the position information, and the moving object is identified based on the target image frame set, which can further improve the performance of the moving object. detection efficiency and accuracy.
  • the identification device can receive the real-time image frame collected by the image acquisition device based on the color mode and the DVS event stream collected by the dynamic visual sensor, and when determining the position information of the target of interest according to the DVS event stream, it can determine The real-time image frame corresponding to the DVS event stream, for example, can time-align the DVS event stream and the real-time image frame, determine the real-time image frame corresponding to the detected DVS event stream of the target of interest, and intercept the real-time image frame The image frame corresponding to the location information. For example, spatial alignment may be performed on the DVS event stream and real-time image frames, and according to the location information, partial images corresponding to the location information in the real-time image frames may be intercepted to obtain image frames in the second image frame set.
  • the recognition device may control the image acquisition device based on the color mode to acquire real-time image frames within the time period of the target, for example, send the image acquisition device based on the color mode to capture or stop
  • the instruction of collecting real-time image frames receives the real-time image frames sent by the image acquisition device based on the color mode, and intercepts the image frames corresponding to the location information according to the received real-time image frames and location information. For example, spatial alignment may be performed on the DVS event stream and real-time image frames, and according to the location information, partial images corresponding to the location information in the real-time image frames may be intercepted to obtain image frames in the second image frame set.
  • the identification device when it detects the target of interest based on the DVS event stream, it can obtain the indication information of the target of interest, the location information of the target of interest, etc., for example, input the DVS event stream into the pre-trained neural network, and can output Indication information of the target of interest, location information of the target of interest, etc.
  • the identification device can determine the real-time image frames where the target of interest appears based on the indication information of the target of interest, and then intercept partial images containing the target of interest in these real-time image frames.
  • the position information is expressed as (x, y, w, h), wherein, x and y respectively represent the horizontal and vertical coordinates of the center point of the target of interest, and w and h represent the width and height of the target of interest respectively
  • the recognition device can intercept the above-mentioned real-time image frame containing sense A rectangular area of the target of interest; or in the case where the position information is used to indicate the contour of the target of interest, the image may be intercepted based on the contour of the target of interest to obtain the image frames in the second set of image frames.
  • the above steps S103 and S104 may be executed synchronously. That is to say, at the moment when the target image frame is acquired in step S103, the step of identifying the moving object in step S104 may be started synchronously.
  • an image acquisition device based on a color mode can be used to start capturing images synchronously, and synchronously
  • the collected image frames are used for moving object recognition.
  • the embodiment of the present disclosure can perform the synchronization processing of the recognition calculation while the image is captured, which is beneficial to improve the image quality.
  • Recognition computing efficiency which in turn can improve the recognition efficiency of moving objects.
  • the target image frame set includes image frames collected by an image acquisition device based on a color mode; for an event-based visual sensor and an image acquisition device based on a color mode, both are located in the same image data acquisition device, and The distance between the cameras included in the two is less than a preset distance threshold.
  • the event-based visual sensor and the image acquisition device based on the color mode can be integrated into a fixed device, and in the fixed device, the event-based visual sensor and the image acquisition device based on the color mode Close enough to reduce the chance of angular parallax.
  • FIG. 2 shows a structural diagram of a network system in an embodiment of the present disclosure.
  • the network system includes a data collection module 210 , a target location module 220 and a recognition analysis module 230 .
  • the data acquisition module 210 includes an image acquisition device based on a color mode and an event-based visual sensor, such as an RGB image acquisition device and a dynamic visual sensor DVS shown in FIG.
  • the dynamic vision sensor DVS can transmit a DVS event stream to the target location module 220 .
  • the object location module 220 can detect an object of interest according to the DVS event stream, and determine a target image frame set based on the object of interest.
  • the identification and analysis module 230 can acquire the target image frame set transmitted by the target positioning module 220, and perform identification and analysis of the moving object on the image frames in the target image frame set, and output the result.
  • a set of target image frames determined based on the object of interest is acquired and analyzed to identify the moving object, thereby reducing the consumption of computing resources to The probability on the image frame without actual content saves computing resources, improves computing efficiency, and can reduce the difficulty of finding moving objects, so as to effectively find moving objects.
  • Fig. 3 is a schematic structural diagram of an apparatus for identifying a moving object provided by an embodiment of the present disclosure.
  • the apparatus 300 for identifying running objects includes the following modules.
  • the first acquiring module 301 is configured to acquire event flow data collected by an event-based visual sensor.
  • the detection module 302 is configured to detect the target of interest according to the collected event stream data, and obtain a detection result.
  • the second acquiring module 303 is used to acquire a target image frame set when the detection result is that the target of interest is detected; wherein, the target image frame set includes image frames within the target time period, and the target time period includes the appearance of the target of interest time period;
  • the recognition module 304 is configured to recognize a moving object according to the set of target image frames.
  • the detection module 302 is specifically configured to: input the event flow data or the pulse sequence corresponding to the event flow data into the pre-trained neural network, and detect the object of interest through the pre-trained neural network to determine whether the object of interest is detected. target, to obtain the detection result of the target of interest; wherein, the pulse sequence corresponding to the DVS event stream is obtained based on encoding the DVS event stream.
  • the pre-trained neural network includes any one of the following: a spiking neural network; a neural network fused with a spiking neural network and an artificial neural network.
  • the image frame is collected by a color mode RGB image collection device, and the DVS event stream is collected by a dynamic visual sensor DVS.
  • the set of target image frames includes the first set of image frames
  • the second acquisition module 303 is specifically configured to: in the case of detecting the target of interest, acquire the images captured by the image acquisition device based on the color mode within the target time period image frames to obtain the first set of image frames.
  • the second acquisition module 303 includes: a first sending unit, configured to send a first instruction to the image acquisition device based on the color mode at a first moment, where the first moment is the start moment of the target time period, and the first instruction It is used to instruct the image acquisition device based on the color mode to collect and return the real-time image frame;
  • the receiving unit is used to receive the real-time image frame sent by the image acquisition device based on the color mode;
  • the second sending unit is used to send the image frame based on the color at the second moment
  • the image acquisition device of the mode sends a second instruction, the second moment is the end moment of the target time period, and the second instruction is used to instruct the image acquisition device based on the color mode to stop collecting real-time image frames;
  • the real-time image frames are obtained to obtain the first image frame set.
  • the set of target image frames includes the first set of image frames
  • the identification device 300 of the running object further includes: a receiving module, configured to receive the real-time image frames sent by the image acquisition device based on the color mode; wherein, the image frames are based on In the process of collecting the event stream data by the image acquisition device in the color mode, it is obtained through synchronous acquisition by the image acquisition device based on the color mode; the second acquisition module 303 is specifically used to: acquire the image frames within the target time period from the received image frames, Obtain the first set of image frames.
  • the set of target image frames includes a second set of image frames
  • the second acquisition module 303 includes: a determination unit configured to determine the position information of the target of interest according to the event stream data; From the real-time image frames, the image frames corresponding to the location information are intercepted to obtain the second image frame set.
  • the set of target image frames includes image frames collected by an image acquisition device based on a color mode; for an event-based visual sensor and an image acquisition device based on a color mode, both are located in the same image data acquisition device, and both The distance between included cameras is less than a preset distance threshold.
  • Fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • an embodiment of the present disclosure also provides an electronic device, which includes: at least one processor 401, at least one memory 402, and one or more I/O interfaces 403 connected between the processor 501 and Between the memory 502; wherein, the memory 502 stores one or more computer programs that can be executed by at least one processor 501, and one or more computer programs are executed by at least one processor 501, so that at least one processor 501 can execute The above-mentioned identification method of the moving object.
  • the electronic devices in the embodiments of the present disclosure include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • An embodiment of the present disclosure also provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, each process of the above embodiment of the method for identifying an operating object is implemented. For the sake of brevity, here No longer.
  • a readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.
  • ROM computer read-only memory
  • RAM random access memory
  • magnetic disk or an optical disk and the like.
  • An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are run in a processor of an electronic device , the processor in the electronic device executes the above-mentioned method for identifying a moving object.
  • the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof.
  • the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute.
  • Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit .
  • Such software may be distributed on computer readable storage media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • Computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable program instructions, data structures, program modules, or other data. volatile, removable and non-removable media.
  • Computer storage media include, but are not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable Compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disk storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or any other device that can be used to store desired information and can be accessed by a computer any other medium.
  • communication media typically embodies computer-readable program instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery medium.
  • Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as “C” or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA)
  • FPGA field programmable gate array
  • PDA programmable logic array
  • the computer program products described here can be specifically realized by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
  • These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供一种运动对象的识别方法、装置、电子设备及可读存储介质，其中，方法包括：获取基于事件的视觉传感器采集的事件流数据；根据采集的事件流数据，进行感兴趣目标检测，得到检测结果；在检测结果为检测到感兴趣目标的情况下，获取目标图像帧集合；其中，目标图像帧集合包括目标时间段内的图像帧，目标时间段包括感兴趣目标出现的时间段；根据目标图像帧集合，识别运动对象。根据该方法，可以节约计算资源，提高计算效率。

Description

运动对象的识别方法、装置、电子设备及可读存储介质 技术领域
本公开实施例涉及图像检测技术领域，尤其涉及一种运动对象的识别方法、装置、电子设备及可读存储介质。
背景技术
目前，出于安防管理的需要，监控设备布满了街道、社区、楼宇等各种公共场合。当监控设备采集的视频中出现运动对象的时间相对稀疏时，在对监控设备采集的视频进行分析时，通常包括大量无实际内容的图像帧，从而需要耗费大量不必要的计算资源，计算效率也较低。
发明内容
本公开实施例提供了一种运动对象的识别方法、装置、电子设备及可读存储介质，以节约对运动对象进行识别的计算资源，提高计算效率。
第一方面，本公开实施例提供了一种运动对象的识别方法，包括：获取基于事件的视觉传感器采集的事件流数据；根据采集的事件流数据，进行感兴趣目标检测，得到检测结果；在检测结果为检测到感兴趣目标的情况下，获取目标图像帧集合；其中，目标图像帧集合包括目标时间段内的图像帧，目标时间段包括感兴趣目标出现的时间段；根据目标图像帧集合，识别运动对象。
第二方面，本公开实施例提供了一种运动对象的识别装置，包括：第一获取模块，用于获取基于事件的视觉传感器采集的事件流数据；检测模块，用于根据采集的事件流数据，进行感兴趣目标检测，得到检测结果；第二获取模块，用于在检测结果为检测到感兴趣目标的情况下，获取目标图像帧集合；其中，目标图像帧集合包括目标时间段内的图像帧，目标时间段包括感兴趣目标出现的时间段；识别模块，用于根据目标图像帧集合，识别运动对象。
第三方面，本公开实施例还提供了一种电子设备，其特征在于，包括处理器，存储器及存储在存储器上并可在处理器上运行的程序或指令，程序或指令被处理器执行时实现如第一方面的方法的步骤。
第四方面，本公开实施例还提供了一种可读存储介质，可读存储介质上存储程序或指令，程序或指令被处理器执行时实现如第一方面的方法的步骤。
第五方面，本公开实施例还提供一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，其中，当计算机可读代码在电子设备的处理器中运行时，电子设备中的处理器执行用于实现如第一方面的方法的步骤。
本公开实施例中，获取动态视觉传感器DVS事件流；根据DVS事件流，进行感兴趣目标检测；在检测到感兴趣目标的情况下，获取目标图像帧集合；目标图像帧集合包 括目标时间段内的图像帧，目标时间段包括感兴趣目标出现的时间段；根据目标图像帧集合，识别运动对象。在基于DVS事件流检测到感兴趣目标的情况下，确定存在感兴趣目标，获取基于感兴趣目标确定的目标图像帧集合并进行分析识别运动对象，从而节约计算资源，提高计算效率。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
图1是本公开实施例提供的运动对象的识别方法的流程图；
图2是本公开实施例的网络系统的结构图；
图3是本公开实施例提供的运动对象的识别装置的结构框图；
图4是本公开实施例提供的电子设备的结构框图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。另外还需要说明的是，为了便于描述，附图中仅示出了与本公开相关的部分而非全部结构。
在本公开的描述中，需要理解的是，术语“第一”、“第二”仅由于描述目的，且不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。因此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者多个该特征。本发明的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在视频监控领域的一些场景中，为了确保监控内容的实时、连贯，会将全部监控到的图像画面信息送给内容监控模块进行分析。这种方式下，无论画面中是否存在有意义的目标，后续分析模块都会进行计算操作。然而，对实际监控录像内容分析可知，视频场景中出现运动对象(监控对象、车等)的时间点相对稀疏，为减少分析模块将大量计算资源消耗到无实际内容的画面帧上，人们考虑只对有运动物体出现时的画面进行分析。
示例性地，无实际内容的画面帧中的画面可以是如下情形中的任一种：画面全黑或全蓝、画面不清晰(画面的清晰度小于预设清晰度阈值)、或画面具有较大面积遮挡(被遮挡画面在视频画面整体面积中占比超过预定比例阈值)。
在一些场景中，可以通过帧差法(也可称为是帧间差分法)来进行运动对象的识别；具体地，帧差法是将色彩模式画面的前后帧对应的像素点进行差运算以得到差值，当该差值的绝对值超过特定阈值，即可以判定画面中是否有运动对象(或移动物体)，以对运动对象进行识别。
在本公开实施例中，色彩模式是工业界的一种颜色标准。作为示例，色彩模式例如可以包括但不限于是：红绿蓝(RED Green Blue，RGB)色彩模式、标准RGB(Standard Red Green Blue，sRGB)色彩模式、Adobe RGB中的一种。其中，RGB色彩模式通过对红色(RED，R)、绿色(Green，G)、蓝色(Blue，B)三个颜色通道的变化以及互相之间的叠加得到多种颜色的；sRGB色彩模式可以基于独立的色彩坐标，使色彩在不同的设备使用传输中对应于同一的色彩坐标体系；Adobe RGB色彩模式与sRGB色彩模式相比具有更大的色域空间。
在一些实施例中，基于色彩模式的图像采集装置例如可以包括如下设备中的至少一种：基于色彩模式的摄像头感光元件、基于色彩模式的相机镜头、基于色彩模式的高速摄像机。
在实际应用场景中，色彩模式例如可以包括RGB色彩模式、sRGB色彩模式和Adobe RGB色彩模式；应理解，色彩模式还可以包括更多类型，具体可以根据图像采集需求进行选择，本公开实施例不做具体限定。
在一些场景中，以色彩模式为RGB色彩模式为例，可以将基于RGB色彩模式简称为是RGB或RGB域；以及，可以将基于色彩模式的图像采集装置采集的图像帧和视频信息，简称为是RGB图像帧和RGB域视频信息。
在通过帧差法识别运动对象时，对于监控区域光照较暗的地方出现的深色的运动目标，受限于基于RGB色彩模式的摄像头感光元件的动态范围，连续图像在运动目标的运动区域的画面像素变化极小，此时使用帧间差方法发现运动对象的难度较大，不利于有效发现运动对象。
对于监控区域中存在高亮度光源的场景，如监控画面中某处有高亮度照明灯发出的光源直射到基于RGB色彩模式的相机镜头，拍摄场景在高亮光源附近会产生一片白色模糊现象，类似人眼看到的炫光模糊，导致在该模糊区域中使用帧差法发现运动对象的难度较大，不利于有效发现运动对象。
第一方面，本公开实施例提供了一种运动对象的识别方法。
本公开实施例的运动对象的识别方法可由相应的运动对象的识别装置执行，该装置可采用软件和/或硬件的方式实现，并一般可集成于电子设备中。
图1是本公开实施例提供的运动对象的识别方法的流程图。参见图1，本公开实施例的运动对象的识别方法包括：
S101、获取基于事件的视觉传感器采集的事件流数据；
S102、根据采集的事件流数据，进行感兴趣目标检测，得到检测结果；
S103、在检测结果为检测到感兴趣目标的情况下，获取目标图像帧集合；其中，目标图像帧集合包括目标时间段内的图像帧，目标时间段包括感兴趣目标出现的时间段；
S104、根据目标图像帧集合，识别运动对象。
本公开实施例的运动对象的识别方法，在基于根据采集的事件流数据确定存在感兴趣目标时，获取基于感兴趣目标确定的目标图像帧集合，并进行分析识别运动对象，通 过该方法，可以对基于事件的视觉传感器发现的存在感兴趣目标的时间段的图像帧集合进行后续分析处理，不必将所有时间段的图像帧全部进行后续分析处理，从而可以大幅降低对运动对象(例如移动目标)进行检测的场景中进行目标识别和分析处理的计算量，进而可以节约计算资源，提高计算效率。
在一些实施例中，该运动对象的识别方法可以应用于相应的识别装置，识别装置可以为计算机、服务器等具备数据处理功能的设备或者数据平台。具备数据处理功能的设备例如可以是嵌入式设备。事件流数据可以由识别装置的动态视觉传感模块采集，也可以由独立于识别装置的动态视觉传感器采集后传输至识别装置；图像帧可以由识别装置的基于色彩模式的图像采集模块采集，也可以由独立于识别装置的基于色彩模式的图像采集装置采集后传输至识别装置。
在一些实施例中，基于事件的视觉传感器包括如下任一传感器中的至少一者：动态视觉传感器(Dynamic Vision Sensor，DVS)和事件性相机(Event Camera)传感器。
其中，DVS也可以称为动态事件传感器，传感器内部可以包括多个像素单元构成的像素单元阵列，其中每个像素单元在感应到光强变化时，可以响应并记录光强快速变化的区域；事件性相机传感器可以通过检测每个像素的亮度变化来生成一个事件，具有高动态范围、低延时和无运动模糊的优势；对于事件性相机传感器而言，在检测正在运动的物体时，无论是低光照场景还是高曝光场景，均可以发挥作用。
在本公开实施例中，基于事件的视觉传感器可以采用事件触发的处理机制输出事件流数据。事件流数据可以包括：光强变化的时间戳、光强值以及被触发像素单元的坐标位置等信息。
在本公开实施例的运动对象的识别方法中，由于DVS的事件触发的处理机制，使得DVS可以侦测到更高速率运动的高速物体，具有更大的动态范围，从而可以在监控区域光照较暗的地方(低光照场景)或者监控区域中存在高亮度光源的场景(高曝光场景)下都能准确感应并输出场景变化，因此，相较于帧差法，采用基于事件的视觉传感器进行感兴趣的目标运动对象的检测，有利于降低发现运动对象的难度，并有利于提高发现运动对应的效率和准确性，从而有效发现运动目标。
在一些实施例中，事件流数据是基于事件机制，采用基于事件的视觉传感器，对于捕获的每个像素点位置，当光强度变化超过第一预设阈值时生成该位置的事件信号。具体的，对于捕获的每个像素点位置，当光强度变化超过第一预设阈值，且该像素点位置从低亮度跳变至高亮度时，可以生成“+1”事件信号；当光强度变化超过第一预设阈值，且该像素点位置从高亮度跳变至低亮度时可以生成“-1”事件信号；当光强度变化不超过第一预设阈值时不发送事件信号。
在本公开实施例中，根据事件流数据进行感兴趣目标检测，在检测到感兴趣目标的情况下，可以获取感兴趣目标出现的时间段内的图像帧，得到目标图像帧集合，基于目标图像帧集合中的图像帧，可以通过连续帧间差分法，对目标图像帧集合中的相邻两帧对差分运算，以计算帧间像素变化是否超过第二预设阈值来确定是否存在运动对象。需 要说明的是，本公开实施例也可以参考除帧差法之外的其他识别运动对象的方法，在此不再赘述。
以基于事件的视觉传感器为DVS为例，本公开实施例提供的运动对象的识别方法，可以根据获取的事件流数据，进行感兴趣目标检测，得到检测结果；在检测结果为检测到感兴趣目标的情况下，获取与感兴趣目标相关联的目标图像帧集合；再根据目标图像帧集合，识别运动对象。在基于事件流数据确定存在感兴趣目标时，获取基于感兴趣目标确定的目标图像帧集合并进行分析识别运动对象，从而节约计算资源，提高计算效率，并可以降低发现运动对象的难度，从而有效发现运动目标。
在本公开实施例的可选实施方式中，步骤S102具体可以包括：
将事件流数据或者事件流数据对应的脉冲序列输入预训练的神经网络中，通过预训练的神经网络进行感兴趣目标检测，以确定是否检测到感兴趣目标，得到感兴趣目标的检测结果；其中，事件流数据对应的脉冲序列基于对事件流数据编码得到。
本公开实施例中，可以将事件流数据或者事件流数据对应的脉冲序列输入到预训练的神经网络中，以检测感兴趣目标。
需要说明的是，可以使用各种基于事件的视觉传感器采集的事件流数据或者事件流数据对应的脉冲序列来完成输入预训练的神经网络，以进行感兴趣目标检测。例如，可以基于DVS、事件性相机传感器等图像传感器所采集的事件流数据来进行感兴趣目标检测。为了简化描述起见，本文下述的多个实施例以DVS采集的事件流数据(简称DVS事件流数据)为例来阐述进行感兴趣目标检测的具体实施方式。但该描述并不能被解读为限制本方案的范围或实施可能性，对DVS以外的其他基于事件的视觉传感器所采集的事件流数据的处理方法与对DVS事件流数据的处理方法保持一致。
在一种实现形式中，可以将DVS事件流数据输入到预训练的神经网络中。预训练的神经网络可以以采样周期为T的DVS事件流数据作为训练样本进行训练，预训练的神经网络的输入信息可以包括采样周期为T的DVS事件流数据，输出信息可以包括但不限于感兴趣目标指示信息、存在感兴趣目标的置信度、感兴趣目标的位置信息中的至少一项。其中，感兴趣目标指示信息用于指示是否存在感兴趣目标；感兴趣目标的位置信息可以包括四个维度，示例性的，位置信息可以表示为(x，y，w，h)，其中，x、y分别表示感兴趣目标的中心点的横纵坐标，w，h分别表示感兴趣目标的宽和高。
在另一种实现形式中，可以先将DVS事件流数据进行编码得到DVS事件数据对应的脉冲序列，再将脉冲序列输入到预训练的脉冲神经网络中，脉冲神经网络可以表征时空信息，能够提高感兴趣目标检测的准确性。
在一些实施例中，预训练的神经网络为以下任意一项：脉冲神经网络；脉冲神经网络和人工神经网络融合的神经网络。
在本公开实施例的可选实施方式中，图像帧可以由基于色彩模式的图像采集装置采集得到，事件流数据由基于事件的视觉传感器采集得到，例如：DVS事件流数据由动态视觉传感器DVS采集得到的事件流数据。
在步骤S103，在从采集到的事件流数据中检测到感兴趣目标的情况下，可以确定感兴趣目标出现的时间段的数量为至少一个。
作为示例，感兴趣目标出现的每个时间段，可以由每次检测到该感兴趣目标出现的起始时刻和该次检测到该感兴趣目标消失的时刻来确定；作为示例，感兴趣目标出现的时间段，也可以由检测到该感兴趣目标首次出现的起始时刻和最后一次检测到该感兴趣目标消失的时刻来确定。
作为具体示例，若在t1时刻检测到感兴趣目标出现，在t1时刻之后的t2时刻检测到该感兴趣目标消失；在t2时刻之后的t3时刻再次检测到该感兴趣目标出现，在t3时刻之后的t4时刻检测到该感兴趣目标消失，则：可以将从t1时刻到t2时刻的时间段作为一个感兴趣目标出现的时间段，将从t3时刻到t4时刻的时间段作为另一个感兴趣目标出现的时间段；或者，也可以将从t1时刻到t4时刻的时间段为感兴趣目标出现的时间段。
在一些场景中，可以将感兴趣目标消失时刻之后预定时长阈值的时刻，作为感兴趣目标消失的时刻；其中，预定时长阈值的取值大于或等于零，具体可以根据实际情况进行设置。
在一些实施例中，目标图像帧集合包括第一图像帧集合；步骤S103中在检测结果为检测到感兴趣目标的情况下，获取目标图像帧集合的步骤，具体可以包括：S11，在检测到感兴趣目标的情况下，获取通过基于色彩模式的图像采集装置采集的在目标时间段内的图像帧，得到第一图像帧集合。
在一些实施例中，步骤S11中获取的图像帧的数量大于或等于1且小于或等于N，N为目标时间段内图像帧的总数量。
例如，在监控场景例如违章车辆拍摄场景中，通常只需要获取若干(数量大于或等于1且小于N)图像帧，就可以准确率较高的运动对象识别结果。为了降低基于色彩模式的图像采集装置的能耗和节约内存空间，可以设置获取目标时间段内的图像帧的数量为大于或等于1且小于N，设置的数量的具体取值可以根据实际需要来确定，本公开实施例不做具体限定。
在本公开实施例中，事件流数据可以由独立于识别装置的动态视觉传感器采集，识别装置可以从动态视觉传感器获取事件流数据。图像帧由独立于识别装置的基于色彩模式的图像采集装置采集，识别装置可以从通过图像采集装置获取图像帧。
在一些实施例中，步骤S11中获取通过基于色彩模式的图像采集装置采集的在目标时间段内的图像帧，得到第一图像帧集合的步骤，具体可以包括如下步骤。
S21，在第一时刻向基于色彩模式的图像采集装置发送第一指令，第一时刻为目标时间段的起始时刻，第一指令用于指示基于色彩模式的图像采集装置采集并返回实时图像帧；S22，接收基于色彩模式的图像采集装置发送的实时图像帧；S23，在第二时刻向基于色彩模式的图像采集装置发送第二指令，第二时刻为目标时间段的终止时刻，第二指令用于指示基于色彩模式的图像采集装置停止采集实时图像帧；S24，根据接收的从 起始时刻到终止时刻的实时图像帧，得到第一图像帧集合。
在本公开实施例中，在检测到感兴趣目标时，可以将检测到感兴趣目标的当前时刻确定为目标时间段的初始时刻，在此表示为第一时刻。识别装置可以在第一时刻向基于色彩模式的图像采集装置发送第一指令，以指示基于色彩模式的图像采集装置开始采集实时图像帧，并将实时图像帧返回识别装置。当感兴趣目标消失后，可以确定目标时间段的终止时刻，在此表示为第二时刻。识别装置可以在第二时刻向基于色彩模式的图像采集装置发送第二指令，以指示基于色彩模式的图像采集装置停止采集实时图像帧。识别装置可以通过接收第一时刻至第二时刻之间的实时图像帧，实时图像帧是指基于色彩模式的图像采集装置采集每一帧原始图像得到第一图像帧集合。
具体实现时，第一时刻可以为初次检测到存在感兴趣目标的时刻，第二时刻的确定可以基于检测到感兴趣目标消失的时刻延后预定时长阈值。由于感兴趣目标的出现可能不连续，目标时间段的数量可以为多个，第一时刻为每次感兴趣目标出现时的时刻，第二时刻为每次感兴趣目标消失的时刻，第一图像帧集合包括多个目标时间段内接收到的实时图像帧，具体可根据实际情况决定，本公开实施例在此不作限定。
本实现形式中，基于色彩模式的图像采集装置可以仅需采集目标时间段内的实时图像帧，在其他时间，基于色彩模式的图像采集可以不采集实时图像帧，从而可以降低基于色彩模式的图像采集装置的能耗。同时，识别装置可以仅存储第一图像帧集合，节约内存空间，提高了内存利用率。
在一些实施例中，目标图像帧集合包括第一图像帧集合，运动对象的识别方法还包括：接收基于色彩模式的图像采集装置发送的实时图像帧；其中，图像帧是在基于色彩模式的图像采集装置采集事件流数据的过程中，通过基于色彩模式的图像采集装置同步采集得到。
在该实施例中，步骤S11中获取通过基于色彩模式的图像采集装置采集的在目标时间段内的图像帧，得到第一图像帧集合的步骤，具体可以包括：S31，从接收的图像帧中获取目标时间段内的实时图像帧，得到第一图像帧集合。
在该实施例中，在基于事件的视觉传感器采集事件流数据的过程中，基于色彩模式的图像采集装置(例如基于RGB色彩模式的相机，简称RGB相机)可以同步采集实时图像帧，DVS事件流数据可以与实时图像帧在时间上对齐，例如，一定时间区间内的DVS事件流数据可以对应一实时图像帧。识别装置可以接收DVS事件流数据以及接收实时图像帧，当根据DVS事件流数据检测到存在感兴趣目标时，识别装置可以基于当前的DVS事件流数据，对应获取与其时间对齐的实时图像帧，从而获取目标时间段内的至少部分实时图像帧，得到目标图像帧集合。本实现形式获取的目标图像帧集合可以更加准确地对应感兴趣目标出现时的实时图像帧，提高了运动对象识别的准确性。
在一些实施例中，目标图像帧集合包括第二图像帧集合。
在该实施例中，步骤S103中的在检测到存在感兴趣目标的情况下，获取目标图像帧集合的步骤，具体可以包括：S41，根据事件流数据，确定感兴趣目标的位置信息； S42，在基于色彩模式的图像采集装置采集的实时图像帧中，截取位置信息对应的图像帧，得到第二图像帧集合。
本公开实施例中，基于DVS事件流可以捕获存在相对运动且光强度变化超过第一预设阈值的像素点，而这些像素点通常分布在对象的轮廓或者边界周围，因此感兴趣目标所在的位置，很有可能存在运动对象。识别装置可以在基于色彩模式的图像采集装置采集的实时图像帧中，截取感兴趣目标的位置信息对应的图像，得到目标图像帧集合。也就是说，图像帧集合中的图像帧在基于色彩模式的图像采集装置采集的实时图像帧中，对应于该位置信息的局部图像，基于该目标图像帧集合识别运动对象，可以进一步提高运动对象检测的效率和准确性。
在本公开实施例中，识别装置可以接收基于色彩模式的图像采集装置采集的实时图像帧以及动态视觉传感器采集的DVS事件流，在根据DVS事件流，确定感兴趣目标的位置信息时，可以确定与DVS事件流对应的实时图像帧，例如，可以对DVS事件流以及实时图像帧进行时间对齐，确定与检测到存在感兴趣目标DVS事件流对应的实时图像帧，并在该实时图像帧上截取位置信息对应的图像帧。例如，可以对DVS事件流以及实时图像帧进行空间对齐，并根据位置信息，截取实时图像帧中位置信息对应的局部图像，得到第二图像帧集合中的图像帧。
可选的，识别装置可以在检测到存在感兴趣目标的情况下，控制基于色彩模式的图像采集装置采集目标时间段内的实时图像帧，例如，向基于色彩模式的图像采集装置发送采集或者停止采集实时图像帧的指令，接收基于色彩模式的图像采集装置发送的实时图像帧，并根据接收到的实时图像帧以及位置信息，截取位置信息对应的图像帧。例如，可以对DVS事件流以及实时图像帧进行空间对齐，并根据位置信息，截取实时图像帧中位置信息对应的局部图像，得到第二图像帧集合中的图像帧。
可选的，识别装置在基于DVS事件流进行感兴趣目标检测时，可以得到感兴趣目标的指示信息、感兴趣目标的位置信息等，例如，将DVS事件流输入预训练的神经网络，可以输出感兴趣目标的指示信息、感兴趣目标的位置信息等。识别装置可以基于感兴趣目标的指示信息，确定感兴趣目标出现的实时图像帧，再截取这些实时图像帧中包含感兴趣目标的局部图像，示例性的，在位置信息表示为(x，y，w，h)，其中，x、y分别表示感兴趣目标的中心点的横纵坐标，w，h分别表示感兴趣目标的宽和高的情况下，识别装置可以截取上述实时图像帧中包含感兴趣目标的矩形区域；或者在位置信息用于指示感兴趣目标的轮廓的情况下，可以基于感兴趣目标的轮廓截取图像，得到第二图像帧集合中的图像帧。
在一些实施例中，上述步骤S103和S104可以同步执行。也就是说，在步骤S103中获取到目标图像帧的时刻，可以同步开始执行步骤S104中对运动对象进行识别的步骤。
在本公开实施例中，在根据事件流数据，确定存在感兴趣目标例如运动对象等动态目标的开始时刻，可以使用基于色彩模式的图像采集装置(例如RGB相机)同步开始 采集图像，并同步对采集的图像帧进行运动对象识别。相较于在获取目标图像帧集合后，才开始执行根据目标图像帧集合识别运动对象的识别计算，本公开实施例可以在采集到图像的同时，进行识别计算的同步处理，从而有利于提高图像识别计算效率，进而可以提高运动对象的识别效率。
在一些实施例中，目标图像帧集合中包括基于色彩模式的图像采集装置采集的图像帧；对于基于事件的视觉传感器和基于色彩模式的图像采集装置，二者位于同一图像数据采集设备中，且二者所包含摄像头之间的距离小于预设距离阈值。
在本公开实施例中，可以将基于事件的视觉传感器和基于色彩模式的图像采集装置集成到一个固定的设备中，在该固定的设备中，基于事件的视觉传感器和基于色彩模式的图像采集装置足够接近，以减少出现角度视差的出现概率。
在不违背逻辑的情况下，本公开不同实施例之间可以相互结合，不同实施例描述有所侧重，未侧重描述的部分可参见其他实施例的记载。
图2示出本公开实施例的网络系统的结构图。在图2中，该网络系统包括数据采集模块210、目标定位模块220和识别分析模块230。
其中，数据采集模块210包括基于色彩模式的图像采集装置和基于事件的视觉传感器，例如图2中示出的RGB图像采集装置和动态视觉传感器DVS，RGB图像采集装置可以向目标定位模块220传输采集的图像帧，动态视觉传感器DVS可以向目标定位模块220传输DVS事件流。目标定位模块220可以根据DVS事件流检测感兴趣目标，并基于感兴趣目标确定目标图像帧集合。识别分析模块230可以获取目标定位模块220传输的目标图像帧集合，并对目标图像帧集合中的图像帧进行运动对象的识别和分析，并输出结果。
根据本公开实施例提供的运动对象的识别方法，在基于DVS事件流确定存在感兴趣目标时，获取基于感兴趣目标确定的目标图像帧集合并进行分析识别运动对象，从而降低将计算资源消耗到无实际内容的图像帧上的概率，节约了计算资源，提高了计算效率，并可以降低发现运动对象的难度，从而有效发现运动目标。
图3为本公开实施例提供的运动对象的识别装置的结构示意图。
如图3所示，运行对象的识别装置300包括如下模块。
第一获取模块301，用于获取基于事件的视觉传感器采集的事件流数据。
检测模块302，用于根据采集的事件流数据，进行感兴趣目标检测，得到检测结果。
第二获取模块303，用于在检测结果为检测到感兴趣目标的情况下，获取目标图像帧集合；其中，目标图像帧集合包括目标时间段内的图像帧，目标时间段包括感兴趣目标出现的时间段；
识别模块304，用于根据目标图像帧集合，识别运动对象。
可选的，检测模块302具体用于：将事件流数据或者事件流数据对应的脉冲序列输入预训练的神经网络中，通过预训练的神经网络进行感兴趣目标检测，以确定是否检测到感兴趣目标，得到感兴趣目标的检测结果；其中，DVS事件流对应的脉冲序列基于对 DVS事件流编码得到。
可选的，预训练的神经网络包括以下任意一项：脉冲神经网络；脉冲神经网络和人工神经网络融合的神经网络。
可选的，图像帧由色彩模式RGB图像采集装置采集，DVS事件流由动态视觉传感器DVS采集。
可选的，目标图像帧集合包括第一图像帧集合，第二获取模块303具体用于：在检测到感兴趣目标的情况下，获取通过基于色彩模式的图像采集装置采集的在目标时间段内的图像帧，得到第一图像帧集合。
可选的，第二获取模块303包括：第一发送单元，用于在第一时刻向基于色彩模式的图像采集装置发送第一指令，第一时刻为目标时间段的起始时刻，第一指令用于指示基于色彩模式的图像采集装置采集并返回实时图像帧；接收单元，用于接收基于色彩模式的图像采集装置发送的实时图像帧；第二发送单元，用于在第二时刻向基于色彩模式的图像采集装置发送第二指令，第二时刻为目标时间段的终止时刻，第二指令用于指示基于色彩模式的图像采集装置停止采集实时图像帧；根据接收的从起始时刻到终止时刻的实时图像帧，得到第一图像帧集合。
可选的，目标图像帧集合包括第一图像帧集合，运行对象的识别装置300还包括：接收模块，用于接收基于色彩模式的图像采集装置发送的实时图像帧；其中，图像帧是在基于色彩模式的图像采集装置采集事件流数据的过程中，通过基于色彩模式的图像采集装置同步采集得到；第二获取模块303具体用于：从接收的图像帧中获取目标时间段内的图像帧，得到第一图像帧集合。
可选的，目标图像帧集合包括第二图像帧集合，第二获取模块303包括：确定单元，用于根据事件流数据，确定感兴趣目标的位置信息；在基于色彩模式的图像采集装置采集的实时图像帧中，截取位置信息对应的图像帧，得到第二图像帧集合。
可选的，目标图像帧集合中包括基于色彩模式的图像采集装置采集的图像帧；对于基于事件的视觉传感器和基于色彩模式的图像采集装置，二者位于同一图像数据采集设备中，且二者所包含摄像头之间的距离小于预设距离阈值。
本公开实施例提供的运动对象的识别装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现和技术效果可参照上文方法实施例的描述，为了简洁，这里不再赘述。
图4示出本公开实施例的电子设备的结构示意图。
如图4所示，本公开实施例还提供一种电子设备，该电子设备包括：至少一个处理器401，至少一个存储器402，以及一个或多个I/O接口403，连接在处理器501与存储器502之间；其中，存储器502存储有可被至少一个处理器501执行的一个或多个计算机程序，一个或多个计算机程序被至少一个处理器501执行，以使至少一个处理器501能够执行上述的运动对象的识别方法。
在本公开实施例中，该计算机程序被处理器401执行时实现运行对象的识别方法实 施例的各个过程，为了简洁，这里不再赘述。
需要注意的是，本公开实施例中的电子设备包括上述的移动电子设备和非移动电子设备。
本公开实施例还提供一种可读存储介质，可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现上述运行对象的识别方法实施例的各个过程，为了简洁，这里不再赘述。
其中，处理器为上述实施例中的电子设备中的处理器。可读存储介质，包括计算机可读存储介质，如计算机只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等。
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当计算机可读代码在电子设备的处理器中运行时，电子设备中的处理器执行上述的运动对象的识别方法。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读存储介质上，计算机可读存储介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。
如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读程序指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM)、静态随机存取存储器(SRAM)、闪存或其他存储器技术、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读程序指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里所描述的计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
这里参照根据本公开实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意 的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
本文已经公开了示例实施例，并且虽然采用了具体术语，但它们仅用于并仅应当被解释为一般说明性含义，并且不用于限制的目的。在一些实例中，对本领域技术人员显而易见的是，除非另外明确指出，否则可单独使用与特定实施例相结合描述的特征、特性和/或元素，或可与其他实施例相结合描述的特征、特性和/或元件组合使用。因此，本领域技术人员将理解，在不脱离由所附的权利要求阐明的本公开的范围的情况下，可进行各种形式和细节上的改变。

Claims (12)

  1. 一种运动对象的识别方法，其特征在于，包括：
    获取基于事件的视觉传感器采集的事件流数据；
    根据采集的所述事件流数据，进行感兴趣目标检测，得到检测结果；
    在所述检测结果为检测到所述感兴趣目标的情况下，获取目标图像帧集合；其中，所述目标图像帧集合包括目标时间段内的图像帧，所述目标时间段包括所述感兴趣目标出现的时间段；
    根据所述目标图像帧集合，识别运动对象。
  2. 根据权利要求1所述的方法，其特征在于，所述根据采集的所述事件流数据，进行感兴趣目标检测，得到检测结果，包括：
    将所述事件流数据或者所述事件流数据对应的脉冲序列输入预训练的神经网络中，通过所述预训练的神经网络进行感兴趣目标检测，以确定是否检测到所述感兴趣目标，得到所述感兴趣目标的检测结果；
    其中，所述事件流数据对应的脉冲序列基于对所述事件流数据编码得到。
  3. 根据权利要求2所述的方法，其特征在于，所述预训练的神经网络包括以下任意一项：脉冲神经网络；脉冲神经网络和人工神经网络融合的神经网络。
  4. 根据权利要求1所述的方法，其特征在于，所述目标图像帧集合包括第一图像帧集合；所述在所述检测结果为检测到所述感兴趣目标的情况下，获取目标图像帧集合，包括：
    在检测到所述感兴趣目标的情况下，获取通过基于色彩模式的图像采集装置采集的在所述目标时间段内的图像帧，得到所述第一图像帧集合。
  5. 根据权利要求4所述的方法，其特征在于，所述获取通过基于色彩模式的图像采集装置采集的在所述目标时间段内的图像帧，得到所述第一图像帧集合，包括：
    在第一时刻向所述基于色彩模式的图像采集装置发送第一指令，所述第一时刻为所述目标时间段的起始时刻，所述第一指令用于指示所述基于色彩模式的图像采集装置采集并返回实时图像帧；
    接收所述基于色彩模式的图像采集装置发送的实时图像帧；
    在第二时刻向所述基于色彩模式的图像采集装置发送第二指令，所述第二时刻为所述目标时间段的终止时刻，所述第二指令用于指示所述基于色彩模式的图像采集装置停止采集实时图像帧；
    根据接收的从所述起始时刻到所述终止时刻的实时图像帧，得到所述第一图像帧集合。
  6. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    接收基于色彩模式的图像采集装置发送的实时图像帧；其中，所述图像帧是在所述基于色彩模式的图像采集装置采集事件流数据的过程中，通过基于色彩模式的图像采集 装置同步采集得到；
    所述获取通过基于色彩模式的图像采集装置采集的在所述目标时间段内的图像帧，得到所述第一图像帧集合，包括：
    从接收的所述图像帧中获取所述目标时间段内的图像帧，得到所述第一图像帧集合。
  7. 根据权利要求1所述的方法，其特征在于，所述目标图像帧集合包括第二图像帧集合，所述在所述检测结果为检测到所述感兴趣目标的情况下，获取目标图像帧集合，包括：
    根据所述事件流数据，确定所述感兴趣目标的位置信息；
    在基于色彩模式的图像采集装置采集的实时图像帧中，截取所述位置信息对应的图像帧，得到所述第二图像帧集合。
  8. 根据权利要求1-7中任一项所述的方法，其特征在于，
    所述目标图像帧集合中包括基于色彩模式的图像采集装置采集的图像帧；
    对于所述基于事件的视觉传感器和所述基于色彩模式的图像采集装置，二者位于同一图像数据采集设备中，且二者所包含摄像头之间的距离小于预设距离阈值。
  9. 一种运动对象的识别装置，其特征在于，包括：
    第一获取模块，用于获取基于事件的视觉传感器采集的事件流数据；
    检测模块，用于根据采集的所述事件流数据，进行感兴趣目标检测，得到检测结果；
    第二获取模块，用于在所述检测结果为检测到所述感兴趣目标的情况下，获取目标图像帧集合；其中，所述目标图像帧集合包括目标时间段内的图像帧，所述目标时间段包括所述感兴趣目标出现的时间段；
    识别模块，用于根据所述目标图像帧集合，识别运动对象。
  10. 一种电子设备，其特征在于，包括处理器，存储器及存储在所述存储器上并可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现包括如权利要求1至8中任一项所述的方法的步骤。
  11. 一种可读存储介质，其特征在于，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现包括权利要求1至8中任一项所述的方法的步骤。
  12. 一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，其中，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行用于实现权利要求1-8中的任一项所述的方法。
PCT/CN2022/090390 2021-05-08 2022-04-29 运动对象的识别方法、装置、电子设备及可读存储介质 WO2022237591A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110501305.0A CN113096158A (zh) 2021-05-08 2021-05-08 运动对象的识别方法、装置、电子设备及可读存储介质
CN202110501305.0 2021-05-08

Publications (1)

Publication Number Publication Date
WO2022237591A1 true WO2022237591A1 (zh) 2022-11-17

Family

ID=76664710

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/090390 WO2022237591A1 (zh) 2021-05-08 2022-04-29 运动对象的识别方法、装置、电子设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN113096158A (zh)
WO (1) WO2022237591A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113096158A (zh) * 2021-05-08 2021-07-09 北京灵汐科技有限公司 运动对象的识别方法、装置、电子设备及可读存储介质
CN113506324B (zh) * 2021-07-15 2024-04-12 清华大学 图像处理方法及装置、电子设备和存储介质
CN113506229B (zh) * 2021-07-15 2024-04-12 清华大学 神经网络训练和图像生成方法及装置
CN113326820B (zh) * 2021-08-02 2021-11-30 中汽创智科技有限公司 一种行驶环境感知方法、装置、电子设备及存储介质
CN115578771A (zh) * 2022-10-24 2023-01-06 智慧眼科技股份有限公司 活体检测方法、装置、计算机设备及存储介质
CN118365949A (zh) * 2024-04-30 2024-07-19 动微视觉(北京)科技有限公司 基于事件相机的视觉检测方法、装置及视觉检测相机

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073929A (zh) * 2016-11-15 2018-05-25 北京三星通信技术研究有限公司 基于动态视觉传感器的物体检测方法及设备
CN108734739A (zh) * 2017-04-25 2018-11-02 北京三星通信技术研究有限公司 用于时间对齐标定、事件标注、数据库生成的方法及装置
US20180357504A1 (en) * 2017-06-13 2018-12-13 Samsung Electronics Co., Ltd. Event-based image feature extraction
CN111582300A (zh) * 2020-03-20 2020-08-25 北京航空航天大学 一种基于事件相机的高动态目标检测方法
CN111951313A (zh) * 2020-08-06 2020-11-17 北京灵汐科技有限公司 图像配准方法、装置、设备及介质
CN112666550A (zh) * 2020-12-25 2021-04-16 北京灵汐科技有限公司 运动物体检测方法及装置、融合处理单元、介质
CN112669344A (zh) * 2020-12-24 2021-04-16 北京灵汐科技有限公司 一种运动物体的定位方法、装置、电子设备及存储介质
CN112800860A (zh) * 2021-01-08 2021-05-14 中电海康集团有限公司 一种事件相机和视觉相机协同的高速抛撒物检测方法和系统
CN113096158A (zh) * 2021-05-08 2021-07-09 北京灵汐科技有限公司 运动对象的识别方法、装置、电子设备及可读存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110933488A (zh) * 2018-09-19 2020-03-27 传线网络科技（上海）有限公司 视频剪辑方法及装置
CN112183153A (zh) * 2019-07-01 2021-01-05 中国移动通信集团浙江有限公司 一种基于视频分析的对象行为检测方法及装置
WO2021056434A1 (zh) * 2019-09-27 2021-04-01 深圳市大疆创新科技有限公司 一种探测对象的检测方法、探测设备及毫米波雷达
CN111898416A (zh) * 2020-06-17 2020-11-06 绍兴埃瓦科技有限公司 视频流处理方法、装置、计算机设备和存储介质
CN112070047A (zh) * 2020-09-15 2020-12-11 北京金山云网络技术有限公司 视频处理方法、装置及电子设备

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073929A (zh) * 2016-11-15 2018-05-25 北京三星通信技术研究有限公司 基于动态视觉传感器的物体检测方法及设备
CN108734739A (zh) * 2017-04-25 2018-11-02 北京三星通信技术研究有限公司 用于时间对齐标定、事件标注、数据库生成的方法及装置
US20180357504A1 (en) * 2017-06-13 2018-12-13 Samsung Electronics Co., Ltd. Event-based image feature extraction
CN111582300A (zh) * 2020-03-20 2020-08-25 北京航空航天大学 一种基于事件相机的高动态目标检测方法
CN111951313A (zh) * 2020-08-06 2020-11-17 北京灵汐科技有限公司 图像配准方法、装置、设备及介质
CN112669344A (zh) * 2020-12-24 2021-04-16 北京灵汐科技有限公司 一种运动物体的定位方法、装置、电子设备及存储介质
CN112666550A (zh) * 2020-12-25 2021-04-16 北京灵汐科技有限公司 运动物体检测方法及装置、融合处理单元、介质
CN112800860A (zh) * 2021-01-08 2021-05-14 中电海康集团有限公司 一种事件相机和视觉相机协同的高速抛撒物检测方法和系统
CN113096158A (zh) * 2021-05-08 2021-07-09 北京灵汐科技有限公司 运动对象的识别方法、装置、电子设备及可读存储介质

Also Published As

Publication number Publication date
CN113096158A (zh) 2021-07-09

Similar Documents

Publication Publication Date Title
WO2022237591A1 (zh) 运动对象的识别方法、装置、电子设备及可读存储介质
US11856290B2 (en) Method and apparatus of processing a signal from an event-based sensor
US10070053B2 (en) Method and camera for determining an image adjustment parameter
US11790504B2 (en) Monitoring method and apparatus
US20180048894A1 (en) Methods and systems of performing lighting condition change compensation in video analytics
US8280108B2 (en) Image processing system, image processing method, and computer program
CN108734107B (zh) 一种基于人脸的多目标跟踪方法及系统
TWI538510B (zh) 在基於一場景中動作的程度捕捉該場景之一序列影像圖框中最佳化一影像圖框之曝光的攝影機及方法
CN110191320B (zh) 基于像素时序运动分析的视频抖动与冻结检测方法及装置
US8879789B1 (en) Object analysis using motion history
CN108230607B (zh) 一种基于区域特征分析的图像火灾探测方法
EP3518529A1 (en) Method and apparatus of processing a signal from an event-based sensor
CN111489342A (zh) 一种基于视频的火焰检测方法、系统及可读存储介质
KR102584708B1 (ko) 과소 및 과밀 환경을 지원하는 군중위험관리시스템 및 방법
US20120308123A1 (en) Apparatus and method for estimating the number of objects included in an image
CN112184771B (zh) 一种社区的人员轨迹跟踪的方法及装置
CN115760912A (zh) 运动目标跟踪方法、装置、设备及计算机可读存储介质
CN109841022B (zh) 一种目标移动轨迹侦测报警方法、系统及存储介质
CN111783563A (zh) 基于双光谱的人脸抓拍和监控方法及系统、设备
CN104574340A (zh) 一种基于历史图像的视频入侵检测方法
CN103729624A (zh) 一种基于肤色识别的测光方法及测光系统
KR102150661B1 (ko) 객체를 검출하기 위한 전처리 장치 및 방법
TWI476735B (zh) 攝影機異常種類辨識方法及可偵測攝影異常的監視主機
JP3736836B2 (ja) 物体検出方法及び物体検出装置及びプログラム
CN113038017A (zh) 图像采集方法、装置及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22806559

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12-03-2024)