WO2020156341A1 - 移动目标的检测方法、装置及电子设备和存储介质 - Google Patents

移动目标的检测方法、装置及电子设备和存储介质 Download PDF

Info

Publication number
WO2020156341A1
WO2020156341A1 PCT/CN2020/073327 CN2020073327W WO2020156341A1 WO 2020156341 A1 WO2020156341 A1 WO 2020156341A1 CN 2020073327 W CN2020073327 W CN 2020073327W WO 2020156341 A1 WO2020156341 A1 WO 2020156341A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature points
frame image
moving target
distances
image
Prior art date
Application number
PCT/CN2020/073327
Other languages
English (en)
French (fr)
Inventor
王再冉
郭小燕
郑文
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2020156341A1 publication Critical patent/WO2020156341A1/zh
Priority to US17/138,452 priority Critical patent/US11176687B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30221—Sports video; Sports image
    • G06T2207/30224—Ball; Puck
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition

Definitions

  • This application relates to the field of image processing technology, and in particular to detection methods, devices, electronic equipment, and storage media for moving targets.
  • the shooting of moving objects is a common shooting scene. For example, in sports competitions, shooting athletes' poses in intense competition; in animal photography, recording the real scenes of animals running.
  • the shooting of people or objects moving at high speed requires professional shooting skills and rich experience, but many users usually do not have professional shooting knowledge, so the shots are not satisfactory.
  • the background difference method is a pixel-level background modeling and foreground detection algorithm. Its core idea is the update strategy of the background model. The sample of the pixel to be replaced is randomly selected, and the neighboring pixels are randomly selected for update. When the model of pixel change cannot be determined, the random update strategy can simulate the uncertainty of pixel change to a certain extent.
  • a background model needs to be created and initialized, and the background model needs to be updated.
  • the background difference method can only recognize moving objects in images collected by a fixed camera.
  • the first stage of the object segmentation method needs to obtain the edges of the moving object according to the adjacent frame images, and then generate the blur estimation of the pixels inside the moving object.
  • an apparent model is automatically generated through resampling based on fuzzy estimation, and then the apparent model is used for precise spatial segmentation and static segmentation.
  • the object segmentation method is used to identify moving objects, it is necessary to generate an apparent model, and it is necessary to calculate the optical flow of each pixel in the image to get the edge of the moving object.
  • the inventor realizes that no matter whether the background difference method or the object segmentation method is used to recognize a moving object, it is necessary to create a related model. Moreover, the background difference method has higher requirements for the camera position of the captured image, and the object separation method needs to calculate the optical flow of each pixel.
  • the present application discloses a detection method, device, electronic equipment and storage medium for moving objects.
  • a method for detecting a moving target including:
  • the moving target is detected according to a plurality of the third characteristic points.
  • a device for detecting a moving target including:
  • An image matrix acquisition unit configured to acquire adjacent first frame images and second frame images, and a rotation matrix and translation matrix between the first frame image and the second frame image, the first The frame image and the second frame image both contain the same moving target;
  • the feature point extraction unit is configured to extract a plurality of first feature points from the first frame image
  • the feature point determining unit is configured to determine a plurality of second features corresponding to the plurality of first feature points from the second frame image according to the second frame image and the plurality of first feature points point;
  • the distance calculation unit is configured to calculate, according to the rotation matrix and the translation matrix, a plurality of distances between the plurality of second feature points and the corresponding plurality of epipolar lines, and the corresponding plurality of epipolar lines is more Multiple epilines of the first feature points on the second frame image;
  • the feature point screening unit is configured to screen out a plurality of third feature points located on the moving target according to a plurality of the distances;
  • the target detection unit is configured to detect the moving target according to a plurality of the third characteristic points.
  • an electronic device including:
  • a memory for storing executable instructions of the processor
  • the processor is configured to: when the one or more programs are executed by the one or more processors, the one or more processors implement the following processes:
  • the moving target is detected according to a plurality of the third characteristic points.
  • a non-transitory computer-readable storage medium When instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is made to execute as described in the first aspect. The detection method of the moving target.
  • a computer program product when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can perform the detection of the moving target as described in the first aspect method.
  • the embodiment of the application provides a detection solution for a moving target.
  • the first feature point and the second feature point are determined according to two adjacent frames of images, and the rotation matrix and the translation matrix between the first frame image and the second frame image are obtained. Calculate the distance between the second feature point and the corresponding epipolar line through the rotation matrix and the translation matrix, and filter the third feature point for detecting the moving target according to the distance.
  • the embodiment of the application does not specifically limit whether the first frame image and the second frame image are taken at different positions, which reduces the requirements for the first frame image and the second frame image; on the other hand, the embodiment of the application There is no need to perform related calculations on each pixel of the first frame image and the second frame image, only the first feature point and the second feature point need to be extracted, which reduces the amount of data calculation; on the other hand, the embodiment of the present application does not The need to create related models reduces the detection steps of moving targets.
  • Fig. 1 is a schematic flowchart showing a method for detecting a moving target according to an exemplary embodiment
  • Fig. 2 is a schematic diagram showing the principle of epipolar geometry according to an exemplary embodiment
  • Fig. 3 is a schematic diagram showing a moving target detection process according to an exemplary embodiment
  • Fig. 4 is a block diagram showing a device for detecting a moving target according to an exemplary embodiment
  • Fig. 5 is a block diagram showing an electronic device for detecting a moving target according to an exemplary embodiment
  • Fig. 6 is a block diagram showing a device for detecting a moving target according to an exemplary embodiment.
  • Fig. 1 is a schematic flow chart showing a method for detecting a moving target according to an exemplary embodiment.
  • the method for detecting a moving target is applied to a mobile terminal, and the method may include the following steps.
  • Step S11 acquiring adjacent first frame image and second frame image, as well as the rotation matrix and translation matrix between the first frame image and the second frame image, the first frame image and the second frame image both contain the same movement aims.
  • the images in the embodiments of the present application may be captured by image capture devices such as cameras or video cameras.
  • the image can be acquired by a Complementary Metal-Oxide-Semiconductor (CMOS) sensor in the image acquisition device.
  • CMOS Complementary Metal-Oxide-Semiconductor
  • the adjacent first frame image and second frame image may be captured by a camera or video camera and other image capture devices at the same location, or may be captured by a camera or video camera and other image capture devices at different locations.
  • the embodiments of the application do not impose specific restrictions on the shooting positions of the first frame image and the second frame image.
  • the embodiments of the application do not impose any basic information such as the resolution, format, and capacity of the first frame image and the second frame image. Specific restrictions.
  • the rotation matrix and the translation matrix between the first frame image and the second frame image can be obtained according to an inertial measurement unit (International Mathematical Union, IMU) sensor.
  • IMU International Mathematical Union
  • IMU is a device that measures the three-axis attitude angle (or angular rate) and acceleration of an object. In practical applications, the IMU can obtain the rotation matrix and the translation matrix according to the posture relationship between the first frame image and the second frame image.
  • Epipolar geometry is the basis of view geometry theory.
  • Epipolar geometry describes the visual geometric relationship between two frames of images in the same scene.
  • the epipolar geometry only depends on the parameters in the camera and the relative pose between the two images.
  • a point P (X, Y, Z) in the three-dimensional space is projected to the left image plane IL and the right image plane IR.
  • the projection points are respectively PL and PR, and the points OL and OR are two planes IL, IR's camera center.
  • the points OL, OR, and P form a polar plane in the three-dimensional space.
  • the intersection line PLeL of the polar plane and the left image plane IL is referred to as a polar line corresponding to the projection point PR.
  • the intersection line PReR between the polar plane and the right image plane IR is referred to as the polar line corresponding to the projection point PL.
  • the line segment between the points OL and OR is called the baseline B, and the distance from the center of the camera to the respective epipolar line is f.
  • the IMU can obtain the attitude relationship according to the built-in accelerometer (three-axis), gyroscope (three-axis), and magnetic field meter (three-axis), such as using a nine-axis fusion algorithm to obtain the attitude relationship.
  • the accelerometer is used to detect the gravitational acceleration of the moving target on three axes
  • the gyroscope is used to measure the rotation rate of the moving target on the three axes.
  • the magnetic field meter can be a compass, which can coordinate the three axes of the accelerometer and the three axes of the gyroscope. The data in the six axes are yaw corrected.
  • the gravitational acceleration obtained by the accelerometer can determine the state of the moving target, and the rotation rate measured by the gyroscope can be used to detect the instantaneous state of the moving target, such as flipping, rotating speed, etc.
  • the movement state of the moving target can be obtained. There is a slight difference between the integration calculation and the real state, and the effect is small in a short time, but this error will always accumulate, and as the use time increases, there will be a significant deviation. Therefore, it is necessary to introduce a magnetic field meter to find the correct direction for correction.
  • Common nine-axis fusion algorithms can include Kalman filtering, particle filtering, complementary filtering algorithms and so on.
  • the posture relationship between the first frame of image and the second frame of image can include the following two types: the first posture relationship, a straight line on another frame of image can be determined by epipolar geometry points on one frame of image; the second Posture relationship, through the mapping relationship between the points in the first posture relationship to the straight line, a point on one frame of image can determine a point on another frame of image, and the point on another frame of image can be the first frame of image
  • the first posture relationship can be represented by a basic matrix
  • the second posture relationship can be represented by a homography matrix.
  • the essential matrix is a special case of the fundamental matrix, which belongs to the fundamental matrix under the normalized image coordinates.
  • the essential matrix is a 3x3 matrix with 5 degrees of freedom, the translation matrix contains 3 degrees of freedom and the rotation matrix contains 3 degrees of freedom, and 1 degree of freedom is removed from the scale uncertainty (the essential matrix is a homogeneous quantity).
  • One function of the essential matrix is to give a point on a frame of image and multiply it with the essential matrix. The result is the epipolar line of this point on another frame of image. When matching, the search range can be greatly reduced; Another function is to calculate the rotation matrix R and the translation matrix t.
  • the IMU can use a random sampling consensus algorithm to determine the rotation matrix and Translation matrix.
  • first frame of image and the second frame of image may be two adjacent frames of images that both contain the same moving target, where the moving target may be a person, an animal, other objects, etc., and other objects may include but are not limited to: plants , Vehicles and any objects that can move actively or passively.
  • Step S12 extracting multiple first feature points from the first frame of image.
  • the FAST feature point detection algorithm can be used to extract multiple first feature points from the first frame of image.
  • the FAST feature point detection algorithm belongs to a feature point detection algorithm.
  • the principle of detection is: if there are a certain number of pixels around a pixel with different pixel values, the pixel is considered to be a corner point, that is, extreme Value point. For example, taking a certain pixel point p on the first frame of image as the center, there are 16 pixels on a circle with a radius of 3 pixels, which are respectively p1, p2, ..., p16. Calculate the pixel value difference between the pixel points p1 and p9 and the center p.
  • point p cannot be the first feature point; otherwise, point p is determined as the candidate first feature point. If point p is the candidate first feature point, then calculate the pixel value difference between pixel points p1, p9, p5, p13 and the center p. If their absolute values exceed the pixel threshold at least 3, then point p is determined to be the first candidate Feature point; otherwise, point p cannot be the first feature point. If point p is the candidate first feature point, calculate the pixel value difference between the 16 pixels p1 to p16 and the center p. If at least 9 of them exceed the pixel threshold, then point p is the first feature point; otherwise , P point cannot be the first feature point.
  • a non-maximum value suppression method in order to avoid the multiple first feature points extracted from the first frame image from being concentrated in a small area, can be used to extract the first frame image from the Extract multiple first feature points uniformly.
  • Non-maximum suppression is to calculate the Euclidean distance between two adjacent first feature points to ensure that the Euclidean distance is greater than the set distance threshold.
  • the embodiment of the present application does not specifically limit the values and units of the pixel threshold and the distance threshold.
  • Step S13 according to the second frame image and the multiple first feature points, determine multiple second feature points corresponding to the multiple first feature points from the second frame image.
  • the optical flow algorithm can be used to obtain multiple second feature points corresponding to multiple first feature points from the second frame of image.
  • Optical flow is the coordinate displacement of each pixel on the image. For example, the position coordinate of point A on the t-th frame image is (x1, y1), and the position coordinate of point A on the t+1-th frame image is (x2, y2), where x1 is not equal to x2, and/or y1 is not equal to y2. Then it can be determined that point A is a moving point.
  • Optical flow can be understood as the instantaneous velocity of the pixel movement of a spatially moving object on the observation imaging plane.
  • optical flow is generated by the movement of the target itself in the scene, the movement of the camera, or the joint movement of the two.
  • the optical flow algorithm can use the changes in the time domain of the pixels in the image sequence and the correlation between adjacent frames to find the correspondence between the previous frame image and the current frame image, so as to calculate the adjacent frame image A method of moving information between objects.
  • the first frame image and the second frame image may be subjected to local contrast normalization processing, and the local contrast normalization
  • the processing can ensure that the contrast of the first frame image and the second frame image is normalized on each small window instead of being normalized as a whole.
  • the local contrast normalization process can change the first frame image and the second frame image more, discarding all image areas of the same intensity, and aims to ensure that a robust image is obtained under different lighting conditions.
  • Step S14 Calculate multiple distances between multiple second feature points and multiple corresponding epipolar lines according to the rotation matrix and the translation matrix.
  • step S14 if the distance between the second feature point and the corresponding epipolar line is calculated, the rotation matrix and the translation matrix can be used to determine the first feature point in the second frame according to the principle of epipolar geometry as shown in FIG. The polar line on the image, and then calculate the second characteristic point and the corresponding polar line according to the coordinate value of the second characteristic point and the polar line of the first characteristic point on the second frame of the image in the normalized camera coordinate system the distance.
  • the embodiment of the present application does not specifically limit the technical means used to calculate the distance between the second feature point and the corresponding epipolar line.
  • Step S15 Filter out a plurality of third characteristic points located on the moving target according to a plurality of distances.
  • the distance between the non-moving feature points located in the background area of the image and the corresponding epipolar line is 0, and the moving feature points located in the target object area in the image will deviate from the corresponding epipolar line.
  • the distance between the two feature points and the corresponding epipolar line separates the second feature point located on the moving target and the second feature point located in the background area.
  • the distance from the second feature point to the corresponding extreme line can be compared with a preset distance threshold. If the distance from the second feature point to the corresponding extreme line is greater than the distance threshold, the second feature point is determined Is the third feature point located on the moving target; if the distance from the second feature point to the corresponding epipolar line is less than or equal to the distance threshold, the second feature point is determined as the second feature point located on the background area.
  • step S16 a moving target is detected according to a plurality of third characteristic points.
  • the smallest outer rectangle containing all the third feature points can be used as the region to which the moving target belongs.
  • the method for detecting a moving target combines two kinds of sensors, CMOS and IMU.
  • CMOS the adjacent first frame image and second frame image can be obtained, and according to IMU, the rotation matrix and translation matrix between the first frame image and the second frame image can be obtained, where the first frame image and the second frame image Both contain the same moving target.
  • a plurality of first feature points are extracted from the first frame of image, and a plurality of second feature points corresponding to the plurality of first feature points are determined in the second frame of image.
  • the rotation matrix and the translation matrix calculate the multiple distances from the multiple second feature points to the corresponding multiple epipolar lines, and then filter the multiple third feature points on the moving target according to the multiple distances obtained by the calculation.
  • the moving target is detected according to a plurality of third feature points.
  • the embodiment of the application determines the first feature point and the second feature point according to two adjacent frames of images, and obtains the rotation matrix and translation matrix between the first frame image and the second frame image, and calculates the first feature point and the second feature point through the rotation matrix and the translation matrix. According to the distance between the two feature points and the corresponding epipolar line, the third feature point for detecting the moving target is screened out according to the distance.
  • the embodiment of the present application does not specifically limit whether the first frame image and the second frame image are captured at different positions, which reduces the requirements for the first frame image and the second frame image.
  • the embodiment of the present application does not need to perform correlation calculations on each pixel point of the first frame image and the second frame image, but only needs to extract the first feature point and the second feature point, which reduces the amount of data calculation.
  • the embodiments of the present application do not need to create related models, which reduces the detection steps of moving targets.
  • the technical solutions in the embodiments of the present application can be applied to the moving target detection process shown in FIG. 3 to extract the first feature point from the first frame of image, the first feature point can be a FAST feature point, and then from the first feature point. Extract the second feature point corresponding to the first feature point from the two frames of images. Then, the camera motion parameters, namely the rotation matrix and the translation matrix are calculated according to the first feature point and the second feature point. The distance between the second feature point and the corresponding epipolar line is calculated according to the rotation matrix and the translation matrix, and the moving target is finally separated according to the calculated distance.
  • the principle of optical flow and epipolar geometry is used to detect moving targets in the image. In the application scenario of shooting moving targets, it can automatically focus on moving targets efficiently and quickly and easily shoot accurate and clear moving targets.
  • Fig. 4 is a block diagram showing a device for detecting a moving target according to an exemplary embodiment.
  • the mobile target detection device provided by the embodiment of the present application includes an image matrix acquisition unit 41, a feature point extraction unit 42, a feature point determination unit 43, a distance calculation unit 44, a feature point screening unit 45, and a target Detection unit 46.
  • the image matrix acquisition unit 41 is connected to and/or communicates with the feature point extraction unit 42, the feature point determination unit 43, and the distance calculation unit 44, respectively, and the feature point determination unit 43 is connected to the feature point extraction unit 42 and the distance calculation unit 44, respectively.
  • the feature point screening unit 45 is connected and/or communicated with the distance calculation unit 44 and the target detection unit 46, respectively.
  • the image matrix acquiring unit 41 is configured to acquire adjacent first frame images and second frame images, and the rotation matrix and translation matrix between the first frame image and the second frame image, the first frame image and the second frame image
  • the frame images all contain the same moving target.
  • the feature point extraction unit 42 is configured to extract a plurality of first feature points from the first frame of image.
  • the feature point determining unit 43 is configured to determine a plurality of second feature points corresponding to the plurality of first feature points from the second frame image according to the second frame image and the plurality of first feature points.
  • the distance calculation unit 44 is configured to calculate the multiple distances between the multiple second feature points and the multiple corresponding epipolar lines according to the rotation matrix and the translation matrix. Multiple epipolar lines on the two-frame image.
  • the feature point screening unit 45 is configured to screen out multiple third feature points located on the moving target according to multiple distances.
  • the target detection unit 46 is configured to detect a moving target according to a plurality of third feature points.
  • the distance calculation unit 44 includes:
  • the epipolar line determining module 441 is configured to determine multiple epipolar lines corresponding to the multiple first feature points on the second frame image according to the rotation matrix and the translation matrix.
  • the distance determination module 442 is configured to calculate the relationship between the multiple second feature points and the corresponding multiple epipolar lines according to the coordinate values of the multiple second feature points and the corresponding multiple epipolar lines in the normalized camera coordinate system. Multiple distances between.
  • the feature point screening unit 45 is specifically configured to compare the multiple distances with preset distance thresholds respectively;
  • the second feature points corresponding to the distances greater than the distance threshold among the multiple distances are determined as multiple third feature points located on the moving target.
  • the area detection unit 46 is specifically configured to determine the smallest outer rectangle formed by a plurality of third feature points as the area to which the moving target belongs.
  • the feature point extraction unit 42 is specifically configured to extract a plurality of first feature points from the first frame of image according to the FAST feature point detection algorithm.
  • the feature point determining unit 43 is specifically configured to extract multiple second feature points corresponding to multiple first feature points from the second frame image according to an optical flow algorithm.
  • Fig. 5 is a block diagram showing an electronic device 500 for detecting a moving target according to an exemplary embodiment.
  • the electronic device 500 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
  • the electronic device 500 may include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, a multimedia component 508, an audio component 510, an input/output (I/O) interface 512, a sensor component 514, and a communication component 516.
  • the processing component 502 generally controls overall operations of the electronic device 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the foregoing method.
  • the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components.
  • the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.
  • the memory 504 is configured to store various types of data to support operations in the electronic device 500. Examples of these data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, videos, etc.
  • the memory 504 can be implemented by any type of volatile or non-volatile storage devices or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable and Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable and Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic Disk Magnetic Disk or Optical Disk.
  • the power supply component 506 provides power for various components of the electronic device 500.
  • the power supply component 506 may include a power management system, one or more power supplies, and other components associated with the electronic device 500 generating, managing, and distributing power.
  • the multimedia component 508 includes a screen of an output interface provided between the electronic device 500 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be set as a touch screen to receive input signals from the user.
  • the touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure related to the touch or slide operation.
  • the multimedia component 508 includes a front camera and/or a rear camera. When the electronic device 500 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 510 is configured to output and/or input audio signals.
  • the audio component 510 includes a microphone (MIC).
  • the microphone is configured to receive an external audio signal.
  • the received audio signal may be further stored in the memory 504 or transmitted via the communication component 516.
  • the audio component 510 further includes a speaker for outputting audio signals.
  • the I/O interface 512 provides an interface between the processing component 502 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include but are not limited to: home button, volume button, start button, and lock button.
  • the sensor component 514 includes one or more sensors for providing the electronic device 500 with various aspects of state evaluation.
  • the sensor component 514 can detect the on/off status of the electronic device 500 and the relative positioning of the components.
  • the component is the display and the keypad of the electronic device 500.
  • the sensor component 514 can also detect the electronic device 500 or the electronic device 500.
  • the position of the component changes, the presence or absence of contact between the user and the electronic device 500, the orientation or acceleration/deceleration of the electronic device 500, and the temperature change of the electronic device 500.
  • the sensor component 514 may include a proximity sensor configured to detect the presence of nearby objects when there is no physical contact.
  • the sensor component 514 may also include a light sensor, such as a CMOS or charge-coupled device (Charge-coupled Device, CCD) image sensor, for use in imaging applications.
  • a light sensor such as a CMOS or charge-coupled device (Charge-coupled Device, CCD) image sensor
  • CCD Charge-coupled Device
  • the sensor component 514 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices.
  • the electronic device 500 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G, or 5G), or a combination thereof.
  • the communication component 516 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • the electronic device 500 may be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field A programmable gate array (FPGA), controller, microcontroller, microprocessor, or other electronic components are implemented to implement the above methods.
  • ASIC application specific integrated circuits
  • DSP digital signal processors
  • DSPD digital signal processing devices
  • PLD programmable logic devices
  • FPGA field A programmable gate array
  • controller microcontroller, microprocessor, or other electronic components are implemented to implement the above methods.
  • non-transitory computer-readable storage medium including instructions, such as the memory 504 including instructions, which may be executed by the processor 520 of the electronic device 500 to complete the foregoing method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
  • Fig. 6 is a block diagram showing a device 600 for detecting a moving target according to an exemplary embodiment.
  • the apparatus 600 may be provided as a server or electronic equipment.
  • the apparatus 600 includes a processing component 622, which further includes one or more processors, and a memory resource represented by the memory 632, for storing instructions that can be executed by the processing component 622, such as application programs.
  • the application program stored in the memory 632 may include one or more modules each corresponding to a set of instructions.
  • the processing component 622 is configured to execute instructions to execute the aforementioned method for detecting a moving target.
  • the device 600 may also include a power component 626 configured to perform power management of the device 600, a wired or wireless network interface 650 configured to connect the device 600 to a network, and an input output (I/O) interface 658.
  • the device 600 can operate based on an operating system stored in the memory 632, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.
  • the embodiments of the present application may also provide a computer program product.
  • the instructions in the computer program product are executed by the processor of the server, device, or electronic device, the server, device, or electronic device can execute the above-mentioned method for detecting moving targets. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供了一种移动目标的检测方法、装置及电子设备和存储介质，涉及图像处理技术领域，所述方法包括：获取相邻的第一帧图像和第二帧图像及第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵，第一帧图像和第二帧图像包含同一移动目标；从第一帧图像中提取多个第一特征点；根据第二帧图像和多个第一特征点从第二帧图像中确定与多个第一特征点对应的多个第二特征点；根据旋转矩阵和平移矩阵计算多个第二特征点与对应的多个极线之间的多个距离，对应的多个极线为多个第一特征点在第二帧图像上的多个极线；根据多个距离筛选出位于移动目标上的多个第三特征点；根据多个第三特征点检测得到移动目标。本申请减少了数据计算量和移动目标的检测步骤。

Description

移动目标的检测方法、装置及电子设备和存储介质
相关申请的交叉引用
本申请要求于2019年01月31日提交中国专利局、申请号为201910101425.4，发明名称为“移动目标的检测方法、装置及电子设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及移动目标的检测方法、装置及电子设备和存储介质。
背景技术
运动物体的拍摄是一种常见的拍摄场景，如在体育比赛中，拍摄运动员激烈比赛的姿态；在动物摄影中，记录动物奔跑时的真实场景。针对高速运动的人或物的拍摄是需要专业的拍摄技巧和丰富经验的，但是很多用户通常并不具备专业拍摄知识，因此，拍摄的作品不尽如人意。
相关技术中，可以采用背景差分法或者物体分割法等方法对运动物体进行识别。其中，背景差分法是一种像素级的背景建模、前景检测算法，其核心思想是背景模型的更新策略，随机选择需要替换的像素的样本，随机选择邻域像素进行更新。在无法确定像素变化的模型时，随机的更新策略，在一定程度上可以模拟像素变化的不确定性。但是，在采用背景差分法识别运动物体时，需要创建并初始化背景模型，还需要对背景模型进行更新，而且，背景差分法仅能识别固定的相机采集到的图像中的运动物体。物体分割法的第一阶段需要根据相邻帧图像得到运动物体的边缘，进而生成存在运动物体内部像素的模糊估计。第二阶段通过基于模糊估计的重采样自动产生一个表观模型，随后利用表观模型进行精确的空间分割和静止状态的分割。但是，在采用物体分割法识别运动物体时，需要生成表观模型，而且需要计算图像 中每个像素点的光流才可以得到运动物体的边缘。
发明人意识到，无论采用背景差分法还是物体分割法对运动物体进行识别，均需要创建相关的模型。而且，背景差分法对采集图像的相机位置具有较高要求，物体分隔法需要计算每个像素点的光流。
发明内容
为克服相关技术中对运动物体进行识别均需要创建相关模型的问题，本申请公开一种移动目标的检测方法、装置及电子设备和存储介质。
根据本申请实施例的第一方面，提供一种移动目标的检测方法，包括：
获取相邻的第一帧图像和第二帧图像，以及，所述第一帧图像和所述第二帧图像之间的旋转矩阵和平移矩阵，所述第一帧图像和所述第二帧图像均包含同一移动目标；
从所述第一帧图像中提取得到多个第一特征点；
根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点；
根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，对应的多个所述极线为多个所述第一特征点在所述第二帧图像上的多个极线；
根据多个所述距离筛选出位于所述移动目标上的多个第三特征点；
根据多个所述第三特征点检测得到所述移动目标。
根据本申请实施例的第二方面，提供一种移动目标的检测装置，包括：
图像矩阵获取单元，被配置为获取相邻的第一帧图像和第二帧图像，以及，所述第一帧图像和所述第二帧图像之间的旋转矩阵和平移矩阵，所述第一帧图像和所述第二帧图像均包含同一移动目标；
特征点提取单元，被配置为从所述第一帧图像中提取得到多个第一特征点；
特征点确定单元，被配置为根据所述第二帧图像和多个所述第一特征点， 从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点；
距离计算单元，被配置为根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，对应的多个所述极线为多个所述第一特征点在所述第二帧图像上的多个极线；
特征点筛选单元，被配置为根据多个所述距离筛选出位于所述移动目标上的多个第三特征点；
目标检测单元，被配置为根据多个所述第三特征点检测得到所述移动目标。
根据本申请实施例的第三方面，提供一种电子设备，包括：
处理器；
用于存储所述处理器可执行指令的存储器；
其中，所述处理器被配置为：当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现下列过程：
获取相邻的第一帧图像和第二帧图像，以及，所述第一帧图像和所述第二帧图像之间的旋转矩阵和平移矩阵，所述第一帧图像和所述第二帧图像均包含同一移动目标；
从所述第一帧图像中提取得到多个第一特征点；
根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点；
根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，对应的多个所述极线为多个所述第一特征点在所述第二帧图像上的多个极线；
根据多个所述距离筛选出位于所述移动目标上的多个第三特征点；
根据多个所述第三特征点检测得到所述移动目标。
根据本申请实施例的第四方面，提供一种非临时性计算机可读存储介质，当所述存储介质中的指令由移动终端的处理器执行时，使得所述移动终端执行如第一方面所述的移动目标的检测方法。
根据本申请实施例的第五方面，提供一种计算机程序产品，当所述存储介质中的指令由移动终端的处理器执行时，使得移动终端能够执行如第一方面所述的移动目标的检测方法。
本申请实施例提供一种移动目标的检测方案，根据相邻的两帧图像确定第一特征点和第二特征点，并得到第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵，通过旋转矩阵和平移矩阵计算第二特征点到对应的极线之间的距离，根据距离筛选出用于检测移动目标的第三特征点。一方面，本申请实施例对第一帧图像和第二帧图像是否在不同位置拍摄所得不作具体限制，降低了对第一帧图像和第二帧图像的要求；另一方面，本申请实施例不需要对第一帧图像和第二帧图像的每个像素点进行相关计算，只需要提取得到第一特征点和第二特征点，减少了数据计算量；再一方面，本申请实施例不需要创建相关模型，减少了移动目标的检测步骤。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。
图1是根据一示例性实施例示出的一种移动目标的检测方法的示意流程图；
图2是根据一示例性实施例示出的对极几何原理示意图；
图3是根据一示例性实施例示出的一种移动目标检测过程示意图；
图4是根据一示例性实施例示出的一种移动目标的检测装置的框图；
图5是根据一示例性实施例示出的一种用于检测移动目标的电子设备的框图；
图6是根据一示例性实施例示出的一种用于检测移动目标的装置的框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的方法和装置的例子。
图1是根据一示例性实施例示出的一种移动目标的检测方法的示意流程图。
如图1所示，本实施例所提供的移动目标的检测方法应用于移动终端中，该方法可以包括以下步骤。
步骤S11，获取相邻的第一帧图像和第二帧图像，以及，第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵，第一帧图像和第二帧图像均包含同一移动目标。
本申请实施例中的图像可以由相机或摄像机等图像采集设备拍摄得到。具体可以由图像采集设备中的互补金属氧化物半导体(Complementary Metal-Oxide-Semiconductor，CMOS)传感器采集得到图像。其中，相邻的第一帧图像和第二帧图像可以由相机或摄像机等图像采集设备在同一位置处拍摄所得，也可以由相机或摄像机等图像采集设备在不同位置处拍摄所得。本申请实施例对第一帧图像和第二帧图像的拍摄位置等不作具体限制，同时，本申请实施例对第一帧图像和第二帧图像的分辨率、格式、容量等基本信息也不作具体限制。
在获取第一帧图像和第二帧图像之后，可以根据惯性测量单元(International Mathematical Union，IMU)传感器得到第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵。IMU是测量物体三轴姿态角(或角速率)以及加速度的装置。在实际应用中，IMU可以根据第一帧图像与第二帧图像之间的姿态关系得到旋转矩阵和平移矩阵。
在介绍姿态关系之前需要先介绍对极几何原理，对极几何是视图几何理 论的基础，对极几何描述了同一场景下两帧图像之间的视觉几何关系。对极几何只依赖于摄像机内参数和两帧图像之间的相对姿态。
如图2所示，三维空间中一点P(X，Y，Z)，投影到左图像平面IL和右图像平面IR，投影点分别为PL、PR，点OL和OR分别是两个平面IL、IR的相机中心。点OL、OR和P在三维空间内构成一个极平面。极平面与左图像平面IL的交线PLeL称之为对应于投影点PR的极线。同理，极平面与右图像平面IR的交线PReR称之为对应于投影点PL的极线。点OL、OR之间的线段称为基线B，相机中心到各自对应的极线的距离为f。
IMU可以根据内置的加速度计(三轴)、陀螺仪(三轴)、磁场计(三轴)获取姿态关系，如采用九轴融合算法获取姿态关系。加速度计用于检测移动目标在三轴上的重力加速度，陀螺仪用于测量移动目标在三轴上的旋转速率，磁场计可以是指南针，可以对加速度计的三轴和陀螺仪的三轴共六轴中的数据进行偏航校正。加速度计获取到的重力加速度可以确定移动目标摆放的状态，陀螺仪测量到的旋转速率可以用于检测移动目标的瞬时状态，如翻转、旋转的快慢等。通过加速度计和陀螺仪的积分运算，可以获得到移动目标的运动状态。积分运算与真实状态存在微小差值，短时间内影响很小，但这个误差会一直累积，随着使用时间增加，就会有明显的偏离，因此需要引入磁场计，找到正确的方向进行校正。常见的九轴融合算法可以包括卡尔曼滤波、粒子滤波、互补滤波算法等。
第一帧图像与第二帧图像之间的姿态关系可以包括以下两种：第一种姿态关系，通过对极几何一帧图像上的点可以确定另外一帧图像上的一条直线；第二种姿态关系，通过第一种姿态关系中的点到直线之间的映射关系，一帧图像上的点可以确定另外一帧图像上的一个点，另外一帧图像上的点可以是第一帧图像通过光心和图像点的射线与一个平面的交点在第二帧图像上的影像。
第一种姿态关系可以用基本矩阵来表示，第二种姿态关系可以用单应矩阵来表示。而本质矩阵则是基本矩阵的一种特殊情况，属于在归一化图像坐 标下的基本矩阵。本质矩阵使用的是摄像机坐标系，本质矩阵为E＝t ∧R，其中，t为平移矩阵，R为旋转矩阵，∧为反对称符号，作用是将平移矩阵t转换成反对称形式。本质矩阵是一个3x3矩阵，有5个自由度，平移矩阵包含的3个自由度与旋转矩阵包含的3个自由度，去掉尺度不确定性的1个自由度(本质矩阵为齐次量)。本质矩阵的一个作用是给定一帧图像上的一个点，和本质矩阵相乘，其结果为此点在另一帧图像上的极线，在匹配时，可以大大缩小搜索范围；本质矩阵的另一个作用是用于计算旋转矩阵R和平移矩阵t。
在本申请的一种其他实施例中，由于第一特征点和第二特征点中可能存在移动或者误匹配的特征点，为了提高鲁棒性，IMU可以使用随机抽样一致性算法确定旋转矩阵和平移矩阵。
需要说明的是，第一帧图像和第二帧图像可以为均包含同一移动目标的相邻两帧图像，其中，移动目标可以为人、动物和其他物体等，其他物体可以包括但不限于：植物、交通工具及可以主动移动或被动移动的任意物体。
步骤S12，从第一帧图像中提取得到多个第一特征点。
在实际应用中，可以利用FAST特征点检测算法从第一帧图像中提取得到多个第一特征点。该FAST特征点检测算法属于一种特征点检测算法，检测的原理为：若一个像素点周围有一定数量的像素点与该像素点的像素值不同，则认为该像素点为角点，即极值点。例如，以第一帧图像上某像素点p为中心，半径为3个像素点的圆上存在16个像素点，分别为p1、p2、……、p16。计算像素点p1、p9与中心p的像素值差，若它们的绝对值都小于预设的像素阈值，则p点不可能是第一特征点；否则，p点确定为候选第一特征点。若p点是候选第一特征点，则计算像素点p1、p9、p5、p13与中心p的像素值差，若它们的绝对值有至少3个超过像素阈值，则p点确定为候选第一特征点；否则，p点不可能是第一特征点。若p点是候选第一特征点，则计算像素点p1到p16这16个像素点与中心p的像素值差，若它们有至少9个超过像素阈值，则p点是第一特征点；否则，p点不可能是第一特征点。
在本申请的一种其他实施例中，为避免从第一帧图像中提取得到的多个 第一特征点集中在较小的区域内，可以采用非极大值抑制的方式从第一帧图像中均匀地提取多个第一特征点。非极大值抑制即计算相邻两个第一特征点之间的欧式距离，确保该欧氏距离大于设定的距离阈值。本申请实施例对上述像素阈值和距离阈值的数值和单位等不作具体限制。
步骤S13，据第二帧图像和多个第一特征点，从第二帧图像中确定与多个第一特征点对应的多个第二特征点。
在实际应用中，可以利用光流算法从第二帧图像中得到与多个第一特征点对应的多个第二特征点。光流即图像上每个像素点的坐标位移量，比如，第t帧图像上的A点的位置坐标为(x1，y1)，第t+1帧图像上A点的位置坐标为(x2，y2)，其中，x1不等于x2，和/或，y1不等于y2。则可以确定A点为移动的点。
光流可以理解为空间运动物体在观察成像平面上的像素运动的瞬时速度，一般而言，光流是由于场景中目标本身的移动、相机的运动，或者两者的共同运动所产生的。光流算法可以是利用图像序列中像素在时间域上的变化、以及相邻帧之间的相关性来找到上一帧图像与当前帧图像之间存在的对应关系，从而计算出相邻帧图像之间物体的运动信息的一种方法。
在本申请的一种其他实施例中，在利用光流算法确定多个第二特征点之前，可以先对第一帧图像和第二帧图像进行局部对比度归一化处理，局部对比度归一化处理可以确保第一帧图像和第二帧图像的对比度在每个小窗口上被归一化，而不是作为整体被归一化。局部对比度归一化处理可以更多地改变第一帧图像和第二帧图像，丢弃了所有相同强度的图像区域，目的在于保证在不同的光照条件下获得鲁棒的图像。
步骤S14，根据旋转矩阵和平移矩阵计算多个第二特征点与对应的多个极线之间的多个距离。
在步骤S14中，若计算第二特征点到对应的极线之间的距离，可以先根据旋转矩阵和平移矩阵，按照如图2所示的对极几何原理确定第一特征点在第二帧图像上的极线，然后根据第二特征点和第一特征点在第二帧图像上的 极线在归一化的相机坐标系中的坐标值计算第二特征点与对应的极线之间的距离。本申请实施例对计算第二特征点到对应的极线之间的距离所采用的技术手段不作具体限制。
步骤S15，根据多个距离筛选出位于移动目标上的多个第三特征点。
根据对极几何原理，非移动的位于图像中背景区域的特征点到对应的极线的距离为0，移动的位于图像中目标对象区域的特征点会偏离对应的极线，因此，可以根据第二特征点到对应的极线的距离分离出位于移动目标上的第二特征点和位于背景区域的第二特征点。
在实际应用中，可以将第二特征点到对应的极线的距离与预设的距离阈值进行比较，若第二特征点到对应的极线的距离大于距离阈值，则将第二特征点确定为位于移动目标上的第三特征点；若第二特征点到对应的极线的距离小于或等于距离阈值，则将第二特征点确定为位于背景区域上的第二特征点。
步骤S16，根据多个第三特征点检测得到移动目标。
在确定出多个第三特征点之后，可以将包含全部的第三特征点的最小外包矩形作为移动目标所属的区域。
本申请实施例提供的移动目标的检测方法，融合了CMOS和IMU两种传感器。根据CMOS可以获取相邻的第一帧图像和第二帧图像，根据IMU可以获取第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵，其中，第一帧图像和第二帧图像均包含同一移动目标。在第一帧图像中提取得到多个第一特征点，在第二帧图像中确定得到与多个第一特征点对应的多个第二特征点。根据旋转矩阵和平移矩阵计算多个第二特征点到对应的多个极线之间的多个距离，再根据计算得到的多个距离筛选出位于移动目标上的多个第三特征点，以根据多个第三特征点检测得到移动目标。
本申请实施例根据相邻的两帧图像确定第一特征点和第二特征点，并得到第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵，通过旋转矩阵和平移矩阵计算第二特征点到对应的极线之间的距离，根据距离筛选出用于检测 移动目标的第三特征点。一方面，本申请实施例对第一帧图像和第二帧图像是否在不同位置拍摄所得不作具体限制，降低了对第一帧图像和第二帧图像的要求。另一方面，本申请实施例不需要对第一帧图像和第二帧图像的每个像素点进行相关计算，只需要提取得到第一特征点和第二特征点，减少了数据计算量。再一方面，本申请实施例不需要创建相关模型，减少了移动目标的检测步骤。
本申请实施例中的技术方案，可以应用在如图3所示的移动目标检测过程中可以从第一帧图像中提取第一特征点，该第一特征点可以为FAST特征点，再从第二帧图像中提取与第一特征点对应的第二特征点。然后根据第一特征点和第二特征点计算相机运动参数，即旋转矩阵和平移矩阵。根据旋转矩阵和平移矩阵计算第二特征点到对应的极线之间的距离，最终根据计算得到的距离分离出移动目标。采用光流和对极几何原理，检测图像中的移动目标，在拍摄移动目标的应用场景下，可以高效地对移动目标进行自动对焦，方便、快捷地拍摄准确、清晰的移动目标。
图4是根据一示例性实施例示出的一种移动目标的检测装置的框图。如图4所示，本申请实施例所提供的移动目标的检测装置，包括图像矩阵获取单元41、特征点提取单元42、特征点确定单元43、距离计算单元44、特征点筛选单元45和目标检测单元46。其中，图像矩阵获取单元41分别与特征点提取单元42、特征点确定单元43和距离计算单元44连接和/或通信，特征点确定单元43分别与特征点提取单元42和距离计算单元44连接和/或通信，特征点筛选单元45分别与距离计算单元44和目标检测单元46连接和/或通信。
图像矩阵获取单元41，被配置为获取相邻的第一帧图像和第二帧图像，以及，第一帧图像和第二帧图像之间的旋转矩阵和平移矩阵，第一帧图像和第二帧图像均包含同一移动目标。
特征点提取单元42，被配置为从第一帧图像中提取得到多个第一特征点。
特征点确定单元43，被配置为根据第二帧图像和多个第一特征点，从第 二帧图像中确定与多个第一特征点对应的多个第二特征点。
距离计算单元44，被配置为根据旋转矩阵和平移矩阵计算多个第二特征点与对应的多个极线之间的多个距离，对应的多个极线为多个第一特征点在第二帧图像上的多个极线。
特征点筛选单元45，被配置为根据多个距离筛选出位于移动目标上的多个第三特征点。
目标检测单元46，被配置为根据多个第三特征点检测得到移动目标。
其中，距离计算单元44，包括：
极线确定模块441，被配置为根据旋转矩阵和平移矩阵，确定多个第一特征点在第二帧图像上对应的多个极线。
距离确定模块442，被配置为根据多个第二特征点和对应的多个极线在归一化的相机坐标系中的坐标值，计算多个第二特征点与对应的多个极线之间的多个距离。
在一种可能的实施方式中，特征点筛选单元45，具体被配置为将多个距离分别与预设的距离阈值进行比较；
将多个距离中大于距离阈值的距离对应的第二特征点确定为位于移动目标上的多个第三特征点。
在一种可能的实施方式中，区域检测单元46，具体被配置为将多个第三特征点形成的最小外包矩形确定为移动目标所属的区域。
在一种可能的实施方式中，特征点提取单元42，具体被配置为根据FAST特征点检测算法从第一帧图像中提取得到多个第一特征点。
在一种可能的实施方式中，特征点确定单元43，具体被配置为根据光流算法从第二帧图像中提取得到与多个第一特征点对应的多个第二特征点。
关于上述实施例中的装置，其中各个单元、模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
图5是根据一示例性实施例示出的一种用于检测移动目标的电子设备500的框图。电子设备500可以是移动电话、计算机、数字广播终端、消息收发 设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等。电子设备500可以包括以下一个或多个组件：处理组件502、存储器504、电源组件506、多媒体组件508、音频组件510、输入/输出(I/O)的接口512、传感器组件514、以及通信组件516。
处理组件502通常控制电子设备500的整体操作，诸如与显示、电话呼叫、数据通信、相机操作和记录操作相关联的操作。处理组件502可以包括一个或多个处理器520来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件502可以包括一个或多个模块，便于处理组件502和其他组件之间的交互。例如，处理组件502可以包括多媒体模块，以方便多媒体组件508和处理组件502之间的交互。
存储器504被配置为存储各种类型的数据以支持在电子设备500的操作。这些数据的示例包括用于在电子设备500上操作的任何应用程序或方法的指令、联系人数据、电话簿数据、消息、图片、视频等。存储器504可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件506为电子设备500的各种组件提供电力。电源组件506可以包括电源管理系统，一个或多个电源，及其他与电子设备500生成、管理和分配电力相关联的组件。
多媒体组件508包括在电子设备500和用户之间提供的一个输出接口的屏幕。在一些其他实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被设置为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与触摸或滑动操作相关的持续时间和压力。在一些其他实施例中，多媒体组件508包括一个前置摄像头和/或后置摄像头。当电子设备500处于操作模 式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件510被配置为输出和/或输入音频信号。例如，音频组件510包括一个麦克风(MIC)，当电子设备500处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器504或经由通信组件516发送。在一些其他实施例中，音频组件510还包括一个扬声器，用于输出音频信号。
I/O接口512为处理组件502和外围接口模块之间提供接口，上述外围接口模块可以是键盘、点击轮、按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件514包括一个或多个传感器，用于为电子设备500提供各个方面的状态评估。例如，传感器组件514可以检测到电子设备500的打开/关闭状态、组件的相对定位，例如所述组件为电子设备500的显示器和小键盘，传感器组件514还可以检测电子设备500或电子设备500一个组件的位置改变、用户与电子设备500接触的存在或不存在、电子设备500方位或加速/减速和电子设备500的温度变化。传感器组件514可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件514还可以包括光传感器，如CMOS或电荷耦合元件(Charge-coupled Device，CCD)图像传感器，用于在成像应用中使用。在一些其他实施例中，传感器组件514还可以包括加速度传感器、陀螺仪传感器、磁传感器、压力传感器或温度传感器等。
通信组件516被配置为便于电子设备500和其他设备之间有线或无线方式的通信。电子设备500可以接入基于通信标准的无线网络，如WiFi，运营商网络(如2G、3G、4G或5G)，或它们的组合。在一个示例性实施例中，通信组件516经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，通信组件516还包括近场通信(NFC)模 块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备500可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器504，上述指令可由电子设备500的处理器520执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
图6是根据一示例性实施例示出的一种用于检测移动目标的装置600的框图。例如，装置600可以被提供为一服务器或电子设备。如图6所示，装置600包括处理组件622，其进一步包括一个或多个处理器，以及由存储器632所代表的存储器资源，用于存储可由处理组件622的执行的指令，例如应用程序。存储器632中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件622被配置为执行指令，以执行上述移动目标的检测方法。
装置600还可以包括一个电源组件626被配置为执行装置600的电源管理，一个有线或无线网络接口650被配置为将装置600连接到网络，和一个输入输出(I/O)接口658。装置600可以操作基于存储在存储器632的操作系统，例如Windows ServerTM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTM或类似。
本申请实施例还可以提供一种计算机程序产品，当所述计算机程序产品中的指令由服务器、装置或电子设备的处理器执行时，使得服务器、装置或电子设备能够执行上述移动目标的检测方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本 申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (14)

  1. 一种移动目标的检测方法，包括：
    获取相邻的第一帧图像和第二帧图像，以及，所述第一帧图像和所述第二帧图像之间的旋转矩阵和平移矩阵，所述第一帧图像和所述第二帧图像均包含同一移动目标；
    从所述第一帧图像中提取得到多个第一特征点；
    根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点；
    根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，对应的多个所述极线为多个所述第一特征点在所述第二帧图像上的多个极线；
    根据多个所述距离筛选出位于所述移动目标上的多个第三特征点；
    根据多个所述第三特征点检测得到所述移动目标。
  2. 根据权利要求1所述的移动目标的检测方法，所述根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，包括：
    根据所述旋转矩阵和所述平移矩阵，确定多个所述第一特征点在所述第二帧图像上对应的多个所述极线；
    根据多个所述第二特征点和对应的多个所述极线在归一化的相机坐标系中的坐标值，计算多个所述第二特征点与对应的多个所述极线之间的多个所述距离。
  3. 根据权利要求1所述的移动目标的检测方法，所述根据多个所述距离筛选出位于所述移动目标上的多个第三特征点，包括：
    将多个所述距离分别与预设的距离阈值进行比较；
    将多个所述距离中大于所述距离阈值的距离对应的所述第二特征点确定为位于所述移动目标上的多个所述第三特征点。
  4. 根据权利要求1所述的移动目标的检测方法，所述根据多个所述第三特征点检测得到所述移动目标，包括：
    将多个所述第三特征点形成的最小外包矩形确定为所述移动目标所属的区域。
  5. 根据权利要求1至4中任一项所述的移动目标的检测方法，所述从所述第一帧图像中提取得到多个第一特征点，包括：
    根据FAST特征点检测算法从所述第一帧图像中提取得到多个所述第一特征点。
  6. 根据权利要求1至4中任一项所述的移动目标的检测方法，所述根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点，包括：
    根据光流算法从所述第二帧图像中提取得到与多个所述第一特征点对应的多个所述第二特征点。
  7. 一种移动目标的检测装置，包括：
    图像矩阵获取单元，被配置为获取相邻的第一帧图像和第二帧图像，以及，所述第一帧图像和所述第二帧图像之间的旋转矩阵和平移矩阵，所述第一帧图像和所述第二帧图像均包含同一移动目标；
    特征点提取单元，被配置为从所述第一帧图像中提取得到多个第一特征点；
    特征点确定单元，被配置为根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点；
    距离计算单元，被配置为根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，对应的多个所述极线为多个所述第一特征点在所述第二帧图像上的多个极线；
    特征点筛选单元，被配置为根据多个所述距离筛选出位于所述移动目标上的多个第三特征点；
    目标检测单元，被配置为根据多个所述第三特征点检测得到所述移动目 标。
  8. 一种电子设备，包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为：当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现下列过程：
    获取相邻的第一帧图像和第二帧图像，以及，所述第一帧图像和所述第二帧图像之间的旋转矩阵和平移矩阵，所述第一帧图像和所述第二帧图像均包含同一移动目标；
    从所述第一帧图像中提取得到多个第一特征点；
    根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点；
    根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，对应的多个所述极线为多个所述第一特征点在所述第二帧图像上的多个极线；
    根据多个所述距离筛选出位于所述移动目标上的多个第三特征点；
    根据多个所述第三特征点检测得到所述移动目标。
  9. 根据权利要求8所述的电子设备，所述根据所述旋转矩阵和所述平移矩阵计算多个所述第二特征点与对应的多个极线之间的多个距离，包括：
    根据所述旋转矩阵和所述平移矩阵，确定多个所述第一特征点在所述第二帧图像上对应的多个所述极线；
    根据多个所述第二特征点和对应的多个所述极线在归一化的相机坐标系中的坐标值，计算多个所述第二特征点与对应的多个所述极线之间的多个所述距离。
  10. 根据权利要求8所述的电子设备，所述根据多个所述距离筛选出位于所述移动目标上的多个第三特征点，包括：
    将多个所述距离分别与预设的距离阈值进行比较；
    将多个所述距离中大于所述距离阈值的距离对应的所述第二特征点确定为位于所述移动目标上的多个所述第三特征点。
  11. 根据权利要求8所述的电子设备，所述根据多个所述第三特征点检测得到所述移动目标，包括：
    将多个所述第三特征点形成的最小外包矩形确定为所述移动目标所属的区域。
  12. 根据权利要求8至11中任一项所述的电子设备，所述从所述第一帧图像中提取得到多个第一特征点，包括：
    根据FAST特征点检测算法从所述第一帧图像中提取得到多个所述第一特征点。
  13. 根据权利要求8至11中任一项所述的电子设备，所述根据所述第二帧图像和多个所述第一特征点，从所述第二帧图像中确定与多个所述第一特征点对应的多个第二特征点，包括：
    根据光流算法从所述第二帧图像中提取得到与多个所述第一特征点对应的多个所述第二特征点。
  14. 一种非临时性计算机可读存储介质，当所述存储介质中的指令由移动终端的处理器执行时，使得所述移动终端执行权利要求1至6中任一项所述的移动目标的检测方法。
PCT/CN2020/073327 2019-01-31 2020-01-20 移动目标的检测方法、装置及电子设备和存储介质 WO2020156341A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/138,452 US11176687B2 (en) 2019-01-31 2020-12-30 Method and apparatus for detecting moving target, and electronic equipment

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910101425.4 2019-01-31
CN201910101425.4A CN109902725A (zh) 2019-01-31 2019-01-31 移动目标的检测方法、装置及电子设备和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/138,452 Continuation US11176687B2 (en) 2019-01-31 2020-12-30 Method and apparatus for detecting moving target, and electronic equipment

Publications (1)

Publication Number Publication Date
WO2020156341A1 true WO2020156341A1 (zh) 2020-08-06

Family

ID=66944637

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/073327 WO2020156341A1 (zh) 2019-01-31 2020-01-20 移动目标的检测方法、装置及电子设备和存储介质

Country Status (3)

Country Link
US (1) US11176687B2 (zh)
CN (1) CN109902725A (zh)
WO (1) WO2020156341A1 (zh)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109902725A (zh) 2019-01-31 2019-06-18 北京达佳互联信息技术有限公司 移动目标的检测方法、装置及电子设备和存储介质
CN110490131B (zh) * 2019-08-16 2021-08-24 北京达佳互联信息技术有限公司 一种拍摄设备的定位方法、装置、电子设备及存储介质
CN110619664B (zh) * 2019-09-17 2023-06-27 武汉理工大学 基于激光图案辅助的摄像机距离姿态计算方法及服务器
CN110880187B (zh) * 2019-10-17 2022-08-12 北京达佳互联信息技术有限公司 一种相机位置信息确定方法、装置、电子设备及存储介质
CN112771575A (zh) * 2020-03-30 2021-05-07 深圳市大疆创新科技有限公司 距离确定方法、可移动平台及计算机可读存储介质
CN113822102B (zh) * 2020-06-19 2024-02-20 北京达佳互联信息技术有限公司 一种姿态估计方法、装置、电子设备及存储介质
CN111860224A (zh) * 2020-06-30 2020-10-30 北京百度网讯科技有限公司 图像处理的方法、装置、电子设备和计算机可读存储介质
CN111882583B (zh) * 2020-07-29 2023-11-14 成都英飞睿技术有限公司 一种运动目标检测方法、装置、设备及介质
CN112819889B (zh) * 2020-12-30 2024-05-10 浙江大华技术股份有限公司 位置信息的确定方法及装置、存储介质、电子装置
CN114323010B (zh) * 2021-12-30 2024-03-01 北京达佳互联信息技术有限公司 初始特征确定方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080137940A1 (en) * 2005-02-23 2008-06-12 Aisin Seiki Kabushiki Kaisha Object Recognition Apparatus and Object Recognition Method Using Epipolar Geometry
CN102243764A (zh) * 2010-05-13 2011-11-16 东软集团股份有限公司 运动特征点检测方法及装置
CN104428624A (zh) * 2012-06-29 2015-03-18 富士胶片株式会社 三维测定方法、装置及系统、以及图像处理装置
CN106504265A (zh) * 2015-09-08 2017-03-15 株式会社理光 运动估计优化方法、设备和系统
CN109902725A (zh) * 2019-01-31 2019-06-18 北京达佳互联信息技术有限公司 移动目标的检测方法、装置及电子设备和存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100446636B1 (ko) * 2002-11-21 2004-09-04 삼성전자주식회사 이동체의 움직임 및 이동체 전방에 위치한 물체의 3차원정보 측정 기능을 구비한 이동체 및 그 방법
WO2005081178A1 (en) * 2004-02-17 2005-09-01 Yeda Research & Development Co., Ltd. Method and apparatus for matching portions of input images
CN103810718B (zh) * 2012-11-15 2016-07-06 浙江大华技术股份有限公司 一种剧烈运动目标检测方法和装置
CN103745474B (zh) * 2014-01-21 2017-01-18 南京理工大学 基于惯性传感器和摄像机的图像配准方法
CN104197928B (zh) * 2014-08-29 2017-01-18 西北工业大学 多摄像机协同的无人机检测、定位及跟踪方法
CN104240267B (zh) * 2014-09-19 2017-06-27 南京理工大学 一种基于三维结构坐标约束的运动目标检测方法
US10019637B2 (en) * 2015-11-13 2018-07-10 Honda Motor Co., Ltd. Method and system for moving object detection with single camera
US9911198B2 (en) * 2015-12-17 2018-03-06 Canon Kabushiki Kaisha Method, system and apparatus for matching moving targets between camera views
EP3364336B1 (en) * 2017-02-20 2023-12-20 Continental Autonomous Mobility Germany GmbH A method and apparatus for estimating a range of a moving object
CN108109163A (zh) * 2017-12-18 2018-06-01 中国科学院长春光学精密机械与物理研究所 一种航拍视频的运动目标检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080137940A1 (en) * 2005-02-23 2008-06-12 Aisin Seiki Kabushiki Kaisha Object Recognition Apparatus and Object Recognition Method Using Epipolar Geometry
CN102243764A (zh) * 2010-05-13 2011-11-16 东软集团股份有限公司 运动特征点检测方法及装置
CN104428624A (zh) * 2012-06-29 2015-03-18 富士胶片株式会社 三维测定方法、装置及系统、以及图像处理装置
CN106504265A (zh) * 2015-09-08 2017-03-15 株式会社理光 运动估计优化方法、设备和系统
CN109902725A (zh) * 2019-01-31 2019-06-18 北京达佳互联信息技术有限公司 移动目标的检测方法、装置及电子设备和存储介质

Also Published As

Publication number Publication date
CN109902725A (zh) 2019-06-18
US20210118152A1 (en) 2021-04-22
US11176687B2 (en) 2021-11-16

Similar Documents

Publication Publication Date Title
WO2020156341A1 (zh) 移动目标的检测方法、装置及电子设备和存储介质
JP6043856B2 (ja) Ｒｇｂｄカメラを用いた頭部ポーズ推定
US9674395B2 (en) Methods and apparatuses for generating photograph
US9986155B2 (en) Image capturing method, panorama image generating method and electronic apparatus
CN111105454B (zh) 一种获取定位信息的方法、装置及介质
WO2023103377A1 (zh) 标定方法及装置、电子设备、存储介质及计算机程序产品
CN110853095B (zh) 相机定位方法、装置、电子设备及存储介质
WO2021031790A1 (zh) 一种信息处理方法、装置、电子设备、存储介质和程序
CN106503682B (zh) 视频数据中的关键点定位方法及装置
JP7167244B2 (ja) 遮蔽された画像の検出方法、装置、及び媒体
WO2022017140A1 (zh) 目标检测方法及装置、电子设备和存储介质
CN109241875B (zh) 姿态检测方法及装置、电子设备和存储介质
WO2023077754A1 (zh) 目标跟踪方法、装置及存储介质
KR20220123218A (ko) 타깃 포지셔닝 방법, 장치, 전자 기기, 저장 매체 및 프로그램
WO2023273498A1 (zh) 深度检测方法及装置、电子设备和存储介质
CN114581525A (zh) 姿态确定方法及装置、电子设备和存储介质
KR20200135998A (ko) 위치자세 검출 방법 및 장치, 전자 기기 및 저장 매체
JP2013179614A (ja) 撮像装置
CN108648238B (zh) 虚拟角色驱动方法及装置、电子设备和存储介质
CN114600162A (zh) 用于捕捉摄像机图像的场景锁定模式
CN114422736B (zh) 一种视频处理方法、电子设备及计算机存储介质
WO2023240489A1 (zh) 一种拍照方法、装置及存储介质
CN112070681B (zh) 图像处理方法及装置
CN110458962B (zh) 一种图像处理方法、装置、电子设备及存储介质
CN110276841B (zh) 应用于增强现实设备的运动轨迹确定方法、装置及终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20748477

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20748477

Country of ref document: EP

Kind code of ref document: A1