CN111753609B - Target identification method and device and camera - Google Patents
Target identification method and device and camera Download PDFInfo
- Publication number
- CN111753609B CN111753609B CN201910713004.7A CN201910713004A CN111753609B CN 111753609 B CN111753609 B CN 111753609B CN 201910713004 A CN201910713004 A CN 201910713004A CN 111753609 B CN111753609 B CN 111753609B
- Authority
- CN
- China
- Prior art keywords
- target
- depth image
- video frame
- image
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012544 monitoring process Methods 0.000 claims abstract description 18
- 238000013507 mapping Methods 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 15
- 238000013136 deep learning model Methods 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 241001465754 Metazoa Species 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The application provides a target identification method, a target identification device and a camera, wherein the method comprises the following steps: acquiring a video frame and a depth image of a monitoring area; detecting an object in motion from the video frame; when the target triggering preset event rule is tracked, acquiring a target video frame and a target depth image at the moment when the target triggers the event rule; identifying a target type of a target in the target video frame; determining a physical size of the target based on the target depth image; and determining whether the target is a specified target or not according to the physical size and the target type. According to the embodiment of the application, the targets with unreasonable sizes can be filtered by combining the depth map, the appointed targets can be accurately determined, the false alarm probability is reduced, and the accuracy of perimeter precaution is improved.
Description
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a method and apparatus for target recognition, and a camera.
Background
With the continuous progress of society, the application range of video monitoring systems is becoming wider and wider. The existing digital monitoring system can not meet the requirements of many application occasions, mainly reflected in the fact that the intelligent degree is not high enough, and is in a semi-automatic state, and many occasions also need manual intervention, for example, when abnormal conditions (moving targets and the like) are found, the targets can not be automatically identified, and because of hysteresis of manual operation, a lot of important information is lost. In order to solve the problems, the perimeter guard is applied, a moving target can be automatically detected through the perimeter guard, alarm information of the target is generated according to rules configured by a user, and relevant personnel are immediately notified to process the alarm information, so that the method is an active monitoring mode, and the practical application value of monitoring is greatly exerted.
The information of the man-made objects in the video is a concerned part in video monitoring, so that the detection and alarm of the human body are also a core function of perimeter precaution. The problems to be solved under the current technical framework are as follows:
1. the target at a far distance is inaccurate in detection, and whether the target is a real person or a vehicle or only one false alarm is difficult to distinguish.
2. The interference of leaves, lights, animals, rainwater and the like is large, and false alarms are easy to generate.
Disclosure of Invention
In view of the foregoing, the present application provides a method, an apparatus, and a camera for object recognition.
Specifically, the application is realized by the following technical scheme:
in a first aspect, an embodiment of the present application provides a method for identifying an object, where the method includes:
acquiring a video frame and a depth image of a monitoring area;
detecting an object in motion from the video frame;
when the target triggering preset event rule is tracked, acquiring a target video frame and a target depth image at the moment when the target triggers the event rule;
identifying a target type of a target in the target video frame;
determining a physical size of the target based on the target depth image;
and determining whether the target is a specified target or not according to the physical size and the target type.
Optionally, the detected object includes a circumscribed rectangular box of the object, and the identifying the object type of the object in the object video frame includes:
intercepting a local image containing the target from the target video frame, wherein the local image is intercepted by taking a circumscribed rectangular frame of the target as a boundary;
and inputting the local image into a trained deep learning model, carrying out target recognition on the local image by the deep learning model, and outputting the target type of the target.
Optionally, the determining the physical size of the target based on the target depth image includes:
mapping each pixel point of the circumscribed rectangular frame of the target in the target depth image to determine a corresponding mapping point of each pixel point in the target depth image;
acquiring point cloud data of each mapping point;
and calculating the boundary length of the circumscribed rectangular frame according to the point cloud data of the mapping points corresponding to the vertex pixel points of the circumscribed rectangular frame, and taking the boundary length as the physical size of the target.
Optionally, the determining whether the target is a specified target according to the physical size and the target type includes:
when the target type and the physical size are matched with the specified target, judging that the target is the specified target;
and when the object type is not matched with any of the physical dimensions and the specified object, judging that the object is not the specified object.
Optionally, detecting whether the target triggers a preset event rule by:
and if the circumscribed rectangle frame of the target is detected to be intersected with a preset warning line or a warning area, judging that the target frame object triggers a preset event rule.
Optionally, the method further comprises:
and triggering alarm processing after determining that the target is a specified target.
In a second aspect, embodiments of the present application provide an object recognition apparatus, including:
the image acquisition module is used for acquiring video frames and depth images of the monitoring area;
the target detection module is used for detecting a target in a motion state from the video frame;
the target image determining module is used for acquiring a target video frame and a target depth image at the moment when the target triggers the event rule when the target triggers the preset event rule is tracked;
the target type identification module is used for identifying the target type of the target in the target video frame;
a physical size determining module for determining a physical size of the target based on the target depth image;
and the target judging module is used for combining the physical size and the target type to determine whether the target is a specified target.
Optionally, the detected object includes a circumscribed rectangular box of the object, and the object type identification module includes:
a local image intercepting sub-module, configured to intercept a local image including the target from the target video frame, where the local image is intercepted with a circumscribed rectangular frame of the target as a boundary;
and the target type determining submodule is used for inputting the local image into a trained deep learning model so as to carry out target identification on the local image by the deep learning model and output the target type of the target.
Optionally, the physical size determining module includes:
the pixel point mapping sub-module is used for mapping each pixel point of the circumscribed rectangular frame of the target in the target depth image so as to determine a corresponding mapping point of each pixel point in the target depth image;
the point cloud data acquisition sub-module is used for acquiring point cloud data of each mapping point;
and the physical dimension calculation sub-module is used for calculating the boundary length of the circumscribed rectangular frame according to the point cloud data of the mapping points corresponding to the vertex pixel points of the circumscribed rectangular frame and taking the boundary length as the physical dimension of the target.
In a third aspect, embodiments of the present application provide a video camera comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method described above when the program is executed.
The embodiment of the application has the following beneficial effects:
the embodiment can acquire the video frame and the depth image of the monitoring area, detect the target in the motion state according to the video frame, and acquire the target video frame and the target depth image at the moment when the target triggers the event rule when the target triggers the preset event rule is tracked. The method comprises the steps of determining the target type of a target through a target video frame, determining the physical size of the target through a target depth image, finally combining the physical size of the target and the target type to determine whether the target is a specified target, combining a depth image can filter out the target with unreasonable size, accurately determining the specified target, reducing the false alarm probability and improving the accuracy of perimeter precaution.
Drawings
FIG. 1 is a flow chart illustrating steps of an embodiment of a method for object recognition according to an exemplary embodiment of the present application;
FIG. 2 is a schematic diagram of area intrusion event detection as shown in an exemplary embodiment of the present application;
FIG. 3 is a schematic diagram illustrating cross-line intrusion event detection according to an exemplary embodiment of the present application;
FIG. 4 is a hardware configuration diagram of the apparatus in which the device of the present application is located;
fig. 5 is a block diagram illustrating an embodiment of an object recognition apparatus according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first message may also be referred to as a second message, and similarly, a second message may also be referred to as a first message, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
Referring to fig. 1, a flowchart illustrating steps of an embodiment of a method for object recognition according to an exemplary embodiment of the present application may include the following steps:
step 101, obtaining a video frame and a depth image of a monitoring area.
In this step, the monitoring area may be a monitoring range of the camera, and the video frame may be acquired by an image sensor of the camera.
The gray value of each pixel of the Depth map may be used to characterize how far or near a point in the scene is from the camera. When the method is implemented, a general depth image acquisition mode can be adopted to acquire the depth image of the monitoring area. For example, the depth image of the monitoring area may be acquired as follows, but the embodiment is not limited thereto:
a passive ranging sensing method. One such passive ranging sensing method is binocular stereo vision, two images of the same scene are acquired simultaneously by two sensors separated by a certain distance, corresponding pixels in the two images are found by a stereo matching algorithm, parallax information is calculated according to the principle of trigonometry, and the parallax information can be used for representing depth information of objects in the scene by conversion. Based on the stereo matching algorithm, a depth image of the scene can also be obtained by taking a group of images of different angles under the same scene. In addition, the depth image of the scene can be obtained by analyzing and indirectly estimating the photometric features, the bright and dark features and other features of the image.
An active ranging sensing method. The difference between active ranging sensing and passive ranging sensing is: the device itself needs to transmit energy to complete the acquisition of depth information. This also ensures that the depth image is acquired independently of the color image. Methods of active ranging sensing mainly include TOF (Time of Flight), structured light, lidar scanning, etc. Wherein,
the principle of acquiring depth images by a TOF camera is as follows: by emitting successive near infrared pulses to the target scene, the light pulses reflected back by the object are then received by the sensor. By comparing the phase difference between the emitted light pulses and the light pulses reflected by the object, the transmission delay between the light pulses can be calculated, so that the distance between the object and the emitter can be obtained, and finally a depth image can be obtained.
The depth image acquisition principle based on structured light is: structured light is projected onto the scene and a corresponding pattern with structured light is captured by the image sensor. Because the pattern of the structured light deforms according to the shape of the object, the depth information of each point in the scene can be obtained by calculating the position and deformation degree of the pattern image in the captured image by using the triangle principle.
The laser radar ranging technology obtains three-dimensional information of a scene in a laser scanning mode. The basic principle is that laser is emitted to the space according to a certain time interval, signals of all scanning points are recorded, the signals reach objects in a tested scene from the laser radar, and then the distance between the surface of the object and the laser radar is calculated through the interval time that the objects reflect back to the laser radar.
It should be noted that, when acquiring the video frame and the depth image of the monitoring area, if at least two sensors are involved, the at least two sensors may be integrated into one camera or may be disposed in different cameras, which is not limited in this embodiment.
In one possible implementation scenario, if the at least two sensors are integrated in the same camera, this embodiment may be performed by the camera when the camera is mounted with a processing chip. When the camera is not processing a chip, different sensors in the camera may transmit the acquired data to the designated platform, which performs the present embodiment.
In another possible implementation scenario, when multiple sensors are deployed at different cameras, respectively, different ones of the different cameras may transmit the acquired data to a designated platform, which performs the present embodiment.
The embodiments of the present application may be applied to any of the above scenarios, but the embodiments of the present application are not limited thereto, and all embodiments according to the ideas of the present application are within the scope of protection of the present application.
Step 102, detecting an object in a motion state from the video frame.
For example, the detected object may include a circumscribed rectangular box of the object.
In one embodiment, the object in motion may be identified from the video frame by a method of moving object detection. The moving object detection refers to a process of effectively extracting an object generating spatial position change by subtracting redundant information in time and space in a video through a computer vision method. For example, a background model may be established, and then the background difference method is used to classify the pixels of the moving object and the background, so as to detect the object in the moving state. For example, a median method may be used to build the background model, i.e. using the median of the pixel values of a sequence of consecutive N frames of images as the background model. Or background modeling can be performed in a single Gaussian or mixed Gaussian mode, and whether the pixel points belong to the foreground is judged by utilizing a threshold value; alternatively, the pixel point is estimated by using a standard kernel function through the latest image sample information, and the moving object is extracted. Or, randomly selecting a certain number of pixel values for background modeling according to a certain rule for each pixel point, and classifying the foreground and the background for the pixel points by adopting Euclidean distance.
In one embodiment, after moving objects are detected, each object may be tracked to determine whether each object is a stable object, i.e., whether the object is in motion. For example, an object in motion may be detected in every video frame and have a stable displacement for the object.
And step 103, when the target triggering preset event rule is tracked, acquiring a target video frame and a target depth image at the moment when the target triggers the event rule.
In this step, during the tracking of the target, logic analysis may also be performed on the target to determine whether the target triggers a preset event rule.
In one possible implementation manner of this embodiment, the following manner may be adopted to determine whether the target triggers a preset event rule:
and if the circumscribed rectangle frame of the target is detected to be intersected with a preset warning line or a warning area, judging that the target frame object triggers a preset event rule.
By way of example, the preset event rules may include alarm events such as area intrusion, cross-line intrusion, etc.
For example, as shown in the area intrusion event detection schematic diagram of fig. 2, assuming that the warning area generated in advance is F1, the circumscribed rectangular frame of the target human body is F2, when the target human body starts to enter the limiting area F1, and when the F2 intersects with the F1, the circumscribed rectangular frame F2 is divided into A, B parts by the boundary of the F1, so that it can be determined that the position of the target human body is within the limiting area F1, and then it can be determined that the target triggers the event rule of the preset area intrusion event.
As another example, as shown in the cross line intrusion event detection schematic diagram of fig. 3, assuming that the pre-generated warning line is L, and the circumscribed rectangular frame of the target human body is F2, when the target human body crosses the warning line L, the warning line L intersects with F2, that is, the L divides the circumscribed rectangular frame F2 into two parts A, B, so that it can be determined that the position of the target human body triggers the warning line L, and then it can be determined that the target triggers the event rule of the preset cross line intrusion event.
In this embodiment, when it is determined that the target triggers the preset event rule, the target video frame and the target depth image at the moment of the target triggering event rule may be acquired.
In one possible implementation, the rule judging module may judge whether the target triggers a preset event rule, if the rule judging module judges that the target triggers the preset event rule, the rule judging module may send an alarm signal to the video frame identifying module and the depth image identifying module, the video frame identifying module may take a video frame obtained at the moment of receiving the alarm signal as a target video frame, and the depth image identifying module may take a depth image obtained at the moment of receiving the alarm signal as a target depth image.
Step 104, identifying a target type of a target in the target video frame.
In this step, after the target video frame is determined, target detection may be performed on the target video frame to determine a target type of the target.
In one example, the object type of the object may be determined by the video frame identification module based on the object video frame. As one example, the target type may include a human, motor vehicle, small animal, or the like type.
In one possible implementation of this embodiment, step 104 may include the following sub-steps:
and S11, intercepting a local image containing the target from the target video frame, wherein the local image is intercepted by taking a circumscribed rectangular frame of the target as a boundary.
In one example, after the video frame identification module determines the target video frame according to the alarm signal, a matting process may be performed, and the circumscribed rectangular frame containing the target is cut out from the target video frame, so as to obtain a local image.
And a substep S12, inputting the local image into a trained deep learning model to perform target recognition on the local image by the deep learning model, and outputting the target type of the target.
In this step, after the partial image is cut out according to the circumscribed rectangular frame of the target, the partial image may be input into a trained deep learning model, and target recognition may be performed on the partial image by the deep learning model, so as to output a target type of the target as a person, a motor vehicle, a small animal, or a false alarm.
Step 105, determining a physical size of the target based on the target depth image.
In this step, after the target depth image is determined, the physical size of the target may be determined from the target depth image.
In one example, the physical size of the target may be determined based on the target depth image by a depth image recognition module.
In one possible implementation of the present embodiment, step 105 may include the following sub-steps:
and S21, mapping each pixel point of the circumscribed rectangular frame of the target in the target depth image to determine a corresponding mapping point of each pixel point in the target depth image.
In implementation, the video frame and the depth image corresponding to the same moment are accurate, and the pixels of the video frame and the depth image are mapped one by one, that is, for one pixel on the depth image, one pixel on the color image (i.e., the video frame) can be found to correspond to the pixel.
For the present embodiment, the target video frame and the target depth image are also aligned with each other, and correspond to each other on the pixels one by one. After the circumscribed rectangle frame of the target is determined in the target video frame, according to the positions of the pixel points in the circumscribed rectangle frame of the target, the pixel point corresponding to each pixel point can be searched in the target depth image and used as a mapping point corresponding to the pixel point in the target depth image.
In one embodiment, each pixel in the circumscribed rectangular frame of the object may include all pixels within the circumscribed rectangular frame and pixels at the boundary of the circumscribed rectangular frame. In other embodiments, in order to reduce the calculated data amount, the mapping process may be performed by taking only the pixels that circumscribe the boundary of the rectangular frame.
In the substep S22, point cloud data of each mapping point is acquired.
In one possible implementation, the target depth image may first be converted into a point cloud image.
The point cloud image is a massive set of points which are characterized by the surface of a target, each point in the point cloud image represents the position of the actual space geometry, each point comprises three-dimensional coordinates, and some points possibly also comprise color information (RGB) or reflection Intensity information (Intensity).
In this embodiment, a general method may be employed to convert the depth image into a point cloud image. In one implementation, the internal parameters and the external parameters of the camera can be combined, and the corresponding three-dimensional point cloud image is calculated according to the depth image, so that three-dimensional coordinate information of any position is obtained.
By way of example, one of the conversion processes may be:
the depth image is a depth value matrix of each pixel, and the depth value matrix is two-dimensionally arranged, and the two dimensions represent the number of rows and columns of the pixel in the depth image. If the camera cannot collect the depth value of a certain pixel, the depth value of the pixel is set to a specific value, such as 0. The point cloud coordinates of each pixel point in the depth image can be calculated according to the internal parameters of the camera, and the point cloud coordinates are three-dimensional coordinates and can be expressed as (x, y, z).
Specifically, a UV rectangular coordinate system may be defined in the depth image, where the pixel coordinates (u, v) of each pixel point in the depth image respectively represent the number of rows and columns of the pixel point in the depth image. The origin O of the UV rectangular coordinate system represents the intersection point of the camera optical axis and the plane in which the depth image is located, and the coordinates of the origin O in the UV rectangular coordinate system are (u) 0 ,v 0 ) The physical dimensions of each pixel point in the direction u and the direction v are dx and dy, and the focal length of the camera lens is f. Generally u will 0 、v 0 F/dx and f/dy are referred to as camera references.
After determining the pixel coordinates (u, v) of each pixel point and the depth value z of the pixel point in the depth image, the following formula may be used to calculate x and y corresponding to each pixel point, where the formula is: x=z (u-u) 0 )*dx/f；y＝z*(v-v 0 )*dy/f。
Further, point cloud coordinates (x, y, z) of each pixel point in the depth image may be determined.
In other embodiments, in order to save the calculation amount, the point cloud data corresponding to each mapping point of the circumscribed rectangular frame may be directly calculated according to the above-mentioned manner of calculating the point cloud coordinates, without calculating the point cloud map of the entire target depth image.
And S23, calculating the boundary length of the circumscribed rectangular frame according to the point cloud data of the mapping points corresponding to the vertex pixel points of the circumscribed rectangular frame, and taking the boundary length as the physical size of the target.
In this step, after the point cloud data of the mapping point corresponding to each pixel point of the circumscribed rectangular frame of the target is obtained, the boundary length of the circumscribed rectangular frame may be calculated according to the point cloud data corresponding to the vertex pixel point located at the vertex of the circumscribed rectangular frame, for example, the width and the height of the circumscribed rectangular frame may be calculated as the physical size of the target.
And 106, determining whether the target is a specified target or not according to the physical size and the target type.
In one possible implementation of this embodiment, step 106 may include the following sub-steps:
when the target type and the physical size are matched with the specified target, judging that the target is the specified target; and when the object type is not matched with any of the physical dimensions and the specified object, judging that the object is not the specified object.
For example, assuming that the target type of the judgment target is a human body, but its physical size is clearly different from the size of the human body, and assuming that the size of the motor vehicle is the size, it may be determined that the current target is not the human body. On the contrary, if the target type of the target is determined to be a human body and its physical size is matched with the size of the human body, it can be determined that the current target is a human body.
In one possible implementation manner of the embodiment, the method may further include the following steps:
and triggering alarm processing after determining that the target is a specified target.
In this step, when the target is determined to be the specified target, alarm processing may be performed, for example, an alarm signal such as an alarm sound is sent, or alarm information is sent to related personnel, etc., so as to implement a function of perimeter precaution.
The embodiment can acquire the video frame and the depth image of the monitoring area, detect the target in the motion state according to the video frame, and acquire the target video frame and the target depth image at the moment when the target triggers the event rule when the target triggers the preset event rule is tracked. The method comprises the steps of determining the target type of a target through a target video frame, determining the physical size of the target through a target depth image, finally combining the physical size of the target and the target type to determine whether the target is a specified target, combining a depth image can filter out the target with unreasonable size, accurately determining the specified target, reducing the false alarm probability and improving the accuracy of perimeter precaution.
Corresponding to the embodiments of the method described above, the present application also provides an embodiment of the object recognition device.
The device embodiment of the application can be applied to electronic equipment such as radar or cameras. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking a software implementation as an example, the device in a logic sense is formed by reading corresponding computer program instructions in a nonvolatile memory into a memory by a processor of a device where the device is located for operation. In terms of hardware, as shown in fig. 4, a hardware structure diagram of a device where an apparatus of the present application is located is shown in fig. 4, and in addition to a processor, a memory, a network interface, and a nonvolatile memory shown in fig. 4, the device where the apparatus is located in an embodiment generally may include other hardware according to an actual function of the apparatus, which is not described herein again.
Referring to fig. 5, a block diagram illustrating an embodiment of an object recognition device according to an exemplary embodiment of the present application may specifically include the following modules:
the image acquisition module 501 is configured to acquire a video frame and a depth image of a monitored area;
an object detection module 502, configured to detect an object in a motion state from the video frame;
a target image determining module 503, configured to acquire a target video frame and a target depth image at a moment when the target triggers a preset event rule when the target triggers the event rule;
a target type identifying module 504, configured to identify a target type of a target in the target video frame;
a physical size determining module 505, configured to determine a physical size of the target based on the target depth image;
and a target determining module 506, configured to determine whether the target is a specified target in combination with the physical size and the target type.
In one possible implementation manner of this embodiment, the detected object includes a circumscribed rectangular frame of the object, and the object type identifying module 504 may include the following sub-modules:
a local image intercepting sub-module, configured to intercept a local image including the target from the target video frame, where the local image is intercepted with a circumscribed rectangular frame of the target as a boundary;
and the target type determining submodule is used for inputting the local image into a trained deep learning model so as to carry out target identification on the local image by the deep learning model and output the target type of the target.
In one possible implementation manner of this embodiment, the physical dimension determining module includes:
the pixel point mapping sub-module is used for mapping each pixel point of the circumscribed rectangular frame of the target in the target depth image so as to determine a corresponding mapping point of each pixel point in the target depth image;
the point cloud data acquisition sub-module is used for acquiring point cloud data of each mapping point;
and the physical dimension calculation sub-module is used for calculating the boundary length of the circumscribed rectangular frame according to the point cloud data of the mapping points corresponding to the vertex pixel points of the circumscribed rectangular frame and taking the boundary length as the physical dimension of the target.
In one possible implementation manner of this embodiment, the target determining module 506 is specifically configured to:
when the target type and the physical size are matched with the specified target, judging that the target is the specified target;
and when the object type is not matched with any of the physical dimensions and the specified object, judging that the object is not the specified object.
In one possible implementation manner of this embodiment, the following manner is adopted to detect whether the target triggers a preset event rule:
and if the circumscribed rectangle frame of the target is detected to be intersected with a preset warning line or a warning area, judging that the target frame object triggers a preset event rule.
In a possible implementation manner of this embodiment, the apparatus further includes:
and the alarm module is used for triggering alarm processing after determining that the target is the designated target.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present application. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
The embodiment of the application also provides a camera, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the embodiment of the method when executing the program.
Embodiments of the subject matter and the functional operations described in this specification can be implemented in: digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and structural equivalents thereof, or a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions may be encoded on a manually-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode and transmit information to suitable receiver apparatus for execution by data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
Computers suitable for executing computer programs include, for example, general purpose and/or special purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read only memory and/or a random access memory. The essential elements of a computer include a central processing unit for carrying out or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks, etc. However, a computer does not have to have such a device. Furthermore, the computer may be embedded in another device, such as a vehicle-mounted terminal, a mobile phone, a Personal Digital Assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) flash drive, to name a few.
Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disk or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features of specific embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. On the other hand, the various features described in the individual embodiments may also be implemented separately in the various embodiments or in any suitable subcombination. Furthermore, although features may be acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Furthermore, the processes depicted in the accompanying drawings are not necessarily required to be in the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.
The foregoing description of the preferred embodiments of the present invention is not intended to limit the invention to the precise form disclosed, and any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. A method of object recognition, the method comprising:
acquiring a video frame and a depth image of a monitoring area;
detecting an object in motion from the video frame;
when the target triggering preset event rule is tracked, acquiring a target video frame and a target depth image at the moment when the target triggers the event rule;
identifying a target type of a target in the target video frame;
determining a physical size of the target based on the target depth image;
determining whether the target is a specified target by combining the physical size and the target type;
wherein the detected object includes a circumscribed rectangular box of the object, and the identifying the object type of the object in the object video frame includes:
intercepting a local image containing the target from the target video frame, wherein the local image is intercepted by taking a circumscribed rectangular frame of the target as a boundary;
and inputting the local image into a trained deep learning model, carrying out target recognition on the local image by the deep learning model, and outputting the target type of the target.
2. The method of claim 1, wherein the determining the physical size of the target based on the target depth image comprises:
mapping each pixel point of the circumscribed rectangular frame of the target in the target depth image to determine a corresponding mapping point of each pixel point in the target depth image;
acquiring point cloud data of each mapping point;
and calculating the boundary length of the circumscribed rectangular frame according to the point cloud data of the mapping points corresponding to the vertex pixel points of the circumscribed rectangular frame, and taking the boundary length as the physical size of the target.
3. The method according to claim 1 or 2, wherein said determining whether the target is a specified target based on the physical size and the target type comprises:
when the target type and the physical size are matched with the specified target, judging that the target is the specified target;
and when the object type is not matched with any of the physical dimensions and the specified object, judging that the object is not the specified object.
4. The method of claim 1, wherein detecting whether the target triggers a preset event rule is performed by:
and if the circumscribed rectangle frame of the target is detected to be intersected with a preset warning line or a warning area, judging that the target frame object triggers a preset event rule.
5. The method according to claim 1 or 2 or 4, wherein the method further comprises:
and triggering alarm processing after determining that the target is a specified target.
6. An object recognition apparatus, characterized in that the apparatus comprises:
the image acquisition module is used for acquiring video frames and depth images of the monitoring area;
the target detection module is used for detecting a target in a motion state from the video frame;
the target image determining module is used for acquiring a target video frame and a target depth image at the moment when the target triggers the event rule when the target triggers the preset event rule is tracked;
the target type identification module is used for identifying the target type of the target in the target video frame;
a physical size determining module for determining a physical size of the target based on the target depth image;
the target judging module is used for combining the physical size and the target type to determine whether the target is a specified target or not;
wherein the detected object includes a circumscribed rectangular frame of the object, and the object type identification module includes:
a local image intercepting sub-module, configured to intercept a local image including the target from the target video frame, where the local image is intercepted with a circumscribed rectangular frame of the target as a boundary;
and the target type determining submodule is used for inputting the local image into a trained deep learning model so as to carry out target identification on the local image by the deep learning model and output the target type of the target.
7. The apparatus of claim 6, wherein the physical size determining module comprises:
the pixel point mapping sub-module is used for mapping each pixel point of the circumscribed rectangular frame of the target in the target depth image so as to determine a corresponding mapping point of each pixel point in the target depth image;
the point cloud data acquisition sub-module is used for acquiring point cloud data of each mapping point;
and the physical dimension calculation sub-module is used for calculating the boundary length of the circumscribed rectangular frame according to the point cloud data of the mapping points corresponding to the vertex pixel points of the circumscribed rectangular frame and taking the boundary length as the physical dimension of the target.
8. A video camera comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1-5 when the program is executed by the processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910713004.7A CN111753609B (en) | 2019-08-02 | 2019-08-02 | Target identification method and device and camera |
| PCT/CN2020/106202 WO2021023106A1 (en) | 2019-08-02 | 2020-07-31 | Target recognition method and apparatus, and camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910713004.7A CN111753609B (en) | 2019-08-02 | 2019-08-02 | Target identification method and device and camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111753609A CN111753609A (en) | 2020-10-09 |
| CN111753609B true CN111753609B (en) | 2023-12-26 |
Family
ID=72672694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910713004.7A Active CN111753609B (en) | 2019-08-02 | 2019-08-02 | Target identification method and device and camera |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111753609B (en) |
| WO (1) | WO2021023106A1 (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112530021B (en) * | 2020-12-24 | 2023-06-23 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing data |
| CN112995432B (en) * | 2021-02-05 | 2022-08-05 | 杭州叙简科技股份有限公司 | Depth image identification method based on 5G double recorders |
| CN113591777B (en) * | 2021-08-11 | 2023-12-08 | 宁波未感半导体科技有限公司 | Laser radar signal processing method, electronic equipment and storage medium |
| CN113974933B (en) * | 2021-08-18 | 2024-03-12 | 宁波星巡智能科技有限公司 | Infant quilt kicking detection method and device, electronic equipment and medium |
| CN113869201A (en) * | 2021-09-27 | 2021-12-31 | 杭州海康威视系统技术有限公司 | Image identification method and electronic equipment |
| CN114764963B (en) * | 2021-10-12 | 2023-05-12 | 青岛民航凯亚系统集成有限公司 | Multi-defense area airport intelligent enclosure monitoring system and method |
| CN113963029A (en) * | 2021-10-29 | 2022-01-21 | 深圳市商汤科技有限公司 | Track splicing and event detection method, device, equipment and computer storage medium |
| CN114327341A (en) * | 2021-12-31 | 2022-04-12 | 江苏龙冠影视文化科技有限公司 | Remote interactive virtual display system |
| CN115511807B (en) * | 2022-09-16 | 2023-07-28 | 北京远舢智能科技有限公司 | Method and device for determining position and depth of groove |
| CN116469025B (en) * | 2022-12-30 | 2023-11-24 | 以萨技术股份有限公司 | Processing method for identifying task, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103761505A (en) * | 2013-12-18 | 2014-04-30 | 微软公司 | Object tracking embodiments |
| CN107289855A (en) * | 2016-04-13 | 2017-10-24 | 西克股份有限公司 | For the method and system for the size for measuring destination object |
| WO2018095082A1 (en) * | 2016-11-28 | 2018-05-31 | 江苏东大金智信息系统有限公司 | Rapid detection method for moving target in video monitoring |
| CN108507541A (en) * | 2018-03-01 | 2018-09-07 | 广东欧珀移动通信有限公司 | Building recognition method and system and mobile terminal |
| CN109357630A (en) * | 2018-10-30 | 2019-02-19 | 南京工业大学 | A kind of polymorphic type batch workpiece vision measurement system and method |
| CN109794948A (en) * | 2019-03-04 | 2019-05-24 | 北京国电富通科技发展有限责任公司 | Distribution network live line work robot and recognition positioning method |
| CN110014426A (en) * | 2019-03-21 | 2019-07-16 | 同济大学 | A method of utilizing low precision depth camera high-precision crawl symmetrical shape workpiece |
| CN110059676A (en) * | 2019-04-03 | 2019-07-26 | 北京航空航天大学 | A kind of aviation plug hole location recognition methods based on deep learning Yu multiple target distribution sorting |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7583815B2 (en) * | 2005-04-05 | 2009-09-01 | Objectvideo Inc. | Wide-area site-based video surveillance system |
| JP4541316B2 (en) * | 2006-04-06 | 2010-09-08 | 三菱電機株式会社 | Video surveillance search system |
| CN104821056B (en) * | 2015-04-30 | 2018-03-20 | 湖南华诺星空电子技术有限公司 | Intelligent warning method based on radar and video fusion |
| CN105336074A (en) * | 2015-10-28 | 2016-02-17 | 小米科技有限责任公司 | Alarm method and device |
| CN106919895B (en) * | 2016-07-01 | 2020-03-27 | 湖南拓视觉信息技术有限公司 | Tracking method and system for moving object |
| CN108527940B (en) * | 2018-04-12 | 2020-01-21 | 曹芸畅 | Method for manufacturing packaging box |
| CN109919009A (en) * | 2019-01-24 | 2019-06-21 | 北京明略软件系统有限公司 | The monitoring method of target object, apparatus and system |
-
2019
- 2019-08-02 CN CN201910713004.7A patent/CN111753609B/en active Active
-
2020
- 2020-07-31 WO PCT/CN2020/106202 patent/WO2021023106A1/en active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103761505A (en) * | 2013-12-18 | 2014-04-30 | 微软公司 | Object tracking embodiments |
| CN107289855A (en) * | 2016-04-13 | 2017-10-24 | 西克股份有限公司 | For the method and system for the size for measuring destination object |
| WO2018095082A1 (en) * | 2016-11-28 | 2018-05-31 | 江苏东大金智信息系统有限公司 | Rapid detection method for moving target in video monitoring |
| CN108507541A (en) * | 2018-03-01 | 2018-09-07 | 广东欧珀移动通信有限公司 | Building recognition method and system and mobile terminal |
| CN109357630A (en) * | 2018-10-30 | 2019-02-19 | 南京工业大学 | A kind of polymorphic type batch workpiece vision measurement system and method |
| CN109794948A (en) * | 2019-03-04 | 2019-05-24 | 北京国电富通科技发展有限责任公司 | Distribution network live line work robot and recognition positioning method |
| CN110014426A (en) * | 2019-03-21 | 2019-07-16 | 同济大学 | A method of utilizing low precision depth camera high-precision crawl symmetrical shape workpiece |
| CN110059676A (en) * | 2019-04-03 | 2019-07-26 | 北京航空航天大学 | A kind of aviation plug hole location recognition methods based on deep learning Yu multiple target distribution sorting |
Non-Patent Citations (1)
| Title |
|---|
| 基于多尺度卷积的船舶行为识别方法;王立林 等;《计算机应用》;第39卷(第12期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021023106A1 (en) | 2021-02-11 |
| CN111753609A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111753609B (en) | Target identification method and device and camera | |
| CN110163904B (en) | Object labeling method, movement control method, device, equipment and storage medium | |
| CN110988912B (en) | Road target and distance detection method, system and device for automatic driving vehicle | |
| CN111160302B (en) | Obstacle information identification method and device based on automatic driving environment | |
| CN110147706B (en) | Obstacle recognition method and device, storage medium, and electronic device | |
| US9165190B2 (en) | 3D human pose and shape modeling | |
| JP2023523243A (en) | Obstacle detection method and apparatus, computer device, and computer program | |
| US10643472B2 (en) | Monitor apparatus and monitor system | |
| JP5922257B2 (en) | Vehicle periphery monitoring device | |
| CN110799989A (en) | Obstacle detection method, equipment, movable platform and storage medium | |
| CN110442120B (en) | Method for controlling robot to move in different scenes, robot and terminal equipment | |
| CN110865393A (en) | Positioning method and system based on laser radar, storage medium and processor | |
| CN102792314A (en) | Cross traffic collision alert system | |
| KR20200095888A (en) | Method for context awareness of unmanned ship system and apparatus for the same | |
| Nair | Camera-based object detection, identification and distance estimation | |
| CN111913177A (en) | Method and device for detecting target object and storage medium | |
| WO2022067647A1 (en) | Method and apparatus for determining pavement elements | |
| CN114359714A (en) | Unmanned body obstacle avoidance method and device based on event camera and intelligent unmanned body | |
| CN112562005A (en) | Space calibration method and system | |
| CN112683228A (en) | Monocular camera ranging method and device | |
| KR20170106823A (en) | Image processing device identifying object of interest based on partial depth map | |
| TW202029134A (en) | Driving detection method, vehicle and driving processing device | |
| CN112733678A (en) | Ranging method, ranging device, computer equipment and storage medium | |
| CN115792945B (en) | Floating obstacle detection method and device, electronic equipment and storage medium | |
| Hadi et al. | Fusion of thermal and depth images for occlusion handling for human detection from mobile robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |