CN112734795A - Method and equipment for judging motion trend and direction of object - Google Patents
Method and equipment for judging motion trend and direction of object Download PDFInfo
- Publication number
- CN112734795A CN112734795A CN202011632905.2A CN202011632905A CN112734795A CN 112734795 A CN112734795 A CN 112734795A CN 202011632905 A CN202011632905 A CN 202011632905A CN 112734795 A CN112734795 A CN 112734795A
- Authority
- CN
- China
- Prior art keywords
- motion
- image
- detection frame
- moving
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 204
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000001514 detection method Methods 0.000 claims abstract description 55
- 230000008569 process Effects 0.000 claims abstract description 23
- 230000011218 segmentation Effects 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 230000000007 visual effect Effects 0.000 claims description 18
- 230000015654 memory Effects 0.000 claims description 17
- 108091026890 Coding region Proteins 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 5
- 150000001875 compounds Chemical class 0.000 claims description 3
- 230000007774 longterm Effects 0.000 abstract description 8
- 239000000284 extract Substances 0.000 abstract description 2
- 230000004438 eyesight Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- 208000010415 Low Vision Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000004303 low vision Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and equipment for judging the motion trend and direction of an object, which solve the technical problem that the current and future motion trends and directions of a long-term moving object are difficult to judge in the prior art. The method comprises the following steps: object motion prediction process: and obtaining an object motion state sequence by repeatedly executing an object motion detection process, judging and outputting the current and future motion trends and directions of the object by using an object motion prediction model, and updating the motion prediction of the detected object in real time. Wherein, the object motion detection process comprises: detecting and tracking a moving object by using a detection frame; segmenting the image in the detection frame and acquiring an object region gray image; and extracting the motion state information of the object according to the gray-scale image of the object region and coding the motion state information of the object. The invention can extract the long-term motion state information of the moving object, judge the motion trend and direction of the object in the current and future period of time in an all-round way, and provide important basis for accurate obstacle avoidance early warning.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a method and equipment for judging the motion trend and direction of an object.
Background
The blind or visually impaired people make one of the special groups in the social group difficult to freely cope with in working, learning or other activities like the general population due to the visual disability. In China, the blind people are a large group, 1350 tens of thousands of blind people with visual disabilities exist in China, about 550 thousands of blind people account for 18 percent of the total number of the blind people in the world, and the rest 880 thousands of blind people are low-vision people. The safe trip of the blind is always a topic which is concerned by the society. The existing blind guiding tool has the advantages that the blind guiding stick is narrow in function range, objects or obstacles at a position far away cannot be detected, the number of blind guiding dogs is small, the training period is long, and the cultivation cost is high. Therefore, the blind guiding instrument based on the modern computer vision instrument and the intelligent decision system is beneficial to solving the social problem of the safety of the blind in going.
The current blind guiding instrument based on computer vision utilizes a target detection algorithm to collect surrounding target information for obstacle avoidance decision, and has the problem that the existing target detection algorithm cannot realize long-term attention to a detected object. In fact, the detected object in the current frame of the target detection algorithm is unrelated to the detected object in the next or previous frame. The method has the following consequences that the current blind guiding instrument based on computer vision can only judge static or low-speed objects in a short distance and carry out frequent obstacle avoidance alarm, is difficult to carry out complex obstacle avoidance planning and danger alarm based on long-term motion information of the objects, and lacks the judging and early warning capability of high-speed objects at far positions. This clearly limits its applicable scenarios and practical utility.
Disclosure of Invention
In view of the foregoing problems, embodiments of the present invention provide a method and an apparatus for determining a motion trend and a direction of an object, so as to solve the technical problem that it is difficult to determine a current motion trend and a future motion trend and a direction of a long-term moving object in the prior art.
In a first aspect, an embodiment of the present invention provides a method for determining a movement trend and a direction of an object, including:
object motion prediction process: analyzing a section of video image by repeatedly executing an object motion detection process to obtain an object motion state sequence, inputting the object motion state sequence into an object motion prediction model, and judging and outputting the current and future motion trends and directions of the object by using the object motion prediction model;
repeatedly executing the motion prediction process, and updating the motion prediction of the detected object in real time;
the object motion detection process includes:
acquiring a video image in a visual field range in real time, tracking a moving object detected from the video image and marking the moving object by a detection frame;
segmenting the image in each detection frame in real time and acquiring an object area gray level image;
and extracting the motion state information of the object according to the gray-scale image of the object region and coding the motion state information of the object.
In one embodiment of the present invention, the tracking of the object detected as moving from the video image and the marking with the detection frame comprises:
and tracking a moving object appearing in the video image in real time through an SORT-Yolov3 algorithm or a DeepsORT algorithm and marking the moving object by using a unique detection box.
In an embodiment of the present invention, segmenting the image in each detection frame in real time and acquiring the gray-scale image of the object region includes:
segmenting a foreground pixel class and a background pixel class from the image in each detection frame to obtain a binary segmentation image;
calculating the mass center of the segmentation graph to generate a two-dimensional Gaussian distribution image taking the mass center coordinate as the center;
and multiplying the segmentation image with the Gaussian distribution image to obtain an object region gray level image.
In an embodiment of the present invention, the segmenting the image in each detection frame into foreground pixel classes and background pixel classes to obtain a segmentation map to be binarized includes:
setting a threshold value t, defining the pixels of the image in the detection frame which are smaller than the threshold value as foreground pixel classes, defining the pixels of the image in the detection frame which are larger than the threshold value as background pixel classes, and defining the variance of the image after binarization as,
in the formula (I), the compound is shown in the specification,
representing the variance, ω, of the image after binarization1(t) represents the probability of occurrence of foreground pixel classes, ω2(t) represents the probability of the background pixel class appearing, μ1(t) represents the mean gray value of the foreground pixels, μ2(t) represents the foreground pixel mean gray value;
wherein, ω is1(t) is calculated by the following formula,
wherein p (i) represents a value obtained by normalizing the gradation histogram;
μ1(t) is calculated by the following formula,
in the formula, x (i) represents the value of the center of the ith gradation histogram bin, and ω can be found for bins larger than the threshold value in the same manner as above2(t) and μ2(t), traversing all possible thresholds t, enabling
The threshold value reaching the maximum value is set as a segmentation threshold value, and the image in the detection frame is segmented by the segmentation threshold value.
In an embodiment of the present invention, the extracting the object motion state information according to the object region gray scale map includes:
creating an object motion history map according to the object region gray scale map;
defining an object motion history map as
In the formula, Hτ(x, y, t) is the current object motion history map,
the gray scale image of the object region, tau is a parameter describing the time range of motion, delta is an attenuation parameter of the object motion history image, and xi is an object region binary segmentation threshold.
In an embodiment of the present invention, the creating an object motion history map according to the object region gray scale map includes:
judging whether the object appears in the visual field range for the first time;
if the object appears for the first time, a new object motion history map is created;
if the object does not appear for the first time, updating the object motion history map;
a motion history map of the object when the object is lost from view.
In an embodiment of the present invention, the encoding the object motion state information includes:
performing pixel gradient vector calculation on the object motion historical image to obtain a motion vector of a moving object and a vector horizontal included angle;
encoding the motion vector modulo-1 to + 1;
and normalizing the value of the horizontal included angle of the vector and coding in a 360-degree horizontal direction space.
In an embodiment of the present invention, repeatedly performing an object motion detection process to analyze a video image to obtain an object motion state sequence, inputting the object motion state sequence into an object motion prediction model, and determining and outputting a current and future motion trend and direction of an object by using the object motion prediction model includes:
acquiring a motion vector coding sequence according to the coded motion vector;
acquiring a motion direction coding sequence according to the coded vector horizontal included angle;
inputting the motion vector coding sequence into a trained first hidden Markov model, performing model inference by using a Viterbi algorithm, and judging whether the motion trend of the object is forward or backward;
and inputting the motion direction coding sequence into a trained second hidden Markov model, performing model inference by using a Viterbi algorithm, and judging whether the horizontal motion direction of the object is leftward or rightward.
In an embodiment of the present invention, the performing the motion prediction process multiple times, and updating the motion prediction of the detected object in real time includes:
formulating respective unique codes corresponding to moving objects appearing in the visual field;
maintaining a key value table according to the unique codes, wherein the unique codes in the key value table are index keys, and the values in the key value table are motion history maps of the moving objects;
the object motion prediction model updates the motion prediction of the detected object in real time according to the key value table.
In a second aspect, the present invention provides an apparatus for determining a movement trend and a direction of an object, including:
a processor, a memory, an interface to communicate with a gateway;
the memory is used for storing programs and data, and the processor calls the programs stored in the memory to execute the method for judging the movement trend and direction of the object provided by any one of the first aspect.
From the above description, it can be seen that the method for determining the motion trend and direction of an object according to the embodiment of the present invention continuously tracks all moving objects in a field of view, extracts long-term motion state information of the moving objects, inputs the information into an object motion prediction model, and accurately determines the motion trend and direction of the object in a current period and a future period of time in an all-around manner by using the object motion prediction model, so that the object appearing or existing in the field of view for the first time can be captured and continuously tracked in the first time no matter how close or far the object is, and further determines whether the object can hinder the movement of the object in the future period of time.
Drawings
Fig. 1 is a flowchart illustrating a method for determining a movement trend and a direction of an object according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of an apparatus for determining a movement trend and a direction of an object according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer and more obvious, the present invention is further described below with reference to the accompanying drawings and the detailed description. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 shows a method for determining a movement trend and a direction of an object according to an embodiment of the present invention. In fig. 1, the present embodiment includes:
s1: object motion prediction process: analyzing a section of video image by repeatedly executing an object motion detection process to obtain an object motion state sequence, inputting the object motion state sequence into an object motion prediction model, and judging and outputting the current and future motion trends and directions of the object by using the object motion prediction model.
In particular, from the code containing the object motion state information, a sequence of object motion states may be generated over time. The object motion state sequence is used as the input data of an object motion prediction model, and the object motion prediction model can be a hidden Markov model, an RNN (recurrent neural network) model or an LSTM (long short term memory network) model. The object motion prediction model is combined with a dynamic planning algorithm to judge the current object motion trend and direction, and predict future object motion and trend in a forward-looking manner.
S2: and repeatedly executing the motion prediction process, and updating the motion prediction of the detected object in real time.
Specifically, objects moving within the field of view are changing in real time, both moving objects that newly appear in the field of view and moving objects that disappear in the field of view. The moving object in the visual field range can be detected in real time and the moving trend and direction of the moving object can be updated in real time by executing the motion prediction process for multiple times.
The object motion detection process comprises the following steps:
s3: video images in the visual field range are acquired in real time, and moving objects are detected from the video images and marked by detection frames.
Specifically, for a moving object existing in a visual field range, a monitoring camera can be used for acquiring a video image of the moving object in real time and tracking the moving object. The detection of all moving objects is finished before tracking, and a plurality of moving objects which are identified and tracked can be framed out by a detection frame and marked by the detection frame according to a multi-target tracking algorithm, so that newly appeared objects are added or disappeared objects are deleted in real time.
S4: and segmenting the image in each detection frame in real time and acquiring a gray level image of the object region.
Specifically, the images of a plurality of moving objects can be obtained through step 3, so that the amount of calculation is reduced on the premise of ensuring that the effective information of the original image is not lost in order to extract the information in the images, and the images in each detection frame can be segmented to obtain the gray-scale image of the object region.
S5: and extracting the motion state information of the object according to the gray-scale image of the object region and coding the motion state information of the object.
Specifically, the obtained moving object is a long-term moving object, and includes motion state information such as its own motion trend, motion direction, motion speed, and the like, and the motion state information of the object may be embodied by a motion history map or an optical flow method, and the like. And coding the object motion state information of the moving object obtained each time, and providing effective input data for the object motion prediction model.
In the embodiment, all moving objects in the visual field range are continuously tracked, long-term motion state information of the moving objects is extracted to create the object motion prediction model, the motion trend and direction of the objects in the current and future periods are accurately judged in an all-around mode by the object motion prediction model, therefore, the objects which are firstly appeared or exist in the visual field range can be captured and continuously tracked in the first period no matter how close or far the objects are, and then whether the objects can hinder own actions in the future period is judged.
In an embodiment of the present invention, in S3, the tracking of the object that moves from the video image and the marking of the detection frame may be implemented by using a SORT-Yolov3 algorithm or a DeepSORT algorithm, both of which may track the moving object appearing in the video image in real time. In the embodiment, moving objects are preferably detected by using an SORT-Yolov3 algorithm, each detected moving object is marked by a unique detection frame, the SORT-Yolov3 algorithm is partially based on Kalman filtering and Hungary matching algorithm to associate the detected objects appearing in multiple frames, multi-target tracking can be realized, the detection speed and the detection precision are also balanced, small objects can be detected, the detection of the moving objects appearing in a far distance in a visual field range is met, and the detection is more comprehensive.
In this embodiment, can detect to all objects that appear in the field of vision scope, detect more comprehensively, and the object that appears for the first time of generally far away in the field of vision scope is less difficult to detect, can take it into detection range through this embodiment, avoids the emergence of the condition of lou examining.
Based on the above embodiments, in an embodiment of the present invention, S4 is specifically realized by the following steps,
firstly, segmenting a foreground pixel class and a background pixel class from an image in each detection frame to obtain a binary segmentation image;
the specific generation process of the segmentation graph is as follows:
setting a threshold value t, defining the pixels of the image in the detection frame which are smaller than the threshold value as foreground pixel classes, defining the pixels of the image in the detection frame which are larger than the threshold value as background pixel classes, and defining the variance of the image after binarization as,
in the formula (I), the compound is shown in the specification,
representing the variance, ω, of the image after binarization1(t) represents the probability of occurrence of foreground pixel classes, ω2(t) represents the probability of the background pixel class appearing, μ1(t) represents the mean gray value of the foreground pixels, μ2(t) represents the foreground pixel mean gray value;
wherein, ω is1(t) is calculated by the following formula,
wherein p (i) represents a value obtained by normalizing the gradation histogram;
μ1(t) is calculated by the following formula,
in the formula, x (i) represents the value of the center of the ith gradation histogram bin, and ω can be found for bins larger than the threshold value in the same manner as above2(t) and μ2(t), traversing all possible thresholds t, enabling
The threshold reaching the maximum value is set as a segmentation threshold, and the image in the detection frame is segmented by the segmentation threshold, so that the segmented image has more obvious effect.
Then, calculating the mass center of the segmentation graph to generate a two-dimensional Gaussian distribution image taking the mass center coordinate as the center;
and finally, multiplying the segmentation image with the Gaussian distribution image to obtain an object region gray level image.
In this embodiment, the object area gray scale map can be obtained, and the size of the whole data is greatly reduced under the condition that the effective data of the video image is not lost, so that the detection efficiency can be improved, the detection time can be saved, and the reduction of the delay is facilitated.
Based on the above embodiments, in an embodiment of the present invention, the extraction of the object motion state information in S5 is preferably embodied by an object motion history map;
and creating an object motion history map according to the object region gray scale map, so that the motion state information of the object is included in the object motion history map.
Defining an object motion history map as
In the formula, Hτ(x, y, t) is the current object motion history map,
the gray scale image of the object region, tau is a parameter describing the time range of motion, delta is an attenuation parameter of the object motion history image, and xi is an object region binary segmentation threshold.
Based on the above embodiment, in an embodiment of the present invention, because the objects appearing in the view field are not uniform, the created object motion history map also needs to be updated in real time along with the objects in the view field, so as to avoid redundancy and repeated calculation.
The specific steps of creating and updating the object motion history map are as follows,
firstly, judging whether an object appears in a visual field range for the first time;
if the object appears for the first time, a new object motion history map is created;
if the object does not appear for the first time, updating the object motion history map;
a motion history map of the object when the object is lost from view.
The encoding of the object motion state information in S1 specifically includes:
performing pixel gradient vector calculation on the object motion historical image to obtain a motion vector of a moving object and a vector horizontal included angle;
encoding the motion vector modulo-1 to + 1;
and normalizing the value of the horizontal included angle of the vector and coding in a 360-degree horizontal direction space.
In S1, analyzing a video image by repeatedly performing an object motion detection process to obtain an object motion state sequence, inputting the object motion state sequence into an object motion prediction model, and determining and outputting a current and future motion trend and direction of the object by using the object motion prediction model specifically includes:
acquiring a motion vector coding sequence according to the coded motion vector;
acquiring a motion direction coding sequence according to the coded vector horizontal included angle;
inputting the motion vector coding sequence into a trained first hidden Markov model, performing model inference by using a Viterbi algorithm, and judging whether the motion trend of the object is forward or backward;
and inputting the motion direction coding sequence into a trained second hidden Markov model, performing model inference by using a Viterbi algorithm, and judging whether the horizontal motion direction of the object is leftward or rightward.
In this embodiment, the object motion state information included in the object motion history map is encoded in combination with the object motion history map, a hidden markov model is preferably selected to model the object motion state, and the hidden markov model is used to determine the motion trend and the motion direction of the object, so that accurate data can be provided for obstacle avoidance early warning.
Based on the foregoing embodiments, in an embodiment of the present invention, S2 specifically includes:
formulating respective unique codes corresponding to moving objects appearing in the visual field;
maintaining a key value table according to the unique codes, wherein the unique codes in the key value table are index keys, and the values in the key value table are object motion history maps;
and updating the motion prediction of the detected object in real time by the prediction model according to the key value table.
Specifically, the moving objects appearing in the visual field correspond to a unique code corresponding to the moving objects, and the motion state information of each moving object from appearance to disappearance is embodied through an object motion history map, so that each unique code in the key value table corresponds to an object motion history map, the key value table updates the object motion history maps of the moving objects in the current visual field range in real time within each window time, and the prediction model only needs to call the corresponding object motion history maps to facilitate management and query, thereby being beneficial to improving the continuity and the rigor of the whole process.
The embodiment of the present application further provides a specific implementation manner of an apparatus for determining a motion trend and a motion direction of an object, which is capable of implementing all steps in the method in the foregoing embodiment, and referring to fig. 2, the apparatus 100 specifically includes the following contents:
the processor 110, the memory 120 and the communication unit 130 complete communication with each other through the bus 140; the communication unit 130 is used for implementing information transmission between server-side devices and terminal devices and other related devices.
The processor 110 is used to call the computer program in the memory 120, and the processor executes the computer program to implement all the steps of the method in the above-described embodiments.
Those of ordinary skill in the art will understand that: the Memory may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory is used for storing programs, and the processor executes the programs after receiving the execution instructions. Further, the software programs and modules within the aforementioned memories may also include an operating system, which may include various software components and/or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.
The processor may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method for judging the motion trend and direction of an object is characterized by comprising the following steps:
object motion prediction process: analyzing a section of video image by repeatedly executing an object motion detection process to obtain an object motion state sequence, inputting the object motion state sequence into an object motion prediction model, and judging and outputting the current and future motion trends and directions of the object by using the object motion prediction model;
repeatedly executing the motion prediction process to update the motion prediction of the detected object in real time;
the object motion detection process includes:
acquiring a video image in a visual field range in real time, tracking a moving object detected from the video image and marking the moving object by a detection frame;
segmenting the image in each detection frame in real time and acquiring an object area gray level image;
and extracting the motion state information of the object according to the gray-scale image of the object region and coding the motion state information of the object.
2. The method for determining the moving trend and direction of an object according to claim 1, wherein the tracking of the moving object from the video image and the marking of the moving object with the detection frame comprises:
and tracking a moving object appearing in the video image in real time through an SORT-Yolov3 algorithm or a DeepsORT algorithm and marking the moving object by using a unique detection box.
3. The method for determining the movement trend and direction of an object according to claim 1, wherein the segmenting the image in each detection frame in real time and obtaining the gray-scale map of the object region comprises:
segmenting a foreground pixel class and a background pixel class from the image in each detection frame to obtain a binary segmentation image;
calculating the mass center of the segmentation graph to generate a two-dimensional Gaussian distribution image taking the mass center coordinate as the center;
and multiplying the segmentation image with the Gaussian distribution image to obtain an object region gray level image.
4. The method for determining the movement tendency and direction of an object according to claim 3, wherein the obtaining of the segmentation map to be binarized by segmenting the foreground pixel class and the background pixel class from the image in each detection frame comprises:
setting a threshold value t, defining the pixels of the image in the detection frame which are smaller than the threshold value as foreground pixel classes, defining the pixels of the image in the detection frame which are larger than the threshold value as background pixel classes, and defining the variance of the image after binarization as,
in the formula (I), the compound is shown in the specification,
representing the variance, ω, of the image after binarization1(t) represents the probability of occurrence of foreground pixel classes, ω2(t) represents the probability of the background pixel class appearing, μ1(t) represents the mean gray value of the foreground pixels, μ2(t) represents the foreground pixel mean gray value;
wherein, ω is1(t) is calculated by the following formula,
wherein p (i) represents a value obtained by normalizing the gradation histogram;
μ1(t) is calculated by the following formula,
in the formula, x (i) represents the value of the center of the ith gradation histogram bin, and ω can be found for bins larger than the threshold value in the same manner as above2(t) and μ2(t), traversing all possible thresholds t, enabling
The threshold value reaching the maximum value is set as a segmentation threshold value, and the image in the detection frame is segmented by the segmentation threshold value.
5. The method according to claim 1, wherein the extracting the motion state information of the object according to the object region gray map comprises:
creating an object motion history map according to the object region gray scale map;
defining the object motion history map as
In the formula, Hτ(x, y, t) is the current object motion history map,
the gray scale image of the object region, tau is a parameter describing the time range of motion, delta is an attenuation parameter of the object motion history image, and xi is an object region binary segmentation threshold.
6. The method for determining the movement tendency and direction of an object according to claim 5, wherein the creating of the object movement history map according to the object region gray scale map comprises:
judging whether the object appears in the visual field range for the first time;
if the object appears for the first time, a new object motion history map is created;
if the object does not appear for the first time, updating the object motion history map;
a motion history map of the object when the object is lost from view.
7. The method according to claim 5, wherein the encoding the motion state information of the object comprises:
performing pixel gradient vector calculation on the object motion historical image to obtain a motion vector of a moving object and a vector horizontal included angle;
encoding the motion vector modulo-1 to + 1;
and normalizing the value of the horizontal included angle of the vector and coding in a 360-degree horizontal direction space.
8. The method of claim 7, wherein the analyzing a video image to obtain an object motion state sequence by repeatedly performing an object motion detection process, inputting the object motion state sequence into an object motion prediction model, and determining and outputting current and future motion trends and directions of the object by using the object motion prediction model comprises:
acquiring a motion vector coding sequence according to the coded motion vector;
acquiring a motion direction coding sequence according to the coded vector horizontal included angle;
inputting the motion vector coding sequence into a trained first hidden Markov model, performing model inference by using a Viterbi algorithm, and judging whether the motion trend of the object is forward or backward;
and inputting the motion direction coding sequence into a trained second hidden Markov model, performing model inference by using a Viterbi algorithm, and judging whether the horizontal motion direction of the object is leftward or rightward.
9. The method according to claim 8, wherein the motion prediction process is repeated, and the updating the motion prediction of the detected object in real time comprises:
formulating respective unique codes corresponding to moving objects appearing in the visual field;
maintaining a key value table according to the unique codes, wherein the unique codes in the key value table are index keys, and the values in the key value table are object motion history maps;
the object motion prediction model updates the motion prediction of the detected object in real time according to the key value table.
10. An apparatus for determining a movement tendency and direction of an object, comprising:
a processor, a memory, an interface to communicate with a gateway;
the memory is used for storing programs and data, and the processor calls the programs stored in the memory to execute the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011632905.2A CN112734795A (en) | 2020-12-31 | 2020-12-31 | Method and equipment for judging motion trend and direction of object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011632905.2A CN112734795A (en) | 2020-12-31 | 2020-12-31 | Method and equipment for judging motion trend and direction of object |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112734795A true CN112734795A (en) | 2021-04-30 |
Family
ID=75608436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011632905.2A Pending CN112734795A (en) | 2020-12-31 | 2020-12-31 | Method and equipment for judging motion trend and direction of object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112734795A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822879A (en) * | 2021-11-18 | 2021-12-21 | 南京智谱科技有限公司 | Image segmentation method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101950425A (en) * | 2010-09-26 | 2011-01-19 | 新太科技股份有限公司 | Motion behavior detection-based intelligent tracking arithmetic |
| CN102521579A (en) * | 2011-12-21 | 2012-06-27 | Tcl集团股份有限公司 | Method for identifying pushing action based on two-dimensional planar camera and system |
| CN102592112A (en) * | 2011-12-20 | 2012-07-18 | 四川长虹电器股份有限公司 | Method for determining gesture moving direction based on hidden Markov model |
| CN102663429A (en) * | 2012-04-11 | 2012-09-12 | 上海交通大学 | Method for motion pattern classification and action recognition of moving target |
| CN105741321A (en) * | 2016-01-31 | 2016-07-06 | 华南理工大学 | Video target movement tendency analysis method based on trace point distribution |
| CN109697387A (en) * | 2017-10-23 | 2019-04-30 | 北京京东尚科信息技术有限公司 | Direction of motion prediction technique, device, electronic equipment and storage medium |
| CN109902592A (en) * | 2019-01-30 | 2019-06-18 | 浙江大学 | A kind of blind person's secondary row path method based on deep learning |
-
2020
- 2020-12-31 CN CN202011632905.2A patent/CN112734795A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101950425A (en) * | 2010-09-26 | 2011-01-19 | 新太科技股份有限公司 | Motion behavior detection-based intelligent tracking arithmetic |
| CN102592112A (en) * | 2011-12-20 | 2012-07-18 | 四川长虹电器股份有限公司 | Method for determining gesture moving direction based on hidden Markov model |
| CN102521579A (en) * | 2011-12-21 | 2012-06-27 | Tcl集团股份有限公司 | Method for identifying pushing action based on two-dimensional planar camera and system |
| CN102663429A (en) * | 2012-04-11 | 2012-09-12 | 上海交通大学 | Method for motion pattern classification and action recognition of moving target |
| CN105741321A (en) * | 2016-01-31 | 2016-07-06 | 华南理工大学 | Video target movement tendency analysis method based on trace point distribution |
| CN109697387A (en) * | 2017-10-23 | 2019-04-30 | 北京京东尚科信息技术有限公司 | Direction of motion prediction technique, device, electronic equipment and storage medium |
| CN109902592A (en) * | 2019-01-30 | 2019-06-18 | 浙江大学 | A kind of blind person's secondary row path method based on deep learning |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822879A (en) * | 2021-11-18 | 2021-12-21 | 南京智谱科技有限公司 | Image segmentation method and device |
| CN113822879B (en) * | 2021-11-18 | 2022-06-21 | 南京智谱科技有限公司 | Image segmentation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109086873B (en) | Training method, recognition method and device of recurrent neural network and processing equipment | |
| CN113221677A (en) | Track abnormity detection method and device, road side equipment and cloud control platform | |
| CN113177968A (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN113869449A (en) | Model training method, image processing method, device, equipment and storage medium | |
| CN113947188A (en) | Training method of target detection network and vehicle detection method | |
| CN111161325A (en) | Three-dimensional multi-target tracking method based on Kalman filtering and LSTM | |
| CN116027324A (en) | Fall detection method and device based on millimeter wave radar and millimeter wave radar equipment | |
| CN117372917A (en) | Security abnormal behavior identification method based on multidimensional feature fusion | |
| CN112509126A (en) | Method, device, equipment and storage medium for detecting three-dimensional object | |
| CN112734795A (en) | Method and equipment for judging motion trend and direction of object | |
| CN113705380A (en) | Target detection method and device in foggy days, electronic equipment and storage medium | |
| CN111563541B (en) | Training method and device of image detection model | |
| CN113378837A (en) | License plate shielding identification method and device, electronic equipment and storage medium | |
| CN112308917A (en) | Vision-based mobile robot positioning method | |
| CN114663347B (en) | Unsupervised object instance detection method and unsupervised object instance detection device | |
| CN116958529A (en) | Robot remote control method, system and medium | |
| CN114998387A (en) | Object distance monitoring method and device, electronic equipment and storage medium | |
| CN115359468A (en) | Target website identification method, device, equipment and medium | |
| CN117557790A (en) | Training method of image mask generator and image instance segmentation method | |
| CN113936158A (en) | Label matching method and device | |
| CN115431968B (en) | Vehicle controller, vehicle and vehicle control method | |
| Beknazarova et al. | Machine learning algorithms are used to detect and track objects on video images | |
| CN114445395B (en) | Distribution line tree obstacle intelligent positioning method and device based on Internet and big data | |
| CN116704445A (en) | Method and device for identifying landfill image, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210430 |
|
| RJ01 | Rejection of invention patent application after publication |