CN117765424A

CN117765424A - Article monitoring method, article monitoring device, computer readable medium and electronic equipment

Info

Publication number: CN117765424A
Application number: CN202211132221.5A
Authority: CN
Inventors: 赵剑飞
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd
Priority date: 2022-09-16
Filing date: 2022-09-16
Publication date: 2024-03-26

Abstract

The embodiment of the application provides an article monitoring method, an article monitoring device, a computer readable medium and electronic equipment. The article monitoring method comprises the following steps: acquiring multi-frame video frame images of a monitoring video, wherein the monitoring video is a monitoring video aiming at a preset monitoring area; determining a motion track of a target object based on the multi-frame video frame image, wherein the target object is an object positioned in a preset monitoring area; obtaining predicted position information based on the motion trail of the target object, wherein the predicted position information is the position information of the predicted target object in the video frame image of the target frame; positioning a target object in the target frame video frame image based on the predicted position information so as to track the target object; and if the target object is continuously tracked to be kept static in the preset monitoring area, sending alarm information. The problem that the manual monitoring is extravagant manpower and reliability are poor has been solved to this application. Meanwhile, the method has the advantages of high response speed, real-time operation, high efficiency and capability of detecting whether the object falls in a short time.

Description

Article monitoring method, article monitoring device, computer readable medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an article monitoring method, an article monitoring device, a computer readable medium, and an electronic device.

Background

With the development of the logistics industry, a large amount of articles are always piled up in a logistics place such as a transfer bin, a sorting bin and the like. In these places, sometimes because of untimely operation, articles are scattered in places where articles should not appear, such as aisles or the ground, the articles are lost if the articles are light, and certain potential safety hazards are possibly caused if the articles are heavy.

Most of the existing methods rely on manual monitoring of the articles, and naked eyes check which areas are monitored for articles falling and scattering. The manual monitoring system has the advantages that human resources are wasted, meanwhile, the reliability of manual monitoring is low, and the problems that part of dropped articles are difficult to find, the dropped articles are not timely reported, even the reporting is forgotten to report and the like easily occur.

Disclosure of Invention

The embodiment of the application provides an article monitoring method, an article monitoring device, a computer-readable medium and electronic equipment, and further at least to a certain extent, the problems that manpower is wasted and reliability is poor in the existing manual monitoring can be overcome.

Other features and advantages of the present application will be apparent from the following detailed description, or may be learned in part by the practice of the application.

According to one aspect of embodiments of the present application, there is provided an article monitoring method, the method including:

acquiring multi-frame video frame images of a monitoring video, wherein the monitoring video is a monitoring video aiming at a preset monitoring area;

determining a motion trail of a target object based on a multi-frame video frame image, wherein the target object is an object positioned in a preset monitoring area;

obtaining predicted position information based on the motion trail of the target object, wherein the predicted position information is the predicted position information of the target object in a target frame video frame image;

positioning the target object in the target frame video frame image based on the predicted position information so as to track the target object;

and if the target object is continuously tracked to be kept static in the preset monitoring area, sending alarm information.

In one embodiment of the present application, the determining a motion track of a target object based on the multi-frame video frame image specifically includes:

inputting a plurality of frames of video frame images into a preset article detection network one by one, wherein the detection network outputs corresponding actual position information one by one, and the actual position information is the position information of the target article in the video frame images;

And determining the motion trail of the target object based on the actual position information corresponding to the video frame images.

In one embodiment of the present application, the inputting the plurality of frames of the video frame images into a preset article detection network one by one, where the detection network outputs the corresponding actual position information one by one, specifically includes:

inputting multiple frames of video frame images one by one into a detection network, wherein the detection network recognizes the position information of all articles in the multiple frames of video frame images one by one;

and filtering out the position information of the objects positioned outside the preset monitoring area in the multi-frame video frame image according to a preset area mask, so as to obtain the actual position information, wherein the area mask is used for dividing the preset monitoring area.

In one embodiment of the present application, before inputting the video frame image into a preset item detection network, the method further comprises:

acquiring a video frame image sample set, wherein each video frame image sample is calibrated with position information of all objects in the corresponding video frame image in advance;

respectively inputting the data of each video frame image sample into an initial article detection network to obtain the position information of all articles in the video frame images output by the initial article detection network;

If the position information of all the objects in the video frame image obtained after the data of the video frame image sample are input into an initial object detection network is inconsistent with the position information of all the objects in the video frame image which is calibrated in advance for the video frame image sample, adjusting the coefficient of the initial object detection network until the position information of all the objects in the video frame image sample is consistent with the position information of all the objects in the video frame image;

after the data of all the video frame image samples are input into a detection network, the obtained position information of all the articles in the video frame image is consistent with the position information of all the articles in the video frame image calibrated in advance for the video frame image samples, and the training is finished, so that the article detection network is obtained.

In one embodiment of the present application, the positioning the target object in the target frame video frame image based on the predicted position information to achieve tracking of the target object specifically includes:

acquiring the video frame image of the target frame;

inputting the target frame video frame image into a detection network to obtain the actual position information;

and matching the predicted position information with the actual position information to track the target object.

In one embodiment of the present application, the matching the predicted position information with the actual position information to track the target object specifically includes:

obtaining IoU values of the predicted position information and the actual position information;

determining if the predicted and actual location information match based on the IoU value to track the target item.

In an embodiment of the present application, the obtaining the predicted position information based on the motion trail of the target object specifically includes:

inputting the motion trail of the target object into a filter, and outputting the predicted position information by the filter.

In one embodiment of the present application, if the target object is continuously tracked to be kept stationary in the predetermined monitoring area, sending the alarm information specifically includes:

and if the time for tracking the target object to remain stationary in the preset monitoring area exceeds a preset time threshold, sending alarm information.

and if the number of frames for tracking the target object to remain stationary in the preset monitoring area exceeds the preset frame number threshold, sending alarm information.

According to an aspect of the embodiments of the present application, there is provided an article monitoring apparatus including:

the video frame image acquisition module is used for acquiring multi-frame video frame images of a monitoring video, wherein the monitoring video is a monitoring video aiming at a preset monitoring area;

the motion trail determining module is used for determining the motion trail of a target object based on the multi-frame video frame images, wherein the target object is an object positioned in a preset monitoring area;

the position information prediction module is used for obtaining predicted position information based on the motion trail of the target object, wherein the predicted position information is the predicted position information of the target object in a target frame video frame image;

the target object tracking module is used for positioning the target object in the target frame video frame image based on the predicted position information so as to track the target object;

and the alarm information sending module is used for sending alarm information if the target object is continuously tracked to be kept static in a preset monitoring area.

According to an aspect of an embodiment of the present application, there is provided an electronic device including: one or more processors; and a storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the item monitoring method as described in the above embodiments.

In the technical scheme provided by some embodiments of the application, through detecting algorithms and models such as a network, the position information of the article can be stably detected, real-time track tracking is carried out on the article, the target article is positioned in real time, whether the article is scattered or not can be determined, and whether an alarm is needed or not is further determined, so that the labor cost of manual monitoring is saved, errors caused by human negligence can be avoided in the monitoring of the algorithms and the models, the reliability of article monitoring is improved, and the problems of wasting labor and poor reliability in the existing manual monitoring are solved. Meanwhile, the algorithm and the model are low in complexity, high in response speed, capable of running in real time and high in efficiency, and whether the object falls off or not can be detected in a short time.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:

Fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application may be applied.

Fig. 2 schematically shows a flow chart of an item monitoring method according to an embodiment of the present application.

Fig. 3 is a flowchart showing a specific implementation of step S200 in the method for monitoring an article according to the corresponding embodiment of fig. 2.

Fig. 4 is a flowchart showing a specific implementation of step S210 in the method for monitoring an article according to the corresponding embodiment of fig. 2.

Fig. 5 is a schematic structural diagram of a region mask in the method for monitoring an article according to the corresponding embodiment of fig. 4.

Fig. 6 is a flowchart showing a specific implementation of step S400 in the method for monitoring an article according to the corresponding embodiment of fig. 2.

Fig. 7 schematically illustrates a block diagram of an item monitoring device according to one embodiment of the present application.

Fig. 8 shows a schematic diagram of a computer system suitable for use in implementing the electronic device of the embodiments of the present application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present application. One skilled in the relevant art will recognize, however, that the aspects of the application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

As shown in fig. 1, the system architecture may include a terminal device (such as one or more of the smartphone 101, tablet 102, and portable computer 103 shown in fig. 1, but of course, a desktop computer, etc.), a network 104, and a server 105. The network 104 is the medium used to provide communication links between the terminal devices and the server 105. The network 104 may include various connection types, such as wired communication links, wireless communication links, and the like.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. For example, the server 105 may be a server cluster formed by a plurality of servers.

A user may interact with the server 105 through the network 104 using the terminal device as a node, to receive or send messages, etc. The server 105 may be a server providing various services. For example, the user uploads the multi-frame video frame image of the monitoring video to the server 105 by using the terminal device 103 (may also be the terminal device 101 or 102), and the server 105 may determine the motion trail of the target object based on the multi-frame video frame image, where the target object is an object located in a predetermined monitoring area; obtaining predicted position information based on the motion trail of the target object, wherein the predicted position information is the predicted position information of the target object in a target frame video frame image; positioning the target object in the target frame video frame image based on the predicted position information so as to track the target object; and if the target object is continuously tracked to be kept static in the preset monitoring area, sending alarm information.

It should be noted that, the method for monitoring an article according to the embodiment of the present application is generally performed by the server 105, and accordingly, the article monitoring device is generally disposed in the server 105. However, in other embodiments of the present application, the terminal device may also have a similar function as the server, so as to execute the solution for monitoring the article provided in the embodiments of the present application.

The implementation details of the technical solutions of the embodiments of the present application are described in detail below:

fig. 2 shows a flow chart of an item monitoring method according to an embodiment of the present application, which may be performed by a server, which may be the server shown in fig. 1. Referring to fig. 2, the article monitoring method may specifically include the steps of:

step S100, acquiring multi-frame video frame images of a monitoring video, wherein the monitoring video is a monitoring video aiming at a preset monitoring area.

Step S200, determining a motion track of a target object based on the multi-frame video frame image, wherein the target object is an object located in a preset monitoring area.

Step S300, obtaining predicted position information based on the motion trail of the target object, wherein the predicted position information is the predicted position information of the target object in the target frame video frame image.

Step S400, positioning the target object in the target frame video frame image based on the predicted position information, so as to track the target object.

And step S500, if the target object is continuously tracked to be kept static in a preset monitoring area, sending alarm information.

In this embodiment, firstly, a multi-frame video frame image of a monitoring video is acquired, a motion track of the target object is determined based on the multi-frame video frame image, then, based on the motion track of the target object, the position of the target object is predicted, the target object is tracked in real time according to a prediction result, and finally, if the target object is continuously tracked to be kept stationary in a preset monitoring area, the target object is proved to be a scattered object, and alarm information is sent for reminding. According to the embodiment of the application, the position information of the article can be stably detected through the algorithms and the models such as the detection network, real-time track tracking can be carried out on the article, the target article is positioned in real time, whether the article is scattered or not can be determined, whether an alarm is needed or not is further determined, the labor cost of manual monitoring is saved, errors caused by human negligence can be avoided through the monitoring of the algorithms and the models, the reliability of article monitoring is improved, and the problems that the labor is wasted and the reliability is poor in the conventional manual monitoring are solved. Meanwhile, the algorithm and the model are low in complexity, high in response speed, capable of running in real time and high in efficiency, and whether the object falls off or not can be detected in a short time.

In step S100, there are various ways to obtain video frame images of the monitoring video, generally, the monitoring video obtained in real time is split into multiple frame images, and each split image is a video frame image. The target item is an item located in a predetermined monitoring area. The predetermined monitoring area is an area to be monitored, which may be a non-stock area where articles such as aisles, floors, etc. should not be present, and may be scattered articles when the articles are present in the predetermined monitoring area for a long time.

In step S200, the multiple frames of video frame images are arranged according to a time sequence, and generally, the multiple frames of video frame images are continuous video frame images, if monitoring of the target object is required, the motion track of the target object needs to be determined first, and if the motion track of the target object needs to be determined, analysis is required based on the multiple frames of video frame images arranged according to the time sequence.

The method for determining the motion track of the target object includes various modes, for example, determining a position change parameter of the target object based on the position of the target object in a plurality of frames of video frame images, so as to obtain the motion track, or determining a track change function of the target object based on the position of the target object in the plurality of frames of video frame images, wherein the track change function can comprise a speed change function in the plurality of frames of video frame images with time, a speed change function in the plurality of frames of video frame images with frame numbers, an acceleration change function in the plurality of frames of video frame images with time and the like, and determining the motion track of the target object based on the determined track change function.

In some embodiments, the specific implementation of step S200 may refer to fig. 3. Fig. 3 is a detailed description of step S200 in the article monitoring method according to the corresponding embodiment of fig. 2, where step S200 may include the following steps:

step S210, inputting a plurality of frames of video frame images into a preset article detection network one by one, wherein the detection network outputs corresponding actual position information one by one, and the actual position information is the position information of the target article in the video frame images;

step S220, determining the motion trail of the target object based on the actual position information corresponding to the video frame images of the multiple frames.

In this embodiment, firstly, a plurality of frames of video images of a monitoring video are obtained, then, article identification is performed on the plurality of frames of video images, actual position information corresponding to the plurality of frames of video images is determined, and finally, a motion track of the target article is determined based on the actual position information corresponding to the plurality of frames of video images.

In step S210, the target object in the video frame image is identified by the detection network, and the actual position information is obtained.

Specifically, in some embodiments, the specific implementation of step S210 may refer to fig. 4. Fig. 4 is a detailed description of step S210 in the article monitoring method according to the corresponding embodiment of fig. 3, where step S210 may include the following steps:

Step S211, inputting the multiple frames of the video frame images into a detection network one by one, wherein the detection network recognizes the position information of all objects in the multiple frames of the video frame images one by one.

Step S212, filtering out the position information of the objects located outside the preset monitoring area in the multiple frames of video frame images according to a preset area mask, so as to obtain the actual position information, wherein the area mask is used for dividing the preset monitoring area.

In this embodiment, all the articles in the video frame image and the position information corresponding to the articles are identified through the detection network, then the articles located outside the predetermined monitoring area are filtered according to the area mask, and only the articles located in the predetermined monitoring area remain, wherein the articles are target articles.

In step S211, the position information of the target object may be expressed in various manners, for example, in one embodiment, the position information of the target object may be expressed as (x, y, w, h), where x and y respectively represent an abscissa and an ordinate of a center point of the identification frame corresponding to the target object, and w and h respectively represent a width and a height of the identification frame corresponding to the target object. In another embodiment, the position information of the target object may be represented as (a, b, c, d), where a and b represent an abscissa and an ordinate of a first vertex of the identification frame corresponding to the target object, respectively, and c and d represent an abscissa and an ordinate of a second vertex of the identification frame corresponding to the target object, respectively, and the first vertex is a diagonal point of the second vertex.

As shown in fig. 5, the area mask 200 is used to divide a predetermined monitoring area, and includes an active area 210 and an inactive area 220, wherein the active area 210 corresponds to the predetermined monitoring area, and the inactive area 220 corresponds to the other area 12 outside the predetermined monitoring area. In step S220, the area mask 200 is overlaid on the video frame image, and the target object in the ineffective area of the position information identified in step S210 is filtered out.

Specifically, the training method of the detection network includes: acquiring a video frame image sample set, wherein each video frame image sample is calibrated with position information of all objects in the corresponding video frame image in advance; respectively inputting the data of each video frame image sample into a detection network to obtain the position information of all objects in the video frame images output by the flow strategy; if the position information of all the articles in the video frame image obtained after the data of the video frame image sample are input into a detection network is inconsistent with the position information of all the articles in the video frame image which is calibrated in advance for the video frame image sample, adjusting the coefficient of the flow strategy until the position information of all the articles in the video frame image sample is consistent with the position information of all the articles in the video frame image; after the data of all the video frame image samples are input into a detection network, the obtained position information of all the articles in the video frame image is consistent with the position information of all the articles in the video frame image calibrated in advance for the video frame image samples, and training is finished.

In the training process of the detection network, cross-validation, for example, 10-fold cross-validation (10-fold cross validation), divides the data set into ten parts, takes 9 parts of the data set as training set data and 1 part of the data set as verification set data in turn, takes the average value of the 10 results as estimation on algorithm precision, and generally needs to perform multiple times of 10-fold cross-validation to obtain the average value, for example: 10-fold cross-validation for a more accurate point. The model training error may be determined by means of a number of calculations.

After the motion trail of the target object is obtained in step S200, the target object may be continuously tracked. In step S300, the position where the target object appears may be predicted based on the motion trajectory of the target object, and there may be a plurality of predicted positions to achieve tracking of the target object.

Specifically, in some embodiments, the following embodiments may be referred to for a specific implementation of step S300. The embodiment is a detailed description of step S300 in the article monitoring method according to the corresponding embodiment shown in fig. 2, where step S300 may include the following steps:

In this embodiment, the prediction of the target item position is performed by a kalman filter.

Specifically, in this embodiment, after at least 2 frames of identification frame data of the target object are obtained, motion data of the object may be obtained by analysis in a kalman filtering manner, and based on the motion data of the target object obtained by analysis, the target object is subjected to kalman tracking with the current frame video frame image as a starting point, so as to obtain position information of the target object in the target frame until the target object is not detected or the target object is detected to move outside the predetermined monitoring area. It will be appreciated that the kalman tracking of the location of the target item is initiated after at least 2 frames of identification frame data of the target item are obtained.

Specifically, according to the detection of the actual position information corresponding to the previous frame of video frame and the actual position information corresponding to the current frame of video frame, the current position information and the movement speed corresponding to the target object are obtained through a Kalman filter, the current position information is assigned to the position information when the Kalman tracking is started on the historical record target object, the movement speed is assigned to the Kalman parameter, and the position information of the target object predicted by the Kalman is obtained.

In step S400, the target object is locked by matching the root predicted position information with the actual position information, and specifically, the tracking of the target object is achieved by comparing the predicted position information with the actual position information one by one, determining whether the two position information can be matched, and determining that the target object is the target object to be tracked based on the two position information, so as to achieve the tracking of the target object.

Specifically, in some embodiments, the specific implementation of step S400 may refer to fig. 6. Fig. 6 is a detailed description of step S400 in the article monitoring method according to the corresponding embodiment of fig. 2, where step S400 may include the following steps:

step S410, acquiring the video frame image of the target frame.

Step S420, inputting the target frame video frame image into a detection network, to obtain the actual position information.

Step S430, matching the predicted position information with the actual position information to track the target object.

In this embodiment, a target frame video frame image is acquired first, and image recognition is performed on the target frame video frame image to obtain corresponding actual position information, then the actual position information and the predicted position information are matched, and whether the target object is a target object to be tracked or not is determined based on whether the two position information can be matched, so that tracking of the target object is achieved.

In step S430, there are various ways to perform the matching, and the following embodiments may be referred to for details.

Specifically, in some embodiments, the following embodiments may be referred to for a specific implementation of step S430. The embodiment is a detailed description of step S430 in the article monitoring method according to the corresponding embodiment of fig. 6, where step S430 may include the following steps:

and IoU values of the predicted position information and the actual position information are obtained.

In this embodiment, the matching manner is to compare the plurality of predicted position information and the actual position information determined in step S200 one by one to obtain a plurality of IoU values, and combine them to obtain a IoU distance matrix. And then matching the track of the target object in the current frame video frame image with the position information of the target object in the target frame video frame image by using a Hungary algorithm, and if the track and the position information can be matched, updating the state and continuously tracking the target object. If no position information of the target object can be matched, the target object is considered to be not in the preset monitoring area, and tracking information is directly initialized.

In the embodiment of the application, the corresponding tracker is matched for each target object by using the Hungary algorithm, so that the calculated amount of tracking data can be reduced, the matching time of the tracker is shortened, and the tracking efficiency of tracking the target is improved. In addition, the method predicts the tracking target according to the motion-related data, does not comprise any complex matrix operation, further reduces the calculated amount of the tracking data, and improves the tracking efficiency of the tracking target; and the invention only uses the data related to the movement, reduces the storage memory of the tracking data and saves the storage space.

In some embodiments of the present application, if all the tracks cannot be matched when using the hungarian algorithm, the kernel correlation filtering (Kernelized Correlation Filters, KCF) algorithm searches the target frame video frame image for the position of the corresponding track, and matches the position information of the unmatched current frame video frame image again, and then performs the state update. If the result of re-matching is that the track is not matched, the number of frames which are not matched is recorded, if the number of continuous frames which are not matched exceeds a preset matching frame number threshold value, the track is considered to be disappeared, and the tracking information is initialized. And if the result of the re-matching is that the track and the position information are matched, updating the state, and continuously tracking the target object. If the matching result is that no position information of the target object can be matched, the target object is considered to be not in the preset monitoring area, and the tracking information is directly initialized.

The IoU distance matrix at least comprises IoU values of static target articles in target frame video frames, ioU values of uniform motion of the target articles in the target frame video frames and IoU values of uniform variable motion of the target articles in the target frame video frames.

In step S500, if the target object is continuously tracked to be stationary in the predetermined monitoring area, it is proved that the target object is always at a position where the target object should not appear, such as an aisle, a ground, etc., and the probability of being a scattered object is high, and at this time, the target object is determined to be a scattered object, and an alarm message is sent.

The manner of determining whether the target object remains stationary in the predetermined monitoring area may be determined by time or by the number of frames, for example.

Specifically, in some embodiments, the following embodiments may be referred to for a specific implementation of step S500. The embodiment is a detailed description of step S500 in the article monitoring method according to the corresponding embodiment shown in fig. 2, where step S500 may include the following steps:

In the present embodiment, the manner of determining whether the target article remains stationary in the predetermined monitoring area is a transit time determination. Specifically, when it is tracked that one position of the target object in the predetermined monitoring area remains motionless for a predetermined time threshold, whether the target object remains motionless in the predetermined monitoring area can be considered, and at this time, the target object is judged to be a scattered object, and alarm information is sent.

In the present embodiment, the manner of judging whether or not the target article remains stationary in the predetermined monitoring area is through frame number judgment. Specifically, when the number of frames of continuous video frames tracking that a position of the target object remains unchanged in a predetermined monitoring area reaches a predetermined frame number threshold, whether the target object remains static in the predetermined monitoring area can be considered, and at the moment, the target object is judged to be a scattered object, and alarm information is sent.

The following describes embodiments of the apparatus of the present application that may be used to perform the article surveillance methods of the above-described embodiments of the present application. For details not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the method for monitoring an article described in the present application.

Fig. 7 shows a block diagram of an item monitoring device according to one embodiment of the present application.

Referring to fig. 7, an article monitoring device 900 according to an embodiment of the present application includes:

the video frame image obtaining module 910 is configured to obtain a multi-frame video frame image of a surveillance video, where the surveillance video is a surveillance video for a predetermined surveillance area.

The motion trajectory determining module 920 is configured to determine a motion trajectory of a target object, where the target object is an object located in a predetermined monitoring area, based on the multiple frames of video frame images.

The position information prediction module 930 is configured to obtain predicted position information based on the motion track of the target object, where the predicted position information is predicted position information of the target object in the video frame image of the target frame.

And a target object tracking module 940, configured to locate the target object in the target frame video frame image based on the predicted position information, so as to track the target object.

And the alarm information sending module 950 is configured to send alarm information if the target object is continuously tracked and kept stationary in the predetermined monitoring area.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

Furthermore, although the steps of the methods in the present disclosure are depicted in a particular order in the drawings, this does not require or imply that the steps must be performed in that particular order or that all illustrated steps be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.

It should be noted that, the computer system of the electronic device shown in fig. 8 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.

As shown in fig. 8, the computer system includes a central processing unit (Central Processing Unit, CPU) 1801, which can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1802 or a program loaded from a storage section 1808 into a random access Memory (Random Access Memory, RAM) 1803. In the RAM 1803, various programs and data required for system operation are also stored. The CPU 1801, ROM 1802, and RAM 1803 are connected to each other via a bus 1804. An Input/Output (I/O) interface 1805 is also connected to the bus 1804.

The following components are connected to the I/O interface 1805: an input section 1806 including a keyboard, a mouse, and the like; an output portion 1807 including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and a speaker, etc.; a storage section 1808 including a hard disk or the like; and a communication section 1809 including a network interface card such as a LAN (Local Area Network ) card, a modem, or the like. The communication section 1809 performs communication processing via a network such as the internet. The drive 1810 is also connected to the I/O interface 1805 as needed. Removable media 1811, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memory, and the like, is installed as needed on drive 1810 so that a computer program read therefrom is installed as needed into storage portion 1808.

In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network via the communication portion 1809, and/or installed from the removable medium 1811. The computer programs, when executed by a Central Processing Unit (CPU) 1801, perform the various functions defined in the system of the present application.

It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with a computer-readable computer program embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present application may be implemented by means of software, or may be implemented by means of hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.

As another aspect, the present application also provides a computer-readable medium that may be contained in the electronic device described in the above embodiment; or may exist alone without being incorporated into the electronic device. The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit, in accordance with embodiments of the present application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains.

It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An article monitoring method, comprising:

2. The method for monitoring an object according to claim 1, wherein the determining a motion trajectory of the object based on the multi-frame video frame image specifically comprises:

3. The article monitoring method according to claim 2, wherein the inputting the plurality of frames of the video frame images one by one into a preset article detection network, the detection network outputting the corresponding actual position information one by one, specifically comprises:

4. The article surveillance method of claim 3, wherein prior to inputting the video frame image into a preset article detection network, the method further comprises:

5. The method according to any one of claims 1 to 4, wherein positioning the target object in the target frame video frame image based on the predicted position information to achieve tracking of the target object, specifically comprises:

acquiring the video frame image of the target frame;

6. The method of claim 5, wherein said matching said predicted location information with actual location information to track said target item, comprises:

7. The method for monitoring an object according to claim 1, wherein the obtaining predicted position information based on the motion trajectory of the target object specifically includes:

8. An article monitoring device, comprising:

9. A computer readable medium on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the article monitoring method according to any one of claims 1 to 7.

10. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs which when executed by the one or more processors cause the one or more processors to implement the item monitoring method of any of claims 1 to 7.