CN116503814A - Personnel tracking method and system for analysis - Google Patents
Personnel tracking method and system for analysis Download PDFInfo
- Publication number
- CN116503814A CN116503814A CN202310594119.5A CN202310594119A CN116503814A CN 116503814 A CN116503814 A CN 116503814A CN 202310594119 A CN202310594119 A CN 202310594119A CN 116503814 A CN116503814 A CN 116503814A
- Authority
- CN
- China
- Prior art keywords
- monitoring
- information
- moment
- monitoring equipment
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004458 analytical method Methods 0.000 title description 5
- 238000012544 monitoring process Methods 0.000 claims abstract description 176
- 238000012806 monitoring device Methods 0.000 claims abstract description 76
- 230000008447 perception Effects 0.000 claims abstract description 28
- 230000002159 abnormal effect Effects 0.000 claims abstract description 22
- 230000009471 action Effects 0.000 claims description 18
- 238000001514 detection method Methods 0.000 claims description 14
- 230000005856 abnormality Effects 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 8
- 239000000725 suspension Substances 0.000 claims description 4
- 230000007704 transition Effects 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 16
- 230000015654 memory Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000006378 damage Effects 0.000 description 3
- 230000001276 controlling effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Alarm Systems (AREA)
Abstract
The invention is applicable to the field of computers, and provides a personnel tracking method and system to be analyzed, wherein the method comprises the following steps: acquiring first trace information of a suspicious target; determining a first area associated with the suspicious target according to the first track information; determining a first monitoring device with a monitoring state which is changed from normal to abnormal at a first moment in the monitoring devices of the first area; positioning a suspended local area range in which the first monitoring equipment is positioned, and acquiring foreign matter perception information acquired in the suspended local area range; judging whether foreign matters appear in the suspended local area range at a certain moment according to the foreign matter perception information, wherein the certain moment is positioned after the first moment, and the method has the beneficial effects that: the first area where the trace-associated monitoring equipment of the suspicious target is located can be judged, and a reliable basis is provided for tracking the suspicious target.
Description
Technical Field
The invention belongs to the field of computers, and particularly relates to a personnel tracking method and system for analysis.
Background
The smart city originates in the media field, and is characterized in that various information technologies or innovative concepts are utilized to break through and integrate the systems and services of the city, so that the efficiency of resource utilization is improved, city management and services are optimized, the lives of citizens are improved, and along with the development of smart city and safe city business, the monitoring technology becomes an indispensable product and technology.
In the prior art, a monitoring system is basic equipment in video, security and traffic guidance, a target tracking technology is realized based on a plurality of cameras in a management area of the monitoring system, the primary step of target tracking is to determine the position of a target, and then the cameras are effectively scheduled according to the position of the target to track the target. In the prior art, the normal work of the camera cannot be ensured, so that the camera may have the disturbed behavior, such as the blocked view of the camera, the destroyed main body structure of the camera, and the like, so that the target cannot be monitored and tracked, and the tracking judgment of the target is affected.
Disclosure of Invention
The embodiment of the invention aims to provide a personnel tracking method and system for analysis, which aim to solve the problems in the background technology.
The embodiment of the invention is realized in such a way that, on one hand, a personnel tracking method to be analyzed comprises the following steps:
acquiring first trace information of a suspicious target;
determining a first area associated with the suspicious target according to the first track information;
determining a first monitoring device with a monitoring state which is changed from normal to abnormal at a first moment in the monitoring devices of the first area;
positioning a suspended local area range in which the first monitoring equipment is positioned, and acquiring foreign matter perception information acquired in the suspended local area range;
judging whether foreign matters appear in the suspended local area range at a certain moment according to the foreign matter perception information, wherein the certain moment is positioned after the first moment;
if yes, determining that the trace of the suspicious object is associated with the first area, and the suspicious object is related to the monitoring state change of the monitoring equipment at the first moment.
As a further aspect of the present invention, after determining that the trace of the suspicious object is associated with the first region, the method further includes:
determining second monitoring equipment acting on the first area according to the first monitoring, wherein the second monitoring equipment is monitoring equipment except the first monitoring equipment;
respectively acquiring a first coordinate of the first monitoring equipment and a second coordinate of the second monitoring equipment, and generating a first relative deflection angle between the second monitoring equipment and the first monitoring equipment according to a determination result;
generating first deflection scheduling information for scheduling the second monitoring equipment according to the current deflection scheduling information of the second monitoring equipment and the first relative deflection angle;
target characteristics of a suspicious target are obtained, and a directional monitoring instruction is generated according to the target characteristics and the first deflection scheduling information;
and issuing the directional monitoring instruction to the second monitoring equipment so that the monitoring angle of the second monitoring equipment after the dispatching covers the local range of the first monitoring equipment, and reporting the monitoring information about the suspicious target when the suspicious target is identified.
As still further aspect of the present invention, the generating the first yaw schedule information for scheduling the second monitoring device according to the current yaw schedule information and the first relative yaw angle of the second monitoring device specifically includes:
identifying a first limit deflection angle and a second limit deflection angle in the current deflection scheduling information;
identifying a starting angle of the first relative deflection angle, and determining a limit deflection angle with a smaller difference value between the first limit deflection angle and the second limit deflection angle and the starting angle;
identifying an effective deflection angle corresponding to a coverage local area range in the first relative deflection angle, determining a first action parameter in the effective deflection angle, and acquiring a second action parameter in the current deflection scheduling information;
and fusing the current deflection scheduling information with the effective deflection angle to generate first deflection scheduling information for controlling the second monitoring equipment to switch back and forth at the limit deflection angle position according to parameters to be switched, wherein the parameters to be switched are a first action parameter and a second action parameter.
As a still further aspect of the present invention, the acquiring the first track information of the suspicious object includes:
acquiring input of trace information about a suspicious target;
or acquiring monitoring information containing suspicious targets in the second area.
As a further aspect of the present invention, the method further includes: and determining the earliest moment when the suspicious object enters the first area according to the first track information, wherein the earliest moment comprises the latest moment when the suspicious object disappears in the second area.
As a further aspect of the present invention, the first monitoring device for determining that the monitoring state in the monitoring device of the first area is changed from normal to abnormal at the first moment specifically includes:
detecting the working state information reported by the monitoring equipment, wherein the monitoring equipment reports first monitoring information according to a set mode after the self-checking is successful, the set mode comprises the step of reporting at regular time, and a monitoring picture of the first monitoring information accords with set definition;
when the fact that a certain monitoring device does not report the first monitoring information is detected, the monitoring state of the corresponding monitoring device is judged to be abnormal.
As a further aspect of the present invention, after determining that the monitoring status of the corresponding monitoring device is abnormal, the method further includes:
issuing a detection feedback instruction to a certain monitoring device, wherein the detection feedback instruction is used for indicating the certain monitoring device to perform self-checking again and reporting first fault information;
if the first fault information is received, judging whether the monitoring state abnormality of the corresponding monitoring equipment is related to the monitoring lens or not according to the first fault information, and if so, identifying a first fault moment of the monitoring lens in the first fault information;
if the first fault information is not received, determining theoretical reporting time corresponding to the setting mode when the first monitoring information is not reported, and taking the theoretical reporting time as a first fault moment;
and judging whether the first fault moment is later than the earliest moment, if so, determining that the monitoring equipment corresponding to the first fault moment which is later than the latest moment is the first monitoring equipment.
As a further aspect of the present invention, the acquiring foreign object perception information acquired in a floating local area range includes:
and issuing an acquisition instruction to at least one sensing device in the suspended local area range so as to acquire sensing information reported by the at least one sensing device, wherein the foreign matter sensing information comprises sensing information, and the at least one sensing device comprises at least one of distance sensing devices and infrared sensing devices.
As a further aspect of the present invention, the determining whether a foreign object appears in the suspended local area at a certain moment according to the foreign object sensing information includes:
judging whether foreign matters appear in the suspended local area range according to the foreign matter perception information;
if so, based on the foreign matter perception information, the moment when the foreign matter appears is judged to be a certain moment.
As a further aspect of the present invention, in another aspect, a personnel tracking system to be analyzed, the system includes:
the area determining module is used for acquiring first trace information of the suspicious target and determining a first area associated with the suspicious target according to the first trace information;
a state transition determining module, configured to determine a first monitoring device whose monitoring state is changed from normal to abnormal at a first time in the monitoring devices in the first area;
the local positioning module is used for positioning a suspension local range in which the first monitoring equipment is positioned and acquiring foreign matter perception information acquired in the suspension local range;
the foreign matter judging module is used for judging whether foreign matters appear in the suspended local area range at a certain moment according to the foreign matter perception information, and the certain moment is positioned after the first moment;
and the condition judging module is used for judging that the trace of the suspicious object is associated with the first area if the foreign matter appears, and the suspicious object is related to the monitoring state change of the monitoring equipment at the first moment.
According to the personnel tracking method and system to be analyzed, from the perspective of monitoring state abnormality of the monitoring equipment, after the first area associated with the suspicious target is determined according to the first trace information, if foreign matter perception information recorded in the suspended local area is passed, if foreign matter appears in the suspended local area based on a certain moment after the first moment, the first area is judged to be associated with the trace of the suspicious target, namely the suspicious target is very likely to enter the position of the first monitoring equipment in the first area or pass through the position of the first monitoring equipment in the first area, and a reliable basis is provided for tracking the suspicious target.
Drawings
FIG. 1 is a flow chart of a method for tracking personnel to be analyzed.
Fig. 2 is a flowchart of adjusting a second monitoring device and reporting monitoring information in a personnel tracking method to be analyzed.
Fig. 3 is a flow chart of a method of personnel tracking to be analyzed for generating a first deflection schedule for scheduling a second monitoring device.
Fig. 4 is a flowchart of determining that a monitoring device corresponding to a first fault time later than a latest time is a first monitoring device in a personnel tracking method to be analyzed.
Fig. 5 is a flowchart for judging whether a foreign object appears in the floating local area range at a certain moment according to the foreign object perception information.
Fig. 6 is a main structural diagram of a person tracking system to be analyzed.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Specific implementations of the invention are described in detail below in connection with specific embodiments.
The personnel tracking method and system for analysis provided by the invention solve the technical problems in the background technology.
As shown in fig. 1, a main flow chart of a method for tracking a person to be analyzed according to an embodiment of the present invention includes:
step S10: acquiring first track information of a suspicious target, and determining a first area associated with the suspicious target according to the first track information; suspicious targets are generally referred to as "personnel to be tracked", and in some scenarios, robots and the like are also possible; the first tracking information is mainly used for representing the preliminary movement of the first tracking information, for example, the first tracking information may enter the first area through other areas, or is always in the first area, or suspicious targets are not identified in a plurality of adjacent areas, but the suspicious targets are most likely to exist in the first area;
step S11: determining a first monitoring device with a monitoring state which is changed from normal to abnormal at a first moment in the monitoring devices of the first area; a plurality of monitoring devices are distributed in the first area, so that the first area can be monitored more comprehensively, and the monitoring devices in the first area can cause abnormal monitoring states due to interference or damage; of course, the abnormal state of the device can be caused by other reasons, such as self-failure; it is thus necessary to determine the specific cause of the abnormality in the monitoring state thereof;
step S12: positioning a suspended local area range in which the first monitoring equipment is positioned, and acquiring foreign matter perception information acquired in the suspended local area range; the suspended local area generally refers to the portion of the local area of the first monitoring device that is not in contact with the ground, as the first monitoring device is generally installed in the air; selecting a space region with a set size where the first monitoring equipment is located; the foreign matter sensing information can be acquired in the suspended local area range, and can also be called as foreign matter detection information; the foreign matter sensing information is generally collected by a sensing device (such as a sensor) and is used for judging whether foreign matters exist in a suspended local area range; the arrangement of the sensing equipment is not easy to find, and the sensing equipment has the characteristic of being hidden;
step S13: judging whether foreign matters appear in the suspended local area range at a certain moment according to the foreign matter perception information, wherein the certain moment is positioned after the first moment; such foreign objects may be suspicious targets or may be destructive tools employed by suspicious targets (or combinations of suspicious targets) in an attempt to interfere with or destroy the monitoring, such as face masks; generally, no foreign matter exists in the suspended local area; as for the time when the foreign matter appears, it is only necessary to determine that it is located after the first time, thereby seeing that there is a possibility of implementing the destruction;
step S14: if yes, determining that the trace of the suspicious object is associated with the first area, and the suspicious object is related to the monitoring state change of the monitoring equipment at the first moment. It should be noted that, it is determined that the trace of the suspicious object is associated with the first area, and a condition that a certain moment is located at the first moment is required to exist, and in a general case that it is determined that the first object enters the first area, the monitoring device monitors abnormality and detects foreign matters in a suspended local area range of the first object, so that the first object is indicated to enter the first area with a high probability, so as to implement vandalism or finger vandalism, and attempt to evade monitoring of the monitoring device.
In the application of the embodiment, from the perspective of monitoring state abnormality of the monitoring device, after the first area associated with the suspicious target is determined according to the first trace information, if foreign matter sensing information recorded in the suspended local area is passed, if foreign matter appears in the suspended local area based on the moment after the first moment, it is determined that the trace of the suspicious target is associated with the first area, that is, the suspicious target is very likely to enter the position of the first monitoring device in the first area or pass through the position of the first monitoring device in the first area, so that reliable basis is provided for tracking the suspicious target.
As shown in fig. 2, as a preferred embodiment of the present invention, after determining that the trace of the suspicious object is associated with the first region, the method further includes:
step S20: determining second monitoring equipment acting on the first area according to the first monitoring, wherein the second monitoring equipment is monitoring equipment except the first monitoring equipment; in the first area, there may be a plurality of second monitoring devices, for example in one industrial area;
step S21: respectively acquiring a first coordinate of the first monitoring equipment and a second coordinate of the second monitoring equipment, and generating a first relative deflection angle between the second monitoring equipment and the first monitoring equipment according to a determination result; determining a first relative deflection angle according to the current angles of the monitoring equipment on the first coordinate and the second coordinate, for example, by determining the coordinates of the first and the second coordinates on an electronic map, forming a connecting line of the first and the second coordinates through the coordinates; because the relative angle on the connecting line is 180 degrees, the deflection angle of the two relative connecting lines is calculated, and then the first relative deflection angle can be calculated according to the fact that the two are positioned on the same side or different sides of the connecting line, the part is equivalent to translating the two to the same coordinate to calculate the angle difference of the two, and the smaller angle difference is generally taken as the first relative deflection angle;
step S22: generating first deflection scheduling information for scheduling the second monitoring equipment according to the current deflection scheduling information of the second monitoring equipment and the first relative deflection angle; the current deflection scheduling information comprises the covered azimuth (specific azimuth) and the rotation speed, pitch angle and the like of monitoring rotation; the first deflection scheduling information passes through a first relative deflection angle from the original coverage azimuth and is positioned at a new scheduling monitoring angle;
step S23: target characteristics of a suspicious target are obtained, and a directional monitoring instruction is generated according to the target characteristics and the first deflection scheduling information; the target characteristics of the suspicious target can be extracted from the existing target pictures and can be one or more of facial features, action features and clothing features of people;
step S24: and issuing the directional monitoring instruction to the second monitoring equipment so that the monitoring angle of the second monitoring equipment after the dispatching covers the local range of the first monitoring equipment, and reporting the monitoring information about the suspicious target when the suspicious target is identified. Specifically, the real-time monitoring video data (monitoring information) can be decoded, and the real-time video data is decomposed into real-time images frame by frame. Taking a suspicious target as an example, the suspicious target can be identified in the real-time image by extracting a reference image of the target, inquiring and comparing the reference image in the real-time image;
it will be appreciated that by means of the target characteristics of the suspicious target, when the second monitoring device changes the original coverage area, if the suspicious target is identified, it will report the monitoring information containing the suspicious target, so as to facilitate determining the track of the suspicious target, and considering that the track movement of the suspicious target can be monitored before the second monitoring device is damaged, i.e. at least before the second monitoring device may be damaged, the second monitoring device becomes a new first monitoring device after the second monitoring device is damaged, and then the suspicious target is exposed to the new location of the first monitoring device.
As shown in fig. 3, optionally, the generating the first yaw schedule information for scheduling the second monitoring device according to the current yaw schedule information of the second monitoring device and the first relative yaw angle specifically includes:
step S221: identifying a first limit deflection angle and a second limit deflection angle in the current deflection scheduling information; for example, the covered azimuth is 60 degrees from the north to 40 degrees from the south, and the limit deflection angles are respectively 60 degrees from the north to 40 degrees from the south;
step S222: identifying a starting angle of the first relative deflection angle, and determining a limit deflection angle with a smaller difference value between the first limit deflection angle and the second limit deflection angle and the starting angle; for example, the starting angle is 20 degrees south west; therefore, the limit deflection angle with smaller difference value between the first limit deflection angle and the second limit deflection angle and the initial angle is 40 degrees of southeast deflection;
step S223: identifying an effective deflection angle corresponding to a coverage local area range in the first relative deflection angle, determining a first action parameter in the effective deflection angle, and acquiring a second action parameter in the current deflection scheduling information; the local area range is actually the local area range in which the above-mentioned floating local area range is located; the effective deflection angle may occupy only a portion of the first relative deflection angle; the first action parameters comprise a pitching angle within an effective deflection angle, an angular speed of the rotating speed and the like;
step S224: and fusing the current deflection scheduling information with the effective deflection angle to generate first deflection scheduling information for controlling the second monitoring equipment to switch back and forth at the limit deflection angle position according to parameters to be switched, wherein the parameters to be switched are a first action parameter and a second action parameter. Likewise, the second motion parameter may be modified with reference to the parameter prior to the change, for example, by increasing the rotational speed at which it rotates, thereby temporarily steering the monitored center of gravity within the effective yaw angle; as for the rotational speed in the non-effective yaw angle in the first relative yaw angle, the faster and better the rotational speed, the initial pitch angle in the first action parameter can be adjusted in advance in the process; illustratively, the first motion parameter is 30rad/s and the pitch angle is 20 degrees horizontally upwards; the second action parameter is 50rad/s, and the pitching angle is 10 degrees upwards;
according to the method, the current deflection scheduling information and the effective deflection angle are fused to generate the first deflection scheduling information, the second monitoring device can be controlled to deflect at the minimum limit deflection angle at the limit deflection position, corresponding actions are respectively executed before and after deflection according to the first action parameter and the second action parameter, monitoring influence on an original monitoring area is reduced as much as possible, whether a suspicious target is 'out' from the position of the first monitoring device or not can be identified, and therefore track movement of the suspicious target is conveniently monitored.
As a preferred embodiment of the present invention, the acquiring the first track information of the suspicious object includes:
step S101: acquiring input of trace information about a suspicious target;
step S102: or acquiring monitoring information containing suspicious targets in the second area. As a source of the first trace information, which may be descriptive information from the experimenter, witnessing that the suspicious object enters the first area, it may also be that the monitoring device at other locations has monitored that the suspicious object has been transferred from the second area to the first area or has disappeared at a location within the second area closest to the first area; thus in combination with the above description, the following determination is made of the earliest moment at which a suspicious object enters the first area, the method further comprising:
step S30: and determining the earliest moment when the suspicious object enters the first area according to the first track information, wherein the earliest moment comprises the latest moment when the suspicious object disappears in the second area.
When the embodiment is applied, in the first trace information, the input of the trace information may be that the suspicious target is seen to enter the first area, so that the input of the trace information includes a corresponding time node; if the second area contains monitoring information of the suspicious object, the monitoring information indicates the latest moment when the suspicious object disappears in the second area, the latest moment is taken as the earliest moment when the suspicious object enters the first area, and the second area is generally an area adjacent to the first area.
As a preferred embodiment of the present invention, the first monitoring device for determining that the monitored state in the monitoring device of the first area is changed from normal to abnormal at the first time specifically includes:
step S40: detecting the working state information reported by the monitoring equipment, wherein the monitoring equipment reports first monitoring information according to a set mode after the self-checking is successful, the set mode comprises the step of reporting at regular time, and a monitoring picture of the first monitoring information accords with set definition; setting a mode such as regular reporting, for example, periodically reporting, for example, changing reporting frequency according to the monitored number of people information, for example, the more people are, the higher the reporting frequency is;
step S41: when the fact that a certain monitoring device does not report the first monitoring information is detected, the monitoring state of the corresponding monitoring device is judged to be abnormal.
It can be understood that, in the preset program of the monitoring device, the monitoring device reports the monitoring information including the monitoring picture when the picture monitored by the lens or the front end portion thereof meets the set definition, so if it is detected that a certain monitoring device does not report the first monitoring information, the whole monitoring device may be damaged, the lens portion may be damaged or blocked, etc.
As shown in fig. 4, as a preferred embodiment of the present invention, after determining that the monitoring status of the corresponding monitoring device is abnormal, the method further includes:
step S50: issuing a detection feedback instruction to a certain monitoring device, wherein the detection feedback instruction is used for indicating the certain monitoring device to perform self-checking again and reporting first fault information; the purpose of the re-self-checking is to acquire first fault information thereof; for example, the front-end module is mainly used for collecting data and comprises a camera module; or is divided into a photographing part, a transmission part, a control part and a display part from the composition part;
step S51: if the first fault information is received, judging whether the monitoring state abnormality of the corresponding monitoring equipment is related to the monitoring lens or not according to the first fault information, and if so, identifying a first fault moment of the monitoring lens in the first fault information; when the first fault information is received, at least indicating that the transmission part is not abnormal; in the aspect of external damage of the monitoring state abnormality, the last possibility is that the shooting part is damaged or blocked, i.e. the monitoring lens is abnormal;
step S52: if the first fault information is not received, determining theoretical reporting time corresponding to the setting mode when the first monitoring information is not reported, and taking the theoretical reporting time as a first fault moment; when the first fault information is not received, it is obvious that the transmission part is abnormal, and the control module is also possibly abnormal; it is highly likely that the monitoring device body portion is damaged; determining theoretical reporting time as a first fault moment; illustratively, the monitoring device should be at 16:40, a step of performing a; 16:45;16:48;16:52 report the monitoring information separately, but at 16:45, after reporting the monitoring information, at 16:48 does not detect its reporting monitoring information, at which point 16:48 as a first failure moment;
step S53: and judging whether the first fault moment is later than the earliest moment, if so, determining that the monitoring equipment corresponding to the first fault moment which is later than the latest moment is the first monitoring equipment.
In the application of the embodiment, the barrier information of the target can be obtained and reported through the indication monitoring self-checking, and the embodiment respectively identifies the (earliest) time, namely the first fault moment, which theoretically causes the monitoring equipment from the point of not obtaining the fault information and obtaining the fault information, and if the moment is later than the earliest moment, namely after the target enters the first area, the target or the partner of the target is indicated that the time for destroying the monitoring is possible to a great extent.
As a preferred embodiment of the present invention, the acquiring foreign object perception information acquired in a floating local area range includes:
step S121: and issuing an acquisition instruction to at least one sensing device in the suspended local area range so as to acquire sensing information reported by the at least one sensing device, wherein the foreign matter sensing information comprises sensing information, and the at least one sensing device comprises at least one of distance sensing devices and infrared sensing devices. For the distance sensing equipment in the sensing equipment, the distance between the target object (such as the target or a tool used by the target) and the monitoring equipment can be measured through the distance sensor, and when the distance reaches a certain threshold value, the monitoring equipment can be judged to be interfered or damaged; for the infrared sensing device, the infrared sensor is provided with a pair of infrared emitting and receiving pipes, the emitting pipes emit infrared rays with certain frequency, when the detection direction meets an obstacle (reflecting surface), the infrared rays are reflected back to be received by the receiving pipes, and in fact, the infrared sensor is one of photoelectric sensors and can also detect the distance of the foreign matters; the working principle of the infrared sensor is that the temperature rise is caused after the sensitive element of the detector absorbs the radiant energy by utilizing the thermal effect of the infrared radiation, so that certain related physical parameters are changed, and the infrared radiation absorbed by the detector is determined to come from the foreign matters by measuring the change of the physical parameters;
further, the sensing device may also be a lidar, which may use light and emit the light in pulses. With respect to measuring scattered light with respect to laser irradiation, and analyzing the long distance to an object and the characteristics of the object, it is noted that the light waves used for laser radars are 905nm and 1550nm, which are not in the light wave length range of visible light (380-780 nm). The sensing device can be provided with one or more sensing devices, can be uniformly distributed on the installation objects near the monitoring device or on the building, and can be embedded on the monitoring device, so that the sensing device has the greatest advantages that the sensing device is difficult to discover when detecting foreign matters, the working process is difficult to discover, and the sensing device has good concealment.
As shown in fig. 5, further, the determining whether the foreign object appears in the floating local area at a certain moment according to the foreign object sensing information includes:
step S131: judging whether foreign matters appear in the suspended local area range according to the foreign matter perception information; for example, for the distance sensing device, the detection of the foreign matter in the local area where the monitoring device is located can be realized through one or more of a distance sensor, a photoelectric sensor and an infrared sensor, which can also be called as obstacle detection;
step S132: if so, based on the foreign matter perception information, the moment when the foreign matter appears is judged to be a certain moment. Generally, the foreign object detection is divided into two layers, and the low-level perception is also called obstacle detection, and only an obstacle is detected in a certain direction (such as the front) of the monitoring equipment; the high-level perception can be regarded as target recognition, and obstacle information can be further classified; in the application, the lowest sensing is achieved only through low-pass sensing in the suspended local area, because in the suspended local area where the monitoring equipment is located, no foreign matters exist generally, and if the monitoring equipment is located, the monitoring equipment is very likely to be interfered and destroyed by a target or a tool brought by the target;
for high-level perception, the foreign object detection can extract potential foreign objects from the acquired data (point cloud data) to obtain information such as azimuth, size, shape, orientation and the like, and the information can be generally described by adding or polygonal frames through a frame regression algorithm in target detection.
As another preferred embodiment of the present invention, as shown in fig. 6, in another aspect, a person-to-be-analyzed tracking system includes:
the area determining module 100 is configured to obtain first track information of a suspicious target, and determine a first area associated with the suspicious target according to the first track information;
a state transition determining module 200, configured to determine a first monitoring device whose monitoring state is changed from normal to abnormal at a first time in the monitoring devices in the first area;
the local positioning module 300 is configured to position a suspended local area range in which the first monitoring device is located, and acquire foreign matter perception information acquired in the suspended local area range;
the foreign matter judging module 400 is configured to judge whether a foreign matter appears in the suspended local area at a certain moment according to the foreign matter sensing information, where the certain moment is located after the first moment;
the condition judgment module 500 is configured to judge that the trace of the suspicious object is associated with the first area if the foreign object is present, and the suspicious object is related to the monitoring state change of the monitoring device at the first moment.
According to the method for tracking the personnel to be analyzed, the system for tracking the personnel to be analyzed is provided based on the method for tracking the personnel to be analyzed, the first area is judged to be associated with the trace of the suspicious target if the occurrence of the foreign matter in the suspended local area is judged to be based on a moment after the first moment through the foreign matter perception information recorded in the suspended local area after the first area is determined according to the first trace information from the point of abnormal monitoring state of the monitoring equipment, namely the suspicious target is very likely to enter the position of the first monitoring equipment in the first area or pass through the position of the first monitoring equipment in the first area, and reliable basis is provided for tracking the suspicious target.
In order to be able to load the method and system described above to function properly, the system may include more or less components than those described above, or may combine some components, or different components, in addition to the various modules described above, for example, may include input and output devices, network access devices, buses, processors, memories, and the like.
The processor may be a central processing unit (CentralProcessingUnit, CPU), other general purpose processors, digital signal processors (DigitalSignalProcessor, DSP), application specific integrated circuits (ApplicationSpecificIntegratedCircuit, ASIC), off-the-shelf programmable gate arrays (Field-ProgrammableGateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, which is a control center of the above system, and various interfaces and lines are used to connect the various parts.
The memory may be used to store a computer and a system program and/or module, and the processor may perform the various functions described above by running or executing the computer program and/or module stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as an information acquisition template presentation function, a product information distribution function, etc.), and the like. The storage data area may store data created according to the use of the berth status display system (e.g., product information acquisition templates corresponding to different product types, product information required to be released by different product providers, etc.), and so on. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart memory card (SmartMediaCard, SMC), secure digital (SecureDigital, SD) card, flash card (FlashCard), at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
It should be understood that, although the steps in the flowcharts of the embodiments of the present invention are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in various embodiments may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (10)
1. A method for tracking a person to be analyzed, the method comprising:
acquiring first trace information of a suspicious target;
determining a first area associated with the suspicious target according to the first track information;
determining a first monitoring device with a monitoring state which is changed from normal to abnormal at a first moment in the monitoring devices of the first area;
positioning a suspended local area range in which the first monitoring equipment is positioned, and acquiring foreign matter perception information acquired in the suspended local area range;
judging whether foreign matters appear in the suspended local area range at a certain moment according to the foreign matter perception information, wherein the certain moment is positioned after the first moment;
if yes, determining that the trace of the suspicious object is associated with the first area, and the suspicious object is related to the monitoring state change of the monitoring equipment at the first moment.
2. The personnel tracking method to be analyzed according to claim 1, wherein after determining that the trace of the suspicious object is associated with the first area, the method further comprises:
determining second monitoring equipment acting on the first area according to the first monitoring, wherein the second monitoring equipment is monitoring equipment except the first monitoring equipment;
respectively acquiring a first coordinate of the first monitoring equipment and a second coordinate of the second monitoring equipment, and generating a first relative deflection angle between the second monitoring equipment and the first monitoring equipment according to a determination result;
generating first deflection scheduling information for scheduling the second monitoring equipment according to the current deflection scheduling information of the second monitoring equipment and the first relative deflection angle;
target characteristics of a suspicious target are obtained, and a directional monitoring instruction is generated according to the target characteristics and the first deflection scheduling information;
and issuing the directional monitoring instruction to the second monitoring equipment so that the monitoring angle of the second monitoring equipment after the dispatching covers the local range of the first monitoring equipment, and reporting the monitoring information about the suspicious target when the suspicious target is identified.
3. The method for tracking personnel to be analyzed according to claim 2, wherein the generating the first yaw schedule information for scheduling the second monitoring device according to the current yaw schedule information of the second monitoring device and the first relative yaw angle specifically includes:
identifying a first limit deflection angle and a second limit deflection angle in the current deflection scheduling information;
identifying a starting angle of the first relative deflection angle, and determining a limit deflection angle with a smaller difference value between the first limit deflection angle and the second limit deflection angle and the starting angle;
identifying an effective deflection angle corresponding to a coverage local area range in the first relative deflection angle, determining a first action parameter in the effective deflection angle, and acquiring a second action parameter in the current deflection scheduling information;
and fusing the current deflection scheduling information with the effective deflection angle to generate first deflection scheduling information for controlling the second monitoring equipment to switch back and forth at the limit deflection angle position according to parameters to be switched, wherein the parameters to be switched are a first action parameter and a second action parameter.
4. The method for tracking personnel to be analyzed according to claim 1, wherein the acquiring the first track information of the suspicious target includes:
acquiring input of trace information about a suspicious target;
or acquiring monitoring information containing suspicious targets in the second area.
5. The personnel tracking method to be analyzed according to claim 1, further comprising: and determining the earliest moment when the suspicious object enters the first area according to the first track information, wherein the earliest moment comprises the latest moment when the suspicious object disappears in the second area.
6. The person to be analyzed tracking method according to claim 5, wherein the first monitoring device determining that the monitoring state in the monitoring device of the first area is changed from normal to abnormal at the first time specifically includes:
detecting the working state information reported by the monitoring equipment, wherein the monitoring equipment reports first monitoring information according to a set mode after the self-checking is successful, the set mode comprises the step of reporting at regular time, and a monitoring picture of the first monitoring information accords with set definition;
when the fact that a certain monitoring device does not report the first monitoring information is detected, the monitoring state of the corresponding monitoring device is judged to be abnormal.
7. The person-to-be-analyzed tracking method according to claim 6, wherein after determining that the corresponding monitoring device monitors the status abnormality, the method further comprises:
issuing a detection feedback instruction to a certain monitoring device, wherein the detection feedback instruction is used for indicating the certain monitoring device to perform self-checking again and reporting first fault information;
if the first fault information is received, judging whether the monitoring state abnormality of the corresponding monitoring equipment is related to the monitoring lens or not according to the first fault information, and if so, identifying a first fault moment of the monitoring lens in the first fault information;
if the first fault information is not received, determining theoretical reporting time corresponding to the setting mode when the first monitoring information is not reported, and taking the theoretical reporting time as a first fault moment;
and judging whether the first fault moment is later than the earliest moment, if so, determining that the monitoring equipment corresponding to the first fault moment which is later than the latest moment is the first monitoring equipment.
8. The method according to claim 7, wherein the acquiring foreign object perception information acquired in a floating local area range includes:
and issuing an acquisition instruction to at least one sensing device in the suspended local area range so as to acquire sensing information reported by the at least one sensing device, wherein the foreign matter sensing information comprises sensing information, and the at least one sensing device comprises at least one of distance sensing devices and infrared sensing devices.
9. The method according to claim 1, wherein the determining whether a foreign object appears in the floating local area at a certain time according to the foreign object sensing information comprises:
judging whether foreign matters appear in the suspended local area range according to the foreign matter perception information;
if so, based on the foreign matter perception information, the moment when the foreign matter appears is judged to be a certain moment.
10. A personnel tracking system to be analyzed, the system comprising:
the area determining module is used for acquiring first trace information of the suspicious target and determining a first area associated with the suspicious target according to the first trace information;
a state transition determining module, configured to determine a first monitoring device whose monitoring state is changed from normal to abnormal at a first time in the monitoring devices in the first area;
the local positioning module is used for positioning a suspension local range in which the first monitoring equipment is positioned and acquiring foreign matter perception information acquired in the suspension local range;
the foreign matter judging module is used for judging whether foreign matters appear in the suspended local area range at a certain moment according to the foreign matter perception information, and the certain moment is positioned after the first moment;
and the condition judging module is used for judging that the trace of the suspicious object is associated with the first area if the foreign matter appears, and the suspicious object is related to the monitoring state change of the monitoring equipment at the first moment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310594119.5A CN116503814B (en) | 2023-05-24 | 2023-05-24 | Personnel tracking method and system for analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310594119.5A CN116503814B (en) | 2023-05-24 | 2023-05-24 | Personnel tracking method and system for analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116503814A true CN116503814A (en) | 2023-07-28 |
| CN116503814B CN116503814B (en) | 2023-10-24 |
Family
ID=87323078
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310594119.5A Active CN116503814B (en) | 2023-05-24 | 2023-05-24 | Personnel tracking method and system for analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116503814B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116886876A (en) * | 2023-09-06 | 2023-10-13 | 隆基盛泰(北京)科技有限公司 | Security monitoring system and method thereof |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101465033A (en) * | 2008-05-28 | 2009-06-24 | 丁国锋 | Automatic tracking recognition system and method |
| JP2011035571A (en) * | 2009-07-30 | 2011-02-17 | Masafumi Hagiwara | Suspicious behavior detection method and suspicious behavior detector |
| CN102457712A (en) * | 2010-10-28 | 2012-05-16 | 鸿富锦精密工业（深圳）有限公司 | System and method for identifying and tracking suspicious target |
| JP2013050926A (en) * | 2011-08-31 | 2013-03-14 | Secom Co Ltd | Suspicious object body monitoring system |
| CN103929592A (en) * | 2014-04-22 | 2014-07-16 | 杭州道联电子技术有限公司 | All-dimensional intelligent monitoring equipment and method |
| CN104616322A (en) * | 2015-02-10 | 2015-05-13 | 山东省科学院海洋仪器仪表研究所 | Onboard infrared target image identifying and tracking method and device |
| CN111885360A (en) * | 2020-07-31 | 2020-11-03 | 贵州东冠科技有限公司 | Intelligent monitoring system for block chain production workshop |
| WO2020224122A1 (en) * | 2019-05-06 | 2020-11-12 | 平安科技（深圳）有限公司 | Video monitoring method and apparatus, computer device, and storage medium |
| CN112365522A (en) * | 2020-10-19 | 2021-02-12 | 中标慧安信息技术股份有限公司 | Method for tracking personnel in park across borders |
| CN115223100A (en) * | 2022-08-02 | 2022-10-21 | 上海三力信息科技有限公司 | Intelligent park abnormal person identification method, system, equipment and storage medium |
| CN115311735A (en) * | 2022-05-19 | 2022-11-08 | 重庆理工大学 | Intelligent recognition early warning method for abnormal behaviors |
-
2023
- 2023-05-24 CN CN202310594119.5A patent/CN116503814B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101465033A (en) * | 2008-05-28 | 2009-06-24 | 丁国锋 | Automatic tracking recognition system and method |
| JP2011035571A (en) * | 2009-07-30 | 2011-02-17 | Masafumi Hagiwara | Suspicious behavior detection method and suspicious behavior detector |
| CN102457712A (en) * | 2010-10-28 | 2012-05-16 | 鸿富锦精密工业（深圳）有限公司 | System and method for identifying and tracking suspicious target |
| JP2013050926A (en) * | 2011-08-31 | 2013-03-14 | Secom Co Ltd | Suspicious object body monitoring system |
| CN103929592A (en) * | 2014-04-22 | 2014-07-16 | 杭州道联电子技术有限公司 | All-dimensional intelligent monitoring equipment and method |
| CN104616322A (en) * | 2015-02-10 | 2015-05-13 | 山东省科学院海洋仪器仪表研究所 | Onboard infrared target image identifying and tracking method and device |
| WO2020224122A1 (en) * | 2019-05-06 | 2020-11-12 | 平安科技（深圳）有限公司 | Video monitoring method and apparatus, computer device, and storage medium |
| CN111885360A (en) * | 2020-07-31 | 2020-11-03 | 贵州东冠科技有限公司 | Intelligent monitoring system for block chain production workshop |
| CN112365522A (en) * | 2020-10-19 | 2021-02-12 | 中标慧安信息技术股份有限公司 | Method for tracking personnel in park across borders |
| CN115311735A (en) * | 2022-05-19 | 2022-11-08 | 重庆理工大学 | Intelligent recognition early warning method for abnormal behaviors |
| CN115223100A (en) * | 2022-08-02 | 2022-10-21 | 上海三力信息科技有限公司 | Intelligent park abnormal person identification method, system, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 张见雨: "安防监控下人体异常行为检测算法研究", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》, pages 138 - 1890 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116886876A (en) * | 2023-09-06 | 2023-10-13 | 隆基盛泰(北京)科技有限公司 | Security monitoring system and method thereof |
| CN116886876B (en) * | 2023-09-06 | 2023-11-24 | 闵长顺 | Security monitoring system and method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116503814B (en) | 2023-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10553044B2 (en) | Self-diagnosis of faults with a secondary system in an autonomous driving system | |
| CN102859565B (en) | Method and system for security system tampering detection | |
| US11145146B2 (en) | Self-diagnosis of faults in an autonomous driving system | |
| US8310554B2 (en) | Method and apparatus for performing coordinated multi-PTZ camera tracking | |
| US8693725B2 (en) | Reliability in detecting rail crossing events | |
| US8619140B2 (en) | Automatic adjustment of area monitoring based on camera motion | |
| US8731241B2 (en) | Activity mapping system | |
| CN116503814B (en) | Personnel tracking method and system for analysis | |
| KR101852058B1 (en) | unexpected accident detecting method using dual camera | |
| CN103929592A (en) | All-dimensional intelligent monitoring equipment and method | |
| US20220246005A1 (en) | Security cameras with thermal imaging sensors | |
| Adrian et al. | An overview of intelligent surveillance systems development | |
| CN116434148A (en) | Data processing system and processing method based on Internet of things | |
| Nam | Real-time abandoned and stolen object detection based on spatio-temporal features in crowded scenes | |
| US20220035003A1 (en) | Method and apparatus for high-confidence people classification, change detection, and nuisance alarm rejection based on shape classifier using 3d point cloud data | |
| CN114966714A (en) | Window occlusion detection method and device | |
| CN111966126A (en) | Unmanned aerial vehicle patrol method and device and unmanned aerial vehicle | |
| KR102630275B1 (en) | Multi-camera fire detector | |
| WO2016147704A1 (en) | Monitoring device, monitoring system, monitoring method, and computer-readable recording medium | |
| CN111414836A (en) | Identification method and device of crossing gate, computer equipment and storage medium | |
| CN116704411A (en) | Security control method, system and storage medium based on Internet of things | |
| CN207530963U (en) | A kind of illegal geofence system based on video monitoring | |
| CN113570622A (en) | Obstacle determination method and device, electronic equipment and storage medium | |
| CN112883856A (en) | Monitoring method, monitoring device, electronic equipment and storage medium | |
| KR102591598B1 (en) | A method for operating a security device using sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |